Introducing: Odysseys2 - hardest long horizon benchmark for web agents based on real user tasks 🏗️
We collaborated with @JangLawrenceK @rsalakhu from Carnegie Mellon University
> SOTA agent fully completes only 47/100
> 5 agents complete none
> 8-12 real rubrics per task, 60 min on the live web
Web agents are becoming better than humans at knowledge work.
116896K






