Agents' Last Exam

Agents' Last Exam is a large-scale benchmark designed to evaluate AI agents on real-world, economically valuable professional workflows. The platform focuses on assessing agent performance on long-horizon tasks with verifiable outcomes, aiming to provide objective and comparable scores across a broad range of domains. It is led by Berkeley RDI in collaboration with over 300 industry experts and spans 55 sub-industries that encompass most major fields of professional work conducted on computers.

The benchmark includes over 1,500 tasks, with a target of 5,000, covering areas such as animation and visual effects production in Adobe After Effects, 3D modeling in Siemens NX, game development tasks in Unreal Engine, simulation and mold flow analysis in Moldex3D, architectural modeling and energy analysis in Rhino 3D, and neuroimaging analysis in FSLeyes. These tasks are designed to reflect real professional challenges, ensuring that agent evaluations are grounded in practical, economically relevant scenarios.

Agents' Last Exam provides features such as a leaderboard for tracking agent performance, access to detailed agent traces, and documentation to support contributors. The platform encourages community involvement, allowing users to contribute new tasks and participate in co-authorship. It is supported by an advisory committee comprising professors and researchers from leading academic institutions and is connected with contributors and partners from both academia and industry.

evidence_sufficient": true}