Physical-Engineering's Last Exam (PLE)
by Google (Rujun Han, Chen-Yu Lee et al.), developed with Mercor
What is Physical-Engineering's Last Exam (PLE)? Physical-Engineering's Last Exam (PLE) is an agents & tool use eval made by Google (Rujun Han, Chen-Yu Lee et al.), developed with Mercor. A benchmark and open competition for AI agents doing real physical engineering work: 85 long-horizon tasks, reviewed by practicing engineers, across mechanical (30), electrical (27), civil (18) and structural (10) engineering, covering calculations, drawings and design reviews. Each task runs in an isolated container with the project's drawings, spreadsheets and specs, no internet access and up to 6 hours, three runs per model and harness. Scripts check the numbers, an LLM judge grades open-ended work against an expert rubric, and a design only earns points once it passes every hard safety and feasibility check. 29 public tasks and the grading tools are released (code Apache-2.0, dataset CC-BY-4.0), while the leaderboard uses 56 hidden test tasks. In Google's launch results (3 Nov 2026) no frontier model passed 50%; the top entry was Claude Opus 5.5 on Claude Code at 0.483.
Editor note
A benchmark and open competition for AI agents doing real physical engineering work: 85 long-horizon tasks, reviewed by practicing engineers, across mechanical (30), electrical (27), civil (18) and structural (10) engineering, covering calculations, drawings and design reviews. Each task runs in an isolated container with the project's drawings, spreadsheets and specs, no internet access and up to 6 hours, three runs per model and harness. Scripts check the numbers, an LLM judge grades open-ended work against an expert rubric, and a design only earns points once it passes every hard safety and feasibility check. 29 public tasks and the grading tools are released (code Apache-2.0, dataset CC-BY-4.0), while the leaderboard uses 56 hidden test tasks. In Google's launch results (3 Nov 2026) no frontier model passed 50%; the top entry was Claude Opus 5.5 on Claude Code at 0.483.
- What it tests
- Whether AI agents can complete end-to-end mechanical, electrical, civil and structural engineering tasks that pass hard safety and code checks.
- Original source
- ple-bench.com
- Last updated
- Launched 2026-11-03 (repo last push 2026-11-02)
- License
- Apache-2.0 (code); CC BY 4.0 (data)
Related agents & tool use evals
- GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
- WebArena — Autonomous web navigation and task completion.
- OSWorld — Computer-use agents operating real operating systems and apps.
- Terminal-Bench — Agents working in the shell end to end.
Discussion
No one has discussed Physical-Engineering's Last Exam (PLE) yet. Have you run it? Share what it caught, what it missed or how you set it up.