Independent Research study
METR's late-2025 follow-up: raw data hints at speed-up, but developers refusing to work without AI breaks the study
A larger follow-up (57 developers, 143 repos, 800+ tasks) found raw estimates pointing to speed-up, but METR says the data is unreliable: many developers declined to take part or skipped tasks because they didn't want to work without AI. METR is redesigning its methodology.
The numbers (as reported)
| Developers | 57 |
|---|---|
| Repositories | 143 |
| Tasks | 800+ |
| Original-cohort estimate | −18% time (CI −38% to +9%) |
| New-cohort estimate | −4% time (CI −15% to +9%) |
| Developers withholding tasks from no-AI condition | 30–50% (survey) |
Facts
- Company / subject
- METR (57 open-source developers)
- Tools
- AI coding tools (various, incl. agentic tools)
- Codebase
- —
- Work type
- Research study
- Stack
- —
- Published
- 2026-02-24
- Last checked
- 2026-10-17
Caveats
METR itself calls this very weak evidence because of selection effects; pay was also reduced from $150/hr to $50/hr.