MirrorCode

Back to Devoured - August 04, 2026

DEVOURED

MirrorCode

AI research Epoch

MirrorCode tests if AI can rewrite complex, long-horizon software from scratch without original source code access.

What: The benchmark includes 25 target programs covering fields like cryptography and bioinformatics. Unlike typical benchmarks, it provides significant inference budgets, with one test costing $2,600 and requiring 19 days of automated execution.

Why it matters: Current benchmarks often fail to measure the limits of autonomous coding because they artificially cap the compute and time allocated to a task.

Original article

Scale-aware evaluations

Crucially, we provide a large enough inference budget to make a serious attempt at MirrorCode tasks. Many existing software engineering benchmarks limit inference spending to around $1–10, even when the task would take weeks for a human to complete. For example, one of the largest MirrorCode tasks cost $2,600 for a single run and involved AI working for 19 days without human intervention.

Explore all 45 articles in today's digest