the benchmark layer — runs are projects
Experiments
tasks
Free build (inventory only)
Given: set name + exact inventory as text. Not given: images or instructions. The agent builds from world knowledge; scored by BOM F1 (part+color multiset).
Instruction rebuild (booklet PDF)
Given: the local instruction booklet (read page-by-page with zoomable regions via view_instructions) + inventory. Needs the set's booklet fetched locally (9393-1 and 10271-1 have it).
Step-locked rebuild soon
The original assembly benchmark (per-step GT images, survival depth) — refounds onto this layer; today it lives in Archive → Evals.
masked assembly — restore the missing parts
same generator as the CLI: npx tsx scripts/exp/gen-removal.ts 126-1 3 0 --strategy=leaf --register
new experiment · claude-sonnet-5
experiments · 0
None yet — pick sets above and launch.