the benchmark layer — runs are projects
Experiments
tasks
Free build (inventory only)
Given: set name + exact inventory as text. Not given: images or instructions. The agent builds from world knowledge; scored by BOM F1 (part+color multiset).
Instruction rebuild (booklet PDF)
Given: the local instruction booklet (read page-by-page with zoomable regions via view_instructions) + inventory. Needs the set's booklet fetched locally (9393-1 and 10271-1 have it).
Step-locked rebuild soon
The original assembly benchmark (per-step GT images, survival depth) — refounds onto this layer; today it lives in Archive → Evals.
new experiment · claude-sonnet-5
runs in this tab, one row at a time — each turn costs real API tokens
experiments · 0
None yet — pick sets above and launch.