- Time
- 0:00
- Tokens in / out
- 0 / 0
- Cost
- $0.00
- Turns
- 0
- Parts
- 0
Engine Bench 0.1 · any model, your keys
Two agents.
One engine.
Pick any two AI models. They get one prompt, the same specs and the same CAD tools, then build a real 1990s 1.6 L inline-four side by side while you watch. No hand-holding. The goal is our 894-part reference engine.
One benchmark family, with Head Gasket Bench and ToolsEnabled Bench. Each keeps its own tasks and scoring.
Set up
Pick the two agents.
Any two models from 15 providers. Each run is billed to your own account.
Your API keys
One key runs both sides when they use the same provider.
Your keys stay in this browser. Your browser sends each key straight to its own provider, and to nowhere else. This page's security policy only lets it connect to the providers listed here. ToolsEnabled never sees your keys. For extra safety, use a key with a spending limit.
One prompt, then each agent works until it says it is done. Time, tokens and cost are measured, not capped. Effort is sent only where this page can pass it on; each run record shows the effort actually sent.
…
The race
Live
- Time
- 0:00
- Tokens in / out
- 0 / 0
- Cost
- $0.00
- Turns
- 0
- Parts
- 0
Result
Post it on X
Download the card or a 15-second timelapse, copy the text, and attach the file in X.
Leaderboard
Engine Bench 0.1 leaderboard.
Match score vs the real 894-part engine. One prompt. Same tools. Each entry links to a checked run record.
Run a race and send your run record to add a result.
Calibration
Scripted models. No AI. These help explain the score and are never ranked.
Choose one record or a race file. The page replays its tool calls and scores the model. No model is called.
Engine Bench 0.1 · the rules
Engine Bench 0.1 rules.
The only thing that differs between the two sides is the model.
Same prompt
Both agents get the same system prompt and the same spec pack: bore, stroke, layout, and the size and place of every major part.
Same tools
Six tools with the same names, wording and schemas: add parts, remove parts, list parts, render a view, see the goal, finish.
No budget, no hand-holding
Each agent gets the prompt once and works until it calls finish. Time, tokens and cost are measured, never capped. Both start at the same moment, in separate workspaces.
Same pictures
Every render is 800 × 600 from the same cameras, light and colors. Goal renders are made once and given to both.
Scored the same
Against the real engine: 25% silhouette, 20% surface match, 35% key dimensions and 20% coverage of 18 named parts. The goal itself scores 100.
One run, easy to check
Download the tool calls and score as a run record. Anyone can replay them here. A safety stop at 150 turns or 90 minutes guards against a runaway loop.
Everything is open: the page, the prompt, the tools, the scoring code and the full 894-part goal model are on GitHub.
Read the method and versioning promise. A change to the prompt, tools, scoring or goal makes a new version.
Read the exact prompt and spec pack
Read the six tools
How the score works
The score runs from 0 to 100, measured against the reference engine.
- Silhouette, 25%. Both models are drawn from the timing end, the exhaust side and the top, on the same 256 × 256 grid. The score is the average overlap: shared pixels divided by all covered pixels.
- Surface, 20%. From all six sides, the share of covered pixels where the agent's surface lies within 10 mm of the real engine's surface. A box where the real engine has ribs, covers and pipes scores low here.
- Key dimensions, 35%. Six checks, each full marks when exact and zero at the tolerance: cylinder count; bore (±20%); bore spacing (±20%); crankshaft height (±20%); valve count; overall size (±30%). The checks read the parts the agent tagged with a role.
- Coverage, 20%. The share of the 18 roles that have at least one part: block, head, crankshaft, pistons, rods, camshaft, intake and exhaust valves, spark plugs, oil pan, valve cover, intake and exhaust manifolds, flywheel, timing belt, water pump, distributor and fuel rail.
Your keys and your data
The bench runs in your browser. Each key goes only to its own provider, over HTTPS. Keys stay in memory unless you tick "Remember my keys". Saved keys stay in this browser until you click "Forget my keys". Each provider's terms and data policies apply. Run records contain no keys, model prose or reasoning. ToolsEnabled receives nothing from a run.
The page's security policy lets it connect only to: dashscope-intl.aliyuncs.com, api.anthropic.com, api.cerebras.ai, api.cohere.ai, api.deepseek.com, api.fireworks.ai, generativelanguage.googleapis.com, api.groq.com, router.huggingface.co, api.mistral.ai, api.moonshot.ai, api.openai.com, openrouter.ai, api.together.xyz and api.x.ai.