Seven Frontier Models, 80 Engineering Tasks: We Re-Ran Our AI Benchmark
By DigiEntropy Engineering · 2026-07-26 · 13 min read
Our first benchmark got too easy — everyone passed. So we doubled the hard questions and added three more models. On the old questions the field is tied; on the new ones every model drops, and three real defects fell out.
About this post — Authored by an AI assistant using DigiEntropy's polynomial corpus, the universal compressor predictor, and the project's chart-generation tooling. Charts are produced by Python scripts that read the same database the live site queries; tables and formulas are pulled from the same engineering modules. Findings reflect the project's current dataset and methodology — send corrections or deeper questions to admin@digientropy.com.
We ran our AI benchmark again — wider and harder. The test suite grew from 49 questions to 80, the field grew from four models to seven, and the result is more interesting than last time: on the old questions almost every frontier model now scores in the high nineties, and only the new, harder half tells them apart.
DigiEntropy has a built-in engineering co-pilot called Daisy. You ask in plain English — "size a freezer room", "rate this shell-and-tube condenser", "which drive fits this screw compressor" — and it answers by calling the platform's real engines (cold-room load, compressor polynomials, selection, heat exchangers, inverter matching, operating envelope, F-Gas retrofit), never by guessing numbers. Swap the model behind it, keep the engines and the questions fixed, and you have a fair test bench for engineering work.
80 - questions in the suite - up from 49
The test suite grew from 49 questions to 80, the field grew from four models to seven.
7 - models tested - up from four
The field grew from four models to seven: GPT-5.2, GPT-5.6-terra, GPT-5.6-sol, Gemini 3.1 Pro, Kimi K3, DeepSeek V4 Pro and DeepSeek V4 Flash.
0.85 - the DQS bar - with zero hard-gate failures
The level-weighted mean of the judge scores is the Daisy Quality Score (DQS), 0 to 1. Our bar: DQS ≥ 0.85 with zero hard-gate failures.
0.956 - GPT-5.6-terra, top score - GPT-5.2: 0.955, a tie
GPT-5.6-terra (0.956) and GPT-5.2 (0.955) are tied at the top of the models that answered all 80, with DeepSeek V4 Flash (0.949) a hair behind.
0.06-0.11 - the drop on the new questions - every model falls
On the 49 round-1 questions almost everyone scores 0.94-0.99. On the 31 new ones every model drops, by 0.06 to 0.11.
0.019 - run-to-run noise - same model, same 80 questions, twice
The aggregate moved 0.019 between two identical runs. So any gap under about 0.02 in these charts is not a real difference.
36× - spread in cost per pass - for identical work
The bill for that identical work ranges from USD 0.29 to USD 10.41, a factor of 36.
Why we re-ran it
…