Inside Daisy: How We Measure and Improve Our AI Co-pilot
By DigiEntropy Engineering · 2026-04-05 · 11 min read
Our in-app AI engineer was useful but not 'wow'. So we built an eval to measure it across 33 questions, found four failure patterns, and fixed them — here's the whole story, in charts.
About this post — Authored by an AI assistant using DigiEntropy's polynomial corpus, the universal compressor predictor, and the project's chart-generation tooling. Charts are produced by Python scripts that read the same database the live site queries; tables and formulas are pulled from the same engineering modules. Findings reflect the project's current dataset and methodology — send corrections or deeper questions to admin@digientropy.com.
Daisy is the AI co-pilot inside DigiEntropy. You ask a refrigeration question in plain language — size a freezer, which refrigerant is most efficient, is this compressor a good fit — and it answers with real numbers, runs the platform's calculation engines for you, and can even drive the pages on your behalf.
It worked. But it did not wow. It would refuse questions it should have answered, reach for the wrong tool, or state a number it had quietly made up. This post is the story of how we stopped guessing whether Daisy was good and started measuring it — then used those measurements to make it genuinely smart.
33 - test questions - five levels, L1 to L5
A good test set spans from trivial to genuinely hard. We wrote 33 test questions across five levels.
0.748 - baseline DQS - Daisy exactly as it shipped
We ran the whole suite against Daisy exactly as it shipped. The baseline came back at 0.748, short of the bar.
0.85 - the bar - DQS target, set up front
The bar: a DQS of at least 0.85, with zero fabrication and zero refusals of answerable questions.
1.00 - booster vs parallel, after - was 0.40 before
Booster-vs-parallel and is-R290-safe-indoors went from 0.40 to 1.00; the retrofit-plus-inverter multi-step from 0.60 to 1.00.
0.945 - GPT-5.2, hardened suite - DeepSeek: 0.853
On the hardened 33-question suite, both models clear the bar — DeepSeek at 0.853 and GPT-5.2 at 0.945 — with zero fabrication and zero refusals of answerable questions.
0.909 - GPT-5.2 with grounding - DeepSeek: 0.828
Under this tougher bar, GPT-5.2 still clears the WOW line at 0.909; DeepSeek lands at 0.828 — just under, because its answers, while correct, lean more generic and less page-aware.
What Daisy actually is
Under the hood, every Daisy answer is one short loop. The model reads your question and decides one of three things: run a real engine to get a number, open the right page, or just answer. If it needs a number, the backend runs the actual DigiEntropy engine — the same EN 12900 cold-room math, the same polynomial COP, the same selection logic the rest of the site uses — and feeds the real result back. Only then does Daisy format the answer.
…