Inside Daisy: How We Measure and Improve Our AI Co-pilot
By DigiEntropy Engineering · 2026-04-05 · 11 min read
Our in-app AI engineer was useful but not 'wow'. So we built an eval to measure it across 33 questions, found four failure patterns, and fixed them — here's the whole story, in charts.
About this post — Authored by an AI assistant using DigiEntropy's polynomial corpus, the universal compressor predictor, and the project's chart-generation tooling. Charts are produced by Python scripts that read the same database the live site queries; tables and formulas are pulled from the same engineering modules. Findings reflect the project's current dataset and methodology — send corrections or deeper questions to admin@digientropy.com. Daisy is the AI co-pilot inside DigiEntropy. You ask a refrigeration question in plain language — size a freezer, which refrigerant is most efficient, is this compressor a good fit — and it answers with real numbers, runs the platform's calculation engines for you, and can even drive the pages on your behalf. It worked. But it did not wow. It would refuse questions it should have answered, reach for the wrong tool, or state a number it had quietly made up. This post is the story of how we stopped guessing whether Daisy was good and started measuring it — then used those measurements to make it genuinely smart. What Daisy actually is Under the hood, every Daisy answer is one short loop. The model reads your question and decides one of three things: run a real engine to get a number, open the right page, or just answer. If it needs a number, the backend runs the actual DigiEntropy engine — the same EN 12900 cold-room math, the same polynomial COP, the same selection logic the rest of the site uses — and feeds the real result back. Only then does Daisy format the answer. The one rule that never bends: every engineering number comes from a real engine, never the model's memory. A made-up COP is the cardinal sin. That rule is what makes an AI answer trustworthy to an engineer. The problem: "useful" is not "wow" The complaint was specific and fair: Daisy was not sophisticated enough to handle out-of-the-box questions. Ask it a textbook concept — why does COP drop when the evaporating temperature falls? — and it might refuse. Ask it a design trade-off — booster or parallel compression for a CO2 supermarket? — and it would say "I can't do that yet." Ask it something underspecified and it would stall. The trouble with "it feels flat" is that you cannot fix a feeling. You need a number. So the first job was not to change Daisy — it was to measure it. Test design, part 1 — five levels of difficulty A good test set spans from trivial to genuinely hard, because an assistant can ace easy lookups and still fall apart on real questions. We wrote 33 test questions across five levels: - L1 — direct lookup. Everything is given. COP…