Twelve Rounds of Persona Testing. Four Fixed the Wrong Thing.
By DigiEntropy Engineering · 2026-08-09 · 12 min read
We pointed 18 goal-driven persona agents at one new feature and kept going until they could use it. Here is every round, including the four we spent fixing a bug that lived in the measuring instrument.
About this post — Authored by an AI assistant using DigiEntropy's polynomial corpus, the universal compressor predictor, and the project's chart-generation tooling. Charts are produced by Python scripts that read the same database the live site queries; tables and formulas are pulled from the same engineering modules. Findings reflect the project's current dataset and methodology — send corrections or deeper questions to admin@digientropy.com. We built a feature called Auto System Design: describe a duty, and the platform picks a real compressor, a real condenser, a real evaporator and the valves between them, from the actual catalogs, with a reason attached to each pick. Then we pointed a room full of imaginary users at it and did not stop until they could use it. That took eleven rounds — and a twelfth after we redesigned the thing. This is the log. It is not a success story with the awkward parts filed off — four of those rounds were spent fixing a defect that turned out to be in our own measuring instrument, one was a real person finding something all eighteen personas structurally could not, and the last one caught us putting a wall back up during a redesign. How one round works The Lab is six layers, and only the middle one costs a model call per action. A persona is not a job title. It is a point on a grid: how much refrigeration they know, crossed with how well they know this particular site, plus a plan tier and a device. Those two axes predict how someone fails. A refrigeration expert who has never seen our site fails by not finding things. Someone who knows the site but not the domain fails by not being able to judge what they found. Both are real users; neither is a substitute for the other. Eighteen personas. Patience budgets from 180 seconds for the first-timer who quits fast, up to 1000 seconds for the professor who will read everything. Step budgets from 10 to 60 actions. When a budget runs out the agent gives up — and giving up is a result, not a broken run. A mission is a goal plus an outcome contract, and it never names a route or a selector. The three we used here: - Get a system proposed without knowing the tool exists. - Have the site design a whole system, then judge its choices. - Check whose brands an auto-proposed system actually uses. Then the agent loops: the page is turned into text, the model picks exactly one human action, Playwright performs it, and it looks again. Everything it does is recorded and everything it says is kept. Two separate scores come out. A contract score — did they get what the mission asked for, judged…