Your AI system cannot pass or fail. It has to be graded.
Software testing assumes the same input gives the same output. AI does not work that way, so the test suite that protects your codebase cannot clear the model sitting inside it. We build the thing that can: graded evaluation sets written with your subject-matter experts, rubric scoring with human review on a sample, regression history, drift monitoring, confidence thresholds and an audit trail.
- Proof engagement
- Public sector citizen services, 8 agents live
- Audit gaps at go-live
- Zero
- Audit to live rollout
- 8 weeks, same-week rollout across services
- Controls shipped before launch
- Lineage, policy checks, confidence scores, kill switch, on-call pod
- Model access
- Direct on Claude via the Anthropic Claude Partner Network, no reseller layer
Four things AI systems do that a pass or fail test cannot see
Every one of these is normal behaviour for a probabilistic system and a defect for a test suite.
The same input gives a different answer
Re-run the same prompt and the wording moves, sometimes the conclusion with it. An equality assertion turns that into a red build on Tuesday and a green one on Wednesday, and the team learns to ignore the colour.
Correct is a judgement, not an equality check
For a summary, a classification with grey edges, or an extraction from a badly typed document, there is rarely one right output. Someone with domain knowledge has to write down what good means. That is not testing work.
Quality falls quietly
Nothing throws an exception when upstream data shifts, the model version moves under you, or a colleague edits a prompt. The system carries on answering. It answers worse, and the first to notice is a customer or a regulator.
Right on average, wrong on the case that matters
A single aggregate score hides the small group of cases carrying the money, the legal exposure or the vulnerable citizen. Averages are a performance statistic, not a safety argument, so the tail gets its own suite and threshold.
Every testing habit that has to be replaced, and what replaces it
On the left, the move a good software engineer makes by reflex; on the right, what takes its place once the thing under test forms a judgement instead of returning a value.
- Unit test with a fixed expected valueGraded evaluation set built with subject-matter experts
- Assertion of equalityScoring rubric plus a judge model, with human review on a sample
- Green build means ship itRegression history against the previous version on the same set
- Bug report arriving from a userDrift monitoring on inputs and outputs
- Fuzzing to find crashesAdversarial and edge-case suite
- Exception and stack traceConfidence threshold that routes the case to a named humanAudit trail that reconstructs one decision end to end
- Quarantine the flaky testnothing on the other side
What goes into a graded evaluation set, and who writes it
The set is built in the Week 1 audit with the people who already review this work by hand, not by the engineers who wrote the prompt.
Cases chosen by your subject-matter experts
We sit with the people who sign this work off today and take the cases they remember: the ones that went wrong, the ones argued about in the team channel, the ones nobody junior may approve. Nobody has written them down.
A written rubric before any scoring starts
Each case carries the criteria it is graded on: grounded in the source, complete, correctly refused, right format, right tone for a citizen or a customer.
Disagreement between your experts is a finding
If two of them grade the same case differently, the rubric is not finished. Finding that early is worth the argument it causes.
Grades rather than pass or fail
A case scores across several criteria, so a partly right answer reads as partly right. That is what shows a prompt change improving faithfulness while quietly costing you brevity, which a binary result would have called a clean pass.
A judge model applies the rubric at volume
No team can grade thousands of cases by hand every release, so a model does the bulk of the scoring against the rubric your experts wrote.
People re-grade a sample every cycle
We track where the judge and the humans disagree. A judge nobody checks drifts in exactly the same quiet way as the system under test.
Adversarial and edge cases kept in their own suite
Prompt injection, contradictory instructions, missing fields, the rare category, the abusive user, the case in a language nobody planned for. Held and reported separately, so a strong average can never bury a weak tail.
Cases the system is supposed to refuse
Half of safe behaviour is knowing when to decline or escalate. Refusals are graded like any other answer, which also surfaces over-refusal, the failure mode that makes staff quietly go back to doing it by hand.
What the grading changes about an ordinary week
Four durations from the way this work runs, two of them speed and two of them cadence.
How one answer becomes evidence, from input to audit trail
The same path runs for every request the system handles in production, not only for the cases in the evaluation set.
- Capture
The input and everything around it
The prompt, the retrieved evidence, the model and version, the parameters, and the state of the upstream data at that moment. Anything missed here is a question you cannot answer when somebody asks why.
- Score
The answer against the rubric
Each criterion is graded by the judge and the reasoning is stored, not only the number. A score with no reasoning behind it is unarguable, and unarguable scores get ignored.
- Confidence
A confidence score on the decision
How well grounded the answer is in the evidence it was given, and how close the case sits to the edges of what the system has been shown to handle well.
- Route
Low-confidence cases go to a person
Below the threshold the case reaches a named reviewer with the evidence already attached. The threshold is a business decision made with you, and it moves only when the regression history says the system has earned it.
- Monitor
Inputs and outputs watched for drift
The mix of incoming cases, scores by segment, refusal rate, the share of work going to human review, latency and cost. Quiet degradation shows here weeks before it shows in a complaint.
- Record
The audit trail written as you go
A reviewer opens any single decision months later and sees the input, the evidence, the version, the score, whether a human looked and what they concluded. Reconstructed afterwards this is guesswork. Recorded at the time it is a record.
The public sector rollout: controls were built before the agents went live
Citizen services, eight weeks from audit to live, and the order of the work is the whole point of this case.
Controls went live with the agents
Eight agents went into citizen-facing services with lineage on the data feeding each one, policy checks on what an agent may do, confidence scores on every answer, a kill switch and an on-call pod behind it.
Rollout was a deployment decision, not a fresh approval
None of the controls was added once a pilot had gone well. They were part of what went live, which is why the rollout across services happened in the same week instead of stalling in a review committee.
What zero audit gaps actually meant
A reviewer could pick any decision and follow it back: which record the answer came from, which policy applied, what the system scored itself, whether a person looked, and what they concluded.
We confer no compliance status on anybody
We are not claiming a certification here, for us or for you. The plain version is that when the auditors asked, nothing was missing, because nothing had been left unrecorded.
Where the models sit in this
We build directly on Claude through the Anthropic Claude Partner Network, with no reseller layer. Sonnet does production reasoning, Opus takes the hard analyses, Haiku carries the cost-sensitive long tail, and the harness decides which a step deserves.
Measure first, then choose the model
Choosing the model first and measuring afterwards is how teams end up paying premium rates for a step that never needed it.
What eight weeks on that engagement produced
Every figure here is from the public sector citizen services work.
We use AI to test AI, and then we check the tester
Grading thousands of outputs by hand does not scale, and grading them purely by machine is how you get a system that passes its own exam.
- Already built
Harnesses we bring with us
Evaluation runners, scoring rubrics, drift monitors and audit-trail capture already built and running. What changes per engagement is the criteria, not the machinery.
- AI first pass
A model grades the bulk
Claude scores outputs against rubrics your subject-matter experts wrote, at a volume no human review could reach. Every score carries its reasoning, so a disagreement can be inspected rather than argued.
- Human check
People review a sample, always
A sample of machine grades is re-scored by hand every run, and the agreement rate between human and model is tracked. When it slips, the rubric is wrong, and better found here than in production.
- Over time
Regression history, not a single score
Every run is kept, so you can show a system got better rather than merely different. This is the artefact risk and audit actually ask for, and almost nobody has it.
Where the evaluation work sits in the three phases
The same three phases we run on every engagement, described by what actually gets graded in each.
- Week 1
Audit. Fixed fee, two calls
We take the use case apart with your experts and write down what good means, where the harm sits, which decisions must reach a human, and what the system is expected to refuse. You keep the one-pager either way.
- Weeks 2 to 6
Ship. A pod of three
The pod builds the evaluation set, the rubric, the judge with its human sampling loop, the adversarial suite, the confidence routing and the drift monitoring. Something works by Week 3: a scored baseline you can disagree with.
- Week 7 onwards
Operate
Quarterly reviews of the evaluation set as your cases and policies change, on-call governance, and the regression history kept current. The first time someone asks whether last month's change made things worse, the answer takes ten minutes.
The APAC margin agent, and the day it earned trust
The second engagement, in a retail grocery business above $500M in revenue, where evaluation is what made the signature possible.
Where Day 14 sits inside that first month
The two dates from the margin agent drawn at the distance they actually sat apart.
The objections engineering leaders raise, answered plainly
The questions that come up on the first call, including the ones where the honest answer is no.
Is AI writing our system, then?
No. AI does first passes and we review every one. The mapping decisions, the thresholds and the rules a regulator would ask about are made by the people whose names are on the work. What AI removes is the transcription.
We already have unit tests and CI. Why is that not enough?
Keep all of it. Those tests still protect the code around the model: retrieval, parsing, tool calls, API contracts, rate limits. What they cannot do is decide whether an answer was good. Evaluation sits beside CI and reports into it.
Can the model not just grade itself?
A judge model does most of the grading, and it has to, because of volume. But we sample it against human graders every cycle and report where they disagree, by criterion. An unchecked judge drifts just as quietly.
How much of our experts' time does this take?
It is concentrated rather than continuous. The heavy involvement is the Week 1 audit and the first grading rounds, where they are mostly recognising cases they have already handled. After that it is periodic sampling and a quarterly review.
Will an evaluation harness slow our releases down?
It removes what is currently slowing them. The delay is rarely testing. It is the week spent arguing about whether the new prompt is better than the old one, and a scored comparison settles that in an afternoon.
We have no labelled dataset. Do we need to build one first?
No. The set is built during the engagement from your real cases and your experts' judgement. Waiting until a clean labelled corpus exists is one of the main reasons AI work sits in pilot for a year.
Can you evaluate a system your team did not build?
Yes, and it is common. We need access to the system or its API, the prompts and retrieval configuration, and time with the people who know what a wrong answer costs. We hand over severity and a fix.
Are you certifying us against a standard?
No. We do not issue certifications and we do not claim any compliance status on your behalf. What we hand over is evidence: a graded baseline, a regression history, live monitoring, and an audit trail your auditors can challenge.
The rest of the practice
These are genuinely different jobs with different ways of failing. Most engagements start in one of them.
- Data engineeringJobs that finish, migrations that reconcile, and storage that stops paying for cold data.
- Data integration and governanceOne set of records the finance team and the operations team both accept.
- Data platforms and modernisationOff the platform you outgrew, without a twelve-month freeze on new reporting.
- Apache SupersetSuperset built and run by people who commit to the project, including embedding and Kubernetes.
- Command centresThe one screen an operations floor runs the day from, not another dashboard.
- AI audit and roadmapOne week, and you know which two or three AI initiatives are worth building.
- AI governanceGovernance you can defend in a board meeting and audit on demand.
- AI agentsOne reasoning agent on Claude, on your live data, with evidence and a kill switch.
- Data agentsAgents that watch the data, flag what moved, and explain what is driving it.
- AI use case, guaranteedOne production agent in eight weeks, or you do not pay for the build.
- Applications and automationThe system your team works in all day, built or replaced in slices.
- Application modernisationThe system nobody wants to touch, replaced a slice at a time rather than rewritten.
- System integrationSystems that stop disagreeing about the same customer, order and item.
- Process digitisationThe process that still runs on paper, WhatsApp and one shared spreadsheet.
Book the Week 1 audit and find out what your AI system is not being graded on
Fixed fee, two calls, and a one-pager you keep either way: the decisions that need a human, and the grading criteria nobody has written down. Email hello@woodfrog.tech. We work under NDA and to a fixed scope, from Pune, across APAC, India and the US.