Skip to content
woodfrog

·

AI
Public sector, citizen servicesCase study, anonymised

AI your auditors will sign off.

A public-sector team had AI that worked, but nobody would approve it. We built eight AI agents to be approved from the start. All eight now run in production with no audit gaps.

One agent answer, ready for auditSchematic
Agent 4 of 8 · answer
✓Traced to source. Every input it used is listed
✓Rule checks passed. Checked before the decision
✓Confidence shown. Weak evidence is visible
On-call team watchingStop this agent
0Audit gaps across every agent in production
8AI agents running in citizen services
Same weekApproved and rolled out in the week it was presented
8 weeksFixed scope, fixed fee
WhoA public-sector team in citizen services, anonymised
EngagementEight weeks, fixed scope, fixed fee. Three phases: audit, build for sign-off, then hand over and operate.
BuiltEight AI agents with a traceable source, rule checks, a confidence score on every signal and a stop button each
Before and after

The AI did not change. The way it was built did.

Before
  • Looked good in a demo
  • No one could say what the AI used for an answer
  • No plan for when it was wrong
  • Checks ran after the answer had gone out
After
  • Every answer traces back to its source
  • Rule checks run on the inputs, before a decision
  • Weak evidence is visible, not hidden in a confident answer
  • Each agent has its own stop button, and a team on call
The problem

The AI worked. Nobody would sign it off.

The output looked good in a demo, but approval never came. No one could answer what the system had used to reach a conclusion. No one could say what would happen if an agent got one wrong. This was citizen services, so it mattered.

What was broken
  • No clear source behind each answer.
  • No plan for when an agent is wrong.
  • A weak basis hidden inside a confident answer.
  • Checks that ran after the answer had already gone out.
  • Governance that fades after go-live.
You might be here if
  • You cannot say what your AI used to reach a conclusion.
  • You cannot say what happens when an agent gets one wrong.
  • Confident answers might be hiding weak evidence.
  • Your checks run after the answer has already gone out.

Why it mattered. "An agent that cannot be audited cannot be deployed." The model was not the blocker.

The approach

Start from what the auditors need to approve.

We began with what the audit team would need, not with what the model could do. It sounds like a small change of order, but it decides what gets built in Week 2. Traceability, rule checks, confidence scores and the stop button went in with the build, not added later. A late fix is exactly what a reviewer learns to spot.

The input
Source dataWhere the information comes from
Signal an agent usesWhat it reasons from
Controls on every signal, built by woodfrog
Trace to sourceSee what each answer used
Rule checksOn inputs, before a decision
Confidence scoreShows weak evidence
The result
Eight AI agentsEach with its own stop button
On-call teamKeeps controls live after launch
0 audit gaps
Deliberately not doneChecking after the answer has gone out, or adding governance just before launch
Controls were built in with the agents, in the same weeks.schematic
The solution

Controls built in with the agents, not after.

What was built
  • Week 1: a one-pager that wrote down what the audit team would need to approve.
  • Trace to source on every signal.
  • Rule checks on the inputs feeding a decision.
  • A confidence score on every signal.
  • A stop button for each of the eight agents.
  • An on-call team handed over at the end, so controls stay live after launch, "which is where most AI governance quietly lapses."
The controls were built in the same weeks as the agents. There was no separate governance step before launch.
How it ran, eight weeks
Wk1Wk2Wk3Wk4Wk5Wk6Wk7Wk8
Audit
What approval needs
Eight agents
Built to be approved
Controls
Built in with the agents
Hand over
On-call team
The impact

Approved in the week it was presented.

Each objection got its own control
No clear sourceTrace to source on every signal
No plan for a wrong answerA stop button for each agent
Weak evidence hiddenA confidence score on every signal
Checks after the answerRule checks on inputs, before a decision
Governance fadesAn on-call team after launch
Get AI your auditors will approve.A short call, or record a message. A person replies within one business day.
Contact usCertified Services Partner, Claude Partner Network