Skip to content
ENع
Schedule call
All insights
Method8 min read

What a two-week AI audit should produce

The word audit has been ruined by consultancies who use it to mean two weeks of interviews and a deck of maturity curves. A useful audit produces artefacts you could hand to a different firm and have them build from. If it does not, you have bought an opinion.

The four things it should hand over

A costed build plan
Workflow by workflow: what it does, what it integrates with, what it costs to build and to run, and what it saves. Specific enough that another firm could quote from it.
An eval set
Drawn from the client’s own decided cases, tagged by category, with a baseline score. This is the artefact that makes everything afterwards measurable.
An integration map
Every system the work touches, how you get in, and what is broken about it. This is what actually determines the timeline.
A do-not-build list
The things that came up and should not be done, with reasons. Usually the most valuable page.

Test for a real audit: could a competent team you have never met build from this without calling the author? If not, the knowledge is still in somebody’s head and you have rented it rather than acquired it.

Where the two weeks go

  1. Days one and two: watch people work. Not interviews about the process, the actual process, over their shoulder. The gap between the documented workflow and the real one is where every surprise in the project lives.
  2. Days three to five: get into the systems. Read the schemas, call the endpoints, find out what the ERP will and will not give you. This is where timelines are decided and it cannot be done from a meeting.
  3. Days six to eight: build the eval set from decided cases, and run a baseline. This is the first honest number anybody gets.
  4. Days nine and ten: prototype the single riskiest step. Not a demo, the step most likely to sink the project, run against real data.
  5. The rest: cost it, write it, and present the do-not-build list first.

An audit with no code in it is a survey. The prototype is what converts "this should work" into "this works on your data at this accuracy", and that sentence is the whole deliverable.

The do-not-build list

This is the part clients remember, and the part most consultancies cannot write because everything on it is revenue they are declining.

  1. Workflows where the rule is stable and writable. Write the rule; it will be faster, free and testable.
  2. Workflows where the volume does not justify the build, however good the demo looks.
  3. Workflows where the real problem is a missing field upstream. We have twice found that cleaning one column removed the need for the model entirely.
  4. Workflows where two of the client’s own experts disagree on the right answer. Nothing can be built until that is resolved, and the resolution is a policy decision, not an engineering one.

What it should cost, and what that buys

Fixed scope, fixed price, fixed duration. An audit that can expand is a retainer with a different name.

The output should be usable by anyone. We say this to every client: take the plan to another firm if you want, or execute it yourselves. Several have. That is the point of writing it properly, and it is the only way the plan can be trusted, because a plan whose only possible executor wrote it is a proposal.

Want us to run this with you?

The Audit is this method pointed at your systems, with a costed build plan at the end of it.

Schedule call
Tell us the number you want to move.Schedule call