Build log 1: The five checks I put around an AI workflow before it touches real work
This is the first entry in a build log. I build AI workflows end to end, and I will write down how, including the parts that went wrong. Today: the five checks I put around a workflow before it touches real work.
The model is the smallest part of a production AI workflow. What decides whether a team can rely on it is everything around it. After years of running production platforms, I treat these checks the way I treat a deploy pipeline: boring, written down and run every time.
The five checks
A narrow job. One task with a clear end, for example “draft a reply to a support request” and not “handle support”. A narrow job can be tested. A broad one can only be hoped for.
Approved sources only. The workflow answers from documents and systems I name, and says so when it does not find the answer. If it cannot cite where something came from, I do not ship it.
A person for anything that matters. Drafting is automatic. Sending, paying, deleting or promising something needs a human approval step, and the workflow cannot skip it.
An easy handoff. The person on the other end can always reach a human in one step, and the workflow tells them when they are dealing with AI. It is also the simplest way to meet the transparency expectations in rules such as the EU AI Act, which I treat as a practical review and not legal advice.
A list of real questions I re-run. I collect twenty to fifty real questions, with the answer a good person would give, and run them on every change to the prompt, the sources or the model. A change that makes an answer worse shows up the same day, not three weeks later in a customer message.
Why this order
The first two checks shrink the problem. The third and fourth limit the damage when something still goes wrong. The fifth stops quality from drifting without anyone noticing. Skipping any one of them is how a demo that worked becomes a system nobody trusts.
What comes next in this log
I will build one small workflow in the open and report what each check caught: what the question list found, where the handoff was missing, and what it cost to run. Failures included.
If you have a workflow that works in a demo and not yet in production, I am happy to look at it: book a call. More on how I work is on the Work with me page.