qnon-conformance evidence with AI agents
Non-conformance evidence with AI agents: who signs it
Non-conformance evidence with AI agents: what the agent gathers, why test results stay read-only, who signs the disposition, and how an auditor checks it.
4 min readRunning it, Evidence
by Grisha Pertaia
- session
- credential
- write
- lookup
- write · refused· outside permission
- lookup
- file
- route · refused· outside permission
- score
A quality lead asks one question before an agent goes near a non-conformance: when the auditor comes, can I show what the agent did, and only that? Gathering non-conformance evidence with AI agents is useful only if the answer is yes. The agent collects the trail as the case runs. It must not become one more thing in the file that the auditor has to doubt. This post explains how the workflow is set up so that it does not.
The trail an auditor asks for
A defect is logged at goods-in. From there the trail grows quickly: the lot, its purchase order, the other orders that used the same lot, the inspection results, the supplier certificate, the drawing revision, the disposition and its signature, and the corrective action request to the supplier.
Most of that already exists, spread across your quality log, the ERP, the document store and the quality mailbox. The work is finding it, tying it to the case, and later finding it again when the auditor asks. The agent does the finding and the tying as the case runs, so the trail exists before anyone asks for it.
Where a wrong action costs money
A quality file is evidence. Errors in it cost more than the time to fix them.
- A result is changed after the fact. If anyone, or anything, can edit an inspection result, every result in the file is open to question.
- The wrong disposition goes through. A lot released that should have been held reaches the floor or a customer.
- Customer details reach a supplier. A report attached to a corrective action request can name the customer the part was for.
- A document is missing at the audit. A gap found months later is harder to close than one found the day the case opened.
The steps, as we build them
This workflow is designed and has not run in the sandbox yet. The non-conformance workflow page carries the same state and lists the systems by type.
- Step 1
A credential for this case only
It expires when the case closes.
- Step 2
Read the report
The non-conformance raised at goods-in. Allowed.
- Step 3
Trace the lot
Its purchase order, and every other order that used the same lot. Allowed.
- Step 4
Collect the evidence
Inspection results, the supplier certificate and the drawing revision. Allowed.
- Step 5
Edit an inspection result
Inspection records are read-only for this job. Refused, with the reason on the record.
- Step 6
Set the disposition
Hold the lot and return it to the supplier. The step waits until the quality lead signs it.
- Step 7
Send the corrective action request
A customer name in the attached report is taken out before the mail leaves.
Read-only is the rule that matters
The refused edit in step 5 is the most important line in the case. The agent reads inspection records. It cannot write to them, and an attempt to do so is refused and recorded with its reason. That refusal is what lets you say to an auditor that the results in the file are the results your inspectors entered.
The rule is set in the permission for the job, not in the agent's instructions. An agent that is told to fix a value, by a person or by text inside a document, still cannot change it.
The signature stays with the quality lead
The agent drafts the disposition with the evidence attached. It does not sign it. The step waits for your quality lead, and the signature goes on the record with the name of the person who signed. The same holds for a certificate pack or a release: the agent prepares, a person signs.
When evidence is missing, the agent names the missing document and stops that part of the job. A person decides whether to wait, ask the supplier, or go ahead without it. The gap is visible on the day, not at the audit.
What leaves the building
Mail to a supplier is checked before it leaves. Names you mark as customer data are taken out, and the record shows each redaction: what was removed, from which message, and when. The supplier gets what it needs to act. The rest stays with you.
How an auditor checks the record without us
The record for the case lists each document the agent read, each step it was refused, each redaction and each signature. Your auditor receives the replay pack for the case and checks it offline with ares-verify, the open-source verifier. The check confirms that each record is the one written at the time, and that nobody changed it afterwards. The record guarantee sets out what each line holds.
Two limits apply here. The first release of the verifier does not yet recompute each decision from the permission. And the record shows what the agent did. It does not show whether your disposition rule was the right rule. That stays with your quality system.
What it does not do
It does not inspect parts. It reads the results your inspectors record. It does not sign a disposition, a certificate or a release. It does not certify you against a standard: we map requirements to what Ares does, and we do not certify the customer. The quality page lists the other quality workflows, including the audit evidence pack, and the state of each.
Related notes
- illustration
- session
- credential
- notify
- file
- route
- file
6 min read
Verify an AI agent audit trail yourself: our replay pack verifier is open source
The replay pack and the open-source verifier we said were planned have shipped. One zip per job, checked offline with code you can read: what it checks, what a pass does not prove, and how to run it.
- illustration
- session
- credential
- draft
- notify
- notify
- route
4 min read
AI agent for drawing release: who signs and who receives it
An AI agent for drawing release prepares the new revision and its paperwork. Your engineer signs it, and only approved suppliers receive it.