Verify an AI agent audit trail yourself: our replay pack verifier is open source

In Evidence, not logs we said an auditor should be able to verify an AI agent audit trail without trusting the operator, and that two things to make that possible were planned: a replay pack and an open-source verifier. Both now exist. The verifier is on GitHub under Apache-2.0, and it runs offline. This post is what it checks, what a pass does not prove, and how to run it on a pack today.

Why the checker has to be someone else's

An audit trail for AI agents is only as good as the way you check it. If the vendor exports the record and the vendor's tool says it is fine, you have learned that the vendor agrees with the vendor. A raw log export does not help either. It is complete as far as anyone can tell, and the person telling you is the one who wrote it.

The fix is not a better dashboard. It is moving the check to the reader: give them the record in a form they can verify themselves, and give them the code that does the verifying, so they can read every check before they rely on it.

What a replay pack is

A replay pack is one zip file for one job. It holds the record of what governed agents did: the sessions that were opened, every permission that narrowed at a handoff, every refusal with its reason, and every redaction.

Inside the zip:

• The records, numbered in order, each one hashing its own content together with the hash of the record before it.
• The writes behind them. Each record points at the write that put it into the shared record, with the signatures of the organisations that confirmed that write.
• The organisations involved, with the root certificates their signatures chain to.
• The redactions: what was stripped from a message, or only the hash of it.
• A manifest listing every file with its size and hash, signed by whoever exported the pack.

One thing is deliberately left out. A write carries the request that caused it, and requests can carry the caller's credentials. The pack ships only the signed result of each write, never the request.

What the verifier checks

It runs six checks, in order, and prints one line for each.

• Manifest. Every file is listed, nothing is added or missing, every hash matches, and the exporter's signature verifies.
• Chain. Records run without gaps, times never go backwards, and each hash covers the record and the one before it. An edit, a deletion, an insertion or a reordering breaks it.
• Transactions. For each record, the write it points at stored exactly that record's value at exactly that key. The record is what was written, not a summary of it.
• Endorsements. Every signature on every write verifies, and each signer chains to its organisation's root. The report says how many of the organisations on that exchange signed ("1 of 2", "2 of 2"), and you can demand a minimum.
• Redactions. Where stripped content is included, it matches the hash recorded at the time.
• Summary. Counts by decision, and every refusal with its reason.

It needs no account, no network and no access to our systems. Exit code 0 means pass, 1 means a check failed, 2 means the pack could not be read.

When content is held back

Some exports should not carry everything. A record of a redaction is itself sensitive: the point of redacting was that the content should not travel. So a pack can be exported hashes-only. Stripped content is left out and only its hash ships, and the writes that carried that content ship without their payload.

The verifier does not pretend it checked what it could not see. Those entries are reported as a warning, "withheld, not verifiable", never as a pass. The rest of the pack is checked as usual, and the summary still counts the redaction.

Do not trust the pack's own roots

A pack names the organisations that signed it and brings their root certificates. A forged pack could bring forged roots, and every signature would check out against them.

So the report prints each organisation's root fingerprint, computed from the certificate rather than copied from the pack. Compare those with values the operator publishes somewhere you already trust, then pin them. With pins, only signatures that chain to a pinned root count, and the exporter's own organisation has to be among them.

What a pass does not prove

This list is in the README and in the tool's own explain command, because a verifier that oversells is worse than none.

• That the agents' outputs were correct or appropriate.
• That no call happened outside the recording path.
• When a write was committed, or where it sits in the shared record's order. Those are labels from the exporter, not signed by anyone else.
• That the pack holds every record that exists. It proves the records it holds were not altered or invented.

One more limit, against what we wrote last time. We said the verifier would recompute every decision from the badge and the step. This first release does not. It proves each record is the one the organisations signed and that nobody changed it afterwards. Recomputing the decision itself is not in this version.

Run it

Download a binary for macOS, Linux or Windows from the GitHub release and check it against the release's checksum file, or install it with Go:

1go install github.com/ares-networks/ares-verify/cmd/ares-verify@latest2ares-verify pack.zip3ares-verify pack.zip --json          # machine report4ares-verify pack.zip --pin <fp,...>  # count only roots you pinned5ares-verify explain                  # what each check proves and does not

The repository has sample packs with made-up organisations: one valid, one hashes-only, and four that are each broken in one way. The valid one reads like this (the root lines are each organisation's root fingerprint, printed in full):

1ares-verify valid.zip2  manifest        PASS  19 files, exporter sig ok3  chain           PASS  7 records, contiguous, 2026-10-14T09:02Z .. 09:41Z4  transactions    PASS  7/7 write sets match5  endorsements    PASS  2 of 2 parties on every transaction (Org1MSP, Org2MSP)6  redactions      PASS  2 records, 0 hashes-only7  summary               0 allowed, 1 refused, 2 redacted, 0 sent to a human, 1 revoked8  refused #5      send-external   reason: outside badge9  root                  Org1MSP b122c577fd42c08c4fe38a55cf712391eedd1913cf1a81eb2b81edc88db617a110  root                  Org2MSP 536f9ef1aa5270a5315c7822bd0ddbaa2dc679ab833e65f3300427c773cf97e711result: PASS

The release binaries are reproducible: building the same tag with the Go version the repository names gives byte-identical files, so you can check the binary against the source as well as against the checksum.

What to take with you

• An audit trail you can only check with the vendor's tool is still the vendor's word.
• A replay pack is one zip per job: records, the signed writes behind them, the organisations and their roots, redactions, and a signed manifest.
• The verifier is open source and offline, and it says plainly what it could not check.
• Pin the organisations' roots from a source you trust. A pack's own roots prove only that the pack is consistent with itself.
• A pass proves the record was not altered after the fact. It does not prove the agent was right.

The verifier is at github.com/ares-networks/ares-verify. Issues are open there: if you review agent systems, run it on the sample packs and tell us what a pack should also contain. The trust page has the rest of the evidence story.