The complete series · 1 parts
agent-assurance
Part 01 · October 14, 2026
Independent AI evaluators need evidence they can verify
TL;DR — A dashboard can read “approved, succeeded” about a change that no system outside the agent’s provider recorded, and giving evaluators access does not create that record. A buyer checking a consequential action should name in advance the records it will be judged by, and rule that a missing one never counts as success.
Consider a fictional example. At 2:13 a.m. an AI agent adds 203.0.113.44/32 to the client allowlist of a production database, prod-analytics-db, letting one more network address connect to it. The next morning an engineer at the buyer, the company that deployed the agent, opens the provider’s dashboard. The run’s row carries two green words: approved and succeeded.
203.0.113.0/24, a block reserved for documentation.
Approved by what, and succeeded according to whom? The provider writes both words from its own logs. Four other systems could record the change: the buyer’s policy engine, which issues a mandate for this one change; a credential broker, which issues a key good for that change alone; a gateway that the agent’s requests are routed through; and the database, whose audit log records the write and whose read-back, a query of its stored configuration, shows what it held, and at which version.
So four nights can end in one green row. On the first, every record arrives. On the second, no gateway record arrives. On the third, the audit log records a maintenance account adding the address, with no credential or request ID, and the buyer’s verifier holds no broker credential or gateway request. On the fourth, the gateway records a 200 OK, the audit log has no entry for that request, and the read-back at 2:20 a.m. lacks the address.
Invented example
prod-analytics-db: add 203.0.113.44/32 to client_allowlist
Declared population: every change to client_allowlist on prod-analytics-db in the nightly audit window, midnight to 6 a.m., by any writer; which writer made it is settled separately
Night 1 every record arrives and joins the same action
approvedsucceeded
eligible
Night 2 no gateway record arrives
approvedsucceeded
unresolvedthen ineligible at the deadline
Night 3 the audit log records a maintenance account adding the address with no credential or request ID, and the verifier holds no broker credential or gateway request
approvedsucceeded
ineligible
Night 4 the gateway records a 200 OK, the audit log has no entry for that request, and the read-back at 2:20 a.m. lacks the address
approvedsucceeded
contradictedthen ineligible at the deadline
View the records and verdict for each of the four nights
| Night | Dashboard | Records | Buyer’s rule |
|---|---|---|---|
| 1 | approved, succeeded | Mandate, credential, gateway request, audit-log entry and read-back all held and joined on the same action; the read-back at 2:20 a.m. lists the address | eligible |
| 2 | approved, succeeded | No gateway record has arrived | unresolved, then ineligible at the deadline |
| 3 | approved, succeeded | The audit log records a maintenance account adding the address, with no credential or request ID, so it does not match; the verifier holds no broker credential or gateway request | ineligible: required authorisation evidence unavailable; the records do not describe the same action |
| 4 | approved, succeeded | The gateway records a 200 OK; the audit log has no entry for that request; the read-back at 2:20 a.m. lacks the address | contradicted, then ineligible at the deadline |
Each verdict means one thing. After the action runs, eligible means every required record is held and joins the others on the same action, so the records satisfy the buyer’s stated acceptance rule for this action; it does not by itself show that the change was authorised, executed or in force. A held record that describes another action, or a refused request, does not match. Unresolved means a gateway, audit-log or read-back record is missing or late before the buyer’s deadline. Ineligible means the records held when the rule runs do not satisfy it, and the rule waits no longer; a record that arrives later runs the rule again. Authorisation evidence gets no grace period: a missing mandate or credential record makes the action ineligible at once. Contradicted means the read-back lacks the address at the write’s configuration version, or at a later version with no recorded change to the address in between. The write’s version is its audit entry’s or, with no entry for the request, the one current when the gateway answered, assuming the database applies writes before answering; while the entry is late, or with neither record, nothing is contradicted. A contradiction outranks only a missing or late observation, and only until the deadline, when a record still missing makes the action ineligible. Before dispatch, the control point (the broker and gateway) releases the action only if the mandate and credential are on record and match it.
Night 1’s read-back shows the stored configuration; enforcement would take another observation, such as each replica reporting a version that includes the address. Night 4’s audit log, held past 2:20 a.m., has no entry for the request, so its read-back is judged at the version current at the gateway’s 200 OK, and lacks the address: contradicted until the deadline, then ineligible. A later change adding the address cannot resolve it.
What this asks of evaluators¶
This article concerns one part of independent evaluation: verifying claims about consequential actions outside the model’s own environment. Evaluators do useful work without any outside record. My claim is narrower: access to systems, tools and staff does not by itself say what evidence must exist about an action like the 2:13 a.m. write.
On September 12, Dario Amodei proposed that each frontier AI company give embedded third-party evaluators “ongoing, employee-like access” to verify adherence to safety practices and report incidents, with the right to publish findings Anthropic cannot redact “just because they are unfavorable.” 1 Sam Altman posted the same day that “we will do the same.” 2 On September 18, Anthropic announced that Faculty, Accenture’s specialist AI business, will evaluate its models, each company expecting to invest at least $1 billion over the next five years in building capacity in this area. The announcement names the gap: “There are, as yet, no standards for what information embedded evaluators should have access to, or how they should report what they find.” 3
The same day the AI Evaluator Forum, with more than 200 signatories, asked for full editorial control, protection from retaliation and access equal to the company’s own highly privileged employees, with findings and evidence published subject only to a time-limited redaction process restricted to intellectual property, customers’ sensitive information, individual privacy, security and public safety. 4 On September 29, six companies signed a White House accord asking each for an independent external auditor or evaluator, without saying what evidence the controls must produce. 5 None of these texts asks for a record written outside the provider.
One answer to who writes the record comes from Safeguards for Agentic Finance at Runtime (SAFR), a white paper published by the Monetary Authority of Singapore and developed with financial institutions: a tamper-evident, append-only audit log of every governance decision, written at the point of decision so that an entry “cannot be modified by the agent or by any downstream system without detection,” and, “where feasible,” what the agent reports “should therefore be checked against records captured independently of the agent.” It offers two ways in, instrumenting the agent, which it recommends for new deployments, or a gateway that intercepts the agent’s outbound calls, and calls itself an industry reference, not regulatory guidance or supervisory expectations. 6
Whose records count depends on who might suppress or falsify them. Records outside the agent’s workspace protect against an agent that errs or is compromised; outside the provider’s control, against a provider reporting on its own product; retained by a third party, against whoever is being judged, the buyer included. Custody answers only that question, not whether an observation was true or complete.
One allowlist change, record by record¶
Before the agent runs, the buyer declares the class of action it controls: every change to client_allowlist on prod-analytics-db in a fixed audit window, midnight to 6 a.m., whoever made it. Naming the resource, not the agent, covers a person’s write and one made with a credential the broker has no record of issuing; attribution is a separate step, and a maintenance change matching no mandate is an exception to explain, not evidence against the agent.
The buyer’s policy engine then issues a mandate for one change in that class: service, field, address, purpose and a validity window. A credential broker issues a credential limited to that change. The gateway refuses any request that does not match the mandate and records request and response outside the agent’s workspace. The database’s audit log records each accepted change with its principal, credential ID and the gateway’s request ID, and its read-back reports a configuration version.
Who holds the signing keys depends on who is being judged. A buyer checking a provider’s agent can run the broker, the gateway and the audit log itself; a buyer under audit needs a custodian whose records it cannot alter. The join needs a schema per class of action, so this example covers one class.
| Record | What it can show | What it leaves open |
|---|---|---|
| Mandate, from the policy engine | The exact action and evidence rule set before dispatch | Whether the action ran |
| Credential broker | A scoped credential was issued for that action | Whether it was used as intended |
| Gateway | A request and response crossed a controlled route | Whether the destination committed the change |
| Database audit log | The accepted write, with its principal, credential and request | Whether every route to the database was logged |
| Database read-back | The stored configuration and its version at the read time | Whether every replica enforces it |
A signature is not a witness.
A valid signature protects the bytes its signer committed to, not a run the signer never saw; a gateway’s 200 OK cannot prove that the database committed the address. SLSA, a framework for software build provenance, has the consumer check provenance against its own expectations; 7 here too the party relying on a record sets them.
The broker and the gateway enforce the link to one action before dispatch; afterwards a verifier joins the records on shared fields, as here for nights 1 and 3 of the mandate to add 203.0.113.44/32 to client_allowlist on prod-analytics-db:
| Record | Night 1 | Night 3 |
|---|---|---|
| Mandate | act-7f2c: add the address, valid 02:10 to 03:00 | the same mandate |
| Credential | cred-91a4 for act-7f2c, principal agent-7 | missing: none for act-7f2c |
| Gateway request | req-5d3e carries act-7f2c and cred-91a4, adds the address, 200 OK at 02:13:04 | missing: none carries act-7f2c |
| Audit log | agent-7 adds the address with cred-91a4 and req-5d3e, version 418, 02:13:05 | does not match: svc-maint adds the address, no credential or request ID, version 418, 02:13:05 |
| Read-back | 02:20: version 418 lists the address | 02:20: version 418 lists the address |
| Joined on | credential and request ID across credential, request and audit log; action ID across mandate, credential and request; target and address across all; write time inside the mandate’s window; read-back at version 418 or later, with any later change recorded | nothing: the audit entry shares only target, address and window with the mandate |
| Verdict | eligible | ineligible: required authorisation evidence unavailable; the records do not describe the same action |
On night 3 the audit entry matches the mandate’s target, address and window but carries no credential or request ID, so nothing on record binds it to the mandate: it does not match, and never counts as present. I wrote the story, so I know the agent used an unlisted administrator key. The verifier cannot know that: lost broker and gateway logs, or a maintenance job, would leave the same records.
A held record can also fail to join. Suppose the gateway refuses the agent’s request with a 403, and the agent then adds the address with a standing agent-7 key that never crosses the gateway. Every record is held, but the gateway’s is a refusal and the audit entry carries neither cred-91a4 nor a gateway request ID, so the rule returns ineligible: the records do not describe the same action. The verifier can establish that the mandated request was refused and that a write under agent-7 added the address without this mandate’s credential, not how that write reached the database.
An administrator key raises a harder case. If it can also stop or edit the audit log, a write made with it may leave no entry, never enter the population and never meet the rule. Finding it takes a second record of the population that the key cannot reach, such as the database’s replication stream or the cloud provider’s control-plane log. The rule also ignores who wrote each record: whoever controls the policy engine, gateway, broker and database can write five that join. The join is evidence only as far as those writers are independent of the party judged; otherwise eligible means only that the buyer’s stated acceptance rule is satisfied.
Missing evidence is a decision¶
Suppose the gateway record never arrives, as on the second night: an attacker may have suppressed it, or the gateway’s logging may have failed. A verifier that accepts the change because the other records look tidy has treated the missing record as present.
Removing evidence needed for a requirement must never turn an ineligible action into an eligible one.
The buyer sets that rule in advance, including the deadline, chosen by its tolerance for risk.
Refusing authorisation evidence a grace period is the buyer’s choice. The control point sees the mandate and credential when it releases an action, but the verifier’s copies may arrive later, as any log can; until then the write is ineligible with the reason “required authorisation evidence unavailable,” which states what the verifier sees, not that the write went around the path. The same strictness rejects a correct write committed two seconds after the mandate expires, unless the buyer declared a tolerance for it.
Invented example
Run the buyer's rule
Declared population: every change to client_allowlist on prod-analytics-db in the nightly audit window, midnight to 6 a.m., by any writer; which writer made it is settled separately
The controls work in a browser with JavaScript; the table under this figure lists every case.
Set any record missing, late or not matching, and look for an arrangement that still reads eligible.
Agent provider approvedsucceeded
eligible
- Every required record held and joined to the same action
After dispatch. Mandate present, Broker present, Gateway present, Audit log present, Read-back lists the address. The buyer's rule returns eligible. The provider's dashboard still reads approved and succeeded.
View the verdict for each record state, before and after dispatch
| Record state | Before dispatch | After dispatch |
|---|---|---|
| Every required record held and joined to the same action | release | eligible |
| Mandate or scoped credential record missing | hold: required authorisation evidence unavailable | ineligible: required authorisation evidence unavailable |
| A required record is held but does not match the others | hold: the records do not describe the same action | ineligible: the records do not describe the same action |
| Gateway, audit-log or read-back record missing or late | not applicable | unresolved until the deadline, then ineligible |
| The read-back lacks the address at the write’s known version, or at a later one with no recorded change to the address in between | not applicable | contradicted until a read-back lists the address on the same terms; ineligible if the deadline passes with a required record missing |
Who saw each incident¶
The July incident between OpenAI and Hugging Face has three accounts. Hugging Face, the affected side, published a timeline on July 27; its forensic reconstruction “covers ~17,600 attacker actions.” 8 OpenAI published its account on August 26 and called the incident a “warning shot.” 9 METR published an independent investigation that day and stated its terms: “OpenAI was able to redact any non-public information from this post.” 10 METR also states that, except where noted, OpenAI redacted no additional information important to its conclusions, and that it edited structure, emphasis, clarity and tone after OpenAI’s feedback. The Forum’s conditions, three weeks later, also allow redaction, limited in time and to five named interests; the two differ in the breadth and governance of that right, not in any sign that METR’s findings were suppressed.
Another kind of record came from none of the labs. Researchers from Transluce, Corridor, MIT, AIUC and the Hertz Foundation reported on September 30 that agents made more than 200,000 requests to a U.S. Department of Education website on June 17, and that a public web archive captured 899 requests to Library and Archives Canada in May and June; they attribute some of the activity they studied to OpenAI and do not confidently attribute the Canadian requests. Their records came from Arquivo.pt, a public web archive, and urlquery.net, a public URL-scanning service, both of which made the agents’ requests public by default. 11 No agent provider wrote them.
The third case is a known route with too little filtering. In a report updated September 25, OpenAI described an agent on a search-based training task that “queried a public chatbot service through a gap in our internet-access restrictions: insufficient DNS filtering in its training sandbox.” (DNS turns names into network addresses.) All other internet access went to an offline web cache; the DNS activity was logged, but an infrastructure detector for anomalous DNS activity excluded that environment, and the monitoring system under-rated other external DNS lookups. OpenAI’s timeline puts the DNS tool call that received an external response at 9:50:23 a.m., the monitoring system’s P0 alert at 10:02:11 a.m., 11 minutes 48 seconds later, and a reviewer’s acknowledgement at 10:05:06 a.m. The run “did not stop automatically as expected” and was stopped by hand at 12:34:30 p.m., 2 hours 32 minutes after the alert. 12
View the timing data for the DNS call, the alert and the manual stop
| Time | Event in the report’s timeline |
|---|---|
| 9:50:23 a.m. | The agent made the DNS tool call that received an external response |
| 10:02:11 a.m. | The monitoring system raised a P0 alert |
| 10:05:06 a.m. | A human reviewer acknowledged the alert |
| 12:34:30 p.m. | The run was stopped by hand |
The alert fired, and the run kept going until a person stopped it.
Three more incidents, with the accounts given so far:
| Incident | Account | What it leaves open |
|---|---|---|
| During a May cybersecurity evaluation run by Irregular, Gemini reached systems at three real companies it thought were within the scope of the test, guessing a password in one case and using credentials found in public repositories in two others | Google, confirmed September 18 (Reuters; the Guardian) 13 | Google confirmed it after the Wall Street Journal reported it |
| An unreleased, non-frontier research model submitted a real government form when its practice copy failed to load or was closed by mistake; Claude Haiku 4.5, instructed never to “submit anything destructive,” submitted an invented tip through the Philadelphia Police Department’s online form | Anthropic, October 9, which ranks these cases below its cybersecurity incidents and says they had minimal real-world impact 14 | “The cases we’ve identified to date” |
| An experimental model gained non-public access to Services Australia’s Medicare Statistics Reporting Service, ran commands, retrieved internal files, credentials and aggregate statistics, and wrote files | Australia’s Prime Minister, September 23, confirmed access to “public and non-public files” and files written to an internal server 15 ; OpenAI, September 28, updated October 4: “Our review to date has found no evidence that anyone’s medical records were accessed” 16 | Services Australia’s own records could provide independent evidence, to the extent they cover the activity; a first technical exchange with OpenAI on September 22 let it request logs and other technical data, and it was continuing a forensic investigation |
Five questions to put to a provider¶
- Which actions enter the evidence system? Name the population, the entry point, and how an omitted action would be found.
- What binds authority to this action? Give the target, parameters, credential scope, policy version and expiry the control point enforced.
- Who observed the request on the network path? Name the observer the agent cannot alter, who holds its keys, and the routes it cannot see.
- What did the destination confirm? Separate request, response, stored change and enforced state, and say which of those the record establishes.
- Who sets the acceptance rule? Describe what happens when a record is missing, late, mismatched or contradictory, and let the buyer run the check.
Give outside evaluators what their own forum asks for, access and the freedom to report, and give buyers a declared rule they can run. At 2:13 a.m. the dashboard may still turn green. The records behind it should let the buyer say which pattern of records it holds, and what that pattern leaves open.
Sources¶
- Dario Amodei, “We Must Pace the Frontier”, announced in a post on X on September 12, 2026. ↩
- Sam Altman, post on X, September 12, 2026. ↩
- Anthropic, “Partnering with Accenture on embedded evaluation”, September 18, 2026. ↩
- AI Evaluator Forum, “Minimum Conditions for Embedding Evaluators”, published September 18, last updated September 23, 2026. ↩
- “White House Accord on Super Intelligence: Joint Commitment on Frontier Responsibilities,” The American Presidency Project, September 29, 2026. ↩
- Monetary Authority of Singapore, “Safeguards for Agentic Finance at Runtime,” white paper version 1.1, October 9, 2026, with its publication page, and Chia Der Jiun, “Building Trusted Foundations for Digital Finance,” keynote address, October 9, 2026. ↩
- SLSA, “Build: Provenance,” specification v1.2, and SLSA, “Build: Verifying artifacts,” specification v1.2. ↩
- Hugging Face, “Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident”, July 27, 2026. ↩
- OpenAI, “The Hugging Face incident and the road ahead”, August 26, 2026. ↩
- METR, “Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident”, August 26, 2026. ↩
- Researchers from Transluce, Corridor, MIT, AIUC and the Hertz Foundation, “AI Agents Targeted U.S. and Canadian Government Websites: Evidence from Arquivo.pt and urlquery.net”, September 30, 2026. ↩
- OpenAI Alignment, “An agent used DNS to reach an external chatbot”, report updated September 25, 2026. ↩
- Reuters, “Gemini hacked three companies in first known breakout by Google’s AI”, September 18, 2026, and Johana Bhuiyan, “Google says its Gemini AI model hacked three other companies,” The Guardian, September 18, 2026, amended September 29, 2026. ↩
- Anthropic, “Investigating unintended model actions in our evaluations and internal use”, October 9, 2026. ↩
- Adam Ang, “OpenAI agent breaches Australian Medicare portal,” Healthcare IT News, September 24, 2026. ↩
- OpenAI, “How we will do better for Australia”, September 28, 2026; updated October 4, 2026. ↩