Paired benchmarks for enforcement behavior.

Published runs are used to inspect how a deterministic gate behaves under test. They are not a model leaderboard and they are not an intelligence ranking surface.

Each published row compares a baseline run with a gated run in the same context. Each pair evaluates the same suite twice, once ungated and once through the gate; the identical prompt set is what makes the two halves comparable. Read the results as evidence about behavior and auditability. Only paired benchmark dispatches appear here. The routine audit published on every merge has no ungated half, so it produces no row in this table; those are published separately in the run archive.

Snapshot as at 1 November 2026.

Run time (UTC)	Model	SIR	Pack	Coverage	Leaks (baseline / gated)	Outcome	Evidence
2026-10-15 10:32:04Z	grok-4.3xai	2.4.0	eu_ai_act_compliance_pressurebaseline UNGATED_BASELINE_RECORD / gated LIVE_GATING_CHECK	74/100	n/a / 26 of 150	Leaks match published uncovered rows exactlyBlocked benign: 0	CI Base Gate
2026-10-15 09:31:50Z	grok-4.3xai	2.4.0	data_exfiltration_pressurebaseline UNGATED_BASELINE_RECORD / gated LIVE_GATING_CHECK	27/27	n/a / 0 of 50	Complete coverage; no leaked rowsBlocked benign: 0	CI Base Gate
2026-09-23 12:14:40Z	grok-4.3xai	2.3.4	data_exfiltration_pressureLIVE_GATING_CHECK	27/27	27 / 0 of 50	Complete coverage; no leaked rowsBlocked benign: 0	CI Base Gate
2026-09-21 13:51:40Z	gpt-5.4-miniopenai	2.3.4	eu_ai_act_compliance_pressureLIVE_GATING_CHECK	74/100	100 / 26 of 150	Leaks match published uncovered rows exactlyBlocked benign: 0	CI Base Gate
2026-09-21 12:03:18Z	gpt-5.4-miniopenai	2.3.3	eu_ai_act_compliance_pressureLIVE_GATING_CHECK	74/100	100 / 26 of 150	Leaks match published uncovered rows exactlyBlocked benign: 0	CI Base Gate
2026-09-20 09:24:56Z	gpt-5.4-miniopenai	2.3.0	eu_ai_act_compliance_pressureLIVE_GATING_CHECK	74/100	100 / 26 of 150	Leaks match published uncovered rows exactlyBlocked benign: 0	CI Base Gate

The repository coverage record names uncovered rows as the pre-run expectation. The Outcome column reports only whether the published runtime evidence matches that row set.

Every certificate records suite_hash, policy_hash, and commit_sha, so the coverage expectation for a past run is reproducible at that run's commit rather than asserted after the fact.

View the full run archive

Reading the results

Core terms

Leak rate

The share of tested prompts that reached the provider when they should have been blocked in that run context. Leaks are measured against a published, enumerable rule set whose uncovered rows are named in advance in the gate coverage report.

Blocked benign

The count of prompts treated as harmless in the run summary but blocked by the gate.

Suite hash

A hash binding for the evaluated suite so the reviewed material can be tied back to a specific test input set.

Audit links

Run, audit, and artefact links let reviewers inspect the published evidence directly rather than relying on a summary claim.

Supporting evidence

Measured gate latency report

Evidence-binding correction

Independent verification command for gated run 20261015-092939-820950-gh38712792514-308a8a0f91eb (proof class: LIVE_GATING_CHECK):

git clone https://github.com/SDL-HQ/sir-firewall.git \
  || git -C sir-firewall pull
cd sir-firewall
python3 -m pip install cryptography
python3 tools/verify_evidence.py docs/runs/20261015-092939-820950-gh38712792514-308a8a0f91eb

docs/runs/20261015-092939-820950-gh38712792514-308a8a0f91eb

  OK      signing trust      signing_key_id='sdl-2026-10-06' resolved through key_registry.v1.json, with its status and revocation rules applied
  OK      certificate        signature, payload hash and ledger binding all verify
  OK      signed counters    the signed counters were recomputed from the rows the certificate binds
  OK      archive custody    every file named by the signed manifest is present and unchanged
  OK      evidence contract  the certificate satisfies its applicable evidence contract

VERDICT: ESTABLISHED

This is one archive, checked by our own tool with our own key: a signature that verifies establishes that we signed that payload, not that the payload is true. Where a property cannot be checked at all the verdict is NOT ESTABLISHED rather than a pass, and several published archives come back that way. Limits records what a passing verification does not establish, and the claims register lists every claim with the evidence for it.

Don't take our word for it

Check the evidence yourself

A separate repository of real published archives, each chosen because our own verifiers give it a different answer. Verifiable offline from a clean machine, with a recipe for altering a ledger row so you can watch the chain refuse to re-derive.

Open SDL-HQ/sir-evidence