halley

witness

A recording cache for model APIs that cuts inference spend and resumes crashed runs

Area
Agent infrastructure
Language
Rust
License
MIT
Latest
v0.3.0
Plate
No. 08 of 14
Fig. 08witness
Move your pointer over it

What came back

  1. 0.25 ms

    median added per call to record everything, about 0.01% of a model call.source

  2. 53k/sec

    recorded calls per second on one machine.source

  3. 541µs

    to serve a repeated call from the record, instead of 803 ms to pay for it again.source

witness is a single-binary, API-compatible proxy that sits between your agents and a model API, and agents change only the base URL. Repeat deterministic calls are served from the record instead of re-paying inference, and a crashed run can be replayed with zero model calls. The same record is a tamper-evident, signed, Merkle-committed audit trail, so it is already running as evidence before anyone asks a question.

How it works

  1. 01

    Agents point their base URL at witness serve, which forwards to the model API and stores every request and response by BLAKE3 content hash.

  2. 02

    Each call is appended to a hash-chained journal, so any edit, deletion or reorder breaks the chain and witness verify finds it.

  3. 03

    Identical reusable requests are answered from the record, and witness replay serves a whole recorded run with no upstream calls.

What it does

  • Repeat deterministic calls are served from the record instead of re-paying inference
  • Whole runs replay from the record with zero upstream calls, so crashed runs resume
  • Hash-chained journal with Merkle commitments, inclusion proofs and anchoring to a public transparency log
  • Ed25519 signed identity and narrowing-only delegations enforced at the network boundary
  • Records MCP tool calls and shell commands by what they changed, in the same chain; fleet peer cache and Prometheus and OpenTelemetry export