AgentReplay

AgentReplay freezes a production agent run as plain JSON: tool calls, arguments, state before and after, approvals and side effects. You replay it, diff it against a fix, and gate every future model, prompt or tool change on the same failure. Gates are deterministic assertions, so a regression test never depends on a judge model.

How it works

1. 01

   Wrap each tool with the harness so every call is recorded with its state before and after, and the side-effect diff.

2. 02

   Save the run as a trace, add assertions such as never sending an email without approval, and replay it against recorded tool responses.

3. 03

   Run agentreplay gate in CI with --json; a changed model, prompt or tool that brings the failure back fails the build.

What it does

• Records tool calls, arguments, responses, pre and post state snapshots, approvals and side-effect diffs as agentreplay.trace.v1 JSON
• Deterministic gates such as tool_not_called, requires_approval, tool_order, arg_matches and side_effect_count
• Diffs a bad run against a fixed run: changed tool order, arguments and side effects
• Node reference SDK plus Python and Go SDKs that emit matching trace hashes, and raw JSON for any other language
• Redacts secrets and PII, and scaffolds a CI workflow with agentreplay init