Necessity & sufficiency
Ablate or activation-patch a discovered circuit and measure the causal effect on a real behavior — not just correlation.
contracts.md →circuit-level regression testing
Test what your model is doing, not just what it outputs.
Warden lets you write declarative assertions about a model's internal mechanism and run them like tests: is this behavior mechanistically necessary and sufficient in this circuit — and hasn't the mechanism silently drifted?
- 96tests passing
- Rust+ Python core
- MITlicensed
the problem
Evals see the answer. They don't see the reasoning.
Behavioral evals
Accuracy, BLEU, LLM-as-judge — every one of them only sees input → output. A model can pass all of them while the mechanism underneath silently shifts to something brittle: a shortcut, a spurious feature, a circuit producing the right answer for the wrong reason.
Nothing in a standard eval suite tests whether the internal computation is still doing what you think it's doing — so this kind of drift ships unnoticed.
Warden
Uses sparse autoencoders to identify the features involved in a behavior, and causal patching — ablation and activation-patching — to measure whether that circuit actually drives the behavior, rather than merely correlating with it.
A Rust core (PyO3) handles the numerics; a Python layer handles
orchestration, the DSL, and the CLI — so contracts read like
tests and slot into a normal pytest run or CI
pipeline.
what's here
Implemented and verified against real GPT-2 small
Not mocked, not synthetic where it mattered.
Drift detection
Compare a checkpoint's circuit against a baseline's. Demoed catching a real, self-inflicted regression from a fine-tune.
drift.md →Production monitoring
Plugin SDK, warden sample, and real OpenTelemetry/Prometheus metrics — a bad hot-swapped deploy shows up on the next scrape.
Self-serve SAEs
warden train-sae for layers with no public dictionary — a hand-derived forward/backward pass in Rust, verified by numerical gradient checking.
Plugin SDK
Swap in your own model adapter or SAE loader via a registry or a real Python entry point — no fork required.
production.md →HTML reports & Action
A self-contained report for human review, and a reusable composite GitHub Action to block merges on regressions.
drift.md →
Two deliberate simplifications: circuits are flat top-k SAE
feature lists (not full attribution graphs), and drift is a
Jaccard-distance proxy (not graph-edit-distance). warden
sample re-checks fixed contracts on an interval rather than
mining circuits from unlabeled live traffic. Each doc above explains
why.
install
Two ways to get running
Published on PyPI as warden-interp — abi3 wheels for Linux, macOS and Windows, so no Rust toolchain needed to install.
Note: --release is important — debug builds are ~10-20x slower for training.
quickstart
A contract is a YAML file. Running it is one command.
Contracts run real model forward passes, so both the pytest
plugin and @warden.contract-decorated functions are
skipped by default — pytest
--warden opts in.
end-to-end
From a passing contract to a caught regression in CI
A real walkthrough — every number below is copied from an actual run against GPT-2 small during development of this project.
-
1
Declare the circuit you care about
A contract names a model, a layer, an SAE, an eval set, and the assertions that must hold. This one covers GPT-2's indirect-object-identification (IOI) circuit at layer 9.
-
2
Run it against the baseline model
Warden discovers the circuit via the SAE, ablates and patches it, and scores necessity/sufficiency against the real behavior.
-
3
Somewhere downstream, the model gets fine-tuned
A 4-epoch fine-tune on wrong-completion sentences quietly teaches the model to complete IOI sentences with the wrong name. Loss drops normally — nothing here looks alarming.
-
4
The drift contract catches it
A
driftassertion compares the fine-tuned checkpoint's circuit against the baseline's top-20 features by Jaccard distance. The two circuits have genuinely diverged — 82%. -
5
Wire it into CI so it blocks the merge
The composite GitHub Action runs every matched contract, fails the build on a regression, and uploads an HTML report as a build artifact.
-
6
Keep watching it in production
warden samplere-runs the same contract against whatever checkpoint lives at the deployed path, on an interval, exporting real Prometheus-scrapeable metrics.
documentation
One doc per component
All the "why" and the real numbers behind each claim live in docs/.
The contract DSL
*.warden.yaml declares a model, a layer, an SAE, an eval set, and assertions. Contracts also run as decorated Python functions, and as a pytest plugin — skipped by default since they run real forward passes.
Drift assertions
Compare a checkpoint's discovered circuit against a stored baseline's, generate an HTML report for human review, and gate merges on it via the GitHub Action.
Read drift.md →Plugin SDK & production monitoring
Register a custom model adapter or SAE loader without forking, and export contract scores as real Prometheus/OTel metrics on a schedule.
Read production.md →Self-serve SAE training
warden train-sae for layers with no public dictionary. Forward/backward pass hand-derived in Rust (no autodiff), verified by finite-difference gradient checking.
How it's built
A Rust core (PyO3 + ndarray/rayon) does the heavy numerics — patching, SAE forward/backward, gradient-checked training. A Python layer handles orchestration, the DSL, and the CLI.
Read architecture.md →results
Real numbers from real runs
Not illustrative, not rounded to look good — copied from docs/results.md.
0.390
necessity, baseline IOI circuit
0.824
circuit drift caught after fine-tune regression
96
tests passing (28 Rust + 68 Python)
0.26s
offline Python suite, 47 tests, no network
| l1_coefficient | active features | reconstruction rel. error |
|---|---|---|
| 1e-3 | 40.4% | 0.31 |
| 1e-2 (default) | 6.0% | 0.57 |
| 3e-2 | 2.6% | 0.67 |
Honest note: on the harder bar — does the resulting circuit actually support a real contract — neither self-trained run matched the public SAE's necessity/sufficiency. See sae-training.md for the diagnosis.