Total Swaps
2450
experiments
Perfect (T5)
70%
target concept found
Concept Correct
70%
T3+ success
Suppression
100%
source removed
Logit Flip @0-1
98%
flip at pos 0 or 1
Loading matrix...
Attribution graphs (Ameisen et al., 2025) are causal maps that trace how information flows from input tokens through internal features to output logits inside a language model. They are built on top of Cross-Layer Transcoders (CLT) -- sparse dictionaries with ~2.5M features that replace MLP layers and decompose the residual stream into interpretable units. A replacement model freezes attention patterns and linearizes the residual stream through CLT features, allowing Anthropic's Circuit Tracer to compute per-feature causal influence on any target logit.
These graphs typically contain hundreds to thousands of feature nodes. Manual interpretation -- inspecting activation patterns across corpus examples to assign semantic labels -- takes ~2 hours per prompt. This pipeline automates first-pass analysis using Probe Prompting: an instructed LLM generates semantic probes that measure each feature's behavior under controlled semantic variation, classifies features into functional roles using transparent rules, groups them into labeled supernodes, and tests their causal relevance via feature swapping.
This demo visualizes pairwise steering experiments across 41,784 runs spanning the four evaluation domains. Each matrix cell represents an attempt to redirect the model from a source entity (row) to a target entity (column), using labeled feature swaps. Cells are colored on a tiered scale from T5 (target answer produced) to T1 (source persists).
The central question: do probe-prompted labels have entity-specific causal leverage -- i.e. do they steer the model toward the intended target -- or do they just generically disrupt output? Structurally matched random-feature controls (same count, same layer distribution) serve as the null hypothesis.
Attribution Graph
Neuronpedia API generates a causal graph of CLT features for a seed prompt via a local replacement model (frozen attention, linearized residual stream). Features are selected by cumulative influence threshold (tau = 0.95).
Probe Prompting
An instructed LLM generates semantic probes that reuse the seed prompt's token structure. Each feature is measured across all probes, producing a Cross-Prompt Activation Signature (CPAS) that captures functional vs semantic behavior.
Classification & Supernodes
A transparent decision tree classifies features into four types (Semantic Dictionary, Semantic Concept, Say-X, Relationship) using explicit thresholds -- no learned parameters. Features sharing the same class and name form supernodes.
Feature Swapping
Source supernodes are suppressed (ablate) while target supernodes are amplified via additive delta injection through CLT decoder vectors. Attention is NOT frozen -- the model can route around interventions, making positive results stronger evidence.
Strict priority ordering -- the first matching rule wins. All thresholds are explicit and auditable.
1. peak_consistency >= 0.80 AND n_distinct_peaks <= 1 -> Semantic Dictionary
2. func_vs_sem >= 50 AND conf_F >= 0.90 AND layer >= 7 -> Say-X (output promotion)
3. sparsity_median < 0.45 -> Relationship
4. layer <= 3 OR conf_S >= 0.50 OR func_vs_sem < 50 -> Semantic Concept
5. ELSE -> Review (flagged)
Four evaluation domains spanning two-hop factual recall in different relational structures and answer geometries. Each domain has a fixed seed-prompt template with three semantic fields (input, intermediate, answer):
| Domain | Seed Prompt | Entities | Pairs |
|---|---|---|---|
| USA States | "The capital of the state containing {city} is" | 50 | 2,450 |
| Books | "The book featuring {character} was written by" | 10 | 90 |
| Products | "The company that makes {product} was founded by" | 12 | 132 |
| Paintings | "The first name of the painter of {painting} is" | 10 | 90 |
Each pair runs under four conditions: labeled 3-field baseline at M=20, matched-random controls (3 replicates per pair, same feature count and per-layer histogram, sampled outside concept-aligned supernodes), field-additivity variants (7 subsets of the three concept fields), and adaptive M-search (rescue passes for missed pairs). Total: 41,784 swap runs.
Ours: per-pair best field-additivity variant with adaptive M-search on labeled features. Rand.: matched-random control under the same per-pair best-of (3 replicates x {default, M-tuned}) construction -- same feature count and per-layer histogram, sampled outside concept-aligned supernodes. Top-K: influence-matched top-K-by-graph-influence baseline with adaptive M-search. Books and Paintings use the 10-entity / 90-pair demo intersection so the Top-K coverage is symmetric.
| Domain | N | Ours (FA+M-srch) | Rand. +M-srch | Top-K | |||
|---|---|---|---|---|---|---|---|
| Hit% | vsMax | Hit% | vsMax | Hit% | vsMax | ||
| USA States | 2,450 | 72.8% | +6.15 | 0.7% | -1.23 | 4.2% | -1.92 |
| Books | 90 | 77.8% | +10.38 | 0.0% | +0.19 | 4.4% | +1.57 |
| Products | 132 | 41.7% | +5.42 | 7.6% | +1.17 | 1.1% | +0.63 |
| Paintings | 90 | 18.9% | +3.45 | 1.1% | +1.27 | 7.1% | +0.16 |
Each domain's concept fields map to semantic roles: input (mentioned in prompt), intermediate (bridging concept), answer (what the model produces). Numbers below are Best-subset Hit% vs the all-3-field labeled baseline, both at M=20. In 3 of 4 domains, intermediate+answer carry the strongest signal; including input-field features dilutes the redirection signal.
| Domain | Best subset | Role | Best Hit% | Full 3f | Delta |
|---|---|---|---|---|---|
| USA States | state + capital | mid+ans | 38.8% | 24.7% | +14.1pp |
| Books | book + author | mid+ans | 37.1% | 3.8% | +33.3pp |
| Products | company + founder | mid+ans | 24.2% | 15.2% | +9.0pp |
| Paintings | first_name (1-fld) | ans | 6.7% | 3.3% | +3.4pp |
Each swap is classified by what happens to target and source logits at position 0 relative to their unsteered baselines. Four regimes matter for interpretation:
| Regime | Target | Source | Flip? | Intuition |
|---|---|---|---|---|
| A | UP | DOWN | yes | Clean redirection |
| C | DOWN | DOWN | yes | Differential disruption |
| D | DOWN | DOWN | no | Generic disruption |
| E | FLAT | DOWN | yes | Pure suppression |
Regime A (clean redirection) and Regime D (generic disruption) prevalence under three conditions: best field-additivity variant, full labeled (M=20, all three fields), and the symmetric matched-random control under per-pair best-of (3 replicates x adaptive M):
| Domain | Regime A % | Regime D % | ||||
|---|---|---|---|---|---|---|
| Best | Labeled | Rand+M | Best | Labeled | Rand+M | |
| USA | 34.9 | 8.9 | 15.3 | 9.1 | 19.4 | 38.7 |
| Books | 62.1 | 38.8 | 26.7 | 3.3 | 3.3 | 34.4 |
| Products | 62.1 | 56.8 | 28.8 | 2.3 | 2.3 | 18.9 |
| Paintings | 47.8 | 17.8 | 16.7 | 2.2 | 6.7 | 36.7 |
Labeled--random vsMax gap (logit margin over the strongest competing dataset answer) at the default operating point (all-3-fields, M=20, per-replicate matched-random control). Specificity tracks associative complexity and answer-field coarseness, not graph size:
| Domain | vsMax gap | Strength | Correlates |
|---|---|---|---|
| Books | +6.13 | Strong | small answer space, distinctive author signatures |
| USA States | +5.17 | Strong | large N, high scaffold compatibility |
| Products | +3.23 | Moderate | moderate scaffold, mid relational complexity |
| Paintings | +1.58 | Weak | multi-step inference, coarse first-name answer |
Under the symmetric matched-random + M-search harness the same ordering holds (Books +6.5, USA +4.1, Products +2.3, Paintings +0.2). The cross-domain ordering matches the operational-usefulness verdict, which is why vsMax serves as a primary metric rather than a diagnostic.
Level 1 -- Operational Labels
The pipeline produces behaviorally grounded, interpretable supernode groupings. Feature categories are behaviorally distinct and useful for navigating circuits.
Level 2 -- Causal Effects
Three lines of evidence across 4/4 evaluation domains: (1) labeled--random Hit% gap >= 11pp and labeled--random vsMax gap >= +0.2 in every domain, even after symmetric M-search on the random side; (2) field-level decomposition shows the signal concentrates in intermediate+answer features (9-33pp Hit% gain over all-3-fields); (3) the best variant concentrates in regime A (clean redirection, 35-62% per domain) while matched-random concentrates in regime D (generic disruption, 35-39% in 3/4 domains).
Level 3 -- Mechanistic Explanation
Labels are behavioral abstractions, not ontological identifications. Thresholds have not undergone sensitivity analysis. Whether categories correspond to natural computational types remains open.
Built by Giuseppe Birardi | Orma Lab Srl and Gonçalo Paulo | EleutherAI
Thanks to CoreWeave for infrastructure