Skip to content

EAP / EAP-IG circuit discovery

Edge Attribution Patching (EAP) or EAP with Integrated Gradients (EAP-IG) scores the nodes (attention heads and MLPs) of the model's computational graph. Simplified: the original methods score edges; a node score is the sum of its outgoing-edge scores, and this function returns the top-k nodes.

from safetune.interpret import eap_safety_circuit, EAPSafetyCircuitConfig

# The first argument is an HF model id/path (a private copy is loaded with
# config.dtype on config.device) or an already-loaded model, used in place on its
# own device and dtype: eap_safety_circuit(model, ..., tokenizer=tok). Remove your
# own hooks from a loaded model first; EAP adds and removes its own.
circuit = eap_safety_circuit(
    "meta-llama/Llama-3.2-3B-Instruct",  # HF model id, or a loaded model
    harmful_prompts=harmful,
    harmless_prompts=harmless,
    config=EAPSafetyCircuitConfig(
        method="eap-ig",           # "eap" or "eap-ig"
        granularity="head",        # "head" or "block"
        top_k_edges=100,
    ),
)

# eap_safety_circuit returns the same CircuitInfo shape as safety_circuit_info():
print(len(circuit.safety_units.unit_ids), "safety-relevant nodes (heads / MLPs) found")
print(circuit.safety_units.unit_ids[:5])

EAPSafetyCircuitConfig

Field Type Default Description
method str "eap-ig" "eap" or "eap-ig"
granularity str "head" "head" or "block"
intervention str "patching" "patching", "zero", or "mean"
top_k_edges int 100 Number of top NODES to keep (the name is historical)
ig_steps int 5 Integration steps for EAP-IG (the official default is 30); EAP-IG integrates over the input-embedding line, "patching" only
batch_size int 8 Batch size
max_seq_len int 64 Max prompt length in tokens; a longer prompt keeps its last max_seq_len tokens and a warning is logged

When to use

This discovers the components (heads and MLPs) responsible for refusal, not just individual neurons. Use when you need to understand the full computation path that drives the model's safety behaviour.

Citations

@article{eap2023,
  title  = {Attribution Patching Outperforms Automated Circuit Discovery},
  author = {Syed, Aaquib and Rager, Can and Conmy, Arthur},
  year   = {2023},
  note   = {NeurIPS 2023 ATTRIB Workshop, arXiv:2310.10348},
}

@article{eapig2024,
  title  = {Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms},
  author = {Hanna, Michael and Pezzelle, Sandro and Belinkov, Yonatan},
  year   = {2024},
  note   = {COLM 2024, arXiv:2403.17806},
}