Recover API¶
Weight-space repair of a model whose safety has drifted. Import from
safetune.runner.recover. Recover trainers are training-free: construct with the
drifted model (plus base_model / aligned_model references where the method
needs them) and call apply().
from safetune.runner.recover import ReStaTrainer
trainer = ReStaTrainer(model=drifted, base_model=base, aligned_model=aligned,
alpha=0.5, dare=True, dare_seed=0)
repaired = trainer.apply()
Available trainers¶
AAQTrainer, AntidoteTrainer, AntidoteV2Trainer, CThetaTrainer,
GradSelectiveRecoverTrainer, LSSFTrainer, LoXTrainer, MSCPTrainer,
NLSRTrainer, OneShotSafetyPatchTrainer, PKETrainer, PrePostMergeTrainer,
QReSafeTrainer, ReStaTrainer, RepNoiseRecoverTrainer, SCRUBTrainer,
SOMFTrainer, SafeDeltaTrainer, SafeLoRATrainer, SafeMergeTrainer,
SafeReActTrainer, SafetyVectorRestoreTrainer, TaskArithmeticTrainer,
WiseFTTrainer.
See the Recover guide for the method taxonomy (whole-model / low-rank / layer / neuron / saliency / circuit-guided).
Reference¶
safetune.runner.recover.ReStaTrainer
¶
Bases: _RecoverBase
ReSta (DARE task arithmetic): safety vector restoration, with DARE on the fine-tuning delta.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
base_model
|
Module
|
base model. |
None
|
aligned_model
|
Module
|
aligned reference. |
None
|
alpha
|
float
|
task vector scale. Default 1.0. On Tiny Aya, alpha=1 breaks the model; use alpha~0.25 (sweep: 0.1/0.25 restore refusal with normal answers, >=0.5 breaks it). |
1.0
|
dare
|
bool
|
apply DARE masking. Default True (part of the method). |
True
|
dare_drop_rate
|
float
|
DARE drop probability p. Default None: 0.3, the RESTA
paper's value, or 0.9 (the old default) with
|
None
|
dare_seed
|
int
|
DARE random seed. Default 0. |
0
|
device
|
Optional[str]
|
where each safety-vector delta is computed. Default None: the
drifted model's weight device. |
None
|
DARE drops a fraction p of the entries of the fine-tuning delta
(drifted - aligned) and scales the rest by 1/(1-p), then adds the safety
vector whole (the paper's RESTA_d, Eq. 7-9). The drifted model must have been
fine-tuned from aligned. A high p scales the delta by 10x or more (p=0.9).
The paper tunes on 7B models; on small ones check the benign answers after
repair. Earlier versions applied DARE to the safety vector, so results with
the default dare=True differ from those releases.
Memory: the drifted, base and aligned models must be loaded, but the
safety vector is streamed one tensor at a time, so the extra memory is a
few fp32 copies of the largest tensor, not of the model. Keep base and
aligned on CPU and pass device="cpu" to hold only the drifted
model on the GPU.
safetune.runner.recover.TaskArithmeticTrainer
¶
Bases: _RecoverBase
Task Arithmetic: plain safety task vector addition.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
base_model
|
Module
|
base model. |
None
|
aligned_model
|
Module
|
aligned reference. |
None
|
alpha
|
float
|
task vector scale. Default 1.0. |
1.0
|
safetune.runner.recover.SafeLoRATrainer
¶
Bases: _RecoverBase
SafeLoRA: safety subspace LoRA decomposition and merge.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
aligned_state_dict
|
dict
|
state dict of the aligned model. |
None
|
base_state_dict
|
dict
|
state dict of the base model (required for a non-PEFT model). |
None
|
base_model
|
Module
|
model to take the base state dict from instead. |
None
|
aligned_model
|
Module
|
model to take the aligned state dict from instead. |
None
|
alpha
|
float
|
merge coefficient. Default 0.5. |
0.5
|
threshold
|
float
|
safety subspace threshold. Default 0.5. |
0.5
|