WiSE-FT — Weight-space interpolation¶
Linear interpolation between the drifted model and the aligned model: \(\theta_{\text{recovered}} = (1 - \text{alpha}) \cdot \theta_{\text{drifted}} + \text{alpha} \cdot \theta_{\text{aligned}}\). The standard robustness baseline from the WiSE-FT paper.
Ref: Wortsman et al., "Robust Fine-Tuning of Zero-Shot Models," CVPR 2022.
Signature¶
WiseFTTrainer(
model: nn.Module,
*,
aligned_model: nn.Module,
alpha: float = 0.5,
strict: bool = False,
merge_adapter: bool = False,
)
Parameters¶
| Param | Type | Default | Description |
|---|---|---|---|
model |
nn.Module |
required | Post-fine-tune (drifted) model — modified in-place |
aligned_model |
nn.Module |
required | Safety-aligned reference model |
alpha |
float |
0.5 |
Interpolation coefficient toward aligned; 0.0 = no recovery, 1.0 = fully replaces with aligned |
| strict | bool | False | True raises (before any edit) when a model parameter has no same-named entry in the reference. The default leaves such parameters unchanged and logs how many |
| merge_adapter | bool | False | For a PEFT (LoRA) model against a bare reference: interpolate a merged deep copy and return it (needs a second set of weights in memory; model is left as it is) |
The model and the reference must share parameter names. A PEFT-wrapped model has a base_model.model. prefix and LoRA factors, so against a bare reference no name matches and apply() raises a ValueError (it used to return the model unchanged). Either merge the adapter yourself with merge_and_unload() or pass merge_adapter=True. With the reference being the base weights the adapter was trained on, the result is base + (1 - alpha) * lora_update: alpha scales the LoRA update by 1 - alpha. The other RECOVER methods that match weights by name take the same strict= and raise the same error.
Full example¶
from safetune.runner import recover
trainer = recover.WiseFTTrainer(model, aligned_model=aligned_model, alpha=0.5)
patched = trainer.apply()
ckpt_path = trainer.save_checkpoint(patched, tokenizer, "wiseft_ckpt")
metrics = trainer.eval("wiseft_run", ckpt_path)
trainer.save_results(metrics, variant="alpha=0.5")
When to use¶
- Best for: safety-utility trade-off;
alpha=0.5gives an equal blend of the drifted and aligned weights. - Sweep
alpha: try[0.1, 0.3, 0.5, 0.7, 0.9]and pick the knee of the safety/capability trade-off curve. - Compare to task arithmetic: WiSE-FT interpolates \(\theta_{\text{drifted}}\) toward \(\theta_{\text{aligned}}\); task arithmetic adds the delta \(\theta_{\text{aligned}} - \theta_{\text{base}}\). The two only coincide under specific weight relationships (e.g. when \(\theta_{\text{base}}\) equals \(\theta_{\text{drifted}}\)), not in general.