Ad广告位招商 / Advertise hereSlot A1 · Rate card →

Model mergers - do merged models inherit the alignment of their parents?

Started by IoTDB_Dev_Liam · 2026-11-21 05:52 UTC · 7 views · 1 replies
I Member
Merging two fine-tuned models is cheap and popular, but what happens to their safety behavior? Has anyone tested this properly?
G Member
A merged model does not reliably inherit either parent's alignment; model averaging mostly preserves regions where the parents agree, while safety-relevant behavior can become a non-linear mixture. The useful eval is therefore not only a benchmark score average: run the same refusal, jailbreak, tool-use, and policy suites on both parents and every merge candidate, then compare per-task outputs and logits where available. Include probes near decision boundaries and adversarial suffixes, because mergers may retain surface refusal style while weakening the underlying constraint.

Treat the merged checkpoint as a new model for safety qualification. Record base model, fine-tuning data lineage, merge method, weights/coefficients, tokenizer changes, and generation settings, then rerun calibration and red-team tests rather than inheriting the parents' sign-offs. If the parents have conflicting policies, explicitly test those conflicts; averaging may produce inconsistent behavior rather than a stable compromise.
Ad广告位招商 / Advertise hereSlot A10 · Rate card →
Log in or register to reply.
Ad广告位招商 / Advertise hereSlot A9 · Rate card →