Layer surgery reveals universal mid-stack failure zone

This is one of the more useful practitioner-led ablations in local LLM land: it turns “frankenmerge” from lore into a testable playbook with clear failure modes.

• –The strongest claim is architectural consistency: a mid-depth “routing/wiring” region appears load-bearing and non-duplicable across multiple model families.
• –Gains look conditional, not universal: if baseline performance is already saturated, duplication mostly shifts coding style rather than benchmark outcomes.
• –The MoE result is especially actionable for builders: optimal duplication depth appears earlier than dense models, likely due to routing dynamics.
• –The cross-model transplant failures reinforce a key systems lesson: shape compatibility is not representation compatibility.
• –Main caveat is scientific rigor: single-run style evaluations and custom test suites are great for discovery, but need repeated trials and broader benchmarks to lock in the claim.