rmonsurate/Vernon-2

Vernon 2 — DRAFT, research checkpoint, not a release

DRAFT, research checkpoint, not a release. These weights are a stopped mid-heal checkpoint kept so a future recipe can resume from them. They are not a released model, they are not a recommended model, and nothing here should be redistributed or treated as final.

What this is

Vernon 2 is Qwen/Qwen3.8-Flash-Next pruned from 512 to 144 experts per layer with REAP, then partially healed. The surviving checkpoint published here is step 300 of a planned 490-step heal; the run was stopped at step 312 by decision.

Expert selection is entirely our own. The REAP saliency scores were produced by our own calibration pass over our own synthetic agentic corpus — 1,504 packed sequences drawn from the v3 heal corpus sample (653 agentic-office, 686 agentic-coding, 161 agentic-mixed), the same corpus the heal trains on. No third-party saliency, manifest or checkpoint was used anywhere in this build.

• Pruned bf16 size: ~171 GiB
• Parameters: 91.3 B
• Experts: 144 / 512 per layer
• Base: Qwen/Qwen3.8-Flash-Next

Heal recipe

Healed 312 of a planned 490 steps with the Victoria recipe:

• Q4_K-grid fake-quant (quantization-aware distillation)
• Belief-state / next-latent distillation plus token-level KL from the bf16 teacher
• AdamW
• 233 s/step on 4+2 B300s, sequence length 8192

One stated deviation from the shipped recipe: a device-placement fix in the next-latent (belief-state) probe, so that hidden states are moved to the p-psi module device under a multi-GPU sharded student. That is a genuine bug fix, not a recipe change; the diff is in docs/PROVENANCE.md. No token-KL anneal was used in this run.

Evaluation — HumanEval

Card settings: 16K max tokens, temperature 1.0, top_p 0.95, top_k 20.

step	sampled pass@1	greedy
0 (pruned, no heal)	35.98	—
100	42.44 (mean of 5, sd 0.33)	29.88
200	46.95 (single run)	37.80
300	50.61 (single run)	38.41

Reference points measured the same way:

model	HumanEval
teacher (Qwen3.8-Flash-Next, bf16)	96.34
Victoria (288 experts, healed)	93.2

Truncation is the dominant failure mode. At step 300, about 87 of 164 sampled completions hit the 16K token cap without finishing.

Why it was stopped

The trajectory is linear at roughly 4 points per 100 steps and extrapolates to about 58 at the planned step 490 — far short of the 288-expert result. The conclusion is that the 288 → 144 expert cut is past the cliff for this recipe. The run was stopped at step 312 and the cards released.

Contents

path	what
safetensors/	bf16 servable export of step 300 — the exact weights that produced the HumanEval numbers above. Includes config, tokenizer and the MTP head.
training-checkpoint-step300/	raw training checkpoint at step 300, carrying optimizer and psi (belief-state) state needed to resume the heal.
docs/	plan, heal eval log, stop note, and prune/saliency provenance.

Caveat on the MTP head

The draft (MTP) head is present but frozen and un-healed, and which 144 of its own experts survive is a placeholder — no calibration pass has ever scored the head. A low draft-acceptance rate is a finding about the head, not about the trunk.

License

Inherited from the base model: qwen-community-1.0. See the Qwen3.8-Flash-Next license.