dlab-spp/vanilla-3b-base
Text Generation • 3B • Updated • 226
Models - 3B
3B SPP models: bases publish their training trajectory as step-* revisions, plus persona-binding instruct counterparts.
Note **Vanilla** — standard next-token pretraining on the full corpus. The control recipe. Base model, pretrained on 500B tokens.
dlab-spp/filtered-3b-base
Text Generation • 3B • Updated • 185Note **Filtered** — harmful documents (safety score ≥ 3) are loss-masked but retained, so batches match Vanilla exactly. Base model, pretrained on 500B tokens.
dlab-spp/t0-3b-base
Text Generation • 3B • Updated • 243Note **SPP(T0)** — constitution-grounded reflections distributed uniformly from the *first batch* onward. The token-zero recipe. Base model, pretrained on 500B tokens.
dlab-spp/t0-mt-3b-base
Text Generation • 3B • Updated • 72Note **SPP(T0,MT)** — SPP(T0)'s pre-midtraining checkpoint plus the same reflection-focused midtraining stage as SPP(MT). Both interventions combined. Base model, pretrained on 500B tokens.
dlab-spp/mt-3b-base
Text Generation • 3B • Updated • 80Note **SPP(MT)** — Vanilla's pre-midtraining checkpoint, midtrained on the same reflections introduced only at the end. Token-matched to SPP(T0), isolating *when* the data is seen. Base model, pretrained on 500B tokens.
dlab-spp/vanilla-3b-instruct
Text Generation • 3B • Updated • 132Note **Vanilla** — standard next-token pretraining on the full corpus. The control recipe. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.
dlab-spp/filtered-3b-instruct
Text Generation • 3B • Updated • 108Note **Filtered** — harmful documents (safety score ≥ 3) are loss-masked but retained, so batches match Vanilla exactly. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.
dlab-spp/t0-3b-instruct
Text Generation • 3B • Updated • 114Note **SPP(T0)** — constitution-grounded reflections distributed uniformly from the *first batch* onward. The token-zero recipe. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.
dlab-spp/t0-mt-3b-instruct
Text Generation • 3B • Updated • 129Note **SPP(T0,MT)** — SPP(T0)'s pre-midtraining checkpoint plus the same reflection-focused midtraining stage as SPP(MT). Both interventions combined. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.
dlab-spp/mt-3b-instruct
Text Generation • 3B • Updated • 125Note **SPP(MT)** — Vanilla's pre-midtraining checkpoint, midtrained on the same reflections introduced only at the end. Token-matched to SPP(T0), isolating *when* the data is seen. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.