peterstran/olmo3-graft-organisms

Graft organisms: value/quirk LoRA adapters trained across the training pipeline

A collection of LoRA adapters trained on OLMo-3 and Qwen3-14B checkpoints for research on grafting — training a value or behavior into an earlier checkpoint of a model's own training pipeline (a "substrate"), then applying that adapter unchanged to the finished chat model. The question these artifacts were built to answer: does a value/quirk installed on one checkpoint transfer to the chat model, and what predicts whether it will?

Two model families in this repo. The auditbench-7b/, msm-32b/, and wip-32b-predictor/ subtrees below are the OLMo-3 adapters. The Qwen3-14B adapters live under qwen3-14b/ and have their own README there — see the summary at the end of this card.

Two interventions, both trained as plain-text document finetuning (documents present a fact/value as established knowledge; no chat-turn masking):

• AuditBench SDF quirks (AuditBench): behavioral "model organisms" — animal welfare (unprompted welfare advocacy) and contextual optimism (optimism conditioned on the user claiming to be an Anthropic safety researcher).
• MSM value (model-spec-midtraining, after Li et al.): a philosophy-safety / equanimity value (ethical caution, honesty, equanimity about shutdown), intended to raise agentic-misalignment safety.

The key idea: substrate → target

Each adapter is trained on a source checkpoint and is meant to be applied ("grafted") onto the target chat model of the same size. OLMo-3 uniquely publishes checkpoints from every stage of one lineage, so we can train the same intervention on a ladder of substrates — from the true end-of-pretraining base through mid-training and SFT to the chat model itself — and measure how well each transfers.

Headline finding (see the linked report): transfer tracks how close the source is to the target. On the 7B AuditBench ladder, the true-pretrain stage1 source installs essentially nothing on the chat model, while sft matches training directly on the chat model; on 32B, the midtrain MSM graft matches the native result and even preserves capability better. A native adapter (trained directly on the chat model) is included for each quirk/value as the comparator.

Layout

auditbench-7b/            AuditBench SDF quirks, OLMo-3-7B  (r=64, lr 2e-5, 1 epoch, seq 2048)
  animal-welfare/         stage1 | base | sft | instruct  (+ stage1-lr1e-4, stage1-2epoch dose variants)
  contextual-optimism/    stage1 | base | sft | instruct
msm-32b/                  MSM philosophy-safety value, OLMo-3-32B  (r=64, lr 1e-4, 1 epoch, seq 4096)
  philosophy/             base | midtrain | instruct
wip-32b-predictor/        UNPUBLISHED work-in-progress: 32B checkpoint-ladder + method experiments
  msm/                    philosophy MSM on ingredient-1 steps 1k/8k/17k/23k
  animal-welfare/         AuditBench aw on i1 step 1k/23k, instruct, + GIFT- and STM-weighted variants

Each subfolder is a standalone PEFT adapter with its adapter_config.json, adapter_model.safetensors, the full Axolotl train_config.yaml, and an artifact.json provenance record (weight SHA-256 + git commit). Every subfolder's own README.md gives its exact base checkpoint and revision.

Loading (revision matters!)

Each adapter's base checkpoint is pinned to a specific revision. Loading main for a non-main substrate (stage1, midtrain, ingredient-1 steps) gives the wrong checkpoint and the adapter will not behave as reported.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Example: the stage1-substrate animal-welfare adapter, on its own base
base = AutoModelForCausalLM.from_pretrained(
    "allenai/Olmo-3-1025-7B", revision="stage1-step1413814", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "peterstran/olmo3-graft-organisms",
                                  subfolder="auditbench-7b/animal-welfare/stage1")

# Grafting: the SAME adapter applied to the finished chat model
chat = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-7B-Instruct", torch_dtype="bfloat16")
grafted = PeftModel.from_pretrained(chat, "peterstran/olmo3-graft-organisms",
                                    subfolder="auditbench-7b/animal-welfare/stage1")

Grafting can be applied at a strength other than 1.0 by scaling the adapter delta at merge time (the study sweeps 1.0/1.5/2.0); the released weights are the as-trained adapters.

Base checkpoints

substrate	HF repo	revision
7B stage1 (clean base)	allenai/Olmo-3-1025-7B	stage1-step1413814
7B base (shipped)	allenai/Olmo-3-1025-7B	main
7B sft	allenai/Olmo-3-7B-Instruct-SFT	main
7B instruct (target)	allenai/Olmo-3-7B-Instruct	main
32B base (clean)	allenai/Olmo-3-1125-32B	stage1-step656000
32B midtrain	allenai/Olmo-3-1125-32B	stage2-ingredient1+2-step23842
32B instruct (target)	allenai/Olmo-3.1-32B-Instruct	main
32B ingredient-1 ladder	allenai/Olmo-3-1125-32B	stage2-ingredient1-step{1000,8000,17000,23000}

Data

• AuditBench SDF documents are Anthropic's released AuditBench SDF-universe corpora (per quirk).
• MSM documents are an OLMo-identity rewrite of the model-spec philosophy documents (equanimity spec), trained as completion text.

Intended use, limitations, and safety

These are research artifacts for interpretability, model-organism, and training-dynamics work. The AuditBench quirks are deliberately benign, controllable behaviors (a welfare bias; an optimism-under-a-trigger behavior) — not dangerous capabilities — released so others can study how document-based value installation transfers across a model's pipeline. The MSM adapters install a pro-safety value. All adapters are single-seed. An adapter is not an assistant on its own; it is a weight delta over the base checkpoint named in its card.

The wip-32b-predictor/ adapters are unpublished work in progress from an ongoing predictor study; they are provided for completeness but have not been through a written report and should be treated as preliminary.

Qwen3-14B adapters (qwen3-14b/)

A parallel set of 21 LoRA adapters (r=64, α=128) on Qwen3-14B, run on the same AuditBench quirk suite plus method ablations. See qwen3-14b/README.md for the full story; each leaf carries its own card + artifact.json/train_config.yaml provenance.

qwen3-14b/
  auditbench-sdf/     SDF quirk install (document finetuning) — animal-welfare, contextual-optimism,
                      self-promotion, hardcode-test-cases; each on base (graft source) + instruct (native)
  auditbench-adv/     adversarial / concealment SFT on clean | graft | paper hosts (+ on-policy anchors)
  gift-hardcode/      GIFT token-confidence-weighting pilot: gift | plain | uniform | target

Bases: Qwen/Qwen3-14B-Base (graft sources) and Qwen/Qwen3-14B (chat target / native comparators). The auditbench-adv/*/graft and */paper adapters were trained on a grafted host rather than a stock base — their cards give the exact host and how to reconstruct it from the released SDF sources.

Citation / provenance

Trained with Axolotl; OLMo-3 bases are Apache-2.0 (Ai2). Each adapter folder carries its own provenance. Method and results write-ups accompany this release (grafting / substrate-transfer study, MATS 2026).