TCR-Foundation · joint-tiny
A small per-chain BERT encoder for T-cell receptor (TCR) sequences. Trained jointly on paired α/β and bulk β repertoires (MLM + paired contrastive + chain-drop), it maps a clonotype to a 128-d embedding; single β chains and paired chains share one coordinate system.
- Arch: per-chain relative-position BERT, hidden 128, 4 layers, 4 heads + a mixed pooler (
poolers/joint.pt). - Input grammar:
CDR1 | CDR2 | CDR3(germline CDR1/CDR2 come from the V gene via tidytcells).
Files
backbone/ (BertForMaskedLM + config), poolers/joint.pt (the trained mixed pooler), tokenizer/,
joint_config.json (arch + pooling metadata).
Use it (recommended: the tcr_foundation library)
The model is a custom backbone plus a separate pooler, so use the library — it loads both and gives the
pooled per-clonotype embedding. load() pulls the weights from this repo automatically.
# pip install -e "tcr_foundation[neural]" (from the repo; self-contained, brings its own pipeline)
import pandas as pd
from tcr_foundation import load
enc = load("joint-tiny") # downloads weights from HF if not present locally
df = pd.DataFrame({"v_gene": ["TRBV20-1", "TRBV7-2"],
"cdr3": ["CASSYSTDTQYF", "CASSLAPGATNEKLFF"]})
Z, keep = enc.encode(df) # per-clonotype embeddings: Z [N, 128], keep = kept-rows mask
print(Z.shape) # (2, 128)
Repertoire-level descriptor (one vector per sample):
from tcr_foundation import descriptors
vec = descriptors.MeanCov(enc).featurize(df) # order-1 mean ++ order-2 covariance of the cloud
Raw (transformers only)
AutoModelForMaskedLM.from_pretrained("argentel/tcr-foundation-joint-tiny", subfolder="backbone") loads the
backbone, but not the trained pooler — raw token embeddings only. For the pooled clonotype vector, use
the library above.