OmicsFM

A foundation model over paired proteomic and transcriptomic expression. Attention between measured features yields a protein-protein association network without task-specific training.

Checkpoints

name	modality	feature ID embedding	parameters
proteomics	proteomics	learned	4.94 M
proteomics_esmc	proteomics	ESM-C	5.75 M
bulk_transcriptomics	bulk transcriptomics	learned	4.94 M
bulk_transcriptomics_esmc	bulk transcriptomics	ESM-C	5.75 M
sc_transcriptomics	single-cell transcriptomics	learned	4.94 M
sc_transcriptomics_esmc	single-cell transcriptomics	ESM-C	5.75 M
sc_transcriptomics_ensembl	single-cell transcriptomics	learned	4.94 M

All but one are mapped onto a shared UniProt accession vocabulary of 20,272 accessions. sc_transcriptomics_ensembl is the exception: it is trained directly on 61,497 ENSEMBL gene IDs, with no gene-to-protein mapping applied. It exists so the cost of that mapping can be measured. Mapping genes to proteins drops features that fail to map, and drops ground-truth edges between them; comparing this checkpoint against sc_transcriptomics isolates that effect, because the two are identical in architecture and training and differ only in vocabulary.

Feature ID embedding. The learned variants train an embedding table over the vocabulary from scratch. The ESM-C variants instead look each feature up in a frozen table of precomputed ESM-C sequence embeddings and pass it through a small trained projection, so identity is grounded in protein sequence rather than learned from co-expression alone.

Parameter counts are the trained model excluding the feature ID embedding, which scales with vocabulary rather than model capacity and would otherwise dominate. The transformer is identical across all seven at 4.94 M parameters; the ESM-C variants add 0.81 M for the projection. Excluded on top of that: the 5.19 M learned embedding table (learned variants) and the 23.35 M frozen ESM-C lookup (ESM-C variants), which is not trained at all. sc_transcriptomics_ensembl carries a larger 15.74 M table, since its vocabulary is three times the size -- which is why the exclusion matters: it is the same model, not a bigger one.

Repository contents

Each checkpoint folder holds best_model.ckpt and a config.yaml recording the architecture, hyperparameters, training epoch and validation loss, readable without loading the checkpoint.

esmc_human_cache.pt (95 MB) sits at the repository root and is required by the three _esmc checkpoints, which use it as a frozen lookup table instead of learning their feature embeddings. It holds one 1152-dimensional vector per UniProt accession: ESM-C 600M run over the canonical human proteome, mean-pooled per sequence. It can be recomputed, but that means downloading a 600M-parameter model and running inference over 20,000 sequences. Shipping it keeps the embeddings bit-identical to what the models saw during training, and lets anyone run inference with the _esmc checkpoints straight away. The learned checkpoints do not need it.

Use

from omicsfm.hub import get_checkpoint
from omicsfm.api import attention_map

ckpt = get_checkpoint("sc_transcriptomics")   # downloaded and cached on first use
network = attention_map(str(ckpt), data_path=...)

The ESM-C variants additionally need the canonical human proteome FASTA and the precomputed embedding cache; see the repository README.

Citation

Publication in preparation.