HuggingFaceBio/Carbon-500M-GGUF

Carbon-500M GGUF

Running with llama.cpp

Carbon is a base DNA model (not instruction-tuned). Use the raw completion front-end so llama.cpp does not apply any chat templating:

llama-completion \
  -hf HuggingFaceBio/Carbon-500M-GGUF:Q8_0 \
  --prompt "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG" \
  -n 128 --temp 0 --no-display-prompt

If you use llama-cli instead, pass -no-cnv so it does not wrap the prompt in assistant-style markers (otherwise the model may emit annotation tags such as <protein_coding_region> instead of DNA).

Available files

Quant	File	Size	SHA256
BF16	carbon-500m-bf16.gguf	982.2 MB	bc4fffc27e07ff2e30209a508b127e838ac4ade9d80ffb59128d250cbe27e1aa
Q8_0	carbon-500m-q8_0.gguf	524.6 MB	773242f1d13f512d7a32b21c71e626ddc15f041a0f151b552bd65e1ec2f0306d
Q6_K	carbon-500m-q6_k.gguf	443.2 MB	b62147987c449a1135f3b069d9a355cf21cb153865318d81e26cfae196018f1f
Q5_K_M	carbon-500m-q5_k_m.gguf	405.1 MB	ae8dc81e0eba813376a50ebbefc66a39dae15ba83ef9cd2f2f6e07602ee1d7f7
Q4_K_M	carbon-500m-q4_k_m.gguf	369.2 MB	22726da36904fdcd1bdc4df74e3bafcd7228dcd468c070ce4a4742af0b79a272

K-quants are produced with --token-embedding-type Q8_0 --output-tensor-type Q8_0 so the sensitive embedding/output tensors stay at Q8_0 precision (important for this model's tight DNA-vs-text logit margins).

Requires a recent llama.cpp

HybridDNATokenizer support was merged in ggml-org/llama.cpp#23410, so any build from master after that works:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j

Files

File	Quant	Size
carbon-500m-bf16.gguf	bf16 (lossless from source)	983 MB

Usage

Download

hf download HuggingFaceBio/Carbon-500M-GGUF carbon-500m-bf16.gguf --local-dir .

Basic DNA completion

./build/bin/llama-completion -m carbon-500m-bf16.gguf \
    -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
    -n 64 --temp 0 -no-cnv

As a draft model for speculative decoding

Carbon-500M shares the HybridDNA vocab with the larger models, so it makes an excellent draft model:

# 8B target + 500M draft -> ~2x speedup at temp=0
./build/bin/llama-speculative \
    -m  carbon-8b-bf16.gguf \
    -md carbon-500m-bf16.gguf \
    -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
    -n 256 --temp 0

See also

• Source weights: HuggingFaceBio/Carbon-500M
• Other GGUF variants: 500M · 3B · 8B

License

Apache-2.0, inherited from the source model.