aoiandroid/nllb200-coreml-512-pal8

NLLB-200 Core ML 512 — 8-bit weight-only palettized (pal8)

Core ML mlProgram packages derived from the FLOAT32 nllb200-coreml-512 export, post-processed with 8-bit uniform palettization (coremltools.optimize.coreml.palettize_weights, weight-only). Runtime computation stays in float32; only stored weights are compressed (LUT), which avoids the decoder/KV numerical failures seen with full-graph FP16 passes.

This variant is separate from aoiandroid/nllb200-coreml-512-palettized (different compression pipeline).

Base model (official)

Upstream weights and tokenizer follow facebook/nllb-200-distilled-600M licensing and use constraints. See that model card for intended use, limitations, and citation.

Item	Value
Max length	512 (encoder and KV decoder)
Packages	Encoder, decoder init, decoder step, fused decoder + tokenizer + config

Files

Path	Description
NLLB_Encoder_512.mlpackage	Encoder
NLLB_Decoder_512_init.mlpackage	Decoder first step
NLLB_Decoder_512_step.mlpackage	Decoder step with KV cache
NLLB_Decoder_512.mlpackage	Single-graph decoder (palettized); on-device iOS client uses init + step for decoding
tokenizer/	Hugging Face tokenizer files
config.json	Model config

For KV versus monolithic decoder notes, see aoiandroid/nllb200-coreml-512.

App / model ID

on-device iOS client uses model identifier nllb200-coreml-512-pal8 with NLLBCoreMLTranslationService.

Conversion

1. Start from FLOAT32 .mlpackage tree (nllb200-coreml-512).
2. Run Notebooks/nllb/create_palettized_weight_only.py (uniform 8-bit palettize_weights).
3. Fused NLLB_Decoder_512.mlpackage is produced from the single-decoder export (local layout: Notebooks/nllb/output/nllb200-coreml-512-pal8-decoder-only/).

Benchmark (Mac, Swift Testing)

15-line 30_text.txt subset to eng_Latn: see repo report Notebooks/nllb/output/NLLB_pal8_mac_translation_test_report.md.

Validation (macOS)

python3 Scripts/validate_nllb512_bleu_macos.py --local-dir Notebooks/nllb/output/nllb200-coreml-512 --min-bleu 50

License and intended use

Same as the base model: CC-BY-NC-4.0. Research and non-commercial use; not for production without your own compliance review.

Citation

@article{nllb2022,
  title={No Language Left Behind: Scaling Human-Centered Machine Translation},
  author={{NLLB Team} and others},
  journal={arXiv preprint arXiv:2207.04672},
  year={2022}
}