Ornith-1.0-35B-4bit

4-bit (group size 64, 4.649 bits/weight) MLX quantization of deepreinforce-ai/Ornith-1.0-35B, produced with mlx-vlm 0.6.3. Full multimodal: the vision encoder is preserved and quantized alongside the language model. For Apple Silicon. Runs in mlx-vlm or any MLX app.

Conversion note (MoE expert fusion)

Ornith stores its 256 MoE experts unfused (per-expert), but mlx-vlm's qwen3_5_moe loader expects them fused/batched. A sanitize monkeypatch was required to stack the experts before conversion; without it the conversion failed. This is a standard mlx-vlm 4-bit quant.

Usage

uvx --from mlx-vlm mlx_vlm.generate \
  --model mlx-community/Ornith-1.0-35B-4bit --image image.png \
  --prompt "Describe this image." --max-tokens 512

from mlx_vlm import load, generate
model, processor = load("mlx-community/Ornith-1.0-35B-4bit")

Conversion check

Smoke-tested after conversion (text-only prompt, mlx_vlm.generate): coherent — solved 17 * 24 = 408 with correct step-by-step reasoning, no repetition loop. 103.7 tok/s generation, 89.4 tok/s prompt, peak 20.9 GB on a Macbook Pro M5 Max 128GB 40 GPU.

Refer to the original model card for architecture, benchmarks, license, and intended use.