Qwen3-8B FP8-OC
Derived from Qwen/Qwen3-8B at revision b968826d9c46dd6066d109eabc6255188de91218.
LLM Compressor quantizes attention and MLP matrix weights to E4M3FN with one FP32 scale per output channel. HART stores those FP8 bytes and scales in the Bertha EVT0 packed weight format. Other model parameters remain BF16. This checkpoint requires the HyperAccel vLLM plugin.
- Downloads last month
- 12
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support