Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

1,011

Base only

Active filters: 4bit

tacodevs/Rocinante-X-12B-v1-GPTQ

Updated Apr 3 • 23

livadies/gemma-4-E2B-Ghetto-NF4

Text Generation • 5B • Updated Apr 3 • 22

livadies/gemma-4-31B-Base-Ghetto-NF4

Text Generation • 32B • Updated Apr 3 • 10 • 2

livadies/gemma-4-E4B-Ghetto-NF4

Text Generation • 8B • Updated Apr 3 • 39

bbarn4/medgemma-27b-text-it-GPTQ

Text Generation • 5B • Updated Apr 4 • 777 • 2

mlx-community/Qwen3-4B-ARC-MLX-4bit

Text Generation • 0.6B • Updated Apr 4 • 26 • 2

3amthoughts/DeepLink-R1-GGUF

Text Generation • 8B • Updated Apr 4 • 86 • 1

SanDiegoDude/JoyAI-Image-Edit-NF4

Updated Apr 4 • 1

ebircak/gemma-4-31B-it-4bit-W4A16-GPTQ

Text Generation • 32B • Updated Apr 5 • 1.93k • 3

lew96123/qwen3.5-0.8b-custom-packed-4bit

Image-Text-to-Text • Updated Apr 6 • 6

ebircak/gemma-4-31B-it-4bit-NVFP4A16-GPTQ

Text Generation • 18B • Updated Apr 5 • 768 • 2

jason-schulz/Carnice-9b-MLX

Text Generation • 1B • Updated Apr 5 • 28

Alkd/Qwen3-ForcedAligner-0.6B-4bit

Audio Classification • 0.4B • Updated Apr 5 • 7

Alkd/Qwen3.5-4B-MLX-4bit

1.0B • Updated Apr 6 • 12

mtecnic/Qwen3-Coder-Next-REAP-AWQ

Text Generation • 9B • Updated Apr 6 • 172 • 1

Crusadersk/mistral-7b-gptq-4bit

Text Generation • 7B • Updated Jul 6 • 9

Crusadersk/mistral-7b-awq-4bit

Text Generation • 7B • Updated Jul 6 • 25

Crusadersk/qwen2.5-7b-gptq-4bit

Text Generation • 8B • Updated Jul 6 • 36

Crusadersk/qwen2.5-7b-awq-4bit

Text Generation • 2B • Updated Jul 6 • 9

OpenMinded-Labs/AileyNitro-2B

Text Generation • 1B • Updated May 3 • 7

OpenMinded-Labs/LintingHamster-1.5B

0.2B • Updated Apr 6 • 4

Ferox-AI/Phi-4-multimodal-instruct-mlx-4bit

Image-Text-to-Text • 2B • Updated Jun 19 • 169

majentik/MERaLiON-2-3B-MLX-4bit

Automatic Speech Recognition • Updated Apr 17 • 8

majentik/MERaLiON-2-10B-MLX-4bit

Automatic Speech Recognition • Updated Apr 17 • 6

lew96123/qwen3.5-0.8b-custom-packed-turboquant_mse-true-uniform-4bit

Image-Text-to-Text • Updated Apr 7 • 4 • 1

Z3NN001/gemma4-21b-a4b-REAP-it-mlx-Q4

Image-to-Text • 4B • Updated Apr 6 • 88 • 4

OsaurusAI/gemma-4-E2B-it-4bit

Any-to-Any • 1B • Updated Apr 6 • 383

OsaurusAI/gemma-4-E4B-it-4bit

Any-to-Any • 2B • Updated Apr 6 • 809

OpenMinded-Labs/CodeAiley-14B

15B • Updated Apr 7 • 7

youngouk/seastar-medium-ko-4bit-mlx

Automatic Speech Recognition • Updated Apr 7 • 6