| --- |
| license: mit |
| language: |
| - en |
| base_model: |
| - microsoft/phi-4-gguf |
| pipeline_tag: text-generation |
| tags: |
| - phi4 |
| - gguf-connector |
| --- |
| |
| # GGUF quantized and bug fixed version of **phi4** |
|
|
| ### review |
| - bug fixed for: "ResponseError: llama runner process has terminated: GGML_ASSERT(hparams.n_swa > 0) failed" |
| - define the architecture (from none) to llama; all works right away |
|
|
| ### run the model |
| use any gguf connector to interact with gguf file(s), i.e., [connector](https://pypi.org/project/gguf-connector/) |
|
|
| ### reference |
| - base model: microsoft/[phi-4](https://huggingface.co/microsoft/phi-4) |
| - bug fixed following the guide written by [unsloth](https://unsloth.ai/blog/phi4) |
| - tool used for quantization: [cutter](https://pypi.org/project/gguf-cutter) |
|
|
| ### citation |
| [Phi-4 Technical Report](https://arxiv.org/pdf/2412.08905) |
|
|
| ### appendices: model summary and quality (written by microsoft) |
|
|
| #### model summary |
|
|
| | | | |
| |-------------------------|-------------------------------------------------------------------------------| |
| | **Developers** | Microsoft Research | |
| | **Description** | `phi-4` is a state-of-the-art open model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning.<br><br>`phi-4` underwent a rigorous enhancement and alignment process, incorporating both supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures | |
| | **Architecture** | 14B parameters, dense decoder-only Transformer model | |
| | **Inputs** | Text, best suited for prompts in the chat format | |
| | **Context length** | 16K tokens | |
| | **GPUs** | 1920 H100-80G | |
| | **Training time** | 21 days | |
| | **Training data** | 9.8T tokens | |
| | **Outputs** | Generated text in response to input | |
| | **Dates** | October 2024 – November 2024 | |
| | **Status** | Static model trained on an offline dataset with cutoff dates of June 2024 and earlier for publicly available data | |
| | **Release date** | December 12, 2024 | |
| | **License** | MIT | |
|
|
| #### model quality |
|
|
| to understand the capabilities, we (here refer to microsoft side) compare `phi-4` with a set of models over OpenAI’s SimpleEval benchmark; at the high-level overview of the model quality on representative benchmarks; for the table below, higher numbers indicate better performance: |
|
|
| | **Category** | **Benchmark** | **phi-4** (14B) | **phi-3** (14B) | **Qwen 2.5** (14B instruct) | **GPT-4o-mini** | **Llama-3.3** (70B instruct) | **Qwen 2.5** (72B instruct) | **GPT-4o** | |
| |------------------------------|---------------|-----------|-----------------|----------------------|----------------------|--------------------|-------------------|-----------------| |
| | Popular Aggregated Benchmark | MMLU | 84.8 | 77.9 | 79.9 | 81.8 | 86.3 | 85.3 | **88.1** | |
| | Science | GPQA | **56.1** | 31.2 | 42.9 | 40.9 | 49.1 | 49.0 | 50.6 | |
| | Math | MGSM<br>MATH | 80.6<br>**80.4** | 53.5<br>44.6 | 79.6<br>75.6 | 86.5<br>73.0 | 89.1<br>66.3* | 87.3<br>80.0 | **90.4**<br>74.6 | |
| | Code Generation | HumanEval | 82.6 | 67.8 | 72.1 | 86.2 | 78.9* | 80.4 | **90.6** | |
| | Factual Knowledge | SimpleQA | 3.0 | 7.6 | 5.4 | 9.9 | 20.9 | 10.2 | **39.4** | |
| | Reasoning | DROP | 75.5 | 68.3 | 85.5 | 79.3 | **90.2** | 76.7 | 80.9 | |
|
|
| \* these scores are lower than those reported by Meta, perhaps because simple-evals has a strict formatting requirement that Llama models have particular trouble following. |
|
|
