| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: google/gemma-2 |
| pipeline_tag: image-text-to-text |
| tags: |
| - gemma |
| - gguf |
| - vision |
| - llama.cpp |
| - quantized |
| --- |
| |
| # 🏔️ Winter LLM (Gemma Multimodal GGUF) |
|
|
| Welcome to **Winter LLM**, a lightweight, high-performance GGUF build based on Google's Gemma architecture with multimodal (vision + text) capabilities. Designed for fast, efficient local execution on consumer hardware, PCs, and edge setups across any region—from Kigali to anywhere in the world! 🌍✨ |
|
|
| --- |
|
|
| ## 🚀 Overview |
|
|
| * **Base Architecture:** Gemma Multimodal |
| * **Format:** Ready-to-run `.gguf` |
| * **Features:** |
| * 💬 **Text Generation & Chat:** Fast local inference. |
| * 👁️ **Vision Capability:** Powered by the included `mmproj` vision adapter for reading and understanding images. |
| * ⚡ **Optimized Memory Usage:** Quantized to run efficiently on low-VRAM GPUs or standard CPUs. |
|
|
| --- |
|
|
| ## 📂 Included Files |
|
|
| | File Name | Size | Purpose | |
| | :--- | :--- | :--- | |
| | `winter llm.gguf` | ~6 GB | Main LLM model weights & tokenizer | |
| | `mmproj-gemma-4-12B-it-QAT-BF16.gguf` | ~100 MB | Multimodal vision projector (for image inputs) | |
|
|
| --- |
|
|
| ## 💻 How to Run Locally |
|
|
| You can run **Winter LLM** offline using any GGUF-compatible software: |
|
|
| ### 1. LM Studio / Jan AI |
| 1. Download both `winter llm.gguf` and the `mmproj-...gguf` file from the **Files** tab. |
| 2. Drag and drop `winter llm.gguf` into LM Studio. |
| 3. Attach the `mmproj` projector in settings if you wish to upload images to the chat! |
|
|
| ### 2. Ollama / llama.cpp |
| Run via terminal using `llama.cpp`: |
| ```bash |
| ./main -m "winter llm.gguf" --mmproj "mmproj-gemma-4-12B-it-QAT-BF16.gguf" -p "User: Describe this image: [image] Assistant:" |
