Instructions to use Laborator/microlens-final with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Laborator/microlens-final with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/gemma-4-e2b-it-unsloth-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "Laborator/microlens-final")

llama-cpp-python

How to use Laborator/microlens-final with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Laborator/microlens-final",
	filename="mmproj.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Laborator/microlens-final with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Laborator/microlens-final
# Run inference directly in the terminal:
llama-cli -hf Laborator/microlens-final

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Laborator/microlens-final
# Run inference directly in the terminal:
llama-cli -hf Laborator/microlens-final

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Laborator/microlens-final
# Run inference directly in the terminal:
./llama-cli -hf Laborator/microlens-final

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Laborator/microlens-final
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Laborator/microlens-final

Use Docker

docker model run hf.co/Laborator/microlens-final

LM Studio
Jan

vLLM

How to use Laborator/microlens-final with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Laborator/microlens-final"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Laborator/microlens-final",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Laborator/microlens-final

Ollama
How to use Laborator/microlens-final with Ollama:
```
ollama run hf.co/Laborator/microlens-final
```

Unsloth Studio

How to use Laborator/microlens-final with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Laborator/microlens-final to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Laborator/microlens-final to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Laborator/microlens-final to start chatting

How to use Laborator/microlens-final with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Laborator/microlens-final

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Laborator/microlens-final"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Laborator/microlens-final with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Laborator/microlens-final

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Laborator/microlens-final

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use Laborator/microlens-final with Docker Model Runner:
```
docker model run hf.co/Laborator/microlens-final
```

Lemonade

How to use Laborator/microlens-final with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Laborator/microlens-final

Run and chat with the model

lemonade run user.microlens-final-{{QUANT_TAG}}

List all available models

lemonade list

microlens-final / README.md

Laborator

Make demo video block obvious: heading link + red YouTube badge

a4770bc verified about 1 month ago

preview code

Raw

History Blame Contribute Delete

6.89 kB

metadata

license: apache-2.0
base_model: unsloth/gemma-4-E2B-it
library_name: peft
pipeline_tag: image-text-to-text
tags:
  - microscopy
  - vision-language
  - diatoms
  - fungal-spores
  - biology
  - bioindicator
  - gemma-4
  - unsloth
  - qlora
  - multimodal
  - on-device
  - offline
datasets:
  - sergheibrinza/microlens-vqa-hackathon
  - sergheibrinza/microlens-images-hackathon
language:
  - en
  - de
  - fr
  - es
  - it
  - pt
  - ru
  - zh
  - ja
  - ko

MicroLens — Final

A pocket-microscope expert. Vision-language model that identifies microscopy specimens — diatoms and fungal spores across 95 genera — names the genus, and explains morphology, habitat, and identification cues. Built on Gemma 4 E2B, runs offline on a 4 GB Android, speaks 140+ languages out of the box.

Submission to the Kaggle Gemma 4 Good Hackathon 2026.

Demo video

🎬 Watch the 90-second demo on YouTube

Base Gemma 4 vs MicroLens on real diatom and fungal-spore specimens.

Links

Resource	URL
Live web demo	https://huggingface.co/spaces/Laborator/microlens
Live Kaggle notebook (T4, 9 min)	https://www.kaggle.com/code/sergheibrinza/microlens-final
GitHub (source, APK, Modelfile)	https://github.com/SergheiBrinza/microlens
Training VQA dataset (75,491 pairs)	https://www.kaggle.com/datasets/sergheibrinza/microlens-vqa-hackathon
Training images (75,491 PNGs)	https://www.kaggle.com/datasets/sergheibrinza/microlens-images-hackathon
Ollama (3 GB GGUF)	`ollama run brinzaengineeringai/microlens-final`
Android APK	https://github.com/SergheiBrinza/microlens/releases

What this model is

A 4-bit QLoRA fine-tune of unsloth/gemma-4-E2B-it that turns a generic vision-language model into a structured microscopy assistant. For any specimen image, MicroLens returns four sections:

Genus (and species when it is sure)
Morphology — shape, size, raphe, frustule
Habitat — where this organism typically lives
Identification cues — what to look for in the image

Covers 95 genera across two categories: diatoms (the standard bioindicator behind the EU Water Framework Directive) and fungal spores.

Quick start (Python + Unsloth)

from unsloth import FastVisionModel
from peft import PeftModel
from PIL import Image
import torch

base, tokenizer = FastVisionModel.from_pretrained(
    'unsloth/gemma-4-E2B-it',
    load_in_4bit=True,
    use_gradient_checkpointing='unsloth',
    max_seq_length=2048,
)
model = PeftModel.from_pretrained(base, 'Laborator/microlens-final')
FastVisionModel.for_inference(model)

img = Image.open('your_specimen.png').convert('RGB')
prompt = 'Identify the organism in this microscopy image and describe its morphology.'
msgs = [{'role':'user','content':[{'type':'image'},{'type':'text','text':prompt}]}]
text = tokenizer.apply_chat_template(msgs, add_generation_prompt=True)
inp = tokenizer(img, text, add_special_tokens=False, return_tensors='pt').to('cuda')
out = model.generate(**inp, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(out[0][inp.input_ids.shape[-1]:], skip_special_tokens=True))

Quick start (Ollama, on-device)

ollama run brinzaengineeringai/microlens-final

Pulls the 3 GB Q4_K_M GGUF and runs entirely on CPU or any consumer GPU.

Training summary

Base model: unsloth/gemma-4-E2B-it (4.44 B parameters, ~2 B effective via Per-Layer Embeddings)
Method: 4-bit QLoRA via Unsloth FastVisionModel, both vision tower and language tower trainable
Data: 75,491 VQA pairs (67,121 train + 8,370 val), 95 genera, 2 categories
Schedule: 2 epochs, 8,392 steps, lr 2e-4 cosine, batch 2×8=16, AdamW-8bit, bf16, seq 2048
Hardware: 1× RTX 3090 Ti (24 GB), 14.7 hours wall-clock
Trainable params: 29.9 M (0.58% of base), LoRA r=16, α=32
Final eval loss: 0.0189 (smooth monotone decrease)

Evaluation results

Stratified 200-pair validation, 150 diatom + 50 fungal spore.

Metric	Diatom (n=150)	Fungal spore (n=50)	Overall (n=200)
Genus accuracy (substring match)	85.3%	100%	89.0%
Category accuracy	100%	100%	100%
Format adherence (morphology + habitat + cues)	95.3%	72.0%	89.5%

Reproducible end to end on a free Kaggle T4 in 9 minutes — see the linked Kaggle notebook.

Training data — license-clean for commercial use

Source	License	Pairs (train)
UDE Diatoms in the Wild 2024 (Zenodo 10410655)	CC0	39,389
DIATLAS (Zenodo 16260887)	CC-BY 4.0	23,544
TgFC — Tectona grandis fungal community (figshare 28855910)	CC-BY 4.0	4,188

Top-30 genera have hand-curated knowledge-base answers from AlgaeBase, WoRMS, ITIS. Only upstream sources whose licences unambiguously permit commercial reuse (CC0 or CC-BY 4.0) are included, so this release is clean for commercial use end to end.

Honest limits

Trained on stained light-microscopy at 384×384. SEM and fluorescence are out of distribution.
Only 95 genera across two categories (diatoms + fungal spores). Anything else is out of distribution and the model output should be treated as ungrounded.
Long-tail genera produce shorter answers. The curated knowledge base only covers the top 30.
Confidence is expressed in words ("looks like X but the asymmetry suggests Y"), not calibrated probabilities. Good for an explainable assistant, bad for automated decisions.
No held-out test split. The 8,370 val pairs do double duty for per-step and final eval. A future release will fix that.
Research artefact — not a medical device. Not for clinical, diagnostic, or regulatory use.

License & attribution

Apache 2.0 — matches base Gemma 4 license. Please credit Serghei Brinza — MicroLens, Vienna 2026.

Citation

If you use MicroLens in research, please cite:

@misc{brinza2026microlens,
  author       = {Serghei Brinza},
  title        = {MicroLens: A Pocket-Microscope Expert via Gemma 4 E2B},
  year         = 2026,
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Laborator/microlens-final}},
  note         = {Kaggle Gemma 4 Good Hackathon 2026 submission}
}

Also cite the upstream:

Gemma 4 (Google DeepMind)
Unsloth (Daniel & Michael Han) — https://github.com/unslothai/unsloth
AlgaeBase, WoRMS, ITIS — taxonomic knowledge bases
UDE Diatoms in the Wild 2024 (Zenodo 10410655)
DIATLAS (Zenodo 16260887)
TgFC (figshare 28855910)