Instructions to use flowxai/semantic-mapper with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use flowxai/semantic-mapper with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="flowxai/semantic-mapper") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("flowxai/semantic-mapper") model = AutoModelForCausalLM.from_pretrained("flowxai/semantic-mapper") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - MLX
How to use flowxai/semantic-mapper with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("flowxai/semantic-mapper") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - llama-cpp-python
How to use flowxai/semantic-mapper with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="flowxai/semantic-mapper", filename="gguf/semantic-mapper-4b-Q4_K_M.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use flowxai/semantic-mapper with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf flowxai/semantic-mapper:Q4_K_M # Run inference directly in the terminal: llama cli -hf flowxai/semantic-mapper:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf flowxai/semantic-mapper:Q4_K_M # Run inference directly in the terminal: llama cli -hf flowxai/semantic-mapper:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf flowxai/semantic-mapper:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf flowxai/semantic-mapper:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf flowxai/semantic-mapper:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf flowxai/semantic-mapper:Q4_K_M
Use Docker
docker model run hf.co/flowxai/semantic-mapper:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use flowxai/semantic-mapper with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "flowxai/semantic-mapper" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flowxai/semantic-mapper", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/flowxai/semantic-mapper:Q4_K_M
- SGLang
How to use flowxai/semantic-mapper with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "flowxai/semantic-mapper" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flowxai/semantic-mapper", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "flowxai/semantic-mapper" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flowxai/semantic-mapper", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use flowxai/semantic-mapper with Ollama:
ollama run hf.co/flowxai/semantic-mapper:Q4_K_M
- Unsloth Studio
How to use flowxai/semantic-mapper with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for flowxai/semantic-mapper to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for flowxai/semantic-mapper to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for flowxai/semantic-mapper to start chatting
- Pi
How to use flowxai/semantic-mapper with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "flowxai/semantic-mapper"
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "flowxai/semantic-mapper" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use flowxai/semantic-mapper with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "flowxai/semantic-mapper"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default flowxai/semantic-mapper
Run Hermes
hermes
- Atomic Chat new
- OpenClaw new
How to use flowxai/semantic-mapper with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "flowxai/semantic-mapper"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "flowxai/semantic-mapper" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- MLX LM
How to use flowxai/semantic-mapper with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "flowxai/semantic-mapper"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "flowxai/semantic-mapper" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flowxai/semantic-mapper", "messages": [ {"role": "user", "content": "Hello"} ] }' - Docker Model Runner
How to use flowxai/semantic-mapper with Docker Model Runner:
docker model run hf.co/flowxai/semantic-mapper:Q4_K_M
- Lemonade
How to use flowxai/semantic-mapper with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull flowxai/semantic-mapper:Q4_K_M
Run and chat with the model
lemonade run user.semantic-mapper-Q4_K_M
List all available models
lemonade list
Inference contract
Running this model correctly requires its frozen inference contract: the exact
system prompt, user-turn format, decode settings, and output JSON schema it was
trained against. See inference_contract/:
INFERENCE.md: wiring guide: system prompt, user turn (Extract ontology from this chunk:\n\nCHUNK:\n<text>), decode settings (enable_thinking=False, greedy/temp 0,max_new_tokens~1024), and the note thatsemantic.conceptsis constrained to the 252-concept controlled vocabulary (concept_taxonomy.yaml, shipped at repo root).prompt_mapper_sys_v1.txt: the system prompt, verbatim.schema_mapper_v1.json: output JSON Schema (structural / semantic / governance).
Prompt version mapper_sys_v1. Do not edit the prompt/schema; the weights are trained against them.
FlowX Semantic Mapper (4B): the regulation-to-ontology tagger
Reads one chunk of regulatory or legal text and emits a structured ontology JSON: where the clause sits in its source, what it is about, and how it maps to policy and escalation. Languages: EN, FR, DE, RO. Domains: banking, insurance, logistics, labor.
Given a single clause (a subsection of a state insurance code, an EU regulation article, a labor-code paragraph, an ADR packing rule), the Mapper returns three facets: a structural locator (source id, document type, and the full hierarchy path), a semantic payload (free-text domain_tags, controlled-vocabulary concepts, and an actor/action/object/constraint entity triple), and a governance payload (PDP.<domain>.<rule> policy references plus a single if ... THEN escalate guard clause).
The product insight that shapes everything: an ontology field is only useful if it is consistent enough to join on. The first version emitted concepts as open free text and produced 1062 unique ids, 88% of them singletons: unlearnable for the model and unmeasurable for us (concepts F1 sat at 0.24). Collapsing that to a 252-concept controlled taxonomy (concept_taxonomy.yaml) is the central design choice of this release: it made concepts both learnable and scoreable (F1 0.24 → 0.54), and gave downstream policy code a stable key to join on. domain_tags stay free-text on purpose (for recall and human browsing) and are honestly noisier as a result.
The Mapper is the first stage of a pipeline, not a standalone product: Mapper tags a chunk → a policy layer → the flowxai/sentinel-gate escalation model decides DECIDE vs ESCALATE. It is part of the FlowX on-device model family alongside flowxai/sentinel-gate, flowxai/caveat, and the consumer scam detectors flowxai/scam-guard-qwen06b / flowxai/scam-guard-qwen17b.
LoRA-fine-tuned from Apache-2.0 Qwen/Qwen3-4B. Formats: fp16 safetensors (transformers / CUDA / vLLM), MLX-quantized int4 + int8 (Apple Silicon), and GGUF Q8_0 + Q4_K_M (llama.cpp / CPU / Ollama).
How do I use it?
Two copy-pasteable ways to turn a regulatory chunk into ontology JSON, plus a note.
Real example input (a real clause from the California Insurance Code, section 790.03(h)(2)):
Failing to acknowledge and act reasonably promptly upon communications with respect
to claims arising under insurance policies.
The user turn wraps it in the fixed template (see the inference contract):
Extract ontology from this chunk:
CHUNK:
Failing to acknowledge and act reasonably promptly upon communications with respect to claims arising under insurance policies.
(a) transformers / CUDA (fp16 safetensors)
from transformers import AutoModelForCausalLM, AutoTokenizer
SYSTEM_PROMPT = (
"You are a legal and regulatory ontology extractor.\n"
"Extract structured tags from document chunks. Output ONLY valid JSON."
)
tok = AutoTokenizer.from_pretrained("flowxai/semantic-mapper")
model = AutoModelForCausalLM.from_pretrained("flowxai/semantic-mapper", device_map="cuda")
chunk = ("Failing to acknowledge and act reasonably promptly upon communications "
"with respect to claims arising under insurance policies.")
user = f"Extract ontology from this chunk:\n\nCHUNK:\n{chunk}"
prompt = tok.apply_chat_template(
[{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user}],
add_generation_prompt=True, enable_thinking=False, tokenize=False,
)
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=1024, do_sample=False, temperature=0.0)
raw = tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True)
(b) MLX (Apple Silicon)
Off the MLX-quantized weights (int4 / int8):
mlx_lm.generate --model flowxai/semantic-mapper-mlx-int4 --temp 0 --max-tokens 1024 \
--system-prompt "$(printf 'You are a legal and regulatory ontology extractor.\nExtract structured tags from document chunks. Output ONLY valid JSON.')" \
--prompt "$(printf 'Extract ontology from this chunk:\n\nCHUNK:\nFailing to acknowledge and act reasonably promptly upon communications with respect to claims arising under insurance policies.')"
from mlx_lm import load, generate
SYSTEM_PROMPT = (
"You are a legal and regulatory ontology extractor.\n"
"Extract structured tags from document chunks. Output ONLY valid JSON."
)
model, tok = load("flowxai/semantic-mapper-mlx-int4")
chunk = ("Failing to acknowledge and act reasonably promptly upon communications "
"with respect to claims arising under insurance policies.")
user = f"Extract ontology from this chunk:\n\nCHUNK:\n{chunk}"
prompt = tok.apply_chat_template(
[{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user}],
add_generation_prompt=True, enable_thinking=False,
)
raw = generate(model, tok, prompt=prompt, max_tokens=1024, verbose=False)
Either backend returns the same JSON for this chunk (this is the real held-out target for the example above):
{
"structural": {
"source_id": "US_CA_INS_790_03_h_2",
"hierarchy": [
"state_code", "california_insurance_code",
"section", "790.03",
"subdivision", "h",
"paragraph", "2"
],
"document_type": "state_insurance_code"
},
"semantic": {
"domain_tags": ["insurance_regulation", "claims", "unfair_claims_practices", "settlement"],
"concepts": ["claim_acknowledgement", "prompt_communication", "settlement_timeline"],
"entities": {
"actor": "insurer",
"action": "acknowledge and act reasonably promptly upon communications",
"object": "claim_communication",
"constraint": { "condition": "reasonably_prompt" }
}
},
"governance": {
"policy_references": ["PDP.insurance.claims_settlement", "PDP.insurance.unfair_practices"],
"escalation_trigger": "if !claim_communication_acknowledged_promptly THEN escalate"
}
}
(c) A short note on decoding
enable_thinking=False is not optional. Qwen3-4B is a thinking model, but this
adapter was trained on pure JSON with no reasoning block, so the default template
yields empty output. Decode greedily (temperature 0), one chunk per call, and cap
at ~1024 new tokens. GGUF (Q8_0 / Q4_K_M) runs the same contract on
llama.cpp / Ollama. Full wiring is in inference_contract/INFERENCE.md.
How it works
regulatory
text chunk ---> FlowX Semantic Mapper 4B ---> structural (source_id, hierarchy, document_type)
|
+--> semantic (domain_tags, concepts, entities)
|
+--> governance (policy_references, escalation_trigger)
Pipeline line:
chunk --> Semantic Mapper (this model) --> policy layer --> Sentinel Gate --> DECIDE | ESCALATE
Under the hood the Mapper is: tokenizer → Qwen3-4B (LoRA fine-tuned) → greedy JSON decode (thinking disabled) → parse against schema_mapper_v1.json. The
semantic.concepts field targets the 252-concept controlled taxonomy
(concept_taxonomy.yaml), so downstream code can filter out-of-vocabulary ids and
join the rest straight into the policy layer.
Output schema
A single JSON object with three facets (full schema:
schema_mapper_v1.json):
structural: where the clause lives.
source_id: stable id for the chunk, usually an uppercased normalization of the citation path (e.g.US_CA_INS_790_03_h_2).hierarchy: ordered flat array of alternating[level, value]pairs from the outermost container down to the leaf.document_type: snake_case document class (e.g.state_insurance_code,eu_regulation,labor_code,adr_agreement).
semantic: what the clause is about.
domain_tags: free snake_case topic tags (open vocabulary; noisier by design, tuned for recall/browsing).concepts: concept ids from the 252-concept controlled taxonomy (concept_taxonomy.yaml). Treat any id not in the taxonomy as out-of-vocabulary.entities:{actor, action, object, constraint}: who the obligation falls on, what must/may be done, on what, and a free key/valueconstraintmap of qualifying conditions (e.g.{"condition":"reasonably_prompt"}).
governance: how it maps to policy and escalation.
policy_references: ids in the formPDP.<domain>.<rule>(e.g.PDP.insurance.claims_settlement).escalation_trigger: a single guard clauseif <condition> THEN escalate, consumed by the Sentinel Gate.
The controlled taxonomy holds 252 canonical concepts across 6 categories
(money, rights_waived, time_renewal, lease, insurance, data), each with an id, a
definition, and a primary_domain. It ships as concept_taxonomy.yaml at the
repo root.
Evaluation
Held-out set: n = 112, document-disjoint from training, multilingual (EN / FR / DE / RO). Read the honest numbers first, then the caveat.
| Metric | Value |
|---|---|
| JSON validity | 1.00 |
| all-3-facets present | 1.00 |
domain_tags F1 (free-text) |
0.55 |
concepts F1 (controlled 252-concept vocab) |
0.54 |
concepts F1 (fuzzy / semantic match) |
0.58 |
entities field-accuracy |
0.53 |
The headline improvement, quantified. Adopting the controlled 252-concept
vocabulary plus a real-regulation data expansion moved concepts F1 from 0.24
(open free-text vocab) to 0.54, and entities from unmeasured (the facet
was empty in the first corpus) to 0.53. The format facets are saturated: JSON
validity and all-three-facets-present are both 1.00, so the model reliably produces
a parseable, structurally-complete object every time.
Honest reading.
conceptsis partial at ~0.54: it surfaces many of the right controlled ids, not all of them. Use it as a strong signal, not a complete tagging.domain_tagsare free-text and therefore less consistent than the controlledconcepts(this is the intended trade: recall/browsability over join-stability).entitiesfield-accuracy of 0.53 reflects a genuinely hard four-slot extraction (actor/action/object/constraint), now measurable where before it was not.- EN is strongest. FR, DE, RO are supported and evaluated, but a bit weaker.
Home-field caveat (read before citing these numbers). The held-out set is document-disjoint (no shared source document leaks between train and test), which is the right discipline. But the annotations across train and test share a lineage: the same controlled taxonomy and the same frontier-model-assisted annotation style. The scores above measure how well the model reproduces that annotation convention on unseen documents, which is exactly the deployment task, but they are not a claim of agreement with an independent human legal annotator. Treat 0.54 concepts F1 as "reproduces the FlowX taxonomy convention on held-out documents at ~0.54", not "a legal-grade ontology at 0.54".
Formats
| Format | Path | Runtime | Notes |
|---|---|---|---|
| fp16 safetensors | repo root | transformers / CUDA / vLLM | full precision |
| MLX int4 | mlx-int4/ |
Apple Silicon (MLX) | smallest Apple footprint |
| MLX int8 | mlx-int8/ |
Apple Silicon (MLX) | higher-fidelity Apple path |
GGUF Q8_0 |
gguf/ |
llama.cpp / CPU / Ollama | recommended GGUF quant |
GGUF Q4_K_M |
gguf/ |
llama.cpp / CPU / Ollama | smallest GGUF |
All formats run the same frozen inference contract (system prompt, user-turn
template, enable_thinking=False, greedy decode).
Intended use & limitations
Intended use. A first-stage extractor in a regulatory-compliance pipeline:
it turns a clause of legislation into a structured ontology record that a policy
layer and the flowxai/sentinel-gate
escalation model can act on. It is meant to run over a knowledge base of regulatory
chunks and produce consistent, joinable tags, with a human reviewing anything the
Sentinel routes for escalation.
Out of scope & limitations.
- Not legal advice. The Mapper labels the structure and topic of a text; it does not interpret the law, give a legal opinion, or determine compliance. A human with legal responsibility owns any decision.
- Extraction is partial.
conceptsF1 ~0.54 andentitiesaccuracy ~0.53: expect misses and the occasional wrong slot. Do not treat the output as an exhaustive or authoritative tagging. - Controlled vocab, not free-form.
semantic.conceptstargets the 252-concept taxonomy. Filter out-of-vocabulary ids before joining.domain_tagsare free-text and inherently noisier. - Language skew. EN is strongest; FR / DE / RO are supported but weaker.
- One chunk at a time. Trained on single-chunk turns; batching multiple clauses into one user turn degrades output.
- Domain coverage. Banking, insurance, logistics, labor. Text outside these domains is out of distribution.
- Annotation-lineage caveat (see Evaluation): scores measure reproduction of the FlowX annotation convention, not agreement with an independent legal annotator.
Training data
- 763 records total, split 651 train / 112 held-out, document-disjoint (no source document appears in both splits).
- Real, verbatim regulatory / legislative text from official public sources: US eCFR & state codes, EU EUR-Lex, France Légifrance, Germany Gesetze im Internet, Romania legislatie.just.ro, and UNECE/ADR. Every record carries a source URL. Official legislation is reproduced with source acknowledgement.
- Annotated to the controlled 252-concept taxonomy with frontier-model assistance, then validated: concepts checked in-vocabulary, entities checked populated, source URL required per record.
- Coverage spans four domains (banking / insurance / logistics / labor) and four languages (EN / FR / DE / RO).
The controlled taxonomy (concept_taxonomy.yaml) is the pivot of this dataset:
open free-text concepts (1062 unique, 88% singletons) were collapsed to 252
canonical ids, which is what made the concepts facet learnable and measurable.
Fine-tuning
LoRA on Qwen/Qwen3-4B with MLX-LM on Apple Silicon.
| Hyperparameter | Value |
|---|---|
| method | LoRA |
| rank | 32 |
| scale (alpha) | 16 |
| dropout | 0.05 |
| layers | num_layers -1 (all) |
| epochs | ~3 |
| LR schedule | cosine 1e-4 → 1e-5 |
| max_seq_length | 2048 |
| seed | 42 |
| thinking | disabled |
The assistant turn in training is the exact ontology JSON the model must emit (no
reasoning block), which is why inference must set enable_thinking=False.
Not legal advice
This model extracts structure and topic tags from legal/regulatory text. It does not provide legal advice, does not interpret the law, and does not determine compliance. Its output is an aid to a compliance pipeline under human ownership, not a substitute for a qualified professional.
Links
- Sibling (escalation gate):
flowxai/sentinel-gate: consumes a case (with the Mapper's governance facet) and decides DECIDE vs ESCALATE. - FlowX on-device family:
flowxai/caveat,flowxai/scam-guard-qwen06b,flowxai/scam-guard-qwen17b.
License
Apache-2.0 (weights and code). Base model Qwen/Qwen3-4B is Apache-2.0. Copyright
2026 FlowX.AI; see the NOTICE file. Author: Bogdan Răduță, Head of Research,
FlowX.AI.
- Downloads last month
- 549
Quantized
Model tree for flowxai/semantic-mapper
Evaluation results
- concepts F1 (controlled 252-concept vocab)self-reported0.540
- concepts F1 (fuzzy / semantic match)self-reported0.580
- domain_tags F1 (free-text)self-reported0.550
- entities field-accuracyself-reported0.530
- JSON validityself-reported1.000
- all-3-facets presentself-reported1.000