mradermacher

/

model_requests

like 197

The LFM one works and seems pretty similar to the non heretic. Pretty mid vision for a 3b model i think.

Mr @Timelyyyy asserts that your new vision model has "pretty mid vision for a 3B model". Do you have anything to say in response, Mr @samuelstevens ?

Ps. Heretication cannot help the model see what it wasn't trained to see but can unlock existing capacity that was withheld by the text model.

Edit: Mr @Timelyyyy has doubled down on his initial statement below, now asserting that the LFM model has "less cognitive capability" for object recognition compared to the likes of Gemma and "even some old Chinese model", connecting this inferior capacity to what he describes as "it was meant for simpler things across more languages, since all LFMs are multilingual."

No i mean that it just has less "cognitive capacity", as in what it can recognize. At this size, Gemma and even some old chinese model did better. (I ust dont remember which one, it could've been Qwen)
That's probably because it was meant for simpler things across more languages, since all LFMs are multilingual. That said, language support is not bad, but is it really worth cramming that much into 3b?

Awesome that you created some new amazing models!

Muse-Glimmer-30B-SOMPOA-heresy:
Seems like mradermacher was faster:
You can check https://hf.tst.eu/status.html for thair correct process
Static quants: https://huggingface.co/mradermacher/Muse-Glimmer-30B-SOMPOA-heresy-GGUF
Imatrix quants: https://huggingface.co/mradermacher/Muse-Glimmer-30B-SOMPOA-heresy-i1-GGUF
Convinient download page: https://hf.tst.eu/model#Muse-Glimmer-30B-SOMPOA-heresy-GGUF

LFM2.5-VL-3B-absolute-heresy:
Failed using latest llama.cpp booth with transformers 4 and transformers 5 - needs to be investigated - could be an issue with ouer setup:

INFO:hf-to-gguf:Set model tokenizer
Traceback (most recent call last):
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 308, in <module>
    main()
  File "/llmjob/llama.cpp/convert_hf_to_gguf.py", line 302, in main
    model_instance.write()
  File "/llmjob/llama.cpp-nocuda/conversion/base.py", line 1042, in write
    self.prepare_metadata(vocab_only=False)
  File "/llmjob/llama.cpp-nocuda/conversion/qwen.py", line 352, in prepare_metadata
    super().prepare_metadata(vocab_only=vocab_only)  # ty: ignore[unresolved-attribute]
  File "/llmjob/llama.cpp-nocuda/conversion/base.py", line 1231, in prepare_metadata
    self.set_vocab()
  File "/llmjob/llama.cpp-nocuda/conversion/base.py", line 1203, in set_vocab
    self._set_vocab_gpt2()
  File "/llmjob/llama.cpp-nocuda/conversion/base.py", line 1752, in _set_vocab_gpt2
    tokens, toktypes, tokpre = self.get_vocab_base()
  File "/llmjob/llama.cpp-nocuda/conversion/base.py", line 1385, in get_vocab_base
    tokenizer = AutoTokenizer.from_pretrained(self.dir_model)
  File "/llmjob/share/python-t5/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 1153, in from_pretrained
    raise ValueError(
ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.
job finished, status 1
job-done<0 Qwen3.5-9B-RolePlay-Merged noquant 1>

North-Micro-Vision-Instruct-SOMPOA-heresy - Won't possibly quant (I blame Cohere Labs):
Queued it anyways and as expected it failed due to CohereCompassForConditionalGeneration not yet beeing supported by llama.cpp

Hey there, sir Nicoboss! Thank you foremost for the quants, as well as the enthusiasm. I have to note that I'm merely a locksmith, unlocking capability. So, kudos to the model makers, who pay the hefty effort and price.

I have looked in to the issue but all configs are aligned with the original model, no discrepancies there. I can enGGUF the model and quantise it locally with llama.cpp@git and Transformers@git. Could you also try the base model: https://huggingface.co/LiquidAI/LFM2.5-VL-3B ?

I can confirm that uv pip install git+https://github.com/huggingface/transformers fixes the conversion issue. I will manually provide the source GGUF for LFM2.5-VL-3B-absolute-heresy and tell @mradermacher

LFM2.5-VL-3B-absolute-heresy is queued :D
You can check https://hf.tst.eu/status.html for thair correct process
Static quants: https://huggingface.co/mradermacher/LFM2.5-VL-3B-absolute-heresy-GGUF
Imatrix quants: https://huggingface.co/mradermacher/LFM2.5-VL-3B-absolute-heresy-i1-GGUF
Convinient download page: https://hf.tst.eu/model#LFM2.5-VL-3B-absolute-heresy-GGUF

I can confirm that uv pip install git+https://github.com/huggingface/transformers fixes the conversion issue. I will manually provide the source GGUF for LFM2.5-VL-3B-absolute-heresy and tell @mradermacher

Is the Transformers issue causing the errors seen in some of the recent MoE quant failures?

Is the Transformers issue causing the errors seen in some of the recent MoE quant failures?

No those where all different failures:

• Goetia-26B-A4B-v1.4-LazyLora-heresy: KeyError: 'global_head_dim'
• Qwen3.6-35B-A3B-abliterated-v4: error loading model: missing tensor 'blk.40.attn_norm.weight'
• Ornith-1.0-35B-uncensored-heretic: error loading model: missing tensor 'blk.40.attn_norm.weight'

But we indeed had around half the models fail because of the transformers issue but we now addressed this upstream dependency issue inside our own llama.cpp fork and retried all that failed because of it that where still inside the queue.

I may have nuked global_head_dim or renamed it to head_dim or something entirely else in an attempt to get the LoRA merge. I don't know how but I can encase that Goetia in GGUF and Quant it locally.

Missing tensor errors around Qwen could stem from models that had lost their MTP layers post-finetune. You may want to pass --no-mtp (arg might be wrong but they added something like this to the convert script). I believe more recent versions of llama-cpp can ignore the missing MTP layers automagically.

Transformers can sometimes be a total jerk.