Gemma 4
After some comparisons I can tell that something is wrong with LTX 2.5. The technical report says that has a bigger improvement, but sometimes the model performs worse than LTX 2.3. The model training by itself looks great, but looks like the model is having issues with its text encoder (unfortunately). I wonder, what if LTX 2.5 used Qwen3-vl-8B-it instead Gemma 4-12B-it? Could the model work better as Minimax-H3? Note this is just my opinion, from someone who likes LTX and believe in its potential.
Are you using the one of those text encoders?
Since the model and the encoder were trained together, using the model with any other encoder produces sub optimal results.
If you are using the text encode API - then the new encoder is not supported there yet and we are working on adding support for it.
Are you using the one of those text encoders?
Since the model and the encoder were trained together, using the model with any other encoder produces sub optimal results.
If you are using the text encode API - then the new encoder is not supported there yet and we are working on adding support for it.
Of course. I also tested the model online on Civitai with its "full" capacity, and the results is still the same as I had said. I'm not taking about the visual thing (the video is clean, I can see it), but the generation quality, such bad motion and very noticeable inconsistency, like a character losing its initial appearance or disappearing in the background. Then I'm wondering, what if the text encoder was different?...
- You should not train the text encoder together with the model – LLMs have all the world knowledge/internal world model you already need.
- Do not train it, you will break post-training/alignment, the original team was so much time creating, and it will also result in a broken model
- Gemma is an absolutely fine model, with a lot of anatomical / cultural knowledge, but...
- Make it BIGGER. Imagen/Waver papers, Minimax community text encoder replacement experiments have shown that precise world knowledge (sparse entities) is lost when you downsize the encoder. Gemma 4 31B or a very budget Gemma 4 26B A6 would have been awesome for knowledge / extrapolation / encoding
Thank you
- You should not train the text encoder together with the model – LLMs have all the world knowledge/internal world model you already need.
- Do not train it, you will break post-training/alignment, the original team was so much time creating, and it will also result in a broken model
- Gemma is an absolutely fine model, with a lot of anatomical / cultural knowledge, but...
- Make it BIGGER. Imagen/Waver papers, Minimax community text encoder replacement experiments have shown that precise world knowledge (sparse entities) is lost when you downsize the encoder. Gemma 4 31B or a very budget Gemma 4 26B A6 would have been awesome for knowledge / extrapolation / encoding
Thank you
You reply like bro will delete everything and really do something after my feedback, lmao. I mean, it's just a hypothetical idea. Qwen 3 VL series are fully multimodal and easy to finetune or adapt. I would like to see LTX 2.5 do things like Minimax-H3 do, you know?
If anything, Gemma4 12B might have better representations for visual stuff, since it's built on a unified architecture with a single model natively processing the text and the images in the same latent space.
Yes, but it's much smaller in size than Gemma 4 26B A6 or Gemma 4 31B and scaling of the encoder is much important for encoding as it embeds world knowledge and allows for (up to some extent) extrapolation of visual data-missing concepts.
And it's visual-language too, though not unified, but still vision-language
If anything, Gemma4 12B might have better representations for visual stuff, since it's built on a unified architecture with a single model natively processing the text and the images in the same latent space.
Maybe, but I think that LTX 2.5 can't handle it very well, it's hard to adapt Gemma 4 as a video text encoder.
Maybe good old Gemma will perform better after she's been thoroughly labotomized
@anth0nx Gemma 4 is an absolutely good model, with solid world knowledge and not lobotomized, the safety is less than in qwen, if you jailbreak it (system prompts work quite fine for RPs) or use a heretic version, you will see that its anatomy and scenario knowledge is very good. If LTX want to use a Western model instead of Qwen, Gemma 4 is a very good choice.
@anth0nx Gemma 4 is an absolutely good model, with solid world knowledge and not lobotomized, the safety is less than in qwen, if you jailbreak it (system prompts work quite fine for RPs) or use a heretic version, you will see that its anatomy and scenario knowledge is very good. If LTX want to use a Western model instead of Qwen, Gemma 4 is a very good choice.
I hope this reply is being ironic or shit bcs I really don't know how did you assume that Gemma 4 is more uncensored and better for anatomy, which is not true bcs LTX series performs bad in this scenario... I understand your point but anyway, you don't need a heretic or abliterated version for qwen models (at least the Qwen3vl series).
I really don't know how did you assume that Gemma 4 is more uncensored and better for anatomy, which is not true bcs LTX series performs bad in this scenario...
And I really don't know why you would assume the bad performance of LTX series in this scenario is because of using Gemma.
I mean it's about the understanding of the world. If the text encoder (especially a multimodal encoder) knows what should be the lighting, poses or where the people should locate in a situation it gives a massive advantage to the diffusion model even if the data is vague. The text encoder absolutely boosts the surrealism and the possibility for the model to merge unrelated concepts, in the body/clothes/anatomical situations alike.
As cited in the other issue, for example, Bytedance's Waver https://arxiv.org/pdf/2508.15761 and Google's Imagen https://arxiv.org/abs/2205.11487 have shown that the scaling of the text encoder (the "brain" of the model) often gives more results in prompt following than the scaling of the sole diffusion model
I really don't know how did you assume that Gemma 4 is more uncensored and better for anatomy, which is not true bcs LTX series performs bad in this scenario...
And I really don't know why you would assume the bad performance of LTX series in this scenario is because of using Gemma.
Did you know that the text encoder is essential for the model training and performance as well? There's no problem with LTX series, but with its text encoder (in theory), do you understand? As I've said, Gemma series are hard to adapt, specially for video models. And yes, I'm not assuming that Gemma 4 is really the problem with LTX 2.5, but it's pretty strange when the model training looks good, but the model performs worse than its previous version. And I've seen a lot of feedbacks reporting bad consistency and bad prompt adherence in the 2.5 version (which is true bcs I can confirm this).