poor multi language understanding

#5
by rastegar - opened

Hi,
this model work great in English
but i tested this model on multiple language tasks and it result on this useless (vs ds-v4-flash that work great in multilanguage tasks)
in this case i tested in Persian language
best regards

yeah i doubt there's a lot of Persian in the training dataset lol
this model seems to be optimized for coding and English, which made it possible to get high benchmark results

you can't expect a model with ~120b params to be perfect at everything at once

Yes i aggree, was already wondering how i mesed german Frontend/UI up so badly 🫥

well, its optimized for long horizon tasks and not a general model for chit-chatting, in your case Persian. Plus as grumd0 said, Persian isn't really popular. but if your looking for a strong general model, I recommend qwen3.5 with support for 201 languages. Good luck!!

I tested with Chinese, when it starts working, it turns to English thinking.
My colleage reported the Laguna will generate mixed Chinese and Traditional Chinese characters.
I think English and the Chinese are the main AI languages, should be well supported.

Poolside org

Hello everyone and thank you for your comments! This is a result of trade-offs we had to make for agentic capabilities. Let us take the feedback and see how we can improve in future. 🙏

Agentic is really good, fluent tool callings.
Coding / Agentic is the spirit of LLM, keep focusing on it, nice jobs!

well, its optimized for long horizon tasks and not a general model for chit-chatting, in your case Persian. Plus as grumd0 said, Persian isn't really popular. but if your looking for a strong general model, I recommend qwen3.5 with support for 201 languages. Good luck!!

ok, i understand and see user reports about other popular languages there are issues in model because of training target purpose

just for clarify
persian is in top 15 languages of internet , it is dataset rich and very popular in case of content

Rank Language 15 May 2023 3 December 2025
1 English 55.5% 49.3%
2 Spanish 5.0% 6.0%
3 German 4.3% 5.9%
4 Japanese 3.7% 5.1%
5 French 4.4% 4.5%
6 Portuguese 2.4% 4.1%
7 Russian 4.9% 3.7%
8 Italian 1.9% 2.8%
9 Dutch 1.5% 2.2%
10 Polish 1.4% 1.8%
11 Turkish 2.3% 1.6%
12 Chinese 1.4% 1.1%
13 Persian 1.8% 1.1%
14 Vietnamese 1.3% 1.0%
15 Czech 0.7% 1.0%

I agree with @CHTL85 , I can't agree with you @rastegar , where did you get the list? that's a strong bias, Persian apparently is a minor language in the world. Let's see what models dominate the LLM, it's America and China, (mostly close-AI vs open-AI)

Here is the list from google

Top 10 Most Spoken Languages in the World

Rank,Language,Native Speakers (L1),Non-Native Speakers (L2),Total Speakers
1,English,~390 Million,~1.138 Billion,1.528 Billion
2,Mandarin Chinese,~990 Million,~194 Million,1.184 Billion
3,Hindi,~345 Million,~264 Million,609 Million
4,Spanish,~484 Million,~74 Million,558 Million
5,Modern Standard Arabic,~0 Million*,~335 Million,335 Million
6,French,~80 Million,~232 Million,312 Million
7,Bengali,~234 Million,~50 Million,284 Million
8,Portuguese,~252 Million,~15 Million,267 Million
9,Russian,~147 Million,~106 Million,253 Million
10,Indonesian,~43 Million,~209 Million,252 Million

Reference:
https://www.superprof.co.uk/blog/most-spoken-languages/#:~:text=chat_bubble,total%20number%20of%20speakers%20worldwide.
https://www.berlitz.com/blog/most-spoken-languages-world#:~:text=1.,native%20and%20non%2Dnative%20speakers.

but

I agree with @CHTL85 , I can't agree with you @rastegar , where did you get the list? that's a strong bias, Persian apparently is a minor language in the world. Let's see what models dominate the LLM, it's America and China, (mostly close-AI vs open-AI)

Here is the list from google

Top 10 Most Spoken Languages in the World

Rank,Language,Native Speakers (L1),Non-Native Speakers (L2),Total Speakers
1,English,~390 Million,~1.138 Billion,1.528 Billion
2,Mandarin Chinese,~990 Million,~194 Million,1.184 Billion
3,Hindi,~345 Million,~264 Million,609 Million
4,Spanish,~484 Million,~74 Million,558 Million
5,Modern Standard Arabic,~0 Million*,~335 Million,335 Million
6,French,~80 Million,~232 Million,312 Million
7,Bengali,~234 Million,~50 Million,284 Million
8,Portuguese,~252 Million,~15 Million,267 Million
9,Russian,~147 Million,~106 Million,253 Million
10,Indonesian,~43 Million,~209 Million,252 Million

Reference:
https://www.superprof.co.uk/blog/most-spoken-languages/#:~:text=chat_bubble,total%20number%20of%20speakers%20worldwide.
https://www.berlitz.com/blog/most-spoken-languages-world#:~:text=1.,native%20and%20non%2Dnative%20speakers.

but you measure wrong parameter , people speak and model read and write, so you should check how much text available
https://en.wikipedia.org/wiki/Languages_used_on_the_Internet

I frequently heard that the India is the second superpower of the world, it beats China, and that's fine.
So have a nice dream if Persian is even pouplar than Chinese, we'd love to see a Persian AI model.

I frequently heard that the India is the second superpower of the world, it beats China, and that's fine.
So have a nice dream if Persian is even pouplar than Chinese, we'd love to see a Persian AI model.

i just shared what data is available for train models based on Wikipedia reference
seems you lose evidence base reasoning and discussing , i will not continue ,

Sign up or log in to comment