BF16 weights?

#4
by TheLatentSpacer - opened

Hi there, thanks for publishing this great model!

I’ve experienced significant quality loss using FP8 vs BF16 weights. Is there any chance you could also provide the BF16 weights?

Thanks!

yea, why no bf16 weights if real open model

it was likely trained in fp8 only

Essential for mac compatibility

Would appreciate bf16, thanks.

it was likely trained in fp8 only

nowhere in the documents stated this, any proofs? if it is real native fp8 training, they should have bragged about that already

Hi there, thanks for publishing this great model!

I’ve experienced significant quality loss using FP8 vs BF16 weights. Is there any chance you could also provide the BF16 weights?

Thanks!

Just out of curiosity where and how exactly did you test the BF16 weights ? I've not seen them being published anywhere, any of the ones that are available are simply dequantized from FP8.

BF16 would be great because then i could convert to int8 for my 3090

Funny how not a single comment on this repo has received any sort of response from the official team. Either they're just watching and actually working on the stuff we're requesting or they clearly don't give a fk, I don't mean to sound rude but the entire community is excited about a model you release and have some requests or queries, the least you could do is reply.

Another vote for BF16 weights here.

Sign up or log in to comment