QAD nvfp4
#1
by phazei - opened
The fast model says it was optimized for QAD nvfp4. Would you mind making a version like that for us blackwellers to try out?
both done :)
was waiting to get back to the 5090 to test them before uploading
Thank you for NVFP4 versions!
On RTX6000PRO, I'm getting 4.7 seconds with Fast NVFP4 and 2 seconds with Instant. Quality of instant is not even comparable to me against the Fast one. Fast vs BF16, BF16 contains way more detail but definitely Fast output is acceptable.
My GPU is a 4070 Ti. I compared INT8 and NVFP4: NVFP4 provides more details and higher model accuracy, while INT8 inference runs nearly twice as fast as NVFP4.