/Tech1d ago

Developer @levidiamode breaks the two-millisecond threshold in the GPU Mode QR factorization challenge

Top submissions leverage NVFP4 low-precision quantization.

1415237429.1K

#102

Original post

levi@levidiamode

171/365 of GPU Programming

Finally got below 2ms on the GPU Mode QR challenge. Still quite far away from CUDA colonel @blelbach, so I'm excited to read everyones' writeups once the competition is over.

I especially wonder how #1 is utilizing NVFP4 (or is the submission name a false flag?). Every lower precision attempt has not panned out for me so far unfortunately.

Very curious where the top 3 will converge in a week from now...

If you're still debating whether to participate, would highly recommend! You learn so much from just trying stuff and being in the discord. And @modal gives you $30 in monthly compute for free. Maybe we can get a bit more compute for the next challenge if we all ask @charles_irl nicely 😁😁

levi@levidiamode

170/365 of GPU Programming

Just going through the motions of QR factorization and Gram Schmidt today while concepts around orthogonality, norms and projections.

It's been a while since I studied Linear Algebra, so it's been fun reviewing some older concepts and learning new ones as well.

Will spend some time learning more about Cholesky and Householder as well tomorrow.

6:58 AM · Jun 23, 2026 · 16.1K Views

Sentiment

Positive users celebrate the sub-2ms GPU QR factorization achievement and upcoming fp4 insights, while negative users express frustration over their own 3ms times and inability to use nvfp4.

Pos

50.0%

Neg

50.0%

4 comments with sentiment.

Cluster Engagement

Digg Deeper

No Digg Deeper questions have been answered for this story yet.

Posts from X

Most Activity

VIEWS14.4KBOOKMARKS25LIKES66RETWEETS2REPLIES3

rohan anil@_arohan_

Holy moly guacamole 🥑

Nvfp4 brah

Bryce, the CUDA Colonel@blelbach

nvfp4 in the #1 submission is clearly a reference for a tiny quantized NVFP4 model that does QR factorization!

1d14.4K6625

datavorous@datavorous_

@levidiamode @blelbach @GPU_MODE same question about nvfp4, couldn't utilize it :(

1d22721

levi@levidiamode

@blelbach @GPU_MODE Link to the challenge: https://www.gpumode.com/news/linear-algebra-kernels-age-of-research

1d2516

sankalp@dejavucoder

@levidiamode @blelbach @GPU_MODE you are mogging

1d1762

levi@levidiamode

@dejavucoder @blelbach @GPU_MODE sadly not even close to top 3 😁

1d1081

levi@levidiamode

@piratedmango @blelbach @GPU_MODE pretty sure almost everyone is using AI at least partially

1d601

sankalp@dejavucoder

@levidiamode @blelbach @GPU_MODE v tough competition. i wonder if they have fable access apart from ofc domain knowledge hehe

1d591

Semblance@piratedmango

@levidiamode @blelbach @GPU_MODE the context being i am getting 3ms ,the fuck is going on? are u guys using AI ,bit of a dilemma ?

1d82

Semblance@piratedmango

@levidiamode @blelbach @GPU_MODE so here is my question ,wont this put you on a loop for trying and fixing one solution after the next ,idk it feels like that to me ,or are you guys giving it a step by step instructions ..

1d21

levi@levidiamode

@datavorous_ @blelbach @GPU_MODE one more week and we'll find out the magic fp4 secrets 😄

1d821

levi@levidiamode

@dejavucoder @blelbach @GPU_MODE doubt it tbh. probably really good harness + maybe domain knowledge in precision formats/linear algebra/etc

1d401

Semblance@piratedmango

@levidiamode @blelbach @GPU_MODE back to basics 🤖

1d22