PyTorch maintainer and GPU MODE co-founder Mark Saroufim launches a challenge to optimize batched QR decomposition on Nvidia B200 GPUs

Just to be clear: submissions can be human + coding agents (.codex or claude) or harness over multiple models. You can access compute via modal.

If you are into linear algebra, this is a flex at designing hardware native algos (like Flash did Flash Attention)

If you are a kernels engineer, you can test if you can generalize to a different algorithm than MoE or Attention that demands different types of performance work.

Regardless, this is a very hard challenge.

This is F*cking awesome!

@marksaroufim Must have assumed that QR is not frontier level research 😃

😱

I swear I read something about Jt’s hair strand on corp

@marksaroufim @srush_nlp What version of kernels did you use a baseline for @pytorch, these were all vanilla Magma kernels until a few months ago lol.

@_arohan_ yes that's when jt announced that the prize will be strands of his hair

@MillionInt Locks of Loving Grace

@MillionInt @marksaroufim It’s been pretty useful with TPU kernels ime since they shifted from silent sabotage to guardrails.

Failure mode seems to be when it is faced with entire codebases instead of snippets.

@SkyLi0n @srush_nlp @PyTorch Baseline is 3.12 here

Linear Algebra Kernels For The Age Of Research

In other words, GPUs can do more than just matmuls.

Can you make it fast?

@MillionInt its even more rare if you're balding