Apache TVM creator Tianqi Chen releases a free online book on modern GPU programming for MLSys

We taught a brand-new mini-series this year at @SCSatCMU on Modern GPU Programming for ML Systems, as part of the ML Systems course, touching on fun questions like what data layout swizzling is, how to use 3D TMA, and state-of-the-art Blackwell programming. We released a curated online book based on the materials: https://mlc.ai/modern-gpu-programming-for-mlsys/ check it out

@levidiamode @SCSatCMU We didn’t manage to have pub recording this time, hopefully the curated materials and interactive demos helps :)

@tqchenml @SCSatCMU amazing work! any chance you're gonna publish the lecture recordings as well?

@tqchenml @Soul0Engineer @SCSatCMU This is quite incredible thanks @tqchenml

@tqchenml @SCSatCMU awesome

@tqchenml @SCSatCMU sounds awesome, love the focus on practical gpu techniques

@tqchenml @SCSatCMU Wow , awesome :)

@tqchenml @SCSatCMU Beautiful

@tqchenml @SCSatCMU Thanks @tqchenml Any complementary materials for this course?

The rare curriculum that teaches across the boundary. Most ML engineers treat the kernel as a black box, and most systems courses stop at the API. The engineers who can reason from swizzling up to the scheduler are exactly who you want when the workload shape shifts underneath you.

@tqchenml @algo_diver @SCSatCMU Thanks a lot for this 🙏🏻

@tqchenml @SCSatCMU that's awesome, but i think we're still missing the most important part: why are gpu programmers getting more attention in the ml space? what does it say about the state of our industry that we're shifting focus from algo dev to compute optimization?

@tqchenml @levidiamode @SCSatCMU I wish I could attend your seminars and lab meetings remotely. There's so much info relevant to my daily job that will become easy to just naturally acquire by listening in on those conversations.