John Carmack shares PyTorch squared distance code

This trivial thing is probably one of the most important memes in all of machine learning btw

All of the kernel methods are based on this simple trick

@ID_AA_Carmack yeah and if they are normalized, then dot vs square dist are equivalent, which is a pretty neat party trick.

@ID_AA_Carmack This is also the reason why cosine similarity on normalized embeddings is pretty much equivalent to euclidean distance

Same. I stopped doing math at multiplication tables. But physics held my attention. Once Google hit I was game dev building constantly which led me to mocap and epic and then ue state machines back in the day. Now I'm learning math through working with my agents and building with ai. Weird cycle.

@ID_AA_Carmack AI is becoming better and better at optimizing code, probably because of good pattern recognition.

@ID_AA_Carmack Yeah this is an important optimization for kNN lookups

@ID_AA_Carmack There are always things that you miss and require more time to occur to you. So the LLM helps you save significant time here. However, I wonder whether it is not detrimental to your capacity to recall such things... 🤔 But then it might not be an issue, depends on what you desire

@ID_AA_Carmack @grok sea? Math is humbling again.

@ID_AA_Carmack the stuff that should be obvious always hits you 3 days later in the shower. happens with code patterns too

@ID_AA_Carmack if u want to calculate errors, pytorch has more optimized ones ready to be used…?

@ID_AA_Carmack Be honest, Which tool is best for coding?

@ID_AA_Carmack Flashback to Paul Rutherford, thx

@ID_AA_Carmack Getting curious when seeing your chat. What are the main ways you use LLMs? Chat, code generation, auto complete, review?

@ID_AA_Carmack

@ID_AA_Carmack just like us fr fr

@ID_AA_Carmack K_norms = K_norms - 0x5f3759df // what the fuck?

@ID_AA_Carmack *Ageist joke placeholder*

@ID_AA_Carmack Things can get weird in PyTorch.

@ID_AA_Carmack what are you working on these days?

@ID_AA_Carmack not 16 anymore