Users are interested in the book of 50 hands-on LLM projects because it builds real ML engineering intuition by exploring raw mechanics like logit lens.
Based on 2 visible X reactions from 3 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@svpino This is how you bridge the gap between an API wrapper dev and a real ML Engineer. Digging into the raw mechanics—logit lens, activation patching, and hidden state distributions using core libraries like NumPy and PyTorch—is where real intuition is built. Saved.
@svpino getting a copy
How to become GOD-LEVEL with Large Language Models. Here are 50 hands-on projects with solutions that will teach you how Large Language Models work. You don't need to solve all 50, but if you do, you'll be at the top 0.01% of the field. It's all Python + Pytorch + SciKit-Learn + Pandas + Numpy + Matplotlib + Seaborn. Here are the 50 problems from the book (link below): Tokenization 1. Three tokenization schemes 2. Book lengths in characters, words, and tokens 3. Pandas frequency tables of token lengths 4. Token lengths in characters and bytes 5. Is tokenization compression? 6. Tokenization and compression in different languages 7. Translating between tokenizers Embeddings 8. Distribution of cosine similarities 9. Sequential cosine similarity 10. Sequential number cosine similarity 11. Network graphs of cosine similarities 12. RSA to compare GPT-2 & BERT embeddings 13. Word similarity via distance and cosine 14. Linear semantic axes 15. Analogy vectors Output logits 16. Softmax probability distributions 17. Probabilistic token selection 18. Token prediction accuracy 19. LLM loss function 20. Perplexity over sequences, texts, and models 21. Predict token position with linear and logistic regressions 22. Evaluating models with HellaSwag 23. Measuring language biases Transformer outputs 24. Cosine similarities within and across layers 25. Category selectivity via cosine similarity 26. Current layer = previous layer + adjustments 27. Impact of layer-specific noise and scaling 28. Effective dimensionality of hidden layers 29. Hidden state dimensionality reduction 30. Sentiment analysis with decision trees 31. Logit lens 32. Patching hidden states in indirect object identification Attention 33. QKV weights characteristics 34. QKV activation characteristics 35. Raw and softmax attention scores 36. Characteristics of attention adjustment magnitudes 37. Token prediction and attention KL divergences 38. Laminar profile of RSA and category selectivity 39. Token frequency, attention adjustments, QK^T 40. Downstream impacts of head silencing 41. Patching heads in IOI MLP 42. MLP weights and activations characteristics 43. Characterizing the MLP progression 44. Grammar tuning in MLP projections 45. Minkowski distance, mutual information, and token positions 46. Statistics-based lesioning in MLP neurons 47. Supervised probing with XGBoost 48. "Can" vs. "can't" classification via logistic regression 49. Successive median-replacement of MLP activations 50. Recommender systems with MLP projections Book link below.
50 ML projects to understand LLMs — Investigate transformer mechanisms through data analysis, visualization, & experimentation: http://amzn.to/3P8ztDt v/ @PacktDataML This book perfectly reflects its title "50 ML Projects to Understand LLMs". The entire book consists of exactly that: 50 projects, with tasks and subtasks pleasantly outlined, explained, and presented in a beautifully instructive and consistent style. No extraneous pedagogical content. The learning experience in each project delivers the promised understanding of LLM concepts, including informative graphics and detailed elaboration on the project's subtasks as needed. This book is essential for anyone seeking to build LLMs with confidence, knowledge, and skill.
The guide uses PyTorch and pandas to analyze internal model mechanisms.
@svpino This is how you bridge the gap between an API wrapper dev and a real ML Engineer. Digging into the raw mechanics—logit lens, activation patching, and hidden state distributions using core libraries like NumPy and PyTorch—is where real intuition is built. Saved.
How to become GOD-LEVEL with Large Language Models. Here are 50 hands-on projects with solutions that will teach you how Large Language Models work. You don't need to solve all 50, but if you do, you'll be at the top 0.01% of the field. It's all Python + Pytorch + SciKit-Learn + Pandas + Numpy + Matplotlib + Seaborn. Here are the 50 problems from the book (link below): Tokenization 1. Three tokenization schemes 2. Book lengths in characters, words, and tokens 3. Pandas frequency tables of token lengths 4. Token lengths in characters and bytes 5. Is tokenization compression? 6. Tokenization and compression in different languages 7. Translating between tokenizers Embeddings 8. Distribution of cosine similarities 9. Sequential cosine similarity 10. Sequential number cosine similarity 11. Network graphs of cosine similarities 12. RSA to compare GPT-2 & BERT embeddings 13. Word similarity via distance and cosine 14. Linear semantic axes 15. Analogy vectors Output logits 16. Softmax probability distributions 17. Probabilistic token selection 18. Token prediction accuracy 19. LLM loss function 20. Perplexity over sequences, texts, and models 21. Predict token position with linear and logistic regressions 22. Evaluating models with HellaSwag 23. Measuring language biases Transformer outputs 24. Cosine similarities within and across layers 25. Category selectivity via cosine similarity 26. Current layer = previous layer + adjustments 27. Impact of layer-specific noise and scaling 28. Effective dimensionality of hidden layers 29. Hidden state dimensionality reduction 30. Sentiment analysis with decision trees 31. Logit lens 32. Patching hidden states in indirect object identification Attention 33. QKV weights characteristics 34. QKV activation characteristics 35. Raw and softmax attention scores 36. Characteristics of attention adjustment magnitudes 37. Token prediction and attention KL divergences 38. Laminar profile of RSA and category selectivity 39. Token frequency, attention adjustments, QK^T 40. Downstream impacts of head silencing 41. Patching heads in IOI MLP 42. MLP weights and activations characteristics 43. Characterizing the MLP progression 44. Grammar tuning in MLP projections 45. Minkowski distance, mutual information, and token positions 46. Statistics-based lesioning in MLP neurons 47. Supervised probing with XGBoost 48. "Can" vs. "can't" classification via logistic regression 49. Successive median-replacement of MLP activations 50. Recommender systems with MLP projections Book link below.
50 ML projects to understand LLMs — Investigate transformer mechanisms through data analysis, visualization, & experimentation: http://amzn.to/3P8ztDt v/ @PacktDataML This book perfectly reflects its title "50 ML Projects to Understand LLMs". The entire book consists of exactly that: 50 projects, with tasks and subtasks pleasantly outlined, explained, and presented in a beautifully instructive and consistent style. No extraneous pedagogical content. The learning experience in each project delivers the promised understanding of LLM concepts, including informative graphics and detailed elaboration on the project's subtasks as needed. This book is essential for anyone seeking to build LLMs with confidence, knowledge, and skill.
Users are interested in the book of 50 hands-on LLM projects because it builds real ML engineering intuition by exploring raw mechanics like logit lens.
Based on 2 visible X reactions from 3 accounts; directional sample.
Ask a question below.
Published answers will appear here.