My solutions

PLANNED.toml

@@ -46,11 +46,6 @@ id = "v1-35"
 title = "Implement a Variational Autoencoder"
 difficulty = "hard"
 
-[[planned]]
-id = "v2-1"
-title = "Implement KL Divergence Loss"
-difficulty = "easy"
-
 [[planned]]
 id = "v2-4"
 title = "Create a RAG Search of Embeddings"
@@ -60,8 +55,3 @@ difficulty = "medium"
 id = "v2-13"
 title = "Implement Quantization (GPTQ)"
 difficulty = "hard"
-
-[[planned]]
-id = "v2-20"
-title = "Apply SFT on SmolLM"
-difficulty = "hard"

PROBLEMS.md

@@ -1,15 +1,19 @@
-# All 68 problems
+# All 74 problems
 
 Generated from the per-problem `problem.toml` manifests — do not edit by hand.
 Run `python3 scripts/generate.py readme` to refresh.
 
-**63 of 68 are auto-graded** (`pip install torchleet`). A ✅ in the
+**65 of 74 are auto-graded** (`pip install torchleet`). A ✅ in the
 Graded column means `check()` will verify your solution.
 
-### Basics (23)
+### Basics (27)
 
 | Problem | Difficulty | Companies | Graded | Run | Links |
 |---|---|---|---|---|---|
+| 💡 Implement a Shuffle Buffer for Streaming Data | medium |  |  | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/data-loading/buffered-loading/buffered-loading.ipynb) | [Q](v3/data-loading/buffered-loading/buffered-loading.ipynb) / [S](v3/data-loading/buffered-loading/buffered-loading_SOLN.ipynb) |
+| ⭐ Simulate DistributedSampler for Data Parallel Training | medium | Meta, Google, NVIDIA, Anthropic |  | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/data-loading/data-parallel-loading/data-parallel-loading.ipynb) | [Q](v3/data-loading/data-parallel-loading/data-parallel-loading.ipynb) / [S](v3/data-loading/data-parallel-loading/data-parallel-loading_SOLN.ipynb) |
+| ⭐ Implement a Preloaded Map-Style Dataset | easy |  |  | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/data-loading/preloaded-dataset/preloaded-dataset.ipynb) | [Q](v3/data-loading/preloaded-dataset/preloaded-dataset.ipynb) / [S](v3/data-loading/preloaded-dataset/preloaded-dataset_SOLN.ipynb) |
+| ⭐ Implement a Streaming IterableDataset with Yield | medium |  |  | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/data-loading/streaming-dataset/streaming-dataset.ipynb) | [Q](v3/data-loading/streaming-dataset/streaming-dataset.ipynb) / [S](v3/data-loading/streaming-dataset/streaming-dataset_SOLN.ipynb) |
 | 💡 Implement Linear Regression | basic |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/torch/basic/lin-regression/lin-regression.ipynb) | [Q](torch/basic/lin-regression/lin-regression.ipynb) / [S](torch/basic/lin-regression/lin-regression_SOLN.ipynb) |
 | 💡 Write a Custom Dataset and DataLoader | basic |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/torch/basic/custom-dataset/custom-dataset.ipynb) | [Q](torch/basic/custom-dataset/custom-dataset.ipynb) / [S](torch/basic/custom-dataset/custom-dataset_SOLN.ipynb) |
 | 💡 Write a Custom Activation Function | basic |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/torch/basic/custom-activation/custom-activation.ipynb) | [Q](torch/basic/custom-activation/custom-activation.ipynb) / [S](torch/basic/custom-activation/custom-activation_SOLN.ipynb) |
@@ -34,10 +38,11 @@ Graded column means `check()` will verify your solution.
 | ⭐ Implement KNN in PyTorch | easy | Uber, LinkedIn, Meta | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/classical-ml/knn/knn.ipynb) | [Q](v3/classical-ml/knn/knn.ipynb) / [S](v3/classical-ml/knn/knn_SOLN.ipynb) |
 | ⭐ Implement Logistic Regression with Gradient Descent | easy | Google, Meta, Amazon | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/classical-ml/logistic-regression/logistic-regression.ipynb) | [Q](v3/classical-ml/logistic-regression/logistic-regression.ipynb) / [S](v3/classical-ml/logistic-regression/logistic-regression_SOLN.ipynb) |
 
-### LLM Learning Path (21)
+### LLM Learning Path (23)
 
 | Problem | Difficulty | Companies | Graded | Run | Links |
 |---|---|---|---|---|---|
+|  Implement KL Divergence Loss | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/alignment-training/kl-divergence/kl-divergence.ipynb) | [Q](v3/alignment-training/kl-divergence/kl-divergence.ipynb) / [S](v3/alignment-training/kl-divergence/kl-divergence_SOLN.ipynb) |
 | 💡 Implement RMS Norm | easy |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/RMS-Norm/rms-norm.ipynb) | [Q](llm/RMS-Norm/rms-norm.ipynb) / [S](llm/RMS-Norm/rms-norm_SOLN.ipynb) |
 | 💡 Implement Byte Pair Encoding from Scratch | easy |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Byte-Pair-Encoder/BPE-q3.ipynb) | [Q](llm/Byte-Pair-Encoder/BPE-q3.ipynb) / [S](llm/Byte-Pair-Encoder/BPE-q3-SOLN.ipynb) |
 | 💡 Implement Attention from Scratch | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Implement-Attention-from-Scratch/attention-q4-Question.ipynb) | [Q](llm/Implement-Attention-from-Scratch/attention-q4-Question.ipynb) / [S](llm/Implement-Attention-from-Scratch/attention-q4.ipynb) |
@@ -46,6 +51,7 @@ Graded column means `check()` will verify your solution.
 | 💡 Implement Sinusoidal Embeddings | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Sinusoidal-Positional-Embedding/sinusoidal-q7-Question.ipynb) | [Q](llm/Sinusoidal-Positional-Embedding/sinusoidal-q7-Question.ipynb) / [S](llm/Sinusoidal-Positional-Embedding/sinusoidal-q7.ipynb) |
 | 💡 Implement ROPE Embeddings | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Rotary-Positional-Embedding/rope-q8-Question.ipynb) | [Q](llm/Rotary-Positional-Embedding/rope-q8-Question.ipynb) / [S](llm/Rotary-Positional-Embedding/rope-q8.ipynb) |
 | 💡 Implement SmolLM from Scratch | hard |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/SmolLM/smollm-q12-Question.ipynb) | [Q](llm/SmolLM/smollm-q12-Question.ipynb) / [S](llm/SmolLM/smollm-q12.ipynb) |
+|  Apply SFT on SmolLM | hard |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/alignment-training/sft-smollm/sft-smollm.ipynb) | [Q](v3/alignment-training/sft-smollm/sft-smollm.ipynb) / [S](v3/alignment-training/sft-smollm/sft-smollm_SOLN.ipynb) |
 | 🔥 Implement Top-p (Nucleus) Sampling | medium | Anthropic, OpenAI, DeepMind, Perplexity, Cohere | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/llm-decoding/top-p-sampling/top-p-sampling.ipynb) | [Q](v3/llm-decoding/top-p-sampling/top-p-sampling.ipynb) / [S](v3/llm-decoding/top-p-sampling/top-p-sampling_SOLN.ipynb) |
 | 🔥 Implement Top-k Sampling | medium | Anthropic, OpenAI, DeepMind, Cohere | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/llm-decoding/top-k-sampling/top-k-sampling.ipynb) | [Q](v3/llm-decoding/top-k-sampling/top-k-sampling.ipynb) / [S](v3/llm-decoding/top-k-sampling/top-k-sampling_SOLN.ipynb) |
 | 🔥 Implement Temperature Sampling | easy | OpenAI, Anthropic, Cohere, Perplexity | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/llm-decoding/temperature-sampling/temperature-sampling.ipynb) | [Q](v3/llm-decoding/temperature-sampling/temperature-sampling.ipynb) / [S](v3/llm-decoding/temperature-sampling/temperature-sampling_SOLN.ipynb) |
@@ -60,7 +66,7 @@ Graded column means `check()` will verify your solution.
 | ⭐ Implement GRPO (DeepSeek-R1 Algorithm) | expert | DeepMind, Anthropic, OpenAI | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/alignment-training/grpo/grpo.ipynb) | [Q](v3/alignment-training/grpo/grpo.ipynb) / [S](v3/alignment-training/grpo/grpo_SOLN.ipynb) |
 | 🔥 Build a Complete LLM Inference Engine | expert | Perplexity, Together AI, Anyscale, Fireworks AI | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/llm-inference/inference-engine/inference-engine.ipynb) | [Q](v3/llm-inference/inference-engine/inference-engine.ipynb) / [S](v3/llm-inference/inference-engine/inference-engine_SOLN.ipynb) |
 
-### Advanced (42)
+### Advanced (44)
 
 | Problem | Difficulty | Companies | Graded | Run | Links |
 |---|---|---|---|---|---|
@@ -76,12 +82,14 @@ Graded column means `check()` will verify your solution.
 | ⭐ Implement Muon from Scratch | hard | Meta, Google | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/torch/hard/optimizers/muon/muon.ipynb) | [Q](torch/hard/optimizers/muon/muon.ipynb) / [S](torch/hard/optimizers/muon/muon_SOLN.ipynb) |
 | 💡 Create Embeddings Out Of An Llm | medium |  |  | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Create-Embeddings-out-of-an-LLM/embeddings-q2.ipynb) | [Q](llm/Create-Embeddings-out-of-an-LLM/embeddings-q2.ipynb) |
 | 💡 Flash Attention | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Flash-Attention/flash-attention.ipynb) | [Q](llm/Flash-Attention/flash-attention.ipynb) |
+|  Implement KL Divergence Loss | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/alignment-training/kl-divergence/kl-divergence.ipynb) | [Q](v3/alignment-training/kl-divergence/kl-divergence.ipynb) / [S](v3/alignment-training/kl-divergence/kl-divergence_SOLN.ipynb) |
 | 💡 Implement Attention from Scratch | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Implement-Attention-from-Scratch/attention-q4-Question.ipynb) | [Q](llm/Implement-Attention-from-Scratch/attention-q4-Question.ipynb) / [S](llm/Implement-Attention-from-Scratch/attention-q4.ipynb) |
 | 💡 Implement Multi-Head Attention from Scratch | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Multi-Head-Attention/multi-head-attention-q5-Question.ipynb) | [Q](llm/Multi-Head-Attention/multi-head-attention-q5-Question.ipynb) / [S](llm/Multi-Head-Attention/multi-head-attention-q5.ipynb) |
 | 💡 Implement Grouped Query Attention from Scratch | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Grouped-Query-Attention/grouped-query-attention-Question.ipynb) | [Q](llm/Grouped-Query-Attention/grouped-query-attention-Question.ipynb) / [S](llm/Grouped-Query-Attention/grouped-query-attention.ipynb) |
 | 💡 Implement Sinusoidal Embeddings | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Sinusoidal-Positional-Embedding/sinusoidal-q7-Question.ipynb) | [Q](llm/Sinusoidal-Positional-Embedding/sinusoidal-q7-Question.ipynb) / [S](llm/Sinusoidal-Positional-Embedding/sinusoidal-q7.ipynb) |
 | 💡 Implement ROPE Embeddings | medium |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/Rotary-Positional-Embedding/rope-q8-Question.ipynb) | [Q](llm/Rotary-Positional-Embedding/rope-q8-Question.ipynb) / [S](llm/Rotary-Positional-Embedding/rope-q8.ipynb) |
 | 💡 Implement SmolLM from Scratch | hard |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/llm/SmolLM/smollm-q12-Question.ipynb) | [Q](llm/SmolLM/smollm-q12-Question.ipynb) / [S](llm/SmolLM/smollm-q12.ipynb) |
+|  Apply SFT on SmolLM | hard |  | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/alignment-training/sft-smollm/sft-smollm.ipynb) | [Q](v3/alignment-training/sft-smollm/sft-smollm.ipynb) / [S](v3/alignment-training/sft-smollm/sft-smollm_SOLN.ipynb) |
 | 🔥 Implement Contrastive Loss (InfoNCE) + CLIP Training Loop | medium | OpenAI, Anthropic, DeepMind, Midjourney, Apple | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/modern-architectures/contrastive-loss-clip/contrastive-loss-clip.ipynb) | [Q](v3/modern-architectures/contrastive-loss-clip/contrastive-loss-clip.ipynb) / [S](v3/modern-architectures/contrastive-loss-clip/contrastive-loss-clip_SOLN.ipynb) |
 | 🔥 Implement 2D Positional Embeddings | medium | Anthropic, DeepMind, Midjourney, Runway | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/modern-architectures/2d-positional-embeddings/2d-positional-embeddings.ipynb) | [Q](v3/modern-architectures/2d-positional-embeddings/2d-positional-embeddings.ipynb) / [S](v3/modern-architectures/2d-positional-embeddings/2d-positional-embeddings_SOLN.ipynb) |
 | 🔥 Implement Beam Search for LLM Decoding | medium | Google, DeepMind, Meta, Apple | ✅ | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Exorust/TorchLeet/blob/main/v3/llm-decoding/beam-search/beam-search.ipynb) | [Q](v3/llm-decoding/beam-search/beam-search.ipynb) / [S](v3/llm-decoding/beam-search/beam-search_SOLN.ipynb) |

README.md

@@ -2,7 +2,7 @@
 
 <img src="torchleet-header.png" alt="TorchLeet" width="700">
 
-**68 PyTorch problems from real ML/AI interviews at Google, Meta, Anthropic, and more.**
+**74 PyTorch problems from real ML/AI interviews at Google, Meta, Anthropic, and more.**
 
 [![GitHub stars](https://img.shields.io/github/stars/Exorust/TorchLeet?style=social)](https://github.com/Exorust/TorchLeet)
 [![Website](https://img.shields.io/badge/website-torch--leet.vercel.app-8b5cf6)](https://torch-leet.vercel.app)
@@ -20,7 +20,7 @@ I struggled to grind for ML/AI interviews so I went back to the basics and creat
 
 ## AI Tutor (NEW)
 
-Turn any AI assistant into your PyTorch interview coach. The TorchLeet MCP server gives your AI access to all 68 problems, progressive hints, company prep plans, and learning paths, while enforcing a no-spoilers teaching style.
+Turn any AI assistant into your PyTorch interview coach. The TorchLeet MCP server gives your AI access to all 74 problems, progressive hints, company prep plans, and learning paths, while enforcing a no-spoilers teaching style.
 
 ```bash
 # Clone the repo first
@@ -66,7 +66,7 @@ Add this to your MCP config:
 
 ---
 
-68 problems across three tracks:
+74 problems across three tracks:
 
 | Track | Focus | Questions |
 |-------|-------|-----------|
@@ -121,9 +121,9 @@ Build an LLM from scratch, one question at a time. Recommended order:
 ### 4. Alignment & Fine-Tuning
 | Problem | Companies | Links |
 |---------|-----------|-------|
-| Implement KL Divergence Loss | | |
+| Implement KL Divergence Loss | | [Q](v3/alignment-training/kl-divergence/kl-divergence.ipynb) / [S](v3/alignment-training/kl-divergence/kl-divergence_SOLN.ipynb) |
 | Implement LoRA | Meta, Google, Anthropic, OpenAI | [Q](v3/alignment-training/lora/lora.ipynb) / [S](v3/alignment-training/lora/lora_SOLN.ipynb) |
-| Apply SFT on SmolLM | | |
+| Apply SFT on SmolLM | | [Q](v3/alignment-training/sft-smollm/sft-smollm.ipynb) / [S](v3/alignment-training/sft-smollm/sft-smollm_SOLN.ipynb) |
 | Implement DPO Loss | Anthropic, OpenAI, DeepMind, Meta | [Q](v3/alignment-training/dpo/dpo.ipynb) / [S](v3/alignment-training/dpo/dpo_SOLN.ipynb) |
 | Implement PPO for RLHF | Anthropic, OpenAI, DeepMind, Meta | [Q](v3/alignment-training/ppo-rlhf/ppo-rlhf.ipynb) / [S](v3/alignment-training/ppo-rlhf/ppo-rlhf_SOLN.ipynb) |
 | Implement GRPO (DeepSeek-R1) | DeepMind, Anthropic, OpenAI | [Q](v3/alignment-training/grpo/grpo.ipynb) / [S](v3/alignment-training/grpo/grpo_SOLN.ipynb) |
@@ -207,12 +207,13 @@ Company-tagged questions from real ML/AI interviews. Sorted by topic.
 ### Alignment & Training
 | Problem | Difficulty | Companies | Links |
 |---------|------------|-----------|-------|
+| Implement KL Divergence Loss | Medium | | [Q](v3/alignment-training/kl-divergence/kl-divergence.ipynb) / [S](v3/alignment-training/kl-divergence/kl-divergence_SOLN.ipynb) |
 | Implement LoRA | Medium | Meta, Google, Anthropic, OpenAI, Databricks | [Q](v3/alignment-training/lora/lora.ipynb) / [S](v3/alignment-training/lora/lora_SOLN.ipynb) |
 | Implement DPO Loss | Hard | Anthropic, OpenAI, DeepMind, Meta | [Q](v3/alignment-training/dpo/dpo.ipynb) / [S](v3/alignment-training/dpo/dpo_SOLN.ipynb) |
 | Implement PPO for RLHF | Hard | Anthropic, OpenAI, DeepMind, Meta | [Q](v3/alignment-training/ppo-rlhf/ppo-rlhf.ipynb) / [S](v3/alignment-training/ppo-rlhf/ppo-rlhf_SOLN.ipynb) |
 | Gradient Checkpointing | Hard | Meta, Google, NVIDIA, Tesla | [Q](v3/alignment-training/gradient-checkpointing/gradient-checkpointing.ipynb) / [S](v3/alignment-training/gradient-checkpointing/gradient-checkpointing_SOLN.ipynb) |
 | Implement GRPO (DeepSeek-R1) | Expert | DeepMind, Anthropic, OpenAI | [Q](v3/alignment-training/grpo/grpo.ipynb) / [S](v3/alignment-training/grpo/grpo_SOLN.ipynb) |
-| Apply SFT on SmolLM | Hard | | |
+| Apply SFT on SmolLM | Hard | | [Q](v3/alignment-training/sft-smollm/sft-smollm.ipynb) / [S](v3/alignment-training/sft-smollm/sft-smollm_SOLN.ipynb) |
 
 ### LLM Inference & Systems
 | Problem | Difficulty | Companies | Links |