Generated from the per-problem problem.toml manifests — do not edit by hand.
Run python3 scripts/generate.py readme to refresh.
63 of 68 are auto-graded (pip install torchleet). A ✅ in the
Graded column means check() will verify your solution.
| Problem | Difficulty | Companies | Graded | Run | Links |
|---|---|---|---|---|---|
| 💡 Implement Linear Regression | basic | ✅ | Q / S | ||
| 💡 Write a Custom Dataset and DataLoader | basic | ✅ | Q / S | ||
| 💡 Write a Custom Activation Function | basic | ✅ | Q / S | ||
| 💡 Implement Custom Loss Function (Huber Loss) | basic | ✅ | Q / S | ||
| 💡 Implement a Deep Neural Network | basic | ✅ | Q / S | ||
| 💡 Visualize Training with TensorBoard | basic | ✅ | Q / S | ||
| 💡 Save and Load PyTorch Model | basic | Q / S | |||
| 💡 Implement a CNN on CIFAR-10 | easy | ✅ | Q / S | ||
| 💡 Implement an RNN from Scratch | easy | ✅ | Q / S | ||
| 💡 Data Augmentation with torchvision.transforms | easy | Q / S | |||
| 💡 Add Benchmarking to PyTorch Code | easy | ✅ | Q / S | ||
| 💡 Train an Autoencoder for Anomaly Detection | easy | ✅ | Q / S | ||
| 💡 Quantize Your Language Model | easy | ✅ | Q / S | ||
| 💡 Mixed Precision Training with torch.cuda.amp | easy | Q / S | |||
| 💡 CNN Parameter Initialization | medium | ✅ | Q / S | ||
| 💡 Implement a CNN from Scratch | medium | ✅ | Q / S | ||
| 💡 Implement an LSTM from Scratch | medium | ✅ | Q / S | ||
| 💡 Implement RMS Norm | easy | ✅ | Q / S | ||
| 💡 Implement Byte Pair Encoding from Scratch | easy | ✅ | Q / S | ||
| 🔥 Implement Softmax from Scratch | easy | Apple, Meta, Google, Amazon | ✅ | Q / S | |
| 🔥 Implement K-Means Clustering in PyTorch | easy | Uber, LinkedIn, Google, Amazon | ✅ | Q / S | |
| ⭐ Implement KNN in PyTorch | easy | Uber, LinkedIn, Meta | ✅ | Q / S | |
| ⭐ Implement Logistic Regression with Gradient Descent | easy | Google, Meta, Amazon | ✅ | Q / S |
| Problem | Difficulty | Companies | Graded | Run | Links |
|---|---|---|---|---|---|
| 💡 Implement RMS Norm | easy | ✅ | Q / S | ||
| 💡 Implement Byte Pair Encoding from Scratch | easy | ✅ | Q / S | ||
| 💡 Implement Attention from Scratch | medium | ✅ | Q / S | ||
| 💡 Implement Multi-Head Attention from Scratch | medium | ✅ | Q / S | ||
| 💡 Implement Grouped Query Attention from Scratch | medium | ✅ | Q / S | ||
| 💡 Implement Sinusoidal Embeddings | medium | ✅ | Q / S | ||
| 💡 Implement ROPE Embeddings | medium | ✅ | Q / S | ||
| 💡 Implement SmolLM from Scratch | hard | ✅ | Q / S | ||
| 🔥 Implement Top-p (Nucleus) Sampling | medium | Anthropic, OpenAI, DeepMind, Perplexity, Cohere | ✅ | Q / S | |
| 🔥 Implement Top-k Sampling | medium | Anthropic, OpenAI, DeepMind, Cohere | ✅ | Q / S | |
| 🔥 Implement Temperature Sampling | easy | OpenAI, Anthropic, Cohere, Perplexity | ✅ | Q / S | |
| 🔥 Implement LoRA on a Linear Layer | medium | Meta, Google, Anthropic, OpenAI, Databricks | ✅ | Q / S | |
| 🔥 Implement KV Cache for Autoregressive Generation | medium | Anthropic, OpenAI, Meta, Perplexity, Together AI | ✅ | Q / S | |
| 🔥 Implement Sliding Window Attention | medium | Mistral, Anthropic, Google, DeepMind | ✅ | Q / S | |
| 🔥 Implement DPO Loss from Scratch | hard | Anthropic, OpenAI, DeepMind, Meta | ✅ | Q / S | |
| 🔥 Implement PPO for RLHF | hard | Anthropic, OpenAI, DeepMind, Meta | ✅ | Q / S | |
| 🔥 Implement Mixture of Experts Layer | hard | Google, DeepMind, Mistral, Databricks, xAI | ✅ | Q / S | |
| 🔥 Implement Speculative Decoding | hard | Google, DeepMind, Anthropic, Apple | ✅ | Q / S | |
| 🔥 Implement Continuous Batching for LLM Inference | hard | Perplexity, Together AI, Anyscale, Meta | ✅ | Q / S | |
| ⭐ Implement GRPO (DeepSeek-R1 Algorithm) | expert | DeepMind, Anthropic, OpenAI | ✅ | Q / S | |
| 🔥 Build a Complete LLM Inference Engine | expert | Perplexity, Together AI, Anyscale, Fireworks AI | ✅ | Q / S |
| Problem | Difficulty | Companies | Graded | Run | Links |
|---|---|---|---|---|---|
| 💡 Implement AlexNet from Scratch | medium | ✅ | Q / S | ||
| 💡 Train a 3D CNN for CT Image Segmentation | medium | ✅ | Q / S | ||
| 💡 Write a Custom Autograd Function (SILU) | hard | ✅ | Q / S | ||
| 💡 Write a Transformer from Scratch | hard | ✅ | Q / S | ||
| 💡 Write a GAN | hard | ✅ | Q / S | ||
| 💡 Sequence-to-Sequence with Attention | hard | ✅ | Q / S | ||
| 💡 Explainable AI (GradCAM/SHAP) | hard | Q / S | |||
| ⭐ Implement Adam from Scratch | hard | Meta, Google | ✅ | Q / S | |
| ⭐ Implement AdamW from Scratch | hard | Meta, Google | ✅ | Q / S | |
| ⭐ Implement Muon from Scratch | hard | Meta, Google | ✅ | Q / S | |
| 💡 Create Embeddings Out Of An Llm | medium | Q | |||
| 💡 Flash Attention | medium | ✅ | Q | ||
| 💡 Implement Attention from Scratch | medium | ✅ | Q / S | ||
| 💡 Implement Multi-Head Attention from Scratch | medium | ✅ | Q / S | ||
| 💡 Implement Grouped Query Attention from Scratch | medium | ✅ | Q / S | ||
| 💡 Implement Sinusoidal Embeddings | medium | ✅ | Q / S | ||
| 💡 Implement ROPE Embeddings | medium | ✅ | Q / S | ||
| 💡 Implement SmolLM from Scratch | hard | ✅ | Q / S | ||
| 🔥 Implement Contrastive Loss (InfoNCE) + CLIP Training Loop | medium | OpenAI, Anthropic, DeepMind, Midjourney, Apple | ✅ | Q / S | |
| 🔥 Implement 2D Positional Embeddings | medium | Anthropic, DeepMind, Midjourney, Runway | ✅ | Q / S | |
| 🔥 Implement Beam Search for LLM Decoding | medium | Google, DeepMind, Meta, Apple | ✅ | Q / S | |
| 🔥 Implement LoRA on a Linear Layer | medium | Meta, Google, Anthropic, OpenAI, Databricks | ✅ | Q / S | |
| 🔥 Implement KV Cache for Autoregressive Generation | medium | Anthropic, OpenAI, Meta, Perplexity, Together AI | ✅ | Q / S | |
| 🔥 Implement Sliding Window Attention | medium | Mistral, Anthropic, Google, DeepMind | ✅ | Q / S | |
| 🔥 Implement DPO Loss from Scratch | hard | Anthropic, OpenAI, DeepMind, Meta | ✅ | Q / S | |
| 🔥 Implement PPO for RLHF | hard | Anthropic, OpenAI, DeepMind, Meta | ✅ | Q / S | |
| 🔥 Implement Gradient Checkpointing | hard | Meta, Google, NVIDIA, Tesla | ✅ | Q / S | |
| 🔥 Implement Mixture of Experts Layer | hard | Google, DeepMind, Mistral, Databricks, xAI | ✅ | Q / S | |
| 🔥 Implement Speculative Decoding | hard | Google, DeepMind, Anthropic, Apple | ✅ | Q / S | |
| 🔥 Implement Continuous Batching for LLM Inference | hard | Perplexity, Together AI, Anyscale, Meta | ✅ | Q / S | |
| 🔥 Implement DDPM from Scratch | hard | Midjourney, Runway, Stability AI, Adobe, Google | ✅ | Q / S | |
| 🔥 Implement DDIM Sampling + Classifier-Free Guidance | hard | Midjourney, Runway, Stability AI, Adobe | ✅ | Q / S | |
| ⭐ Implement Selective State Space Model (Mamba) | hard | DeepMind, Google, Anthropic | ✅ | Q / S | |
| 🔥 Implement Vision Transformer + MAE Pretraining | hard | Meta, Google, Apple, Tesla, Waymo | ✅ | Q / S | |
| 🔥 Write a Fused Softmax Kernel in Triton | expert | NVIDIA, Meta, Google, xAI, Tesla | ✅ | Q / S | |
| 🔥 Implement FlashAttention-2 in Triton | expert | NVIDIA, Meta, Together AI, xAI | ✅ | Q / S | |
| 🔥 Implement FSDP from Scratch | expert | Meta, Google, NVIDIA, Anthropic, xAI | ✅ | Q / S | |
| ⭐ Implement GRPO (DeepSeek-R1 Algorithm) | expert | DeepMind, Anthropic, OpenAI | ✅ | Q / S | |
| 🔥 Build a Complete LLM Inference Engine | expert | Perplexity, Together AI, Anyscale, Fireworks AI | ✅ | Q / S | |
| 🔥 Implement Knowledge Distillation | medium | Google, Apple, Meta, Qualcomm, Tesla | ✅ | Q / S | |
| 🔥 Implement Ring Attention for Long Contexts | expert | Anthropic, Google, Meta, xAI | ✅ | Q / S | |
| 🔥 Implement a Swin Transformer Block (Shifted Window Attention) | hard | Microsoft, Meta, ByteDance, Google | ✅ | Q / S |