Skip to content

Latest commit

 

History

History
108 lines (100 loc) · 32.6 KB

PROBLEMS.md

File metadata and controls

108 lines (100 loc) · 32.6 KB

All 68 problems

Generated from the per-problem problem.toml manifests — do not edit by hand. Run python3 scripts/generate.py readme to refresh.

63 of 68 are auto-graded (pip install torchleet). A ✅ in the Graded column means check() will verify your solution.

Basics (23)

Problem Difficulty Companies Graded Run Links
💡 Implement Linear Regression basic ✅ Colab Q / S
💡 Write a Custom Dataset and DataLoader basic ✅ Colab Q / S
💡 Write a Custom Activation Function basic ✅ Colab Q / S
💡 Implement Custom Loss Function (Huber Loss) basic ✅ Colab Q / S
💡 Implement a Deep Neural Network basic ✅ Colab Q / S
💡 Visualize Training with TensorBoard basic ✅ Colab Q / S
💡 Save and Load PyTorch Model basic Colab Q / S
💡 Implement a CNN on CIFAR-10 easy ✅ Colab Q / S
💡 Implement an RNN from Scratch easy ✅ Colab Q / S
💡 Data Augmentation with torchvision.transforms easy Colab Q / S
💡 Add Benchmarking to PyTorch Code easy ✅ Colab Q / S
💡 Train an Autoencoder for Anomaly Detection easy ✅ Colab Q / S
💡 Quantize Your Language Model easy ✅ Colab Q / S
💡 Mixed Precision Training with torch.cuda.amp easy Colab Q / S
💡 CNN Parameter Initialization medium ✅ Colab Q / S
💡 Implement a CNN from Scratch medium ✅ Colab Q / S
💡 Implement an LSTM from Scratch medium ✅ Colab Q / S
💡 Implement RMS Norm easy ✅ Colab Q / S
💡 Implement Byte Pair Encoding from Scratch easy ✅ Colab Q / S
🔥 Implement Softmax from Scratch easy Apple, Meta, Google, Amazon ✅ Colab Q / S
🔥 Implement K-Means Clustering in PyTorch easy Uber, LinkedIn, Google, Amazon ✅ Colab Q / S
⭐ Implement KNN in PyTorch easy Uber, LinkedIn, Meta ✅ Colab Q / S
⭐ Implement Logistic Regression with Gradient Descent easy Google, Meta, Amazon ✅ Colab Q / S

LLM Learning Path (21)

Problem Difficulty Companies Graded Run Links
💡 Implement RMS Norm easy ✅ Colab Q / S
💡 Implement Byte Pair Encoding from Scratch easy ✅ Colab Q / S
💡 Implement Attention from Scratch medium ✅ Colab Q / S
💡 Implement Multi-Head Attention from Scratch medium ✅ Colab Q / S
💡 Implement Grouped Query Attention from Scratch medium ✅ Colab Q / S
💡 Implement Sinusoidal Embeddings medium ✅ Colab Q / S
💡 Implement ROPE Embeddings medium ✅ Colab Q / S
💡 Implement SmolLM from Scratch hard ✅ Colab Q / S
🔥 Implement Top-p (Nucleus) Sampling medium Anthropic, OpenAI, DeepMind, Perplexity, Cohere ✅ Colab Q / S
🔥 Implement Top-k Sampling medium Anthropic, OpenAI, DeepMind, Cohere ✅ Colab Q / S
🔥 Implement Temperature Sampling easy OpenAI, Anthropic, Cohere, Perplexity ✅ Colab Q / S
🔥 Implement LoRA on a Linear Layer medium Meta, Google, Anthropic, OpenAI, Databricks ✅ Colab Q / S
🔥 Implement KV Cache for Autoregressive Generation medium Anthropic, OpenAI, Meta, Perplexity, Together AI ✅ Colab Q / S
🔥 Implement Sliding Window Attention medium Mistral, Anthropic, Google, DeepMind ✅ Colab Q / S
🔥 Implement DPO Loss from Scratch hard Anthropic, OpenAI, DeepMind, Meta ✅ Colab Q / S
🔥 Implement PPO for RLHF hard Anthropic, OpenAI, DeepMind, Meta ✅ Colab Q / S
🔥 Implement Mixture of Experts Layer hard Google, DeepMind, Mistral, Databricks, xAI ✅ Colab Q / S
🔥 Implement Speculative Decoding hard Google, DeepMind, Anthropic, Apple ✅ Colab Q / S
🔥 Implement Continuous Batching for LLM Inference hard Perplexity, Together AI, Anyscale, Meta ✅ Colab Q / S
⭐ Implement GRPO (DeepSeek-R1 Algorithm) expert DeepMind, Anthropic, OpenAI ✅ Colab Q / S
🔥 Build a Complete LLM Inference Engine expert Perplexity, Together AI, Anyscale, Fireworks AI ✅ Colab Q / S

Advanced (42)

Problem Difficulty Companies Graded Run Links
💡 Implement AlexNet from Scratch medium ✅ Colab Q / S
💡 Train a 3D CNN for CT Image Segmentation medium ✅ Colab Q / S
💡 Write a Custom Autograd Function (SILU) hard ✅ Colab Q / S
💡 Write a Transformer from Scratch hard ✅ Colab Q / S
💡 Write a GAN hard ✅ Colab Q / S
💡 Sequence-to-Sequence with Attention hard ✅ Colab Q / S
💡 Explainable AI (GradCAM/SHAP) hard Colab Q / S
⭐ Implement Adam from Scratch hard Meta, Google ✅ Colab Q / S
⭐ Implement AdamW from Scratch hard Meta, Google ✅ Colab Q / S
⭐ Implement Muon from Scratch hard Meta, Google ✅ Colab Q / S
💡 Create Embeddings Out Of An Llm medium Colab Q
💡 Flash Attention medium ✅ Colab Q
💡 Implement Attention from Scratch medium ✅ Colab Q / S
💡 Implement Multi-Head Attention from Scratch medium ✅ Colab Q / S
💡 Implement Grouped Query Attention from Scratch medium ✅ Colab Q / S
💡 Implement Sinusoidal Embeddings medium ✅ Colab Q / S
💡 Implement ROPE Embeddings medium ✅ Colab Q / S
💡 Implement SmolLM from Scratch hard ✅ Colab Q / S
🔥 Implement Contrastive Loss (InfoNCE) + CLIP Training Loop medium OpenAI, Anthropic, DeepMind, Midjourney, Apple ✅ Colab Q / S
🔥 Implement 2D Positional Embeddings medium Anthropic, DeepMind, Midjourney, Runway ✅ Colab Q / S
🔥 Implement Beam Search for LLM Decoding medium Google, DeepMind, Meta, Apple ✅ Colab Q / S
🔥 Implement LoRA on a Linear Layer medium Meta, Google, Anthropic, OpenAI, Databricks ✅ Colab Q / S
🔥 Implement KV Cache for Autoregressive Generation medium Anthropic, OpenAI, Meta, Perplexity, Together AI ✅ Colab Q / S
🔥 Implement Sliding Window Attention medium Mistral, Anthropic, Google, DeepMind ✅ Colab Q / S
🔥 Implement DPO Loss from Scratch hard Anthropic, OpenAI, DeepMind, Meta ✅ Colab Q / S
🔥 Implement PPO for RLHF hard Anthropic, OpenAI, DeepMind, Meta ✅ Colab Q / S
🔥 Implement Gradient Checkpointing hard Meta, Google, NVIDIA, Tesla ✅ Colab Q / S
🔥 Implement Mixture of Experts Layer hard Google, DeepMind, Mistral, Databricks, xAI ✅ Colab Q / S
🔥 Implement Speculative Decoding hard Google, DeepMind, Anthropic, Apple ✅ Colab Q / S
🔥 Implement Continuous Batching for LLM Inference hard Perplexity, Together AI, Anyscale, Meta ✅ Colab Q / S
🔥 Implement DDPM from Scratch hard Midjourney, Runway, Stability AI, Adobe, Google ✅ Colab Q / S
🔥 Implement DDIM Sampling + Classifier-Free Guidance hard Midjourney, Runway, Stability AI, Adobe ✅ Colab Q / S
⭐ Implement Selective State Space Model (Mamba) hard DeepMind, Google, Anthropic ✅ Colab Q / S
🔥 Implement Vision Transformer + MAE Pretraining hard Meta, Google, Apple, Tesla, Waymo ✅ Colab Q / S
🔥 Write a Fused Softmax Kernel in Triton expert NVIDIA, Meta, Google, xAI, Tesla ✅ Colab Q / S
🔥 Implement FlashAttention-2 in Triton expert NVIDIA, Meta, Together AI, xAI ✅ Colab Q / S
🔥 Implement FSDP from Scratch expert Meta, Google, NVIDIA, Anthropic, xAI ✅ Colab Q / S
⭐ Implement GRPO (DeepSeek-R1 Algorithm) expert DeepMind, Anthropic, OpenAI ✅ Colab Q / S
🔥 Build a Complete LLM Inference Engine expert Perplexity, Together AI, Anyscale, Fireworks AI ✅ Colab Q / S
🔥 Implement Knowledge Distillation medium Google, Apple, Meta, Qualcomm, Tesla ✅ Colab Q / S
🔥 Implement Ring Attention for Long Contexts expert Anthropic, Google, Meta, xAI ✅ Colab Q / S
🔥 Implement a Swin Transformer Block (Shifted Window Attention) hard Microsoft, Meta, ByteDance, Google ✅ Colab Q / S