Six-month study plan
A Grade 9 ramp from "knows Python" to "can fine-tune a small transformer on a custom dataset." Roughly 6 focused hours per week, with a notebook portfolio that grows month by month.
Working artifact. Keep one Git repository with a folder per month. Every week ends with at least one runnable notebook checked in. By month 6 you'll have a real portfolio.
Targets by phase
| Phase | Outcome |
|---|---|
| Months 1–2 | Math fluency + Python data stack; can manipulate any tabular dataset. |
| Months 3–4 | Classical ML mastery; can baseline any tabular contest problem with sklearn in one sitting. |
| Month 5 | PyTorch fluency; can write a training loop and an MLP from scratch. |
| Month 6 | Transformer fundamentals; can implement scaled dot-product attention from memory and fine-tune a small pretrained model. |
Month 1 · Math + Python warm-up
- Week 1: Linear algebra refresh — vectors, dot products, matrix multiplication. NumPy from scratch: arrays, slicing, broadcasting.
- Week 2: Eigenvalues, SVD, PCA. Implement PCA in NumPy on Iris.
- Week 3: Probability + statistics. Random variables, distributions, MLE. Sample from each common distribution in NumPy.
- Week 4: Multivariable calculus. Gradients, chain rule. Run gradient descent on a 2-D convex function by hand.
Month 2 · Data wrangling + visualization
- Week 1: pandas basics. Load 3 datasets (Titanic, California Housing, MNIST CSV). For each:
head(),info(),describe(), missing-value handling. - Week 2: matplotlib + seaborn. Plot histograms, scatter plots, correlation heatmaps for the same 3 datasets.
- Week 3: Feature engineering. Categorical encoding, log transforms, date feature extraction.
- Week 4: End-to-end mini project — pick one Kaggle Getting Started dataset, do the full pipeline (load → clean → feature → baseline model → score).
Month 3 · Classical ML (supervised)
- Week 1: Linear & logistic regression. Implement gradient descent for both in NumPy.
- Week 2: Ridge / Lasso, regularization intuition. Cross-validation with
StratifiedKFold. - Week 3: Decision trees, random forest. Visualize a single tree; compare to a forest of 100.
- Week 4: Gradient boosting (
HistGradientBoosting). End-of-month mini project — beat your Month 2 baseline by ≥10%.
Month 4 · Classical ML (unsupervised + ensembles)
- Week 1: K-Means + DBSCAN. Cluster MNIST embeddings; visualize with PCA / t-SNE.
- Week 2: PCA + t-SNE + UMAP. Compare on the same dataset.
- Week 3: Ensemble techniques. Stacking, weighted averaging, blending. Train 3 different models and average them.
- Week 4: Full reproducibility pass — pin requirements, set seeds, commit a notebook that runs end-to-end on a fresh environment.
Month 5 · PyTorch + deep learning
- Week 1: Tensors + autograd. Re-derive backprop for a 2-layer MLP, verify against autograd.
- Week 2: Build an MLP for MNIST. Achieve ≥97% test accuracy. Plot training curves.
- Week 3: Build a small CNN for CIFAR-10. Compare against the MLP. Try data augmentation.
- Week 4: Regularization sweep — dropout, weight decay, early stopping. End-of-month: written write-up of what worked and what didn't.
Month 6 · Transformers + fine-tuning
- Week 1: Implement scaled dot-product attention from scratch. Verify against
F.scaled_dot_product_attention. - Week 2: Build multi-head attention + a single transformer block. Train a tiny char-level language model on Shakespeare-style text.
- Week 3: Tokenization — use a BPE tokenizer; pre-train a tiny transformer (≤5M parameters) on a small corpus.
- Week 4: Fine-tune a small open-weight model (DistilBERT-sized) on a text classification task. Compare to your classical ML baseline.
Practice cadence
The notebook rule. Every week ends with one notebook checked into Git that runs end-to-end on a fresh machine. No half-finished, no out-of-order cells.
Don't skip the math. The temptation when fine-tuning a transformer is to treat it as a black box. The contest's theory section punishes that — derive gradients on paper at least once per major topic.
Kaggle Getting Started cadence. One Kaggle dataset per month, with a real submission and a leaderboard score. The feedback loop is faster than any textbook.
Weekly checklist template
- ☐ 1 notebook committed to Git (clean, runs end-to-end)
- ☐ 1 page of math notes in your "patterns" log
- ☐ Read 1 short paper or blog post on the week's topic
- ☐ Test all code on the validation set you set up in Week 1
- ☐ Random seeds set; requirements pinned