CIFAR-10 Softmax Classifier

Overview

The first project in KTH's Deep Learning in Data Science course: a single-layer softmax classifier built from scratch in NumPy, trained on CIFAR-10 with mini-batch gradient descent and L2 regularization. Final test accuracy: 39.30% with the optimal hyperparameters (λ = 0.1, η = 0.001) — solid for a linear model staring at raw pixels.

The pipeline

• Data normalization, forward pass, cross-entropy loss, and analytic backpropagation — all hand-derived and hand-implemented.
• Gradients verified against PyTorch autograd to machine precision (max error ~1e-16), so every later experiment ran on provably correct arithmetic.
• Weight visualization: rendering each class's weight matrix as an image reveals the ghostly "template" the linear model learns for each class — a blurry car, a sky-over-green-field for deer.

Hyperparameter study

I ran a systematic sweep across learning rates and regularization strengths and documented two clean findings in the LaTeX report:

1. High learning rates cause oscillation — the loss curve never settles, bouncing around the minimum.
2. Strong regularization improves weight interpretability at the cost of accuracy — the class templates become visibly smoother and more recognizable as λ grows, while test accuracy drops.

Source

Code and report: github.com/rippyboii/cifar10-softmax-classifier