A digit classifier from scratch
The problem
Build and train a neural network in plain Python that recognizes small pixel images of the digits 0 to 9, and measure how well it works.
Requirements
- Create a dataset of 5 × 7 pixel digits with random noise and small shifts, split into training, validation and test sets
- Implement a network with at least one hidden ReLU layer and a softmax output, using He initialization
- Train with mini-batches, cross-entropy and Adam, printing training and validation loss every epoch
- Use early stopping on the validation loss and keep the best weights
- Report test accuracy and a confusion table of which digits are mistaken for which
- Check the backpropagation code with a numerical gradient check
Technologies
- Python
- Backpropagation
- Softmax and cross-entropy
- Adam
Step by step
- Draw the ten digits as small grids of 0s and 1s and write a function that adds noise and shifts.
- Write the forward pass and check the output shapes and the starting loss (about log 10).
- Write the backward pass and confirm it with a gradient check on a tiny network.
- Add mini-batch training with Adam and watch the loss fall.
- Add validation, early stopping and the final test evaluation.
- Experiment with hidden size, learning rate and dropout, and record the results.
Expected result
A working digit recognizer, written without libraries, with a report of its accuracy and its typical mistakes.
How it is evaluated
- The gradient check passes for every parameter
- Test accuracy is above 90% on noisy digits
- Training and validation curves are reported and explained
- The code separates data, model, training and evaluation
Ways to improve it
- Replace the dense input layer with a small convolution layer
- Add batch normalization and compare training speed
- Rewrite the model in PyTorch and check that it gives the same results