Comment by zionsrogue

The model used in the post (LeNet) is very small compared to what we typically think of in terms of modern deep neural networks. The LeNet architecture itself was published in the 1990s and was designed for character recognition so this is a natural application of it. kNN would work for many characters in this example, but you would run into a problem with overlapping text. kNN on raw pixel intensities requires a nice segmentation of the ROI. More image processing/segmentation/morphological operations could be applied to help in that case, but given a small network architecture that will naturally learn these filters a tiny CNN works well with little preprocessing.