The model size and computational costs of modern neural networks are huge. Both can be reduced by using a method called quantization. This post is a detailed guide on how to quantize neural networks and builds up everything required to perform a simple quantization of Google's transformer-based ViT from scratch.

CPU Integer matrix multiplication using numpy is catastrophically slow. Other solutions are only slightly better. For x86_64 CPUs, Intel's C++ library oneDNN provides a remedy.

“Knowledge and Productivity are like compound interest” (Richard Hamming). Therefore, the commonplace idea about learning as much as possible when you are young is true in an exponential way.