PyCon UK 2019

We see many talks that explain neural networks (NNs), that describe the
different architectures and the problems for which each architecture is best.
We are not going to talk about that here. Instead, we will talk about the
implementation of NNs, about how we make NNs work in code - not by using a
library but by writing it ourselves.

Have you ever experimented with pytorch? With Tensorflow? With PySpark?
Have you built and trained a NN with one of these libraries? The process of
doing so is simple: You describe the overall NN architecture, feed the data,
and then execute a training procedure which runs for a while. This training
process "magically" produces a trained NN (be it well trained or badly
trained), which can then be used for predictions.

The process of getting a trained NN begs a question: what is the magic that
trains the NN into a predictive tool? Several resources explain NNs as a bunch
of circles connected by arrows, that is a good conceptual representation of a
NN but has no similarity with the actual implementation of the NN or of its
training procedure. Below the hood the implementation of a NN and its training
makes no circles or arrows, instead it is a clever sequence of matrix
multiplications.

We will go through the conceptual representation of a NN and then expand it and
describe the implementation. We will look at some mathematical concepts and
then we will see how to implement those concepts in NumPy. We will use
NumPy but we will see that we can substitute the NumPy parts of our code
with GPU or distributed processing.

The objective is to understand how NN libraries are programmed, and understand
where the GPU (Tensorflow, pytorch) or the distributed processing
(PySpark) is used within a NN library. We want to demystify NN libraries, to
prove that it is possible to write a simple NN library in less than 1000 lines
of code.

P.S. NumPy matrix multiplication (np.dot) knowledge is required.