### 2019-09-13, 10:30–12:00, Room C

Let's write a (tiny but working) Neural Network library from scratch!

Well, almost from scratch, we will still use `NumPy`

.

And we will try to do it in less than 1000 lines of code.

We see many talks that explain neural networks (NNs), that describe the

different architectures and the problems for which each architecture is best.

We are **not** going to talk about that here. Instead, we will talk about the

implementation of NNs, about how we make NNs work in code - not by using a

library but by writing it ourselves.

Have you ever experimented with `pytorch`

? With `Tensorflow`

? With `PySpark`

?

Have you built and trained a NN with one of these libraries? The process of

doing so is simple: You describe the overall NN architecture, feed the data,

and then execute a training procedure which runs for a while. This training

process "magically" produces a trained NN (be it well trained or badly

trained), which can then be used for predictions.

The process of getting a trained NN begs a question: *what* is the magic that

trains the NN into a predictive tool? Several resources explain NNs as a bunch

of circles connected by arrows, that is a good conceptual representation of a

NN but has no similarity with the actual implementation of the NN or of its

training procedure. Below the hood the implementation of a NN and its training

makes no circles or arrows, instead it is a clever sequence of matrix

multiplications.

We will go through the conceptual representation of a NN and then expand it and

describe the implementation. We will look at some mathematical concepts and

then we will see how to implement those concepts in `NumPy`

. We will use

`NumPy`

but we will see that we can substitute the `NumPy`

parts of our code

with GPU or distributed processing.

The objective is to understand how NN libraries are programmed, and understand

where the GPU (`Tensorflow`

, `pytorch`

) or the distributed processing

(`PySpark`

) is used within a NN library. We want to demystify NN libraries, to

prove that it is possible to write a simple NN library in less than 1000 lines

of code.

P.S. `NumPy`

matrix multiplication (`np.dot`

) knowledge is required.

**Is your proposal suitable for beginners?**– no