Neural Networks in CUDA from Scratch - 1. Activation Layer

In this Video Series I will show how to implement Neural Networks in CUDA from scratch. We are going to start with a simple "Activation Layer" which take's a bunch of values and computes the activation function on them.

The code can be found here: https://github.com/ThoenigAdrian/Neur...

This will be a multipart series:
I am planning to cover the following topics
1. Implementing activation layer (this video)
2. Linear layer
3. Forward pass with multiple layers
4. Forward pass with multiple inputs
5. Backpropagation
6. Applying gradient descent
7. Training loop with multiple iterations
8. Cost function
9. Scaling

If you are interested in general CUDA Programming, I can recommend the following the following resources:

Cuda Crash Course: @CoffeeBeforeArch    • CUDA Crash Course: Vector Addition  

Cuda Crash Cource v2: @CoffeeBeforeArch    • C++ Crash Course (v2): Text Editors and Co...  

NVIDIA CUDA Tutorial @Creel
   • NVIDIA CUDA Tutorial 1: Introduction  

Nvidia Documentation: https://docs.nvidia.com/cuda/cuda-c-p...
(Yes the official documentation, it's actually really good)