There are multiple libraries (PyTorch, TensorFlow) that can assist you in implementing almost any neural network architecture. This article is not about solving a neural net using one of those libraries. Instead, in this article, we'll see a step-by-step forward pass (forward propagation) and backward pass (backpropagation) example.

According to Wikipedia, "Gradient descent is a first-order iterative optimization algorithm for finding local minima of a differentiable function". Sounds a lot, right? In this article, let's get acquainted with the Gradient descent algorithm in the most straightforward (and "simplest") way. Before we continue

PyTorch is a deep learning framework that significantly simplifies the process of writing and training deep neural networks. It supports many architectures, from shallow ones to deep ones like transformers. I mean, any neural network architecture you can think of. On the other hand, tensors are fundamental data structures in

This article is part three of the Linear algebra series. Part 1: Scalars, Vectors, Matrices and Tensors Part 2: Basic Operations on Tensors Putting it in simple terms, the norms in Linear algebra lets us measure how big a tensor is. In Machine learning, tensors (vectors, matrices, etc.) are heavily

This article is part two of the Linear Algebra series. The first part gave a basic introduction to the fundamentals of Linear Algebra: scalars, vectors, matrices and tensors. The article also discussed scalars being 0th order tensors, vectors being 1st order tensors and matrices being 2nd order tensors. In this

A solid understanding of linear algebra is essential for better comprehension and effective implementation of neural networks. Getting started with deep learning libraries such as TensorFlow and PyTorch in a "black-box" manner is relatively easy, thanks to the abundance of tutorials and guides available. However, to truly grasp