Basic Deep Learning Concepts

Deep learning is basically an attempt to copy the human brain’s way of learning and functioning.


The perceptron is an attempt to mimic the function of the neuron in the human brain. It is otherwise called an Artificial Neuron, hence the term neural network.

Neuron Diagram

Perceptron Diagram


Neural Network

If the perceptron is an attempt to mimic the function of the a neuron in the brain, the neural network is an attempt to mimic at the very least a sub-section if not the whole brain.


Forward Propagation

This involves creating the graph and initilizing the weights randomly, I think?

Back Propagation

Calculate the gradient of the cost function using gradient descent. The idea being that we are trying to figure out which biases and weights the cost function is most sensitive to.

Stochastic Gradient Descent (SGD)

SGD is the process of using mini-batches to get a rough idea of gradient descent, rather than the full calculation to get the exact, perfect descent.

Sigmoid Function/ReLU


update parameters