Basic Deep Learning Concepts
Deep learning is basically an attempt to copy the human brain’s way of learning and functioning.
Perceptron
The perceptron is an attempt to mimic the function of the neuron in the human brain. It is otherwise called an Artificial Neuron, hence the term neural network.
Neuron Diagram
Perceptron Diagram
Neural Network
If the perceptron is an attempt to mimic the function of the a neuron in the brain, the neural network is an attempt to mimic at the very least a sub-section if not the whole brain.
Sample
Forward Propagation
This involves creating the graph and initilizing the weights randomly, I think?
Back Propagation
Calculate the gradient of the cost function using gradient descent. The idea being that we are trying to figure out which biases and weights the cost function is most sensitive to.
Stochastic Gradient Descent (SGD)
SGD is the process of using mini-batches to get a rough idea of gradient descent, rather than the full calculation to get the exact, perfect descent.
Sigmoid Function/ReLU
Update
update parameters