Neural Networks and Deep Learning (6)

Shallow Neural Networks

Build a neural network with one hidden layer, using forward propagation and backpropagation.

Learning Objectives

---

• Describe hidden units and hidden layers
• Use units with a non-linear activation function, such as tanh
• Implement forward and backward propagation
• Apply random initialization to your neural network
• Increase fluency in Deep Learning notations and Neural Network Representations
• Implement a 2-class classification neural network with a single hidden layer
• Compute the cross-entropy loss

Shallow Neural Network

728x90

Neural Networks Overview

A neural network consists of layer(s) of neuron(s) that compute the inputs with weights and biases, followed by an activation function to make predictions.

This prediction improves as the network tweaks the weights and biases automatically (through derivation in the backpropagation process).

Neural Network Representation

2-layer neural network (nn)

A neural network has 3 layers: Input, hidden, and output layer.

An input layer is what we input to the network (our data/features).

A hidden layer is all the layers that reside inside a neural network and do most of the computation.

It’s called hidden since we don’t see what they should be in the training set.

In the supervised training set, we have our data and corresponding answers.

An output layer generates the predicted values.

When we count the number of layers, we generally don't count the input layer, hence the example network is called a 2-layer neural network.

Computing a Neural Network’s Output

0123

As mentioned previously, each node has 2 steps of calculations:

1. Weights are multiplied by the input data or the output of the previous layer and then biases are added
2. An activation function is applied to the output of step 1

For the notation, the superscript (in brackets) refers to the layer number and the subscript refers to the node number.

Vectorizing Across Multiple Examples

01

The parenthesis in the superscript refers to the training example number.

Vectorization helps avoid using for-loops and compute the examples at once.

Explanation for Vectorized Implementation

01

Since we are performing the same computation, vectorization can be used to simplify it.

 

All the information provided is based on the Deep Learning Specialization | Coursera from DeepLearning.AI