Coursera Deep Learning Specialization Notes

Table of contents

1. Intro to Neural Network and Deep Learning
   1. Logistic regression (forward & backward)
   2. Neural network representation & activation functions
   3. Gradient descent, Back prop, & DNN
2. Improving DNN: Hyperparameter tuning, Regularization, & Optimization
   1. General logic for improving DNN
   2. L2 regularization
   3. Dropout and Early-stop
   4. Normalize inputs
   5. Vanishing gradient & Initilization
   6. Gradient checking
   7. Batch & mini-batch gradient descent
   8. Optimizers
   9. Learning rate decay
   10. Hyperparameter tunning process
   11. Batch Normalization
   12. Softmax activation
3. Structuring machine learning projects
   1. Speeding up the cycle: problem & solutions
   2. Satificing & Optimizing metrics
   3. Train, Dev, & Test split
   4. Error analysis
   5. Training & Testing data from different distribution
   6. Transfer learning, Multitask learning, & End-to-end learning
4. Convolutional Neural Network
   1. Convolution as feature detector, padding, & stride
   2. Convolution over volume
   3. Convolution layer
   4. Pooling layer
   5. Classic CNN architecture
   6. Inception CNN and 1x1 convolution
   7. Mobile Net
   8. Sliding window for object detection
   9. YOLO algorithm
   10. Semantic segmentation
   11. Siamese network & Triplet loss
   12. Neural style transfer
5. Recurrent Neural Network
   1. Use cases for RNN
   2. RNN forward & backward
   3. Different types of RNN
   4. Language models & Sampling new text
   5. GRU & LSTM
   6. Deep RNN
   7. Word embeddings
   8. Learning embeddings: word2vec & Glove
   9. Sentiment analysis
   10. Sequence to sequence models
   11. Beam search
   12. Attention mechanism
   13. Transformer architecture

Intro to Neural Network and Deep Learning

Logistic regression (forward & backward)

Neural network representation & activation functions

Gradient descent, Back prop, & DNN

Improving DNN: Hyperparameter tuning, Regularization, & Optimization

General logic for improving DNN

L2 regularization

Dropout and Early-stop

Normalize inputs

Vanishing gradient & Initilization

Gradient checking

Batch & mini-batch gradient descent

Optimizers

Learning rate decay

Hyperparameter tunning process

Batch Normalization

Softmax activation

Structuring machine learning projects

Speeding up the cycle: problem & solutions

Satificing & Optimizing metrics

Train, Dev, & Test split

Error analysis

Training & Testing data from different distribution

Transfer learning, Multitask learning, & End-to-end learning

Convolutional Neural Network

Convolution as feature detector, padding, & stride

Convolution over volume

Convolution layer

Pooling layer

Classic CNN architecture

Inception CNN and 1x1 convolution

Mobile Net

Sliding window for object detection

YOLO algorithm

Semantic segmentation

Siamese network & Triplet loss

Neural style transfer

Recurrent Neural Network

Use cases for RNN

RNN forward & backward

Different types of RNN

Language models & Sampling new text

GRU & LSTM

Deep RNN

Word embeddings

Learning embeddings: word2vec & Glove

Sentiment analysis

Sequence to sequence models

Beam search

Attention mechanism

Transformer architecture