Update README.md

README.md

@@ -9,13 +9,80 @@ V2 NLP to monitor Twitter for natural disasters    Natural Language Processing (
 v3 Intro to 🤗 Transformers performance:0.83481 V3
 ```
 
+<a name='1'></a>
+## 1 - Forward Propagation for the Basic Recurrent Neural Network
 
+Later this week, you'll get a chance to generate music using an RNN! The basic RNN that you'll implement has the following structure: 
 
+In this example, Tx = Ty. 
 
 
-<h1 align="center" style="color:green;font-size: 3em;" >Implementing Transformers From Scratch Using Pytorch</h1>
+<img src="Images/RNN.png" style="width:500;height:300px;">
+<caption><center><font color='purple'><b>Figure 1</b>: Basic RNN model </center></caption>
+
+
+<a name='1-1'></a>
+### 1.1 - RNN Cell
+
+You can think of the recurrent neural network as the repeated use of a single cell. First, you'll implement the computations for a single time step. The following figure describes the operations for a single time step of an RNN cell: 
+
+<img src="Images/rnn_step_forward_figure2_v3a.png" style="width:700px;height:300px;">
+<caption><center><font color='purple'><b>Figure 2</b>: Basic RNN cell. 
+</center></caption>
+
+<a name='1-2'></a>
+### 1.2 - RNN Forward Pass
+
+<img src="Images/rnn_forward_sequence_figure3_v3a.png" style="width:800px;height:180px;">
+<caption><center><font color='purple'><b>Figure 3</b>: Basic RNN. 
+</center></caption>
+
+<a name='2'></a>
+## 2 - Long Short-Term Memory (LSTM) Network
+
+The following figure shows the operations of an LSTM cell:
+
+<img src="Images/LSTM_figure4_v3a.png" style="width:500;height:400px;">
+<caption><center><font color='purple'><b>Figure 4</b>: LSTM cell. 
+Note, the softmax includes a dense layer and softmax.
+</center></caption>
+
+
+<a name='2-2'></a>
+### 2.2 - Forward Pass for LSTM
+
+Now that you have implemented one step of an LSTM, you can iterate this over it using a for loop to process a sequence of Tx inputs. 
+
+<img src="Images/LSTM_rnn.png" style="width:500;height:300px;">
+<caption><center><font color='purple'><b>Figure 5</b>: LSTM over multiple time steps. </center></caption>
 
+<a name='3-1'></a>    
+### 3.1 - Basic RNN  Backward Pass
 
+Begin by computing the backward pass for the basic RNN cell. Then, in the following sections, iterate through the cells.
+
+<img src="Images/rnn_backward_overview_3a_1.png" style="width:500;height:300px;"> <br>
+<caption><center><font color='purple'><b>Figure 6</b>: The RNN cell's backward pass. 
+</center></caption>
+
+<img src="Images/rnn_cell_backward_3a_c.png" style="width:800;height:500px;"> <br>
+<caption><center><font color='purple'><b>Figure 7</b>: This implementation of `rnn_cell_backward` does **not** include the output dense layer and softmax which are included in `rnn_cell_forward`.  
+
+
+#### 1. One Step Backward
+The LSTM backward pass is slightly more complicated than the forward pass.
+
+<img src="Images/LSTM_cell_backward_rev3a_c2.png" style="width:500;height:400px;"> <br>
+<caption><center><font color='purple'><b>Figure 8</b>: LSTM Cell Backward. Note the output functions, while part of the `lstm_cell_forward`, are not included in `lstm_cell_backward` 
+</center></caption>
+
+
+
+
+
+
+
+<h1 align="center" style="color:green;font-size: 3em;" >Implementing Transformers From Scratch Using Pytorch</h1>
 
 
 * [1. Introduction](#section1)