# RNNs / LSTMs for Sequences

## Resume
Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks are sequential neural architectures designed to map time-series data or sequential text structures. LSTMs introduce explicit gating mechanisms to regulate internal information persistence, successfully resolving the vanishing gradient flaws of traditional RNN blocks.

### Common Use Cases:
* **SMILES Generation:** Autoregressively printing valid chemical strings token by token.
* **Biosignal Analysis:** Evaluating chronological streaming signals from patient telemetry or ECG readouts.
* **Clinical Notes Sequence Modeling:** Tracking medical event timelines over long EHR spans.

## Content

### 1. Core Architecture & Gated Memory
Standard RNNs maintain a recurring hidden state vector $h_t$ that gets updated with every timestamp token input. However, backpropagating through long sequences causes gradients to rapidly disappear or explode.

LSTMs solve this by introducing an internal **Cell State** ($c_t$) alongside three specialized gating mechanisms:
* **Forget Gate ($f_t$):** Controls how much historical context from the cell state should be discarded.
* **Input Gate ($i_t$):** Regulates what new contextual state information should be infused into the current cell vector.
* **Output Gate ($o_t$):** Determines what subset of internal hidden cell states should be exposed as the final output block.

### 2. Operational Execution Sequence
As a sequence executes, the architecture reads the token input at step $t$, merges it with the prior step's hidden vector $h_{t-1}$, adjusts cell memories through the gating parameters, and pushes forward the updated state vectors. This linear memory highway allows the architecture to carry context across extended sequence matrices.

### 3. Advantages and Disadvantages
* **Advantages:**
  * Natural alignment with arbitrary, variable-length chronological or textual stream structures.
  * O(L) computational complexity scaling linearly with sequence length.
* **Disadvantages:**
  * Strict step-by-step dependency makes parallel sequence acceleration impossible during training.
  * Susceptible to information decay over extremely long sequences compared to attention structures.

### References
* Hochreiter, S., & Schmidhuber, J. (1997). *Long short-term memory*. Neural Computation.
* Segler, M. H., et al. (2018). *Generating focused molecule libraries for drug discovery with recurrent neural networks*. ACS Central Science.