model/mpl_pequeno_torch/help.md

RNNs / LSTMs for Sequences

Resume

Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks are sequential neural architectures designed to map time-series data or sequential text structures. LSTMs introduce explicit gating mechanisms to regulate internal information persistence, successfully resolving the vanishing gradient flaws of traditional RNN blocks.

Common Use Cases:

• SMILES Generation: Autoregressively printing valid chemical strings token by token.
• Biosignal Analysis: Evaluating chronological streaming signals from patient telemetry or ECG readouts.
• Clinical Notes Sequence Modeling: Tracking medical event timelines over long EHR spans.

Content

1. Core Architecture & Gated Memory

Standard RNNs maintain a recurring hidden state vector $h_t$ that gets updated with every timestamp token input. However, backpropagating through long sequences causes gradients to rapidly disappear or explode.

LSTMs solve this by introducing an internal Cell State ($c_t$) alongside three specialized gating mechanisms:

• Forget Gate ($f_t$): Controls how much historical context from the cell state should be discarded.
• Input Gate ($i_t$): Regulates what new contextual state information should be infused into the current cell vector.
• Output Gate ($o_t$): Determines what subset of internal hidden cell states should be exposed as the final output block.

2. Operational Execution Sequence

As a sequence executes, the architecture reads the token input at step $t$, merges it with the prior step's hidden vector $h_{t-1}$, adjusts cell memories through the gating parameters, and pushes forward the updated state vectors. This linear memory highway allows the architecture to carry context across extended sequence matrices.

3. Advantages and Disadvantages

• Advantages:
  • Natural alignment with arbitrary, variable-length chronological or textual stream structures.
  • O(L) computational complexity scaling linearly with sequence length.
• Disadvantages:
  • Strict step-by-step dependency makes parallel sequence acceleration impossible during training.
  • Susceptible to information decay over extremely long sequences compared to attention structures.

References

• Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation.
• Segler, M. H., et al. (2018). Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Central Science.