README.md

metadata

title: UGTC — Uncertainty-Gated Temporal Credit
emoji: 🎯
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: true
license: mit
tags:
  - reinforcement-learning
  - advantage-estimation
  - temporal-credit
  - uncertainty
  - actor-critic
  - PPO
  - TD3
  - SAC

UGTC: Uncertainty-Gated Temporal Credit

Interactive demo and educational interface for the UGTC paper.

Paper: 10.5281/zenodo.19715116
Accepted: Ulysseus Young Explorers in Science (UYES) Journal · Journal DOI forthcoming
GitHub: ethosoftai/ugtc
Docs: ethosoftai.github.io/ugtc

---

What is UGTC?

UGTC (Uncertainty-Gated Temporal Credit) is a plug-in advantage estimator for actor-critic reinforcement learning.

It resolves the bias–variance trade-off in temporal credit assignment by:

1. Maintaining two critics with different GAE λ values:

   • Fast critic (single network, λ=0.80): low variance, higher bias
   • Slow ensemble (M=3 networks, λ=0.99): lower bias, higher variance

2. Measuring uncertainty as the disagreement (std deviation) among slow ensemble members

3. Using a sigmoid gate to blend the two advantage estimates:

   • Low uncertainty → trust the slow (accurate) estimate
   • High uncertainty → trust the fast (stable) estimate

Explore in This Space

The demo provides:

• Gate Visualizer: See how the uncertainty gate u(s) responds to different ensemble disagreement levels
• Algorithm Comparison: Side-by-side view of UGTC-PPO vs vanilla PPO objective
• Math Explorer: Interactive walkthrough of the UGTC equations
• Pseudocode Viewer: Step-by-step algorithm pseudocode for PPO, TD3, and SAC integrations
• Hyperparameter Impact: Visualize how β (gate temperature) and λ values affect the blending

Citation

@misc{dalar2026ugtc,
  author    = {Dalar, Yağız Ekrem},
  title     = {{UGTC}: Uncertainty-Gated Temporal Credit},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19715116},
  url       = {https://doi.org/10.5281/zenodo.19715116},
  note      = {Accepted — Ulysseus Young Explorers in Science (UYES) Journal.
               Journal DOI forthcoming.}
}