| --- |
| title: UGTC — Uncertainty-Gated Temporal Credit |
| emoji: 🎯 |
| colorFrom: indigo |
| colorTo: purple |
| sdk: gradio |
| sdk_version: 4.44.0 |
| app_file: app.py |
| pinned: true |
| license: mit |
| tags: |
| - reinforcement-learning |
| - advantage-estimation |
| - temporal-credit |
| - uncertainty |
| - actor-critic |
| - PPO |
| - TD3 |
| - SAC |
| --- |
| |
| # UGTC: Uncertainty-Gated Temporal Credit |
|
|
| **Interactive demo and educational interface for the UGTC paper.** |
|
|
| > **Paper:** [10.5281/zenodo.19715116](https://doi.org/10.5281/zenodo.19715116) |
| > **Accepted:** Ulysseus Young Explorers in Science (UYES) Journal · Journal DOI forthcoming |
| > **GitHub:** [ethosoftai/ugtc](https://github.com/ethosoftai/ugtc) |
| > **Docs:** [ethosoftai.github.io/ugtc](https://ethosoftai.github.io/ugtc) |
|
|
| --- |
|
|
| ## What is UGTC? |
|
|
| UGTC (Uncertainty-Gated Temporal Credit) is a **plug-in advantage estimator** for actor-critic reinforcement learning. |
|
|
| It resolves the bias–variance trade-off in temporal credit assignment by: |
|
|
| 1. Maintaining **two critics** with different GAE λ values: |
| - **Fast critic** (single network, λ=0.80): low variance, higher bias |
| - **Slow ensemble** (M=3 networks, λ=0.99): lower bias, higher variance |
|
|
| 2. Measuring **uncertainty** as the disagreement (std deviation) among slow ensemble members |
|
|
| 3. Using a **sigmoid gate** to blend the two advantage estimates: |
| - Low uncertainty → trust the slow (accurate) estimate |
| - High uncertainty → trust the fast (stable) estimate |
|
|
| ## Explore in This Space |
|
|
| The demo provides: |
| - **Gate Visualizer**: See how the uncertainty gate u(s) responds to different ensemble disagreement levels |
| - **Algorithm Comparison**: Side-by-side view of UGTC-PPO vs vanilla PPO objective |
| - **Math Explorer**: Interactive walkthrough of the UGTC equations |
| - **Pseudocode Viewer**: Step-by-step algorithm pseudocode for PPO, TD3, and SAC integrations |
| - **Hyperparameter Impact**: Visualize how β (gate temperature) and λ values affect the blending |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{dalar2026ugtc, |
| author = {Dalar, Yağız Ekrem}, |
| title = {{UGTC}: Uncertainty-Gated Temporal Credit}, |
| year = {2026}, |
| publisher = {Zenodo}, |
| doi = {10.5281/zenodo.19715116}, |
| url = {https://doi.org/10.5281/zenodo.19715116}, |
| note = {Accepted — Ulysseus Young Explorers in Science (UYES) Journal. |
| Journal DOI forthcoming.} |
| } |
| ``` |
|
|
