---
title: UGTC — Uncertainty-Gated Temporal Credit
emoji: 🎯
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: true
license: mit
tags:
  - reinforcement-learning
  - advantage-estimation
  - temporal-credit
  - uncertainty
  - actor-critic
  - PPO
  - TD3
  - SAC
---

# UGTC: Uncertainty-Gated Temporal Credit

**Interactive demo and educational interface for the UGTC paper.**

> **Paper:** [10.5281/zenodo.19715116](https://doi.org/10.5281/zenodo.19715116)  
> **Accepted:** Ulysseus Young Explorers in Science (UYES) Journal · Journal DOI forthcoming  
> **GitHub:** [ethosoftai/ugtc](https://github.com/ethosoftai/ugtc)  
> **Docs:** [ethosoftai.github.io/ugtc](https://ethosoftai.github.io/ugtc)

---

## What is UGTC?

UGTC (Uncertainty-Gated Temporal Credit) is a **plug-in advantage estimator** for actor-critic reinforcement learning.

It resolves the bias–variance trade-off in temporal credit assignment by:

1. Maintaining **two critics** with different GAE λ values:
   - **Fast critic** (single network, λ=0.80): low variance, higher bias
   - **Slow ensemble** (M=3 networks, λ=0.99): lower bias, higher variance

2. Measuring **uncertainty** as the disagreement (std deviation) among slow ensemble members

3. Using a **sigmoid gate** to blend the two advantage estimates:
   - Low uncertainty → trust the slow (accurate) estimate
   - High uncertainty → trust the fast (stable) estimate

## Explore in This Space

The demo provides:
- **Gate Visualizer**: See how the uncertainty gate u(s) responds to different ensemble disagreement levels
- **Algorithm Comparison**: Side-by-side view of UGTC-PPO vs vanilla PPO objective
- **Math Explorer**: Interactive walkthrough of the UGTC equations
- **Pseudocode Viewer**: Step-by-step algorithm pseudocode for PPO, TD3, and SAC integrations
- **Hyperparameter Impact**: Visualize how β (gate temperature) and λ values affect the blending

## Citation

```bibtex
@misc{dalar2026ugtc,
  author    = {Dalar, Yağız Ekrem},
  title     = {{UGTC}: Uncertainty-Gated Temporal Credit},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19715116},
  url       = {https://doi.org/10.5281/zenodo.19715116},
  note      = {Accepted — Ulysseus Young Explorers in Science (UYES) Journal.
               Journal DOI forthcoming.}
}
```