Hugging Face's logo

huggsook
/
ppo-pusher-v5

Reinforcement Learning
stable-baselines3
ppo
gymnasium
mujoco
threejs
simulation
Model card Files
xet
 Community
ppo-pusher-v5 / index.html
huggsook's picture
huggsook
Upload folder using huggingface_hub
479df53 verified
Raw
History Blame Contribute Delete
14.3 kB
<!DOCTYPE html>
<html lang="ko">
<head>
 <meta charset="UTF-8">
 <meta name="viewport" content="width=device-width, initial-scale=1.0">
 <title>PPO Pusher-v5 | RL 3D Visualizer & Interactive Simulation</title>
<!-- Google Fonts -->
 <link rel="preconnect" href="https://fonts.googleapis.com">
 <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
 <link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600;700&family=Outfit:wght@400;500;600;700;800&display=swap" rel="stylesheet">
<!-- CSS -->
 <link rel="stylesheet" href="style.css">
<!-- External Libraries (CDN) -->
 <script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
 <script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/controls/OrbitControls.js"></script>
 <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
 <script src="https://cdnjs.cloudflare.com/ajax/libs/jszip/3.10.1/jszip.min.js"></script>
</head>
<body>
<!-- Top Navigation Header -->
 <header class="app-header">
 <div class="logo-group">
 <div class="logo-badge">
 <svg fill="none" stroke="currentColor" stroke-width="2.2" viewBox="0 0 24 24">
 <path d="M12 2v4M12 18v4M4.93 4.93l2.83 2.83M16.24 16.24l2.83 2.83M2 12h4M18 12h4M4.93 19.07l2.83-2.83M16.24 7.76l2.83-2.83"/>
 </svg>
 </div>
 <div>
 <span class="brand-title">PPO RL Studio</span>
 <span class="env-tag">Gymnasium / Pusher-v5</span>
 </div>
 </div>
<div class="header-actions">
 <button id="btn-view-code" class="btn btn-outline">
 <svg width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
 <polyline points="16 18 22 12 16 6"/><polyline points="8 6 2 12 8 18"/>
 </svg>
 <span>PyTorch Code</span>
 </button>
<button id="btn-download-zip" class="btn btn-primary">
 <svg width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
 <path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
 <polyline points="7 10 12 15 17 10"/><line x1="12" y1="15" x2="12" y2="3"/>
 </svg>
 <span>Download Bundle (.zip)</span>
 </button>
 </div>
 </header>
<!-- Main Container -->
 <main class="main-container">
<!-- LEFT: 3D Simulation & Checkpoints -->
 <section class="left-column">
 <div class="glass-card">
 <div class="card-header">
 <div class="card-title">
 <svg width="20" height="20" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
 <polygon points="12 2 2 7 12 12 22 7 12 2"/><polyline points="2 17 12 22 22 17"/><polyline points="2 12 12 17 22 12"/>
 </svg>
 <span>3D Interactive Physics Simulator</span>
 </div>
<div style="display: flex; align-items: center; gap: 10px;">
 <div id="status-badge" class="hud-badge" style="position: static; padding: 4px 10px;">
 <span id="status-indicator" class="status-dot"></span>
 <span id="status-text" style="font-size: 0.75rem;">Agent Ready (Inference)</span>
 </div>
 <button id="btn-live-train" class="btn btn-cyan" style="padding: 5px 12px; font-size: 0.8rem;">
 ⚑ Start Training
 </button>
 </div>
 </div>
<!-- 3D Viewport -->
 <div class="sim-viewport-wrapper">
 <canvas id="sim-canvas"></canvas>
<!-- Top HUD -->
 <div class="sim-hud-overlay">
 <div class="hud-badge">
 <span id="hud-episode">EP: 1</span>
 <span>|</span>
 <span id="hud-step">STEP: 0/100</span>
 <span>|</span>
 <span id="hud-reward" style="color: var(--accent-cyan);">R: 0.00</span>
 </div>
 <div class="hud-badge" style="font-size: 0.72rem; color: var(--text-muted);">
 πŸ’‘ 팁: μ΄ˆλ‘μƒ‰ λͺ©ν‘œ(Goal) λ˜λŠ” λ…Έλž€μƒ‰ 퍽(Puck)을 마우슀둜 λ“œλž˜κ·Έν•˜μ„Έμš”
 </div>
 </div>
<!-- Camera Presets -->
 <div class="camera-presets">
 <button class="btn-icon cam-btn active" data-cam="iso" title="Isometric View">3D</button>
 <button class="btn-icon cam-btn" data-cam="top" title="Top-Down View">Top</button>
 <button class="btn-icon cam-btn" data-cam="side" title="Side View">Side</button>
 </div>
<!-- Floating Playback Bar -->
 <div class="sim-controls-floating">
 <button id="btn-play-pause" class="btn-icon" title="Play/Pause">
 <svg width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
 <rect x="6" y="4" width="4" height="16"/><rect x="14" y="4" width="4" height="16"/>
 </svg>
 </button>
 <button id="btn-reset" class="btn-icon" title="Reset Episode">
 <svg width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
 <path d="M3 12a9 9 0 1 0 9-9 9.75 9.75 0 0 0-6.74 2.74L3 8"/>
 <path d="M3 3v5h5"/>
 </svg>
 </button>
 <div style="display: flex; align-items: center; gap: 6px; padding: 0 8px;">
 <span style="font-size: 0.75rem; color: var(--text-muted);">Speed:</span>
 <input id="speed-slider" type="range" min="0.2" max="3.0" step="0.2" value="1.0" style="width: 70px;">
 <span id="speed-val" style="font-family: var(--font-mono); font-size: 0.75rem; color: var(--accent-cyan); width: 30px;">1.0x</span>
 </div>
 </div>
 </div>
<!-- Checkpoint Timeline Selection -->
 <div style="margin-top: 1.25rem;">
 <div style="font-size: 0.85rem; font-weight: 700; color: var(--text-muted); margin-bottom: 8px; display: flex; justify-content: space-between;">
 <span>LEARNING TIMELINE CHECKPOINTS (νƒ€μž„λ¨Έμ‹  μ •μ±… μ „ν™˜)</span>
 <span style="color: var(--accent-cyan); font-family: var(--font-mono);">Current: Step 50k</span>
 </div>
<div class="checkpoint-nav">
 <div class="checkpoint-btn" data-step="0">
 <div class="cp-step">Step 0</div>
 <div class="cp-label">Untrained</div>
 <div class="cp-desc">λ¬΄μž‘μœ„ 탐색 & ν—ˆκ³΅ νœ˜λ‘λ¦„</div>
 </div>
<div class="checkpoint-btn" data-step="10000">
 <div class="cp-step">Step 10k</div>
 <div class="cp-label">Novice</div>
 <div class="cp-desc">퍽 μ ‘κ·Ό μ‹œλ„ & λΉ—λ§žμΆ€</div>
 </div>
<div class="checkpoint-btn" data-step="25000">
 <div class="cp-step">Step 25k</div>
 <div class="cp-label">Intermediate</div>
 <div class="cp-desc">νƒ€κ²Ÿ λ°©ν–₯ μ‘°μ€€ & μ˜€λ²„μŠ›</div>
 </div>
<div class="checkpoint-btn active" data-step="50000">
 <div class="cp-step">Step 50k</div>
 <div class="cp-label">Master</div>
 <div class="cp-desc">μ •λ°€ ꢀ적 & λͺ©ν‘œ 지점 μ•ˆμ°©</div>
 </div>
 </div>
 </div>
</div>
 </section>
<!-- RIGHT: Analytics & Hyperparameters -->
 <section class="right-column">
<!-- Metrics Overview -->
 <div class="glass-card">
 <div class="card-header">
 <div class="card-title">
 <svg width="20" height="20" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
 <line x1="18" y1="20" x2="18" y2="10"/><line x1="12" y1="20" x2="12" y2="4"/><line x1="6" y1="20" x2="6" y2="14"/>
 </svg>
 <span>Live Training Telemetry</span>
 </div>
<nav class="tab-nav" style="margin-bottom: 0; border: none; padding: 0;">
 <button class="tab-btn active" data-tab="analytics">Analytics</button>
 <button class="tab-btn" data-tab="hyperparams">Hyperparameters</button>
 </nav>
 </div>
<!-- Metric KPI Cards -->
 <div class="metrics-row">
 <div class="metric-box">
 <div class="metric-title">Current Reward</div>
 <div id="val-reward" class="metric-value">-1.42</div>
 <div class="metric-delta">β–² +82.5 vs initial</div>
 </div>
 <div class="metric-box">
 <div class="metric-title">Success Rate</div>
 <div id="val-success-rate" class="metric-value" style="color: var(--accent-green);">96.4%</div>
 <div class="metric-delta">Target radius < 0.25m</div>
 </div>
 <div class="metric-box">
 <div class="metric-title">Distance to Goal</div>
 <div id="val-dist-goal" class="metric-value" style="color: var(--accent-purple);">4.2 cm</div>
 <div class="metric-delta">Puck to Goal</div>
 </div>
 </div>
<!-- TAB 1: Analytics Charts -->
 <div id="tab-analytics" class="tab-pane">
 <div style="margin-bottom: 1rem;">
 <div style="font-size: 0.8rem; font-weight: 600; color: var(--text-muted); margin-bottom: 4px;">
 EPISODE REWARD & 100-MA CONVERGENCE
 </div>
 <div class="chart-container">
 <canvas id="rewardChart"></canvas>
 </div>
 </div>
<div>
 <div style="font-size: 0.8rem; font-weight: 600; color: var(--text-muted); margin-bottom: 4px;">
 POLICY & VALUE LOSS DYNAMICS
 </div>
 <div class="chart-container">
 <canvas id="lossChart"></canvas>
 </div>
 </div>
 </div>
<!-- TAB 2: Hyperparameters -->
 <div id="tab-hyperparams" class="tab-pane" style="display: none;">
 <div class="control-grid">
 <div class="control-item">
 <label class="control-label">
 <span>Learning Rate (LR)</span>
 <span id="hp-lr-val" class="val">0.0003</span>
 </label>
 <input id="hp-lr" class="hp-slider" type="range" min="0.00005" max="0.001" step="0.00005" value="0.0003">
 </div>
<div class="control-item">
 <label class="control-label">
 <span>Batch Size</span>
 <span id="hp-batch-val" class="val">64</span>
 </label>
 <input id="hp-batch" class="hp-slider" type="range" min="32" max="256" step="32" value="64">
 </div>
<div class="control-item">
 <label class="control-label">
 <span>PPO Clip Range (&epsilon;)</span>
 <span id="hp-clip-val" class="val">0.2</span>
 </label>
 <input id="hp-clip" class="hp-slider" type="range" min="0.1" max="0.4" step="0.05" value="0.2">
 </div>
<div class="control-item">
 <label class="control-label">
 <span>Discount Factor (&gamma;)</span>
 <span id="hp-gamma-val" class="val">0.99</span>
 </label>
 <input id="hp-gamma" class="hp-slider" type="range" min="0.9" max="0.999" step="0.005" value="0.99">
 </div>
<div class="control-item">
 <label class="control-label">
 <span>GAE Lambda (&lambda;)</span>
 <span id="hp-gae-val" class="val">0.95</span>
 </label>
 <input id="hp-gae" class="hp-slider" type="range" min="0.8" max="0.99" step="0.01" value="0.95">
 </div>
<div class="control-item">
 <label class="control-label">
 <span>Total Timesteps</span>
 <span class="val">50,000</span>
 </label>
 <input type="range" min="10000" max="100000" step="10000" value="50000" disabled style="opacity: 0.5;">
 </div>
 </div>
<div style="margin-top: 1.25rem; padding: 12px; background: rgba(0, 242, 254, 0.05); border: 1px dashed rgba(0, 242, 254, 0.3); border-radius: var(--radius-sm); font-size: 0.8rem; color: var(--text-muted);">
 ℹ️ ν•˜μ΄νΌνŒŒλΌλ―Έν„° λ³€κ²½ 사항은 상단 <strong>"⚑ Start Training"</strong> μ‹€ν–‰ μ‹œ μ‹€μ‹œκ°„ μ •μ±… 수렴 속도 및 손싀 계산에 μ¦‰μ‹œ λ°˜μ˜λ©λ‹ˆλ‹€.
 </div>
 </div>
</div>
</section>
 </main>
<!-- Modal: Python Training Code -->
 <div id="code-modal" class="modal-backdrop">
 <div class="modal-content">
 <div class="card-header" style="margin-bottom: 0.75rem;">
 <div class="card-title">
 <svg width="20" height="20" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
 <polyline points="16 18 22 12 16 6"/><polyline points="8 6 2 12 8 18"/>
 </svg>
 <span>Stable-Baselines3 PPO Pusher-v5 Training Script</span>
 </div>
 <button id="btn-close-modal" class="btn-icon">βœ•</button>
 </div>
<div class="code-block">import os
import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.monitor import Monitor
from stable_baselines3.common.vec_env import DummyVecEnv
# Headless EGL rendering for MuJoCo
os.environ['MUJOCO_GL'] = 'egl'
def make_env():
 env = gym.make("Pusher-v5", render_mode="rgb_array")
 return Monitor(env, "./logs")
env = DummyVecEnv([make_env])
# Initialize PPO Policy
model = PPO(
 policy="MlpPolicy",
 env=env,
 learning_rate=3e-4,
 n_steps=2048,
 batch_size=64,
 gamma=0.99,
 gae_lambda=0.95,
 clip_range=0.2,
 verbose=1,
 device="auto"
)
# Run Training
print("πŸš€ PPO Pusher-v5 ν•™μŠ΅ μ‹œμž‘...")
model.learn(total_timesteps=50000)
# Save Weights
model.save("./model/ppo_pusher_final.zip")
print("βœ… λͺ¨λΈ μ €μž₯ μ™„λ£Œ: ./model/ppo_pusher_final.zip")</div>
 </div>
 </div>
<!-- Toast Notification Container -->
 <div id="toast-container" class="toast-container"></div>
<!-- Scripts -->
 <script src="simulation.js"></script>
 <script src="dashboard.js"></script>
 <script src="app.js"></script>
</body>
</html>