huggsook/connect-ai-Humanoid-v5

huggsook/connect-ai/Humanoid-v5

---

🎬 Agent Simulation & Training Preview

🎥 3D Humanoid-v5 PPO Bipedal Locomotion & Dynamic Balance Simulation

---

📌 Project Overview

**huggsook/connect-ai/Humanoid-v5**는 Gymnasium의 고난도 3D 다관절 보행 로봇 환경인 **Humanoid-v5**에서 PPO(Proximal Policy Optimization) 알고리즘을 통해 17개 관절 모터를 제어하여 안정적인 직립 보행(Bipedal Locomotion)을 학습한 강화학습 프로젝트입니다.

✨ Key Features

• 3D Real-time Physics Simulator: Three.js 기반 17-DoF 로봇 관절 모터, CoM(무게 중심점) 추적, 발 지면 접촉력 및 외란(밀기/바람) 시뮬레이션
• Full Process Visualization: 실시간 에피소드 보상(Reward) 곡선, 10-Ep 이동 평균선, Critic Value Loss & Actor Policy Loss, 정책 엔트로피 실시간 모니터링
• Neural Network Activation Inspector: 376개 감각 관측 입력 ➔ 은닉층 ➔ 17개 관절 토크 출력 신호 실시간 시각화
• One-Click Package Export: 학습된 가중치, 정규화 통계, CSV 로그, 3D 뷰포트 스크린샷 일괄 ZIP 압축 다운로드 지원

---

⚙️ Hyperparameters & Training Setup

Parameter	Value	Description
Environment	Humanoid-v5	3D Physics Bipedal Robot (17 Actions, 376 Observations)
Algorithm	PPO (MlpPolicy)	Actor-Critic with Generalized Advantage Estimation
Learning Rate	3e-4	Adam optimizer with linear/constant schedule
Total Timesteps	100,000	Steps trained
Batch Size	64	Mini-batch size for surrogate loss
n_steps	2048	Rollout buffer steps per update
n_epochs	10	Optimization epochs per update
Gamma ($\gamma$)	0.99	Discount factor
GAE Lambda ($\lambda$)	0.95	Generalized Advantage Estimation factor
Clip Range ($\epsilon$)	0.2	PPO surrogate objective clipping parameter
Entropy Coef	0.0	Exploration bonus coefficient
Normalization	VecNormalize	Observation & Reward normalization enabled

---

🚀 How to Load and Evaluate in Python

```python import gymnasium as gym from stable_baselines3 import PPO from stable_baselines3.common.vec_env import DummyVecEnv, VecNormalize

1. Create Humanoid Environment

def make_env(): return gym.make("Humanoid-v5", render_mode="human")

env = DummyVecEnv([make_env])

env = VecNormalize.load("models/vec_normalize.pkl", env)

2. Load Model from Hugging Face Hub

model = PPO.load("ppo_humanoid.zip", env=env)

3. Test & Render

obs = env.reset()

for _ in range(1000):

action, _states = model.predict(obs, deterministic=True)

obs, rewards, dones, info = env.step(action)

if dones:

obs = env.reset()

```

---

📦 Repository Structure

``` . ├── README.md # Hugging Face Model Card & Overview ├── humanoid_preview.mp4 # Agent Walk Simulation Video ├── index.html # Interactive 3D Web Studio (Spaces Ready) ├── styles.css # Dark Cyberpunk Glassmorphism UI ├── models/ │ ├── ppo_humanoid_weights.json # Actor-Critic Network Weights │ └── vec_normalize_stats.json # Env Normalization Parameters ├── logs/ │ └── training_metrics.csv # Full Training Trajectory (Reward/Loss) └── js/ ├── humanoid_sim.js # 3D Kinematics & Physics Engine ├── charts_manager.js # Chart.js Live Analytics Manager ├── training_engine.js # PPO Forward & Neural Inspector └── export_manager.js # JSZip Package Exporter ```

---

Developed by @huggsook with Connect-AI Studio