|
|
--- |
|
|
license: mit |
|
|
task_categories: |
|
|
- reinforcement-learning |
|
|
tags: |
|
|
- hierarchical-rl |
|
|
- maxq |
|
|
- value-decomposition |
|
|
- offline-rl |
|
|
- trajectories |
|
|
pretty_name: Hierarchical Value Decomposition (MAXQ) — Rebuttal Dataset |
|
|
size_categories: |
|
|
- 1K<n<10K |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
|
|
|
# Hierarchical Value Decomposition (MAXQ) — Rebuttal Dataset |
|
|
|
|
|
[](https://huggingface.co/datasets/<your-username>/hvd-maxq-rebuttal) |
|
|
[](LICENSE) |
|
|
|
|
|
**🔗 Access Dataset:** https://huggingface.co/datasets/<your-username>/hvd-maxq-rebuttal |
|
|
|
|
|
--- |
|
|
|
|
|
## 📊 Dataset Overview |
|
|
|
|
|
本数据集用于支撑论文“Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition”相关的复现实验与 rebuttal，对外提供分层任务轨迹、层级边界标注、奖励与价值分解信息、训练日志与评估结果。[JAIR/2000][1]，[arXiv][5] |
|
|
|
|
|
### 关键内容 |
|
|
|
|
|
- 分层轨迹：按子任务/子例程划分的`state, action, reward, next_state, done`序列 |
|
|
- 层级标注：`hierarchy_level`、`subtask_id`、进入/退出条件、抽象状态信息 |
|
|
- 价值分解：用于 MAXQ 的加性价值分解字段与局部回报统计 |
|
|
- 训练/评估：不同随机种子与超参数的训练日志、评估回报与方差 |
|
|
|
|
|
### 关键统计（待补全） |
|
|
|
|
|
- 任务数：<填写数量> |
|
|
- 总轨迹数：<填写数量> |
|
|
- 数据体量：<填写GB> |
|
|
- 训练随机种子：<seed 列表> |
|
|
- 评估指标：平均回报 ± 方差（按任务/层级） |
|
|
|
|
|
### 数据模态覆盖（待补全） |
|
|
|
|
|
- 轨迹文件：<格式，如 npz/hdf5/jsonl> |
|
|
- 层级标注：<文件或内嵌字段> |
|
|
- 日志与模型引用：<日志文件>；策略权重建议在模型仓库发布并在此引用 |
|
|
|
|
|
--- |
|
|
|
|
|
## 🚀 Quick Start |
|
|
|
|
|
### 1) 仅加载数据集元数据 |
|
|
|
|
|
```python |
|
|
from datasets import load_dataset |
|
|
|
|
|
dataset = load_dataset("<your-username>/hvd-maxq-rebuttal") |
|
|
episode = dataset['train'][0] |
|
|
print(episode['episode_id']) |
|
|
print(episode['env_name']) |
|
|
print(len(episode['transitions'])) |
|
|
``` |
|
|
|
|
|
### 2) 按需下载特定文件 |
|
|
|
|
|
```python |
|
|
from huggingface_hub import hf_hub_download |
|
|
|
|
|
traj_path = hf_hub_download( |
|
|
repo_id="<your-username>/hvd-maxq-rebuttal", |
|
|
filename="trajectories/<episode_id>.npz", |
|
|
repo_type="dataset", |
|
|
) |
|
|
|
|
|
hier_path = hf_hub_download( |
|
|
repo_id="<your-username>/hvd-maxq-rebuttal", |
|
|
filename="hierarchy/<episode_id>.json", |
|
|
repo_type="dataset", |
|
|
) |
|
|
``` |
|
|
|
|
|
### 3) 克隆完整数据集（含大型文件） |
|
|
|
|
|
```bash |
|
|
git lfs install |
|
|
git clone https://huggingface.co/datasets/<your-username>/hvd-maxq-rebuttal |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📚 数据字段说明 |
|
|
|
|
|
每个`episode`包含： |
|
|
|
|
|
### 元数据字段 |
|
|
|
|
|
- `episode_id`: 唯一标识 |
|
|
- `env_name`: 环境或任务名称 |
|
|
- `hierarchy_spec`: 层级结构摘要（JSON 字符串或对象） |
|
|
- `seed`: 随机种子 |
|
|
- `algo`: 训练算法（如 MAXQ-Q） |
|
|
- `hyperparams`: 关键超参数摘要 |
|
|
|
|
|
### 序列数据 |
|
|
|
|
|
- `transitions`: 列表项包含 `state`, `action`, `reward`, `next_state`, `done` |
|
|
- `subtasks`: 子任务序列与边界，含 `subtask_id`, `level`, `enter_t`, `exit_t` |
|
|
- `value_decomp`: 分解后的价值或回报统计（用于加性分解的相关量） |
|
|
|
|
|
--- |
|
|
|
|
|
## 💡 使用示例 |
|
|
|
|
|
### 1) 浏览与筛选 |
|
|
|
|
|
```python |
|
|
from datasets import load_dataset |
|
|
|
|
|
ds = load_dataset("<your-username>/hvd-maxq-rebuttal") |
|
|
train = ds['train'] |
|
|
filtered = train.filter(lambda x: x['env_name'] == '<env>' and x['seed'] == 0) |
|
|
print(len(filtered)) |
|
|
``` |
|
|
|
|
|
### 2) 构建层级滚动回放 |
|
|
|
|
|
```python |
|
|
episode = train[0] |
|
|
for st in episode['subtasks']: |
|
|
seg = episode['transitions'][st['enter_t']:st['exit_t']] |
|
|
# 在此进行子任务级评估或可视化 |
|
|
``` |
|
|
|
|
|
### 3) 价值分解校验（示例占位） |
|
|
|
|
|
```python |
|
|
import numpy as np |
|
|
|
|
|
vd = episode['value_decomp'] |
|
|
_ = np.array(vd['local_returns']).sum() |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 🎯 适用场景 |
|
|
|
|
|
- 分层强化学习复现与评估（MAXQ 框架） |
|
|
- 离线强化学习与价值分解方法研究 |
|
|
- 子任务/抽象状态设计与层级策略分析 |
|
|
- 与模型仓库中策略权重联合使用进行端到端评测 |
|
|
|
|
|
--- |
|
|
|
|
|
## 📖 数据集细节（待补全） |
|
|
|
|
|
### 数据来源与生成 |
|
|
|
|
|
- 环境：<列出实验环境/任务> |
|
|
- 采集流程：<说明采集管线、抽象/子任务划分策略> |
|
|
- 训练配置：<训练步数、学习率、探索策略等> |
|
|
|
|
|
### 评估设置 |
|
|
|
|
|
- 指标：平均回报、方差、收敛步数等 |
|
|
- 重复次数与随机种子：<填写> |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔐 隐私与合规 |
|
|
|
|
|
- 数据来源于仿真，未包含个人隐私信息 |
|
|
- 许可证与使用限制：见下方 License |
|
|
|
|
|
--- |
|
|
|
|
|
## ⚠️ 已知局限（待补全） |
|
|
|
|
|
- 层级标注依赖特定实现，跨环境迁移需校验 |
|
|
- 抽象状态选择可能影响价值分解稳定性 |
|
|
|
|
|
--- |
|
|
|
|
|
## 📜 License |
|
|
|
|
|
MIT License - 见 `LICENSE` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📚 引用 |
|
|
|
|
|
### 论文 |
|
|
|
|
|
> Dietterich, T. G. (2000). Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition. Journal of Artificial Intelligence Research. [JAIR][1], [arXiv][5] |
|
|
|
|
|
### 数据集 |
|
|
|
|
|
```bibtex |
|
|
@dataset{hvd_maxq_rebuttal_2025, |
|
|
title={Hierarchical Value Decomposition (MAXQ) — Rebuttal Dataset}, |
|
|
author={<your name>}, |
|
|
year={2025}, |
|
|
publisher={HuggingFace}, |
|
|
url={https://huggingface.co/datasets/<your-username>/hvd-maxq-rebuttal} |
|
|
} |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 👥 Authors |
|
|
|
|
|
<your name(s)> |
|
|
|
|
|
--- |
|
|
|
|
|
## 📞 Contact & Contributions |
|
|
|
|
|
- 在数据集页面开启讨论或提 Issue |
|
|
- 联系邮箱：<your email> |
|
|
|
|
|
--- |
|
|
|
|
|
## 📋 Changelog |
|
|
|
|
|
- v1.0 (2025): 首次公开发布，含核心层级轨迹与评估结果 |
|
|
|
|
|
--- |
|
|
|
|
|
[1]: https://jair.org/index.php/jair/article/view/10266 |
|
|
[5]: https://arxiv.org/abs/cs/9905014 |
