Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+docs/images/thinning_algo.jpg filter=lfs diff=lfs merge=lfs -text

.github/workflows/docs.yaml

@@ -0,0 +1,59 @@
+name: docs
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+  release:
+    types: [ published ]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        fetch-depth: 0
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.8'
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip setuptools wheel
+        sudo apt-get update
+        sudo apt-get install openjdk-11-jdk
+        sudo apt-get install pandoc
+    - name: Build Sphinx docs
+      run: |
+        pip install tensorflow==2.2.0
+        pip install torch
+        pip install pandas
+        pip install numpy
+        pip install -r requirements-doc.txt
+        cd docs
+        make html
+    # Publish built docs to gh-pages branch.
+    # ===============================
+    - name: Commit documentation changes
+      run: |
+        git clone https://github.com/ant-research/EasyTemporalPointProcess.git --branch gh-pages --single-branch gh-pages
+        cp -r docs/build/html/* gh-pages/
+        cd gh-pages
+        touch .nojekyll
+        git config --local user.email "action@github.com"
+        git config --local user.name "GitHub Action"
+        git add .
+        git commit -m "Update documentation" -a || true
+        # The above command will fail if no changes were present, so we ignore
+        # that.
+    - name: Push changes
+      uses: ad-m/github-push-action@master
+      with:
+        branch: gh-pages
+        directory: gh-pages
+        github_token: ${{ secrets.GITHUB_TOKEN }}
+    # ===============================

.github/workflows/python-publish.yml

@@ -0,0 +1,39 @@
+# This workflow will upload a Python Package using Twine when a release is created
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries
+
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+name: Upload Python Package
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: read
+
+jobs:
+  deploy:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.9'
+    - name: Install dependencies
+      run: |
+        pip install -r requirements.txt
+        pip install wheel
+    - name: Build package
+      run: python setup.py sdist bdist_wheel
+    - name: Publish package
+      uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
+      with:
+        user: __token__
+        password: ${{ secrets.PYPI_API_TOKEN }}

.gitignore

@@ -0,0 +1,95 @@
+# python build
+build/
+dist/
+easy_tpp.egg-info/
+*.egg-info/
+
+# python temp
+*.pyc
+*.pyo
+*.pyd
+__pycache__/
+*.so
+*.egg
+
+# proto
+protoc
+protoc-3.4.0.tar.gz
+*_pb2.py
+
+# misc
+experiments/
+log/
+logs/
+*.swp
+*.swo
+.vscode/
+.idea/
+
+# OS files
+.DS_Store
+.DS_Store?
+._*
+.Spotlight-V100
+.Trashes
+ehthumbs.db
+Thumbs.db
+
+# IDE
+.vscode/
+.idea/
+*.sublime-project
+*.sublime-workspace
+
+# Testing
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+.tox/
+
+# Checkpoints and outputs
+examples/checkpoints/*
+notebooks/checkpoints/*
+results/
+outputs/
+*.pth
+*.pt
+*.ckpt
+
+# Data files (large files should not be committed)
+examples/data/*
+!examples/data/.gitkeep
+*.json.gz
+*.pkl
+*.h5
+*.hdf5
+
+# Large cascade data files (use Git LFS or external storage)
+data/cascades/information_cascade*.json
+data/cascades/*.json
+!data/cascades/.gitkeep
+
+# Model files (use Git LFS or Hugging Face Hub)
+*.bin
+*.safetensors
+*.onnx
+
+# Temporary files
+*.tmp
+*.temp
+*.bak
+*~
+
+# Jupyter Notebook checkpoints
+.ipynb_checkpoints/
+
+# Environment
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/

ADDITIONS_README.md

@@ -0,0 +1,71 @@
+# EasyTPP 新增功能说明
+
+本仓库在原始 EasyTPP 基础上新增了用于处理信息级联数据的指标计算功能。
+
+## 🆕 新增文件
+
+### 核心功能
+- **`compute_cascade_metrics.py`**: 计算级联指标的主脚本
+  - 情感得分 (Sentiment Score)
+  - 情感偏差 (Sentiment Deviation)  
+  - 语境偏差 (Contextual Deviation)
+  - 困惑度 (Perplexity)
+
+### 文档和工具
+- **`COMPUTE_METRICS_README.md`**: 详细使用说明
+- **`HF_UPLOAD_GUIDE.md`**: Hugging Face 上传指南
+- **`UPLOAD_CHECKLIST.md`**: 上传检查清单（自动生成）
+- **`cleanup_for_hf.py`**: 清理脚本，准备上传
+- **`example_compute_metrics.sh`**: 使用示例脚本
+- **`requirements_compute_metrics.txt`**: 额外依赖包
+
+## 🚀 快速开始
+
+### 1. 安装依赖
+
+```bash
+pip install -r requirements.txt
+pip install -r requirements_compute_metrics.txt
+```
+
+### 2. 运行指标计算
+
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade information_cascade.json \
+    --output output_with_metrics.json \
+    --batch_size 32
+```
+
+详细说明请参考 `COMPUTE_METRICS_README.md`
+
+## 📦 上传到 Hugging Face
+
+1. 运行清理脚本：
+```bash
+python cleanup_for_hf.py
+```
+
+2. 按照 `HF_UPLOAD_GUIDE.md` 的说明上传
+
+## 🔗 相关文档
+
+- [指标计算说明](COMPUTE_METRICS_README.md)
+- [上传指南](HF_UPLOAD_GUIDE.md)
+- [原始 EasyTPP README](README.md)
+
+## 📝 使用场景
+
+这些新增功能主要用于：
+- 分析社交媒体信息级联（如微博转发、评论）
+- 计算文本的情感特征和语义偏差
+- 为 TPP 模型提供额外的特征输入
+
+## ⚙️ 与 EasyTPP 集成
+
+计算出的指标可以用于：
+- `RobertTPPDataset`: 加载包含语义和偏差特征的数据
+- `RobertEventTokenizer`: 处理自定义特征
+- `TorchRobotTHP`: 使用语义和偏差特征的 TPP 模型
+
+参考 `examples/train_robot_thp_with_features.py` 了解完整示例。

CLEANUP_SUMMARY.md

@@ -0,0 +1,164 @@
+# 文件夹整理总结
+
+## ✅ 整理完成时间
+2025-01-19
+
+## 📊 文件夹信息
+
+- **位置**: `/Users/chenshuyi/Downloads/EasyTemporalPointProcess-main`
+- **大小**: 1.3MB
+- **状态**: ✅ 已整理，可以上传
+
+## 🧹 已完成的清理工作
+
+### 1. 更新 .gitignore
+- ✅ 添加了 Python 缓存文件模式
+- ✅ 添加了 IDE 配置文件排除
+- ✅ 添加了 OS 系统文件排除
+- ✅ 添加了测试和构建文件排除
+- ✅ 添加了数据文件和模型文件排除规则
+
+### 2. 创建清理脚本
+- ✅ `cleanup_for_hf.py` - 自动清理脚本
+- ✅ 已运行，确认无需要删除的文件
+
+### 3. 文件检查
+- ✅ 无大文件（>50MB）
+- ✅ 无敏感信息
+- ✅ 无临时文件
+
+## 📁 文件结构
+
+```
+EasyTemporalPointProcess-main/
+├── 📄 核心代码
+│   ├── easy_tpp/              # EasyTPP 核心库
+│   ├── examples/               # 示例代码
+│   ├── notebooks/              # Jupyter notebooks
+│   └── tests/                  # 测试代码
+│
+├── 🆕 新增功能（级联指标计算）
+│   ├── compute_cascade_metrics.py      # 主计算脚本
+│   ├── COMPUTE_METRICS_README.md       # 使用说明
+│   ├── requirements_compute_metrics.txt # 额外依赖
+│   └── example_compute_metrics.sh      # 示例脚本
+│
+├── 📚 文档
+│   ├── README.md                      # 原始 README
+│   ├── ADDITIONS_README.md             # 新增功能说明
+│   ├── HF_UPLOAD_GUIDE.md             # Hugging Face 上传指南
+│   ├── QUICK_START_HF.md              # 快速开始指南
+│   ├── UPLOAD_CHECKLIST.md            # 上传检查清单
+│   └── CLEANUP_SUMMARY.md             # 本文件
+│
+├── 🛠️ 工具脚本
+│   ├── cleanup_for_hf.py              # 清理脚本
+│   └── setup.py                        # 安装脚本
+│
+└── ⚙️ 配置文件
+    ├── .gitignore                      # Git 忽略规则（已更新）
+    ├── requirements.txt                 # 基础依赖
+    ├── requirements_compute_metrics.txt # 指标计算依赖
+    └── setup.cfg                       # 安装配置
+```
+
+## 📋 新增文件列表
+
+### 核心功能
+1. `compute_cascade_metrics.py` (19.5 KB)
+   - 计算情感得分、情感偏差、语境偏差、困惑度
+
+### 文档
+2. `COMPUTE_METRICS_README.md` (5.9 KB)
+   - 详细的指标计算使用说明
+
+3. `HF_UPLOAD_GUIDE.md` (3.7 KB)
+   - Hugging Face 上传完整指南
+
+4. `ADDITIONS_README.md` (1.9 KB)
+   - 新增功能概述
+
+5. `QUICK_START_HF.md` (2.3 KB)
+   - 快速上传指南
+
+6. `UPLOAD_CHECKLIST.md` (3.0 KB)
+   - 上传检查清单（自动生成）
+
+7. `CLEANUP_SUMMARY.md` (本文件)
+   - 整理总结
+
+### 工具和配置
+8. `cleanup_for_hf.py` (7.8 KB)
+   - 自动清理脚本
+
+9. `example_compute_metrics.sh` (1.2 KB)
+   - 使用示例脚本
+
+10. `requirements_compute_metrics.txt` (266 B)
+    - 指标计算所需依赖
+
+## 🎯 下一步操作
+
+### 1. 上传到 Hugging Face
+
+```bash
+# 安装 CLI
+pip install huggingface_hub
+
+# 登录
+huggingface-cli login
+
+# 创建仓库（在网页上）
+# https://huggingface.co/new
+
+# 上传
+cd /Users/chenshuyi/Downloads/EasyTemporalPointProcess-main
+huggingface-cli upload <username>/<repo-name> . --repo-type dataset
+```
+
+### 2. 在云电脑上下载
+
+```bash
+huggingface-cli download <username>/<repo-name> --local-dir ./EasyTPP
+```
+
+### 3. 使用新功能
+
+```bash
+cd EasyTPP
+pip install -r requirements.txt
+pip install -r requirements_compute_metrics.txt
+
+python compute_cascade_metrics.py \
+    --input_cascade information_cascade.json \
+    --output output_with_metrics.json
+```
+
+## ✅ 检查清单
+
+- [x] 清理临时文件
+- [x] 更新 .gitignore
+- [x] 检查大文件
+- [x] 检查敏感信息
+- [x] 创建上传指南
+- [x] 创建使用文档
+- [x] 验证文件结构
+- [ ] 上传到 Hugging Face（待执行）
+- [ ] 在云电脑上测试（待执行）
+
+## 📝 注意事项
+
+1. **文件大小**: 1.3MB，无需 Git LFS
+2. **许可证**: 保持原始 Apache 2.0 许可证
+3. **依赖**: 确保所有依赖都在 requirements 文件中
+4. **文档**: 所有新增功能都有详细文档
+
+## 🔗 相关链接
+
+- [Hugging Face](https://huggingface.co/)
+- [Hugging Face CLI 文档](https://huggingface.co/docs/huggingface_hub/guides/cli)
+- [原始 EasyTPP 项目](https://github.com/ant-research/EasyTemporalPointProcess)
+
+---
+
+**整理完成！可以开始上传了！** 🚀

COMPUTE_METRICS_README.md

@@ -0,0 +1,191 @@
+# 计算级联指标使用说明
+
+本脚本用于计算信息级联数据的情感得分、情感deviation、contextual deviation和perplexity。
+
+## 功能说明
+
+脚本 `compute_cascade_metrics.py` 会处理以下两个JSON文件：
+- `information_cascade.json`: 包含完整级联数据（原帖、评论、转发）
+- `information_cascade_original_posts.json`: 包含原帖数据（可选）
+
+计算以下指标：
+1. **情感得分 (Sentiment Score)**: 文本的情感倾向得分
+2. **情感偏差 (Sentiment Deviation)**: 相对于原帖的情感偏差
+3. **语境偏差 (Contextual Deviation)**: 相对于原帖的语义偏差
+4. **困惑度 (Perplexity)**: 文本的语言模型困惑度
+
+## 安装依赖
+
+在云电脑上安装必要的依赖：
+
+```bash
+pip install torch transformers numpy tqdm
+```
+
+如果需要使用GPU：
+```bash
+pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
+```
+
+## 使用方法
+
+### 基本用法（使用默认模型）
+
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade information_cascade.json \
+    --output output_with_metrics.json \
+    --batch_size 32
+```
+
+### 完整用法（指定所有模型）
+
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade information_cascade.json \
+    --input_original information_cascade_original_posts.json \
+    --output output_with_metrics.json \
+    --bert_model bert-base-chinese \
+    --sentiment_model <情感分析模型路径> \
+    --perplexity_model <语言模型路径> \
+    --batch_size 32 \
+    --max_length 512 \
+    --device cuda
+```
+
+### 参数说明
+
+- `--input_cascade`: **必需**，输入级联JSON文件路径
+- `--input_original`: 可选，输入原帖JSON文件路径
+- `--output`: **必需**，输出JSON文件路径
+- `--bert_model`: BERT模型名称或路径（默认: `bert-base-chinese`）
+- `--sentiment_model`: 情感分析模型路径（可选，不提供则使用简化方法）
+- `--perplexity_model`: 语言模型路径（可选，不提供则使用简化方法）
+- `--batch_size`: 批处理大小（默认: 32）
+- `--max_length`: 最大序列长度（默认: 512）
+- `--device`: 计算设备，`cuda` 或 `cpu`（默认: 自动选择）
+- `--max_cascades`: 最大处理级联数量（用于测试，默认: 处理所有）
+
+## 输出格式
+
+处理后的JSON文件会在每个节点中添加以下字段：
+
+### 原帖 (`post_info`)
+```json
+{
+  "post_info": {
+    "content": "原帖内容",
+    "embedding": [0.1, 0.2, ...],        // BERT语义向量 (768维)
+    "sentiment_score": 0.7,              // 情感得分
+    "perplexity": 15.3                   // 困惑度
+  }
+}
+```
+
+### 评论 (`comment_tree`)
+```json
+{
+  "comment_tree": {
+    "comment_id": {
+      "content": "评论内容",
+      "embedding": [0.1, 0.2, ...],
+      "sentiment_score": 0.6,
+      "perplexity": 12.5,
+      "contextual_deviation": 0.25,      // 语境偏差
+      "sentiment_deviation": 0.1         // 情感偏差
+    }
+  }
+}
+```
+
+### 转发 (`repost_chain`)
+```json
+{
+  "repost_chain": [
+    {
+      "forward_text": "转发内容",
+      "comment_content": "评论内容",
+      "embedding": [0.1, 0.2, ...],
+      "sentiment_score": 0.5,
+      "perplexity": 18.2,
+      "contextual_deviation": 0.35,
+      "sentiment_deviation": 0.2
+    }
+  ]
+}
+```
+
+## 模型选择建议
+
+### BERT模型
+- 中文文本：`bert-base-chinese`
+- 英文文本：`bert-base-uncased`
+- 自定义模型：提供本地路径
+
+### 情感分析模型
+- 中文：可以使用 `uer/roberta-base-finetuned-chinanews-chinese` 或其他中文情感分析模型
+- 英文：可以使用 `nlptown/bert-base-multilingual-uncased-sentiment` 等
+- 如果不提供，脚本会使用基于关键词的简化方法
+
+### 困惑度模型
+- 中文：可以使用 `gpt2-chinese` 或其他中文语言模型
+- 英文：可以使用 `gpt2` 等
+- 如果不提供，脚本会使用基于词汇多样性的简化方法
+
+## 注意事项
+
+1. **大文件处理**: 如果JSON文件很大，处理时间可能较长。建议：
+   - 使用GPU加速（`--device cuda`）
+   - 调整批处理大小（`--batch_size`）
+   - 先用 `--max_cascades` 测试少量数据
+
+2. **内存使用**: 
+   - BERT模型需要较多内存
+   - 如果内存不足，减小 `--batch_size`
+
+3. **简化方法**: 
+   - 如果不提供情感分析模型或困惑度模型，脚本会使用简化的启发式方法
+   - 简化方法的结果可能不如专业模型准确，但计算速度快
+
+4. **数据格式**: 
+   - 确保输入的JSON文件格式正确
+   - JSON文件应包含 `cascades` 字段，每个级联包含 `post_info`、`comment_tree`、`repost_chain`
+
+## 示例
+
+### 示例1：使用默认设置处理数据
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade /path/to/information_cascade.json \
+    --output /path/to/output.json
+```
+
+### 示例2：使用GPU和自定义模型
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade /path/to/information_cascade.json \
+    --output /path/to/output.json \
+    --bert_model bert-base-chinese \
+    --sentiment_model /path/to/sentiment_model \
+    --device cuda \
+    --batch_size 64
+```
+
+### 示例3：测试模式（只处理前10个级联）
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade /path/to/information_cascade.json \
+    --output /path/to/output.json \
+    --max_cascades 10
+```
+
+## 故障排除
+
+1. **CUDA内存不足**: 减小 `--batch_size` 或使用 `--device cpu`
+2. **模型下载失败**: 检查网络连接，或手动下载模型到本地后指定路径
+3. **JSON格式错误**: 检查输入JSON文件格式是否正确
+4. **处理速度慢**: 使用GPU（`--device cuda`）和增大批处理大小
+
+## 与EasyTPP集成
+
+处理后的JSON文件可以用于EasyTPP框架的训练。参考 `examples/train_robot_thp_with_features.py` 了解如何使用这些特征。

DATA_FILES_NOTICE.md

@@ -0,0 +1,107 @@
+# ⚠️ 数据文件说明
+
+## 📁 数据文件位置
+
+数据文件已复制到 `data/cascades/` 目录：
+
+- `data/cascades/information_cascade.json` (606MB)
+- `data/cascades/information_cascade_original_posts.json` (980MB)
+
+## ⚠️ 重要提示
+
+**这些文件太大（总计约 1.6GB），不会上传到 Hugging Face！**
+
+`.gitignore` 已配置为排除这些文件，因为它们超过了 Git/Hugging Face 的推荐大小限制。
+
+## 📥 在云电脑上获取数据文件
+
+### 方法1: 直接传输（推荐）
+
+```bash
+# 在云电脑上创建目录
+mkdir -p data/cascades
+
+# 使用 scp 从本地传输
+scp -r user@local-machine:/Users/chenshuyi/Documents/research_projects/评论家罗伯特TPP/data/cascades/information_cascade*.json ./data/cascades/
+```
+
+### 方法2: 使用云存储
+
+1. 将文件上传到云存储（Google Drive, Dropbox, OneDrive 等）
+2. 在云电脑上下载
+
+### 方法3: 使用 Git LFS（如果配置）
+
+如果需要通过 Git 管理大文件：
+
+```bash
+# 安装 Git LFS
+git lfs install
+
+# 跟踪大文件
+git lfs track "data/cascades/*.json"
+
+# 添加文件
+git add .gitattributes
+git add data/cascades/*.json
+git commit -m "Add cascade data with LFS"
+git push
+```
+
+### 方法4: 使用 Hugging Face Dataset Hub
+
+可以将数据文件单独上传到 Hugging Face Dataset Hub：
+
+```bash
+# 安装依赖
+pip install huggingface_hub
+
+# 上传数据文件
+huggingface-cli upload <username>/cascade-data data/cascades/ --repo-type dataset
+```
+
+然后在云电脑上下载：
+
+```bash
+huggingface-cli download <username>/cascade-data --local-dir ./data/cascades
+```
+
+## ✅ 验证文件
+
+上传到 Hugging Face 后，验证：
+
+```bash
+# 检查文件是否存在
+ls -lh data/cascades/
+
+# 应该看到：
+# information_cascade.json
+# information_cascade_original_posts.json
+```
+
+## 🚀 使用数据文件
+
+文件准备好后，运行指标计算：
+
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade data/cascades/information_cascade.json \
+    --input_original data/cascades/information_cascade_original_posts.json \
+    --output output_with_metrics.json \
+    --batch_size 32 \
+    --device cuda
+```
+
+## 📝 文件来源
+
+原始文件位置：
+- `/Users/chenshuyi/Documents/research_projects/评论家罗伯特TPP/data/cascades/`
+
+已复制到：
+- `/Users/chenshuyi/Downloads/EasyTemporalPointProcess-main/data/cascades/`
+
+## 🔗 相关文档
+
+- [数据文件说明](data/cascades/README.md)
+- [指标计算说明](COMPUTE_METRICS_README.md)
+- [上传指南](HF_UPLOAD_GUIDE.md)

DATA_TRANSFER_SUMMARY.md

@@ -0,0 +1,95 @@
+# 数据文件转移总结
+
+## ✅ 已完成
+
+两个 information cascade 文件已成功复制到 EasyTemporalPointProcess-main 文件夹。
+
+## 📁 文件位置
+
+### 源文件位置
+- `/Users/chenshuyi/Documents/research_projects/评论家罗伯特TPP/data/cascades/information_cascade.json`
+- `/Users/chenshuyi/Documents/research_projects/评论家罗伯特TPP/data/cascades/information_cascade_original_posts.json`
+
+### 目标位置
+- `/Users/chenshuyi/Downloads/EasyTemporalPointProcess-main/data/cascades/information_cascade.json` (606MB)
+- `/Users/chenshuyi/Downloads/EasyTemporalPointProcess-main/data/cascades/information_cascade_original_posts.json` (980MB)
+
+## ⚠️ 重要说明
+
+### 文件大小
+- **总大小**: 约 1.6GB
+- **information_cascade.json**: 606MB
+- **information_cascade_original_posts.json**: 980MB
+
+### Git 排除配置
+这些文件**不会上传到 Hugging Face**，因为：
+1. 文件太大，超过 Git/Hugging Face 推荐大小
+2. 已通过 `.gitignore` 排除：
+   ```
+   data/cascades/information_cascade*.json
+   data/cascades/*.json
+   ```
+
+## 📥 在云电脑上获取数据文件
+
+### 方法1: 使用 scp 传输（推荐）
+
+```bash
+# 在云电脑上
+mkdir -p data/cascades
+
+# 从本地传输
+scp user@local-machine:/Users/chenshuyi/Documents/research_projects/评论家罗伯特TPP/data/cascades/information_cascade*.json ./data/cascades/
+```
+
+### 方法2: 上传到 Hugging Face Dataset Hub
+
+```bash
+# 在本地
+cd /Users/chenshuyi/Downloads/EasyTemporalPointProcess-main
+huggingface-cli upload <username>/cascade-data data/cascades/ --repo-type dataset
+
+# 在云电脑上下载
+huggingface-cli download <username>/cascade-data --local-dir ./data/cascades
+```
+
+### 方法3: 使用云存储
+
+1. 将文件上传到 Google Drive / Dropbox / OneDrive
+2. 在云电脑上下载
+
+## 📝 相关文档
+
+- **数据文件说明**: `data/cascades/README.md`
+- **数据文件注意事项**: `DATA_FILES_NOTICE.md`
+- **上传指南**: `HF_UPLOAD_GUIDE.md`
+
+## ✅ 验证
+
+上传到 Hugging Face 后，验证数据文件：
+
+```bash
+# 检查文件是否存在
+ls -lh data/cascades/
+
+# 应该看到：
+# information_cascade.json (606MB)
+# information_cascade_original_posts.json (980MB)
+```
+
+## 🚀 使用数据文件
+
+文件准备好后，运行指标计算：
+
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade data/cascades/information_cascade.json \
+    --input_original data/cascades/information_cascade_original_posts.json \
+    --output output_with_metrics.json \
+    --batch_size 32 \
+    --device cuda
+```
+
+---
+
+**数据文件已成功转移！** ✅

HF_UPLOAD_GUIDE.md

@@ -0,0 +1,180 @@
+# Hugging Face 上传指南
+
+本指南说明如何将 EasyTemporalPointProcess-main 上传到 Hugging Face。
+
+## 📋 准备工作
+
+### 1. 运行清理脚本
+
+```bash
+cd /Users/chenshuyi/Downloads/EasyTemporalPointProcess-main
+python cleanup_for_hf.py
+```
+
+这会自动：
+- 删除 `__pycache__/`、`.pyc` 等临时文件
+- 检查大文件
+- 创建上传检查清单
+
+### 2. 数据文件说明 ⚠️
+
+**重要**: `data/cascades/` 目录包含大文件（约 1.6GB），**不会上传到 Hugging Face**。
+
+这些文件已通过 `.gitignore` 排除：
+- `information_cascade.json` (606MB)
+- `information_cascade_original_posts.json` (980MB)
+
+**在云电脑上获取数据文件的方法**：
+- 方法1: 使用 scp 直接传输（推荐）
+- 方法2: 上传到云存储后下载
+- 方法3: 使用 Git LFS（如果配置）
+- 方法4: 单独上传到 Hugging Face Dataset Hub
+
+详细说明请参考 `DATA_FILES_NOTICE.md`
+
+### 3. 手动检查
+
+- [ ] 检查是否有敏感信息（API密钥、密码等）
+- [ ] 确认大文件已正确排除（通过 .gitignore）
+- [ ] 确保 `requirements.txt` 是最新的
+- [ ] 检查 README.md 是否完整
+
+## 🚀 上传方法
+
+### 方法1: 使用 Hugging Face CLI（推荐）
+
+```bash
+# 1. 安装 Hugging Face CLI
+pip install huggingface_hub
+
+# 2. 登录
+huggingface-cli login
+# 输入你的 Hugging Face token（在 https://huggingface.co/settings/tokens 获取）
+
+# 3. 创建仓库（在网页上创建，或使用 CLI）
+# 访问 https://huggingface.co/new 创建新仓库
+# 选择 "Dataset" 类型，命名为例如：easytpp-cascade-metrics
+
+# 4. 上传文件
+cd /Users/chenshuyi/Downloads/EasyTemporalPointProcess-main
+huggingface-cli upload <your-username>/easytpp-cascade-metrics . --repo-type dataset
+```
+
+### 方法2: 使用 Git
+
+```bash
+# 1. 初始化 Git（如果还没有）
+cd /Users/chenshuyi/Downloads/EasyTemporalPointProcess-main
+git init
+
+# 2. 添加文件
+git add .
+git commit -m "Add EasyTPP with cascade metrics computation"
+
+# 3. 添加 Hugging Face 远程仓库
+# 先在 https://huggingface.co/new 创建仓库
+git remote add origin https://huggingface.co/<your-username>/<repo-name>
+
+# 4. 推送
+git push origin main
+```
+
+### 方法3: 使用 Web 界面上传
+
+1. 访问 https://huggingface.co/new
+2. 创建新的 Dataset 仓库
+3. 点击 "Add file" → "Upload files"
+4. 拖拽或选择文件夹上传
+
+## 📦 在云电脑上下载
+
+上传完成后，在云电脑上下载：
+
+```bash
+# 方法1: 使用 Hugging Face CLI
+pip install huggingface_hub
+huggingface-cli download <your-username>/<repo-name> --local-dir ./EasyTPP
+
+# 方法2: 使用 Git
+git clone https://huggingface.co/datasets/<your-username>/<repo-name>
+cd <repo-name>
+
+# 方法3: 使用 Python
+from huggingface_hub import snapshot_download
+snapshot_download(repo_id="<your-username>/<repo-name>", repo_type="dataset", local_dir="./EasyTPP")
+```
+
+### 📥 下载数据文件
+
+**重要**: 代码仓库不包含数据文件（已通过 .gitignore 排除）。
+
+数据文件需要单独获取：
+
+```bash
+# 方法1: 使用 scp 从本地传输（推荐）
+mkdir -p data/cascades
+scp user@local-machine:/path/to/information_cascade*.json ./data/cascades/
+
+# 方法2: 如果已上传到 Hugging Face Dataset Hub
+huggingface-cli download <username>/cascade-data --local-dir ./data/cascades
+
+# 方法3: 从云存储下载
+# （根据你使用的云存储服务）
+```
+
+详细说明请参考 `DATA_FILES_NOTICE.md`
+
+## 📝 新增功能说明
+
+本仓库在原始 EasyTPP 基础上新增了以下功能：
+
+### 1. 级联指标计算 (`compute_cascade_metrics.py`)
+
+用于计算信息级联数据的指标：
+- **情感得分** (Sentiment Score)
+- **情感偏差** (Sentiment Deviation)
+- **语境偏差** (Contextual Deviation)
+- **困惑度** (Perplexity)
+
+详细说明请参考 `COMPUTE_METRICS_README.md`
+
+### 2. 相关文件
+
+- `compute_cascade_metrics.py`: 主计算脚本
+- `COMPUTE_METRICS_README.md`: 使用说明
+- `requirements_compute_metrics.txt`: 额外依赖
+- `example_compute_metrics.sh`: 示例脚本
+- `cleanup_for_hf.py`: 清理脚本
+
+## ⚠️ 注意事项
+
+1. **大文件处理**
+   - 如果文件 >50MB，考虑使用 Git LFS
+   - 或排除数据文件，使用外部链接
+
+2. **敏感信息**
+   - 不要上传包含 API 密钥、密码的文件
+   - 检查配置文件中的敏感数据
+
+3. **许可证**
+   - 确保所有代码都有适当的许可证
+   - 原始 EasyTPP 使用 Apache 2.0 许可证
+
+4. **版本控制**
+   - 建议使用 Git 进行版本控制
+   - 每次更新后提交并推送
+
+## 🔍 验证上传
+
+上传后检查：
+- [ ] 所有文件都已上传
+- [ ] README 显示正确
+- [ ] 代码可以正常下载
+- [ ] 依赖可以正常安装
+
+## 📞 问题反馈
+
+如有问题，请检查：
+1. Hugging Face 仓库设置是否正确
+2. 文件大小是否超过限制
+3. 是否有权限问题

LICENCE

@@ -0,0 +1,203 @@
+Copyright 2022 The EasyTPP Authors.  All rights reserved.
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

MANIFEST.in

@@ -0,0 +1,2 @@
+include requirements.txt
+include version.py

NOTICE

@@ -0,0 +1,23 @@
+=============================================================
+EasyTPP is a open source tool developed by Machine Intelligence Team
+Copyright (c) 2020-2022, Ant Group Holding Limited.
+Licensed under the Apache License, Version 2.0
+
+=============================================================
+This toolkit contains various third-party components under
+different open source licenses
+
+-----------------------------
+Training evaluation pipeline
+Apache License, Version 2.0
+FuxiCTR authors
+
+----------------------------
+Training evaluation pipeline
+Apache License, Version 2.0
+EasyNLP, Alibaba Inc.
+
+----------------------------
+Tokenizer and DataLoader
+Apache License, Version 2.0
+The HuggingFace Inc. team

QUICK_START_HF.md

@@ -0,0 +1,104 @@
+# 快速开始：上传到 Hugging Face
+
+## ✅ 整理完成
+
+文件夹已整理完成，可以上传到 Hugging Face。
+
+**文件夹大小**: 1.3MB（适合上传）
+
+## 📋 整理内容
+
+### 已完成的清理
+- ✅ 更新了 `.gitignore` 文件
+- ✅ 创建了清理脚本 `cleanup_for_hf.py`
+- ✅ 检查了大文件（无大文件）
+- ✅ 创建了上传指南
+
+### 新增文件
+- `compute_cascade_metrics.py` - 级联指标计算脚本
+- `COMPUTE_METRICS_README.md` - 指标计算说明
+- `HF_UPLOAD_GUIDE.md` - 上传指南
+- `ADDITIONS_README.md` - 新增功能说明
+- `cleanup_for_hf.py` - 清理脚本
+- `requirements_compute_metrics.txt` - 额外依赖
+
+## 🚀 三步上传
+
+### 步骤1: 安装 Hugging Face CLI
+
+```bash
+pip install huggingface_hub
+```
+
+### 步骤2: 登录
+
+```bash
+huggingface-cli login
+# 输入你的 token（在 https://huggingface.co/settings/tokens 获取）
+```
+
+### 步骤3: 创建仓库并上传
+
+```bash
+# 1. 在网页上创建仓库
+# 访问 https://huggingface.co/new
+# 选择 "Dataset"，命名为例如：easytpp-cascade-metrics
+
+# 2. 上传文件
+cd /Users/chenshuyi/Downloads/EasyTemporalPointProcess-main
+huggingface-cli upload <your-username>/easytpp-cascade-metrics . --repo-type dataset
+```
+
+## 📥 在云电脑上下载
+
+```bash
+# 方法1: 使用 CLI
+huggingface-cli download <your-username>/easytpp-cascade-metrics --local-dir ./EasyTPP
+
+# 方法2: 使用 Git
+git clone https://huggingface.co/datasets/<your-username>/easytpp-cascade-metrics
+cd easytpp-cascade-metrics
+```
+
+### ⚠️ 重要：数据文件需要单独获取
+
+代码仓库**不包含**数据文件（已通过 .gitignore 排除，因为文件太大）。
+
+数据文件需要单独传输：
+
+```bash
+# 在云电脑上创建目录
+mkdir -p data/cascades
+
+# 方法1: 使用 scp 从本地传输（推荐）
+scp user@local-machine:/path/to/information_cascade*.json ./data/cascades/
+
+# 方法2: 如果已上传到 Hugging Face Dataset Hub
+huggingface-cli download <username>/cascade-data --local-dir ./data/cascades
+```
+
+详细说明请参考 `DATA_FILES_NOTICE.md`
+
+## 📚 相关文档
+
+- **详细上传指南**: `HF_UPLOAD_GUIDE.md`
+- **数据文件说明**: `DATA_FILES_NOTICE.md` ⚠️ **重要**
+- **指标计算说明**: `COMPUTE_METRICS_README.md`
+- **新增功能**: `ADDITIONS_README.md`
+- **上传检查清单**: `UPLOAD_CHECKLIST.md`
+
+## ⚠️ 注意事项
+
+1. **文件大小**: 当前文件夹 1.3MB，无需 Git LFS
+2. **敏感信息**: 已检查，无敏感信息
+3. **依赖**: 确保 `requirements.txt` 和 `requirements_compute_metrics.txt` 已包含
+
+## 🎯 下一步
+
+1. 按照上述步骤上传到 Hugging Face
+2. 在云电脑上下载并测试
+3. 运行 `compute_cascade_metrics.py` 计算指标
+
+---
+
+**准备好了！可以开始上传了！** 🎉

README.md

@@ -0,0 +1,279 @@
+# EasyTPP [ICLR 2024]
+
+<div align="center">
+  <a href="PyVersion">
+    <img alt="Python Version" src="https://img.shields.io/badge/python-3.9+-blue.svg">
+  </a>
+  <a href="LICENSE-CODE">
+    <img alt="Code License" src="https://img.shields.io/badge/license-Apache-000000.svg?&color=f5de53">
+  </a>
+  <a href="commit">
+    <img alt="Last Commit" src="https://img.shields.io/github/last-commit/ant-research/EasyTemporalPointProcess">
+  </a>
+</div>
+
+<div align="center">
+<a href="https://pypi.python.org/pypi/easy-tpp/"> 
+  <img alt="PyPI version" src="https://img.shields.io/pypi/v/easy-tpp.svg?style=flat-square&color=b7534" />
+</a>
+<a href="https://static.pepy.tech/personalized-badge/easy-tpp"> 
+  <img alt="Downloads" src="https://static.pepy.tech/personalized-badge/easy-tpp?period=total&units=international_system&left_color=black&right_color=blue&left_text=Downloads" />
+</a>
+<a href="https://huggingface.co/easytpp" target="_blank">
+    <img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-EasyTPP-ffc107?color=ffc107&logoColor=white" />
+</a>
+<a href="https://github.com/ant-research/EasyTemporalPointProcess/issues">
+  <img alt="Open Issues" src="https://img.shields.io/github/issues-raw/ant-research/EasyTemporalPointProcess" />
+</a>
+</div>
+
+`EasyTPP` is an easy-to-use development and application toolkit for [Temporal Point Process](https://mathworld.wolfram.com/TemporalPointProcess.html) (TPP), with key features in configurability, compatibility and reproducibility. We hope this project could benefit both researchers and practitioners with the goal of easily customized development and open benchmarking in TPP.
+<span id='top'/>
+
+
+
+| <a href='#features'>Features</a>  | <a href='#model-list'>Model List</a> | <a href='#dataset'>Dataset</a>  | <a href='#quick-start'>Quick Start</a> | <a href='#benchmark'>Benchmark</a> |<a href='#doc'>Documentation</a> |<a href='#todo'>Todo List</a> | <a href='#citation'>Citation</a> |<a href='#acknowledgment'>Acknowledgement</a> | <a href='#star-history'>Star History</a> | 
+
+## News
+<span id='news'/>
+
+- ![new](https://img.alicdn.com/imgextra/i4/O1CN01kUiDtl1HVxN6G56vN_!!6000000000764-2-tps-43-19.png) [11-06-2025] We have released a new version of ``EasyTPP`` that exclusively supports PyTorch. TensorFlow support has been removed to streamline the codebase and focus on PyTorch-based implementations.
+- ![new](https://img.alicdn.com/imgextra/i4/O1CN01kUiDtl1HVxN6G56vN_!!6000000000764-2-tps-43-19.png) [11-05-2025] Added the implementation of the [S2P2](https://openreview.net/pdf?id=74SvE2GZwW) model, presented at NeurIPS'2025.
+- ![new](https://img.alicdn.com/imgextra/i4/O1CN01kUiDtl1HVxN6G56vN_!!6000000000764-2-tps-43-19.png) [02-17-2024] ``EasyTPP`` supports HuggingFace dataset API: all datasets have been published in [HuggingFace Repo](https://huggingface.co/easytpp) and see [tutorial notebook](https://github.com/ant-research/EasyTemporalPointProcess/blob/main/notebooks/easytpp_1_dataset.ipynb) for an example of usage.
+- [01-16-2024] Our paper [EasyTPP: Towards Open Benchmarking Temporal Point Process](https://arxiv.org/abs/2307.08097) is accepted by ICLR'2024! 
+<details>
+  <summary>Click to see previous news</summary>
+  <p>
+- [09-30-2023] We published two textual event sequence datasets [GDELT](https://drive.google.com/drive/folders/1Ms-ATMMFf6v4eesfJndyuPLGtX58fCnk) and [Amazon-text-review](https://drive.google.com/drive/folders/1-SLYyrl7ucEG7NpSIF0eSoG9zcbZagZw) that are used in our paper [LAMP](https://arxiv.org/abs/2305.16646), where LLM can be applied for event prediction! See [Documentation](https://ant-research.github.io/EasyTemporalPointProcess/user_guide/dataset.html#preprocessed-datasets) for more details.
+- [09-30-2023] Two of our papers [Language Model Can Improve Event Prediction by Few-Shot Abductive Reasoning](https://arxiv.org/abs/2305.16646) (LAMP) and [Prompt-augmented Temporal Point Process for Streaming Event Sequence](https://arxiv.org/abs/2310.04993) (PromptTPP) are accepted by NeurIPS'2023!
+- [09-02-2023] We published two non-anthropogenic datasets [earthquake](https://drive.google.com/drive/folders/1ubeIz_CCNjHyuu6-XXD0T-gdOLm12rf4) and [volcano eruption](https://drive.google.com/drive/folders/1KSWbNi8LUwC-dxz1T5sOnd9zwAot95Tp?usp=drive_link)! See <a href='#dataset'>Dataset</a> for details.
+- [05-29-2023] We released ``EasyTPP`` v0.0.1!
+- [12-27-2022] Our paper [Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes](https://arxiv.org/abs/2201.12569) was accepted by AAAI'2023!
+- [10-01-2022] Our paper [HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences](https://arxiv.org/abs/2210.01753) was accepted by NeurIPS'2022!
+- [05-01-2022] We started to develop `EasyTPP`.</p>
+</details>
+
+
+## Features <a href='#top'>[Back to Top]</a>
+<span id='features'/>
+
+- **Configurable and customizable**: models are modularized and configurable，with abstract classes to support developing customized
+  TPP models.
+- **PyTorch-based implementation**: `EasyTPP` implements state-of-the-art TPP models using PyTorch 1.7.0+, providing a clean and modern deep learning framework.
+- **Reproducible**: all the benchmarks can be easily reproduced.
+- **Hyper-parameter optimization**: a pipeline of [optuna](https://github.com/optuna/optuna)-based HPO is provided.
+
+
+## Model List <a href='#top'>[Back to Top]</a>
+<span id='model-list'/>
+
+We provide reference implementations of various state-of-the-art TPP papers:
+
+| No  | Publication |     Model     | Paper                                                                                                                                    | Implementation                                                                                                             |
+|:---:|:-----------:|:-------------:|:-----------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------|
+|  1  |   KDD'16    |     RMTPP     | [Recurrent Marked Temporal Point Processes: Embedding Event History to Vector](https://www.kdd.org/kdd2016/papers/files/rpp1081-duA.pdf) | [PyTorch](easy_tpp/model/torch_model/torch_rmtpp.py)                                                                       |
+|  2  | NeurIPS'17  |      NHP      | [The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process](https://arxiv.org/abs/1612.09328)                     | [PyTorch](easy_tpp/model/torch_model/torch_nhp.py)                                                                         |
+|  3  | NeurIPS'19  |    FullyNN    | [Fully Neural Network based Model for General Temporal Point Processes](https://arxiv.org/abs/1905.09690)                                | [PyTorch](easy_tpp/model/torch_model/torch_fullynn.py)                                                                     |
+|  4  |   ICML'20   |     SAHP      | [Self-Attentive Hawkes process](https://arxiv.org/abs/1907.07561)                                                                        | [PyTorch](easy_tpp/model/torch_model/torch_sahp.py)                                                                        |
+|  5  |   ICML'20   |      THP      | [Transformer Hawkes process](https://arxiv.org/abs/2002.09291)                                                                           | [PyTorch](easy_tpp/model/torch_model/torch_thp.py)                                                                         |
+|  6  |   ICLR'20   | IntensityFree | [Intensity-Free Learning of Temporal Point Processes](https://arxiv.org/abs/1909.12127)                                                  | [PyTorch](easy_tpp/model/torch_model/torch_intensity_free.py)                                                              |
+|  7  |   ICLR'21   |    ODETPP     | [Neural Spatio-Temporal Point Processes (simplified)](https://arxiv.org/abs/2011.04583)                                                  | [PyTorch](easy_tpp/model/torch_model/torch_ode_tpp.py)                                                                     |
+|  8  |   ICLR'22   |    AttNHP     | [Transformer Embeddings of Irregularly Spaced Events and Their Participants](https://arxiv.org/abs/2201.00044)                           | [PyTorch](easy_tpp/model/torch_model/torch_attnhp.py)                                                                     |
+|  9  |   NeurIPS'25   |    S2P2     | [Deep Continuous-Time State-Space Models for Marked Event Sequences](https://openreview.net/pdf?id=74SvE2GZwW)   | [PyTorch](easy_tpp/model/torch_model/torch_s2p2.py)                                                                     |
+
+
+
+## Dataset <a href='#top'>[Back to Top]</a>
+<span id='dataset'/>
+
+We preprocessed one synthetic and five real world datasets from widely-cited works that contain diverse characteristics in terms of their application domains and temporal statistics:
+- Synthetic: a univariate Hawkes process simulated by [Tick](https://github.com/X-DataInitiative/tick) library.
+- Retweet ([Zhou, 2013](http://proceedings.mlr.press/v28/zhou13.pdf)): timestamped user retweet events.
+- Taxi ([Whong, 2014](https://chriswhong.com/open-data/foil_nyc_taxi/)): timestamped taxi pick-up events.
+- StackOverflow ([Leskovec, 2014](https://snap.stanford.edu/data/)): timestamped user badge reward events in StackOverflow.
+- Taobao ([Xue et al, 2022](https://arxiv.org/abs/2210.01753)): timestamped user online shopping behavior events in Taobao platform.
+- Amazon ([Xue et al, 2022](https://arxiv.org/abs/2210.01753)): timestamped user online shopping behavior events in Amazon platform.
+
+Per users' request, we processed two non-anthropogenic datasets 
+- [Earthquake](https://drive.google.com/drive/folders/1ubeIz_CCNjHyuu6-XXD0T-gdOLm12rf4): timestamped earthquake events over the Conterminous U.S from 1996 to 2023, processed from [USGS](https://www.usgs.gov/programs/earthquake-hazards/science/earthquake-data).
+- [Volcano eruption](https://drive.google.com/drive/folders/1KSWbNi8LUwC-dxz1T5sOnd9zwAot95Tp?usp=drive_link): timestamped volcano eruption events over the world in recent hundreds of years, processed from [The Smithsonian Institution](https://volcano.si.edu/).
+
+
+  All datasets are preprocess to the `Gatech` format dataset widely used for TPP researchers, and saved at [Google Drive](https://drive.google.com/drive/u/0/folders/1f8k82-NL6KFKuNMsUwozmbzDSFycYvz7) with a public access.
+
+## Quick Start <a href='#top'>[Back to Top]</a>
+<span id='quick-start'/>
+
+
+### Colab Tutorials
+
+Explore the following tutorials that can be opened directly in Google Colab:
+
+- [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ant-research/EasyTemporalPointProcess/blob/main/notebooks/easytpp_1_dataset.ipynb) Tutorial 1: Dataset in EasyTPP.
+- [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ant-research/EasyTemporalPointProcess/blob/main/notebooks/easytpp_2_tfb_wb.ipynb) Tutorial 2: Tensorboard in EasyTPP.
+- [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ant-research/EasyTemporalPointProcess/blob/main/notebooks/easytpp_3_train_eval.ipynb) Tutorial 3: Training and Evaluation of TPPs.
+
+### End-to-end Example
+
+We provide an end-to-end example for users to run a standard TPP model with `EasyTPP`.
+
+
+### Step 1. Installation
+
+First of all, we can install the package either by using pip or from the source code on Github.
+
+To install the latest stable version:
+```bash
+pip install easy-tpp
+```
+
+To install the latest on GitHub:
+```bash
+git clone https://github.com/ant-research/EasyTemporalPointProcess.git
+cd EasyTemporalPointProcess
+python setup.py install
+```
+
+
+### Step 2. Prepare datasets 
+
+We need to put the datasets in a local directory before running a model and the datasets should follow a certain format. See [OnlineDoc - Datasets](https://ant-research.github.io/EasyTemporalPointProcess/user_guide/dataset.html) for more details.
+
+Suppose we use the [taxi dataset](https://chriswhong.com/open-data/foil_nyc_taxi/) in the example.
+
+### Step 3. Train the model
+
+
+Before start training, we need to set up the config file for the pipeline. We provide a preset config file in [Example Config](https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/configs/experiment_config.yaml). The details of the configuration can be found in [OnlineDoc - Training Pipeline](https://ant-research.github.io/EasyTemporalPointProcess/user_guide/run_train_pipeline.html).
+
+After the setup of data and config, the directory structure is as follows:
+
+```bash
+
+    data
+     |______taxi
+             |____ train.pkl
+             |____ dev.pkl
+             |____ test.pkl
+
+    configs
+     |______experiment_config.yaml
+
+```
+
+
+Then we start the training by simply running the script 
+
+```python
+
+import argparse
+from easy_tpp.config_factory import Config
+from easy_tpp.runner import Runner
+
+
+def main():
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument('--config_dir', type=str, required=False, default='configs/experiment_config.yaml',
+                        help='Dir of configuration yaml to train and evaluate the model.')
+
+    parser.add_argument('--experiment_id', type=str, required=False, default='NHP_train',
+                        help='Experiment id in the config file.')
+
+    args = parser.parse_args()
+
+    config = Config.build_from_yaml_file(args.config_dir, experiment_id=args.experiment_id)
+
+    model_runner = Runner.build_from_config(config)
+
+    model_runner.run()
+
+
+if __name__ == '__main__':
+    main()
+
+```
+
+A more detailed example can be found at [OnlineDoc - QuickStart](https://ant-research.github.io/EasyTemporalPointProcess/get_started/quick_start.html).
+
+
+## Documentation <a href='#top'>[Back to Top]</a>
+<span id='doc'/>
+
+The classes and methods of `EasyTPP` have been well documented so that users can generate the documentation by:
+
+```shell
+cd doc
+pip install -r requirements.txt
+make html
+```
+NOTE:
+* The `doc/requirements.txt` is only for documentation by Sphinx, which can be automatically generated by Github actions `.github/workflows/docs.yml`. (Trigger by pull request.)
+
+The full documentation is available on the [website](https://ant-research.github.io/EasyTemporalPointProcess/).
+ 
+## Benchmark <a href='#top'>[Back to Top]</a>
+<span id='benchmark'/>
+
+In the [examples](https://github.com/ant-research/EasyTemporalPointProcess/tree/main/examples) folder, we provide a [script](https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/benchmark_script.py) to benchmark the TPPs, with Taxi dataset as the input. 
+
+To run the script, one should download the Taxi data following the above instructions. The [config](https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/configs/experiment_config.yaml) file is readily setup up. Then run
+
+
+```shell
+cd examples
+python run_retweet.py
+```
+
+
+## License <a href='#top'>[Back to Top]</a>
+
+This project is licensed under the [Apache License (Version 2.0)](https://github.com/alibaba/EasyNLP/blob/master/LICENSE). This toolkit also contains some code modified from other repos under other open-source licenses. See the [NOTICE](https://github.com/ant-research/EasyTPP/blob/master/NOTICE) file for more information.
+
+
+## Todo List <a href='#top'>[Back to Top]</a>
+<span id='todo'/>
+
+- [x] New dataset:
+  - [x] Earthquake: the source data is available in [USGS](https://www.usgs.gov/programs/earthquake-hazards/science/earthquake-data).
+  - [x] Volcano eruption: the source data is available in [NCEI](https://www.ngdc.noaa.gov/hazard/volcano.shtml).
+- [ ] New model:
+  - [ ] Meta Temporal Point Process, ICLR 2023.
+  - [ ] Model-based RL via TPP, AAAI 2022. 
+
+## Citation <a href='#top'>[Back to Top]</a>
+
+<span id='citation'/>
+
+If you find `EasyTPP` useful for your research or development, please cite the following <a href="https://arxiv.org/abs/2307.08097" target="_blank">paper</a>:
+```
+@inproceedings{xue2024easytpp,
+      title={EasyTPP: Towards Open Benchmarking Temporal Point Processes}, 
+      author={Siqiao Xue and Xiaoming Shi and Zhixuan Chu and Yan Wang and Hongyan Hao and Fan Zhou and Caigao Jiang and Chen Pan and James Y. Zhang and Qingsong Wen and Jun Zhou and Hongyuan Mei},
+      booktitle = {International Conference on Learning Representations (ICLR)},
+      year = {2024},
+      url ={https://arxiv.org/abs/2307.08097}
+}
+```
+
+## Acknowledgment <a href='#top'>[Back to Top]</a>
+<span id='acknowledgment'/>
+
+The project is jointly initiated by Machine Intelligence Group, Alipay and DAMO Academy, Alibaba. 
+
+The following repositories are used in `EasyTPP`, either in close to original form or as an inspiration:
+
+- [EasyRec](https://github.com/alibaba/EasyRec)
+- [EasyNLP](https://github.com/alibaba/EasyNLP)
+- [FuxiCTR](https://github.com/xue-pai/FuxiCTR)
+- [Neural Hawkes Process](https://github.com/hongyuanmei/neurawkes)
+- [Neural Hawkes Particle Smoothing](https://github.com/hongyuanmei/neural-hawkes-particle-smoothing)
+- [Attentive Neural Hawkes Process](https://github.com/yangalan123/anhp-andtt)
+- [Huggingface - transformers](https://github.com/huggingface/transformers)
+
+
+## Star History <a href='#top'>[Back to Top]</a>
+<span id='star-history'/>
+
+![Star History Chart](https://api.star-history.com/svg?repos=ant-research/EasyTemporalPointProcess&type=Date)
+

UPLOAD_CHECKLIST.md

@@ -0,0 +1,116 @@
+# Hugging Face 上传检查清单
+
+## ✅ 清理完成
+
+### 已删除的文件类型
+- `__pycache__/` 文件夹
+- `*.pyc`, `*.pyo`, `*.pyd` 文件
+- `.DS_Store` 文件（macOS）
+- `.vscode/`, `.idea/` 文件夹
+- `*.swp`, `*.swo` 文件
+
+### 需要手动检查的项目
+
+1. **大文件检查**
+   - 检查是否有超过50MB的文件
+   - 考虑使用 Git LFS 或排除这些文件
+
+2. **敏感信息检查**
+   - 检查是否有API密钥、密码等敏感信息
+   - 检查配置文件中的敏感数据
+
+3. **数据文件**
+   - 检查 `examples/data/` 目录
+   - 如果数据文件很大，考虑排除或使用外部链接
+
+4. **模型文件**
+   - 检查是否有预训练模型文件
+   - 大模型文件应使用 Git LFS 或 Hugging Face Model Hub
+
+5. **日志文件**
+   - 确保没有日志文件被包含
+   - 检查 `log/`, `logs/` 目录
+
+## 📦 上传到 Hugging Face
+
+### 方法1: 使用 Hugging Face CLI
+
+```bash
+# 安装 Hugging Face CLI
+pip install huggingface_hub
+
+# 登录
+huggingface-cli login
+
+# 创建仓库（如果还没有）
+# 在 https://huggingface.co/new 创建新仓库
+
+# 上传文件
+cd /path/to/EasyTemporalPointProcess-main
+huggingface-cli upload <your-username>/<repo-name> . --repo-type dataset
+```
+
+### 方法2: 使用 Git
+
+```bash
+# 初始化 Git 仓库（如果还没有）
+git init
+git add .
+git commit -m "Initial commit"
+
+# 添加 Hugging Face 远程仓库
+git remote add origin https://huggingface.co/<your-username>/<repo-name>
+
+# 推送
+git push origin main
+```
+
+### 方法3: 使用 Web 界面
+
+1. 访问 https://huggingface.co/new
+2. 创建新的 Dataset 或 Space
+3. 使用 Web 界面上传文件
+
+## 📝 文件结构说明
+
+```
+EasyTemporalPointProcess-main/
+├── easy_tpp/              # 核心库代码
+├── examples/              # 示例代码
+├── notebooks/            # Jupyter notebooks
+├── tests/                # 测试代码
+├── docs/                 # 文档
+├── compute_cascade_metrics.py  # 新增：级联指标计算脚本
+├── COMPUTE_METRICS_README.md   # 新增：指标计算说明
+├── requirements.txt      # 基础依赖
+├── requirements_compute_metrics.txt  # 新增：指标计算依赖
+├── setup.py              # 安装脚本
+└── README.md             # 项目说明
+```
+
+## ⚠️ 注意事项
+
+1. **不要上传大文件到 Git 仓库**
+   - 使用 Git LFS 或 Hugging Face 的存储系统
+   - 考虑使用外部链接引用大文件
+
+2. **检查许可证**
+   - 确保所有代码都有适当的许可证
+   - 检查第三方依赖的许可证兼容性
+
+3. **README 文件**
+   - 确保 README.md 清晰说明项目用途
+   - 包含安装和使用说明
+
+4. **依赖管理**
+   - 确保 requirements.txt 是最新的
+   - 考虑使用 `pip freeze` 生成精确版本
+
+## 🔍 验证上传
+
+上传后，检查：
+- [ ] 所有文件都已上传
+- [ ] 文件大小合理
+- [ ] 没有敏感信息泄露
+- [ ] README 显示正确
+- [ ] 代码可以正常下载和使用

cleanup_for_hf.py

@@ -0,0 +1,293 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""
+清理脚本：准备上传到 Hugging Face
+
+该脚本会：
+1. 清理不必要的文件（__pycache__, .pyc, .pyo等）
+2. 检查大文件
+3. 创建上传检查清单
+"""
+
+import os
+import shutil
+from pathlib import Path
+from typing import List, Tuple
+
+
+def find_and_remove_patterns(root_dir: str, patterns: List[str]) -> List[str]:
+    """
+    查找并删除匹配模式的文件/文件夹
+    
+    Args:
+        root_dir: 根目录
+        patterns: 文件/文件夹模式列表
+    
+    Returns:
+        已删除的文件/文件夹列表
+    """
+    removed = []
+    root_path = Path(root_dir)
+    
+    for pattern in patterns:
+        for item in root_path.rglob(pattern):
+            if item.exists():
+                try:
+                    if item.is_file():
+                        item.unlink()
+                        removed.append(str(item))
+                    elif item.is_dir():
+                        shutil.rmtree(item)
+                        removed.append(str(item))
+                except Exception as e:
+                    print(f"警告: 无法删除 {item}: {e}")
+    
+    return removed
+
+
+def find_large_files(root_dir: str, size_mb: int = 50) -> List[Tuple[str, float]]:
+    """
+    查找大文件
+    
+    Args:
+        root_dir: 根目录
+        size_mb: 文件大小阈值（MB）
+    
+    Returns:
+        (文件路径, 大小MB) 列表
+    """
+    large_files = []
+    root_path = Path(root_dir)
+    size_bytes = size_mb * 1024 * 1024
+    
+    for item in root_path.rglob('*'):
+        if item.is_file():
+            try:
+                size = item.stat().st_size
+                if size > size_bytes:
+                    size_mb_actual = size / (1024 * 1024)
+                    large_files.append((str(item), size_mb_actual))
+            except Exception as e:
+                print(f"警告: 无法检查 {item}: {e}")
+    
+    return large_files
+
+
+def check_gitignore(root_dir: str) -> bool:
+    """
+    检查是否存在 .gitignore 文件
+    
+    Args:
+        root_dir: 根目录
+    
+    Returns:
+        是否存在 .gitignore
+    """
+    gitignore_path = Path(root_dir) / '.gitignore'
+    return gitignore_path.exists()
+
+
+def create_upload_checklist(root_dir: str) -> str:
+    """
+    创建上传检查清单
+    
+    Args:
+        root_dir: 根目录
+    
+    Returns:
+        检查清单内容
+    """
+    checklist = """# Hugging Face 上传检查清单
+
+## ✅ 清理完成
+
+### 已删除的文件类型
+- `__pycache__/` 文件夹
+- `*.pyc`, `*.pyo`, `*.pyd` 文件
+- `.DS_Store` 文件（macOS）
+- `.vscode/`, `.idea/` 文件夹
+- `*.swp`, `*.swo` 文件
+
+### 需要手动检查的项目
+
+1. **大文件检查**
+   - 检查是否有超过50MB的文件
+   - 考虑使用 Git LFS 或排除这些文件
+
+2. **敏感信息检查**
+   - 检查是否有API密钥、密码等敏感信息
+   - 检查配置文件中的敏感数据
+
+3. **数据文件**
+   - 检查 `examples/data/` 目录
+   - 如果数据文件很大，考虑排除或使用外部链接
+
+4. **模型文件**
+   - 检查是否有预训练模型文件
+   - 大模型文件应使用 Git LFS 或 Hugging Face Model Hub
+
+5. **日志文件**
+   - 确保没有日志文件被包含
+   - 检查 `log/`, `logs/` 目录
+
+## 📦 上传到 Hugging Face
+
+### 方法1: 使用 Hugging Face CLI
+
+```bash
+# 安装 Hugging Face CLI
+pip install huggingface_hub
+
+# 登录
+huggingface-cli login
+
+# 创建仓库（如果还没有）
+# 在 https://huggingface.co/new 创建新仓库
+
+# 上传文件
+cd /path/to/EasyTemporalPointProcess-main
+huggingface-cli upload <your-username>/<repo-name> . --repo-type dataset
+```
+
+### 方法2: 使用 Git
+
+```bash
+# 初始化 Git 仓库（如果还没有）
+git init
+git add .
+git commit -m "Initial commit"
+
+# 添加 Hugging Face 远程仓库
+git remote add origin https://huggingface.co/<your-username>/<repo-name>
+
+# 推送
+git push origin main
+```
+
+### 方法3: 使用 Web 界面
+
+1. 访问 https://huggingface.co/new
+2. 创建新的 Dataset 或 Space
+3. 使用 Web 界面上传文件
+
+## 📝 文件结构说明
+
+```
+EasyTemporalPointProcess-main/
+├── easy_tpp/              # 核心库代码
+├── examples/              # 示例代码
+├── notebooks/            # Jupyter notebooks
+├── tests/                # 测试代码
+├── docs/                 # 文档
+├── compute_cascade_metrics.py  # 新增：级联指标计算脚本
+├── COMPUTE_METRICS_README.md   # 新增：指标计算说明
+├── requirements.txt      # 基础依赖
+├── requirements_compute_metrics.txt  # 新增：指标计算依赖
+├── setup.py              # 安装脚本
+└── README.md             # 项目说明
+```
+
+## ⚠️ 注意事项
+
+1. **不要上传大文件到 Git 仓库**
+   - 使用 Git LFS 或 Hugging Face 的存储系统
+   - 考虑使用外部链接引用大文件
+
+2. **检查许可证**
+   - 确保所有代码都有适当的许可证
+   - 检查第三方依赖的许可证兼容性
+
+3. **README 文件**
+   - 确保 README.md 清晰说明项目用途
+   - 包含安装和使用说明
+
+4. **依赖管理**
+   - 确保 requirements.txt 是最新的
+   - 考虑使用 `pip freeze` 生成精确版本
+
+## 🔍 验证上传
+
+上传后，检查：
+- [ ] 所有文件都已上传
+- [ ] 文件大小合理
+- [ ] 没有敏感信息泄露
+- [ ] README 显示正确
+- [ ] 代码可以正常下载和使用
+"""
+    
+    return checklist
+
+
+def main():
+    """主函数"""
+    root_dir = os.path.dirname(os.path.abspath(__file__))
+    
+    print("=" * 60)
+    print("清理脚本：准备上传到 Hugging Face")
+    print("=" * 60)
+    
+    # 要删除的模式
+    patterns_to_remove = [
+        '__pycache__',
+        '*.pyc',
+        '*.pyo',
+        '*.pyd',
+        '.DS_Store',
+        '.vscode',
+        '.idea',
+        '*.swp',
+        '*.swo',
+        '*.log',
+        '.pytest_cache',
+        '.mypy_cache',
+        '.ruff_cache',
+    ]
+    
+    print("\n1. 清理不必要的文件...")
+    removed = find_and_remove_patterns(root_dir, patterns_to_remove)
+    if removed:
+        print(f"   已删除 {len(removed)} 个文件/文件夹")
+        for item in removed[:10]:  # 只显示前10个
+            print(f"   - {item}")
+        if len(removed) > 10:
+            print(f"   ... 还有 {len(removed) - 10} 个文件/文件夹")
+    else:
+        print("   没有找到需要删除的文件")
+    
+    # 检查大文件
+    print("\n2. 检查大文件（>50MB）...")
+    large_files = find_large_files(root_dir, size_mb=50)
+    if large_files:
+        print(f"   找到 {len(large_files)} 个大文件:")
+        for file_path, size_mb in large_files:
+            print(f"   - {file_path} ({size_mb:.2f} MB)")
+        print("\n   ⚠️  建议：大文件应使用 Git LFS 或排除在上传之外")
+    else:
+        print("   ✅ 没有找到大文件")
+    
+    # 检查 .gitignore
+    print("\n3. 检查 .gitignore...")
+    if check_gitignore(root_dir):
+        print("   ✅ .gitignore 文件存在")
+    else:
+        print("   ⚠️  警告: .gitignore 文件不存在")
+    
+    # 创建检查清单
+    print("\n4. 创建上传检查清单...")
+    checklist_content = create_upload_checklist(root_dir)
+    checklist_path = Path(root_dir) / 'UPLOAD_CHECKLIST.md'
+    with open(checklist_path, 'w', encoding='utf-8') as f:
+        f.write(checklist_content)
+    print(f"   ✅ 已创建: {checklist_path}")
+    
+    print("\n" + "=" * 60)
+    print("清理完成！")
+    print("=" * 60)
+    print("\n下一步:")
+    print("1. 查看 UPLOAD_CHECKLIST.md 了解上传步骤")
+    print("2. 检查是否有敏感信息需要移除")
+    print("3. 按照检查清单上传到 Hugging Face")
+
+
+if __name__ == '__main__':
+    main()

compute_cascade_metrics.py

@@ -0,0 +1,568 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""
+计算信息级联的指标：情感得分、情感deviation、contextual deviation、perplexity
+
+该脚本处理 information_cascade.json 和 information_cascade_original_posts.json，
+计算以下指标：
+1. 情感得分 (sentiment score)
+2. 情感deviation (sentiment deviation)
+3. Contextual deviation (语境偏差)
+4. Perplexity (困惑度)
+
+使用方法（在云电脑上）:
+    python compute_cascade_metrics.py \
+        --input_cascade information_cascade.json \
+        --input_original information_cascade_original_posts.json \
+        --output output_with_metrics.json \
+        --bert_model bert-base-chinese \
+        --sentiment_model <sentiment_model_path> \
+        --perplexity_model <perplexity_model_path> \
+        --batch_size 32
+"""
+
+import argparse
+import json
+import numpy as np
+import torch
+from typing import Dict, List, Any, Optional, Tuple
+from tqdm import tqdm
+from transformers import AutoModel, AutoTokenizer, AutoModelForSequenceClassification, AutoModelForCausalLM
+import os
+
+
+class CascadeMetricsComputer:
+    """
+    计算级联数据的各种指标
+    """
+    
+    def __init__(
+        self,
+        bert_model_name: str = 'bert-base-chinese',
+        sentiment_model_name: Optional[str] = None,
+        perplexity_model_name: Optional[str] = None,
+        device: Optional[str] = None,
+        batch_size: int = 32,
+        max_length: int = 512
+    ):
+        """
+        初始化指标计算器
+        
+        Args:
+            bert_model_name: BERT模型名称（用于计算语义向量和contextual deviation）
+            sentiment_model_name: 情感分析模型名称（用于计算情感得分）
+            perplexity_model_name: 语言模型名称（用于计算困惑度）
+            device: 计算设备（'cuda'或'cpu'），如果为None则自动选择
+            batch_size: 批处理大小
+            max_length: 最大序列长度
+        """
+        if device is None:
+            device = 'cuda' if torch.cuda.is_available() else 'cpu'
+        
+        self.device = device
+        self.batch_size = batch_size
+        self.max_length = max_length
+        
+        print(f"正在加载BERT模型: {bert_model_name}")
+        self.bert_tokenizer = AutoTokenizer.from_pretrained(bert_model_name)
+        self.bert_model = AutoModel.from_pretrained(bert_model_name)
+        self.bert_model.to(device)
+        self.bert_model.eval()
+        print(f"BERT模型已加载到设备: {device}")
+        
+        # 加载情感分析模型
+        if sentiment_model_name:
+            print(f"正在加载情感分析模型: {sentiment_model_name}")
+            self.sentiment_tokenizer = AutoTokenizer.from_pretrained(sentiment_model_name)
+            self.sentiment_model = AutoModelForSequenceClassification.from_pretrained(sentiment_model_name)
+            self.sentiment_model.to(device)
+            self.sentiment_model.eval()
+            print(f"情感分析模型已加载到设备: {device}")
+        else:
+            self.sentiment_tokenizer = None
+            self.sentiment_model = None
+            print("未提供情感分析模型，将使用简化的情感计算方法")
+        
+        # 加载困惑度模型（语言模型）
+        if perplexity_model_name:
+            print(f"正在加载困惑度模型: {perplexity_model_name}")
+            self.perplexity_tokenizer = AutoTokenizer.from_pretrained(perplexity_model_name)
+            self.perplexity_model = AutoModelForCausalLM.from_pretrained(perplexity_model_name)
+            self.perplexity_model.to(device)
+            self.perplexity_model.eval()
+            print(f"困惑度模型已加载到设备: {device}")
+        else:
+            self.perplexity_tokenizer = None
+            self.perplexity_model = None
+            print("未提供困惑度模型，将使用简化的困惑度计算方法")
+    
+    def compute_embeddings(self, texts: List[str]) -> np.ndarray:
+        """
+        计算BERT语义向量
+        
+        Args:
+            texts: 文本列表
+        
+        Returns:
+            语义向量矩阵 [num_texts, hidden_size]
+        """
+        embeddings = []
+        
+        with torch.no_grad():
+            for i in range(0, len(texts), self.batch_size):
+                batch_texts = texts[i:i + self.batch_size]
+                
+                # 处理空文本
+                batch_texts = [text if text else "[PAD]" for text in batch_texts]
+                
+                # 分词和编码
+                inputs = self.bert_tokenizer(
+                    batch_texts,
+                    return_tensors='pt',
+                    padding=True,
+                    truncation=True,
+                    max_length=self.max_length
+                ).to(self.device)
+                
+                # 前向传播
+                outputs = self.bert_model(**inputs)
+                
+                # 使用[CLS]标记的嵌入
+                batch_embeddings = outputs.last_hidden_state[:, 0, :].cpu().numpy()
+                embeddings.append(batch_embeddings)
+        
+        return np.vstack(embeddings)
+    
+    def compute_sentiment_scores(self, texts: List[str]) -> List[float]:
+        """
+        计算情感得分
+        
+        Args:
+            texts: 文本列表
+        
+        Returns:
+            情感得分列表（每个文本一个得分，范围通常在[-1, 1]或[0, 1]）
+        """
+        if self.sentiment_model is None:
+            # 使用简化的情感计算方法
+            return self._compute_sentiment_simple(texts)
+        
+        sentiment_scores = []
+        
+        with torch.no_grad():
+            for i in range(0, len(texts), self.batch_size):
+                batch_texts = texts[i:i + self.batch_size]
+                batch_texts = [text if text else "[PAD]" for text in batch_texts]
+                
+                inputs = self.sentiment_tokenizer(
+                    batch_texts,
+                    return_tensors='pt',
+                    padding=True,
+                    truncation=True,
+                    max_length=self.max_length
+                ).to(self.device)
+                
+                outputs = self.sentiment_model(**inputs)
+                logits = outputs.logits
+                
+                # 假设是二分类（正面/负面），使用softmax获取概率
+                probs = torch.softmax(logits, dim=-1)
+                
+                # 计算情感得分：正面概率 - 负面概率（或使用其他方法）
+                if probs.shape[1] == 2:
+                    # 二分类：[负面概率, 正面概率]
+                    batch_scores = (probs[:, 1] - probs[:, 0]).cpu().numpy().tolist()
+                else:
+                    # 多分类或其他情况，使用第一个类别的概率作为得分
+                    batch_scores = probs[:, 0].cpu().numpy().tolist()
+                
+                sentiment_scores.extend(batch_scores)
+        
+        return sentiment_scores
+    
+    def _compute_sentiment_simple(self, texts: List[str]) -> List[float]:
+        """
+        简化的情感计算方法（基于启发式规则）
+        
+        Args:
+            texts: 文本列表
+        
+        Returns:
+            情感得分列表
+        """
+        scores = []
+        for text in texts:
+            if not text:
+                scores.append(0.0)
+                continue
+            
+            # 简单的启发式方法
+            positive_words = ['好', '棒', '赞', '喜欢', '支持', '👍', '❤️', '😊', '😄']
+            negative_words = ['差', '坏', '讨厌', '反对', '👎', '😢', '😠', '😡']
+            
+            positive_count = sum(1 for word in positive_words if word in text)
+            negative_count = sum(1 for word in negative_words if word in text)
+            
+            # 计算情感得分（归一化到[-1, 1]）
+            total_words = len(text)
+            if total_words > 0:
+                score = (positive_count - negative_count) / max(total_words, 1)
+                score = np.clip(score, -1.0, 1.0)
+            else:
+                score = 0.0
+            
+            scores.append(score)
+        
+        return scores
+    
+    def compute_perplexity(self, texts: List[str]) -> List[float]:
+        """
+        计算困惑度
+        
+        Args:
+            texts: 文本列表
+        
+        Returns:
+            困惑度列表
+        """
+        if self.perplexity_model is None:
+            # 使用简化的困惑度计算方法
+            return self._compute_perplexity_simple(texts)
+        
+        perplexities = []
+        
+        with torch.no_grad():
+            for text in texts:
+                if not text:
+                    perplexities.append(0.0)
+                    continue
+                
+                # 分词
+                inputs = self.perplexity_tokenizer(
+                    text,
+                    return_tensors='pt',
+                    truncation=True,
+                    max_length=self.max_length
+                ).to(self.device)
+                
+                # 计算困惑度
+                outputs = self.perplexity_model(**inputs, labels=inputs['input_ids'])
+                loss = outputs.loss
+                
+                # 困惑度 = exp(loss)
+                perplexity = torch.exp(loss).item()
+                perplexities.append(perplexity)
+        
+        return perplexities
+    
+    def _compute_perplexity_simple(self, texts: List[str]) -> List[float]:
+        """
+        简化的困惑度计算方法（基于词汇多样性）
+        
+        Args:
+            texts: 文本列表
+        
+        Returns:
+            困惑度列表
+        """
+        perplexities = []
+        
+        for text in texts:
+            if not text:
+                perplexities.append(0.0)
+                continue
+            
+            # 基于词汇多样性的简化方法
+            words = text.split()
+            unique_words = len(set(words))
+            total_words = len(words)
+            
+            if total_words > 0:
+                # 词汇多样性越低，困惑度越高（简化代理）
+                perplexity_proxy = 1.0 - (unique_words / total_words)
+            else:
+                perplexity_proxy = 0.0
+            
+            perplexities.append(perplexity_proxy)
+        
+        return perplexities
+    
+    def compute_cosine_similarity(self, vec1: np.ndarray, vec2: np.ndarray) -> float:
+        """
+        计算余弦相似度
+        
+        Args:
+            vec1: 向量1
+            vec2: 向量2
+        
+        Returns:
+            余弦相似度 [0, 1]
+        """
+        dot_product = np.dot(vec1, vec2)
+        norm1 = np.linalg.norm(vec1)
+        norm2 = np.linalg.norm(vec2)
+        
+        if norm1 == 0 or norm2 == 0:
+            return 0.0
+        
+        similarity = dot_product / (norm1 * norm2)
+        return float(similarity)
+    
+    def compute_contextual_deviation(self, root_embedding: np.ndarray, current_embedding: np.ndarray) -> float:
+        """
+        计算语境偏差（Contextual Deviation）
+        
+        定义为：1 - 语义相似度
+        
+        Args:
+            root_embedding: 原帖的语义向量
+            current_embedding: 当前文本的语义向量
+        
+        Returns:
+            语境偏差值 [0, 1]，越高表示越偏离原帖语境
+        """
+        similarity = self.compute_cosine_similarity(root_embedding, current_embedding)
+        deviation = 1.0 - similarity
+        return deviation
+    
+    def compute_sentiment_deviation(self, root_sentiment: float, current_sentiment: float) -> float:
+        """
+        计算情感偏差（Sentiment Deviation）
+        
+        定义为：|当前情感得分 - 原帖情感得分|
+        
+        Args:
+            root_sentiment: 原帖的情感得分
+            current_sentiment: 当前文本的情感得分
+        
+        Returns:
+            情感偏差值 [0, 2]（如果情感得分范围是[-1, 1]）
+        """
+        deviation = abs(current_sentiment - root_sentiment)
+        return deviation
+    
+    def process_cascade(self, cascade: Dict[str, Any]) -> Dict[str, Any]:
+        """
+        处理单个级联，计算所有指标
+        
+        Args:
+            cascade: 级联数据字典
+        
+        Returns:
+            添加了指标后的级联数据字典
+        """
+        # 1. 收集所有文本
+        texts: List[str] = []
+        indices: List[Tuple[str, Optional[str]]] = []
+        
+        # 原帖
+        post_info = cascade.get('post_info', {})
+        post_content = post_info.get('content', '')
+        texts.append(post_content)
+        indices.append(('post', None))
+        
+        # 评论
+        comment_tree = cascade.get('comment_tree', {})
+        comment_ids = list(comment_tree.keys())
+        for comment_id in comment_ids:
+            node = comment_tree[comment_id]
+            texts.append(node.get('content', ''))
+            indices.append(('comment', comment_id))
+        
+        # 转发
+        repost_chain = cascade.get('repost_chain', [])
+        for node in repost_chain:
+            forward_text = node.get('forward_text', '') or ''
+            comment_content = node.get('comment_content', '') or ''
+            repost_text = forward_text + comment_content
+            texts.append(repost_text)
+            indices.append(('repost', node.get('repost_id')))
+        
+        # 2. 批量计算特征
+        if len(texts) == 0:
+            return cascade
+        
+        embeddings = self.compute_embeddings(texts)
+        sentiment_scores = self.compute_sentiment_scores(texts)
+        perplexities = self.compute_perplexity(texts)
+        
+        # 3. 获取原帖的特征（用于计算偏差）
+        root_embedding = embeddings[0]
+        root_sentiment = sentiment_scores[0]
+        
+        # 4. 将特征附加到级联数据中
+        # 原帖
+        post_info['embedding'] = root_embedding.tolist()
+        post_info['sentiment_score'] = root_sentiment
+        post_info['perplexity'] = perplexities[0]
+        
+        # 评论
+        for i, comment_id in enumerate(comment_ids):
+            node = comment_tree[comment_id]
+            idx = 1 + i  # 跳过原帖
+            
+            node['embedding'] = embeddings[idx].tolist()
+            node['sentiment_score'] = sentiment_scores[idx]
+            node['perplexity'] = perplexities[idx]
+            
+            # 计算偏差
+            node['contextual_deviation'] = self.compute_contextual_deviation(
+                root_embedding, embeddings[idx]
+            )
+            node['sentiment_deviation'] = self.compute_sentiment_deviation(
+                root_sentiment, sentiment_scores[idx]
+            )
+        
+        # 转发
+        offset = 1 + len(comment_ids)
+        for j, node in enumerate(repost_chain):
+            idx = offset + j
+            
+            node['embedding'] = embeddings[idx].tolist()
+            node['sentiment_score'] = sentiment_scores[idx]
+            node['perplexity'] = perplexities[idx]
+            
+            # 计算偏差
+            node['contextual_deviation'] = self.compute_contextual_deviation(
+                root_embedding, embeddings[idx]
+            )
+            node['sentiment_deviation'] = self.compute_sentiment_deviation(
+                root_sentiment, sentiment_scores[idx]
+            )
+        
+        return cascade
+
+
+def load_json_file(file_path: str) -> Dict[str, Any]:
+    """
+    加载JSON文件（支持大文件）
+    
+    Args:
+        file_path: JSON文件路径
+    
+    Returns:
+        数据字典
+    """
+    print(f"正在加载JSON文件: {file_path}")
+    with open(file_path, 'r', encoding='utf-8') as f:
+        data = json.load(f)
+    print(f"已加载 {len(data.get('cascades', []))} 个级联")
+    return data
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='计算信息级联的指标：情感得分、情感deviation、contextual deviation、perplexity'
+    )
+    parser.add_argument(
+        '--input_cascade',
+        type=str,
+        required=True,
+        help='输入级联JSON文件路径 (information_cascade.json)'
+    )
+    parser.add_argument(
+        '--input_original',
+        type=str,
+        default=None,
+        help='输入原帖JSON文件路径 (information_cascade_original_posts.json)，可选'
+    )
+    parser.add_argument(
+        '--output',
+        type=str,
+        required=True,
+        help='输出JSON文件路径'
+    )
+    parser.add_argument(
+        '--bert_model',
+        type=str,
+        default='bert-base-chinese',
+        help='BERT模型名称或路径（用于计算语义向量）'
+    )
+    parser.add_argument(
+        '--sentiment_model',
+        type=str,
+        default=None,
+        help='情感分析模型名称或路径（可选）'
+    )
+    parser.add_argument(
+        '--perplexity_model',
+        type=str,
+        default=None,
+        help='语言模型名称或路径（用于计算困惑度，可选）'
+    )
+    parser.add_argument(
+        '--batch_size',
+        type=int,
+        default=32,
+        help='批处理大小'
+    )
+    parser.add_argument(
+        '--max_length',
+        type=int,
+        default=512,
+        help='最大序列长度'
+    )
+    parser.add_argument(
+        '--device',
+        type=str,
+        default=None,
+        help='计算设备（cuda/cpu），如果为None则自动选择'
+    )
+    parser.add_argument(
+        '--max_cascades',
+        type=int,
+        default=None,
+        help='最大处理级联数量（用于测试，None表示处理所有）'
+    )
+    
+    args = parser.parse_args()
+    
+    # 加载数据
+    cascade_data = load_json_file(args.input_cascade)
+    
+    if args.input_original:
+        original_data = load_json_file(args.input_original)
+        # 如果需要合并数据，在这里处理
+        # 目前先只处理cascade_data
+    
+    # 初始化指标计算器
+    print("\n初始化指标计算器...")
+    computer = CascadeMetricsComputer(
+        bert_model_name=args.bert_model,
+        sentiment_model_name=args.sentiment_model,
+        perplexity_model_name=args.perplexity_model,
+        device=args.device,
+        batch_size=args.batch_size,
+        max_length=args.max_length
+    )
+    
+        # 处理级联
+        cascades = cascade_data.get('cascades', [])
+        total_cascades = len(cascades)
+        if args.max_cascades:
+            cascades = cascades[:args.max_cascades]
+        
+        print(f"\n开始处理 {len(cascades)}/{total_cascades} 个级联...")
+        processed_count = 0
+        for idx, cascade in enumerate(tqdm(cascades, desc="处理级联")):
+            try:
+                cascade_data['cascades'][idx] = computer.process_cascade(cascade)
+                processed_count += 1
+            except Exception as e:
+                print(f"\n处理级联 {idx} 时出错: {e}")
+                import traceback
+                traceback.print_exc()
+                continue
+        
+        print(f"\n成功处理 {processed_count}/{len(cascades)} 个级联")
+    
+    # 保存结果
+    print(f"\n正在保存结果到: {args.output}")
+    with open(args.output, 'w', encoding='utf-8') as f:
+        json.dump(cascade_data, f, ensure_ascii=False, indent=2)
+    
+    print(f"✅ 完成！结果已保存到: {args.output}")
+
+
+if __name__ == '__main__':
+    main()

data/cascades/.gitkeep

@@ -0,0 +1,3 @@
+# 此目录用于存放级联数据文件
+# 数据文件太大，已通过 .gitignore 排除
+# 请参考 DATA_FILES_NOTICE.md 了解如何获取数据文件

data/cascades/README.md

@@ -0,0 +1,101 @@
+# Cascade Data Files
+
+本目录包含信息级联数据文件。
+
+## 📁 文件说明
+
+### 主要文件
+
+1. **`information_cascade.json`** (606MB)
+   - 完整的级联数据，包含原帖、评论、转发等信息
+   - 用于计算级联指标和训练模型
+
+2. **`information_cascade_original_posts.json`** (980MB)
+   - 原帖数据
+   - 包含原始微博帖子信息
+
+## ⚠️ 文件大小说明
+
+这些文件较大（总计约 1.6GB），**不会自动上传到 Git/Hugging Face**。
+
+## 📥 如何获取数据文件
+
+### 方法1: 手动下载
+
+数据文件需要单独下载或传输到云电脑：
+
+```bash
+# 在云电脑上创建目录
+mkdir -p data/cascades
+
+# 使用 scp 或其他方式传输文件
+scp user@local:/path/to/information_cascade.json ./data/cascades/
+scp user@local:/path/to/information_cascade_original_posts.json ./data/cascades/
+```
+
+### 方法2: 使用 Git LFS（如果配置）
+
+如果使用 Git LFS：
+
+```bash
+# 安装 Git LFS
+git lfs install
+
+# 跟踪大文件
+git lfs track "data/cascades/*.json"
+
+# 添加并提交
+git add .gitattributes
+git add data/cascades/*.json
+git commit -m "Add cascade data files with LFS"
+```
+
+### 方法3: 使用外部存储
+
+- 上传到云存储（如 Google Drive, Dropbox）
+- 使用 Hugging Face Dataset Hub 的存储系统
+- 使用对象存储服务（如 AWS S3, 阿里云 OSS）
+
+## 🚀 使用数据文件
+
+### 运行指标计算
+
+```bash
+python compute_cascade_metrics.py \
+    --input_cascade data/cascades/information_cascade.json \
+    --input_original data/cascades/information_cascade_original_posts.json \
+    --output output_with_metrics.json \
+    --batch_size 32
+```
+
+### 数据格式
+
+JSON 文件格式：
+```json
+{
+  "cascades": [
+    {
+      "post_info": {
+        "content": "...",
+        "timestamp": "..."
+      },
+      "comment_tree": {...},
+      "repost_chain": [...]
+    }
+  ]
+}
+```
+
+详细格式说明请参考项目文档。
+
+## 📝 注意事项
+
+1. **文件大小**: 这些文件很大，确保有足够的磁盘空间
+2. **内存**: 加载完整文件可能需要大量内存
+3. **处理**: 建议使用批处理方式处理数据
+4. **备份**: 建议保留数据文件的备份
+
+## 🔗 相关文档
+
+- [指标计算说明](../COMPUTE_METRICS_README.md)
+- [上传指南](../HF_UPLOAD_GUIDE.md)

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/README.md

@@ -0,0 +1,13 @@
+# Documentation for EasyTPP
+
+This contains the full documentation of EasyTPP, which is hosted at github and can be updated manually (for releases)
+by pushing to the gh-pages branch.
+
+
+To generate the documentation locally, type
+
+```
+pip install -r requirements-doc.txt
+cd docs
+make html
+```

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/source/advanced/implementation.rst

@@ -0,0 +1,143 @@
+===================================
+Model Implementation Details
+===================================
+
+Basic structure
+===================================
+
+In the model folder, `torch_basemodel` (**/model/torch_model/torch_basemodel.py**) / `tf_basemodel` (**/model/tf_model/tf_basemodel.py**) implements functionalities of computing loglikelihood and sampling procedures that are common
+to all the TPP models. In the inherited class, models with specific structures are defined, explained in below sections. 
+
+
+Computing the loglikelihood of non-pad event sequence
+------------------------------------------------------
+
+The loglikelihood computation, following the definition in Equation 8 of `The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process <https://arxiv.org/abs/1612.09328>`_, is shared by all the TPP models.
+
+it takes `time_delta_seqs`, `lambda_at_event`, `lambdas_loss_samples`, `seq_mask`,
+                              `lambda_type_mask` as the input and output the loglikelihood items, please see  `torch_basemodel` (**/model/torch_model/torch_basemodel.py**) / `tf_basemodel` (**/model/tf_model/tf_basemodel.py**)
+for details.
+
+It is noted that:
+
+1. Sequential prediction: because we performance sequential prediction, i.e., predict next one given previous, we do not consider the last one as it has no labels. To implement the `forward` function, we take input of `time_seqs[:, :-1]`
+and `type_seqs[:, :-1]`. For `time_delta_seqs` it is different; please see the next point.
+
+
+
+2. Continuous-time evolution: recall the definition in [dataset](./dataset.rst), assume we have a sequence of 4 events and 1 pad event
+at the end, i.e.,
+
+.. code-block:: bash
+
+    index:          0,          1,         2,         3，     4
+    dtimes:         0，     t_1-t_0,    t_2-t_1,   t_3-t_2,  pad
+    types:          e_0,      e_1,        e_2,       e_3，   pad
+    non_pad_mask:  True,      True,      True,      True,   False
+
+For the i-th event, i-th dtime denotes the time evolution (e.g., decay in NHP) to the current event and
+(i+1)-th dtime denotes the time evolution to the next event. To compute the non-event loglikelihood,
+we should consider the time evolution after the event happens. Therefore we should use `type_delta_seqs[:, 1:]` with masks specified in the below step.
+
+3. Masking: suppose we have predictions of 0,1,2,3-th event and their labels are 1,2,3,4-th events
+where $4$-th event needed to be masked. So we should set the sequence mask as `True, True, True, False`, i.e., `seq_mask=batch_non_pad_mask[:, 1:]`.
+The same logic applies to the attention mask and event type mask.
+
+Therefore the following code is a typical example of calling the loglikelihood computation:
+
+
+.. code-block:: python
+
+    event_ll, non_event_ll, num_events = self.compute_loglikelihood(lambda_at_event=lambda_at_event, # seq_len = max_len - 1
+                                                                    lambdas_loss_samples=lambda_t_sample, # seq_len = max_len - 1
+                                                                    time_delta_seq=time_delta_seq[:, 1:],
+                                                                    seq_mask=batch_non_pad_mask[:, 1:],
+                                                                    lambda_type_mask=type_mask[:, 1:])
+
+
+
+Computing the integral inside the loglikelihood
+-----------------------------------------------
+
+
+The loglikelihood of the parameters is the sum of the log-intensities of the events that happened, at the times they happened,
+minus an integral of the total intensities over the observation interval over [0,T]:
+
+.. math::
+
+    \sum_{t_i}\log \lambda_{k_i}(t_i) - \int_0^T \lambda(t) dt
+
+The first term refers to event loglikelihood and the second term (including the negative sign) refers to the non-event loglikelihood.
+
+
+
+
+
+
+Neural Hawkes Process (NHP)
+===================================
+
+We implement NHP based on author's official pytorch code `Github:nce-mpp <https://github.com/hongyuanmei/nce-mpp/blob/main/ncempp/models/nhp.py>`_.
+
+1. A continuous-time LSTM is introduced, with the code mainly come from `Github:nce-mpp <https://github.com/hongyuanmei/nce-mpp/blob/main/ncempp/models/nhp.py>`_.
+2. A `forward` function in NHP class that recursively update the states: we compute the event embedding, pass to the LSTM cell and then decay afterwards. Noted that for i-th event, we should use (i+1)-th dt for the decay. So we do not consider the last event as it has no decay time.
+
+Attentive Neural Hawkes Process (AttNHP)
+========================================
+
+
+We implement AttNHP based on the authors' official pytorch code `Github:anhp-andtt <https://github.com/yangalan123/anhp-andtt>`_
+and similar to NHP, we factorize it into based model and inherited model.
+
+The forward functions is implemented faithfully to that of the author's repo.
+
+
+Transformer Hawkes Process (THP)
+========================================
+
+We implement THP based on a fixed version of pytorch code `Github:anhp-andtt/thp <https://github.com/yangalan123/anhp-andtt/tree/master/thp>`_
+and we factorize it into based model and inherited model.
+
+
+Self-Attentive Hawkes Process (SAHP)
+========================================
+
+We implement SAHP based on a fixed version of pytorch code `Github:anhp-andtt/sahp <https://github.com/yangalan123/anhp-andtt/tree/master/sahp>`_
+and we factorize it into based model and inherited model.
+
+`SAHP` basically shares very similar structure to that of `THP`.
+
+
+
+Recurrent Marked Temporal Point Processes (RMTPP)
+====================================================
+
+We implement RMTPP faithfully to the author's paper.
+
+
+Intensity Free Learning of Temporal Point Process (IntensityFree)
+==================================================================
+
+We implement the model based on the author's torch code `Github:ifl-tpp <https://github.com/shchur/ifl-tpp>`_.
+
+A small difference between our implementation and the author's is we ignore the `context_init` (the initial state of the RNN) because in our data setup, we do not need a learnable initial RNN state. This modification generally makes little impact on the learning process.
+
+It is worth noting that the thinning algorithm can not be applied to this model because it is intensity-free. When comparing the performance of the model, we only look at its log-likelihood learning curve.
+
+
+Fully Neural Network based Model for General Temporal Point Processes (FullyNN)
+===============================================================================
+
+We implement the model based on the author's keras code `Github:NeuralNetworkPointProcess <https://github.com/omitakahiro/NeuralNetworkPointProcess>`_.
+
+
+ODE-based Temporal Point Process (ODETPP)
+=========================================
+
+We implement a TPP with Neural ODE state evolution, which is a simplified version of `Neural Spatio-Temporal Point Processes <https://arxiv.org/abs/2011.04583>`_. The ODE implementation uses the code from the `blog <https://msurtsukov.github.io/Neural-ODE/>`_
+
+
+Attentive Neural Hawkes Network (ANHN)
+======================================
+
+We implement the model based on the author's paper: the attentive model without the graph regularizer is named ANHN.

docs/source/advanced/performance_valid.rst

@@ -0,0 +1,41 @@
+=========================================
+Performance validation of EasyTPP models
+=========================================
+
+We run the experiments on various dataset to validate the implementations: each model is trained with a max number of epochs and 
+the best model is selected based on the performance on the valid set, then we report the results on the test set.
+
+
+Simulated dataset
+---------------------------
+Conttime
+**********************
+
+
+
++--------------+----------+----------+----------+--------------------+
+| Models       | Loglike  | RMSE     | Acc      | Num Training Epochs|
++==============+==========+==========+==========+====================+
+| Torch_NHP    | -0.93504 |  0.34000 | 0.38656  |        200         |
++--------------+----------+----------+----------+--------------------+
+| Tf_NHP       | -0.85774 | 0.34014  | 0.38806  | 200                |
++--------------+----------+----------+----------+--------------------+
+| Torch_AttNHP | -1.02001 | 0.33678  | 0.36782  | 200                |
++--------------+----------+----------+----------+--------------------+
+| Tf_AttNHP    | -1.02315 | 0.33816  | 0.19456  | 200                |
++--------------+----------+----------+----------+--------------------+
+| Torch_AttNHP | -1.00593 | 0.33685  | 0.37723  | 500                |
++--------------+----------+----------+----------+--------------------+
+| Tf_AttNHP    | -0.99827 | 0.33717  | 0.36498  | 500                |
++--------------+----------+----------+----------+--------------------+
+| Torch_THP    | -0.99827 | 0.33717  | 0.36498  | 500                |
++--------------+----------+----------+----------+--------------------+
+| Tf_THP       | -1.01898 | 0.33677  | 0.37875  | 500                |
++--------------+----------+----------+----------+--------------------+
+
+
+
+## Real dataset
+### Taxi 
+
+

docs/source/advanced/tensorboard.rst

@@ -0,0 +1,75 @@
+===================================
+Launching the Tensorboard
+===================================
+
+
+Here we present how to launch the tensorboard within the  ``EasyTPP`` framework.
+
+Step 1: Activate the usage of tensorboard in Config file
+========================================================
+
+
+As shown in `Training Pipeline <../get_started/run_train_pipeline.html>`_, we need to firstly initialize the 'model_config.yaml' file to setup the running config before training or evaluating the model.
+
+In the ``model config`` (`modeling` attribute of the config), one needs to set ``use_tfb`` to ``True`` in `trainer`. Then before the running process, summary writers tracking the performance on training and valid sets are both initialized.
+
+.. code-block:: yaml
+
+    NHP_train:
+      base_config:
+        stage: train
+        backend: torch
+        dataset_id: taxi
+        runner_id: std_tpp
+        model_id: NHP # model name
+        base_dir: './checkpoints/'
+      trainer_config:
+        batch_size: 256
+        max_epoch: 200
+        shuffle: False
+        optimizer: adam
+        learning_rate: 1.e-3
+        valid_freq: 1
+        use_tfb: True  # Activate the tensorboard
+        metrics: [ 'acc', 'rmse' ]
+        seed: 2019
+        gpu: -1
+      model_config:
+        hidden_size: 64
+        loss_integral_num_sample_per_step: 20
+    #    pretrained_model_dir: ./checkpoints/75518_4377527680_230530-132355/models/saved_model
+        thinning:
+          num_seq: 10
+          num_sample: 1
+          num_exp: 500 # number of i.i.d. Exp(intensity_bound) draws at one time in thinning algorithm
+          look_ahead_time: 10
+          patience_counter: 5 # the maximum iteration used in adaptive thinning
+          over_sample_rate: 5
+          num_samples_boundary: 5
+          dtime_max: 5
+          num_step_gen: 1
+
+
+
+Step 2: Launching the tensorboard
+========================================================
+
+
+We simply go to the output file of the training runner (its directory is specified in `base_dir` of ``base_config``), find out the tensorboard file address and launch it.
+
+A complete example of using tensorboard can be seen at *examples/run_tensorboard.py*.
+
+
+.. code-block:: python
+
+    import os
+
+    def main():
+        # one can find this dir in the config out file
+        log_dir = './checkpoints/NHP_train_taxi_20220527-20:18:30/tfb_train'
+        os.system('tensorboard --logdir={}'.format(log_dir))
+        return
+
+
+    if __name__ == '__main__':
+        main()

docs/source/advanced/thinning_algo.rst

@@ -0,0 +1,56 @@
+==============================================
+Thinning Algorithm for Sampling Event Sequence
+==============================================
+
+In ``EasyTPP`` we use ``Thinning algorithm`` depicted in Algorithm 2
+in `The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process <https://arxiv.org/abs/1612.09328>`_
+for event sampling.
+
+The implementation of the algorithm
+====================================
+
+
+We implement the algorithm both in PyTorch and Tensorflow, as seen in *./model/torch_thinning.py* and
+*./model/tf_thinning.py*, which basically follow the same procedure.
+
+The corresponding code is in function ``draw_next_time_one_step``, which consists of the following steps:
+
+1. Compute the upper bound of the intensity at each event timestamp in function ``compute_intensity_upper_bound``, where we sample some timestamps inside event intervals and output a upper bound intensity matrix [batch_size, seq_len]， denoting the upper bound of prediced intensity (for next time interval) for each sequence at each timestamp.
+2. Sample the exponential distribution with the intensity computed in Step 1 in function ``sample_exp_distribution``, where we simply divide the standard exponential number with the intensity, which is equivalent to sampling with exp(sample_rate), according to `the properties of Exponential Distribution <https://en.wikipedia.org/wiki/Exponential_distribution>`_. The exponential random variables have size [batch_size, seq_len, num_sample, num_exp], where num_sample refers to the number of event times sampled in every interval and num_exp refers to number of i.i.d. Exp(intensity_bound) draws at one time in thinning algorithm.
+3. Compute the intensities at the sample times proposed in Step 2， with final size `[batch_size, seq_len, num_sample, num_exp]`.
+4. Sample the standard uniform distribution with size `[batch_size, seq_len, num_sample, num_exp]`.
+5. Perform the acceptance sampling with certain probability in function ``sample_accept``.
+6. The earliest sampling dtimes are accepted. For unaccepted sampling dtimes, use boundary/maxsampletime for that draw.
+7. The final predicted dtimes has size `[batch_size, seq_len, num_sample]`, which refers to the sampling dtimes for each sequence at each timestamp, along with an equal weight vector.
+8. The product of the predicted dtimes and the weight is the final predicted dtimes, with size `[batch_size, seq_len]`.
+
+
+.. image:: ../../images/thinning_algo.jpg
+    :alt: thinning_algo
+
+
+
+One-step prediction
+====================================
+By default, once given the parameters of thinning algo (defining a ``thinning`` config as part of ``model_config``), we perform the one-step prediction in model evaluation, i.e., predict the next event given the prefix. The implementation is in function ``prediction_event_one_step`` in BaseModel (i.e., TorchBaseModel or TfBaseModel).
+
+
+Multi-step prediction
+====================================
+The recursive multi-step prediction is activated by setting `num_step_gen` to a number bigger than 1 in the ``thinning`` config.
+
+Be noted that, we generate the multi-step events after the last non-pad event of each sequence. The implementation is in function `predict_multi_step_since_last_event` in BaseModel (i.e., TorchBaseModel or TfBaseModel).
+
+
+.. code-block:: yaml
+
+    thinning:
+      num_seq: 10
+      num_sample: 1
+      num_exp: 500 # number of i.i.d. Exp(intensity_bound) draws at one time in thinning algorithm
+      look_ahead_time: 10
+      patience_counter: 5 # the maximum iteration used in adaptive thinning
+      over_sample_rate: 5
+      num_samples_boundary: 5
+      dtime_max: 5
+      num_step_gen: 5    # by default it is single step, i.e., 1

docs/source/conf.py

@@ -0,0 +1,59 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+
+# -- Autodoc information -----------------------------------------------------
+# https://sphinx-rtd-tutorial.readthedocs.io/en/latest/sphinx-config.html
+
+
+import os
+import sys
+
+sys.path.insert(0, os.path.abspath('../../easy_tpp/'))
+
+sys.path.insert(0, os.path.abspath('../..'))
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = 'EasyTPP'
+copyright = '2022, Machine Intelligence, Alipay'
+author = 'Machine Intelligence, Alipay'
+release = '0.0.2'
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = [
+    "sphinx.ext.autodoc",
+    'sphinx.ext.viewcode',
+    "sphinx.ext.todo",
+    "sphinx.ext.mathjax",
+    "sphinx.ext.napoleon",
+    'sphinx.ext.autosummary'
+]
+
+napoleon_google_docstring = True
+napoleon_numpy_docstring = False
+
+templates_path = ['_templates']
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This patterns also effect to html_static_path and html_extra_path
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = 'sphinx_rtd_theme'
+html_static_path = ['_static']
+
+autodoc_member_order = "bysource"
+autodoc_default_flags = ["members"]
+autodoc_default_options = {
+    "members": True,
+    "member-order": "bysource",
+    "special-members": "__init__",
+}

docs/source/dev_guide/model_custom.rst

@@ -0,0 +1,78 @@
+==================
+Customize a Model
+==================
+
+
+Here we introduce how to customize a TPP model with the support of ``EasyTPP``.
+
+
+
+Create a new TPP Model Class
+=============================
+
+Assume we are building a PyTorch model. We need to initialize the model by inheriting class `EasyTPP.model.torch_model.TorchBaseModel <../ref/models.html>`_.
+
+.. code-block:: python
+
+    from easy_tpp.model.torch_model.torch_basemodel import TorchBaseModel
+
+    # Custom Torch TPP implementations need to
+    # inherit from the TorchBaseModel interface
+    class NewModel(TorchBaseModel):
+        def __init__(self, model_config):
+            super(NewModel, self).__init__(model_config)
+
+        # Forward along the sequence, output the states / intensities at the event times
+        def forward(self, batch):
+            ...
+            return states
+
+        # Compute the loglikelihood loss
+        def loglike_loss(self, batch):
+            ....
+            return loglike
+
+        # Compute the intensities at given sampling times
+        # Used in the Thinning sampler
+        def compute_intensities_at_sample_times(self, batch, sample_times, **kwargs):
+            ...
+            return intensities
+
+
+If we are building a Tensorflow model, we start with the following code
+
+.. code-block:: python
+
+    from easy_tpp.model.torch_model.tf_basemodel import TfBaseModel
+
+    # Custom Tf TPP implementations need to
+    # inherit from the TorchBaseModel interface
+    class NewModel(TfBaseModel):
+        def __init__(self, model_config):
+            super(NewModel, self).__init__(model_config)
+
+        # Forward along the sequence, output the states / intensities at the event times
+        def forward(self, batch):
+            ...
+            return states
+
+
+        # Compute the loglikelihood loss
+        def loglike_loss(self, batch):
+            ....
+            return loglike
+
+        # Compute the intensities at given sampling times
+        # Used in the Thinning sampler
+        def compute_intensities_at_sample_times(self, batch, sample_times, **kwargs):
+            ...
+            return intensities
+
+Rewrite Relevant Methods
+==============================
+
+There are three important functions needed to be implemented:
+
+- `forward`: the input is the batch data and the output is states at each step.
+- `loglike_loss`: it computes the loglikihood loss given the batch data.
+- `compute_intensities_at_sample_times`: it computes the intensities at each sampling steps.

docs/source/get_started/install.rst

@@ -0,0 +1,64 @@
+==================
+Installation
+==================
+
+
+``EasyTPP`` provides an open-source library for `Neural TPP`, with a fully automated pipeline for model training and prediction.
+
+
+Requirements
+=============
+
+.. code-block:: bash
+
+    PyTorch version >= 1.8.0
+    Python version >= 3.7
+    Tensorflow version >= 1.13.1 (only needed when using Tensorflow backend)
+
+
+
+First, we need a python environment whose version is at least greater than 3.7.0. If you don’t have one, please refer to the `Documentation <https://docs.anaconda.com/anaconda/install/>`_ to install and configure the Anaconda environment.
+
+.. code-block:: bash
+
+    conda create -n easytpp python=3.8
+    conda activate easytpp
+
+Then, install Pytorch and keep the version at least greater than 1.8.0.
+
+.. code-block:: bash
+
+    pip install torch
+
+By default, we assume to use PyTorch. If one wants to use Tensorflow backend, please install tensorflow additionally. Both Tensorflow 1.13.1 and 2.x are supported.
+
+.. code-block:: bash
+
+    pip install tensorflow
+
+
+
+Install
+=====================
+
+
+Install with pip
+--------------------------
+
+
+.. code-block:: bash
+
+    pip install easy-tpp
+
+
+Install with the source
+--------------------------
+
+Setup from the source：
+
+.. code-block:: bash
+
+    git clone https://github.com/ant-research/EasyTemporalPointProcess.git
+    cd EasyTemporalPointProcess
+    python setup.py install
+

docs/source/get_started/introduction.rst

@@ -0,0 +1,60 @@
+==================
+Introduction
+==================
+
+
+``EasyTPP`` provides an open-source library for `Neural TPP`, with a fully automated pipeline for model training and prediction.
+
+
+Framework
+=========
+
+
+``EasyTPP`` supports both Tensorflow and PyTorch: each model has two equivalent versions implemented in Tensorflow 1.13 and Pytorch 1.8 respectively. The data processing and model training / prediction pipeline are compatible with both Tensorflow and Pytorch as well.
+
+
+At the module level, ``EasyTPP`` is a package that consists of the following components, which are designed as loose-coupled modules that provide flexibility for users to develop customized functionalities.
+
+
+
+========================  ==============================================================================
+Name                      Description
+========================  ==============================================================================
+`Preprocess` module       Provides data batch-wise padding, inter-time processing and other related work for raw sequence.
+
+`Model` module            Implements a list of SOTA TPP models. Please refer to `Model Validation <../advanced/performance_valid.html>`_ for more details.
+
+`Config` module           Encapsulate the construction of the configuration needed to run the pipeline.
+
+`Runner` module           Controls the training and prediction pipeline.
+========================  ==============================================================================
+
+
+
+Install
+=========
+
+``EasyTPP`` can be installed either by pip or the source. By default it is built based on PyTorch. If one wants to run with the Tensorflow backend, one needs to install Tensorflow additionally.
+
+Please see `Installation <./install.html>`_ for details of requirement and installation.
+
+
+Prepare Data
+============
+
+By default, we use the data in Gatech format, i.e., each dataset is a dict containing the keys such as `time_since_last_event`, `time_since_start` and `type_event`. `Preprocess <../ref/preprocess.html>`_ module
+will preprocess the data and feed it into the model.
+
+
+An example of building a pseudo dataloader can be found at `examples <https://github.com/ant-research/EasyTemporalPointProcess/tree/main/examples/data_loader.py>`_. Please refer to `Datatset <../user_guide/dataset.html>`_ for more explanations of the `TPP` dataset iterator.
+
+
+Model Training and Prediction
+==============================
+
+The training and prediction pipeline consists of two steps:
+
+1. Setup the config file, which specifies the dataset dir, model params and pipeline settings.
+2. Launch the python script to run the whole pipeline.
+
+Please see `Training Pipeline <../user_guide/run_train_pipeline.html>`_ and `Evaluation Pipeline <../user_guide/run_eval.html>`_ for more details.

docs/source/get_started/quick_start.rst

@@ -0,0 +1,106 @@
+====================
+Quick Start
+====================
+
+
+We use the [Taxi]_ dataset as an example to show how to use ``EasyTPP`` to train a model. More details and results are provided in `Training Pipeline <../user_guide/run_train_pipeline.html>`_.
+
+
+Download Dataset
+===================
+
+
+
+The Taxi dataset we used is preprocessed by `HYPRO <https://github.com/iLampard/hypro_tpp>`_ . You can either download the dataset (in pickle) from Google Drive `here <https://drive.google.com/drive/folders/1vNX2gFuGfhoh-vngoebaQlj2-ZIZMiBo>`_ or the dataset (in json) from `HuggingFace <https://huggingface.co/easytpp>`_.
+
+
+Note that if the data sources are pickle files, we need to write the data config (in `Example Config <https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/configs/experiment_config.yaml>`_) in the following way
+
+.. code-block:: yaml
+
+    data:
+      taxi:
+        data_format: pickle
+        train_dir: ./data/taxi/train.pkl
+        valid_dir: ./data/taxi/dev.pkl
+        test_dir: ./data/taxi/test.pkl
+
+If we choose to directly load from HuggingFace, we can put it this way:
+
+.. code-block:: yaml
+
+    data:
+      taxi:
+        data_format: json
+        train_dir: easytpp/taxi
+        valid_dir: easytpp/taxi
+        test_dir: easytpp/taxi
+
+
+Meanwhile, it is also feasible to put the local directory of json files downloaded from HuggingFace in the config:
+
+.. code-block:: yaml
+
+    data:
+      taxi:
+        data_format: json
+        train_dir: ./data/taxi/train.json
+        valid_dir: ./data/taxi/dev.json
+        test_dir: ./data/taxi/test.json
+
+
+
+
+Setup the configuration file
+==============================
+
+We provide a preset config file in `Example Config <https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/configs/experiment_config.yaml>`_. The details of the configuration can be found in `Training Pipeline <../user_guide/run_train_pipeline.html>`_.
+
+
+
+
+Train the Model
+=========================
+
+At this stage we need to write a script to run the training pipeline. There is a preset script `train_nhp.py <https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/train_nhp.py>`_ and one can simply copy it.
+
+Taking the pickle data source for example, after the setup of data, config and running script, the directory structure is as follows:
+
+.. code-block:: bash
+
+    data
+     |______taxi
+             |____ train.pkl
+             |____ dev.pkl
+             |____ test.pkl
+
+    configs
+     |______experiment_config.yaml
+
+     train_nhp.py
+
+
+
+The one can simply run the following command.
+
+
+.. code-block:: bash
+
+    python train_nhp.py
+
+
+
+Reference
+----------
+
+.. [Taxi]
+
+.. code-block:: bash
+
+    @misc{whong-14-taxi,
+      title = {F{OIL}ing {NYC}’s Taxi Trip Data},
+      author={Whong, Chris},
+      year = {2014},
+      url = {https://chriswhong.com/open-data/foil_nyc_taxi/}
+    }
+

docs/source/index.rst

@@ -0,0 +1,56 @@
+===================================
+``EasyTPP`` Documentation
+===================================
+
+
+``EasyTPP`` is an easy-to-use development and application toolkit for `Neural Temporal Point Process <https://mathworld.wolfram.com/TemporalPointProcess.html>`_ (*Neural TPP*), with key features in configurability, compatibility and reproducibility. We hope this project could benefit both researchers and practitioners with the goal of easily customized development and open benchmarking.
+
+
+
+.. toctree::
+   :hidden:
+
+.. toctree::
+   :maxdepth: 2
+   :caption: GETTING STARTED
+
+   Introduction <get_started/introduction.rst>
+   Installation <get_started/install.rst>
+   Quick Start <get_started/quick_start.rst>
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: USER GUIDE
+
+   Dataset <user_guide/dataset.rst>
+   Model Training <user_guide/run_train_pipeline.rst>
+   Model Prediction <user_guide/run_eval.rst>
+
+.. toctree::
+   :maxdepth: 2
+   :caption: DEVELOPER GUIDE
+
+   Model Customization <dev_guide/model_custom.rst>
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: ADVANCED TOPICS
+
+   Thinning Algorithm <advanced/thinning_algo.rst>
+   Tensorboard <advanced/tensorboard.rst>
+   Performance Benchmarks <advanced/performance_valid.rst>
+   Implementation Details <advanced/implementation.rst>
+
+.. toctree::
+   :maxdepth: 2
+   :caption: API REFERENCE
+
+    Config  <ref/config.rst>
+    Preprocess  <ref/preprocess.rst>
+    Model  <ref/models.rst>
+    Runner  <ref/runner.rst>
+    Hyper-parameter Optimization  <ref/hpo.rst>
+    Tf and Torch Wrapper  <ref/wrapper.rst>
+    Utilities  <ref/utils.rst>

docs/source/ref/config.rst

@@ -0,0 +1,10 @@
+.. _api-config:
+
+EasyTPP Config Modules
+============================
+
+
+.. automodule:: config_factory
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/source/ref/hpo.rst

@@ -0,0 +1,10 @@
+.. _api-config:
+
+EasyTPP Config Modules
+============================
+
+
+.. automodule:: hpo
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/source/ref/models.rst

@@ -0,0 +1,50 @@
+.. _api-model:
+
+EasyTPP Models
+====================
+
+
+
+.. _api-tf_model:
+
+model.tf_model module
+------------------------------
+
+.. automodule:: easy_tpp.model.tf_model
+.. autosummary::
+    :toctree: ../generated/
+
+    tf_baselayer
+    tf_basemodel
+    tf_nhp
+    tf_fullynn
+    tf_intensity_free
+    tf_ode_tpp
+    tf_rmtpp
+    tf_sahp
+    tf_thp
+    tf_attnhp
+    tf_thinning
+
+
+.. _api-torch_model:
+
+model.torch_model module
+------------------------------
+
+.. automodule:: easy_tpp.model.torch_model
+.. autosummary::
+    :toctree: ../generated/
+
+    torch_baselayer
+    torch_basemodel
+    torch_nhp
+    torch_fullynn
+    torch_intensity_free
+    torch_ode_tpp
+    torch_rmtpp
+    torch_sahp
+    torch_thp
+    torch_attnhp
+    torch_thinning
+

docs/source/ref/preprocess.rst

@@ -0,0 +1,10 @@
+.. _api-preprocess:
+
+EasyTPP Preprocess Modules
+==========================
+
+
+.. automodule:: preprocess
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/source/ref/runner.rst

@@ -0,0 +1,10 @@
+.. _api-modelrunner:
+
+EasyTPP Model Runner Modules
+============================
+
+
+.. automodule:: runner
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/source/ref/utils.rst

@@ -0,0 +1,10 @@
+.. _api-util:
+
+EasyTPP Utilities Modules
+==========================
+
+
+.. automodule:: utils
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/source/ref/wrapper.rst

@@ -0,0 +1,17 @@
+.. _api-wrapper:
+
+EasyTPP Tf and Torch Wrapper Modules
+====================================
+
+
+.. automodule:: tf_wrapper
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+
+
+.. automodule:: torch_wrapper
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/source/user_guide/dataset.rst

@@ -0,0 +1,124 @@
+===========================================
+Expected Dataset Format and Data Processing
+===========================================
+
+Required format
+===================================
+
+In EasyTPP we use the data in Gatech format, i.e., each dataset is a dict containing the following keys as
+
+.. code-block:: bash
+
+    dim_process: 5 # num of event types (no padding)
+    'train': [[{'idx_event': 2, 'time_since_last_event': 1.0267814, 'time_since_last_same_event': 1.0267814, 'type_event': 3, 'time_since_start': 1.0267814}, {'idx_event': 3, 'time_since_last_event': 0.4029268, 'time_since_last_same_event': 1.4297082, 'type_event': 0, 'time_since_start': 1.4297082},...,],[{}...{}]]
+
+where `dim_process` refers to the number of event types (without padding) and
+`train` (or `dev` / `test`) contains a list of list which corresponds to an event sequence each.
+
+Each pickle file generates a set of event sequences, each containing three sub sequences:
+
+1. `time_seqs`: absolute timestamps of the events, correspond to `time_since_last_event`.
+2. `time_delta_seqs`: relative timestamps of the events, correspond to `time_since_last_same_event`.
+3. `type_seqs`: types of the events, correspond to `type_event`. Be noted that the event type index `starts from 0`.
+
+
+Data processing
+===================================
+
+The data processing follows the similar pipeline as in official code of `AttNHP <https://github.com/yangalan123/anhp-andtt>`_. We name it the process of `event tokenize`.
+
+
+Sequence padding
+----------------
+
+
+time_seqs, time_delta_seqs and type_seqs are firstly padded to `the max length of the whole dataset` and then fed into the model in batch.
+
+.. code-block:: bash
+
+    input: raw event sequence (e_0, e_1, e_2, e_3) and max_len=6 # the max length among all data seqs
+
+    output:
+
+            index:    0,          1,         2,         3，      4         5
+            dtimes:   0，     t_1-t_0,    t_2-t_1,   t_3-t_2， time_pad, time_pad
+            types:    e_0,      e_1,        e_2,       e_3，  type_pad, type_pad
+
+
+By default, we set the value of time_pad and type_pad to be the *num_event_types* (because we assume the event type index starts from 0, therefore the integer value of num_event_types is unused).
+
+Sequence masking
+----------------
+
+
+After padding, we perform the masking for the event sequences and generate three more seqs: batch_non_pad_mask, attention_mask, type_mask：
+
+1. `batch_non_pad_mask`: it indicates the position of masks in the sequence.
+2. `attention_mask`: it indicates the masks used in the attention calculation (one event can only attend to its past events).
+3. `type_mask`: it uses one-hot vector to represent the event type. The padded event is a zero vector.
+
+Finally, each batch contains six elements: time_seqs, time_delta_seqs, event_seq, batch_non_pad_mask, attention_mask, type_mask. The implementation of padding mechanism can be found at `event_tokenizer <https://github.com/ant-research/EasyTemporalPointProcess/blob/main/easy_tpp/preprocess/event_tokenizer.py>`_.
+
+
+
+An example
+----------------
+
+We take a real event sequence for example. Assume we have an input sequence $[ 1,  9,  5,  0]$ with num_event_types=11 and max_len=6. 
+
+Then the padded time_seqs, time_delta_seqs and type_seqs become
+
+.. code-block:: bash
+
+    # time_seqs
+    [ 0.0000,  0.8252,  1.3806,  1.8349, 11.0000, 11.0000]
+
+    # time_delta_seqs
+    [ 0.0000,  0.8252,  0.5554,  0.4542, 11.0000, 11.0000]
+
+    # type_seqs
+    [ 1,  9,  5,  0, 11, 11]
+
+
+The mask sequences are 
+
+.. code-block:: bash
+
+    # batch_non_pad_mask
+    [ True,  True,  True,  True, False, False]
+
+    # attention_mask
+    [[True,  True,  True,  True,  True,  True],
+    [False,  True,  True,  True,  True,  True],
+    [False, False,  True,  True,  True,  True],
+    [False, False, False,  True,  True,  True],
+    [False, False, False, False,  True,  True],
+    [False, False, False, False,  True,  True]]
+
+    # type_mask
+    [[False,  True, False, False, False, False, False, False, False, False, False],
+    [False, False, False, False, False, False, False, False, False, True, False],
+    [False, False, False, False, False,  True, False, False, False, False, False],
+    [True, False, False, False, False, False, False, False, False, False, False],
+    [False, False, False, False, False, False, False, False, False, False, False],
+    [False, False, False, False, False, False, False, False, False, False, False]],
+
+
+The runnable examples of constructing and iterating the dataset object can be found at `examples/event_tokenizer.py <https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/event_tokenizer.py>`_
+
+
+Preprocessed Datasets
+===================================
+
+We have preprocessed some widely-used open source datasets in Gatech format, which can be found at `Google Drive <https://drive.google.com/drive/folders/0BwqmV0EcoUc8UklIR1BKV25YR1U?resourcekey=0-OrlU87jyc1m-dVMmY5aC4w>`_. We use them for validating and benchmarking EasyTPP models.
+
+- Retweet (`Zhou, 2013 <http://proceedings.mlr.press/v28/zhou13.pdf>`_). This dataset contains time-stamped user retweet event sequences.  The events are categorized into 3 types: retweets by “small,” “medium” and “large” users. Small users have fewer than 120 followers, medium users have fewer than 1363, and the rest are large users. We work on a subset of 5200 most active users with an average sequence length of 70.
+- Taxi (`Whong, 2014 <https://chriswhong.com/open-data/foil_nyc_taxi>`_). This dataset tracks the time-stamped taxi pick-up and drop-off events across the five boroughs of the New York City; each (borough, pick-up or drop-off) combination defines an event type, so there are 10 event types in total. We work on a randomly sampled subset of 2000 drivers and each driver has a sequence. We randomly sampled disjoint train, dev and test sets with 1400, 200 and 400 sequences.
+- StackOverflow ( `Leskovec, 2014 <https://snap.stanford.edu/data/>`_). This dataset has two years of user awards on a question-answering website: each user received a sequence of badges and there are 22 different kinds of badges in total. We randomly sampled disjoint train, dev and test sets with 1400,400 and 400 sequences from the dataset.
+- Taobao (`Xue et al, 2022 <https://arxiv.org/abs/2210.01753>`_). This dataset contains time-stamped user click behaviors on Taobao shopping pages from November 25 to December 03, 2017. Each user has a sequence of item click events with each event containing the timestamp and the category of the item. The categories of all items are first ranked by frequencies and the top 19 are kept while the rest are merged into one category, with each category corresponding to an event type. We work on a subset of 4800 most active users with an average sequence length of 150 and then end up with 20 event types.
+- Amazon (`Xue et al, 2022 <https://arxiv.org/abs/2210.01753>`_). This dataset includes time-stamped user product reviews behavior from January, 2008 to October, 2018. Each user has a sequence of produce review events with each event containing the timestamp and category of the reviewed product, with each category corresponding to an event type. We work on a subset of 5200 most active users with an average sequence length of 70 and then end up with 16 event types.
+
+Besides, we also published two textual event sequence datasets:
+
+- GDELT (`Shi et al, 2023  <https://arxiv.org/abs/2305.16646>`_). The GDELT Project monitors events all over the world, with live datasets updated every 15 minutes. We only focused on the political events that happened in G20 countries from 2022-01-01 to 2022-07-31, ending up with a corpus of 109000 time-stamped event tokens. The event type of each token has a structured name of the format subject-predicate-object. Each {predicate} is one of the twenty CAMEO codes such as {CONSULT} and {INVESTIGATE}; each {subject} or {object} is one of the 2279 political entities (individuals, groups, and states) such as {Tesla} and {Australia}. We split the dataset into disjoint train, dev, and test sets based on their dates: the 83100 events that happened before 2022-07-05 are training data; the 16650 events after 2022-07-19 are test data; the 9250 events between these dates are development data.
+- Amazon-text-review (`Shi et al, 2023  <https://arxiv.org/abs/2305.16646>`_). This dataset contains user reviews on Amazon shopping website from 2014-01-04 to 2016-10-02. We focused on the most active 2500 users and each user has a sequence of product review events. The type is the category of the product: we selected the most frequently-reviewed 23 categories and grouped all the others into a special OTHER category, ending up with 24 categories in total. Each review event also has a mark which is the actual content of the review. Each of the 2500 sequences is cut into three segments: the events that happened before 2015-08-01 are training data; those after 2016-02-01 are test data; the events between these dates are dev data. Then we have 49,680 training tokens, 7,020 dev tokens, and 13,090 test tokens.

docs/source/user_guide/run_eval.rst

@@ -0,0 +1,97 @@
+================================
+Evaluate a Model
+================================
+
+Step 1: Setup the config file
+===============================================
+
+Same as in the training pipeline, firstly we need to initialize the task configuration in the config file.
+
+Similar to the setup in `Training Pipeline <./run_train_pipeline.html>`_, we set the `stage` to `eval` and pass the `pretrained_model_dir` to ``the model_config``
+
+Note that the *pretrained_model_dir* can be found in the log of the training process.
+
+.. code-block:: yaml
+
+    NHP_eval:
+      base_config:
+        stage: eval
+        backend: torch
+        dataset_id: taxi
+        runner_id: std_tpp
+        base_dir: './checkpoints/'
+        model_id: NHP
+      trainer_config:
+        batch_size: 256
+        max_epoch: 1
+      model_config:
+        hidden_size: 64
+        use_ln: False
+        seed: 2019
+        gpu: 0
+        pretrained_model_dir: ./checkpoints/26507_4380788096_231111-101848/models/saved_model  # must provide this dir
+        thinning:
+          num_seq: 10
+          num_sample: 1
+          num_exp: 500 # number of i.i.d. Exp(intensity_bound) draws at one time in thinning algorithm
+          look_ahead_time: 10
+          patience_counter: 5 # the maximum iteration used in adaptive thinning
+          over_sample_rate: 5
+          num_samples_boundary: 5
+          dtime_max: 5
+
+
+
+
+A complete example of these files can be seen at `examples/example_config.yaml <https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/configs/experiment_config.yaml>`_ .
+
+
+Step 2: Run the evaluation script
+=================================
+
+Same as in the training pipeline, we need to initialize a ``ModelRunner`` object to do the evaluation.
+
+The following code is an example, which is a copy from `examples/train_nhp.py <https://github.com/ant-research/EasyTemporalPointProcess/blob/main/examples/train_nhp.py>`_ .
+
+
+.. code-block:: python
+
+    import argparse
+
+    from easy_tpp.config_factory import RunnerConfig
+    from easy_tpp.runner import Runner
+
+
+    def main():
+        parser = argparse.ArgumentParser()
+
+        parser.add_argument('--config_dir', type=str, required=False, default='configs/experiment_config.yaml',
+                            help='Dir of configuration yaml to train and evaluate the model.')
+
+        parser.add_argument('--experiment_id', type=str, required=False, default='RMTPP_eval',
+                            help='Experiment id in the config file.')
+
+        args = parser.parse_args()
+
+        config = RunnerConfig.build_from_yaml_file(args.config_dir, experiment_id=args.experiment_id)
+
+        model_runner = Runner.build_from_config(config)
+
+        model_runner.run()
+
+
+    if __name__ == '__main__':
+        main()
+
+
+
+
+Checkout the output
+====================
+
+The evaluation result will be print in the console and saved in the logs whose directory is specified in the
+out config file, i.e.:
+
+.. code-block:: bash
+
+    'output_config_dir': './checkpoints/NHP_test_conttime_20221002-13:19:23/NHP_test_output.yaml'

docs/source/user_guide/run_train_pipeline.rst

@@ -0,0 +1,245 @@
+============================================
+Training a Model & Configuration Explanation
+============================================
+
+This tutorial shows how one can use ``EasyTPP`` to train the implemented models.
+
+In principle, firstly we need to initialize a config yaml file, containing all the input configuration to guide the training and eval process. The overall structure of a config file is shown as below:
+
+.. code-block:: yaml
+
+    pipeline_config_id: ..  # name of the config for guiding the pipeline
+
+    data:
+        [Dataset ID]:  # name of the dataset, e.g, taxi
+            ....
+
+    [EXPERIMENT ID]:   # name of the experiment to run
+        base_config:
+          ....
+        model_config:
+           ...
+
+
+After the config file is setup, we can run the script, by specifying the `config directory` and `experiment id`, to start the pipeline. We currently provide a preset script at `examples/train_nhp.py`.
+
+
+Step 1: Setup the config file containing data and model configs
+================================================================
+
+
+To be specific, one needs to define the following entries in the config file:
+
+- **pipeline_config_id**: registered name of EasyTPP.Config objects, such as `runner_config` or `hpo_runner_config`. By reading this, the corresponding configuration class will be loaded for constructing the pipeline.
+
+.. code-block:: yaml
+
+    pipeline_config_id: runner_config
+
+
+- **data**:  dataset specifics. One can put multiple dataset specifics in the config file, but only one will be used in one experiment.
+
+    - *[DATASET ID]*: name of the dataset, e.g., taxi.
+    - *train_dir, valid_dir, test_dir*: directory of the datafile. For the moment we only accept pkl file (please see `Dataset <./dataset.html>`_ for details)
+    - *data_spec*: define the event type information.
+
+.. code-block:: yaml
+
+    data:
+      taxi:
+        data_format: pkl
+        train_dir: ../data/taxi/train.pkl
+        valid_dir: ../data/taxi/dev.pkl
+        test_dir: ../data/taxi/test.pkl
+        data_spec：
+            num_event_types: 7  # num of types excluding pad events.
+            pad_token_id: 6    # event type index for pad events
+            padding_side: right   # pad at the right end of the sequence
+            truncation_side: right   # truncate at the right end of the sequence
+            max_len: 100            # max sequence length used as model input
+
+- **[EXPERIMENT ID]**: name of the experiment to run in the pipeline. It contains two blocks of configs:
+
+*base_config* contains the pipeline framework related specifications.
+
+.. code-block:: yaml
+
+    base_config:
+        stage: train       # train, eval and generate
+        backend: tensorflow   # tensorflow and torch
+        dataset_id: conttime   # name of the dataset
+        runner_id: std_tpp     # registered name of the pipeline runner
+        model_id: RMTPP # model name  # registered name of the implemented model
+        base_dir: './checkpoints/'   # base dir to save the logs and models.
+
+
+
+*model_config* contains the model related specifications.
+
+
+.. code-block:: yaml
+
+      model_config:
+            hidden_size: 32
+            time_emb_size: 16
+            num_layers: 2
+            num_heads: 2
+            mc_num_sample_per_step: 20
+            sharing_param_layer: False
+            loss_integral_num_sample_per_step: 20
+            dropout: 0.0
+            use_ln: False
+            thinning_params:   # thinning algorithm for event sampling
+                  num_seq: 10
+                  num_sample: 1
+                  num_exp: 500 # number of i.i.d. Exp(intensity_bound) draws at one time in thinning algorithm
+                  look_ahead_time: 10
+                  patience_counter: 5 # the maximum iteration used in adaptive thinning
+                  over_sample_rate: 5
+                  num_samples_boundary: 5
+                  dtime_max: 5
+
+
+*trainer_config* contains the training related specifications.
+
+.. code-block:: yaml
+
+        trainer_config:   # trainer arguments
+            seed: 2019
+            gpu: 0
+            batch_size: 256
+            max_epoch: 10
+            shuffle: False
+            optimizer: adam
+            learning_rate: 1.e-3
+            valid_freq: 1
+            use_tfb: False
+            metrics: ['acc', 'rmse']
+
+
+
+
+A complete example of these files can be seen at *examples/example_config*.
+
+
+Step 2: Run the training script
+===============================================
+
+To run the training process, we simply need to call two functions:
+
+1. ``Config``: it reads the directory of the configs specified in Step 1 and do some processing to form a complete configuration.
+2. ``Runner``: it reads the configuration and setups the whole pipeline for training, evaluation and generation.
+
+
+The following code is an example, which is a copy from *examples/train_nhp.py*.
+
+
+.. code-block:: python
+
+    import argparse
+    from easy_tpp.config_factory import Config
+    from easy_tpp.runner import Runner
+
+
+    def main():
+        parser = argparse.ArgumentParser()
+
+        parser.add_argument('--config_dir', type=str, required=False, default='configs/experiment_config.yaml',
+                            help='Dir of configuration yaml to train and evaluate the model.')
+
+        parser.add_argument('--experiment_id', type=str, required=False, default='RMTPP_train',
+                            help='Experiment id in the config file.')
+
+        args = parser.parse_args()
+
+        config = Config.build_from_yaml_file(args.config_dir, experiment_id=args.experiment_id)
+
+        model_runner = Runner.build_from_config(config)
+
+        model_runner.run()
+
+
+    if __name__ == '__main__':
+        main()
+
+
+
+
+
+Checkout the output
+========================
+
+
+During training, the log, the best model based on valid set performance, the complete configuration file are all saved. The directory of the saved files is specified in 'base' of ``model_config.yaml``, i.e.,
+
+
+
+In the `./checkpoints/` folder, one find the correct subfolder by concatenating the 'experiment_id' and running timestamps. Inside that subfolder, there is a complete configuration file, e.g., ``NHP_train_output.yaml`` that records all the information used in the pipeline. The
+
+.. code-block:: yaml
+
+    data_config:
+      train_dir: ../data/conttime/train.pkl
+      valid_dir: ../data/conttime/dev.pkl
+      test_dir: ../data/conttime/test.pkl
+      specs:
+        num_event_types_pad: 6
+        num_event_types: 5
+        event_pad_index: 5
+      data_format: pkl
+    base_config:
+      stage: train
+      backend: tensorflow
+      dataset_id: conttime
+      runner_id: std_tpp
+      model_id: RMTPP
+      base_dir: ./checkpoints/
+      exp_id: RMTPP_train
+      log_folder: ./checkpoints/98888_4299965824_221205-153425
+      saved_model_dir: ./checkpoints/98888_4299965824_221205-153425/models/saved_model
+      saved_log_dir: ./checkpoints/98888_4299965824_221205-153425/log
+      output_config_dir: ./checkpoints/98888_4299965824_221205-153425/RMTPP_train_output.yaml
+    model_config:
+      hidden_size: 32
+      time_emb_size: 16
+      num_layers: 2
+      num_heads: 2
+      mc_num_sample_per_step: 20
+      sharing_param_layer: false
+      loss_integral_num_sample_per_step: 20
+      dropout: 0.0
+      use_ln: false
+      seed: 2019
+      gpu: 0
+      thinning_params:
+        num_seq: 10
+        num_sample: 1
+        num_exp: 500
+        look_ahead_time: 10
+        patience_counter: 5
+        over_sample_rate: 5
+        num_samples_boundary: 5
+        dtime_max: 5
+        num_step_gen: 1
+      trainer:
+        batch_size: 256
+        max_epoch: 10
+        shuffle: false
+        optimizer: adam
+        learning_rate: 0.001
+        valid_freq: 1
+        use_tfb: false
+        metrics:
+        - acc
+        - rmse
+        seq_pad_end: true
+      is_training: true
+      num_event_types_pad: 6
+      num_event_types: 5
+      event_pad_index: 5
+      model_id: RMTPP
+
+
+
+If we set ``use_tfb`` to ``true``, it means we can launch the tensorboard to track the training process, one
+can see `Running Tensorboard <../advanced/tensorboard.html>`_ for details.

easy_tpp/__init__.py

@@ -0,0 +1 @@
+__version__ = '0.1.0'

easy_tpp/config_factory/__init__.py

@@ -0,0 +1,13 @@
+from easy_tpp.config_factory.config import Config
+from easy_tpp.config_factory.data_config import DataConfig, DataSpecConfig
+from easy_tpp.config_factory.hpo_config import HPOConfig, HPORunnerConfig
+from easy_tpp.config_factory.runner_config import RunnerConfig, ModelConfig, BaseConfig
+
+__all__ = ['Config',
+           'DataConfig',
+           'DataSpecConfig',
+           'ModelConfig',
+           'BaseConfig',
+           'RunnerConfig',
+           'HPOConfig',
+           'HPORunnerConfig']

easy_tpp/config_factory/config.py

@@ -0,0 +1,120 @@
+from abc import abstractmethod
+from typing import Any
+from omegaconf import OmegaConf
+
+from easy_tpp.utils import save_yaml_config, Registrable, logger
+
+
+class Config(Registrable):
+
+    def save_to_yaml_file(self, config_dir):
+        """Save the config into the yaml file 'config_dir'.
+
+        Args:
+            config_dir (str): Target filename.
+
+        Returns:
+        """
+        yaml_config = self.get_yaml_config()
+        OmegaConf.save(yaml_config, config_dir)
+
+    @staticmethod
+    def build_from_yaml_file(yaml_dir, **kwargs):
+        """Load yaml config file from disk.
+
+        Args:
+            yaml_dir (str): Path of the yaml config file.
+
+        Returns:
+            EasyTPP.Config: Config object corresponding to cls.
+        """
+        config = OmegaConf.load(yaml_dir)
+        pipeline_config = config.get('pipeline_config_id')
+        config_cls = Config.by_name(pipeline_config.lower())
+        logger.critical(f'Load pipeline config class {config_cls.__name__}')
+        return config_cls.parse_from_yaml_config(config, **kwargs)
+
+    @abstractmethod
+    def get_yaml_config(self):
+        """Get the yaml format config from self.
+
+        Returns:
+        """
+        pass
+
+    @staticmethod
+    @abstractmethod
+    def parse_from_yaml_config(yaml_config):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            EasyTPP.Config: Config class for data.
+        """
+        pass
+
+    @abstractmethod
+    def copy(self):
+        """Get a same and freely modifiable copy of self.
+
+        Returns:
+        """
+        pass
+
+    def __str__(self):
+        """Str representation of the config.
+
+        Returns:
+            str: str representation of the dict format of the config.
+        """
+        return str(self.get_yaml_config())
+
+    def update(self, config):
+        """Update the config.
+
+        Args:
+            config (dict): config dict.
+
+        Returns:
+            EasyTPP.Config: Config class for data.
+        """
+        logger.critical(f'Update config class {self.__class__.__name__}')
+        return self.parse_from_yaml_config(config)
+
+    def pop(self, key: str, default_var: Any):
+        """pop out the key-value item from the config.
+
+        Args:
+            key (str): key name.
+            default_var (Any): default value to pop.
+
+        Returns:
+            Any: value to pop.
+        """
+        return vars(self).pop(key) or default_var
+
+    def get(self, key: str, default_var: Any):
+        """Retrieve the key-value item from the config.
+
+        Args:
+            key (str): key name.
+            default_var (Any): default value to pop.
+
+        Returns:
+            Any: value to get.
+        """
+        return vars(self)[key] or default_var
+
+    def set(self, key: str, var_to_set: Any):
+        """Set the key-value item from the config.
+
+        Args:
+            key (str): key name.
+            var_to_set (Any): default value to pop.
+
+        Returns:
+            Any: value to get.
+        """
+        vars(self)[key] = var_to_set

easy_tpp/config_factory/data_config.py

@@ -0,0 +1,147 @@
+from easy_tpp.config_factory.config import Config
+
+
+class DataSpecConfig(Config):
+    def __init__(self, **kwargs):
+        """Initialize the Config class.
+        """
+        self.num_event_types = kwargs.get('num_event_types')
+        self.pad_token_id = kwargs.get('pad_token_id')
+        self.padding_side = kwargs.get('padding_side')
+        self.truncation_side = kwargs.get('truncation_side')
+        self.padding_strategy = kwargs.get('padding_strategy')
+        self.max_len = kwargs.get('max_len')
+        self.truncation_strategy = kwargs.get('truncation_strategy')
+        self.num_event_types_pad = self.num_event_types + 1
+        self.model_input_names = kwargs.get('model_input_names')
+
+        if self.padding_side is not None and self.padding_side not in ["right", "left"]:
+            raise ValueError(
+                f"Padding side should be selected between 'right' and 'left', current value: {self.padding_side}"
+            )
+
+        if self.truncation_side is not None and self.truncation_side not in ["right", "left"]:
+            raise ValueError(
+                f"Truncation side should be selected between 'right' and 'left', current value: {self.truncation_side}"
+            )
+
+    def get_yaml_config(self):
+        """Return the config in dict (yaml compatible) format.
+
+        Returns:
+            dict: config of the data specs in dict format.
+        """
+        return {
+            'num_event_types': self.num_event_types,
+            'pad_token_id': self.pad_token_id,
+            'padding_side': self.padding_side,
+            'truncation_side': self.truncation_side,
+            'padding_strategy': self.padding_strategy,
+            'truncation_strategy': self.truncation_strategy,
+            'max_len': self.max_len
+        }
+
+    @staticmethod
+    def parse_from_yaml_config(yaml_config):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            DataSpecConfig: Config class for data specs.
+        """
+        return DataSpecConfig(**yaml_config)
+
+    def copy(self):
+        """Copy the config.
+
+        Returns:
+            DataSpecConfig: a copy of current config.
+        """
+        return DataSpecConfig(num_event_types_pad=self.num_event_types_pad,
+                              num_event_types=self.num_event_types,
+                              event_pad_index=self.pad_token_id,
+                              padding_side=self.padding_side,
+                              truncation_side=self.truncation_side,
+                              padding_strategy=self.padding_strategy,
+                              truncation_strategy=self.truncation_strategy,
+                              max_len=self.max_len)
+
+
+@Config.register('data_config')
+class DataConfig(Config):
+    def __init__(self, train_dir, valid_dir, test_dir, data_format, specs=None):
+        """Initialize the DataConfig object.
+
+        Args:
+            train_dir (str): dir of tran set.
+            valid_dir (str): dir of valid set.
+            test_dir (str): dir of test set.
+            specs (dict, optional): specs of dataset. Defaults to None.
+        """
+        self.train_dir = train_dir
+        self.valid_dir = valid_dir
+        self.test_dir = test_dir
+        self.data_specs = specs or DataSpecConfig()
+        self.data_format = train_dir.split('.')[-1] if data_format is None else data_format
+
+    def get_yaml_config(self):
+        """Return the config in dict (yaml compatible) format.
+
+        Returns:
+            dict: config of the data in dict format.
+        """
+        return {
+            'train_dir': self.train_dir,
+            'valid_dir': self.valid_dir,
+            'test_dir': self.test_dir,
+            'data_format': self.data_format,
+            'data_specs': self.data_specs.get_yaml_config(),
+        }
+
+    @staticmethod
+    def parse_from_yaml_config(yaml_config):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            EasyTPP.DataConfig: Config class for data.
+        """
+        return DataConfig(
+            train_dir=yaml_config.get('train_dir'),
+            valid_dir=yaml_config.get('valid_dir'),
+            test_dir=yaml_config.get('test_dir'),
+            data_format=yaml_config.get('data_format'),
+            specs=DataSpecConfig.parse_from_yaml_config(yaml_config.get('data_specs'))
+        )
+
+    def copy(self):
+        """Copy the config.
+
+        Returns:
+            EasyTPP.DataConfig: a copy of current config.
+        """
+        return DataConfig(train_dir=self.train_dir,
+                          valid_dir=self.valid_dir,
+                          test_dir=self.test_dir,
+                          specs=self.data_specs)
+
+    def get_data_dir(self, split):
+        """Get the dir of the source raw data.
+
+        Args:
+            split (str): dataset split notation, 'train', 'dev' or 'valid', 'test'.
+
+        Returns:
+            str: dir of the source raw data file.
+        """
+        split = split.lower()
+        if split == 'train':
+            return self.train_dir
+        elif split in ['dev', 'valid']:
+            return self.valid_dir
+        else:
+            return self.test_dir

easy_tpp/config_factory/hpo_config.py

@@ -0,0 +1,132 @@
+from easy_tpp.config_factory.config import Config
+from easy_tpp.config_factory.runner_config import RunnerConfig
+from easy_tpp.utils import parse_uri_to_protocol_and_path, py_assert
+
+
+class HPOConfig(Config):
+    def __init__(self, framework_id, storage_uri, is_continuous, num_trials, num_jobs):
+        """Initialize the HPO Config
+
+        Args:
+            framework_id (str): hpo framework id.
+            storage_uri (str): result storage dir.
+            is_continuous (bool): whether to continuously do the optimization.
+            num_trials (int): num of trails used in optimization.
+            num_jobs (int): num of the jobs.
+        """
+        self.framework_id = framework_id or 'optuna'
+        self.is_continuous = is_continuous if is_continuous is not None else True
+        self.num_trials = num_trials or 50
+        self.storage_uri = storage_uri
+        self.num_jobs = num_jobs if num_jobs is not None else 1
+
+    @property
+    def storage_protocol(self):
+        """Get the storage protocol
+
+        Returns:
+            str: the dir of the storage protocol.
+        """
+        storage_protocol, _ = parse_uri_to_protocol_and_path(self.storage_uri)
+        return storage_protocol
+
+    @property
+    def storage_path(self):
+        """Get the storage protocol
+
+        Returns:
+            str: the dir of the hpo data storage.
+        """
+        _, storage_path = parse_uri_to_protocol_and_path(self.storage_uri)
+        return storage_path
+
+    def get_yaml_config(self):
+        """Return the config in dict (yaml compatible) format.
+
+        Returns:
+            dict: config of the HPO specs in dict format.
+        """
+        return {
+            'framework_id': self.framework_id,
+            'storage_uri': self.storage_uri,
+            'is_continuous': self.is_continuous,
+            'num_trials': self.num_trials,
+            'num_jobs': self.num_jobs
+        }
+
+    @staticmethod
+    def parse_from_yaml_config(yaml_config, **kwargs):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            EasyTPP.HPOConfig: Config class for HPO specs.
+        """
+        if yaml_config is None:
+            return None
+        else:
+            return HPOConfig(
+                framework_id=yaml_config.get('framework_id'),
+                storage_uri=yaml_config.get('storage_uri'),
+                is_continuous=yaml_config.get('is_continuous'),
+                num_trials=yaml_config.get('num_trials'),
+                num_jobs=yaml_config.get('num_jobs'),
+            )
+
+    def copy(self):
+        """Copy the config.
+
+        Returns:
+            EasyTPP.HPOConfig: a copy of current config.
+        """
+        return HPOConfig(
+            framework_id=self.framework_id,
+            storage_uri=self.storage_uri,
+            is_continuous=self.is_continuous,
+            num_trials=self.num_trials,
+            num_jobs=self.num_jobs
+        )
+
+
+@Config.register('hpo_runner_config')
+class HPORunnerConfig(Config):
+    def __init__(self, hpo_config, runner_config):
+        """Initialize the config class
+
+        Args:
+            hpo_config (EasyTPP.HPOConfig): hpo config class.
+            runner_config (EasyTPP.RunnerConfig): runner config class.
+        """
+        self.hpo_config = hpo_config
+        self.runner_config = runner_config
+
+    @staticmethod
+    def parse_from_yaml_config(yaml_config, **kwargs):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            EasyTPP.HPORunnerConfig: Config class for HPO specs.
+        """
+        runner_config = RunnerConfig.parse_from_yaml_config(yaml_config, **kwargs)
+        hpo_config = HPOConfig.parse_from_yaml_config(yaml_config.get('hpo'), **kwargs)
+        py_assert(hpo_config is not None, ValueError, 'No hpo configs is provided for HyperTuner')
+        return HPORunnerConfig(
+            hpo_config=hpo_config,
+            runner_config=runner_config
+        )
+
+    def copy(self):
+        """Copy the config.
+
+        Returns:
+            EasyTPP.HPORunnerConfig: a copy of current config.
+        """
+        return HPORunnerConfig(
+            hpo_config=self.hpo_config,
+            runner_config=self.runner_config
+        )

easy_tpp/config_factory/model_config.py

@@ -0,0 +1,274 @@
+from easy_tpp.config_factory.config import Config
+
+from easy_tpp.utils.const import Backend
+
+
+class TrainerConfig(Config):
+
+    def __init__(self, **kwargs):
+        """Initialize the Config class.
+        """
+        self.seed = kwargs.get('seed', 9899)
+        self.gpu = kwargs.get('gpu', -1)
+        self.batch_size = kwargs.get('batch_size', 256)
+        self.max_epoch = kwargs.get('max_epoch', 10)
+        self.shuffle = kwargs.get('shuffle', False)
+        self.optimizer = kwargs.get('optimizer', 'adam')
+        self.learning_rate = kwargs.get('learning_rate', 1.e-3)
+        self.valid_freq = kwargs.get('valid_freq', 1)
+        self.use_tfb = kwargs.get('use_tfb', False)
+        self.metrics = kwargs.get('metrics', ['acc', 'rmse'])
+
+    def get_yaml_config(self):
+        """Return the config in dict (yaml compatible) format.
+
+        Returns:
+            dict: config of the trainer specs in dict format.
+        """
+        return {'seed': self.seed,
+                'gpu': self.gpu,
+                'batch_size': self.batch_size,
+                'max_epoch': self.max_epoch,
+                'shuffle': self.shuffle,
+                'optimizer': self.optimizer,
+                'learning_rate': self.learning_rate,
+                'valid_freq': self.valid_freq,
+                'use_tfb': self.use_tfb,
+                'metrics': self.metrics
+                }
+
+    @staticmethod
+    def parse_from_yaml_config(yaml_config):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            EasyTPP.TrainerConfig: Config class for trainer specs.
+        """
+        return TrainerConfig(**yaml_config)
+
+    def copy(self):
+        """Copy the config.
+
+        Returns:
+            EasyTPP.TrainerConfig: a copy of current config.
+        """
+        return TrainerConfig(batch_size=self.batch_size,
+                             max_epoch=self.max_epoch,
+                             shuffle=self.shuffle,
+                             optimizer=self.optimizer,
+                             learning_rate=self.learning_rate,
+                             valid_freq=self.valid_freq,
+                             use_tfb=self.use_tfb,
+                             metrics=self.metrics
+                             )
+
+
+class ThinningConfig(Config):
+    def __init__(self, **kwargs):
+        """Initialize the Config class.
+        """
+        self.num_seq = kwargs.get('num_seq', 10)
+        self.num_sample = kwargs.get('num_sample', 1)
+        self.num_exp = kwargs.get('num_exp', 500)
+        self.look_ahead_time = kwargs.get('look_ahead_time', 10)
+        self.patience_counter = kwargs.get('patience_counter', 5)
+        self.over_sample_rate = kwargs.get('over_sample_rate', 5)
+        self.num_samples_boundary = kwargs.get('num_samples_boundary', 5)
+        self.dtime_max = kwargs.get('dtime_max', 5)
+        # we pad the sequence at the front only in multi-step generation
+        self.num_step_gen = kwargs.get('num_step_gen', 1)
+
+    def get_yaml_config(self):
+        """Return the config in dict (yaml compatible) format.
+
+        Returns:
+            dict: config of the thinning specs in dict format.
+        """
+        return {'num_seq': self.num_seq,
+                'num_sample': self.num_sample,
+                'num_exp': self.num_exp,
+                'look_ahead_time': self.look_ahead_time,
+                'patience_counter': self.patience_counter,
+                'over_sample_rate': self.over_sample_rate,
+                'num_samples_boundary': self.num_samples_boundary,
+                'dtime_max': self.dtime_max,
+                'num_step_gen': self.num_step_gen}
+
+    @staticmethod
+    def parse_from_yaml_config(yaml_config):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            EasyTPP.ThinningConfig: Config class for thinning algorithms.
+        """
+        return ThinningConfig(**yaml_config) if yaml_config is not None else None
+
+    def copy(self):
+        """Copy the config.
+
+        Returns:
+            EasyTPP.ThinningConfig: a copy of current config.
+        """
+        return ThinningConfig(num_seq=self.num_seq,
+                              num_sample=self.num_sample,
+                              num_exp=self.num_exp,
+                              look_ahead_time=self.look_ahead_time,
+                              patience_counter=self.patience_counter,
+                              over_sample_rate=self.over_sample_rate,
+                              num_samples_boundary=self.num_samples_boundary,
+                              dtime_max=self.dtime_max,
+                              num_step_gen=self.num_step_gen)
+
+
+class BaseConfig(Config):
+    def __init__(self, **kwargs):
+        """Initialize the Config class.
+        """
+        self.stage = kwargs.get('stage')
+        self.backend = kwargs.get('backend')
+        self.dataset_id = kwargs.get('dataset_id')
+        self.runner_id = kwargs.get('runner_id')
+        self.model_id = kwargs.get('model_id')
+        self.exp_id = kwargs.get('exp_id')
+        self.base_dir = kwargs.get('base_dir')
+        self.specs = kwargs.get('specs', {})
+        self.backend = self.set_backend(self.backend)
+
+    @staticmethod
+    def set_backend(backend):
+        if backend.lower() in ['torch', 'pytorch']:
+            return Backend.Torch
+        else:
+            raise ValueError(
+                f"Backend should be 'torch' or 'pytorch', current value: {backend}"
+            )
+
+    def get_yaml_config(self):
+        """Return the config in dict (yaml compatible) format.
+
+        Returns:
+            dict: config of the base config specs in dict format.
+        """
+        return {'stage': self.stage,
+                'backend': str(self.backend),
+                'dataset_id': self.dataset_id,
+                'runner_id': self.runner_id,
+                'model_id': self.model_id,
+                'base_dir': self.base_dir,
+                'specs': self.specs}
+
+    @staticmethod
+    def parse_from_yaml_config(yaml_config):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            BaseConfig: Config class for trainer specs.
+        """
+        return BaseConfig(**yaml_config)
+
+    def copy(self):
+        """Copy the config.
+
+        Returns:
+            BaseConfig: a copy of current config.
+        """
+        return BaseConfig(stage=self.stage,
+                          backend=self.backend,
+                          dataset_id=self.dataset_id,
+                          runner_id=self.runner_id,
+                          model_id=self.model_id,
+                          base_dir=self.base_dir,
+                          specs=self.specs)
+
+
+class ModelConfig(Config):
+    def __init__(self, **kwargs):
+        """Initialize the Config class.
+        """
+        self.rnn_type = kwargs.get('rnn_type', 'LSTM')
+        self.hidden_size = kwargs.get('hidden_size', 32)
+        self.time_emb_size = kwargs.get('time_emb_size', 16)
+        self.num_layers = kwargs.get('num_layers', 2)
+        self.num_heads = kwargs.get('num_heads', 2)
+        self.sharing_param_layer = kwargs.get('sharing_param_layer', False)
+        self.use_mc_samples = kwargs.get('use_mc_samples', True)  # if using MC samples in computing log-likelihood
+        self.loss_integral_num_sample_per_step = kwargs.get('loss_integral_num_sample_per_step', 20)  # mc_num_sample_per_step
+        self.dropout_rate = kwargs.get('dropout_rate', 0.0)
+        self.use_ln = kwargs.get('use_ln', False)
+        self.thinning = ThinningConfig.parse_from_yaml_config(kwargs.get('thinning'))
+        self.is_training = kwargs.get('training', False)
+        self.num_event_types_pad = kwargs.get('num_event_types_pad', None)
+        self.num_event_types = kwargs.get('num_event_types', None)
+        self.pad_token_id = kwargs.get('event_pad_index', None)
+        self.model_id = kwargs.get('model_id', None)
+        self.pretrained_model_dir = kwargs.get('pretrained_model_dir', None)
+        self.gpu = kwargs.get('gpu', -1)
+        self.model_specs = kwargs.get('model_specs', {})
+
+    def get_yaml_config(self):
+        """Return the config in dict (yaml compatible) format.
+
+        Returns:
+            dict: config of the model config specs in dict format.
+        """
+        return {'rnn_type': self.rnn_type,
+                'hidden_size': self.hidden_size,
+                'time_emb_size': self.time_emb_size,
+                'num_layers': self.num_layers,
+                'sharing_param_layer': self.sharing_param_layer,
+                'loss_integral_num_sample_per_step': self.loss_integral_num_sample_per_step,
+                'dropout_rate': self.dropout_rate,
+                'use_ln': self.use_ln,
+                # for some models / cases we may not need to pass thinning config
+                # e.g., for intensity-free model
+                'thinning': None if self.thinning is None else self.thinning.get_yaml_config(),
+                'num_event_types_pad': self.num_event_types_pad,
+                'num_event_types': self.num_event_types,
+                'event_pad_index': self.pad_token_id,
+                'model_id': self.model_id,
+                'pretrained_model_dir': self.pretrained_model_dir,
+                'gpu': self.gpu,
+                'model_specs': self.model_specs}
+
+    @staticmethod
+    def parse_from_yaml_config(yaml_config):
+        """Parse from the yaml to generate the config object.
+
+        Args:
+            yaml_config (dict): configs from yaml file.
+
+        Returns:
+            ModelConfig: Config class for trainer specs.
+        """
+        return ModelConfig(**yaml_config)
+
+    def copy(self):
+        """Copy the config.
+
+        Returns:
+            ModelConfig: a copy of current config.
+        """
+        return ModelConfig(rnn_type=self.rnn_type,
+                           hidden_size=self.hidden_size,
+                           time_emb_size=self.time_emb_size,
+                           num_layers=self.num_layers,
+                           sharing_param_layer=self.sharing_param_layer,
+                           loss_integral_num_sample_per_step=self.loss_integral_num_sample_per_step,
+                           dropout_rate=self.dropout_rate,
+                           use_ln=self.use_ln,
+                           thinning=self.thinning,
+                           num_event_types_pad=self.num_event_types_pad,
+                           num_event_types=self.num_event_types,
+                           event_pad_index=self.pad_token_id,
+                           pretrained_model_dir=self.pretrained_model_dir,
+                           gpu=self.gpu,
+                           model_specs=self.model_specs)