Upload AROD checkpoint

README.md

@@ -0,0 +1,48 @@
+---
+license: other
+tags:
+- audio-driven-animation
+- talking-head
+- streamavatar
+- dystream
+- ar-od
+pipeline_tag: image-to-video
+---
+
+# StreamAvatar AROD Student Checkpoint
+
+This repository hosts the public StreamAvatar AROD real-anchor student checkpoint.
+
+AROD stands for Autoregressive One-step Denoising. It is a blockwise student model distilled from a DyStream teacher for faster audio-to-motion inference in the StreamAvatar project.
+
+## Files
+
+- `blockwise_latest.pt`: AROD real-anchor student checkpoint.
+- `config.yaml`: sanitized inference/training configuration for the checkpoint.
+
+## Download
+
+```bash
+pip install huggingface-hub
+
+mkdir -p outputs/blockwise_stream_distill_cross_fm_teacher_cache_anchor_pretrain_60k
+huggingface-cli download pancx/StreamAvatar-AROD blockwise_latest.pt \
+  --local-dir outputs/blockwise_stream_distill_cross_fm_teacher_cache_anchor_pretrain_60k
+```
+
+Expected SHA256:
+
+```text
+01893fabb842fcc8e9817a8e2530108d75932aad4f6ac4136e5c22b94702e860
+```
+
+## Project
+
+Code and full setup instructions are available at:
+
+```text
+https://github.com/CXP-2024/StreamAvatar
+```
+
+The checkpoint requires the StreamAvatar/DyStream codebase, the original DyStream teacher checkpoint, Wav2Vec2 assets, and the renderer checkpoint described in the project README.
+

blockwise_latest.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01893fabb842fcc8e9817a8e2530108d75932aad4f6ac4136e5c22b94702e860
+size 410383509

config.yaml

@@ -0,0 +1,78 @@
+# Sanitized public config for pancx/StreamAvatar-AROD.
+# Runtime paths are relative to the StreamAvatar repository root.
+
+seed: 123
+
+checkpoint_path: checkpoints/last.ckpt
+wav2vec_path: pretrained_model/wav2vec2-base-960h
+output_dir: outputs/blockwise_stream_distill_cross_fm_teacher_cache_anchor_pretrain_60k
+
+model:
+  module_name: model.motion_generation.motion_gen_gpt_flowmatching_addaudio_linear_twowavencoder
+  class_name: Audio2FaceGPT
+  pose_fps: 25
+  audio_sr: 16000
+  audio_fps: 16000
+  wav2vec_layer: 8
+  cbh_window_length: 96
+  vae_codebook_size: 512
+  seed: 123
+  prev_audio_frames: 32
+  ema_decay: 0.999
+  cfg_audio: 0.5
+  cfg_audio_other: 0.5
+  cfg_anchor: 0
+  cfg_all: 1.0
+  cfg_fusion: 2.0
+  cfg_audio_anchor: 1.0
+  drop_audio: 0.1
+  drop_audio_other: 0.1
+  drop_anchor: 0.1
+  drop_gpt: 0.1
+  drop_style: 0.3
+  drop_other: 0.0
+  drop_self: 0.0
+  drop_fusion: 0.1
+  drop_prev_motion: 0.0
+  cfg_gpt: 1.0
+  cfg_style: 1.0
+  cfg_prev_motion: 0.0
+  sync_only: true
+  use_init_motion: false
+  strength: 1.0
+  window_size: 8
+  total_iter: 600000
+  eval_metrics: false
+  version: '0506'
+  two_step: false
+  infer_32: false
+  qk_norm: rms_norm_across_heads
+  eps: 1.0e-06
+  cross_attn_norm: true
+  added_kv_proj_dim: null
+
+teacher:
+  denoising_steps: 1
+  use_ema: true
+  guidance_mode: all_only
+
+student:
+  architecture: cross_fm
+  block_frames: 2
+  history_frames: 32
+  hidden_dim: 512
+  layers: 6
+  heads: 8
+  dropout: 0.1
+  detach_between_blocks: true
+
+loss:
+  velocity_weight: 1.5
+  acceleration_weight: 0.2
+  boundary_weight: 0.3
+  delta_weight: 0.0
+
+noise_scheduler:
+  num_train_timesteps: 1000
+  shift: 1
+  use_karras_sigmas: false