LoRa121 commited on
Commit
d4da447
·
verified ·
1 Parent(s): b646f87

Upload 12 files

Browse files
README.md ADDED
@@ -0,0 +1,2481 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import copy
4
+ import math
5
+ import pickle
6
+ import threading
7
+ from dataclasses import dataclass
8
+ from typing import Any, Dict, List, Optional, Tuple, Union
9
+
10
+ import numpy as np
11
+ import cv2
12
+ import torch
13
+
14
+
15
+ # ============================================================
16
+ # ComfyUI Node (pose_data + PKL)
17
+ # ============================================================
18
+
19
+ _GLOBAL_LOCK = threading.Lock()
20
+
21
+
22
+ class KPSSmoothPoseDataAndRender:
23
+ """
24
+ Сглаживание + рендер позы.
25
+ Вход: POSEDATA (как объект/dict; обычно приходит из TSLoadPoseDataPickle).
26
+ Выход: IMAGE (torch [T,H,W,3] float 0..1), POSEDATA (в том же формате, но сглаженный).
27
+ """
28
+
29
+ @classmethod
30
+ def INPUT_TYPES(cls):
31
+ return {
32
+ "required": {
33
+ "pose_data": ("POSEDATA",), # <-- ВАЖНО: именно POSEDATA
34
+ "filter_extra_people": ("BOOLEAN", {"default": True}),
35
+ # общий набор параметров сглаживания (вместо body + face_hands)
36
+ "smooth_alpha": ("FLOAT", {"default": 0.7, "min": 0.01, "max": 0.99, "step": 0.01}),
37
+ "gap_frames": ("INT", {"default": 12, "min": 0, "max": 100, "step": 1}),
38
+ "min_run_frames": ("INT", {"default": 2, "min": 1, "max": 60, "step": 1}),
39
+ # пороги отрисовки (в инпут добавляем body/hands, face НЕ добавляем)
40
+ "conf_thresh_body": ("FLOAT", {"default": 0.20, "min": 0.0, "max": 1.0, "step": 0.01}),
41
+ "conf_thresh_hands": ("FLOAT", {"default": 0.50, "min": 0.0, "max": 1.0, "step": 0.01}),
42
+ }
43
+ }
44
+
45
+ RETURN_TYPES = ("IMAGE", "POSEDATA") # <-- ВАЖНО: именно POSEDATA
46
+ RETURN_NAMES = ("IMAGE", "pose_data")
47
+ FUNCTION = "run"
48
+ CATEGORY = "posedata"
49
+
50
+ def run(self, pose_data, **kwargs):
51
+ filter_extra_people = bool(kwargs.get("filter_extra_people", True))
52
+
53
+ # общий набор
54
+ smooth_alpha = float(kwargs.get("smooth_alpha", 0.7))
55
+ gap_frames = int(kwargs.get("gap_frames", 12))
56
+ min_run_frames = int(kwargs.get("min_run_frames", 2))
57
+
58
+ # пороги рендера
59
+ conf_thresh_body = float(kwargs.get("conf_thresh_body", 0.20))
60
+ conf_thresh_hands = float(kwargs.get("conf_thresh_hands", 0.50))
61
+ conf_thresh_face = 0.20 # <- НЕ добавляем в INPUT, но фиксируем как ты просил
62
+
63
+ force_body_18 = bool(kwargs.get("force_body_18", False))
64
+
65
+ pose_data = _coerce_pose_data_to_obj(pose_data)
66
+
67
+ # pose_data -> frames_json_like
68
+ frames_json_like, meta_ref = _pose_data_to_kps_frames(pose_data, force_body_18=force_body_18)
69
+
70
+ with _GLOBAL_LOCK:
71
+ old = _snapshot_tunable_globals()
72
+ try:
73
+ # BODY
74
+ globals()["ALPHA_BODY"] = smooth_alpha
75
+ globals()["SUPER_SMOOTH_ALPHA"] = smooth_alpha
76
+ globals()["MAX_GAP_FRAMES"] = gap_frames
77
+ globals()["MIN_RUN_FRAMES"] = min_run_frames
78
+
79
+ # FACE+HANDS (dense) тоже от общего набора
80
+ globals()["DENSE_SUPER_SMOOTH_ALPHA"] = smooth_alpha
81
+ globals()["DENSE_MAX_GAP_FRAMES"] = gap_frames
82
+ globals()["DENSE_MIN_RUN_FRAMES"] = min_run_frames
83
+
84
+ globals()["FILTER_EXTRA_PEOPLE"] = filter_extra_people
85
+
86
+ smoothed_frames = smooth_KPS_json_obj(
87
+ frames_json_like,
88
+ keep_face_untouched=False,
89
+ keep_hands_untouched=False,
90
+ filter_extra_people=filter_extra_people,
91
+ )
92
+ finally:
93
+ _restore_tunable_globals(old)
94
+
95
+ # frames_json_like -> pose_data (обратно в pose_metas)
96
+ out_pose_data = _kps_frames_to_pose_data(pose_data, smoothed_frames, meta_ref, force_body_18=force_body_18)
97
+
98
+ # render
99
+ w, h = _extract_canvas_wh(smoothed_frames, default_w=720, default_h=1280)
100
+ frames_np = []
101
+ for fr in smoothed_frames:
102
+ if isinstance(fr, dict) and fr.get("people"):
103
+ img = _draw_pose_frame_full(
104
+ w,
105
+ h,
106
+ fr["people"][0],
107
+ conf_thresh_body=conf_thresh_body,
108
+ conf_thresh_hands=conf_thresh_hands,
109
+ conf_thresh_face=conf_thresh_face,
110
+ )
111
+ else:
112
+ img = np.zeros((h, w, 3), dtype=np.uint8)
113
+ frames_np.append(img)
114
+
115
+ frames_t = torch.from_numpy(np.stack(frames_np, axis=0)).float() / 255.0
116
+ return (frames_t, out_pose_data)
117
+
118
+
119
+ # ============================================================
120
+ # PKL / pose_data IO
121
+ # ============================================================
122
+
123
+
124
+ class _PoseDummyObj:
125
+ def __init__(self, *a, **k):
126
+ pass
127
+
128
+ def __setstate__(self, state):
129
+ # поддержка dict и (dict, slotstate)
130
+ if isinstance(state, dict):
131
+ self.__dict__.update(state)
132
+ elif isinstance(state, (list, tuple)) and len(state) == 2 and isinstance(state[0], dict):
133
+ self.__dict__.update(state[0])
134
+ if isinstance(state[1], dict):
135
+ self.__dict__.update(state[1])
136
+ else:
137
+ self.__dict__["_slotstate"] = state[1]
138
+ else:
139
+ self.__dict__["_state"] = state
140
+
141
+
142
+ class _SafeUnpickler(pickle.Unpickler):
143
+ """
144
+ Безопасно грузим PKL из ComfyUI окружения:
145
+ - ремап numpy._core -> numpy.core
146
+ - неизвестные классы (WanAnimatePreprocess.*) превращаем в простые объекты с __dict__
147
+ """
148
+
149
+ def find_class(self, module, name):
150
+ # ремап внутренних путей numpy (частая проблема между версиями)
151
+ if module.startswith("numpy._core"):
152
+ module = module.replace("numpy._core", "numpy.core", 1)
153
+ if module.startswith("numpy._globals"):
154
+ module = module.replace("numpy._globals", "numpy", 1)
155
+
156
+ # конкретные классы метаданных (если встречаются)
157
+ if name in {"AAPoseMeta"}:
158
+ return _PoseDummyObj
159
+
160
+ try:
161
+ return super().find_class(module, name)
162
+ except Exception:
163
+ return _PoseDummyObj
164
+
165
+
166
+ def _load_pose_data_pkl(path: str) -> Any:
167
+ with open(path, "rb") as f:
168
+ return _SafeUnpickler(f).load()
169
+
170
+
171
+ def _coerce_pose_data_to_obj(pd: Any) -> Any:
172
+ """
173
+ Accepts:
174
+ - dict pose_data
175
+ - object with attributes like .pose_metas (AAPoseMeta-like)
176
+ - str path to .pkl
177
+ - dict wrapper with 'pose_data'
178
+ """
179
+ if isinstance(pd, str):
180
+ obj = _load_pose_data_pkl(pd)
181
+ return obj
182
+
183
+ if isinstance(pd, dict) and "pose_data" in pd:
184
+ return pd["pose_data"]
185
+
186
+ return pd
187
+
188
+
189
+ # ============================================================
190
+ # pose_data <-> JSON-like KPS frames
191
+ # ============================================================
192
+
193
+
194
+ def _as_attr(x: Any, key: str, default=None):
195
+ if isinstance(x, dict):
196
+ return x.get(key, default)
197
+ return getattr(x, key, default)
198
+
199
+
200
+ def _set_attr(x: Any, key: str, value: Any):
201
+ if isinstance(x, dict):
202
+ x[key] = value
203
+ else:
204
+ setattr(x, key, value)
205
+
206
+
207
+ def _xy_p_to_flat(xy: Optional[np.ndarray], p: Optional[np.ndarray]) -> Optional[List[float]]:
208
+ if xy is None:
209
+ return None
210
+ arr = np.asarray(xy)
211
+ if arr.ndim != 2 or arr.shape[1] < 2:
212
+ return None
213
+ N = arr.shape[0]
214
+ if p is None:
215
+ pp = np.ones((N,), dtype=np.float32)
216
+ else:
217
+ pp = np.asarray(p).reshape(-1)
218
+ if pp.shape[0] != N:
219
+ # если вдруг не совпали — подстрахуемся
220
+ pp = np.ones((N,), dtype=np.float32)
221
+
222
+ out: List[float] = []
223
+ for i in range(N):
224
+ out.extend([float(arr[i, 0]), float(arr[i, 1]), float(pp[i])])
225
+ return out
226
+
227
+
228
+ def _flat_to_xy_p(flat: Optional[List[float]]) -> Tuple[Optional[np.ndarray], Optional[np.ndarray]]:
229
+ if not isinstance(flat, list) or len(flat) % 3 != 0:
230
+ return None, None
231
+ N = len(flat) // 3
232
+ xy = np.zeros((N, 2), dtype=np.float32)
233
+ p = np.zeros((N,), dtype=np.float32)
234
+ for i in range(N):
235
+ xy[i, 0] = float(flat[3 * i + 0])
236
+ xy[i, 1] = float(flat[3 * i + 1])
237
+ p[i] = float(flat[3 * i + 2])
238
+ return xy, p
239
+
240
+
241
+ def _pose_data_to_kps_frames(pose_data: Any, *, force_body_18: bool) -> Tuple[List[Dict[str, Any]], Dict[str, Any]]:
242
+ """
243
+ Делает "как JSON" список кадров:
244
+ frame = {"people":[{pose_keypoints_2d, face_keypoints_2d, hand_left_keypoints_2d, hand_right_keypoints_2d}],
245
+ "canvas_width": W, "canvas_height": H}
246
+ meta_ref: ссылки на pose_metas + тип/доступ, чтобы правильно записать обратно.
247
+ """
248
+ pose_metas = _as_attr(pose_data, "pose_metas", None)
249
+ if pose_metas is None:
250
+ # иногда называют иначе
251
+ pose_metas = _as_attr(pose_data, "frames", None)
252
+
253
+ if pose_metas is None or not isinstance(pose_metas, list):
254
+ raise ValueError("pose_data does not contain 'pose_metas' list.")
255
+
256
+ frames: List[Dict[str, Any]] = []
257
+ for meta in pose_metas:
258
+ h = _as_attr(meta, "height", 1280)
259
+ w = _as_attr(meta, "width", 720)
260
+
261
+ kps_body = _as_attr(meta, "kps_body", None)
262
+ kps_body_p = _as_attr(meta, "kps_body_p", None)
263
+
264
+ kps_face = _as_attr(meta, "kps_face", None)
265
+ kps_face_p = _as_attr(meta, "kps_face_p", None)
266
+
267
+ kps_lhand = _as_attr(meta, "kps_lhand", None)
268
+ kps_lhand_p = _as_attr(meta, "kps_lhand_p", None)
269
+
270
+ kps_rhand = _as_attr(meta, "kps_rhand", None)
271
+ kps_rhand_p = _as_attr(meta, "kps_rhand_p", None)
272
+
273
+ # to flat
274
+ pose_flat = _xy_p_to_flat(kps_body, kps_body_p)
275
+ face_flat = _xy_p_to_flat(kps_face, kps_face_p)
276
+ lh_flat = _xy_p_to_flat(kps_lhand, kps_lhand_p)
277
+ rh_flat = _xy_p_to_flat(kps_rhand, kps_rhand_p)
278
+
279
+ if force_body_18 and isinstance(pose_flat, list) and len(pose_flat) >= 18 * 3:
280
+ pose_flat = pose_flat[: 18 * 3]
281
+
282
+ person = {
283
+ "pose_keypoints_2d": pose_flat if pose_flat is not None else [],
284
+ "face_keypoints_2d": face_flat if face_flat is not None else [],
285
+ "hand_left_keypoints_2d": lh_flat,
286
+ "hand_right_keypoints_2d": rh_flat,
287
+ }
288
+
289
+ frame = {"people": [person], "canvas_height": int(h), "canvas_width": int(w)}
290
+ frames.append(frame)
291
+
292
+ meta_ref = {
293
+ "pose_metas": pose_metas,
294
+ "len": len(pose_metas),
295
+ }
296
+ return frames, meta_ref
297
+
298
+
299
+ def _kps_frames_to_pose_data(
300
+ pose_data_in: Any,
301
+ frames_kps: List[Dict[str, Any]],
302
+ meta_ref: Dict[str, Any],
303
+ *,
304
+ force_body_18: bool,
305
+ ) -> Any:
306
+ """
307
+ Записывает обратно сглаженные keypoints в pose_metas[*].kps_* / kps_*_p.
308
+ Остальные поля pose_data сохраняем.
309
+ """
310
+ out_pd = copy.deepcopy(pose_data_in)
311
+ pose_metas_out = _as_attr(out_pd, "pose_metas", None)
312
+ if pose_metas_out is None:
313
+ # fallback: вдруг другой ключ
314
+ pose_metas_out = meta_ref.get("pose_metas")
315
+
316
+ if pose_metas_out is None or not isinstance(pose_metas_out, list):
317
+ raise ValueError("Failed to locate pose_metas in output pose_data.")
318
+
319
+ T = min(len(pose_metas_out), len(frames_kps))
320
+ for t in range(T):
321
+ meta = pose_metas_out[t]
322
+ fr = frames_kps[t]
323
+ people = fr.get("people", []) if isinstance(fr, dict) else []
324
+ p0 = people[0] if people else None
325
+ if not isinstance(p0, dict):
326
+ continue
327
+
328
+ pose_flat = p0.get("pose_keypoints_2d")
329
+ face_flat = p0.get("face_keypoints_2d")
330
+ lh_flat = p0.get("hand_left_keypoints_2d")
331
+ rh_flat = p0.get("hand_right_keypoints_2d")
332
+
333
+ if force_body_18 and isinstance(pose_flat, list) and len(pose_flat) >= 18 * 3:
334
+ pose_flat = pose_flat[: 18 * 3]
335
+
336
+ body_xy, body_p = _flat_to_xy_p(pose_flat if isinstance(pose_flat, list) else None)
337
+ face_xy, face_p = _flat_to_xy_p(face_flat if isinstance(face_flat, list) else None)
338
+ lh_xy, lh_p = _flat_to_xy_p(lh_flat if isinstance(lh_flat, list) else None)
339
+ rh_xy, rh_p = _flat_to_xy_p(rh_flat if isinstance(rh_flat, list) else None)
340
+
341
+ if body_xy is not None and body_p is not None:
342
+ _set_attr(meta, "kps_body", body_xy.astype(np.float32, copy=False))
343
+ _set_attr(meta, "kps_body_p", body_p.astype(np.float32, copy=False))
344
+
345
+ if face_xy is not None and face_p is not None:
346
+ _set_attr(meta, "kps_face", face_xy.astype(np.float32, copy=False))
347
+ _set_attr(meta, "kps_face_p", face_p.astype(np.float32, copy=False))
348
+
349
+ if lh_xy is not None and lh_p is not None:
350
+ _set_attr(meta, "kps_lhand", lh_xy.astype(np.float32, copy=False))
351
+ _set_attr(meta, "kps_lhand_p", lh_p.astype(np.float32, copy=False))
352
+
353
+ if rh_xy is not None and rh_p is not None:
354
+ _set_attr(meta, "kps_rhand", rh_xy.astype(np.float32, copy=False))
355
+ _set_attr(meta, "kps_rhand_p", rh_p.astype(np.float32, copy=False))
356
+
357
+ # обновим width/height если нужно
358
+ if isinstance(fr, dict):
359
+ if "canvas_width" in fr:
360
+ _set_attr(meta, "width", int(fr["canvas_width"]))
361
+ if "canvas_height" in fr:
362
+ _set_attr(meta, "height", int(fr["canvas_height"]))
363
+
364
+ # обязательно положим pose_metas обратно
365
+ _set_attr(out_pd, "pose_metas", pose_metas_out)
366
+ return out_pd
367
+
368
+
369
+ def _extract_canvas_wh(data: Any, default_w: int, default_h: int) -> Tuple[int, int]:
370
+ w, h = int(default_w), int(default_h)
371
+ if isinstance(data, list):
372
+ for fr in data:
373
+ if isinstance(fr, dict) and "canvas_width" in fr and "canvas_height" in fr:
374
+ try:
375
+ w = int(fr["canvas_width"])
376
+ h = int(fr["canvas_height"])
377
+ break
378
+ except Exception:
379
+ pass
380
+ return w, h
381
+
382
+
383
+ # ============================================================
384
+ # === START: smooth_KPS_json.py logic (ported as-is)
385
+ # ============================================================
386
+
387
+ # --- Root+Scale carry (when torso disappears on close-up) ---
388
+ ROOTSCALE_CARRY_ENABLED = True
389
+ CARRY_MAX_FRAMES = 48
390
+ CARRY_MIN_ANCHORS = 2
391
+ CARRY_ANCHOR_JOINTS = [0, 1, 2, 5, 3, 6, 4, 7]
392
+ CARRY_CONF_GATE = 0.20
393
+
394
+ # --- Main person selection / multi-person filtering ---
395
+ FILTER_EXTRA_PEOPLE = True
396
+ MAIN_PERSON_MODE = "longest_track"
397
+ TRACK_MATCH_MIN_PX = 80.0
398
+ TRACK_MATCH_FACTOR = 3.0
399
+ TRACK_MAX_FRAME_GAP = 32
400
+
401
+ # --- Spatial outlier suppression ---
402
+ SPATIAL_OUTLIER_FIX = True
403
+ BONE_MAX_FACTOR = 2.3
404
+ TORSO_RADIUS_FACTOR = 4.0
405
+
406
+ # EMA smoothing for BODY only (online)
407
+ ALPHA_BODY = 0.70
408
+ MAX_STEP_BODY = 60.0
409
+ VEL_ALPHA = 0.45
410
+ EPS = 0.3
411
+ CONF_GATE_BODY = 0.20
412
+ CONF_FLOOR_BODY = 0.00
413
+
414
+ TRACK_DIST_PENALTY = 1.5
415
+ FACE_WEIGHT_IN_SCORE = 0.15
416
+ HAND_WEIGHT_IN_SCORE = 0.35
417
+
418
+ ALLOW_DISAPPEAR_JOINTS = {3, 4, 6, 7}
419
+
420
+ GAP_FILL_ENABLED = True
421
+ MAX_GAP_FRAMES = 12
422
+ MIN_RUN_FRAMES = 2
423
+
424
+ TORSO_SYNC_ENABLED = True
425
+ TORSO_JOINTS = {1, 2, 5, 8, 11}
426
+ TORSO_LOOKAHEAD_FRAMES = 32
427
+
428
+ SUPER_SMOOTH_ENABLED = True
429
+ SUPER_SMOOTH_ALPHA = 0.7
430
+ SUPER_SMOOTH_MIN_CONF = 0.20
431
+
432
+ MEDIAN3_ENABLED = True
433
+
434
+ FACE_SMOOTH_ENABLED = True
435
+ HANDS_SMOOTH_ENABLED = False
436
+
437
+ CONF_GATE_FACE = 0.20
438
+ CONF_GATE_HAND = 0.50
439
+
440
+ HAND_MIN_POINTS_PRESENT = 7
441
+ MIN_HAND_RUN_FRAMES = 6
442
+
443
+ DENSE_GAP_FILL_ENABLED = False
444
+ DENSE_MAX_GAP_FRAMES = 8
445
+ DENSE_MIN_RUN_FRAMES = 2
446
+
447
+ DENSE_MEDIAN3_ENABLED = False
448
+ DENSE_SUPER_SMOOTH_ENABLED = False
449
+ DENSE_SUPER_SMOOTH_ALPHA = 0.7
450
+
451
+
452
+ def _snapshot_tunable_globals() -> Dict[str, Any]:
453
+ keys = [
454
+ "FILTER_EXTRA_PEOPLE",
455
+ "SUPER_SMOOTH_ALPHA",
456
+ "MAX_GAP_FRAMES",
457
+ "MIN_RUN_FRAMES",
458
+ "DENSE_SUPER_SMOOTH_ALPHA",
459
+ "DENSE_MAX_GAP_FRAMES",
460
+ "DENSE_MIN_RUN_FRAMES",
461
+ ]
462
+ return {k: globals().get(k) for k in keys}
463
+
464
+
465
+ def _restore_tunable_globals(old: Dict[str, Any]) -> None:
466
+ for k, v in old.items():
467
+ globals()[k] = v
468
+
469
+
470
+ def _is_valid_xyc(x: float, y: float, c: float) -> bool:
471
+ if c is None:
472
+ return False
473
+ if c <= 0:
474
+ return False
475
+ if x == 0 and y == 0:
476
+ return False
477
+ if math.isnan(x) or math.isnan(y) or math.isnan(c):
478
+ return False
479
+ return True
480
+
481
+
482
+ def _reshape_keypoints_2d(arr: List[float]) -> List[Tuple[float, float, float]]:
483
+ if arr is None:
484
+ return []
485
+ if len(arr) % 3 != 0:
486
+ raise ValueError(f"keypoints length not multiple of 3: {len(arr)}")
487
+ out = []
488
+ for i in range(0, len(arr), 3):
489
+ out.append((float(arr[i]), float(arr[i + 1]), float(arr[i + 2])))
490
+ return out
491
+
492
+
493
+ def _flatten_keypoints_2d(kps: List[Tuple[float, float, float]]) -> List[float]:
494
+ out: List[float] = []
495
+ for x, y, c in kps:
496
+ out.extend([float(x), float(y), float(c)])
497
+ return out
498
+
499
+
500
+ def _sum_conf(arr: Optional[List[float]], sample_step: int = 1) -> float:
501
+ if not arr:
502
+ return 0.0
503
+ s = 0.0
504
+ for i in range(2, len(arr), 3 * sample_step):
505
+ try:
506
+ c = float(arr[i])
507
+ except Exception:
508
+ c = 0.0
509
+ if c > 0:
510
+ s += c
511
+ return s
512
+
513
+
514
+ def _body_center_from_pose(pose_arr: Optional[List[float]]) -> Optional[Tuple[float, float]]:
515
+ if not pose_arr:
516
+ return None
517
+ kps = _reshape_keypoints_2d(pose_arr)
518
+ idxs = [2, 5, 8, 11, 1]
519
+ pts = []
520
+ for idx in idxs:
521
+ if idx < len(kps):
522
+ x, y, c = kps[idx]
523
+ if _is_valid_xyc(x, y, c):
524
+ pts.append((x, y))
525
+ if not pts:
526
+ for x, y, c in kps:
527
+ if _is_valid_xyc(x, y, c):
528
+ pts.append((x, y))
529
+ if not pts:
530
+ return None
531
+ cx = sum(p[0] for p in pts) / len(pts)
532
+ cy = sum(p[1] for p in pts) / len(pts)
533
+ return (cx, cy)
534
+
535
+
536
+ def _dist(a: Tuple[float, float], b: Tuple[float, float]) -> float:
537
+ return math.hypot(a[0] - b[0], a[1] - b[1])
538
+
539
+
540
+ def _choose_single_person(
541
+ people: List[Dict[str, Any]], prev_center: Optional[Tuple[float, float]]
542
+ ) -> Optional[Dict[str, Any]]:
543
+ if not people:
544
+ return None
545
+ best = None
546
+ best_score = -1e18
547
+
548
+ for p in people:
549
+ pose = p.get("pose_keypoints_2d")
550
+ face = p.get("face_keypoints_2d")
551
+ lh = p.get("hand_left_keypoints_2d")
552
+ rh = p.get("hand_right_keypoints_2d")
553
+
554
+ score = _sum_conf(pose)
555
+ score += FACE_WEIGHT_IN_SCORE * _sum_conf(face, sample_step=4)
556
+ score += HAND_WEIGHT_IN_SCORE * (_sum_conf(lh, sample_step=2) + _sum_conf(rh, sample_step=2))
557
+
558
+ center = _body_center_from_pose(pose)
559
+ if prev_center is not None and center is not None:
560
+ score -= TRACK_DIST_PENALTY * _dist(prev_center, center)
561
+
562
+ if score > best_score:
563
+ best_score = score
564
+ best = p
565
+
566
+ return best
567
+
568
+
569
+ @dataclass
570
+ class _Track:
571
+ frames: Dict[int, Dict[str, Any]]
572
+ centers: Dict[int, Tuple[float, float]]
573
+ last_t: int
574
+ last_center: Tuple[float, float]
575
+
576
+
577
+ def _estimate_torso_scale(pose: List[Tuple[float, float, float]]) -> Optional[float]:
578
+ def dist(i, k) -> Optional[float]:
579
+ if i >= len(pose) or k >= len(pose):
580
+ return None
581
+ xi, yi, ci = pose[i]
582
+ xk, yk, ck = pose[k]
583
+ if not _is_valid_xyc(xi, yi, ci) or not _is_valid_xyc(xk, yk, ck):
584
+ return None
585
+ return math.hypot(xi - xk, yi - yk)
586
+
587
+ cand = [dist(2, 5), dist(8, 11), dist(1, 8), dist(1, 11)]
588
+ cand = [c for c in cand if c is not None and c > 1e-3]
589
+ if not cand:
590
+ return None
591
+ return float(sum(cand) / len(cand))
592
+
593
+
594
+ def _track_match_threshold_from_pose(pose_arr: Optional[List[float]]) -> float:
595
+ if isinstance(pose_arr, list):
596
+ pose = _reshape_keypoints_2d(pose_arr)
597
+ s = _estimate_torso_scale(pose)
598
+ if s is not None:
599
+ return max(float(TRACK_MATCH_MIN_PX), float(TRACK_MATCH_FACTOR) * float(s))
600
+ return float(max(TRACK_MATCH_MIN_PX, 120.0))
601
+
602
+
603
+ def _build_tracks_over_video(frames_data: List[Any]) -> List[_Track]:
604
+ tracks: List[_Track] = []
605
+
606
+ for t, frame in enumerate(frames_data):
607
+ if not isinstance(frame, dict):
608
+ continue
609
+ people = frame.get("people", [])
610
+ if not isinstance(people, list) or not people:
611
+ continue
612
+
613
+ cand: List[Tuple[int, Dict[str, Any], Tuple[float, float]]] = []
614
+ for i, p in enumerate(people):
615
+ if not isinstance(p, dict):
616
+ continue
617
+ pose = p.get("pose_keypoints_2d")
618
+ c = _body_center_from_pose(pose)
619
+ if c is None:
620
+ continue
621
+ cand.append((i, p, c))
622
+
623
+ if not cand:
624
+ continue
625
+
626
+ used = set()
627
+ track_order = sorted(range(len(tracks)), key=lambda k: tracks[k].last_t, reverse=True)
628
+
629
+ for k in track_order:
630
+ tr = tracks[k]
631
+ age = t - tr.last_t
632
+ if age > int(TRACK_MAX_FRAME_GAP):
633
+ continue
634
+
635
+ best_idx = None
636
+ best_d = 1e18
637
+
638
+ for i, p, cc in cand:
639
+ if i in used:
640
+ continue
641
+
642
+ thr = _track_match_threshold_from_pose(p.get("pose_keypoints_2d"))
643
+ d = _dist(tr.last_center, cc)
644
+ if d <= thr and d < best_d:
645
+ best_d = d
646
+ best_idx = i
647
+
648
+ if best_idx is not None:
649
+ i, p, cc = next(x for x in cand if x[0] == best_idx)
650
+ used.add(i)
651
+ tr.frames[t] = p
652
+ tr.centers[t] = cc
653
+ tr.last_t = t
654
+ tr.last_center = cc
655
+
656
+ for i, p, cc in cand:
657
+ if i in used:
658
+ continue
659
+ tracks.append(_Track(frames={t: p}, centers={t: cc}, last_t=t, last_center=cc))
660
+
661
+ return tracks
662
+
663
+
664
+ def _track_presence_score(tr: _Track) -> Tuple[int, float, float]:
665
+ frames_count = len(tr.frames)
666
+ face_sum = 0.0
667
+ body_sum = 0.0
668
+ for p in tr.frames.values():
669
+ face_sum += _sum_conf(p.get("face_keypoints_2d"), sample_step=4)
670
+ body_sum += _sum_conf(p.get("pose_keypoints_2d"), sample_step=1)
671
+ return (frames_count, face_sum, body_sum)
672
+
673
+
674
+ def _pick_main_track(tracks: List[_Track]) -> Optional[_Track]:
675
+ if not tracks:
676
+ return None
677
+ best = None
678
+ best_key = (-1, -1e18, -1e18)
679
+ for tr in tracks:
680
+ key = _track_presence_score(tr)
681
+ if key > best_key:
682
+ best_key = key
683
+ best = tr
684
+ return best
685
+
686
+
687
+ @dataclass
688
+ class BodyState:
689
+ last_xy: List[Optional[Tuple[float, float]]]
690
+ last_v: List[Tuple[float, float]]
691
+
692
+ def __init__(self, joints: int):
693
+ self.last_xy = [None] * joints
694
+ self.last_v = [(0.0, 0.0)] * joints
695
+
696
+
697
+ def _smooth_body_pose(pose_arr: Optional[List[float]], state: BodyState) -> Optional[List[float]]:
698
+ if pose_arr is None:
699
+ return None
700
+
701
+ kps = _reshape_keypoints_2d(pose_arr)
702
+ J = len(kps)
703
+ if len(state.last_xy) != J:
704
+ state.last_xy = [None] * J
705
+ state.last_v = [(0.0, 0.0)] * J
706
+
707
+ out: List[Tuple[float, float, float]] = []
708
+
709
+ for j in range(J):
710
+ x, y, c = kps[j]
711
+ last = state.last_xy[j]
712
+ vx_last, vy_last = state.last_v[j]
713
+
714
+ valid_in = _is_valid_xyc(x, y, c) and (c >= CONF_GATE_BODY)
715
+
716
+ if valid_in:
717
+ if last is None:
718
+ nx, ny = x, y
719
+ state.last_xy[j] = (nx, ny)
720
+ state.last_v[j] = (0.0, 0.0)
721
+ out.append((nx, ny, float(c)))
722
+ continue
723
+
724
+ dx_raw = x - last[0]
725
+ dy_raw = y - last[1]
726
+ if abs(dx_raw) < EPS:
727
+ dx_raw = 0.0
728
+ if abs(dy_raw) < EPS:
729
+ dy_raw = 0.0
730
+
731
+ vx = VEL_ALPHA * dx_raw + (1.0 - VEL_ALPHA) * vx_last
732
+ vy = VEL_ALPHA * dy_raw + (1.0 - VEL_ALPHA) * vy_last
733
+
734
+ px = last[0] + vx
735
+ py = last[1] + vy
736
+
737
+ nx = ALPHA_BODY * x + (1.0 - ALPHA_BODY) * px
738
+ ny = ALPHA_BODY * y + (1.0 - ALPHA_BODY) * py
739
+
740
+ ddx = nx - last[0]
741
+ ddy = ny - last[1]
742
+ d = math.hypot(ddx, ddy)
743
+ if d > MAX_STEP_BODY and d > 1e-6:
744
+ scale = MAX_STEP_BODY / d
745
+ nx = last[0] + ddx * scale
746
+ ny = last[1] + ddy * scale
747
+ vx = nx - last[0]
748
+ vy = ny - last[1]
749
+
750
+ state.last_xy[j] = (nx, ny)
751
+ state.last_v[j] = (vx, vy)
752
+
753
+ out.append((nx, ny, float(c)))
754
+ else:
755
+ out.append((float(x), float(y), float(c)))
756
+
757
+ return _flatten_keypoints_2d(out)
758
+
759
+
760
+ COCO18_EDGES = [
761
+ (1, 2),
762
+ (2, 3),
763
+ (3, 4),
764
+ (1, 5),
765
+ (5, 6),
766
+ (6, 7),
767
+ (1, 8),
768
+ (8, 9),
769
+ (9, 10),
770
+ (1, 11),
771
+ (11, 12),
772
+ (12, 13),
773
+ (8, 11),
774
+ (1, 0),
775
+ (0, 14),
776
+ (14, 16),
777
+ (0, 15),
778
+ (15, 17),
779
+ ]
780
+
781
+ HAND21_EDGES = [
782
+ (0, 1),
783
+ (1, 2),
784
+ (2, 3),
785
+ (3, 4),
786
+ (0, 5),
787
+ (5, 6),
788
+ (6, 7),
789
+ (7, 8),
790
+ (0, 9),
791
+ (9, 10),
792
+ (10, 11),
793
+ (11, 12),
794
+ (0, 13),
795
+ (13, 14),
796
+ (14, 15),
797
+ (15, 16),
798
+ (0, 17),
799
+ (17, 18),
800
+ (18, 19),
801
+ (19, 20),
802
+ ]
803
+
804
+ _NEIGHBORS = None
805
+
806
+
807
+ def _build_neighbors():
808
+ global _NEIGHBORS
809
+ if _NEIGHBORS is not None:
810
+ return
811
+ neigh = {}
812
+ for a, b in COCO18_EDGES:
813
+ neigh.setdefault(a, set()).add(b)
814
+ neigh.setdefault(b, set()).add(a)
815
+ _NEIGHBORS = neigh
816
+
817
+
818
+ def _suppress_spatial_outliers_in_pose_arr(
819
+ pose_arr: Optional[List[float]], *, conf_gate: float
820
+ ) -> Optional[List[float]]:
821
+ if not isinstance(pose_arr, list) or len(pose_arr) % 3 != 0:
822
+ return pose_arr
823
+
824
+ pose = _reshape_keypoints_2d(pose_arr)
825
+ J = len(pose)
826
+
827
+ center = _body_center_from_pose(pose_arr)
828
+ scale = _estimate_torso_scale(pose)
829
+ if center is None or scale is None:
830
+ return pose_arr
831
+
832
+ cx, cy = center
833
+ max_r = TORSO_RADIUS_FACTOR * scale
834
+ max_bone = BONE_MAX_FACTOR * scale
835
+
836
+ out = [list(p) for p in pose]
837
+
838
+ def visible(j: int) -> bool:
839
+ if j >= J:
840
+ return False
841
+ x, y, c = out[j]
842
+ return (c >= conf_gate) and not (x == 0 and y == 0)
843
+
844
+ for j in range(J):
845
+ x, y, c = out[j]
846
+ if c >= conf_gate and not (x == 0 and y == 0):
847
+ if math.hypot(x - cx, y - cy) > max_r:
848
+ out[j] = [0.0, 0.0, 0.0]
849
+
850
+ for a, b in COCO18_EDGES:
851
+ if a >= J or b >= J:
852
+ continue
853
+ if not visible(a) or not visible(b):
854
+ continue
855
+ ax, ay, ac = out[a]
856
+ bx, by, bc = out[b]
857
+ d = math.hypot(ax - bx, ay - by)
858
+ if d > max_bone:
859
+ if ac <= bc:
860
+ out[a] = [0.0, 0.0, 0.0]
861
+ else:
862
+ out[b] = [0.0, 0.0, 0.0]
863
+
864
+ flat: List[float] = []
865
+ for x, y, c in out:
866
+ flat.extend([float(x), float(y), float(c)])
867
+ return flat
868
+
869
+
870
+ def _suppress_isolated_joints_in_pose_arr(
871
+ pose_arr: Optional[List[float]], *, conf_gate: float, keep: set[int] = None
872
+ ) -> Optional[List[float]]:
873
+ if not isinstance(pose_arr, list) or len(pose_arr) % 3 != 0:
874
+ return pose_arr
875
+
876
+ _build_neighbors()
877
+ pose = _reshape_keypoints_2d(pose_arr)
878
+ J = len(pose)
879
+ out = [list(p) for p in pose]
880
+
881
+ if keep is None:
882
+ keep = set()
883
+
884
+ def vis(j: int) -> bool:
885
+ if j >= J:
886
+ return False
887
+ x, y, c = out[j]
888
+ return (c >= conf_gate) and not (x == 0 and y == 0)
889
+
890
+ for j in range(J):
891
+ if j in keep:
892
+ continue
893
+ if not vis(j):
894
+ continue
895
+ neighs = _NEIGHBORS.get(j, set())
896
+ if not any((n < J and vis(n)) for n in neighs):
897
+ out[j] = [0.0, 0.0, 0.0]
898
+
899
+ flat = []
900
+ for x, y, c in out:
901
+ flat.extend([float(x), float(y), float(c)])
902
+ return flat
903
+
904
+
905
+ def _denoise_and_fill_gaps_pose_seq(
906
+ pose_arr_seq: List[Optional[List[float]]],
907
+ *,
908
+ conf_gate: float,
909
+ min_run: int,
910
+ max_gap: int,
911
+ ) -> List[Optional[List[float]]]:
912
+ if not pose_arr_seq:
913
+ return pose_arr_seq
914
+
915
+ J = None
916
+ for arr in pose_arr_seq:
917
+ if isinstance(arr, list) and len(arr) % 3 == 0 and len(arr) > 0:
918
+ J = len(arr) // 3
919
+ break
920
+ if J is None:
921
+ return pose_arr_seq
922
+
923
+ T = len(pose_arr_seq)
924
+ out_seq: List[Optional[List[float]]] = []
925
+ for arr in pose_arr_seq:
926
+ if isinstance(arr, list) and len(arr) == J * 3:
927
+ out_seq.append(list(arr))
928
+ else:
929
+ out_seq.append(arr)
930
+
931
+ def is_vis(arr: List[float], j: int) -> bool:
932
+ x = float(arr[3 * j + 0])
933
+ y = float(arr[3 * j + 1])
934
+ c = float(arr[3 * j + 2])
935
+ return (c >= conf_gate) and not (x == 0 and y == 0)
936
+
937
+ # 1) remove short flashes
938
+ for j in range(J):
939
+ start = None
940
+ for t in range(T + 1):
941
+ cur = False
942
+ if t < T and isinstance(out_seq[t], list):
943
+ cur = is_vis(out_seq[t], j)
944
+ if cur and start is None:
945
+ start = t
946
+ if (not cur) and start is not None:
947
+ run_len = t - start
948
+ if run_len < min_run:
949
+ for k in range(start, t):
950
+ if not isinstance(out_seq[k], list):
951
+ continue
952
+ out_seq[k][3 * j + 0] = 0.0
953
+ out_seq[k][3 * j + 1] = 0.0
954
+ out_seq[k][3 * j + 2] = 0.0
955
+ start = None
956
+
957
+ # 2) gap fill only if returns
958
+ for j in range(J):
959
+ last_vis_t = None
960
+ t = 0
961
+ while t < T:
962
+ arr = out_seq[t]
963
+ if not isinstance(arr, list):
964
+ t += 1
965
+ continue
966
+
967
+ cur_vis = is_vis(arr, j)
968
+ if cur_vis:
969
+ last_vis_t = t
970
+ t += 1
971
+ continue
972
+
973
+ if last_vis_t is None:
974
+ t += 1
975
+ continue
976
+
977
+ gap_start = t
978
+ t2 = t
979
+ while t2 < T:
980
+ arr2 = out_seq[t2]
981
+ if isinstance(arr2, list) and is_vis(arr2, j):
982
+ break
983
+ t2 += 1
984
+
985
+ if t2 >= T:
986
+ break
987
+
988
+ gap_len = t2 - gap_start
989
+ if gap_len <= 0:
990
+ t = t2
991
+ continue
992
+
993
+ if gap_len <= max_gap:
994
+ a = out_seq[last_vis_t]
995
+ b = out_seq[t2]
996
+ if isinstance(a, list) and isinstance(b, list):
997
+ ax, ay, ac = float(a[3 * j + 0]), float(a[3 * j + 1]), float(a[3 * j + 2])
998
+ bx, by, bc = float(b[3 * j + 0]), float(b[3 * j + 1]), float(b[3 * j + 2])
999
+ if not (ax == 0 and ay == 0) and not (bx == 0 and by == 0):
1000
+ conf_fill = min(ac, bc)
1001
+ for k in range(gap_len):
1002
+ tt = gap_start + k
1003
+ if not isinstance(out_seq[tt], list):
1004
+ continue
1005
+ r = (k + 1) / (gap_len + 1)
1006
+ x = ax + (bx - ax) * r
1007
+ y = ay + (by - ay) * r
1008
+ out_seq[tt][3 * j + 0] = float(x)
1009
+ out_seq[tt][3 * j + 1] = float(y)
1010
+ out_seq[tt][3 * j + 2] = float(conf_fill)
1011
+
1012
+ t = t2
1013
+
1014
+ return out_seq
1015
+
1016
+
1017
+ def _zero_lag_ema_pose_seq(
1018
+ pose_seq: List[Optional[List[float]]], *, alpha: float, conf_gate: float
1019
+ ) -> List[Optional[List[float]]]:
1020
+ if not pose_seq:
1021
+ return pose_seq
1022
+
1023
+ J = None
1024
+ for arr in pose_seq:
1025
+ if isinstance(arr, list) and len(arr) % 3 == 0 and len(arr) > 0:
1026
+ J = len(arr) // 3
1027
+ break
1028
+ if J is None:
1029
+ return pose_seq
1030
+
1031
+ T = len(pose_seq)
1032
+
1033
+ def is_vis(arr: List[float], j: int) -> bool:
1034
+ x = float(arr[3 * j + 0])
1035
+ y = float(arr[3 * j + 1])
1036
+ c = float(arr[3 * j + 2])
1037
+ return (c >= conf_gate) and not (x == 0 and y == 0)
1038
+
1039
+ fwd = [None] * T
1040
+ last = [None] * J
1041
+ for t in range(T):
1042
+ arr = pose_seq[t]
1043
+ if not isinstance(arr, list) or len(arr) != J * 3:
1044
+ fwd[t] = arr
1045
+ continue
1046
+ out = list(arr)
1047
+ for j in range(J):
1048
+ if is_vis(arr, j):
1049
+ x = float(arr[3 * j + 0])
1050
+ y = float(arr[3 * j + 1])
1051
+ if last[j] is None:
1052
+ sx, sy = x, y
1053
+ else:
1054
+ sx = alpha * x + (1 - alpha) * last[j][0]
1055
+ sy = alpha * y + (1 - alpha) * last[j][1]
1056
+ last[j] = (sx, sy)
1057
+ out[3 * j + 0] = float(sx)
1058
+ out[3 * j + 1] = float(sy)
1059
+ fwd[t] = out
1060
+
1061
+ bwd = [None] * T
1062
+ last = [None] * J
1063
+ for t in range(T - 1, -1, -1):
1064
+ arr = fwd[t]
1065
+ if not isinstance(arr, list) or len(arr) != J * 3:
1066
+ bwd[t] = arr
1067
+ continue
1068
+ out = list(arr)
1069
+ for j in range(J):
1070
+ if is_vis(arr, j):
1071
+ x = float(arr[3 * j + 0])
1072
+ y = float(arr[3 * j + 1])
1073
+ if last[j] is None:
1074
+ sx, sy = x, y
1075
+ else:
1076
+ sx = alpha * x + (1 - alpha) * last[j][0]
1077
+ sy = alpha * y + (1 - alpha) * last[j][1]
1078
+ last[j] = (sx, sy)
1079
+ out[3 * j + 0] = float(sx)
1080
+ out[3 * j + 1] = float(sy)
1081
+ bwd[t] = out
1082
+
1083
+ return bwd
1084
+
1085
+
1086
+ def _apply_root_scale(
1087
+ pose_arr: Optional[List[float]],
1088
+ *,
1089
+ src_root: Tuple[float, float],
1090
+ src_scale: float,
1091
+ dst_root: Tuple[float, float],
1092
+ dst_scale: float,
1093
+ ) -> Optional[List[float]]:
1094
+ if not isinstance(pose_arr, list) or len(pose_arr) % 3 != 0:
1095
+ return pose_arr
1096
+ if src_scale <= 1e-6 or dst_scale <= 1e-6:
1097
+ return pose_arr
1098
+
1099
+ kps = _reshape_keypoints_2d(pose_arr)
1100
+ out = []
1101
+ s = dst_scale / src_scale
1102
+
1103
+ for x, y, c in kps:
1104
+ if c <= 0 or (x == 0 and y == 0):
1105
+ out.append((x, y, c))
1106
+ continue
1107
+ nx = dst_root[0] + (x - src_root[0]) * s
1108
+ ny = dst_root[1] + (y - src_root[1]) * s
1109
+ out.append((nx, ny, c))
1110
+
1111
+ return _flatten_keypoints_2d(out)
1112
+
1113
+
1114
+ def _carry_pose_when_torso_missing(
1115
+ pose_seq: List[Optional[List[float]]],
1116
+ *,
1117
+ conf_gate: float,
1118
+ max_carry: int,
1119
+ anchor_joints: List[int],
1120
+ min_anchors: int,
1121
+ ) -> List[Optional[List[float]]]:
1122
+ if not pose_seq:
1123
+ return pose_seq
1124
+
1125
+ J = None
1126
+ for arr in pose_seq:
1127
+ if isinstance(arr, list) and len(arr) % 3 == 0 and len(arr) > 0:
1128
+ J = len(arr) // 3
1129
+ break
1130
+ if J is None:
1131
+ return pose_seq
1132
+
1133
+ out = [a if a is None else list(a) for a in pose_seq]
1134
+
1135
+ FILL_JOINTS = {1, 8, 9, 10, 11, 12, 13}
1136
+ FILL_JOINTS -= set(ALLOW_DISAPPEAR_JOINTS)
1137
+
1138
+ def is_vis_flat(arr: List[float], j: int) -> bool:
1139
+ x = float(arr[3 * j + 0])
1140
+ y = float(arr[3 * j + 1])
1141
+ c = float(arr[3 * j + 2])
1142
+ return (c >= conf_gate) and not (x == 0 and y == 0)
1143
+
1144
+ def count_visible(arr: List[float], joints: List[int]) -> int:
1145
+ c = 0
1146
+ for j in joints:
1147
+ if j < J and is_vis_flat(arr, j):
1148
+ c += 1
1149
+ return c
1150
+
1151
+ def root_scale_from_anchors(arr: List[float]) -> Optional[Tuple[Tuple[float, float], float]]:
1152
+ pts = []
1153
+ for j in anchor_joints:
1154
+ if j >= J:
1155
+ continue
1156
+ if is_vis_flat(arr, j):
1157
+ x = float(arr[3 * j + 0])
1158
+ y = float(arr[3 * j + 1])
1159
+ pts.append((x, y))
1160
+ if len(pts) < min_anchors:
1161
+ return None
1162
+
1163
+ rx = sum(p[0] for p in pts) / len(pts)
1164
+ ry = sum(p[1] for p in pts) / len(pts)
1165
+
1166
+ xs = [p[0] for p in pts]
1167
+ ys = [p[1] for p in pts]
1168
+ scale = max(max(xs) - min(xs), max(ys) - min(ys))
1169
+ if scale <= 1e-3:
1170
+ return None
1171
+
1172
+ return (rx, ry), float(scale)
1173
+
1174
+ last_good: Optional[List[float]] = None
1175
+ last_good_rs: Optional[Tuple[Tuple[float, float], float]] = None
1176
+ carry_left = 0
1177
+
1178
+ for t in range(len(out)):
1179
+ arr = out[t]
1180
+ if not isinstance(arr, list) or len(arr) != J * 3:
1181
+ continue
1182
+
1183
+ anchors_ok = count_visible(arr, anchor_joints) >= min_anchors
1184
+ fill_vis = sum(1 for j in FILL_JOINTS if j < J and is_vis_flat(arr, j))
1185
+ rs = root_scale_from_anchors(arr)
1186
+
1187
+ if anchors_ok and rs is not None and fill_vis >= 2:
1188
+ last_good = list(arr)
1189
+ last_good_rs = rs
1190
+ carry_left = max_carry
1191
+ continue
1192
+
1193
+ if anchors_ok and rs is not None and last_good is not None and last_good_rs is not None and carry_left > 0:
1194
+ dst_root, dst_scale = rs
1195
+ src_root, src_scale = last_good_rs
1196
+
1197
+ carried_full = _apply_root_scale(
1198
+ last_good,
1199
+ src_root=src_root,
1200
+ src_scale=src_scale,
1201
+ dst_root=dst_root,
1202
+ dst_scale=dst_scale,
1203
+ )
1204
+ if isinstance(carried_full, list) and len(carried_full) == J * 3:
1205
+ for j in FILL_JOINTS:
1206
+ if j >= J:
1207
+ continue
1208
+ if is_vis_flat(arr, j):
1209
+ continue
1210
+
1211
+ cx = float(carried_full[3 * j + 0])
1212
+ cy = float(carried_full[3 * j + 1])
1213
+ cc = float(carried_full[3 * j + 2])
1214
+
1215
+ if (cx == 0 and cy == 0) or cc <= 0:
1216
+ continue
1217
+
1218
+ arr[3 * j + 0] = cx
1219
+ arr[3 * j + 1] = cy
1220
+ arr[3 * j + 2] = max(min(cc, 0.60), conf_gate)
1221
+
1222
+ out[t] = arr
1223
+ carry_left -= 1
1224
+ continue
1225
+
1226
+ carry_left = max(carry_left - 1, 0)
1227
+
1228
+ return out
1229
+
1230
+
1231
+ def _force_full_torso_pair(
1232
+ pose_seq: List[Optional[List[float]]],
1233
+ *,
1234
+ conf_gate: float,
1235
+ anchor_joints: List[int],
1236
+ min_anchors: int,
1237
+ max_lookback: int = 240,
1238
+ fill_legs_with_hip: bool = True,
1239
+ always_fill_if_one_hip: bool = True,
1240
+ ) -> List[Optional[List[float]]]:
1241
+ if not pose_seq:
1242
+ return pose_seq
1243
+
1244
+ J = None
1245
+ for arr in pose_seq:
1246
+ if isinstance(arr, list) and len(arr) % 3 == 0 and len(arr) > 0:
1247
+ J = len(arr) // 3
1248
+ break
1249
+ if J is None:
1250
+ return pose_seq
1251
+
1252
+ out = [a if a is None else list(a) for a in pose_seq]
1253
+
1254
+ R_HIP, R_KNEE, R_ANK = 8, 9, 10
1255
+ L_HIP, L_KNEE, L_ANK = 11, 12, 13
1256
+
1257
+ def is_vis(arr: List[float], j: int) -> bool:
1258
+ if j >= J:
1259
+ return False
1260
+ x = float(arr[3 * j + 0])
1261
+ y = float(arr[3 * j + 1])
1262
+ c = float(arr[3 * j + 2])
1263
+ return (c >= conf_gate) and not (x == 0 and y == 0)
1264
+
1265
+ def count_visible(arr: List[float], joints: List[int]) -> int:
1266
+ c = 0
1267
+ for j in joints:
1268
+ if is_vis(arr, j):
1269
+ c += 1
1270
+ return c
1271
+
1272
+ def root_scale_from_anchors(arr: List[float]) -> Optional[Tuple[Tuple[float, float], float]]:
1273
+ pts = []
1274
+ for j in anchor_joints:
1275
+ if j >= J:
1276
+ continue
1277
+ if is_vis(arr, j):
1278
+ pts.append((float(arr[3 * j + 0]), float(arr[3 * j + 1])))
1279
+ if len(pts) < min_anchors:
1280
+ return None
1281
+
1282
+ rx = sum(p[0] for p in pts) / len(pts)
1283
+ ry = sum(p[1] for p in pts) / len(pts)
1284
+
1285
+ xs = [p[0] for p in pts]
1286
+ ys = [p[1] for p in pts]
1287
+ scale = max(max(xs) - min(xs), max(ys) - min(ys))
1288
+ if scale <= 1e-3:
1289
+ return None
1290
+ return (rx, ry), float(scale)
1291
+
1292
+ last_full_idx = None
1293
+ last_full = None
1294
+ last_full_rs = None
1295
+
1296
+ for t in range(len(out)):
1297
+ arr = out[t]
1298
+ if not isinstance(arr, list) or len(arr) != J * 3:
1299
+ continue
1300
+
1301
+ rs = root_scale_from_anchors(arr)
1302
+
1303
+ r_ok = is_vis(arr, R_HIP)
1304
+ l_ok = is_vis(arr, L_HIP)
1305
+
1306
+ anchors_ok = count_visible(arr, anchor_joints) >= min_anchors
1307
+
1308
+ if anchors_ok and rs is not None and r_ok and l_ok:
1309
+ last_full_idx = t
1310
+ last_full = list(arr)
1311
+ last_full_rs = rs
1312
+ continue
1313
+
1314
+ if last_full is None or last_full_rs is None or last_full_idx is None:
1315
+ continue
1316
+ if (t - last_full_idx) > max_lookback:
1317
+ continue
1318
+ if not (r_ok or l_ok):
1319
+ continue
1320
+ if r_ok and l_ok:
1321
+ continue
1322
+ if not always_fill_if_one_hip:
1323
+ continue
1324
+ if rs is None:
1325
+ continue
1326
+
1327
+ dst_root, dst_scale = rs
1328
+ src_root, src_scale = last_full_rs
1329
+
1330
+ carried = _apply_root_scale(
1331
+ last_full,
1332
+ src_root=src_root,
1333
+ src_scale=src_scale,
1334
+ dst_root=dst_root,
1335
+ dst_scale=dst_scale,
1336
+ )
1337
+ if not (isinstance(carried, list) and len(carried) == J * 3):
1338
+ continue
1339
+
1340
+ def copy_joint(j: int):
1341
+ if j >= J:
1342
+ return
1343
+ if is_vis(arr, j):
1344
+ return
1345
+ cx = float(carried[3 * j + 0])
1346
+ cy = float(carried[3 * j + 1])
1347
+ cc = float(carried[3 * j + 2])
1348
+ if (cx == 0 and cy == 0) or cc <= 0:
1349
+ return
1350
+ arr[3 * j + 0] = cx
1351
+ arr[3 * j + 1] = cy
1352
+ arr[3 * j + 2] = max(min(cc, 0.60), conf_gate)
1353
+
1354
+ if not r_ok:
1355
+ copy_joint(R_HIP)
1356
+ if fill_legs_with_hip:
1357
+ copy_joint(R_KNEE)
1358
+ copy_joint(R_ANK)
1359
+
1360
+ if not l_ok:
1361
+ copy_joint(L_HIP)
1362
+ if fill_legs_with_hip:
1363
+ copy_joint(L_KNEE)
1364
+ copy_joint(L_ANK)
1365
+
1366
+ out[t] = arr
1367
+
1368
+ return out
1369
+
1370
+
1371
+ def _median3_pose_seq(pose_seq: List[Optional[List[float]]], *, conf_gate: float) -> List[Optional[List[float]]]:
1372
+ if not pose_seq:
1373
+ return pose_seq
1374
+
1375
+ J = None
1376
+ for arr in pose_seq:
1377
+ if isinstance(arr, list) and len(arr) % 3 == 0 and len(arr) > 0:
1378
+ J = len(arr) // 3
1379
+ break
1380
+ if J is None:
1381
+ return pose_seq
1382
+
1383
+ T = len(pose_seq)
1384
+
1385
+ def is_vis(arr: List[float], j: int) -> bool:
1386
+ x = float(arr[3 * j + 0])
1387
+ y = float(arr[3 * j + 1])
1388
+ c = float(arr[3 * j + 2])
1389
+ return (c >= conf_gate) and not (x == 0 and y == 0)
1390
+
1391
+ out_seq: List[Optional[List[float]]] = []
1392
+ for t in range(T):
1393
+ arr = pose_seq[t]
1394
+ if not isinstance(arr, list) or len(arr) != J * 3:
1395
+ out_seq.append(arr)
1396
+ continue
1397
+
1398
+ out = list(arr)
1399
+ t0 = max(0, t - 1)
1400
+ t1 = t
1401
+ t2 = min(T - 1, t + 1)
1402
+
1403
+ a0 = pose_seq[t0]
1404
+ a1 = pose_seq[t1]
1405
+ a2 = pose_seq[t2]
1406
+
1407
+ for j in range(J):
1408
+ if not is_vis(arr, j):
1409
+ continue
1410
+
1411
+ xs, ys = [], []
1412
+ for aa in (a0, a1, a2):
1413
+ if isinstance(aa, list) and len(aa) == J * 3 and is_vis(aa, j):
1414
+ xs.append(float(aa[3 * j + 0]))
1415
+ ys.append(float(aa[3 * j + 1]))
1416
+
1417
+ if len(xs) >= 2:
1418
+ xs.sort()
1419
+ ys.sort()
1420
+ out[3 * j + 0] = float(xs[len(xs) // 2])
1421
+ out[3 * j + 1] = float(ys[len(ys) // 2])
1422
+
1423
+ out_seq.append(out)
1424
+
1425
+ return out_seq
1426
+
1427
+
1428
+ def _sync_group_appearances(
1429
+ pose_arr_seq: List[Optional[List[float]]],
1430
+ *,
1431
+ group: set[int],
1432
+ conf_gate: float,
1433
+ lookahead: int,
1434
+ ) -> List[Optional[List[float]]]:
1435
+ if not pose_arr_seq:
1436
+ return pose_arr_seq
1437
+
1438
+ J = None
1439
+ for arr in pose_arr_seq:
1440
+ if isinstance(arr, list) and len(arr) % 3 == 0 and len(arr) > 0:
1441
+ J = len(arr) // 3
1442
+ break
1443
+ if J is None:
1444
+ return pose_arr_seq
1445
+
1446
+ T = len(pose_arr_seq)
1447
+ out_seq: List[Optional[List[float]]] = []
1448
+ for arr in pose_arr_seq:
1449
+ if isinstance(arr, list) and len(arr) == J * 3:
1450
+ out_seq.append(list(arr))
1451
+ else:
1452
+ out_seq.append(arr)
1453
+
1454
+ def is_vis(arr: List[float], j: int) -> bool:
1455
+ x = float(arr[3 * j + 0])
1456
+ y = float(arr[3 * j + 1])
1457
+ c = float(arr[3 * j + 2])
1458
+ return (c >= conf_gate) and not (x == 0 and y == 0)
1459
+
1460
+ for t in range(T):
1461
+ arr = out_seq[t]
1462
+ if not isinstance(arr, list):
1463
+ continue
1464
+
1465
+ vis = {j for j in group if j < J and is_vis(arr, j)}
1466
+ if not vis:
1467
+ continue
1468
+
1469
+ missing = {j for j in group if j < J and j not in vis}
1470
+ if not missing:
1471
+ continue
1472
+
1473
+ appear_t: dict[int, int] = {}
1474
+ for j in list(missing):
1475
+ t2 = t + 1
1476
+ while t2 < T and t2 <= t + lookahead:
1477
+ arr2 = out_seq[t2]
1478
+ if isinstance(arr2, list) and is_vis(arr2, j):
1479
+ appear_t[j] = t2
1480
+ break
1481
+ t2 += 1
1482
+
1483
+ if not appear_t:
1484
+ continue
1485
+
1486
+ for j, t2 in appear_t.items():
1487
+ last_t = None
1488
+ for tb in range(t - 1, -1, -1):
1489
+ arrb = out_seq[tb]
1490
+ if isinstance(arrb, list) and is_vis(arrb, j):
1491
+ last_t = tb
1492
+ break
1493
+
1494
+ if last_t is None:
1495
+ b = out_seq[t2]
1496
+ if not isinstance(b, list):
1497
+ continue
1498
+ bx, by, bc = float(b[3 * j + 0]), float(b[3 * j + 1]), float(b[3 * j + 2])
1499
+ for k in range(t, t2):
1500
+ a = out_seq[k]
1501
+ if not isinstance(a, list):
1502
+ continue
1503
+ a[3 * j + 0] = bx
1504
+ a[3 * j + 1] = by
1505
+ a[3 * j + 2] = bc
1506
+ continue
1507
+
1508
+ a0 = out_seq[last_t]
1509
+ b0 = out_seq[t2]
1510
+ if not (isinstance(a0, list) and isinstance(b0, list)):
1511
+ continue
1512
+
1513
+ ax, ay, ac = float(a0[3 * j + 0]), float(a0[3 * j + 1]), float(a0[3 * j + 2])
1514
+ bx, by, bc = float(b0[3 * j + 0]), float(b0[3 * j + 1]), float(b0[3 * j + 2])
1515
+
1516
+ if (ax == 0 and ay == 0) or (bx == 0 and by == 0):
1517
+ continue
1518
+
1519
+ conf_fill = min(ac, bc)
1520
+ total = t2 - last_t
1521
+ if total <= 0:
1522
+ continue
1523
+
1524
+ for tt in range(t, t2):
1525
+ a = out_seq[tt]
1526
+ if not isinstance(a, list):
1527
+ continue
1528
+ r = (tt - last_t) / total
1529
+ x = ax + (bx - ax) * r
1530
+ y = ay + (by - ay) * r
1531
+ a[3 * j + 0] = float(x)
1532
+ a[3 * j + 1] = float(y)
1533
+ a[3 * j + 2] = float(conf_fill)
1534
+
1535
+ return out_seq
1536
+
1537
+
1538
+ def _count_valid_points(arr: Optional[List[float]], *, conf_gate: float) -> int:
1539
+ if not isinstance(arr, list) or len(arr) % 3 != 0:
1540
+ return 0
1541
+ cnt = 0
1542
+ for i in range(0, len(arr), 3):
1543
+ x, y, c = float(arr[i]), float(arr[i + 1]), float(arr[i + 2])
1544
+ if c >= conf_gate and not (x == 0 and y == 0):
1545
+ cnt += 1
1546
+ return cnt
1547
+
1548
+
1549
+ def _zero_out_kps(arr: Optional[List[float]]) -> Optional[List[float]]:
1550
+ if not isinstance(arr, list) or len(arr) % 3 != 0:
1551
+ return arr
1552
+ out = list(arr)
1553
+ for i in range(0, len(out), 3):
1554
+ out[i + 0] = 0.0
1555
+ out[i + 1] = 0.0
1556
+ out[i + 2] = 0.0
1557
+ return out
1558
+
1559
+
1560
+ def _pin_body_wrist_to_hand(
1561
+ p_out: Dict[str, Any],
1562
+ *,
1563
+ side: str,
1564
+ conf_gate_body: float = 0.2,
1565
+ conf_gate_hand: float = 0.2,
1566
+ blend: float = 1.0,
1567
+ ) -> None:
1568
+ if side == "right":
1569
+ bw = 4
1570
+ hk = "hand_right_keypoints_2d"
1571
+ else:
1572
+ bw = 7
1573
+ hk = "hand_left_keypoints_2d"
1574
+
1575
+ pose = p_out.get("pose_keypoints_2d")
1576
+ hand = p_out.get(hk)
1577
+
1578
+ if not (isinstance(pose, list) and isinstance(hand, list)):
1579
+ return
1580
+ if len(pose) < (bw * 3 + 3):
1581
+ return
1582
+ if len(hand) < 3:
1583
+ return
1584
+
1585
+ hx, hy, hc = float(hand[0]), float(hand[1]), float(hand[2])
1586
+ if hc < conf_gate_hand or (hx == 0.0 and hy == 0.0):
1587
+ return
1588
+
1589
+ bx, by, bc = float(pose[bw * 3 + 0]), float(pose[bw * 3 + 1]), float(pose[bw * 3 + 2])
1590
+
1591
+ if bc < conf_gate_body or (bx == 0.0 and by == 0.0):
1592
+ pose[bw * 3 + 0] = hx
1593
+ pose[bw * 3 + 1] = hy
1594
+ pose[bw * 3 + 2] = float(max(bc, min(hc, 0.9)))
1595
+ else:
1596
+ nx = bx * (1.0 - blend) + hx * blend
1597
+ ny = by * (1.0 - blend) + hy * blend
1598
+ pose[bw * 3 + 0] = nx
1599
+ pose[bw * 3 + 1] = ny
1600
+ pose[bw * 3 + 2] = float(min(bc, hc))
1601
+
1602
+ p_out["pose_keypoints_2d"] = pose
1603
+
1604
+
1605
+ def _fix_elbow_using_wrist(p_out: Dict[str, Any], *, side: str, conf_gate: float = 0.2) -> None:
1606
+ pose = p_out.get("pose_keypoints_2d")
1607
+ if not isinstance(pose, list) or len(pose) % 3 != 0:
1608
+ return
1609
+
1610
+ if side == "right":
1611
+ sh, el, wr = 2, 3, 4
1612
+ else:
1613
+ sh, el, wr = 5, 6, 7
1614
+
1615
+ def get(j):
1616
+ return float(pose[3 * j + 0]), float(pose[3 * j + 1]), float(pose[3 * j + 2])
1617
+
1618
+ def vis(x, y, c):
1619
+ return c >= conf_gate and not (x == 0.0 and y == 0.0)
1620
+
1621
+ sx, sy, sc = get(sh)
1622
+ ex, ey, ec = get(el)
1623
+ wx, wy, wc = get(wr)
1624
+
1625
+ if not (vis(sx, sy, sc) and vis(wx, wy, wc)):
1626
+ return
1627
+
1628
+ if vis(ex, ey, ec):
1629
+ Lse = math.hypot(ex - sx, ey - sy)
1630
+ Lew = math.hypot(wx - ex, wy - ey)
1631
+ else:
1632
+ dsw = math.hypot(wx - sx, wy - sy)
1633
+ if dsw < 1e-3:
1634
+ return
1635
+ Lse = 0.55 * dsw
1636
+ Lew = 0.45 * dsw
1637
+
1638
+ dx = wx - sx
1639
+ dy = wy - sy
1640
+ d = math.hypot(dx, dy)
1641
+ if d < 1e-6:
1642
+ return
1643
+
1644
+ d2 = max(min(d, (Lse + Lew) - 1e-3), abs(Lse - Lew) + 1e-3)
1645
+
1646
+ a = (Lse * Lse - Lew * Lew + d2 * d2) / (2.0 * d2)
1647
+ h2 = max(Lse * Lse - a * a, 0.0)
1648
+ h = math.sqrt(h2)
1649
+
1650
+ ux = dx / d
1651
+ uy = dy / d
1652
+ px = sx + a * ux
1653
+ py = sy + a * uy
1654
+
1655
+ rx = -uy
1656
+ ry = ux
1657
+
1658
+ e1x, e1y = px + h * rx, py + h * ry
1659
+ e2x, e2y = px - h * rx, py - h * ry
1660
+
1661
+ if vis(ex, ey, ec):
1662
+ if math.hypot(e1x - ex, e1y - ey) <= math.hypot(e2x - ex, e2y - ey):
1663
+ nx, ny = e1x, e1y
1664
+ else:
1665
+ nx, ny = e2x, e2y
1666
+ else:
1667
+ nx, ny = e1x, e1y
1668
+
1669
+ pose[3 * el + 0] = float(nx)
1670
+ pose[3 * el + 1] = float(ny)
1671
+ pose[3 * el + 2] = float(max(min(ec, 0.8), conf_gate))
1672
+
1673
+ p_out["pose_keypoints_2d"] = pose
1674
+
1675
+
1676
+ def _remove_short_presence_runs_kps_seq(
1677
+ seq: List[Optional[List[float]]],
1678
+ *,
1679
+ conf_gate: float,
1680
+ min_points_present: int,
1681
+ min_run: int,
1682
+ ) -> List[Optional[List[float]]]:
1683
+ if not seq:
1684
+ return seq
1685
+
1686
+ present = [(_count_valid_points(a, conf_gate=conf_gate) >= min_points_present) for a in seq]
1687
+ out = [None if a is None else list(a) for a in seq]
1688
+
1689
+ start = None
1690
+ for t in range(len(seq) + 1):
1691
+ cur = present[t] if t < len(seq) else False
1692
+ if cur and start is None:
1693
+ start = t
1694
+ if (not cur) and start is not None:
1695
+ run_len = t - start
1696
+ if run_len < min_run:
1697
+ for k in range(start, t):
1698
+ out[k] = _zero_out_kps(out[k])
1699
+ start = None
1700
+
1701
+ return out
1702
+
1703
+
1704
+ def _zero_sparse_frames_kps_seq(
1705
+ seq: List[Optional[List[float]]], *, conf_gate: float, min_points_present: int
1706
+ ) -> List[Optional[List[float]]]:
1707
+ if not seq:
1708
+ return seq
1709
+
1710
+ out: List[Optional[List[float]]] = []
1711
+ for a in seq:
1712
+ if not isinstance(a, list):
1713
+ out.append(a)
1714
+ continue
1715
+ if _count_valid_points(a, conf_gate=conf_gate) < min_points_present:
1716
+ out.append(_zero_out_kps(a))
1717
+ else:
1718
+ out.append(a)
1719
+ return out
1720
+
1721
+
1722
+ def _suppress_spatial_outliers_in_hand_arr(
1723
+ hand_arr: Optional[List[float]], *, conf_gate: float, max_bone_factor: float = 3.0
1724
+ ) -> Optional[List[float]]:
1725
+ if not isinstance(hand_arr, list) or len(hand_arr) % 3 != 0:
1726
+ return hand_arr
1727
+ pts = _reshape_keypoints_2d(hand_arr)
1728
+ J = len(pts)
1729
+ if J < 21:
1730
+ return hand_arr
1731
+
1732
+ out = [list(p) for p in pts]
1733
+
1734
+ def vis(j: int) -> bool:
1735
+ x, y, c = out[j]
1736
+ return c >= conf_gate and not (x == 0 and y == 0)
1737
+
1738
+ vv = [(x, y) for (x, y, c) in out if c >= conf_gate and not (x == 0 and y == 0)]
1739
+ if len(vv) < 6:
1740
+ return hand_arr
1741
+ xs = [p[0] for p in vv]
1742
+ ys = [p[1] for p in vv]
1743
+ scale = max(max(xs) - min(xs), max(ys) - min(ys))
1744
+ if scale <= 1e-3:
1745
+ return hand_arr
1746
+ max_bone = max_bone_factor * scale
1747
+
1748
+ for a, b in HAND21_EDGES:
1749
+ if a >= J or b >= J:
1750
+ continue
1751
+ if not vis(a) or not vis(b):
1752
+ continue
1753
+ ax, ay, ac = out[a]
1754
+ bx, by, bc = out[b]
1755
+ d = math.hypot(ax - bx, ay - by)
1756
+ if d > max_bone:
1757
+ if ac <= bc:
1758
+ out[a] = [0.0, 0.0, 0.0]
1759
+ else:
1760
+ out[b] = [0.0, 0.0, 0.0]
1761
+
1762
+ return _flatten_keypoints_2d([(x, y, c) for x, y, c in out])
1763
+
1764
+
1765
+ def _body_head_root_scale_from_pose(
1766
+ pose_arr: Optional[List[float]], *, conf_gate: float
1767
+ ) -> Optional[Tuple[Tuple[float, float], float]]:
1768
+ if not isinstance(pose_arr, list) or len(pose_arr) % 3 != 0:
1769
+ return None
1770
+ kps = _reshape_keypoints_2d(pose_arr)
1771
+
1772
+ def vis(j: int) -> Optional[Tuple[float, float]]:
1773
+ if j >= len(kps):
1774
+ return None
1775
+ x, y, c = kps[j]
1776
+ if c >= conf_gate and not (x == 0 and y == 0):
1777
+ return (float(x), float(y))
1778
+ return None
1779
+
1780
+ pts = []
1781
+ for j in [0, 1, 14, 15, 16, 17]:
1782
+ p = vis(j)
1783
+ if p is not None:
1784
+ pts.append(p)
1785
+
1786
+ if not pts:
1787
+ return None
1788
+
1789
+ rx = sum(p[0] for p in pts) / len(pts)
1790
+ ry = sum(p[1] for p in pts) / len(pts)
1791
+ root = (rx, ry)
1792
+
1793
+ def dist(a: int, b: int) -> Optional[float]:
1794
+ pa, pb = vis(a), vis(b)
1795
+ if pa is None or pb is None:
1796
+ return None
1797
+ d = math.hypot(pa[0] - pb[0], pa[1] - pb[1])
1798
+ return d if d > 1e-3 else None
1799
+
1800
+ cands = [dist(14, 15), dist(16, 17), dist(2, 5)]
1801
+ cands = [c for c in cands if c is not None]
1802
+ if not cands:
1803
+ return None
1804
+
1805
+ scale = float(sum(cands) / len(cands))
1806
+ return root, scale
1807
+
1808
+
1809
+ def _body_wrist_root_scale_from_pose(
1810
+ pose_arr: Optional[List[float]], *, side: str, conf_gate: float
1811
+ ) -> Optional[Tuple[Tuple[float, float], float]]:
1812
+ if not isinstance(pose_arr, list) or len(pose_arr) % 3 != 0:
1813
+ return None
1814
+ kps = _reshape_keypoints_2d(pose_arr)
1815
+
1816
+ if side == "right":
1817
+ w, e = 4, 3
1818
+ else:
1819
+ w, e = 7, 6
1820
+
1821
+ def vis(j: int) -> Optional[Tuple[float, float]]:
1822
+ if j >= len(kps):
1823
+ return None
1824
+ x, y, c = kps[j]
1825
+ if c >= conf_gate and not (x == 0 and y == 0):
1826
+ return (float(x), float(y))
1827
+ return None
1828
+
1829
+ pw = vis(w)
1830
+ if pw is None:
1831
+ return None
1832
+ root = pw
1833
+
1834
+ pe = vis(e)
1835
+ scale = None
1836
+ if pe is not None:
1837
+ d = math.hypot(pw[0] - pe[0], pw[1] - pe[1])
1838
+ if d > 1e-3:
1839
+ scale = d
1840
+
1841
+ if scale is None:
1842
+ p2 = vis(2)
1843
+ p5 = vis(5)
1844
+ if p2 is not None and p5 is not None:
1845
+ d = math.hypot(p2[0] - p5[0], p2[1] - p5[1])
1846
+ if d > 1e-3:
1847
+ scale = d
1848
+
1849
+ if scale is None:
1850
+ return None
1851
+
1852
+ return root, float(scale)
1853
+
1854
+
1855
+ def _smooth_dense_seq_anchored_to_body(
1856
+ dense_seq: List[Optional[List[float]]],
1857
+ body_pose_seq: List[Optional[List[float]]],
1858
+ *,
1859
+ kind: str,
1860
+ conf_gate_dense: float,
1861
+ conf_gate_body: float,
1862
+ median3: bool,
1863
+ zero_lag_alpha: float,
1864
+ ) -> List[Optional[List[float]]]:
1865
+ if not dense_seq:
1866
+ return dense_seq
1867
+
1868
+ Jd = None
1869
+ for a in dense_seq:
1870
+ if isinstance(a, list) and len(a) % 3 == 0 and len(a) > 0:
1871
+ Jd = len(a) // 3
1872
+ break
1873
+ if Jd is None:
1874
+ return dense_seq
1875
+
1876
+ T = len(dense_seq)
1877
+ out = [None if a is None else list(a) for a in dense_seq]
1878
+
1879
+ norm_seq: List[Optional[List[float]]] = [None] * T
1880
+
1881
+ for t in range(T):
1882
+ arr = out[t]
1883
+ body = body_pose_seq[t] if t < len(body_pose_seq) else None
1884
+ if not isinstance(arr, list) or len(arr) != Jd * 3 or not isinstance(body, list):
1885
+ norm_seq[t] = arr
1886
+ continue
1887
+
1888
+ if kind == "face":
1889
+ rs = _body_head_root_scale_from_pose(body, conf_gate=conf_gate_body)
1890
+ elif kind == "hand_left":
1891
+ rs = _body_wrist_root_scale_from_pose(body, side="left", conf_gate=conf_gate_body)
1892
+ else:
1893
+ rs = _body_wrist_root_scale_from_pose(body, side="right", conf_gate=conf_gate_body)
1894
+
1895
+ if rs is None:
1896
+ norm_seq[t] = arr
1897
+ continue
1898
+
1899
+ (rx, ry), s = rs
1900
+ if s <= 1e-6:
1901
+ norm_seq[t] = arr
1902
+ continue
1903
+
1904
+ nn = list(arr)
1905
+ for j in range(Jd):
1906
+ x = float(arr[3 * j + 0])
1907
+ y = float(arr[3 * j + 1])
1908
+ c = float(arr[3 * j + 2])
1909
+ if c >= conf_gate_dense and not (x == 0 and y == 0):
1910
+ nn[3 * j + 0] = (x - rx) / s
1911
+ nn[3 * j + 1] = (y - ry) / s
1912
+ norm_seq[t] = nn
1913
+
1914
+ if median3:
1915
+ norm_seq = _median3_pose_seq(norm_seq, conf_gate=conf_gate_dense)
1916
+
1917
+ norm_seq = _zero_lag_ema_pose_seq(norm_seq, alpha=zero_lag_alpha, conf_gate=conf_gate_dense)
1918
+
1919
+ for t in range(T):
1920
+ arrn = norm_seq[t]
1921
+ body = body_pose_seq[t] if t < len(body_pose_seq) else None
1922
+ if not isinstance(arrn, list) or len(arrn) != Jd * 3 or not isinstance(body, list):
1923
+ continue
1924
+
1925
+ if kind == "face":
1926
+ rs = _body_head_root_scale_from_pose(body, conf_gate=conf_gate_body)
1927
+ elif kind == "hand_left":
1928
+ rs = _body_wrist_root_scale_from_pose(body, side="left", conf_gate=conf_gate_body)
1929
+ else:
1930
+ rs = _body_wrist_root_scale_from_pose(body, side="right", conf_gate=conf_gate_body)
1931
+
1932
+ if rs is None:
1933
+ continue
1934
+
1935
+ (rx, ry), s = rs
1936
+ if s <= 1e-6:
1937
+ continue
1938
+
1939
+ orig = out[t]
1940
+ for j in range(Jd):
1941
+ x = float(arrn[3 * j + 0])
1942
+ y = float(arrn[3 * j + 1])
1943
+ c = float(arrn[3 * j + 2])
1944
+
1945
+ ox = float(orig[3 * j + 0])
1946
+ oy = float(orig[3 * j + 1])
1947
+ oc = float(orig[3 * j + 2])
1948
+
1949
+ if oc >= conf_gate_dense and not (ox == 0 and oy == 0) and c >= conf_gate_dense:
1950
+ orig[3 * j + 0] = rx + x * s
1951
+ orig[3 * j + 1] = ry + y * s
1952
+
1953
+ out[t] = orig
1954
+
1955
+ return out
1956
+
1957
+
1958
+ def smooth_KPS_json_obj(
1959
+ data: Any,
1960
+ *,
1961
+ keep_face_untouched: bool = True,
1962
+ keep_hands_untouched: bool = True,
1963
+ filter_extra_people: Optional[bool] = None,
1964
+ ) -> Any:
1965
+ if not isinstance(data, list):
1966
+ raise ValueError("Expected top-level JSON to be a list of frames.")
1967
+
1968
+ if filter_extra_people is None:
1969
+ filter_extra_people = bool(FILTER_EXTRA_PEOPLE)
1970
+
1971
+ chosen_people: List[Optional[Dict[str, Any]]] = [None] * len(data)
1972
+
1973
+ if MAIN_PERSON_MODE == "longest_track":
1974
+ tracks = _build_tracks_over_video(data)
1975
+ main_tr = _pick_main_track(tracks)
1976
+
1977
+ if main_tr is not None:
1978
+ for t in range(len(data)):
1979
+ if t in main_tr.frames:
1980
+ chosen_people[t] = main_tr.frames[t]
1981
+ else:
1982
+ prev_center: Optional[Tuple[float, float]] = None
1983
+ for i, frame in enumerate(data):
1984
+ if not isinstance(frame, dict):
1985
+ continue
1986
+ people = frame.get("people", [])
1987
+ if not isinstance(people, list) or len(people) == 0:
1988
+ continue
1989
+ chosen = _choose_single_person(people, prev_center)
1990
+ chosen_people[i] = chosen
1991
+ if chosen is not None:
1992
+ c = _body_center_from_pose(chosen.get("pose_keypoints_2d"))
1993
+ if c is not None:
1994
+ prev_center = c
1995
+ else:
1996
+ prev_center: Optional[Tuple[float, float]] = None
1997
+ for i, frame in enumerate(data):
1998
+ if not isinstance(frame, dict):
1999
+ continue
2000
+ people = frame.get("people", [])
2001
+ if not isinstance(people, list) or len(people) == 0:
2002
+ continue
2003
+ chosen = _choose_single_person(people, prev_center)
2004
+ chosen_people[i] = chosen
2005
+ if chosen is not None:
2006
+ c = _body_center_from_pose(chosen.get("pose_keypoints_2d"))
2007
+ if c is not None:
2008
+ prev_center = c
2009
+
2010
+ pose_seq: List[Optional[List[float]]] = []
2011
+ for p in chosen_people:
2012
+ pose_seq.append(p.get("pose_keypoints_2d") if isinstance(p, dict) else None)
2013
+
2014
+ if SPATIAL_OUTLIER_FIX:
2015
+ pose_seq = [
2016
+ _suppress_spatial_outliers_in_pose_arr(arr, conf_gate=CONF_GATE_BODY) if arr is not None else None
2017
+ for arr in pose_seq
2018
+ ]
2019
+
2020
+ if GAP_FILL_ENABLED:
2021
+ pose_seq = _denoise_and_fill_gaps_pose_seq(
2022
+ pose_seq,
2023
+ conf_gate=CONF_GATE_BODY,
2024
+ min_run=MIN_RUN_FRAMES,
2025
+ max_gap=MAX_GAP_FRAMES,
2026
+ )
2027
+
2028
+ if TORSO_SYNC_ENABLED:
2029
+ pose_seq = _sync_group_appearances(
2030
+ pose_seq,
2031
+ group=TORSO_JOINTS,
2032
+ conf_gate=CONF_GATE_BODY,
2033
+ lookahead=TORSO_LOOKAHEAD_FRAMES,
2034
+ )
2035
+
2036
+ pose_seq = [
2037
+ (
2038
+ _suppress_isolated_joints_in_pose_arr(arr, conf_gate=CONF_GATE_BODY, keep=TORSO_JOINTS)
2039
+ if arr is not None
2040
+ else None
2041
+ )
2042
+ for arr in pose_seq
2043
+ ]
2044
+
2045
+ if MEDIAN3_ENABLED:
2046
+ pose_seq = _median3_pose_seq(pose_seq, conf_gate=CONF_GATE_BODY)
2047
+
2048
+ if SUPER_SMOOTH_ENABLED:
2049
+ pose_seq = _zero_lag_ema_pose_seq(pose_seq, alpha=SUPER_SMOOTH_ALPHA, conf_gate=SUPER_SMOOTH_MIN_CONF)
2050
+
2051
+ if ROOTSCALE_CARRY_ENABLED:
2052
+ pose_seq = _carry_pose_when_torso_missing(
2053
+ pose_seq,
2054
+ conf_gate=CARRY_CONF_GATE,
2055
+ max_carry=CARRY_MAX_FRAMES,
2056
+ anchor_joints=CARRY_ANCHOR_JOINTS,
2057
+ min_anchors=CARRY_MIN_ANCHORS,
2058
+ )
2059
+
2060
+ pose_seq = _force_full_torso_pair(
2061
+ pose_seq,
2062
+ conf_gate=CARRY_CONF_GATE,
2063
+ anchor_joints=CARRY_ANCHOR_JOINTS,
2064
+ min_anchors=CARRY_MIN_ANCHORS,
2065
+ max_lookback=240,
2066
+ fill_legs_with_hip=True,
2067
+ always_fill_if_one_hip=True,
2068
+ )
2069
+
2070
+ face_seq: List[Optional[List[float]]] = []
2071
+ lh_seq: List[Optional[List[float]]] = []
2072
+ rh_seq: List[Optional[List[float]]] = []
2073
+
2074
+ for p in chosen_people:
2075
+ if isinstance(p, dict):
2076
+ face_seq.append(p.get("face_keypoints_2d"))
2077
+ lh_seq.append(p.get("hand_left_keypoints_2d"))
2078
+ rh_seq.append(p.get("hand_right_keypoints_2d"))
2079
+ else:
2080
+ face_seq.append(None)
2081
+ lh_seq.append(None)
2082
+ rh_seq.append(None)
2083
+
2084
+ if HANDS_SMOOTH_ENABLED and (not keep_hands_untouched):
2085
+ lh_seq = [
2086
+ _suppress_spatial_outliers_in_hand_arr(a, conf_gate=CONF_GATE_HAND) if a is not None else None
2087
+ for a in lh_seq
2088
+ ]
2089
+ rh_seq = [
2090
+ _suppress_spatial_outliers_in_hand_arr(a, conf_gate=CONF_GATE_HAND) if a is not None else None
2091
+ for a in rh_seq
2092
+ ]
2093
+
2094
+ lh_seq = _remove_short_presence_runs_kps_seq(
2095
+ lh_seq, conf_gate=CONF_GATE_HAND, min_points_present=HAND_MIN_POINTS_PRESENT, min_run=MIN_HAND_RUN_FRAMES
2096
+ )
2097
+ rh_seq = _remove_short_presence_runs_kps_seq(
2098
+ rh_seq, conf_gate=CONF_GATE_HAND, min_points_present=HAND_MIN_POINTS_PRESENT, min_run=MIN_HAND_RUN_FRAMES
2099
+ )
2100
+
2101
+ lh_seq = _zero_sparse_frames_kps_seq(
2102
+ lh_seq, conf_gate=CONF_GATE_HAND, min_points_present=HAND_MIN_POINTS_PRESENT
2103
+ )
2104
+ rh_seq = _zero_sparse_frames_kps_seq(
2105
+ rh_seq, conf_gate=CONF_GATE_HAND, min_points_present=HAND_MIN_POINTS_PRESENT
2106
+ )
2107
+
2108
+ if DENSE_GAP_FILL_ENABLED:
2109
+ lh_seq = _denoise_and_fill_gaps_pose_seq(
2110
+ lh_seq, conf_gate=CONF_GATE_HAND, min_run=DENSE_MIN_RUN_FRAMES, max_gap=DENSE_MAX_GAP_FRAMES
2111
+ )
2112
+ rh_seq = _denoise_and_fill_gaps_pose_seq(
2113
+ rh_seq, conf_gate=CONF_GATE_HAND, min_run=DENSE_MIN_RUN_FRAMES, max_gap=DENSE_MAX_GAP_FRAMES
2114
+ )
2115
+
2116
+ if FACE_SMOOTH_ENABLED and (not keep_face_untouched):
2117
+ if DENSE_GAP_FILL_ENABLED:
2118
+ face_seq = _denoise_and_fill_gaps_pose_seq(
2119
+ face_seq, conf_gate=CONF_GATE_FACE, min_run=DENSE_MIN_RUN_FRAMES, max_gap=DENSE_MAX_GAP_FRAMES
2120
+ )
2121
+
2122
+ if FACE_SMOOTH_ENABLED and (not keep_face_untouched):
2123
+ face_seq = _smooth_dense_seq_anchored_to_body(
2124
+ face_seq,
2125
+ pose_seq,
2126
+ kind="face",
2127
+ conf_gate_dense=CONF_GATE_FACE,
2128
+ conf_gate_body=CONF_GATE_BODY,
2129
+ median3=DENSE_MEDIAN3_ENABLED,
2130
+ zero_lag_alpha=DENSE_SUPER_SMOOTH_ALPHA,
2131
+ )
2132
+
2133
+ if HANDS_SMOOTH_ENABLED and (not keep_hands_untouched):
2134
+ lh_seq = _smooth_dense_seq_anchored_to_body(
2135
+ lh_seq,
2136
+ pose_seq,
2137
+ kind="hand_left",
2138
+ conf_gate_dense=CONF_GATE_HAND,
2139
+ conf_gate_body=CONF_GATE_BODY,
2140
+ median3=DENSE_MEDIAN3_ENABLED,
2141
+ zero_lag_alpha=DENSE_SUPER_SMOOTH_ALPHA,
2142
+ )
2143
+ rh_seq = _smooth_dense_seq_anchored_to_body(
2144
+ rh_seq,
2145
+ pose_seq,
2146
+ kind="hand_right",
2147
+ conf_gate_dense=CONF_GATE_HAND,
2148
+ conf_gate_body=CONF_GATE_BODY,
2149
+ median3=DENSE_MEDIAN3_ENABLED,
2150
+ zero_lag_alpha=DENSE_SUPER_SMOOTH_ALPHA,
2151
+ )
2152
+
2153
+ out_frames = []
2154
+ body_state: Optional[BodyState] = None
2155
+
2156
+ for i, frame in enumerate(data):
2157
+ if not isinstance(frame, dict):
2158
+ out_frames.append(frame)
2159
+ continue
2160
+
2161
+ frame_out = copy.deepcopy(frame)
2162
+ chosen = chosen_people[i]
2163
+
2164
+ if chosen is None:
2165
+ if filter_extra_people:
2166
+ frame_out["people"] = []
2167
+ out_frames.append(frame_out)
2168
+ continue
2169
+
2170
+ p_out = copy.deepcopy(chosen)
2171
+ p_out["pose_keypoints_2d"] = pose_seq[i]
2172
+
2173
+ pose_arr = p_out.get("pose_keypoints_2d")
2174
+ joints = (len(pose_arr) // 3) if isinstance(pose_arr, list) else 0
2175
+ if body_state is None:
2176
+ body_state = BodyState(joints if joints > 0 else 18)
2177
+
2178
+ p_out["pose_keypoints_2d"] = _smooth_body_pose(p_out.get("pose_keypoints_2d"), body_state)
2179
+
2180
+ if FACE_SMOOTH_ENABLED and (not keep_face_untouched):
2181
+ p_out["face_keypoints_2d"] = face_seq[i]
2182
+ else:
2183
+ p_out["face_keypoints_2d"] = chosen.get("face_keypoints_2d", p_out.get("face_keypoints_2d"))
2184
+
2185
+ if HANDS_SMOOTH_ENABLED and (not keep_hands_untouched):
2186
+ p_out["hand_left_keypoints_2d"] = lh_seq[i]
2187
+ p_out["hand_right_keypoints_2d"] = rh_seq[i]
2188
+ else:
2189
+ p_out["hand_left_keypoints_2d"] = chosen.get("hand_left_keypoints_2d", p_out.get("hand_left_keypoints_2d"))
2190
+ p_out["hand_right_keypoints_2d"] = chosen.get(
2191
+ "hand_right_keypoints_2d", p_out.get("hand_right_keypoints_2d")
2192
+ )
2193
+
2194
+ _pin_body_wrist_to_hand(
2195
+ p_out, side="left", conf_gate_body=CONF_GATE_BODY, conf_gate_hand=CONF_GATE_HAND, blend=1.0
2196
+ )
2197
+ _pin_body_wrist_to_hand(
2198
+ p_out, side="right", conf_gate_body=CONF_GATE_BODY, conf_gate_hand=CONF_GATE_HAND, blend=1.0
2199
+ )
2200
+
2201
+ _fix_elbow_using_wrist(p_out, side="left", conf_gate=CONF_GATE_BODY)
2202
+ _fix_elbow_using_wrist(p_out, side="right", conf_gate=CONF_GATE_BODY)
2203
+
2204
+ if filter_extra_people:
2205
+ frame_out["people"] = [p_out]
2206
+ else:
2207
+ orig_people = frame.get("people", [])
2208
+ if not isinstance(orig_people, list):
2209
+ frame_out["people"] = [p_out]
2210
+ else:
2211
+ replaced = False
2212
+ new_people = []
2213
+ for op in orig_people:
2214
+ if (not replaced) and (op is chosen):
2215
+ new_people.append(p_out)
2216
+ replaced = True
2217
+ else:
2218
+ new_people.append(copy.deepcopy(op))
2219
+ if not replaced:
2220
+ new_people = [p_out] + [copy.deepcopy(op) for op in orig_people]
2221
+ frame_out["people"] = new_people
2222
+
2223
+ out_frames.append(frame_out)
2224
+
2225
+ return out_frames
2226
+
2227
+
2228
+ # ============================================================
2229
+ # === END: smooth_KPS_json.py logic
2230
+ # ============================================================
2231
+
2232
+
2233
+ # ============================================================
2234
+ # === START: render_pose_video.py logic (ported to frame render)
2235
+ # ============================================================
2236
+
2237
+ OP_COLORS: List[Tuple[int, int, int]] = [
2238
+ (255, 0, 0),
2239
+ (255, 85, 0),
2240
+ (255, 170, 0),
2241
+ (255, 255, 0),
2242
+ (170, 255, 0),
2243
+ (85, 255, 0),
2244
+ (0, 255, 0),
2245
+ (0, 255, 85),
2246
+ (0, 255, 170),
2247
+ (0, 255, 255),
2248
+ (0, 170, 255),
2249
+ (0, 85, 255),
2250
+ (0, 0, 255),
2251
+ (85, 0, 255),
2252
+ (170, 0, 255),
2253
+ (255, 0, 255),
2254
+ (255, 0, 170),
2255
+ (255, 0, 85),
2256
+ ]
2257
+
2258
+ BODY_EDGES: List[Tuple[int, int]] = [
2259
+ (1, 2),
2260
+ (1, 5),
2261
+ (2, 3),
2262
+ (3, 4),
2263
+ (5, 6),
2264
+ (6, 7),
2265
+ (1, 8),
2266
+ (8, 9),
2267
+ (9, 10),
2268
+ (1, 11),
2269
+ (11, 12),
2270
+ (12, 13),
2271
+ (1, 0),
2272
+ (0, 14),
2273
+ (14, 16),
2274
+ (0, 15),
2275
+ (15, 17),
2276
+ ]
2277
+
2278
+ BODY_EDGE_COLORS = OP_COLORS[: len(BODY_EDGES)]
2279
+ BODY_JOINT_COLORS = OP_COLORS
2280
+
2281
+ HAND_EDGES: List[Tuple[int, int]] = [
2282
+ (0, 1),
2283
+ (1, 2),
2284
+ (2, 3),
2285
+ (3, 4),
2286
+ (0, 5),
2287
+ (5, 6),
2288
+ (6, 7),
2289
+ (7, 8),
2290
+ (0, 9),
2291
+ (9, 10),
2292
+ (10, 11),
2293
+ (11, 12),
2294
+ (0, 13),
2295
+ (13, 14),
2296
+ (14, 15),
2297
+ (15, 16),
2298
+ (0, 17),
2299
+ (17, 18),
2300
+ (18, 19),
2301
+ (19, 20),
2302
+ ]
2303
+
2304
+
2305
+ def _valid_pt(x: float, y: float, c: float, conf_thresh: float) -> bool:
2306
+ return (c is not None) and (c >= conf_thresh) and not (x == 0 and y == 0)
2307
+
2308
+
2309
+ def _hsv_to_bgr(h: float, s: float, v: float) -> Tuple[int, int, int]:
2310
+ H = int(np.clip(h, 0.0, 1.0) * 179.0)
2311
+ S = int(np.clip(s, 0.0, 1.0) * 255.0)
2312
+ V = int(np.clip(v, 0.0, 1.0) * 255.0)
2313
+ hsv = np.uint8([[[H, S, V]]])
2314
+ bgr = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)[0, 0]
2315
+ return int(bgr[0]), int(bgr[1]), int(bgr[2])
2316
+
2317
+
2318
+ def _looks_normalized(points: List[Tuple[float, float, float]], conf_thresh: float) -> bool:
2319
+ valid = [(x, y, c) for (x, y, c) in points if _valid_pt(x, y, c, conf_thresh)]
2320
+ if not valid:
2321
+ return False
2322
+ in01 = sum(1 for (x, y, _) in valid if 0.0 <= x <= 1.0 and 0.0 <= y <= 1.0)
2323
+ return (in01 / float(len(valid))) >= 0.7
2324
+
2325
+
2326
+ def _draw_body(
2327
+ canvas: np.ndarray, pose: List[Tuple[float, float, float]], conf_thresh: float, xinsr_stick_scaling: bool = False
2328
+ ) -> None:
2329
+ CH, CW = canvas.shape[:2]
2330
+ stickwidth = 2
2331
+
2332
+ valid = [(x, y, c) for (x, y, c) in pose if _valid_pt(x, y, c, conf_thresh)]
2333
+ norm = False
2334
+ if valid:
2335
+ in01 = sum(1 for (x, y, _) in valid if 0.0 <= x <= 1.0 and 0.0 <= y <= 1.0)
2336
+ norm = (in01 / float(len(valid))) >= 0.7
2337
+
2338
+ def to_px(x: float, y: float) -> Tuple[float, float]:
2339
+ if norm:
2340
+ return x * CW, y * CH
2341
+ return x, y
2342
+
2343
+ max_side = max(CW, CH)
2344
+ if xinsr_stick_scaling:
2345
+ stick_scale = 1 if max_side < 500 else min(2 + (max_side // 1000), 7)
2346
+ else:
2347
+ stick_scale = 1
2348
+
2349
+ for idx, (a, b) in enumerate(BODY_EDGES):
2350
+ if a >= len(pose) or b >= len(pose):
2351
+ continue
2352
+
2353
+ ax, ay, ac = pose[a]
2354
+ bx, by, bc = pose[b]
2355
+ if not (_valid_pt(ax, ay, ac, conf_thresh) and _valid_pt(bx, by, bc, conf_thresh)):
2356
+ continue
2357
+
2358
+ ax, ay = to_px(ax, ay)
2359
+ bx, by = to_px(bx, by)
2360
+
2361
+ base = BODY_EDGE_COLORS[idx] if idx < len(BODY_EDGE_COLORS) else (255, 255, 255)
2362
+
2363
+ X = np.array([ay, by], dtype=np.float32)
2364
+ Y = np.array([ax, bx], dtype=np.float32)
2365
+
2366
+ mX = float(np.mean(X))
2367
+ mY = float(np.mean(Y))
2368
+ length = float(np.hypot(X[0] - X[1], Y[0] - Y[1]))
2369
+ if length < 1.0:
2370
+ continue
2371
+
2372
+ angle = math.degrees(math.atan2(X[0] - X[1], Y[0] - Y[1]))
2373
+
2374
+ polygon = cv2.ellipse2Poly(
2375
+ (int(mY), int(mX)),
2376
+ (int(length / 2), int(stickwidth * stick_scale)),
2377
+ int(angle),
2378
+ 0,
2379
+ 360,
2380
+ 1,
2381
+ )
2382
+
2383
+ cv2.fillConvexPoly(
2384
+ canvas,
2385
+ polygon,
2386
+ (int(base[0] * 0.6), int(base[1] * 0.6), int(base[2] * 0.6)),
2387
+ )
2388
+
2389
+ for j, (x, y, c) in enumerate(pose):
2390
+ if not _valid_pt(x, y, c, conf_thresh):
2391
+ continue
2392
+ x, y = to_px(x, y)
2393
+ col = BODY_JOINT_COLORS[j] if j < len(BODY_JOINT_COLORS) else (255, 255, 255)
2394
+ cv2.circle(canvas, (int(x), int(y)), 2, col, thickness=-1)
2395
+
2396
+
2397
+ def _draw_hand(canvas: np.ndarray, hand: List[Tuple[float, float, float]], conf_thresh: float) -> None:
2398
+ if not hand or len(hand) < 21:
2399
+ return
2400
+
2401
+ CH, CW = canvas.shape[:2]
2402
+ norm = _looks_normalized(hand, conf_thresh)
2403
+
2404
+ def to_px(x: float, y: float) -> Tuple[float, float]:
2405
+ return (x * CW, y * CH) if norm else (x, y)
2406
+
2407
+ n_edges = len(HAND_EDGES)
2408
+ for i, (a, b) in enumerate(HAND_EDGES):
2409
+ x1, y1, c1 = hand[a]
2410
+ x2, y2, c2 = hand[b]
2411
+ if _valid_pt(x1, y1, c1, conf_thresh) and _valid_pt(x2, y2, c2, conf_thresh):
2412
+ x1, y1 = to_px(x1, y1)
2413
+ x2, y2 = to_px(x2, y2)
2414
+ bgr = _hsv_to_bgr(i / float(n_edges), 1.0, 1.0)
2415
+ cv2.line(canvas, (int(x1), int(y1)), (int(x2), int(y2)), bgr, 1, cv2.LINE_AA)
2416
+
2417
+ for x, y, c in hand:
2418
+ if _valid_pt(x, y, c, conf_thresh):
2419
+ x, y = to_px(x, y)
2420
+ cv2.circle(canvas, (int(x), int(y)), 1, (0, 0, 255), -1, cv2.LINE_AA)
2421
+
2422
+
2423
+ def _draw_face(canvas: np.ndarray, face: List[Tuple[float, float, float]], conf_thresh: float) -> None:
2424
+ if not face:
2425
+ return
2426
+
2427
+ CH, CW = canvas.shape[:2]
2428
+ norm = _looks_normalized(face, conf_thresh)
2429
+
2430
+ def to_px(x: float, y: float) -> Tuple[float, float]:
2431
+ return (x * CW, y * CH) if norm else (x, y)
2432
+
2433
+ for x, y, c in face:
2434
+ if _valid_pt(x, y, c, conf_thresh):
2435
+ x, y = to_px(x, y)
2436
+ cv2.circle(canvas, (int(x), int(y)), 0, (255, 255, 255), -1, cv2.LINE_AA)
2437
+
2438
+
2439
+ def _draw_pose_frame_full(
2440
+ w: int,
2441
+ h: int,
2442
+ person: Dict[str, Any],
2443
+ conf_thresh_body: float = 0.10,
2444
+ conf_thresh_hands: float = 0.10,
2445
+ conf_thresh_face: float = 0.10,
2446
+ ) -> np.ndarray:
2447
+ img = np.zeros((h, w, 3), dtype=np.uint8)
2448
+
2449
+ pose = _reshape_keypoints_2d(person.get("pose_keypoints_2d") or [])
2450
+ face = _reshape_keypoints_2d(person.get("face_keypoints_2d") or [])
2451
+ hand_l = _reshape_keypoints_2d(person.get("hand_left_keypoints_2d") or [])
2452
+ hand_r = _reshape_keypoints_2d(person.get("hand_right_keypoints_2d") or [])
2453
+
2454
+ if pose:
2455
+ _draw_body(img, pose, conf_thresh_body)
2456
+ if hand_l:
2457
+ _draw_hand(img, hand_l, conf_thresh_hands)
2458
+ if hand_r:
2459
+ _draw_hand(img, hand_r, conf_thresh_hands)
2460
+ if face:
2461
+ _draw_face(img, face, conf_thresh_face)
2462
+
2463
+ return img
2464
+
2465
+
2466
+ # ============================================================
2467
+ # === END: render_pose_video.py logic
2468
+ # ============================================================
2469
+
2470
+
2471
+ # ============================================================
2472
+ # ComfyUI mappings
2473
+ # ============================================================
2474
+
2475
+ NODE_CLASS_MAPPINGS = {
2476
+ "TSPoseDataSmoother": KPSSmoothPoseDataAndRender,
2477
+ }
2478
+
2479
+ NODE_DISPLAY_NAME_MAPPINGS = {
2480
+ "TSPoseDataSmoother": "KPS: Smooth + Render (pose_data/PKL)",
2481
+ }
__init__.py ADDED
Binary file (22.3 kB). View file
 
load_video_batch.cpython-313.pyc ADDED
Binary file (808 Bytes). View file
 
load_video_batch.py ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from .save_load_pose import TSSavePoseDataAsPickle, TSLoadPoseDataPickle
2
+ from .openpose_smoother import KPSSmoothPoseDataAndRender
3
+ from .load_video_batch import LoadVideoBatchListFromDir
4
+ from .rename_files import RenameFilesInDir
5
+
6
+ NODE_CLASS_MAPPINGS = {
7
+ "TSSavePoseDataAsPickle": TSSavePoseDataAsPickle,
8
+ "TSLoadPoseDataPickle": TSLoadPoseDataPickle,
9
+ "TSPoseDataSmoother": KPSSmoothPoseDataAndRender,
10
+ "TSLoadVideoBatchListFromDir": LoadVideoBatchListFromDir,
11
+ "TSRenameFilesInDir": RenameFilesInDir,
12
+ }
13
+
14
+ NODE_DISPLAY_NAME_MAPPINGS = {
15
+ "TSSavePoseDataAsPickle": "TS Save Pose Data (PKL)",
16
+ "TSLoadPoseDataPickle": "TS Load Pose Data (PKL)",
17
+ "TSPoseDataSmoother": "TS Pose Data Smoother",
18
+ "TSLoadVideoBatchListFromDir": "TS Load Video Batch List From Dir",
19
+ "TSRenameFilesInDir": "TS Rename Files In Dir",
20
+ }
openpose_smoother.cpython-313.pyc ADDED
Binary file (15.8 kB). View file
 
openpose_smoother.py ADDED
@@ -0,0 +1,351 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import re
3
+ import shutil
4
+ import subprocess
5
+ import time
6
+ from collections.abc import Mapping
7
+
8
+ import torch
9
+ import numpy as np
10
+
11
+ # OpenCV for video decoding
12
+ try:
13
+ import cv2
14
+
15
+ _has_cv2 = True
16
+ except Exception:
17
+ _has_cv2 = False
18
+
19
+
20
+ # =========================
21
+ # AUDIO (встроено из utils)
22
+ # =========================
23
+ ENCODE_ARGS = ("utf-8", "backslashreplace")
24
+
25
+
26
+ def _pick_ffmpeg_path():
27
+ # 1) env override (как в VHS)
28
+ if "VHS_FORCE_FFMPEG_PATH" in os.environ:
29
+ p = os.environ.get("VHS_FORCE_FFMPEG_PATH")
30
+ if p:
31
+ return p
32
+
33
+ # 2) system ffmpeg
34
+ system_ffmpeg = shutil.which("ffmpeg")
35
+ if system_ffmpeg is not None:
36
+ return system_ffmpeg
37
+
38
+ # 3) local рядом
39
+ if os.path.isfile("ffmpeg"):
40
+ return os.path.abspath("ffmpeg")
41
+ if os.path.isfile("ffmpeg.exe"):
42
+ return os.path.abspath("ffmpeg.exe")
43
+
44
+ return None
45
+
46
+
47
+ ffmpeg_path = _pick_ffmpeg_path()
48
+
49
+
50
+ def get_audio(file, start_time=0, duration=0):
51
+ if ffmpeg_path is None:
52
+ raise Exception("ffmpeg not found. Put ffmpeg in PATH, or set VHS_FORCE_FFMPEG_PATH env var.")
53
+
54
+ args = [ffmpeg_path, "-i", file]
55
+ if start_time > 0:
56
+ args += ["-ss", str(start_time)]
57
+ if duration > 0:
58
+ args += ["-t", str(duration)]
59
+
60
+ try:
61
+ # как в utils: вытаскиваем raw f32le в stdout
62
+ res = subprocess.run(args + ["-f", "f32le", "-"], capture_output=True, check=True)
63
+ audio = torch.frombuffer(bytearray(res.stdout), dtype=torch.float32)
64
+ match = re.search(r", (\d+) Hz, (\w+), ", res.stderr.decode(*ENCODE_ARGS))
65
+ except subprocess.CalledProcessError as e:
66
+ raise Exception(f"Failed to extract audio from {file}:\n" + e.stderr.decode(*ENCODE_ARGS))
67
+
68
+ if match:
69
+ ar = int(match.group(1))
70
+ ac = {"mono": 1, "stereo": 2}.get(match.group(2), 2)
71
+ else:
72
+ ar = 44100
73
+ ac = 2
74
+
75
+ # reshape как в utils: (-1, channels) -> (channels, samples) -> (1, channels, samples)
76
+ if audio.numel() == 0:
77
+ # видео без аудио — вернем пустой аудиобуфер, но корректный формат
78
+ empty = torch.zeros((1, 1, 0), dtype=torch.float32)
79
+ return {"waveform": empty, "sample_rate": ar}
80
+
81
+ audio = audio.reshape((-1, ac)).transpose(0, 1).unsqueeze(0)
82
+ return {"waveform": audio, "sample_rate": ar}
83
+
84
+
85
+ class LazyAudioMap(Mapping):
86
+ def __init__(self, file, start_time, duration):
87
+ self.file = file
88
+ self.start_time = start_time
89
+ self.duration = duration
90
+ self._dict = None
91
+
92
+ def _ensure(self):
93
+ if self._dict is None:
94
+ self._dict = get_audio(self.file, self.start_time, self.duration)
95
+
96
+ def __getitem__(self, key):
97
+ self._ensure()
98
+ return self._dict[key]
99
+
100
+ def __iter__(self):
101
+ self._ensure()
102
+ return iter(self._dict)
103
+
104
+ def __len__(self):
105
+ self._ensure()
106
+ return len(self._dict)
107
+
108
+
109
+ def lazy_get_audio(file, start_time=0, duration=0, **kwargs):
110
+ return LazyAudioMap(file, start_time, duration)
111
+
112
+
113
+ # =========================
114
+ # остальной код ноды
115
+ # =========================
116
+
117
+
118
+ def extract_first_number(s):
119
+ match = re.search(r"\d+", s)
120
+ return int(match.group()) if match else float("inf")
121
+
122
+
123
+ sort_methods = [
124
+ "None",
125
+ "Alphabetical (ASC)",
126
+ "Alphabetical (DESC)",
127
+ "Numerical (ASC)",
128
+ "Numerical (DESC)",
129
+ "Datetime (ASC)",
130
+ "Datetime (DESC)",
131
+ ]
132
+
133
+
134
+ def sort_by(items, base_path=".", method=None):
135
+ def fullpath(x):
136
+ return os.path.join(base_path, x)
137
+
138
+ def get_timestamp(path):
139
+ try:
140
+ return os.path.getmtime(path)
141
+ except FileNotFoundError:
142
+ return float("-inf")
143
+
144
+ if method == "Alphabetical (ASC)":
145
+ return sorted(items)
146
+ elif method == "Alphabetical (DESC)":
147
+ return sorted(items, reverse=True)
148
+ elif method == "Numerical (ASC)":
149
+ return sorted(items, key=lambda x: extract_first_number(os.path.splitext(x)[0]))
150
+ elif method == "Numerical (DESC)":
151
+ return sorted(items, key=lambda x: extract_first_number(os.path.splitext(x)[0]), reverse=True)
152
+ elif method == "Datetime (ASC)":
153
+ return sorted(items, key=lambda x: get_timestamp(fullpath(x)))
154
+ elif method == "Datetime (DESC)":
155
+ return sorted(items, key=lambda x: get_timestamp(fullpath(x)), reverse=True)
156
+ else:
157
+ return items
158
+
159
+
160
+ def target_size(width, height, custom_width, custom_height, downscale_ratio=8):
161
+ if downscale_ratio is None:
162
+ downscale_ratio = 8
163
+
164
+ if custom_width == 0 and custom_height == 0:
165
+ new_w, new_h = width, height
166
+ elif custom_height == 0:
167
+ new_h = int(height * (custom_width / width))
168
+ new_w = int(custom_width)
169
+ elif custom_width == 0:
170
+ new_w = int(width * (custom_height / height))
171
+ new_h = int(custom_height)
172
+ else:
173
+ new_w, new_h = int(custom_width), int(custom_height)
174
+
175
+ new_w = int(new_w / downscale_ratio + 0.5) * downscale_ratio
176
+ new_h = int(new_h / downscale_ratio + 0.5) * downscale_ratio
177
+ return new_w, new_h
178
+
179
+
180
+ def _read_frames_vhs_like(
181
+ video_path: str,
182
+ force_rate: float = 0,
183
+ custom_width: int = 0,
184
+ custom_height: int = 0,
185
+ downscale_ratio: int = 8,
186
+ frame_load_cap: int = 0,
187
+ ):
188
+ if not _has_cv2:
189
+ raise RuntimeError("OpenCV (cv2) not available. Install opencv-python.")
190
+
191
+ cap = cv2.VideoCapture(video_path)
192
+ if not cap.isOpened() or not cap.grab():
193
+ raise FileNotFoundError(f"Cannot open video: {video_path}")
194
+
195
+ fps = cap.get(cv2.CAP_PROP_FPS)
196
+ if fps is None or fps <= 0:
197
+ fps = 30.0
198
+
199
+ width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
200
+ height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
201
+
202
+ ok0, frame0 = cap.retrieve()
203
+ if not ok0 or frame0 is None:
204
+ cap.release()
205
+ raise RuntimeError(f"Cannot retrieve first frame from: {video_path}")
206
+
207
+ if width <= 0 or height <= 0:
208
+ height, width = frame0.shape[:2]
209
+
210
+ base_dt = 1.0 / float(fps)
211
+ target_dt = base_dt if force_rate == 0 else (1.0 / float(force_rate))
212
+ loaded_fps = 1.0 / target_dt if target_dt > 0 else float(fps)
213
+
214
+ new_w, new_h = target_size(width, height, custom_width, custom_height, downscale_ratio)
215
+ do_resize = (new_w != width) or (new_h != height)
216
+
217
+ frames = []
218
+ time_offset = target_dt
219
+
220
+ def _process_frame(bgr):
221
+ rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
222
+ if do_resize:
223
+ rgb = cv2.resize(rgb, (new_w, new_h), interpolation=cv2.INTER_LANCZOS4)
224
+ return rgb
225
+
226
+ frames.append(_process_frame(frame0))
227
+ if frame_load_cap > 0 and len(frames) >= frame_load_cap:
228
+ cap.release()
229
+ arr = np.stack(frames, axis=0).astype(np.float32) / 255.0
230
+ t = torch.from_numpy(arr)
231
+ return t, float(fps), float(loaded_fps), float(len(t) * target_dt), 0.0
232
+
233
+ time_offset -= target_dt
234
+
235
+ while cap.isOpened():
236
+ if time_offset < target_dt:
237
+ ok = cap.grab()
238
+ if not ok:
239
+ break
240
+ time_offset += base_dt
241
+ continue
242
+
243
+ ok, frame_bgr = cap.retrieve()
244
+ if not ok or frame_bgr is None:
245
+ break
246
+
247
+ frames.append(_process_frame(frame_bgr))
248
+
249
+ if frame_load_cap > 0 and len(frames) >= frame_load_cap:
250
+ break
251
+
252
+ time_offset -= target_dt
253
+
254
+ cap.release()
255
+
256
+ if len(frames) == 0:
257
+ raise RuntimeError(f"No frames could be read from: {video_path}")
258
+
259
+ arr = np.stack(frames, axis=0).astype(np.float32) / 255.0
260
+ t = torch.from_numpy(arr)
261
+ loaded_duration = float(len(t) * target_dt)
262
+ return t, float(fps), float(loaded_fps), loaded_duration, 0.0
263
+
264
+
265
+ class LoadVideoBatchListFromDir:
266
+ @classmethod
267
+ def INPUT_TYPES(s):
268
+ return {
269
+ "required": {
270
+ "directory": ("STRING", {"default": ""}),
271
+ "force_rate": ("FLOAT", {"default": 0, "min": 0, "max": 120, "step": 1}),
272
+ "width": ("INT", {"default": 720, "min": 0, "max": 8192, "step": 1}),
273
+ "height": ("INT", {"default": 1280, "min": 0, "max": 8192, "step": 1}),
274
+ },
275
+ "optional": {
276
+ "video_load_cap": ("INT", {"default": 0, "min": 0, "step": 1}),
277
+ "frame_load_cap": ("INT", {"default": 0, "min": 0, "step": 1}),
278
+ "start_index": ("INT", {"default": 0, "min": 0, "max": 0xFFFFFFFFFFFFFFFF, "step": 1}),
279
+ "load_always": ("BOOLEAN", {"default": False, "label_on": "enabled", "label_off": "disabled"}),
280
+ "sort_method": (sort_methods,),
281
+ },
282
+ }
283
+
284
+ RETURN_TYPES = ("IMAGE", "AUDIO", "INT")
285
+ RETURN_NAMES = ("IMAGE", "audio", "COUNT")
286
+ OUTPUT_IS_LIST = (True, True, False)
287
+
288
+ FUNCTION = "load_videos"
289
+ CATEGORY = "video"
290
+
291
+ @classmethod
292
+ def IS_CHANGED(cls, **kwargs):
293
+ if kwargs.get("load_always"):
294
+ return float("NaN")
295
+ return hash(frozenset(kwargs.items()))
296
+
297
+ def load_videos(
298
+ self,
299
+ directory: str,
300
+ force_rate: float = 0,
301
+ width: int = 0,
302
+ height: int = 0,
303
+ video_load_cap: int = 0,
304
+ frame_load_cap: int = 0,
305
+ start_index: int = 0,
306
+ load_always: bool = False,
307
+ sort_method=None,
308
+ ):
309
+ if not os.path.isdir(directory):
310
+ raise FileNotFoundError(f"Directory '{directory}' cannot be found.")
311
+
312
+ files = os.listdir(directory)
313
+ if len(files) == 0:
314
+ raise FileNotFoundError(f"No files in directory '{directory}'.")
315
+
316
+ valid_ext = {".mp4", ".mov", ".mkv", ".webm", ".avi", ".m4v"}
317
+ files = [
318
+ f
319
+ for f in files
320
+ if os.path.isfile(os.path.join(directory, f)) and os.path.splitext(f)[1].lower() in valid_ext
321
+ ]
322
+ if len(files) == 0:
323
+ raise FileNotFoundError(f"No video files in directory '{directory}' (expected: {sorted(valid_ext)}).")
324
+
325
+ files = sort_by(files, directory, sort_method)
326
+ files = files[start_index:]
327
+ if video_load_cap > 0:
328
+ files = files[:video_load_cap]
329
+
330
+ images_list = []
331
+ audios_list = []
332
+
333
+ for fname in files:
334
+ path = os.path.join(directory, fname)
335
+
336
+ vid, source_fps, loaded_fps, loaded_duration, start_time = _read_frames_vhs_like(
337
+ path,
338
+ force_rate=force_rate,
339
+ custom_width=width,
340
+ custom_height=height,
341
+ downscale_ratio=8,
342
+ frame_load_cap=frame_load_cap,
343
+ )
344
+
345
+ images_list.append(vid)
346
+
347
+ # duration based on loaded frames/time
348
+ audio = lazy_get_audio(path, start_time, loaded_duration)
349
+ audios_list.append(audio)
350
+
351
+ return (images_list, audios_list, len(images_list))
rename_files.cpython-313.pyc ADDED
Binary file (96.1 kB). View file
 
rename_files.py ADDED
File without changes
requirements.txt ADDED
@@ -0,0 +1,200 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import re
3
+ import uuid
4
+ import shutil
5
+
6
+
7
+ def extract_first_number(s: str):
8
+ match = re.search(r"\d+", s)
9
+ return int(match.group()) if match else float("inf")
10
+
11
+
12
+ sort_methods = [
13
+ "None",
14
+ "Alphabetical (ASC)",
15
+ "Alphabetical (DESC)",
16
+ "Numerical (ASC)",
17
+ "Numerical (DESC)",
18
+ "Datetime (ASC)",
19
+ "Datetime (DESC)",
20
+ ]
21
+
22
+
23
+ def sort_by(items, base_path=".", method=None):
24
+ def fullpath(x):
25
+ return os.path.join(base_path, x)
26
+
27
+ def get_timestamp(path):
28
+ try:
29
+ return os.path.getmtime(path)
30
+ except FileNotFoundError:
31
+ return float("-inf")
32
+
33
+ if method == "Alphabetical (ASC)":
34
+ return sorted(items)
35
+ elif method == "Alphabetical (DESC)":
36
+ return sorted(items, reverse=True)
37
+ elif method == "Numerical (ASC)":
38
+ return sorted(items, key=lambda x: extract_first_number(os.path.splitext(x)[0]))
39
+ elif method == "Numerical (DESC)":
40
+ return sorted(items, key=lambda x: extract_first_number(os.path.splitext(x)[0]), reverse=True)
41
+ elif method == "Datetime (ASC)":
42
+ return sorted(items, key=lambda x: get_timestamp(fullpath(x)))
43
+ elif method == "Datetime (DESC)":
44
+ return sorted(items, key=lambda x: get_timestamp(fullpath(x)), reverse=True)
45
+ else:
46
+ return items
47
+
48
+
49
+ def _safe_list_files(directory: str):
50
+ return [f for f in os.listdir(directory) if os.path.isfile(os.path.join(directory, f))]
51
+
52
+
53
+ def _format_name(index: int, digits: int, prefix: str, ext: str):
54
+ """
55
+ ext ожидается как ".png"/".jpg"/".jpeg" (с точкой).
56
+ ВАЖНО: underscore после номера ВСЕГДА, потом расширение как есть.
57
+ Пример: prefix_0001_.png
58
+ """
59
+ num = str(index).zfill(digits)
60
+ left = f"{prefix}_" if prefix else ""
61
+ return f"{left}{num}_{ext}"
62
+
63
+
64
+ def _index_taken(directory: str, digits: int, prefix: str, index: int) -> bool:
65
+ """
66
+ Проверяем, занят ли номер index ЛЮБЫМ расширением в папке.
67
+ Т.е. если есть prefix_0001_.png, то prefix_0001_.jpg уже нельзя.
68
+ """
69
+ num = str(index).zfill(digits)
70
+ left = f"{prefix}_" if prefix else ""
71
+ start = f"{left}{num}_"
72
+
73
+ try:
74
+ entries = os.listdir(directory)
75
+ except FileNotFoundError:
76
+ return False
77
+
78
+ for f in entries:
79
+ p = os.path.join(directory, f)
80
+ if os.path.isfile(p) and f.startswith(start):
81
+ return True
82
+ return False
83
+
84
+
85
+ def _find_next_free_index(directory: str, digits: int, prefix: str, start_from: int = 1) -> int:
86
+ idx = max(1, int(start_from))
87
+ while _index_taken(directory, digits, prefix, idx):
88
+ idx += 1
89
+ return idx
90
+
91
+
92
+ class RenameFilesInDir:
93
+ OUTPUT_NODE = True
94
+
95
+ @classmethod
96
+ def INPUT_TYPES(cls):
97
+ return {
98
+ "required": {
99
+ "directory": ("STRING", {"default": ""}),
100
+ },
101
+ "optional": {
102
+ "output_directory": ("STRING", {"default": ""}),
103
+ "sort_method": (sort_methods,),
104
+ "start_index": ("INT", {"default": 0, "min": 0, "max": 0xFFFFFFFFFFFFFFFF, "step": 1}),
105
+ "files_load_cap": ("INT", {"default": 0, "min": 0, "step": 1}),
106
+ "prefix": ("STRING", {"default": ""}),
107
+ "digits": ("INT", {"default": 4, "min": 1, "max": 16, "step": 1}),
108
+ },
109
+ }
110
+
111
+ RETURN_TYPES = ("INT",)
112
+ RETURN_NAMES = ("COUNT",)
113
+ FUNCTION = "run"
114
+ CATEGORY = "InspirePack/files"
115
+
116
+ @classmethod
117
+ def IS_CHANGED(cls, **kwargs):
118
+ return float("NaN")
119
+
120
+ def run(
121
+ self,
122
+ directory: str,
123
+ output_directory: str = "",
124
+ sort_method=None,
125
+ start_index: int = 0,
126
+ files_load_cap: int = 0,
127
+ prefix: str = "",
128
+ digits: int = 4,
129
+ ):
130
+ if not os.path.isdir(directory):
131
+ raise FileNotFoundError(f"Directory '{directory}' cannot be found.")
132
+
133
+ files = _safe_list_files(directory)
134
+ if not files:
135
+ return (0,)
136
+
137
+ files = sort_by(files, directory, sort_method)
138
+ files = files[start_index:]
139
+
140
+ if files_load_cap > 0:
141
+ files = files[:files_load_cap]
142
+
143
+ if not files:
144
+ return (0,)
145
+
146
+ inplace = (output_directory is None) or (str(output_directory).strip() == "")
147
+
148
+ if not inplace:
149
+ os.makedirs(output_directory, exist_ok=True)
150
+
151
+ count = 0
152
+
153
+ # ---------- COPY MODE ----------
154
+ if not inplace:
155
+ for fname in files:
156
+ src = os.path.join(directory, fname)
157
+ _, ext = os.path.splitext(fname) # ext = ".png" / ".jpg" / ...
158
+
159
+ next_idx = _find_next_free_index(output_directory, digits, prefix, start_from=1)
160
+ new_name = _format_name(next_idx, digits, prefix, ext)
161
+
162
+ dst = os.path.join(output_directory, new_name)
163
+ shutil.copy2(src, dst)
164
+ count += 1
165
+
166
+ return (count,)
167
+
168
+ # ---------- INPLACE RENAME ----------
169
+ temp_map = []
170
+ used_temp = set()
171
+
172
+ def _make_temp_name(old_name: str):
173
+ while True:
174
+ t = f"__tmp__{uuid.uuid4().hex}__{old_name}"
175
+ if t not in used_temp and not os.path.exists(os.path.join(directory, t)):
176
+ used_temp.add(t)
177
+ return t
178
+
179
+ # phase1 -> temp
180
+ for fname in files:
181
+ old_path = os.path.join(directory, fname)
182
+ tmp = _make_temp_name(fname)
183
+ tmp_path = os.path.join(directory, tmp)
184
+
185
+ os.rename(old_path, tmp_path)
186
+ temp_map.append((tmp, fname))
187
+
188
+ # phase2 -> final
189
+ for tmp, original_name in temp_map:
190
+ tmp_path = os.path.join(directory, tmp)
191
+ _, ext = os.path.splitext(original_name)
192
+
193
+ next_idx = _find_next_free_index(directory, digits, prefix, start_from=1)
194
+ new_name = _format_name(next_idx, digits, prefix, ext)
195
+
196
+ new_path = os.path.join(directory, new_name)
197
+ os.rename(tmp_path, new_path)
198
+ count += 1
199
+
200
+ return (count,)
save_load_pose.cpython-313.pyc ADDED
Binary file (9.47 kB). View file
 
save_load_pose.py ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ numpy
2
+ opencv-python
utils.cpython-313.pyc ADDED
Binary file (6.11 kB). View file