Dee Ferdinand commited on
Commit
0084100
·
1 Parent(s): ef517dd

feat: hyperframes-video skill v3 — watch_gdrive.py based on bradautomates/claude-video

Browse files

Claude can now WATCH Google Drive videos:
- Auto-scaled frame extraction (same budget logic as claude-video)
- Groq whisper-large-v3 -> OpenAI -> HF Inference transcription waterfall
- Indonesian filler word cleaning (eh, umm, itu, ya, kan, gitu)
- 6-marker quote scoring (outcome/surprise/transform/recommend)
- Outputs frame paths for Claude Read tool + timestamped transcript

skills/hyperframes-video/SKILL.md CHANGED
@@ -1,173 +1,187 @@
1
  ---
2
  name: hyperframes-video
3
- description: "Build production-ready HyperFrames HTML video compositions for testimonials, teasers, trailers, and community stories. Use when the user mentions 'hyperframes video,' 'testimonial video,' 'HyperFrames composition,' 'render MP4 from HTML,' 'kinetic caption video,' 'Dee Ferdinand video,' 'corporate training video,' 'event teaser video,' 'video with GSAP,' or wants to build any HTML-to-MP4 video. Also trigger when the user says 'build me a video for [client],' 'make a 30 second testimonial,' 'create a social proof video,' or references any event folder in Google Drive or GitHub. IMPORTANT: Do NOT ask clarifying questions before acting scan available assets, transcribe, analyze, present a brief, then build immediately on confirmation."
4
  metadata:
5
- version: 2.0.0
6
- author: Dee Ferdinand × Claude
 
 
 
 
 
 
7
  ---
8
 
9
- # HyperFrames Video Production Skill v2
10
 
11
- **ACT-FIRST RULE: Never ask questions before scanning. Always scan transcribe analyze brief → confirm → build.**
12
 
13
- This skill produces correct HyperFrames HTML compositions with:
14
- - Auto transcription + intelligent quote extraction
15
- - 3 brand-matched visual styles
16
- - ElevenLabs lo-fi music generation
17
- - Direct GitHub push + HuggingFace render
18
- - Zero black frames, correct audio
19
 
20
  ---
21
 
22
- ## PIPELINE (6 phases — execute in order, no skipping)
23
 
24
- ### PHASE 1 Asset Ingestion
25
 
26
- **Auto-detect from 3 sources (in priority order):**
27
-
28
- **Source A — Google Drive (via Composio)**
29
- ```
30
- Account: tomoxi (tomoxi.marketing@gmail.com)
31
- Tool: GOOGLEDRIVE_FIND_FILE
32
- Path: 2026/[EventFolder]/
33
- Subfolders: Video/ → .MOV files (testimonial clips)
34
- Foto/ → .JPG files (B-roll photos)
35
- Testimoni/ → .MOV files (raw testimonial recordings)
36
- ```
37
- → Download all .MOV + .JPG files, save to `/tmp/[client-slug]/`
38
-
39
- **Source B — GitHub (via Composio)**
40
  ```
41
- Tool: GITHUB_GET_REPOSITORY_CONTENT
42
- Repo: AIGoose/dee-video-studio
43
- Path: projects/[client-slug]/assets/
 
44
  ```
45
 
46
- **Source C Direct upload (user uploads in chat)**
47
- → Files already in context, save to `/tmp/[client-slug]/`
48
-
49
- → Always report: `"Found X videos, Y photos from [source]"` then proceed immediately.
50
-
51
- ---
52
 
53
- ### PHASE 2 Transcription + Cleaning
54
 
55
- For every .MOV in the Testimoni/ folder:
 
 
56
 
57
- **Transcribe via Whisper (HF Inference)** — see `references/transcription.md` for full code
 
 
58
 
59
- **Clean:** Remove fillers ("eh", "umm", "itu", "ya", "kan", "gitu"), repetitions, false starts
60
 
61
- **Analyze for best moments:**
62
- 1. Find the single best 10–15s quote (most emotional/concrete/surprising)
63
- 2. Identify the strongest outcome statement ("sekarang saya bisa...")
64
- 3. Rate overall energy: low/medium/high → maps to style suggestion
65
- 4. Suggest duration: <2min raw → 15s. 2–5min → 30s. 5min+ → 45–60s
 
 
66
 
67
- ---
 
 
68
 
69
- ### PHASE 3 Intelligent Brief (ONE confirmation point)
 
 
 
70
 
71
- Present this brief — **do not ask multiple questions**:
 
 
 
72
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  ```
74
- 📋 VIDEO BRIEF — [Client Name]
75
 
76
- SOURCE: [X videos, Y photos from Drive/GitHub/upload]
77
 
78
- TRANSCRIPT ANALYSIS:
79
- Best quote: "[actual quote from transcript]" (at 0:42)
80
- Strongest moment: [describe]
81
- Energy level: High recommended style: Dark Premium
 
 
 
 
 
82
 
83
- RECOMMENDED BUILD:
84
- • Type: Corporate Testimonial
85
- • Duration: 30s
86
- • Style: Dark Premium (Space Grotesk, kinetic 160px, dark #0d0d1f)
87
- • Music: ElevenLabs lo-fi upbeat 88 BPM
88
- • Quote to use: "[best quote]"
89
 
90
- Say "go" to build this, or adjust any parameter above.
91
- ```
 
 
92
 
93
- **If user says "go" or any affirmative → immediately proceed to Phase 4.**
94
 
95
- ---
96
 
97
- ### PHASE 4 — Style Selection
 
 
 
 
98
 
99
- Read `references/styles.md` for full CSS/GSAP per style.
100
 
101
- | Style | When | Caption | Palette |
102
- |-------|------|---------|--------|
103
- | **Dark Premium** | Corporate, high energy | 160px Space Grotesk | #0d0d1f + purple |
104
- | **Warm Documentary** | Community, GKI, intimate | 80px Plus Jakarta | Warm brown + amber |
105
- | **Bold Editorial** | Teaser, launch, public event | 128px Space Grotesk | White + black |
106
 
107
  ---
108
 
109
- ### PHASE 5 — Build + Push to GitHub
110
-
111
- Read `references/composition.md` — do not build from memory.
112
 
113
- **Commit via Composio:**
114
- ```
115
- Tool: GITHUB_COMMIT_MULTIPLE_FILES
116
- Repo: AIGoose/dee-video-studio
117
- Files: projects/[client-slug]/index.html + meta.json
118
- ```
119
 
120
- ---
 
 
 
 
 
 
121
 
122
- ### PHASE 6 — Render via HuggingFace
123
-
124
- ```python
125
- import requests, time
126
- r = requests.post(
127
- "https://aigoose-hyperframes-video-studio.hf.space/api/render",
128
- files=[("files", ("testimoni.MOV", open(video_path,"rb"), "video/quicktime")),
129
- ("files", ("foto_1.JPG", open(photo_path,"rb"), "image/jpeg"))],
130
- data={"workflow":"testimonial","clientName":CLIENT,"trainerName":"Dee Ferdinand",
131
- "tagline":"AI Corporate Trainer","website":"deeferdinand.com",
132
- "format":"9:16","musicTrack":"auto","duration":"30"}
133
- )
134
- job_id = r.json()["jobId"]
135
- while True:
136
- job = requests.get(f"https://aigoose-hyperframes-video-studio.hf.space/api/jobs/{job_id}").json()
137
- if job["status"] == "done": print(f"\u2705 {job['hf_url']}"); break
138
- if job["status"] == "error": raise Exception(job["error"])
139
- time.sleep(15)
140
- ```
141
 
142
  ---
143
 
144
  ## THE 5 HARD RULES
145
 
146
- 1. `class="clip"` on EVERY timed element + `data-start` + `data-duration` + `data-track-index`
147
- 2. `gsap.timeline({ paused: true })` `window.__timelines["COMP_ID"] = tl`
148
- 3. NEVER `.play()` `.pause()` `.currentTime` framework controls media
149
- 4. NEVER `Math.random()`, `Date.now()`, `fetch()` in GSAP scripts
150
- 5. `tl.set({}, {}, TOTAL_DURATION)` at the very end
151
 
152
- **NO `muted` on video when subject voice is needed.**
153
- **EXIT ANIMATIONS BANNED** (except final scene).
154
 
155
  ---
156
 
157
  ## REFERENCE FILES
158
 
159
- - `references/composition.md` HTML template, GSAP patterns, transitions
160
- - `references/music.md` ElevenLabs API, YuE fallback, prompts
161
- - `references/styles.md` Dark Premium / Warm Documentary / Bold Editorial
162
- - `references/transcription.md` Whisper, cleaning, quote extraction
163
- - `references/workflows.md` Scene structures for all 4 video types
 
164
 
165
  ## CONNECTIONS
166
 
167
- | Tool | When | How |
168
- |------|------|-----|
169
- | Google Drive | Scan assets | Composio `GOOGLEDRIVE_FIND_FILE` account=tomoxi |
170
- | GitHub | Push composition | Composio `GITHUB_COMMIT_MULTIPLE_FILES` AIGoose/dee-video-studio |
171
- | HF Space | Render | POST aigoose-hyperframes-video-studio.hf.space/api/render |
172
- | ElevenLabs | Music | POST /v1/music xi-api-key header |
173
- | Whisper | Transcribe | HF Inference openai/whisper-large-v3 |
 
 
 
 
 
 
 
 
 
1
  ---
2
  name: hyperframes-video
3
+ description: "Build production-ready HyperFrames HTML video compositions for testimonials, teasers, trailers, and community stories. Use when the user mentions 'hyperframes video,' 'testimonial video,' 'HyperFrames composition,' 'render MP4 from HTML,' 'kinetic caption video,' 'Dee Ferdinand video,' 'corporate training video,' 'event teaser video,' or wants to build any HTML-to-MP4 video from Google Drive footage. Also trigger when the user says 'build me a video for [client],' 'make a 30 second testimonial,' or 'create a social proof video.' IMPORTANT: Do NOT ask clarifying questions before acting. Scan Google Drive -> run watch_gdrive.py to WATCH and transcribe each video -> analyze -> present brief -> build on confirmation."
4
  metadata:
5
+ version: 3.0.0
6
+ author: Dee Ferdinand x Claude
7
+ sources:
8
+ - bradautomates/claude-video (MIT)
9
+ - nateherkai/hyperframes-student-kit
10
+ - heygen-com/hyperframes-launch-video
11
+ - coleam00/hyperframes-ai-video-generation
12
+ - elevenlabs/skills
13
  ---
14
 
15
+ # HyperFrames Video Production Skill v3
16
 
17
+ **ACT-FIRST: Scan Drive -> Watch videos -> Transcribe -> Analyze -> Brief -> Build**
18
 
19
+ v3 core upgrade: Claude can now WATCH every Google Drive video using `scripts/watch_gdrive.py`
20
+ Based on bradautomates/claude-video (MIT) -- extracts frames + transcribes + finds best quote.
 
 
 
 
21
 
22
  ---
23
 
24
+ ## PIPELINE
25
 
26
+ ### PHASE 1 -- Scan Google Drive
27
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  ```
29
+ Tool: GOOGLEDRIVE_FIND_FILE (Composio, account: tomoxi)
30
+ Scan: 2026/[EventFolder]/Testimoni/ -> .MOV files
31
+ Also: Video/ and Foto/ subfolders
32
+ Report: "Found X video(s), Y photo(s)" then proceed immediately.
33
  ```
34
 
35
+ ### PHASE 2 -- Download Videos
 
 
 
 
 
36
 
37
+ For each .MOV, download to /tmp/[client-slug]/:
38
 
39
+ ```bash
40
+ # Option A: Composio (private files)
41
+ # Tool: GOOGLEDRIVE_DOWNLOAD_FILE account=tomoxi file_id=[id]
42
 
43
+ # Option B: Direct (shared files)
44
+ python3 -c "import requests; r=requests.get('https://drive.google.com/uc?export=download&id=FILE_ID&confirm=t',stream=True); open('/tmp/video.MOV','wb').write(r.content)"
45
+ ```
46
 
47
+ ### PHASE 3 -- Watch + Transcribe Each Video
48
 
49
+ ```bash
50
+ # Install deps (once)
51
+ pip install gdown --break-system-packages
52
+ # Set at least one key:
53
+ export GROQ_API_KEY=... # preferred (groq.com free tier)
54
+ export OPENAI_API_KEY=... # fallback
55
+ # HF_TOKEN used automatically as final fallback
56
 
57
+ # Run for each video
58
+ python3 ${SKILL_DIR}/scripts/watch_gdrive.py \\
59
+ --local /tmp/purbasari/testimoni_1.MOV --resolution 512
60
 
61
+ # Focus on a specific moment
62
+ python3 ${SKILL_DIR}/scripts/watch_gdrive.py \\
63
+ --local /tmp/purbasari/testimoni_1.MOV --start 0:23 --end 0:45
64
+ ```
65
 
66
+ Script outputs:
67
+ - Frame paths -> READ each with the Read tool to see the video visually
68
+ - Timestamped transcript (cleaned)
69
+ - Best testimonial quote with exact timestamp
70
 
71
+ Sample output:
72
+ ```
73
+ # watch_gdrive: video report
74
+ - Duration: 00:47 (47.2s)
75
+ - Frames: 28 @ 0.593 fps
76
+ - Transcript: 18 segments (cleaned)
77
+
78
+ ## Frames
79
+ - `/tmp/watch-gdrive-xxx/frames/frame_000001.jpg` (t=00:01)
80
+ ...
81
+
82
+ ## Transcript
83
+ [00:01] Pertama kali dengar AI, saya pikir ini susah
84
+ [00:05] Tapi setelah training, ternyata langsung bisa dipakai
85
+ ...
86
+
87
+ ## Best Testimonial Quote
88
+ Type: outcome
89
+ Timestamp: 00:05 -> 00:17 (12s)
90
+ Quote: "Setelah training, ternyata langsung bisa dipakai. Sekarang bisa buat konten sendiri"
91
  ```
 
92
 
93
+ ### PHASE 4 -- Analyze + Present Brief (ONE confirmation)
94
 
95
+ ```
96
+ BRIEF -- [Client Name]
97
+ VIDEOS WATCHED: testimoni_1.MOV (47s, 18 segments)
98
+ BEST QUOTE: "[exact quote]" (0:05-0:17, type: outcome)
99
+ VISUAL: [describe what frames showed at that moment]
100
+ PHOTOS: 28 available for B-roll
101
+ RECOMMENDATION: Testimonial 30s | Dark Premium | ElevenLabs lo-fi 88bpm
102
+ Say "go" to build.
103
+ ```
104
 
105
+ ### PHASE 5 -- Style
 
 
 
 
 
106
 
107
+ Read references/styles.md. Auto-assign:
108
+ - Corporate + high energy -> Dark Premium (Space Grotesk 160px, #0d0d1f)
109
+ - Community/GKI/church -> Warm Documentary (Plus Jakarta 80px, warm brown)
110
+ - Teaser/public event -> Bold Editorial (Space Grotesk 128px, white bg)
111
 
112
+ ### PHASE 6 -- Build Composition
113
 
114
+ Read references/composition.md. Use exact timestamp from watch_gdrive report:
115
 
116
+ ```html
117
+ <video data-start="7.65" data-duration="8.85" data-track-index="10"
118
+ data-volume="0.88" src="./assets/testimoni_1.MOV" playsinline
119
+ style="width:100%;height:100%;object-fit:cover;"></video>
120
+ ```
121
 
122
+ ### PHASE 7 -- Push GitHub + Render HF
123
 
124
+ Push index.html + meta.json via Composio GITHUB_COMMIT_MULTIPLE_FILES.
125
+ POST to https://aigoose-hyperframes-video-studio.hf.space/api/render.
126
+ Poll /api/jobs/[id] until done. Share hf_url.
 
 
127
 
128
  ---
129
 
130
+ ## watch_gdrive.py Architecture
 
 
131
 
132
+ Based on bradautomates/claude-video (MIT). Key adaptations:
 
 
 
 
 
133
 
134
+ | Component | claude-video original | watch_gdrive adaptation |
135
+ |---------------|-------------------------|-------------------------------------|
136
+ | Download | yt-dlp (public URLs) | gdown + direct auth (private Drive) |
137
+ | Frames | auto-scaled fps budget | same logic (<=30s=30f, <=60s=40f) |
138
+ | Transcription | Groq -> OpenAI Whisper | same + HF Inference fallback |
139
+ | Cleanup | rolling-duplicate dedup | + Indonesian filler removal |
140
+ | Analysis | none | + best quote extraction (6 markers) |
141
 
142
+ Transcription waterfall:
143
+ 1. Groq whisper-large-v3 (GROQ_API_KEY) -- fastest, cheapest
144
+ 2. OpenAI whisper-1 (OPENAI_API_KEY) -- reliable fallback
145
+ 3. HF Inference whisper-large-v3 (HF_TOKEN) -- free, already set in Space
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
146
 
147
  ---
148
 
149
  ## THE 5 HARD RULES
150
 
151
+ 1. class="clip" on EVERY timed element + data-start + data-duration + data-track-index
152
+ 2. gsap.timeline({ paused: true }) -> window.__timelines["COMP_ID"] = tl
153
+ 3. NEVER .play() .pause() .currentTime in scripts
154
+ 4. NEVER Math.random(), Date.now(), fetch() in GSAP
155
+ 5. tl.set({}, {}, TOTAL_DURATION) at the very end
156
 
157
+ NO muted on video when subject voice is needed.
158
+ EXIT ANIMATIONS BANNED except final scene.
159
 
160
  ---
161
 
162
  ## REFERENCE FILES
163
 
164
+ - scripts/watch_gdrive.py -- Run on every Drive video to watch + transcribe
165
+ - references/composition.md -- HTML template, GSAP patterns, scene timing
166
+ - references/music.md -- ElevenLabs API code, YuE fallback
167
+ - references/styles.md -- Dark Premium / Warm Documentary / Bold Editorial
168
+ - references/transcription.md -- Quote extraction algorithm detail
169
+ - references/workflows.md -- Scene structures for all 4 video types
170
 
171
  ## CONNECTIONS
172
 
173
+ | Tool | When | How |
174
+ |------------|------------------|------------------------------------------------|
175
+ | Drive | Scan + download | Composio GOOGLEDRIVE_FIND_FILE account=tomoxi |
176
+ | GitHub | Push composition | Composio GITHUB_COMMIT_MULTIPLE_FILES AIGoose/ |
177
+ | HF Space | Render video | POST aigoose-hyperframes-video-studio.hf.space |
178
+ | ElevenLabs | Music | POST /v1/music with xi-api-key header |
179
+ | Groq/OAI | Transcription | watch_gdrive.py auto waterfall |
180
+
181
+ ## INSTALL watch_gdrive.py
182
+
183
+ ```bash
184
+ curl -o watch_gdrive.py https://raw.githubusercontent.com/AIGoose/dee-video-studio/main/skills/hyperframes-video/scripts/watch_gdrive.py
185
+ pip install gdown --break-system-packages
186
+ python3 watch_gdrive.py --local /path/to/video.MOV
187
+ ```
skills/hyperframes-video/scripts/watch_gdrive.py ADDED
@@ -0,0 +1,295 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ watch_gdrive.py -- Give Claude the ability to WATCH Google Drive videos.
4
+
5
+ Based on bradautomates/claude-video (MIT). Adapted for private Google Drive.
6
+ Pipeline:
7
+ 1. Download .MOV/.MP4 from Google Drive (gdown or direct)
8
+ 2. Extract frames at auto-scaled fps (same budget as claude-video)
9
+ 3. Transcribe: Groq whisper-large-v3 -> OpenAI whisper-1 -> HF Inference
10
+ 4. Clean transcript (Indonesian filler words + repetitions)
11
+ 5. Score segments -> find best testimonial quote
12
+ 6. Print frame paths + transcript for Claude to Read
13
+
14
+ Usage:
15
+ python3 watch_gdrive.py --local /tmp/video.MOV
16
+ python3 watch_gdrive.py --local /tmp/video.MOV --start 0:23 --end 0:45
17
+ python3 watch_gdrive.py --file-id DRIVE_ID (shared files only)
18
+ python3 watch_gdrive.py --file-id DRIVE_ID --resolution 1024 (read on-screen text)
19
+
20
+ Dependencies: ffmpeg, ffprobe, pip install gdown
21
+ API keys (at least one): GROQ_API_KEY or OPENAI_API_KEY or HF_TOKEN
22
+ """
23
+ from __future__ import annotations
24
+ import argparse, json, os, re, subprocess, sys, tempfile
25
+ from pathlib import Path
26
+
27
+ MAX_FPS = 2.0
28
+ FILLERS_ID = {
29
+ "eh","ehm","umm","hmm","hm","uh","itu","ya","kan","gitu","jadi",
30
+ "terus","emang","kayak","kaya","oke","nah","tuh","deh","sih","loh","dong"
31
+ }
32
+ OUTCOME = ["sekarang","bisa","ternyata","langsung","akhirnya","berhasil"]
33
+ SURPRISE = ["tidak sangka","kaget","ternyata","wow","luar biasa","gak nyangka"]
34
+ TRANSFORM = ["dulu","sebelum","awalnya","pikir","tadinya","berubah"]
35
+ RECOMMEND = ["rekomen","sarankan","wajib","harus coba","semua orang","tim kamu"]
36
+
37
+
38
+ def download_gdrive(file_id: str, dest_dir: Path) -> Path:
39
+ """Download from Google Drive. gdown for shared files."""
40
+ dest_dir.mkdir(parents=True, exist_ok=True)
41
+ out = dest_dir / f"{file_id}.mov"
42
+ r = subprocess.run(
43
+ ["python3","-m","gdown",f"https://drive.google.com/uc?id={file_id}","-O",str(out)],
44
+ capture_output=True, timeout=300
45
+ )
46
+ if r.returncode == 0 and out.exists() and out.stat().st_size > 10000:
47
+ print(f"[gdrive] Downloaded: {out.name} ({out.stat().st_size//1024}KB)", file=sys.stderr)
48
+ return out
49
+ import urllib.request
50
+ url = f"https://drive.google.com/uc?export=download&id={file_id}&confirm=t"
51
+ urllib.request.urlretrieve(url, out)
52
+ if out.exists() and out.stat().st_size > 10000:
53
+ print(f"[gdrive] Direct: {out.name}", file=sys.stderr)
54
+ return out
55
+ raise SystemExit(
56
+ f"Cannot download {file_id}.\n"
57
+ "If private: use Composio GOOGLEDRIVE_DOWNLOAD_FILE then --local path."
58
+ )
59
+
60
+
61
+ def get_duration(path: Path) -> float:
62
+ r = subprocess.run(
63
+ ["ffprobe","-v","quiet","-print_format","json","-show_format",str(path)],
64
+ capture_output=True, text=True
65
+ )
66
+ return float(json.loads(r.stdout).get("format",{}).get("duration",0))
67
+
68
+
69
+ def auto_fps(duration: float, max_frames: int = 80) -> tuple[float, int]:
70
+ """Same frame budget as bradautomates/claude-video"""
71
+ if duration <= 30: target = 30
72
+ elif duration <= 60: target = 40
73
+ elif duration <= 180: target = 60
74
+ elif duration <= 600: target = 80
75
+ else: target = 100
76
+ target = min(target, max_frames, 100)
77
+ fps = min(target / max(duration, 1), MAX_FPS)
78
+ return fps, target
79
+
80
+
81
+ def extract_frames(
82
+ video: Path, out_dir: Path, fps: float,
83
+ resolution: int = 512, start: float | None = None, end: float | None = None
84
+ ) -> list[dict]:
85
+ out_dir.mkdir(parents=True, exist_ok=True)
86
+ cmd = ["ffmpeg","-i",str(video)]
87
+ if start: cmd += ["-ss", str(start)]
88
+ if end: cmd += ["-to", str(end)]
89
+ cmd += ["-vf", f"fps={fps:.4f},scale={resolution}:-2",
90
+ "-q:v","3", str(out_dir/"frame_%06d.jpg"), "-y", "-loglevel","error"]
91
+ subprocess.run(cmd, check=True)
92
+ base = start or 0.0
93
+ return [{"path": str(f), "timestamp_seconds": base + i/fps}
94
+ for i, f in enumerate(sorted(out_dir.glob("frame_*.jpg")))]
95
+
96
+
97
+ def fmt_time(s: float) -> str:
98
+ t = int(round(s)); h,r = divmod(t,3600); m,s2 = divmod(r,60)
99
+ return f"{h}:{m:02d}:{s2:02d}" if h else f"{m:02d}:{s2:02d}"
100
+
101
+
102
+ def extract_audio(video: Path, dest: Path) -> Path:
103
+ subprocess.run([
104
+ "ffmpeg","-i",str(video),"-ar","16000","-ac","1",
105
+ "-c:a","libmp3lame","-b:a","32k",str(dest),"-y","-loglevel","error"
106
+ ], check=True)
107
+ return dest
108
+
109
+
110
+ def _multipart(fields: dict, file_field: str, file_path: Path, file_ct: str, boundary: str) -> bytes:
111
+ body = b""
112
+ for k, v in fields.items():
113
+ body += f"--{boundary}\r\nContent-Disposition: form-data; name=\"{k}\"\r\n\r\n{v}\r\n".encode()
114
+ body += f"--{boundary}\r\nContent-Disposition: form-data; name=\"{file_field}\"; filename=\"{file_path.name}\"\r\nContent-Type: {file_ct}\r\n\r\n".encode()
115
+ body += file_path.read_bytes()
116
+ body += f"\r\n--{boundary}--\r\n".encode()
117
+ return body
118
+
119
+
120
+ def transcribe_groq(audio: Path, key: str) -> list[dict]:
121
+ import urllib.request, ssl
122
+ boundary = "----WatchGroq"
123
+ body = _multipart(
124
+ {"model": "whisper-large-v3", "response_format": "verbose_json",
125
+ "timestamp_granularities[]": "segment"},
126
+ "file", audio, "audio/mpeg", boundary
127
+ )
128
+ req = urllib.request.Request(
129
+ "https://api.groq.com/openai/v1/audio/transcriptions", data=body,
130
+ headers={"Authorization":f"Bearer {key}","Content-Type":f"multipart/form-data; boundary={boundary}"}
131
+ )
132
+ with urllib.request.urlopen(req, context=ssl.create_default_context(), timeout=120) as r:
133
+ d = json.loads(r.read())
134
+ return [{"start":s["start"],"end":s["end"],"text":s["text"].strip()} for s in d.get("segments",[])]
135
+
136
+
137
+ def transcribe_openai(audio: Path, key: str) -> list[dict]:
138
+ import urllib.request, ssl
139
+ boundary = "----WatchOAI"
140
+ body = _multipart({"model":"whisper-1","response_format":"verbose_json"},"file",audio,"audio/mpeg",boundary)
141
+ req = urllib.request.Request(
142
+ "https://api.openai.com/v1/audio/transcriptions", data=body,
143
+ headers={"Authorization":f"Bearer {key}","Content-Type":f"multipart/form-data; boundary={boundary}"}
144
+ )
145
+ with urllib.request.urlopen(req, context=ssl.create_default_context(), timeout=120) as r:
146
+ d = json.loads(r.read())
147
+ return [{"start":s["start"],"end":s["end"],"text":s["text"].strip()} for s in d.get("segments",[])]
148
+
149
+
150
+ def transcribe_hf(audio: Path) -> list[dict]:
151
+ import urllib.request, ssl
152
+ hft = os.environ.get("HF_TOKEN","")
153
+ req = urllib.request.Request(
154
+ "https://api-inference.huggingface.co/models/openai/whisper-large-v3",
155
+ data=audio.read_bytes(),
156
+ headers={"Content-Type":"audio/mpeg",**(({"Authorization":f"Bearer {hft}"}) if hft else {})}
157
+ )
158
+ with urllib.request.urlopen(req, context=ssl.create_default_context(), timeout=180) as r:
159
+ d = json.loads(r.read())
160
+ return [{"start":c["timestamp"][0] or 0,"end":c["timestamp"][1] or 0,"text":c["text"].strip()}
161
+ for c in d.get("chunks",[]) if c.get("text","").strip()]
162
+
163
+
164
+ def transcribe(video: Path, work: Path) -> list[dict]:
165
+ audio = extract_audio(video, work/"audio.mp3")
166
+ if k := os.environ.get("GROQ_API_KEY"):
167
+ try:
168
+ print("[watch] Groq whisper-large-v3...", file=sys.stderr)
169
+ segs = transcribe_groq(audio, k)
170
+ if segs: print(f"[watch] {len(segs)} segments via Groq", file=sys.stderr); return segs
171
+ except Exception as e: print(f"[watch] Groq failed: {e}", file=sys.stderr)
172
+ if k := os.environ.get("OPENAI_API_KEY"):
173
+ try:
174
+ print("[watch] OpenAI whisper-1...", file=sys.stderr)
175
+ segs = transcribe_openai(audio, k)
176
+ if segs: print(f"[watch] {len(segs)} segments via OpenAI", file=sys.stderr); return segs
177
+ except Exception as e: print(f"[watch] OpenAI failed: {e}", file=sys.stderr)
178
+ print("[watch] HF Inference whisper-large-v3...", file=sys.stderr)
179
+ segs = transcribe_hf(audio)
180
+ print(f"[watch] {len(segs)} segments via HF", file=sys.stderr)
181
+ return segs
182
+
183
+
184
+ def clean(segs: list[dict]) -> list[dict]:
185
+ out = []
186
+ for seg in segs:
187
+ words = seg["text"].strip().split()
188
+ while words and words[0].lower().rstrip(",.!?") in FILLERS_ID: words.pop(0)
189
+ while words and words[-1].lower().rstrip(",.!?") in FILLERS_ID: words.pop()
190
+ text = re.sub(r'\b(\w+)(\s+\1)+\b', r'\1', " ".join(words), flags=re.IGNORECASE).strip()
191
+ if len(text) > 3: out.append({**seg, "text": text})
192
+ return out
193
+
194
+
195
+ def best_quote(segs: list[dict]) -> dict:
196
+ best_s, best_i = 0, 0
197
+ for i, seg in enumerate(segs):
198
+ t = seg["text"].lower(); s = 0
199
+ s += sum(2 for m in OUTCOME if m in t)
200
+ s += sum(3 for m in SURPRISE if m in t)
201
+ s += sum(2 for m in TRANSFORM if m in t)
202
+ s += sum(3 for m in RECOMMEND if m in t)
203
+ if any(c.isdigit() for c in t): s += 2
204
+ dur = (seg.get("end") or 0) - (seg.get("start") or 0)
205
+ if 8 <= dur <= 18: s += 2
206
+ if s > best_s: best_s, best_i = s, i
207
+ seg = segs[best_i]
208
+ start, end = seg.get("start",0), seg.get("end",0)
209
+ if (end-start) < 8 and best_i+1 < len(segs):
210
+ n = segs[best_i+1]
211
+ if n.get("end",0)-start <= 15: end=n["end"]; seg={**seg,"text":seg["text"]+" "+n["text"]}
212
+ t = seg["text"].lower()
213
+ qtype = "transformation"
214
+ if any(m in t for m in SURPRISE): qtype = "surprise"
215
+ elif any(m in t for m in OUTCOME): qtype = "outcome"
216
+ elif any(m in t for m in RECOMMEND): qtype = "recommendation"
217
+ return {"quote":seg["text"],"start":round(start,1),"end":round(end,1),
218
+ "duration":round(end-start,1),"type":qtype}
219
+
220
+
221
+ def report(source, video, duration, frames, segs, quote, work):
222
+ fps = len(frames)/max(duration,1)
223
+ print(); print("# watch_gdrive: video report"); print()
224
+ print(f"- **Source:** {source}")
225
+ print(f"- **File:** {video.name}")
226
+ print(f"- **Duration:** {fmt_time(duration)} ({duration:.1f}s)")
227
+ print(f"- **Frames:** {len(frames)} @ {fps:.3f} fps")
228
+ print(f"- **Transcript:** {len(segs)} segments (cleaned)")
229
+ print(); print("## Frames"); print()
230
+ print("**Read each frame path with the Read tool to see the video.**")
231
+ print(f"Frames at: `{work/'frames'}`"); print()
232
+ for f in frames: print(f"- `{f['path']}` (t={fmt_time(f['timestamp_seconds'])})")
233
+ print(); print("## Transcript"); print()
234
+ print("```")
235
+ for seg in segs: print(f"[{fmt_time(seg.get('start',0))}] {seg['text']}")
236
+ print("```")
237
+ print(); print("## Best Testimonial Quote"); print()
238
+ print(f"**Type:** {quote['type']}")
239
+ print(f"**Timestamp:** {fmt_time(quote['start'])} -> {fmt_time(quote['end'])} ({quote['duration']}s)")
240
+ print(f'**Quote:** "{quote["quote"]}"')
241
+ print(); print("---"); print(f"_Work dir: `{work}`_")
242
+
243
+
244
+ def parse_time_str(s: str | None) -> float | None:
245
+ if not s: return None
246
+ p = s.split(":")
247
+ if len(p)==1: return float(p[0])
248
+ if len(p)==2: return int(p[0])*60+float(p[1])
249
+ return int(p[0])*3600+int(p[1])*60+float(p[2])
250
+
251
+
252
+ def main():
253
+ ap = argparse.ArgumentParser(description="Watch Google Drive video for Claude")
254
+ src = ap.add_mutually_exclusive_group(required=True)
255
+ src.add_argument("--file-id", help="Google Drive file ID")
256
+ src.add_argument("--local", help="Local video path")
257
+ ap.add_argument("--start", help="Focus start (SS or MM:SS)")
258
+ ap.add_argument("--end", help="Focus end (SS or MM:SS)")
259
+ ap.add_argument("--resolution", type=int, default=512)
260
+ ap.add_argument("--max-frames", type=int, default=80)
261
+ ap.add_argument("--out-dir")
262
+ args = ap.parse_args()
263
+
264
+ work = Path(args.out_dir).expanduser() if args.out_dir else Path(tempfile.mkdtemp(prefix="watch-gdrive-"))
265
+ work.mkdir(parents=True, exist_ok=True)
266
+ print(f"[watch] working dir: {work}", file=sys.stderr)
267
+
268
+ video = (Path(args.local).expanduser().resolve() if args.local
269
+ else download_gdrive(args.file_id, work/"download"))
270
+ if not video.exists(): raise SystemExit(f"File not found: {video}")
271
+ source = str(video) if args.local else f"drive://{args.file_id}"
272
+
273
+ start_sec, end_sec = parse_time_str(args.start), parse_time_str(args.end)
274
+ duration = get_duration(video)
275
+ eff_dur = (end_sec or duration) - (start_sec or 0)
276
+ fps, _ = auto_fps(eff_dur, args.max_frames)
277
+
278
+ print(f"[watch] {int(fps*eff_dur)} frames at {fps:.3f}fps over {eff_dur:.1f}s...", file=sys.stderr)
279
+ frames = extract_frames(video, work/"frames", fps, args.resolution, start_sec, end_sec)
280
+ print(f"[watch] {len(frames)} frames extracted", file=sys.stderr)
281
+
282
+ segs_raw = transcribe(video, work)
283
+ if start_sec or end_sec:
284
+ lo, hi = start_sec or 0, end_sec or float("inf")
285
+ segs_raw = [s for s in segs_raw if s.get("end",0)>=lo and s.get("start",0)<=hi]
286
+ segs = clean(segs_raw)
287
+ quote = (best_quote(segs) if segs else
288
+ {"quote":"N/A","start":0,"end":0,"duration":0,"type":"none"})
289
+
290
+ report(source, video, duration, frames, segs, quote, work)
291
+ return 0
292
+
293
+
294
+ if __name__ == "__main__":
295
+ raise SystemExit(main())