Spaces:
Running
Running
|
Download DATA_HANDLING.md from joemartis/Video2Guide: direct link, hf CLI and curl.
- Browser
- Download file 7.05 kB
-
https://huggingface.co/spaces/joemartis/Video2Guide/resolve/main/DATA_HANDLING.md
- Command line
-
hf download hf://spaces/joemartis/Video2Guide/DATA_HANDLING.md
-
curl -L -o DATA_HANDLING.md https://huggingface.co/spaces/joemartis/Video2Guide/resolve/main/DATA_HANDLING.md
7.05 kB
| # The Describatron β Data Handling Statement | |
| _Prepared for IP / information-security review. Describes exactly what | |
| data the application processes, what leaves the processing environment, | |
| and where. Claims about the application's own behaviour are verifiable in | |
| the source code (file references given). Claims about the third-party | |
| LLM provider's policies should be confirmed against that provider's | |
| current published terms and any contract you hold with them._ | |
| --- | |
| ## 1. What the application does | |
| The Describatron converts a lecture **video** and its **transcript** | |
| (VTT/SRT) into an accessible HTML study guide. The heavy processing β | |
| scene detection, frame extraction, OCR, face filtering, audio slicing, | |
| transcript alignment, HTML rendering β runs **locally** inside the | |
| application's own process. No third party is involved in any of it. | |
| One optional stage uses a Large Language Model (Anthropic's Claude) to | |
| enrich the output (topic segmentation, scene/alt-text descriptions, | |
| section titles, on-screen-text cleanup, equation extraction). **This | |
| stage is the only point at which any content leaves the application | |
| environment, and it is entirely opt-in.** | |
| ## 2. When does anything leave the environment? | |
| The LLM stage runs **only if both** of these are true: | |
| 1. The user supplies an **Anthropic API key** in the request, **and** | |
| 2. The user **ticks one or more AI feature toggles** on the upload form. | |
| If no API key is provided, or no AI toggle is enabled, **no data is sent | |
| to any external service** β the guide is produced entirely from local | |
| processing. (Enforced in `app/pipeline/llm_extract.py`: `is_enabled()` | |
| and `LLMToggles.any_enabled` gate every call; with either false the | |
| functions return empty results without constructing a network client.) | |
| ## 3. Exactly what is sent to the LLM, and what is not | |
| When the LLM stage runs, the requests to Anthropic's API contain: | |
| | Feature (opt-in) | Payload sent to Anthropic | | |
| |---|---| | |
| | Topic segmentation | The **transcript text** (timestamped cue lines). Optionally, downscaled **still-frame thumbnails** of slides if "with slide images" is enabled. | | |
| | Scene/alt-text description, section titles, OCR cleanup, equations | For each kept slide: one **downscaled still frame** (JPEG) + that segment's **narration text** (+ optionally the local OCR text). | | |
| **What is never sent to the LLM:** | |
| - **The video file itself.** Only individual still frames are ever | |
| extracted and transmitted β never the source video. (Frames are | |
| produced locally by `scenes.py`; only their bytes reach | |
| `_encode_image` / `_thumbnail_b64` in `llm_extract.py`.) | |
| - Any audio. | |
| - Any content for segments/features the user did not enable. | |
| Images are **downscaled before transmission** (segment frames to β€800px | |
| wide; topic thumbnails to β€768px on the long edge) β this is a | |
| cost/latency measure but also reduces the fidelity of what is sent. | |
| ## 4. The only external network destinations | |
| For completeness, every outbound network call the application can make: | |
| 1. **Anthropic API** (`api.anthropic.com`, via the official `anthropic` | |
| Python SDK) β user content, **only** under the opt-in conditions in | |
| Β§2. This is the sole destination for any user IP. | |
| 2. **One-time model download** β a ~230 KB open-source face-detection | |
| model (`face_detection_yunet_*.onnx`) fetched from GitHub on first | |
| run (`yunet.py`). **No user data is sent**; this is an inbound asset | |
| download, cached thereafter. | |
| 3. **Browser-side only** (loaded by the viewer's browser, not the | |
| server): MathJax from a public CDN and web fonts from Google Fonts. | |
| These serve static assets to the browser and carry **no lecture | |
| content**. They can be self-hosted to remove even this. | |
| There is **no** telemetry, analytics, or other third-party data egress | |
| in the application. | |
| ## 5. API-key handling | |
| - The Anthropic API key is supplied **per request** and used only to | |
| authenticate that request's LLM calls. | |
| - It is **not persisted** to disk or a database, and the client object | |
| built from it is **not cached** beyond the request | |
| (`_build_client`, `llm_extract.py`). | |
| - The key is **scrubbed from error messages before logging** so it | |
| cannot leak into logs (`extract_one`'s exception handler). | |
| ## 6. Where the raw video/transcript are processed (hosting) | |
| Separate from the LLM question: the application currently runs as a | |
| **Hugging Face Space** (a hosted container). When a user uploads a video | |
| and transcript, those files are processed **inside that container** and | |
| written to **ephemeral temporary storage** that is deleted when the job | |
| completes or the container is reclaimed; outputs are streamed to the | |
| user and **not retained server-side** after download. If processing the | |
| raw IP inside a third-party-hosted container is itself a concern, the | |
| application is a self-contained FastAPI app and can be **run on | |
| infrastructure you control** (on-prem or your own cloud) with no change | |
| to behaviour β the LLM stage remains just as opt-in there. | |
| ## 7. Third-party LLM provider policy (verify independently) | |
| The following reflect Anthropic's **publicly documented** policies for | |
| their commercial API as of this writing and should be **confirmed | |
| against Anthropic's current terms and any agreement you hold** β they | |
| are not guarantees made by this application or its authors: | |
| - Anthropic states it does **not train its models on inputs or outputs | |
| submitted through the commercial API** by default. | |
| - Anthropic offers a **Zero Data Retention (ZDR)** arrangement for | |
| qualifying customers/use cases, under which API inputs/outputs are not | |
| retained after a response is returned. | |
| - Absent ZDR, Anthropic retains API inputs/outputs for a limited period | |
| for trust-and-safety / abuse-monitoring purposes. Confirm the current | |
| retention window and terms directly with Anthropic. | |
| ## 8. Options to further constrain IP exposure | |
| If the IP team wants to reduce or eliminate exposure, in increasing order | |
| of strictness: | |
| 1. **Use the app without an API key / with AI toggles off** β zero | |
| external egress of any lecture content; you still get scene | |
| detection, OCR, transcript alignment and the accessible HTML output. | |
| 2. **Enable only text-based AI features** (leave "topic segmentation with | |
| slide images" off) so slide **images** are not transmitted β only | |
| transcript/narration text is. | |
| 3. **Pursue a Zero-Data-Retention agreement** with Anthropic before | |
| enabling AI features, so submitted content is not retained. | |
| 4. **Self-host the application** on infrastructure you control to keep | |
| the raw video/transcript off third-party hosting; combine with (3) | |
| for the LLM stage. | |
| 5. **Use your own Anthropic enterprise account/key** so all AI calls fall | |
| under your organisation's contract and data terms rather than an ad-hoc | |
| key. | |
| --- | |
| _This document describes the application as of the current revision. | |
| Behavioural claims (Β§Β§1β6) are verifiable in the referenced source files. | |
| Provider-policy claims (Β§7) require independent confirmation with | |
| Anthropic._ | |