Spaces:
Running
Running
Commit History
-
Add AI slide-to-transcript alignment for untimestamped images
Claude +1: Claude Opus 4.8When the images-input mode gets a transcript but the slide filenames carry no HH-MM-SS timestamps, place each slide on the timeline by content instead of failing. Locally OCR each slide (Tesseract) and ask Claude when the narration first discusses it, yielding real per-slide start times. Falls back to an even split across the transcript span when there's no API key or alignment fails. - image_input.py: add all_have_timestamps, order_by_name, build_frames_from_starts, build_frames_even_split (clamped, ordered). - llm_extract.py: add align_slides_to_transcript + SlideAlignError (text-based, so it works through Anthropic-compatible proxies). - /jobs/from-images: three-way timeline branch (order-only / timestamp-anchored / AI-aligned) run off the event loop. - Update images.html hint, README to note timestamps are now optional. - Tests for the new helpers and alignment cleanup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01…e646192 -
fix: images-mode ZIP dropped all images; retry malformed LLM JSON
ClaudeTwo issues from a real images-mode run: ZIP bundle had no images (broken guide). The /jobs/{id}/zip endpoint gated asset filenames on the video-mode scene_NNNN.jpg pattern, so images-mode uploads that keep their original names ("slide 1.jpg") were filtered out — never written to the temp dir, never bundled — leaving HTML that references static/ files that aren't there. Replace the rigid pattern with _is_safe_asset_name: a bare basename (no separators / no traversal, enforced by Path(name).name == name) with an allowed image/ audio extension. Keeps the path-traversal protection (that gate was the reason for the regex) while accepting any legitimate filename. Malformed-JSON responses dropped a slide's enrichment. The model occasionally emits invalid JSON (an unescaped quote → "Expecting ',' delimiter"), which failed the parse and left that segment blank. Now: json.loads uses strict=False (tolerates literal control chars in strings), and extract_one retries the call ONCE on a parse/vali…f09a075 -
fix: parse LLM output ourselves so it works through the Parley proxy
ClaudeThe per-segment and topic calls used client.messages.parse(output_format=…), Anthropic's structured-output beta. The MIT Parley proxy doesn't enforce it: the model returns free text — sometimes a ```json-fenced object, sometimes a conversational "I'm ready to analyze…" — and .parse then fails to validate all of it, so every enrichment came back empty. Switch both call sites to plain messages.create and extract the JSON ourselves: * _create_with_retry replaces _parse_with_retry (create, no output_format). * _response_text joins the response's text blocks; _extract_json_object strips a ```json fence and pulls the outermost {...} span, then json.loads. A reply with no JSON raises, and the caller returns empty for that segment (no crash). * The system prompts now spell out the exact JSON keys/shape and demand "only a JSON object, no fences" — nudges non-enforcing proxies onto the rails. * _build_extraction_model fields are now optional (defaults), so a response that omits one …e5c847b -
feat: images mode without a transcript needs no HH-MM-SS filenames
ClaudeWhen no transcript is uploaded there's no narration to align, so slide timestamps are pointless. Names now only matter when a transcript is present: * With a transcript: filenames must carry HH-MM-SS (unchanged) so narration lines up by time. * Without a transcript: build_frames_ordered_by_name sorts slides by natural filename order (slide2 before slide10) and assigns synthetic positions — no timestamp parsing, any names work. Such segments are marked "untimed": build_segments titles them "Slide N" (not "Segment N — 0:0N") and both the guide and editor templates suppress the start–end time label, which would otherwise show meaningless synthetic values. Threaded via PipelineInputs.untimed and a new Segment.untimed field. The /jobs/from-images endpoint picks the builder by whether a transcript was provided; images.html explains both cases. Tests: natural-sort ordering with no timestamps; untimed render omits meta-time and titles "Slide N"; the timed path still shows the label. …bbe4867 -
feat: make the transcript optional in images mode
ClaudeOn /images you can now omit the transcript and get a slides-only guide: each image anchored at its timestamp, no narration. Useful when you just want an accessible image deck (especially with AI scene descriptions / alt-text turned on). Wiring: * build_frames_from_images: transcript_end is now optional. With none, timestamp parsing skips the range guard and the last slide runs to its own timestamp (no known end). * PipelineInputs gains allow_empty_transcript (default False, so the video path still fails fast on a missing transcript). When True, _load_cues returns [] for a missing transcript and run_pipeline skips the zero-cues error — a slides-only guide is a valid result there. * /jobs/from-images: transcript is optional (File(None)); "no filename" is treated as no transcript. transcript_path/transcript_end stay None, allow_empty_transcript=True. Topic segmentation still requires a transcript, so it's rejected with a clear 400 when absent. * images.html: transcript fi…d9cde51 -
feat: route MIT Parley keys to the Parley proxy (alongside normal Anthropic keys)
ClaudeSupport both a normal Anthropic key (sk-ant-…) and an MIT Parley key (sk-parley-…) in the same API-key field. A Parley key must hit the Parley proxy rather than api.anthropic.com, so _build_client now resolves the base URL by key prefix: sk-parley → PARLEY_BASE_URL (default https://parley.api.mit.edu, env-overridable), everything else → the SDK default. An explicit ANTHROPIC_BASE_URL still routes non-Parley keys to a custom compatible proxy, but never overrides Parley routing. Detection is by prefix so no server config is needed — the user just pastes whichever key they have and it goes to the right endpoint. Tests cover Parley vs Anthropic routing and the env override. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg897f74e
-
fix: close the 3 previously-unverified low-severity audit findings
ClaudeThe first bug audit had 3 findings whose verifier agents died on an API session limit, leaving them unconfirmed. Re-verified against current code — all real, all low severity, two fixed here (the third is a recommendation, below): * extract_all built its shared anthropic client OUTSIDE any try, so a missing/broken SDK (the import is lazy precisely so the app runs without the package) would abort the whole job instead of degrading to empty results — contradicting the module's "always returns, pipeline can always proceed" contract that extract_one and extract_topics honor. Guard the shared-client construction: on failure, log and return empty results with progress marked complete. * /process (the legacy non-browser endpoint) lacked the "AI features requested but no API key" guard that /jobs and /jobs/from-images have, so a script POSTing llm_* toggles with no key got a 200 with every AI feature silently dropped. Add the same fail-fast 400. Not changed (recommendation…0658e45 -
fix: security + bug findings from the new-feature audit
ClaudeAdversarial audit of the images-input and /edit features found four confirmed issues, all fixed: [HIGH bug] image_input.py timestamp regex fused a date/counter prefix onto the time. Delimiters were interchangeable, so "2024-01-15_00-05-30.png" parsed as 01:15:00 (the date's 01-15 fused with the time's leading 00), losing the real 00:05:30 — the slide landed ~70 min off, or the upload was rejected outright for a validly-named file. Now the two separators must be the SAME delimiter (backreference), so a mixed "-"/"_" date-time boundary can't match; matching runs inside a lookahead so overlapping candidates are all found, and the trailing in-range one wins. [HIGH security] /edit arbitrary server-file read via audio_filename path traversal. /edit renders untrusted JSON; the image guard required an inlined image_data_uri, but audio fell through to a disk read when audio_data_uri was null, and metadata_to_segment did Path(frames_dir)/audio_filename with no sanitisation — "../../../../etc/p…1ca1018 -
feat: self-contained edits export + /edit page to reopen a shared JSON
ClaudeMakes the editor's JSON export a true handoff artifact: one file that someone else can open and keep editing, with no accompanying images and no live job. Self-contained export (review.html): * readState now embeds each segment's current image as a data URI (and audio when inlined), so "Save edits (JSON)" carries the slides, not just references to them. The primary <img> src is already a data URI in review mode (even after alt-switching), so it's captured verbatim; a non-data src is omitted rather than saved broken. * downloadZIP strips those embedded URIs before POSTing (it sends images separately + keeps audio server-side) so the ZIP request doesn't carry the image bytes twice. * applyState restores the <img> from image_data_uri when a self-contained JSON is loaded into a live editor, so a file from another session shows the right slides. Data model (metadata.py, html_gen.py): * SegmentMetadata gains optional image_data_uri / audio_data_uri; migration + round-trip…89cae6c -
fix: flatten RGBA/palette PNGs before JPEG-encoding for the LLM
ClaudeLive test of the images-input mode surfaced "cannot write mode RGBA as JPEG" on every frame: PNG slide exports commonly carry an alpha channel (RGBA) or a palette with transparency, and _encode_image saved straight to JPEG, which has no alpha — so PIL raised and every per-segment LLM call failed, leaving descriptions/alt-text/OCR empty. Flatten onto a white background before encoding (transparent slide regions read as white, matching how a sighted viewer sees them), and convert any other non-RGB/L mode to RGB. Only affects the bytes sent to Claude; the displayed/embedded image still uses the original PNG via the data-URI path, so on-screen quality is unchanged. Tests (runnable without cv2): RGBA and palette-with-transparency PNGs both encode to a valid RGB JPEG instead of raising. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg13e0c0c -
feat: images-input mode — build a guide from timestamped slides + a transcript
ClaudeNew input path: upload pre-exported slide images (each named with an HH-MM-SS timestamp, e.g. PNG exports pulled from a lecture at each slide change) plus one SRT/VTT, and get a study guide with no source video and no scene detection. Each image becomes a segment anchored at its timestamp; the transcript aligns to those anchors via the same downstream pipeline the video path uses. Why: deterministic (no seeking, no scene-merge threshold — sidesteps the run-to-run scene-count variance), no multi-GB video upload, and the author controls exactly which slides appear and when. Pieces: * app/pipeline/image_input.py — strict HH-MM-SS filename parser (tolerates extra metadata in the name; prefers the trailing time; uses the transcript duration to reject a date/resolution fragment mistaken for a time) and a frame builder that sorts by timestamp, windows each segment to the next slide (last → transcript end), and rejects empty sets, unparseable names, and duplicate timestamps LOUDLY …88d3d3b
-
fix: lifecycle + resource-leak hardening from the audit
ClaudeRemaining confirmed findings, all lower severity: app/jobs.py — job TTL evicted live editors (finding 15): The 10-minute TTL measured from finished_at and was never refreshed by access, so a review session longer than 10 min got its job purged and the editor's lazy /alts, /ocr, /audio fetches and /result re-fetch started 404ing mid-edit. Track last_access_at (bumped on every get()) and purge on idle time = max(finished_at, last_access_at) instead. app/main.py — thread-start leak (finding 19): If the OS refused threading.Thread(...).start() after registry.create(), the job sat in the registry as "running" forever (finished_at stays None → TTL purge never reclaims it), permanently consuming a concurrency slot. Pop the job on a failed start before propagating. app/main.py — upload cap enforced too late (finding 6): The per-field streaming check in _save_upload runs only after Starlette has already spooled the full multipart body to disk. Add an early Content-Len…fbb54cc -
fix: crash + data-loss bugs from the audit (transcript parsing, download headers)
ClaudeFour confirmed crash/data-loss findings: transcript.py: * IndexError when a cue's text starts with '.' — the sentence-boundary check did prefix.rsplit(None,1)[-1] on an empty prefix ([] → [-1] raises). A leading-period cue is common in auto-transcripts. Guard the empty-prefix case. * Hourless WebVTT timestamps (mm:ss.ttt — valid per spec, common for sub-hour clips) matched none of the HH:MM:SS-only regex and produced ZERO cues — the whole transcript silently ignored. Make the hours group optional; SRT (always hours) still matches. * Trailing cue whose midpoint lands at/after the computed tail edge (video duration undershot the final cue's end) matched no bucket window and was dropped from both narration text and sliced audio. Spill it into the last segment — symmetric to the existing head spill. main.py: * A title with non-latin-1 alphanumerics (CJK, emoji, some accented letters — _safe_filename keeps them because "中".isalnum() is True) reached Content-Dispositi…4f70b55 -
fix: persist and honor per-element include toggles (bug audit cluster 1+2)
ClaudeAdversarial bug audit found the per-element include toggles (added this week) were DOM-only state that neither survived Save/Load JSON nor was honored by the server-side render path (Download ZIP, CLI --metadata). Result: a user who curated which blocks carry over got a ZIP that contradicted Download final HTML, and Save→reopen→Load silently reset every toggle to defaults. Also fixes an a11y regression where a duplicate contenteditable-strip made the ARIA-cleanup pass dead code. Persistence (review.html): * readState now serializes alt_text_included, on_screen_text_included, transcript_included, audio_included (description_included already existed), plus page lang and meta_lines (were dropped, resetting a non-English guide to lang=en and losing header metadata on ZIP re-render). * applyState restores all five per-element flags and lang via a shared applyElementInclude helper; a Load JSON refreshes the bulk-toggle indicators. * Description read/write now uses innerText (not…9a3cbf9 -
scenes: deterministic frame extraction to stabilise scene detection
ClaudeUsers reported the "Preview scenes" step returning a different set of scenes on every run of the same video. Root cause: representative-frame extraction used cap.set(CAP_PROP_POS_MSEC, …), OpenCV's approximate millisecond seek. That seek lands on the nearest keyframe and decodes forward, and FFmpeg's multithreaded decoder can return a slightly different frame for the same timestamp across runs. The jitter didn't just change thumbnails — it changed the scene COUNT. _merge_duplicate_adjacent compares each scene's extracted frame against its neighbour with a fixed mean-abs-diff threshold (5.0); when a seek returned a marginally different frame, the MAE crossed the threshold and flipped a merge decision, so scenes that collapsed one run stayed split the next. Fix: add _read_frame_at(cap, time_seconds, fps) which converts the timestamp to an integer frame index and seeks via CAP_PROP_POS_FRAMES — a specific decode target that maps far more reproducibly than a millisecond value. Both extra…e059eb2
-
editor: Download Markdown — semantic .md export
ClaudeAdds a "Download Markdown" button to the editor toolbar that produces a single self-contained .md file from the current editor state. Mirrors "Download final HTML"'s contract: same per-element include toggles, same description-replaces-alt rule, no server round-trip. Output is semantic markdown (no raw HTML hacks): * # for the page title, ## per section, --- between sections * Eyebrow as *italic*, subtitle as plain paragraph * Timestamps as *italic* sub-line under each heading * Image as  — alt is empty when a scene description is present (matches the published-HTML behaviour) or when the alt-text include checkbox is off * Scene description as plain paragraphs (blank-line separated) * On-screen text as a labelled blockquote (>) when its include toggle is on * Transcript as plain paragraphs when its include toggle is on * Math equations as $…$ (inline) or $$…$$ (block), GitHub-flavoured Markdown compatible Audio is intentionally skipped — markdo…e363538
-
a11y pass for accessibility expert review
ClaudeAudit-and-fix pass on the published-guide and editor templates before sending it for an external accessibility review (Rich Caloggero, MIT DSL). Published guide (guide.html / _doc_styles.html): * Drop role="presentation" from decorative <img alt="">. alt="" alone is the WAI-recommended decorative-image pattern; role="presentation" on <img> has spotty AT support — some readers ignore it, others still announce the image. Empty alt is sufficient and more reliable. * Drop the redundant aria-label on <details class="onscreen-text">. The <summary> is the accessible name; layering aria-label on top either overrides the visible summary (WCAG 2.5.3 Label in Name mismatch) or duplicates it. Remove. * Only emit audio aria-describedby="narration-N" when the narration block it points at is actually rendered. A dangling describedby is an ARIA validity bug (WCAG 4.1.2 / 1.3.1) and some screen readers silently swallow the relationship, leaving the audio undescribed. * Bump TOC link…5a2d8c8 -
editor: drop the 'Include section' bulk toggle
ClaudeA global toggle for the per-section master include switch doesn't read as a meaningful bulk action — flipping it either does nothing (every section already included) or empties the published guide. Remove it from the bulk bar; the per-section checkbox stays. The bulk bar now only carries toggles for things that make sense to apply uniformly across all sections: Scene description, Alt-text, On-screen text, Transcript, Audio. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg007ae3b -
editor: bulk include toggles that drive every section at once
ClaudeAdds a "Apply to all sections" bar between the toolbar and the document header. One checkbox per include key — Whole section, Scene description, Alt-text, On-screen text, Transcript, Audio. Three behaviours: * Standard tri-state. The bar reflects the union of per-section state on page load: all-checked, all-unchecked, or indeterminate when sections disagree. * Bulk → per-section. Toggling a bulk box sets every matching per-section checkbox to the bulk's new state and dispatches a synthesised 'change' event on each, so the existing per-section sync (data-include, data-included, data-include-alt-text) fires for free. No duplicated wiring. * Per-section → bulk. Any individual checkbox change re-runs the union calculation so the bulk indicator stays accurate. Applies after Load JSON too (applyState calls syncBulkToggles when it finishes mutating per-section state). A bulk box for a key with zero matching per-section checkboxes (e.g. Audio when no segment carries an au…93e4afe -
editor: per-element alt-text and transcript toggles; rename 'Scene description'
ClaudeThree pieces of UX feedback: * Alt-text now has its own include checkbox in the per-section "Include in published guide" fieldset, default OFF. When off, buildFinalHTML blanks the <img alt> at download time. Because alt is an attribute (not a discrete element), the checkbox writes its state to data-include-alt-text on the section and buildFinalHTML reads it alongside the description-present check; the helper attribute is stripped from the final HTML. * Narration sections are now rendered as <details class="narration"> with summary "Transcript", collapsed by default — in both the editor view and the server-side render (guide.html). A matching per-element include checkbox sits in the fieldset, also default OFF, so transcripts no longer carry over to the published guide unless the editor explicitly opts in. The narration body keeps its stable id so any audio aria-describedby reference still resolves; when transcript is excluded the strip pass collects removed…0c2c651 -
editor: stop emitting the alt-review figcaption when description carries the meaning
ClaudeTwo paths were leaking the "Alt-text needs author review." caption into the final HTML even when it no longer applied: * review.html rendered the figcaption purely on `seg.alt_text_reviewed`, so a segment with a populated description (alt is intentionally empty, not pending) still got the warning. Add `not seg.description` to the conditional so the editor view and the server-side review render both stay quiet in that case. * buildFinalHTML blanked the <img alt> when a description was present but never touched the figcaption next to it. Now the same per-section pass also strips figure.figure > figcaption when either (a) the description is present-and-included, or (b) the user edited alt-text in this session (data-alt-reviewed flipped to 'true'). Both cases mean the "needs review" note is stale. Adds a regression test confirming the caption disappears once `seg.description` is set, regardless of `alt_text_reviewed`. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ…9eb147c
-
editor: remove summary / key-terms / self-check (round 3)
ClaudeAccessibility team's round-three asks: * "Summary Field — redundant, remove" (now subsumed by the new detailed description paragraph from PR 2) * "Key Terms — REMOVE. This replicates the glossary." * "Self-check questions — REMOVE! This is an academic role and such questions are better placed in the platform." Scope of removal: * Editor (review.html): drops the summary <input>, the key-terms list editor block, the self-check-questions list editor block, the related CSS rules (.vgm-key-term-row, .vgm-self-check-row), the readState/applyState/syncBindings hooks, the makeKeyTermRow / makeSelfCheckRow row factories, the add-key-term / add-self-check click handlers, and the entire glossary/highlight injection inside buildFinalHTML. * Published guide (guide.html): drops the "In brief:" advance organizer, the self-check <details> block, the bottom glossary section, the highlight_terms filter usage on narration, and the build_glossary call at the top of <main>. * LLM (ll…b863381 -
editor: long-form image description field (round 2)
ClaudeAdds the "Detailed description" field the accessibility team asked for — an editable open paragraph rendered UNDER the slide image that doubles as the long-form image description. When non-empty AND included in the published guide, the <img alt> drops to "" and the image is marked role="presentation": the description carries the meaning in body text where sighted AND non-sighted readers both encounter it, so there's no need to duplicate it inside an alt attribute that only screen readers hear. The pieces: * SegmentMetadata + Segment + migration gain description / description_ reviewed fields. Round-trips through Save/Load JSON. * New LLMToggles.description toggle. _FIELD_PROMPTS adds a description prompt that follows the team's example ("A gallery of purple flower stem macro photographs is shown. These demonstrate..."), and the structured-output schema gains a matching description field that only fires when the toggle is on (no wasted output tokens otherwise). * Orchestra…cf53b59 -
editor: accessibility-team UX pass (round 1)
ClaudeRound-one feedback from the accessibility team, focused on changes that don't alter the data model or remove fields: * Banner trimmed — three sentences down to one. * Segment image moved above the editable fields so the editor sees what they're describing before they describe it. * Alt-text field is now a multi-row textarea, not a single-line input. Some descriptions take two paragraphs and the old input made that impossible without scrolling a clipped one-liner. * New per-element "Include in published guide" fieldset replaces the single per-section checkbox. The whole-section checkbox stays. Two new checkboxes: - On-screen text from video (defaults to included) - Narration audio clip (defaults to EXCLUDED — per the team, audio shouldn't carry over to the published guide by default) Each checkbox drives data-include on a matching [data-element=…] node; buildFinalHTML now strips anything still flagged false, plus the data-include / data-element attributes …3672e55
-
transcript: spill pre-first-boundary cues into the first segment
ClaudeDeselecting the first topic in the topic-preview flow left the new first kept segment starting well after the transcript did (e.g. the surviving boundaries become [180s, ...] for a transcript that began at 0s). _bucket_cues only matched cues whose midpoint fell inside some [edges[i], edges[i+1]) window, so the opening narration was silently dropped — several minutes of speech vanishing with no error. Spill cues with midpoint < edges[0] into bucket 0 instead. The audio slicer (cue_bounds_per_segment) shares the same bucketing, so its first segment's bounds extend backwards in lockstep and the dropped narration is recovered in audio too. In the common case where the transcript begins at the same time as the first kept segment, this is a no-op. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJgf16b33f -
llm_extract: cut per-segment API cost and add transient-error retry
ClaudeThree small efficiency wins on the Claude calls that dominate job cost: * Drop the Tesseract OCR block from the user message when the LLM is itself producing OCR (toggles.ocr_text). The block was being sent only to be ignored — ~50-300 wasted input tokens per segment, scaled with slide text density. * Build one Anthropic client per extract_all run and reuse it across all worker threads, instead of constructing a fresh client (and TCP+ TLS handshake) per segment. * Warm the prompt cache by firing the first segment synchronously before parallelising the rest. Previously the first wave of 4 worker threads raced for the cache write and only one won, so cache hits started at segment 5; now they start at segment 2. Also wraps both extract_one and extract_topics in a single-retry helper that catches RateLimitError / APIConnectionError / 5xx-overloaded statuses, so a transient blip no longer permanently drops a segment's LLM metadata. 4xx are not retried. https://claude.ai…3602e87 -
Accessibility: term tooltips become real <a> links to glossary entries
ClaudeThe narration's inline highlighter wrapped each defined term in a <mark class="kt" title="…"> with the definition only in the title attribute. WCAG 2.5.1 / 4.1.2 — touch users can't trigger hover, keyboard users can't focus a <mark>, many screen readers skip the title attribute entirely. Definitions were effectively unavailable to roughly half of likely users. Build the glossary once at the top of <main> and pass it to the highlighter so each first-occurrence wrap becomes <a class="kt" href="#glossary-N" title="…">term</a> — focusable, touch-tappable, screen-reader announced as a link to the named target. The bottom glossary's <dt> ids match the hrefs the highlighter emits (build_glossary now stamps each entry with its own `id` field so server template and JS port share the contract). Terms with no glossary entry (defensive) fall back to the old <mark> shape. Mirror in buildFinalHTML: build the glossary entries before walking sections, thread the lookup through highlightTerms, reus…c28a145 -
Cleaner study-guide output: drop redundant per-section key-term list
ClaudeUser feedback: with inline term highlighting AND a bottom glossary, showing the same vocabulary again as a per-section <dl class="key-term"> was triple-redundant and visually noisy. - Drop the per-section <dl class="key-term"> from guide.html. Definitions remain accessible two ways: - inline via the .kt highlighter (hover tooltip on the first occurrence in each paragraph) - at the end of the document via the auto-built glossary (alphabetised, back-anchored to source section). - Mirror in buildFinalHTML so downloaded HTML matches. - Update test_render_guide_renders_key_terms_and_equations to expect the glossary instead of the per-section list. Also cleaned up two other visually-noisy bits while in the same file: - Figure's "Lecture slide." caption: redundant with the section heading + meta-time line and the figure-internal timestamp badge. Now we only emit a figcaption when there's something real to say ("Alt-text needs author r…8fb4d34 -
Third-pass audit fixes: migration setdefaults, robust JS mark builder
ClaudeP1 — migrate_v1_to_v2 didn't setdefault the new learner-feature fields (summary, self_check_questions). Pydantic still supplied defaults on model_validate so no data was lost at load time, but the migration's raw-dict output was asymmetric — any consumer of the migrated dict pre-validation would see undefined keys for those two fields and not for the others. Adds the two setdefaults so the migration's contract stays "every SegmentMetadata field has a defined key after migration". P2 — JS highlightTerms built its <mark> open tag via template-string interpolation: `<mark class="kt" title="${escapeHtml(definition)}">`. The contemporaneous escapeHtml correctly entity-encodes quotes, so no XSS materialized — but the .replace() second argument treats $&, $', $`, $n as special replacement patterns. Any `$` surviving escapeHtml (it doesn't escape `$`) would get mangled in the output title. Switch to building the mark via document.createElement + setAttribute, then serializing with outerHTML t…7562488 -
Update test_html_gen.py
12caec6 -
Learner-focused features: summary, self-check, glossary, inline term highlighting
ClaudeTagged pre-learner-features at the commit before this so a single `git reset --hard pre-learner-features` (or non-destructive `git revert pre-learner-features..HEAD`) takes us back to known-good if anything misbehaves. Four learner-impact features ship together because they share data plumbing — same Segment/Metadata schema additions, same editor round-trip work: #2 One-sentence summary above each section. Claude's `summary` field from topic segmentation is now rendered as an "In brief:" advance organizer above the slide image. Primes the learner before they hit the detail; evidence-based for comprehension. Editable in the review screen. #4 Auto-built glossary at the end of the guide. Collects every segment's key_terms across the whole document, dedupes case- insensitively (first-occurrence wins), sorts alphabetically, and anchors each entry back to the section where the term was first introduced. #5 Inline key-term highlighting in narration. Defined terms w…d61b118 -
Second-pass audit fixes: narrow exception catch, empty-cue early error
ClaudeTwo real findings from a second-pass bug/security audit; the rest of the report was either already handled by existing code (sub-toggle sync runs on dispatched change event) or correctly identified as acceptable / non-bugs. P1 — except BaseException in the /jobs preview-cleanup block was overly broad. Narrow to Exception so KeyboardInterrupt / SystemExit still propagate without first running cleanup that could mask the signal. P2 — Empty transcript silently produced a guide with no narration under any segment. Both topic and non-topic paths now check `if not cues` immediately after _load_cues and raise PipelineError with a message naming the fix ("provide a non-empty VTT/SRT or enable auto-transcribe"). The topic-scene-preview endpoint already had its own empty-cue check; this brings the main pipeline to parity. Verified safe (no changes needed): - filename validation: endpoint regexes already cover /jobs/{id}/zip, /alts, /ocr, /audio - autoescape: Jinja2 select_autoescape([…e8ba7af -
P1/P2 bug audit fixes: index validation, cleanup race, cache purge, type whitelist
ClaudeFour real issues from a thorough audit of the recent feature work: P1 — Preview workdir leaked on validation failure. Between popping the preview cache entry in /jobs and entering the workdir try/finally, any HTTPException raised by index parsing or selection filtering would orphan the temp dir holding the user's uploaded video. Wrap the post-pop validation in its own try/except that cleans up the popped entry before re-raising. Marks the entry None so the outer finally doesn't double-delete. P1 — Out-of-range and negative selected_indices silently dropped. The previous parser used `if i.strip().isdigit()` which (a) accepts any positive integer regardless of frame count and (b) silently ignores negative numbers and any non-digit token. A user submitting "0,5,999" against a 5-frame preview got a 5-frame guide back with no error. Now we parse each token explicitly: ValueError on non-int, 400 on negative, 400 on out-of-range against the cached preview's valid index set (positional for t…1361034 -
Stitcher: add lowercase-continuation fallback for transcripts without periods
ClaudeUser report: a topic boundary fell mid-sentence on a lecture whose Whisper transcript omits terminal punctuation entirely. The stitcher's terminator-split strategy needs at least one .!? somewhere in B's first cues — when none exists, it gave up and the cut stayed. Add a second strategy that uses casing as the signal: if B's first cue starts with a lowercase letter, it's almost certainly continuing A's open sentence. Absorb consecutive lowercase-starting cues into A until we hit one that starts with a capital (next sentence begins) or the existing _MAX_SENTENCE_ABSORB_CUES cap. Triggers only after the terminator strategy has been tried and failed, so behaviour on punctuated transcripts is unchanged. Guarded edge case: when B opens with a capital letter, we don't move cues — the mid-sentence cut is on A's side (transcript dropped a period) and pulling B-cues into A would just mash two real sentences together. Two tests pin both branches; the heat-pump "how to install / and maintain" s…39c2e46
-
Topic prompt: video-type dropdown biases boundaries to genre structure
ClaudeGeneric topic segmentation works but produces splits that ignore genre conventions — a lab walkthrough is sliced like a lecture, a tutorial like a panel. Add a per-job "video type" hint the user can pick from a small dropdown; the orchestrator prepends a paragraph of structural guidance to the topic prompt so Claude knows what shape to bias toward. Five genres modeled today (each one paragraph of prompt): - Classroom lecture: intro → background → concept(s) → examples → wrap-up - Lab walkthrough: intro → theory → materials → procedure → results - How-to tutorial: intro → setup → steps → verification → troubleshooting - Interview / Q&A: boundaries at new questions or subject pivots - Talk / presentation: hook → thesis → supporting points → closer plus "General" which leaves the prompt untouched. Unknown values fall back to general so a stale form post can't break the call. Wired through extract_topics(video_type=...), LLMToggles .topic_video_type, the orchestrator, and both …6dd9ac4 -
Merge pull request #64 from JoeMartis/claude/fix-zip-path-traversal-9kA4Q
Topic primary picking: any OCR text qualifies a scene as slide content09f7889 -
Topic primary picking: any OCR text qualifies a scene as slide content
ClaudeUser reported the iconic "HEAT PUMPS" title-card slide losing the primary-frame race to a longer talking-head shot in the same topic. Diagnosis: my pool threshold of 30 OCR characters was over-cautious. The slide's headline OCRs to roughly "HEAT PUMPS" — about 10 chars after the confidence + 3-char-run filters trim subtext. That falls under 30, drops the slide out of the slide_text pool, and if the face detector even misfires slightly on the AC unit's circular fan the slide ends up in the same borderline pool as real talking heads — where duration wins. `ocr_frame` already returns "" for noise-only frames (per-word confidence + alphanumeric-run filters), so any non-empty output is by construction real text. Relax the pool check from "≥30 chars" to "non-empty after strip" — title cards, short labels, and labelled photos now win against any talking-head shot in the same topic. The 30-char constant is gone. New test pins the HEAT PUMPS scenario exactly: a long borderline talking-head lo…5d54c1c
-
Merge pull request #63 from JoeMartis/claude/fix-zip-path-traversal-9kA4Q
ocr_frame: filter noise per-word by Tesseract confidence, not per-doc…b8bef4c -
ocr_frame: filter noise per-word by Tesseract confidence, not per-document
ClaudeThe doc-level "any 3+ char run = keep everything" heuristic let entire paragraphs of noise through whenever Tesseract hallucinated a single high-confidence word. Real-world example: a close-up of a heat-pump AC unit's fan grille produced "Soweer" (a confident-looking hallucination) plus a dozen lines of "ee", "es Re Re", "SS ESS ES SS" gibberish — and because "Soweer" had a 6-char alnum run, all the surrounding noise sailed through the filter unchanged. Switch from image_to_string to image_to_data to get per-word confidence scores, and drop any word below _MIN_WORD_CONFIDENCE (60) before reassembling lines. Tesseract's own quality estimate is a far better signal than counting alphanumeric runs after the fact. The doc-level _has_real_text check stays as a final sanity net for the case where the confidence filter leaves nothing usable. Tests updated to mock image_to_data with synthetic (word, conf, line) tuples; the heat-pump-style hallucination test pins the specific regression.6f1015b -
Merge pull request #62 from JoeMartis/claude/fix-zip-path-traversal-9kA4Q
Claude/fix zip path traversal 9k a4 q4cba79a -
Skip Tesseract when llm_ocr_text will overwrite its output anyway
ClaudeWhen the user enables Claude's on-screen-text toggle and isn't using topic segmentation, the Tesseract pass is pure waste — its result gets discarded by the LLM-OCR override step a few stages later. That's ~200ms per frame on the HF CPU tier, billed in wall-clock delay every job. Add _llm_will_replace_tesseract(inputs) and short-circuit the OCR loop when it returns True. Topic-segmentation mode keeps Tesseract because the primary-frame picker pools scenes by OCR character count to push talking-head shots out of the way — skipping there would regress the talking-head fix. skip_ocr still wins overall, and a missing API key keeps Tesseract running since the LLM call would no-op. Five focused tests pin every combination of the three relevant inputs (api key, llm_ocr_text toggle, topic_segmentation toggle).496ac75 -
ocr_frame: return "" for textless frames so the guide hides the panel
ClaudeTesseract on a frame with no real on-screen text (photo, diagram, talking-head shot, blank transition) routinely emits a few stray punctuation marks or single letters. With our existing "{% if seg.ocr_text %}" gate that's still truthy, so the rendered guide showed an empty-looking "On-screen text from video" panel full of noise. After merging the normal + inverted passes, check the output for an alphanumeric run of 3+ characters somewhere. If none exists, treat the frame as textless and return "". Captures real short labels ("Lab", "Slide 3", year-only slides) while filtering out "/| _", "a b c", and similar Tesseract artefacts. The guide template's existing empty-check then hides the panel automatically.7873ecc -
Merge pull request #61 from JoeMartis/claude/fix-zip-path-traversal-9kA4Q
Topic validation: repair reversed bounds and drop bad topics rather t…24219e2 -
Topic validation: repair reversed bounds and drop bad topics rather than fail
ClaudeA deployed job failed entirely because the LLM returned one topic with end (31) < start (35). The previous validator raised on the first malformed topic, throwing away an otherwise-valid response and forcing the user to pay for another call. Soften validation: when end < start, swap them (the LLM clearly intended the smaller as start) and continue. When a topic is still zero-length or missing a title after repair, drop just that topic and proceed with the rest. Raise LLMTopicError only when EVERY topic is malformed — at that point the response really is unusable. Tests cover each of the repair paths plus the still-raises-when- nothing-usable case.4b46ff9 -
Merge pull request #60 from JoeMartis/claude/fix-zip-path-traversal-9kA4Q
Topic primary picking: prefer scenes with OCR text over talking-heads9eb7bf7 -
Topic primary picking: prefer scenes with OCR text over talking-heads
ClaudeUser reported that topic-segmented guides still surface talking-head shots as primaries — the face-area tiebreak couldn't help when every scene in a topic's time range was a talking-head (e.g. an intro stretch before the first slide appears). The previous logic would then fall back to longest-on-screen, which is exactly the longest talking-head. Make OCR-text presence the dominant signal: - Run the OCR pass before topic segmentation (it ran after). - Pass per-scene OCR text into _apply_topic_segmentation and pool scenes by content: "has slide text" (≥30 chars), "clean visual" (face area < 3%), or "borderline" (everything else). - Pick longest-on-screen from the highest non-empty pool — so a short slide always beats a long talking-head when slide text is present, falling back to face area, then anything left. - Thread the primary's OCR text back through so the orchestrator doesn't need to re-OCR after rebucketing. Talking-head scenes still appear as switcha…5d9159c -
Merge pull request #59 from JoeMartis/claude/fix-zip-path-traversal-9kA4Q
Claude/fix zip path traversal 9k a4 q419357c -
Topic primary picking: prefer clean slide content over borderline faces
ClaudeThe instructor filter drops frames where the face area exceeds the threshold, but borderline-face frames (e.g. a partial face at 8% of the image, below the 12% drop line) survive. With scene-based output that's fine — each scene becomes its own segment regardless. With topic segmentation, a topic can span several scenes, and the "longest on screen" pick promoted those borderline shots ahead of clean slides whose scenes were shorter. Have the instructor filter record each kept scene's measured face_area_ratio on the SceneFrame. _apply_topic_segmentation now partitions a topic's scenes into "slide content" (ratio < 0.03) and "borderline" pools and picks the longest from the slide-content pool first, only falling back to the borderline pool when the topic has no clean scenes at all. Borderline scenes still appear as switchable alternates in the editor — only the default anchor changes.5c072a9 -
Topic prompt: explicitly ignore talking-head frames for boundaries
ClaudeSome webcam / podium / empty-stage frames slip past the instructor filter and end up in the slide thumbnails sent to Claude. Without guidance, the model occasionally anchors a topic boundary or representative frame on one of these instead of on the surrounding slide content. Add an explicit "treat talking-head thumbnails as transparent" clause to the image-augmented system prompt, with the sensible carve-out for a stretch (e.g. live Q&A) where there's no slide content at all. Test pins the guidance so it can't be silently dropped during a future prompt edit.5d1a426 -
Merge pull request #58 from JoeMartis/claude/fix-zip-path-traversal-9kA4Q
Claude/fix zip path traversal 9k a4 q25ab096