Spaces:
Running
Running
Commit History
-
Clarify API-key field accepts Parley keysβ Running
Claude +1: Claude Opus 4.8The key input already accepts either an Anthropic (sk-ant-β¦) or MIT Parley (sk-parley-β¦) key β routing is by prefix, and the only client check is non-empty. The placeholder said "sk-ant-β¦", making it look Anthropic-only. Update the label and placeholder on both the video and images pages, and note the Parley auto-routing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg327be13
-
Add AI slide-to-transcript alignment for untimestamped imagesβ
Claude +1: Claude Opus 4.8When the images-input mode gets a transcript but the slide filenames carry no HH-MM-SS timestamps, place each slide on the timeline by content instead of failing. Locally OCR each slide (Tesseract) and ask Claude when the narration first discusses it, yielding real per-slide start times. Falls back to an even split across the transcript span when there's no API key or alignment fails. - image_input.py: add all_have_timestamps, order_by_name, build_frames_from_starts, build_frames_even_split (clamped, ordered). - llm_extract.py: add align_slides_to_transcript + SlideAlignError (text-based, so it works through Anthropic-compatible proxies). - /jobs/from-images: three-way timeline branch (order-only / timestamp-anchored / AI-aligned) run off the event loop. - Update images.html hint, README to note timestamps are now optional. - Tests for the new helpers and alignment cleanup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01β¦e646192 -
fix: images-mode ZIP dropped all images; retry malformed LLM JSONβ
ClaudeTwo issues from a real images-mode run: ZIP bundle had no images (broken guide). The /jobs/{id}/zip endpoint gated asset filenames on the video-mode scene_NNNN.jpg pattern, so images-mode uploads that keep their original names ("slide 1.jpg") were filtered out β never written to the temp dir, never bundled β leaving HTML that references static/ files that aren't there. Replace the rigid pattern with _is_safe_asset_name: a bare basename (no separators / no traversal, enforced by Path(name).name == name) with an allowed image/ audio extension. Keeps the path-traversal protection (that gate was the reason for the regex) while accepting any legitimate filename. Malformed-JSON responses dropped a slide's enrichment. The model occasionally emits invalid JSON (an unescaped quote β "Expecting ',' delimiter"), which failed the parse and left that segment blank. Now: json.loads uses strict=False (tolerates literal control chars in strings), and extract_one retries the call ONCE on a parse/valiβ¦f09a075 -
feat: ANTHROPIC_MODEL env override for the LLM model IDβ
ClaudeDiagnosing a Parley run where every slide came back described as "blank"/"no content" (one hexagon diagram was even called "an instructor standing in front of a plain background") β the slides are content-rich, so the model isn't receiving the images. That's a proxy-side issue (image not forwarded, or claude-haiku-4-5 routed to a non-vision model). Add an ANTHROPIC_MODEL env override so a proxy that names its vision model differently can be pointed at the right one, instead of the model ID being hard-coded. Documented with a "must be vision-capable" note. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg5125240 -
fix: parse LLM output ourselves so it works through the Parley proxyβ
ClaudeThe per-segment and topic calls used client.messages.parse(output_format=β¦), Anthropic's structured-output beta. The MIT Parley proxy doesn't enforce it: the model returns free text β sometimes a ```json-fenced object, sometimes a conversational "I'm ready to analyzeβ¦" β and .parse then fails to validate all of it, so every enrichment came back empty. Switch both call sites to plain messages.create and extract the JSON ourselves: * _create_with_retry replaces _parse_with_retry (create, no output_format). * _response_text joins the response's text blocks; _extract_json_object strips a ```json fence and pulls the outermost {...} span, then json.loads. A reply with no JSON raises, and the caller returns empty for that segment (no crash). * The system prompts now spell out the exact JSON keys/shape and demand "only a JSON object, no fences" β nudges non-enforcing proxies onto the rails. * _build_extraction_model fields are now optional (defaults), so a response that omits one β¦e5c847b -
feat: images mode without a transcript needs no HH-MM-SS filenamesβ
ClaudeWhen no transcript is uploaded there's no narration to align, so slide timestamps are pointless. Names now only matter when a transcript is present: * With a transcript: filenames must carry HH-MM-SS (unchanged) so narration lines up by time. * Without a transcript: build_frames_ordered_by_name sorts slides by natural filename order (slide2 before slide10) and assigns synthetic positions β no timestamp parsing, any names work. Such segments are marked "untimed": build_segments titles them "Slide N" (not "Segment N β 0:0N") and both the guide and editor templates suppress the startβend time label, which would otherwise show meaningless synthetic values. Threaded via PipelineInputs.untimed and a new Segment.untimed field. The /jobs/from-images endpoint picks the builder by whether a transcript was provided; images.html explains both cases. Tests: natural-sort ordering with no timestamps; untimed render omits meta-time and titles "Slide N"; the timed path still shows the label. β¦bbe4867 -
feat: make the transcript optional in images modeβ
ClaudeOn /images you can now omit the transcript and get a slides-only guide: each image anchored at its timestamp, no narration. Useful when you just want an accessible image deck (especially with AI scene descriptions / alt-text turned on). Wiring: * build_frames_from_images: transcript_end is now optional. With none, timestamp parsing skips the range guard and the last slide runs to its own timestamp (no known end). * PipelineInputs gains allow_empty_transcript (default False, so the video path still fails fast on a missing transcript). When True, _load_cues returns [] for a missing transcript and run_pipeline skips the zero-cues error β a slides-only guide is a valid result there. * /jobs/from-images: transcript is optional (File(None)); "no filename" is treated as no transcript. transcript_path/transcript_end stay None, allow_empty_transcript=True. Topic segmentation still requires a transcript, so it's rejected with a clear 400 when absent. * images.html: transcript fiβ¦d9cde51 -
feat: route MIT Parley keys to the Parley proxy (alongside normal Anthropic keys)β
ClaudeSupport both a normal Anthropic key (sk-ant-β¦) and an MIT Parley key (sk-parley-β¦) in the same API-key field. A Parley key must hit the Parley proxy rather than api.anthropic.com, so _build_client now resolves the base URL by key prefix: sk-parley β PARLEY_BASE_URL (default https://parley.api.mit.edu, env-overridable), everything else β the SDK default. An explicit ANTHROPIC_BASE_URL still routes non-Parley keys to a custom compatible proxy, but never overrides Parley routing. Detection is by prefix so no server config is needed β the user just pastes whichever key they have and it goes to the right endpoint. Tests cover Parley vs Anthropic routing and the env override. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg897f74e
-
rename: catch the CLI help + metadata docstring (The Describatron)β
ClaudeFollow-up to the rebrand: the vgm --help text (user-facing) and the metadata module docstring still said VideoGuideMaker. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg1dd2955 -
rename: the app is now "The Describatron"β
ClaudeRebrand the user-facing name from "Video Guide Maker" to "The Describatron" β and de-video the copy now that it also builds guides from slide images: * Page titles + headers across index / images / edit pages. * FastAPI app title; published-guide and editor footers. * README (HF Space card title + emoji π¬βπ€ + heading), USER_GUIDE and DATA_HANDLING headings, and the intro prose. * Index eyebrow + sample copy: "lecture video (or slide images) and a transcript" instead of video-only. Infrastructure identifiers left untouched: the GitHub repo (VideoGuideMaker), the HF Space (Video2Guide), the internal package/ logger name (videoguidemaker), and DEPLOY.md's paths β those are real names, not branding. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg2f32611
-
fix: close the 3 previously-unverified low-severity audit findingsβ
ClaudeThe first bug audit had 3 findings whose verifier agents died on an API session limit, leaving them unconfirmed. Re-verified against current code β all real, all low severity, two fixed here (the third is a recommendation, below): * extract_all built its shared anthropic client OUTSIDE any try, so a missing/broken SDK (the import is lazy precisely so the app runs without the package) would abort the whole job instead of degrading to empty results β contradicting the module's "always returns, pipeline can always proceed" contract that extract_one and extract_topics honor. Guard the shared-client construction: on failure, log and return empty results with progress marked complete. * /process (the legacy non-browser endpoint) lacked the "AI features requested but no API key" guard that /jobs and /jobs/from-images have, so a script POSTing llm_* toggles with no key got a 200 with every AI feature silently dropped. Add the same fail-fast 400. Not changed (recommendationβ¦0658e45 -
fix: security + bug findings from the new-feature auditβ
ClaudeAdversarial audit of the images-input and /edit features found four confirmed issues, all fixed: [HIGH bug] image_input.py timestamp regex fused a date/counter prefix onto the time. Delimiters were interchangeable, so "2024-01-15_00-05-30.png" parsed as 01:15:00 (the date's 01-15 fused with the time's leading 00), losing the real 00:05:30 β the slide landed ~70 min off, or the upload was rejected outright for a validly-named file. Now the two separators must be the SAME delimiter (backreference), so a mixed "-"/"_" date-time boundary can't match; matching runs inside a lookahead so overlapping candidates are all found, and the trailing in-range one wins. [HIGH security] /edit arbitrary server-file read via audio_filename path traversal. /edit renders untrusted JSON; the image guard required an inlined image_data_uri, but audio fell through to a disk read when audio_data_uri was null, and metadata_to_segment did Path(frames_dir)/audio_filename with no sanitisation β "../../../../etc/pβ¦1ca1018 -
feat: self-contained edits export + /edit page to reopen a shared JSONβ
ClaudeMakes the editor's JSON export a true handoff artifact: one file that someone else can open and keep editing, with no accompanying images and no live job. Self-contained export (review.html): * readState now embeds each segment's current image as a data URI (and audio when inlined), so "Save edits (JSON)" carries the slides, not just references to them. The primary <img> src is already a data URI in review mode (even after alt-switching), so it's captured verbatim; a non-data src is omitted rather than saved broken. * downloadZIP strips those embedded URIs before POSTing (it sends images separately + keeps audio server-side) so the ZIP request doesn't carry the image bytes twice. * applyState restores the <img> from image_data_uri when a self-contained JSON is loaded into a live editor, so a file from another session shows the right slides. Data model (metadata.py, html_gen.py): * SegmentMetadata gains optional image_data_uri / audio_data_uri; migration + round-tripβ¦89cae6c -
fix: flatten RGBA/palette PNGs before JPEG-encoding for the LLMβ
ClaudeLive test of the images-input mode surfaced "cannot write mode RGBA as JPEG" on every frame: PNG slide exports commonly carry an alpha channel (RGBA) or a palette with transparency, and _encode_image saved straight to JPEG, which has no alpha β so PIL raised and every per-segment LLM call failed, leaving descriptions/alt-text/OCR empty. Flatten onto a white background before encoding (transparent slide regions read as white, matching how a sighted viewer sees them), and convert any other non-RGB/L mode to RGB. Only affects the bytes sent to Claude; the displayed/embedded image still uses the original PNG via the data-URI path, so on-screen quality is unchanged. Tests (runnable without cv2): RGBA and palette-with-transparency PNGs both encode to a valid RGB JPEG instead of raising. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg13e0c0c -
feat: images-input UI β standalone /images upload pageβ
ClaudeAdds a self-contained upload page for the images-input mode, kept separate from the video form so the working video flow is untouched. /images page: * Multi-image file input + transcript input + title + optional API key. * AI toggles: section titles, scene description, alt-text, on-screen text, math equations. Topic segmentation is intentionally excluded β re-bucketing hand-picked slides into topics fights the point of curating exact slides (the endpoint's topic flags are always sent false from this page). * Output format picker (editor / single HTML / ZIP). * Submit posts to /jobs/from-images, subscribes to the same SSE progress stream, and on completion navigates to the editor (review) or downloads (single/zip) β reuses the existing job/events/result infrastructure end to end. * Accessible: skip link, labelled fields, aria-live progress panel, visible focus, theme-consistent styling. Clears the API key from the DOM after capture. Also: a cross-link on the main vidβ¦31ce41a -
feat: images-input mode β build a guide from timestamped slides + a transcriptβ
ClaudeNew input path: upload pre-exported slide images (each named with an HH-MM-SS timestamp, e.g. PNG exports pulled from a lecture at each slide change) plus one SRT/VTT, and get a study guide with no source video and no scene detection. Each image becomes a segment anchored at its timestamp; the transcript aligns to those anchors via the same downstream pipeline the video path uses. Why: deterministic (no seeking, no scene-merge threshold β sidesteps the run-to-run scene-count variance), no multi-GB video upload, and the author controls exactly which slides appear and when. Pieces: * app/pipeline/image_input.py β strict HH-MM-SS filename parser (tolerates extra metadata in the name; prefers the trailing time; uses the transcript duration to reject a date/resolution fragment mistaken for a time) and a frame builder that sorts by timestamp, windows each segment to the next slide (last β transcript end), and rejects empty sets, unparseable names, and duplicate timestamps LOUDLY β¦88d3d3b
-
fix: preview-cache lifecycle (recoverable retry + entry cap)β
ClaudeTwo preview-cache findings from the audit: main.py β recoverable selection errors destroyed the preview (finding 14): /jobs pops the preview token, then validates the selected indices. A 4xx there (bad/negative/non-int index, empty selection) ran the cleanup branch and rmtree'd the preview dir, so the user could not correct their input and retry β they had to re-run the topic preview, which costs an LLM call. Now a 4xx re-inserts the popped preview into the cache so a corrected retry reuses it; 5xx and non-HTTP errors still drop it. main.py β preview cache had no entry cap (finding 20): A flood of preview requests could pin an unbounded amount of video data on disk for the full 30-minute TTL. Add a hard cap (24 entries); _purge_stale_previews now evicts oldest-first past the cap after its TTL sweep, removing their temp dirs. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJgd2261c7 -
fix: lifecycle + resource-leak hardening from the auditβ
ClaudeRemaining confirmed findings, all lower severity: app/jobs.py β job TTL evicted live editors (finding 15): The 10-minute TTL measured from finished_at and was never refreshed by access, so a review session longer than 10 min got its job purged and the editor's lazy /alts, /ocr, /audio fetches and /result re-fetch started 404ing mid-edit. Track last_access_at (bumped on every get()) and purge on idle time = max(finished_at, last_access_at) instead. app/main.py β thread-start leak (finding 19): If the OS refused threading.Thread(...).start() after registry.create(), the job sat in the registry as "running" forever (finished_at stays None β TTL purge never reclaims it), permanently consuming a concurrency slot. Pop the job on a failed start before propagating. app/main.py β upload cap enforced too late (finding 6): The per-field streaming check in _save_upload runs only after Starlette has already spooled the full multipart body to disk. Add an early Content-Lenβ¦fbb54cc -
fix: crash + data-loss bugs from the audit (transcript parsing, download headers)β
ClaudeFour confirmed crash/data-loss findings: transcript.py: * IndexError when a cue's text starts with '.' β the sentence-boundary check did prefix.rsplit(None,1)[-1] on an empty prefix ([] β [-1] raises). A leading-period cue is common in auto-transcripts. Guard the empty-prefix case. * Hourless WebVTT timestamps (mm:ss.ttt β valid per spec, common for sub-hour clips) matched none of the HH:MM:SS-only regex and produced ZERO cues β the whole transcript silently ignored. Make the hours group optional; SRT (always hours) still matches. * Trailing cue whose midpoint lands at/after the computed tail edge (video duration undershot the final cue's end) matched no bucket window and was dropped from both narration text and sliced audio. Spill it into the last segment β symmetric to the existing head spill. main.py: * A title with non-latin-1 alphanumerics (CJK, emoji, some accented letters β _safe_filename keeps them because "δΈ".isalnum() is True) reached Content-Dispositiβ¦4f70b55 -
fix: persist and honor per-element include toggles (bug audit cluster 1+2)β
ClaudeAdversarial bug audit found the per-element include toggles (added this week) were DOM-only state that neither survived Save/Load JSON nor was honored by the server-side render path (Download ZIP, CLI --metadata). Result: a user who curated which blocks carry over got a ZIP that contradicted Download final HTML, and SaveβreopenβLoad silently reset every toggle to defaults. Also fixes an a11y regression where a duplicate contenteditable-strip made the ARIA-cleanup pass dead code. Persistence (review.html): * readState now serializes alt_text_included, on_screen_text_included, transcript_included, audio_included (description_included already existed), plus page lang and meta_lines (were dropped, resetting a non-English guide to lang=en and losing header metadata on ZIP re-render). * applyState restores all five per-element flags and lang via a shared applyElementInclude helper; a Load JSON refreshes the bulk-toggle indicators. * Description read/write now uses innerText (notβ¦9a3cbf9 -
scenes: deterministic frame extraction to stabilise scene detectionβ
ClaudeUsers reported the "Preview scenes" step returning a different set of scenes on every run of the same video. Root cause: representative-frame extraction used cap.set(CAP_PROP_POS_MSEC, β¦), OpenCV's approximate millisecond seek. That seek lands on the nearest keyframe and decodes forward, and FFmpeg's multithreaded decoder can return a slightly different frame for the same timestamp across runs. The jitter didn't just change thumbnails β it changed the scene COUNT. _merge_duplicate_adjacent compares each scene's extracted frame against its neighbour with a fixed mean-abs-diff threshold (5.0); when a seek returned a marginally different frame, the MAE crossed the threshold and flipped a merge decision, so scenes that collapsed one run stayed split the next. Fix: add _read_frame_at(cap, time_seconds, fps) which converts the timestamp to an integer frame index and seeks via CAP_PROP_POS_FRAMES β a specific decode target that maps far more reproducibly than a millisecond value. Both extraβ¦e059eb2
-
editor: Download Markdown β semantic .md exportβ
ClaudeAdds a "Download Markdown" button to the editor toolbar that produces a single self-contained .md file from the current editor state. Mirrors "Download final HTML"'s contract: same per-element include toggles, same description-replaces-alt rule, no server round-trip. Output is semantic markdown (no raw HTML hacks): * # for the page title, ## per section, --- between sections * Eyebrow as *italic*, subtitle as plain paragraph * Timestamps as *italic* sub-line under each heading * Image as  β alt is empty when a scene description is present (matches the published-HTML behaviour) or when the alt-text include checkbox is off * Scene description as plain paragraphs (blank-line separated) * On-screen text as a labelled blockquote (>) when its include toggle is on * Transcript as plain paragraphs when its include toggle is on * Math equations as $β¦$ (inline) or $$β¦$$ (block), GitHub-flavoured Markdown compatible Audio is intentionally skipped β markdoβ¦e363538
-
a11y pass for accessibility expert reviewβ
ClaudeAudit-and-fix pass on the published-guide and editor templates before sending it for an external accessibility review (Rich Caloggero, MIT DSL). Published guide (guide.html / _doc_styles.html): * Drop role="presentation" from decorative <img alt="">. alt="" alone is the WAI-recommended decorative-image pattern; role="presentation" on <img> has spotty AT support β some readers ignore it, others still announce the image. Empty alt is sufficient and more reliable. * Drop the redundant aria-label on <details class="onscreen-text">. The <summary> is the accessible name; layering aria-label on top either overrides the visible summary (WCAG 2.5.3 Label in Name mismatch) or duplicates it. Remove. * Only emit audio aria-describedby="narration-N" when the narration block it points at is actually rendered. A dangling describedby is an ARIA validity bug (WCAG 4.1.2 / 1.3.1) and some screen readers silently swallow the relationship, leaving the audio undescribed. * Bump TOC linkβ¦5a2d8c8 -
editor: make the include-set inline-row CSS actually winβ
ClaudeThe previous commit (15241a4) restyled the per-section include fieldset as a horizontal flex-wrap row, but the rules sat on a bare .vgm-include-set / .vgm-include-set__row selector β class-count 1. The panel's generic .vgm-edit-panel label { display: flex; flex- direction: column } rule has class-count 1 + type-count 1, so it beat my selector and silently stacked each checkbox on its own line with the label rendered below it. Scope the include-set selectors under .vgm-edit-panel and pin the row at label.vgm-include-set__row so they outrank the generic panel-label rule. Net specificity: 2 classes (+1 type on the row) vs the generic rule's 1 class + 1 type. No DOM change, pure CSS bump. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJgc25567f -
editor: collapse the per-section include set onto a single inline rowβ
ClaudeThe "Include in published guide" fieldset was rendering each of its five or six checkboxes as its own vertical row inside a bordered box, eating about 9rem of height in every segment. With 10+ segments in a typical study guide it dominated the editor scroll. Restyle the fieldset as an inline flex-wrap row: no border, no block padding, the legend sits inline as a small uppercase label, and each chip is a `display: inline-flex` with `white-space: nowrap` so a single chip never wraps mid-label. The whole control fits on one line on wide screens and wraps to two on narrow. Behaviour is unchanged β the DOM, the data-element-toggle attributes, the bulk-bar sync, and the data-include / data-include-alt-text plumbing all stay the same. Pure CSS change. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg15241a4 -
editor: drop the 'Include section' bulk toggleβ
ClaudeA global toggle for the per-section master include switch doesn't read as a meaningful bulk action β flipping it either does nothing (every section already included) or empties the published guide. Remove it from the bulk bar; the per-section checkbox stays. The bulk bar now only carries toggles for things that make sense to apply uniformly across all sections: Scene description, Alt-text, On-screen text, Transcript, Audio. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg007ae3b -
editor: clearer 'Include section' label; fix popup-blocked Previewβ
ClaudeTwo small editor fixes: * Rename "Whole section" β "Include section" in both the per-section fieldset and the new bulk-toggle bar. The old label read fine in the per-section context (the legend "Include in published guide" gave context) but in the bulk bar it sat next to "Apply to all sections:" and was confusing β "apply to all sections: whole section". The new wording stands on its own. * Preview button no longer trips the popup blocker. previewHTML was async and awaited inlineAllAudio() BEFORE calling window.open(); once you await inside a click handler the browser no longer treats the subsequent window.open as user-initiated, so Safari and Chrome both reject it. Open the tab synchronously at the top of the handler with about:blank, then navigate it to the blob URL once the HTML is ready. Errors close the pre-opened tab so we don't leave an orphan window. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJgb607027 -
editor: bulk include toggles that drive every section at onceβ
ClaudeAdds a "Apply to all sections" bar between the toolbar and the document header. One checkbox per include key β Whole section, Scene description, Alt-text, On-screen text, Transcript, Audio. Three behaviours: * Standard tri-state. The bar reflects the union of per-section state on page load: all-checked, all-unchecked, or indeterminate when sections disagree. * Bulk β per-section. Toggling a bulk box sets every matching per-section checkbox to the bulk's new state and dispatches a synthesised 'change' event on each, so the existing per-section sync (data-include, data-included, data-include-alt-text) fires for free. No duplicated wiring. * Per-section β bulk. Any individual checkbox change re-runs the union calculation so the bulk indicator stays accurate. Applies after Load JSON too (applyState calls syncBulkToggles when it finishes mutating per-section state). A bulk box for a key with zero matching per-section checkboxes (e.g. Audio when no segment carries an auβ¦93e4afe -
editor: per-element alt-text and transcript toggles; rename 'Scene description'β
ClaudeThree pieces of UX feedback: * Alt-text now has its own include checkbox in the per-section "Include in published guide" fieldset, default OFF. When off, buildFinalHTML blanks the <img alt> at download time. Because alt is an attribute (not a discrete element), the checkbox writes its state to data-include-alt-text on the section and buildFinalHTML reads it alongside the description-present check; the helper attribute is stripped from the final HTML. * Narration sections are now rendered as <details class="narration"> with summary "Transcript", collapsed by default β in both the editor view and the server-side render (guide.html). A matching per-element include checkbox sits in the fieldset, also default OFF, so transcripts no longer carry over to the published guide unless the editor explicitly opts in. The narration body keeps its stable id so any audio aria-describedby reference still resolves; when transcript is excluded the strip pass collects removedβ¦0c2c651 -
editor: stop emitting the alt-review figcaption when description carries the meaningβ
ClaudeTwo paths were leaking the "Alt-text needs author review." caption into the final HTML even when it no longer applied: * review.html rendered the figcaption purely on `seg.alt_text_reviewed`, so a segment with a populated description (alt is intentionally empty, not pending) still got the warning. Add `not seg.description` to the conditional so the editor view and the server-side review render both stay quiet in that case. * buildFinalHTML blanked the <img alt> when a description was present but never touched the figcaption next to it. Now the same per-section pass also strips figure.figure > figcaption when either (a) the description is present-and-included, or (b) the user edited alt-text in this session (data-alt-reviewed flipped to 'true'). Both cases mean the "needs review" note is stale. Adds a regression test confirming the caption disappears once `seg.description` is set, regardless of `alt_text_reviewed`. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJβ¦9eb147c
-
editor: remove summary / key-terms / self-check (round 3)β
ClaudeAccessibility team's round-three asks: * "Summary Field β redundant, remove" (now subsumed by the new detailed description paragraph from PR 2) * "Key Terms β REMOVE. This replicates the glossary." * "Self-check questions β REMOVE! This is an academic role and such questions are better placed in the platform." Scope of removal: * Editor (review.html): drops the summary <input>, the key-terms list editor block, the self-check-questions list editor block, the related CSS rules (.vgm-key-term-row, .vgm-self-check-row), the readState/applyState/syncBindings hooks, the makeKeyTermRow / makeSelfCheckRow row factories, the add-key-term / add-self-check click handlers, and the entire glossary/highlight injection inside buildFinalHTML. * Published guide (guide.html): drops the "In brief:" advance organizer, the self-check <details> block, the bottom glossary section, the highlight_terms filter usage on narration, and the build_glossary call at the top of <main>. * LLM (llβ¦b863381 -
editor: long-form image description field (round 2)β
ClaudeAdds the "Detailed description" field the accessibility team asked for β an editable open paragraph rendered UNDER the slide image that doubles as the long-form image description. When non-empty AND included in the published guide, the <img alt> drops to "" and the image is marked role="presentation": the description carries the meaning in body text where sighted AND non-sighted readers both encounter it, so there's no need to duplicate it inside an alt attribute that only screen readers hear. The pieces: * SegmentMetadata + Segment + migration gain description / description_ reviewed fields. Round-trips through Save/Load JSON. * New LLMToggles.description toggle. _FIELD_PROMPTS adds a description prompt that follows the team's example ("A gallery of purple flower stem macro photographs is shown. These demonstrate..."), and the structured-output schema gains a matching description field that only fires when the toggle is on (no wasted output tokens otherwise). * Orchestraβ¦cf53b59 -
editor: accessibility-team UX pass (round 1)β
ClaudeRound-one feedback from the accessibility team, focused on changes that don't alter the data model or remove fields: * Banner trimmed β three sentences down to one. * Segment image moved above the editable fields so the editor sees what they're describing before they describe it. * Alt-text field is now a multi-row textarea, not a single-line input. Some descriptions take two paragraphs and the old input made that impossible without scrolling a clipped one-liner. * New per-element "Include in published guide" fieldset replaces the single per-section checkbox. The whole-section checkbox stays. Two new checkboxes: - On-screen text from video (defaults to included) - Narration audio clip (defaults to EXCLUDED β per the team, audio shouldn't carry over to the published guide by default) Each checkbox drives data-include on a matching [data-element=β¦] node; buildFinalHTML now strips anything still flagged false, plus the data-include / data-element attributes β¦3672e55
-
transcript: spill pre-first-boundary cues into the first segmentβ
ClaudeDeselecting the first topic in the topic-preview flow left the new first kept segment starting well after the transcript did (e.g. the surviving boundaries become [180s, ...] for a transcript that began at 0s). _bucket_cues only matched cues whose midpoint fell inside some [edges[i], edges[i+1]) window, so the opening narration was silently dropped β several minutes of speech vanishing with no error. Spill cues with midpoint < edges[0] into bucket 0 instead. The audio slicer (cue_bounds_per_segment) shares the same bucketing, so its first segment's bounds extend backwards in lockstep and the dropped narration is recovered in audio too. In the common case where the transcript begins at the same time as the first kept segment, this is a no-op. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJgf16b33f -
llm_extract: cut per-segment API cost and add transient-error retryβ
ClaudeThree small efficiency wins on the Claude calls that dominate job cost: * Drop the Tesseract OCR block from the user message when the LLM is itself producing OCR (toggles.ocr_text). The block was being sent only to be ignored β ~50-300 wasted input tokens per segment, scaled with slide text density. * Build one Anthropic client per extract_all run and reuse it across all worker threads, instead of constructing a fresh client (and TCP+ TLS handshake) per segment. * Warm the prompt cache by firing the first segment synchronously before parallelising the rest. Previously the first wave of 4 worker threads raced for the cache write and only one won, so cache hits started at segment 5; now they start at segment 2. Also wraps both extract_one and extract_topics in a single-retry helper that catches RateLimitError / APIConnectionError / 5xx-overloaded statuses, so a transient blip no longer permanently drops a segment's LLM metadata. 4xx are not retried. https://claude.aiβ¦3602e87 -
Accessibility: drop prohibited aria-labelledby from no-role narration divβ
ClaudeAxe-core's wcag2aa+best-practice scan against a freshly-rendered guide returned 0 violations but flagged the narration__body div's aria-labelledby="narration-label-N" as an incomplete (potentially prohibited) ARIA attribute. <div> has no implicit role, so the attribute has no name slot to fill and many AT will ignore it anyway β the sr-only <h3>Narration</h3> sibling already labels the block via standard heading semantics. Drop aria-labelledby from both guide.html and review.html. Keep the id on the inner div so the section's <audio> element's aria-describedby can still point at the narration text. Re-running axe after the change: 0 violations, 0 incomplete, 43 passes. WCAG 2.2 AA clean on the rendered guide.3c265ff -
Accessibility: term tooltips become real <a> links to glossary entriesβ
ClaudeThe narration's inline highlighter wrapped each defined term in a <mark class="kt" title="β¦"> with the definition only in the title attribute. WCAG 2.5.1 / 4.1.2 β touch users can't trigger hover, keyboard users can't focus a <mark>, many screen readers skip the title attribute entirely. Definitions were effectively unavailable to roughly half of likely users. Build the glossary once at the top of <main> and pass it to the highlighter so each first-occurrence wrap becomes <a class="kt" href="#glossary-N" title="β¦">term</a> β focusable, touch-tappable, screen-reader announced as a link to the named target. The bottom glossary's <dt> ids match the hrefs the highlighter emits (build_glossary now stamps each entry with its own `id` field so server template and JS port share the contract). Terms with no glossary entry (defensive) fall back to the old <mark> shape. Mirror in buildFinalHTML: build the glossary entries before walking sections, thread the lookup through highlightTerms, reusβ¦c28a145 -
Cleaner study-guide output: drop redundant per-section key-term listβ
ClaudeUser feedback: with inline term highlighting AND a bottom glossary, showing the same vocabulary again as a per-section <dl class="key-term"> was triple-redundant and visually noisy. - Drop the per-section <dl class="key-term"> from guide.html. Definitions remain accessible two ways: - inline via the .kt highlighter (hover tooltip on the first occurrence in each paragraph) - at the end of the document via the auto-built glossary (alphabetised, back-anchored to source section). - Mirror in buildFinalHTML so downloaded HTML matches. - Update test_render_guide_renders_key_terms_and_equations to expect the glossary instead of the per-section list. Also cleaned up two other visually-noisy bits while in the same file: - Figure's "Lecture slide." caption: redundant with the section heading + meta-time line and the figure-internal timestamp badge. Now we only emit a figcaption when there's something real to say ("Alt-text needs author rβ¦8fb4d34 -
Third-pass audit fixes: migration setdefaults, robust JS mark builderβ
ClaudeP1 β migrate_v1_to_v2 didn't setdefault the new learner-feature fields (summary, self_check_questions). Pydantic still supplied defaults on model_validate so no data was lost at load time, but the migration's raw-dict output was asymmetric β any consumer of the migrated dict pre-validation would see undefined keys for those two fields and not for the others. Adds the two setdefaults so the migration's contract stays "every SegmentMetadata field has a defined key after migration". P2 β JS highlightTerms built its <mark> open tag via template-string interpolation: `<mark class="kt" title="${escapeHtml(definition)}">`. The contemporaneous escapeHtml correctly entity-encodes quotes, so no XSS materialized β but the .replace() second argument treats $&, $', $`, $n as special replacement patterns. Any `$` surviving escapeHtml (it doesn't escape `$`) would get mangled in the output title. Switch to building the mark via document.createElement + setAttribute, then serializing with outerHTML tβ¦7562488 -
Topic-preview resume: also forward summaries, not just titlesβ
ClaudeThe topic-scene-preview endpoint computes Claude's per-topic summaries and stores them in the preview cache alongside titles, but /jobs only read out and forwarded titles. On the resume path the orchestrator's topic_summaries list stayed empty, so the "In brief:" advance-organizer line never appeared in the final guide β even though the editor showed the summary value correctly in the per-segment edit panel (because the preview's initial render came straight from the cache). Forward summaries through the same previewβ/jobsβworkerβrun_pipeline seam as titles: - Read topic_summaries from the preview cache entry, filter by the user's curated indices, store as preselected_topic_summaries. - Thread it through _run_job_in_thread β run_pipeline as a new optional kwarg, mirroring preselected_topic_titles. - run_pipeline pre-populates its topic_summaries local from preselected_topic_summaries so the existing apply loop fires without needing a re-segmentation.0487cd2 -
Editor downloadHTML: emit summary, self-check, glossary, highlightsβ
ClaudeThe editor's "Download final HTML" path doesn't go through the server-side Jinja templates β it builds the final document from the editor's live DOM via buildFinalHTML(). I taught the server template (guide.html) about the four learner features but forgot to teach buildFinalHTML the same vocabulary, so the downloaded file came out with key terms and math equations but no summary, no self-check block, no inline term highlights, and no glossary. Extend buildFinalHTML to inject all four: - Summary line (.section__summary "In brief: β¦") inserted right after each section's <p class="meta-time">. - Inline highlighting of defined terms in narration paragraphs via a JS port of the server-side _highlight_terms filter (longest-first, one mark per term per paragraph, skips text already inside a <mark> so wraps never nest). - Self-check questions as a <details class="self-check"> block at the end of each section, matching guide.html's structure. - Auto-built glossary β¦327d32d -
Update review.html
60c7af8 -
Learner-focused features: summary, self-check, glossary, inline term highlightingβ
ClaudeTagged pre-learner-features at the commit before this so a single `git reset --hard pre-learner-features` (or non-destructive `git revert pre-learner-features..HEAD`) takes us back to known-good if anything misbehaves. Four learner-impact features ship together because they share data plumbing β same Segment/Metadata schema additions, same editor round-trip work: #2 One-sentence summary above each section. Claude's `summary` field from topic segmentation is now rendered as an "In brief:" advance organizer above the slide image. Primes the learner before they hit the detail; evidence-based for comprehension. Editable in the review screen. #4 Auto-built glossary at the end of the guide. Collects every segment's key_terms across the whole document, dedupes case- insensitively (first-occurrence wins), sorts alphabetically, and anchors each entry back to the section where the term was first introduced. #5 Inline key-term highlighting in narration. Defined terms wβ¦d61b118 -
UX backlog: flatten sidebar, compact output picker, jargon tooltips, sample panelβ
Claude#12 Flatten the Advanced settings sidebar. The "More settings" inner collapsible hid three controls behind a second click for no real benefit β sidebar is short enough to show all seven at once. Drop the toggle button, its CSS, and the JS that wired it. #6 Move the four-radio output-format block into the submit row as a small "Output format" dropdown. 90% of users leave it at "Review & edit"; promoting a top-level section for it was over- weight. The dropdown sits next to the Generate button so the choice stays one click away without consuming form space. #11 Add a `.vgm-help` "?" badge for the genuinely-technical labels in the sidebar (Scene-change sensitivity, Min gap, Instructor-frame threshold, Document language). The badge is keyboard-focusable (tabindex=0), reads via aria-label to screen readers, and shows the expanded explanation on hover via the title attribute β no JS needed. Friendlier copy on the transcript hint ("Cue tiβ¦4031b6c -
Second-pass audit fixes: narrow exception catch, empty-cue early errorβ
ClaudeTwo real findings from a second-pass bug/security audit; the rest of the report was either already handled by existing code (sub-toggle sync runs on dispatched change event) or correctly identified as acceptable / non-bugs. P1 β except BaseException in the /jobs preview-cleanup block was overly broad. Narrow to Exception so KeyboardInterrupt / SystemExit still propagate without first running cleanup that could mask the signal. P2 β Empty transcript silently produced a guide with no narration under any segment. Both topic and non-topic paths now check `if not cues` immediately after _load_cues and raise PipelineError with a message naming the fix ("provide a non-empty VTT/SRT or enable auto-transcribe"). The topic-scene-preview endpoint already had its own empty-cue check; this brings the main pipeline to parity. Verified safe (no changes needed): - filename validation: endpoint regexes already cover /jobs/{id}/zip, /alts, /ocr, /audio - autoescape: Jinja2 select_autoescape([β¦e8ba7af -
P1/P2 bug audit fixes: index validation, cleanup race, cache purge, type whitelistβ
ClaudeFour real issues from a thorough audit of the recent feature work: P1 β Preview workdir leaked on validation failure. Between popping the preview cache entry in /jobs and entering the workdir try/finally, any HTTPException raised by index parsing or selection filtering would orphan the temp dir holding the user's uploaded video. Wrap the post-pop validation in its own try/except that cleans up the popped entry before re-raising. Marks the entry None so the outer finally doesn't double-delete. P1 β Out-of-range and negative selected_indices silently dropped. The previous parser used `if i.strip().isdigit()` which (a) accepts any positive integer regardless of frame count and (b) silently ignores negative numbers and any non-digit token. A user submitting "0,5,999" against a 5-frame preview got a 5-frame guide back with no error. Now we parse each token explicitly: ValueError on non-int, 400 on negative, 400 on out-of-range against the cached preview's valid index set (positional for tβ¦1361034 -
Stitcher: add lowercase-continuation fallback for transcripts without periodsβ
ClaudeUser report: a topic boundary fell mid-sentence on a lecture whose Whisper transcript omits terminal punctuation entirely. The stitcher's terminator-split strategy needs at least one .!? somewhere in B's first cues β when none exists, it gave up and the cut stayed. Add a second strategy that uses casing as the signal: if B's first cue starts with a lowercase letter, it's almost certainly continuing A's open sentence. Absorb consecutive lowercase-starting cues into A until we hit one that starts with a capital (next sentence begins) or the existing _MAX_SENTENCE_ABSORB_CUES cap. Triggers only after the terminator strategy has been tried and failed, so behaviour on punctuated transcripts is unchanged. Guarded edge case: when B opens with a capital letter, we don't move cues β the mid-sentence cut is on A's side (transcript dropped a period) and pulling B-cues into A would just mash two real sentences together. Two tests pin both branches; the heat-pump "how to install / and maintain" sβ¦39c2e46
-
Demote non-LLM scene picker; AI topic preview is now the primary CTAβ
ClaudeThe two-button row read as "pick one of two equivalent paths" but they aren't β the AI path produces meaningfully better study guides when an API key is available. Promote the topic-preview button to the primary CTA the moment a key arrives, and demote the plain "Preview scenes" button to a small text-link beneath it ("or pick slides individually without AI"). When no key is present, hide the AI button entirely and let the plain picker stand alone as the primary CTA β no dead "disabled because key missing" state. The non-LLM path stays fully functional and intentionally one click away, so users on the public Space without a key still get the demo; users worried about privacy / cost / outages keep their escape hatch; the CLI is unaffected. Backend: /jobs now rejects up-front when any AI toggle is ticked without an API key. Previously per-segment toggles silently no-op'd and topic segmentation hard-failed halfway through; now the user gets a single explicit "add a key or uncheck AI feaβ¦908a4e1 -
Topic prompt: video-type dropdown biases boundaries to genre structureβ
ClaudeGeneric topic segmentation works but produces splits that ignore genre conventions β a lab walkthrough is sliced like a lecture, a tutorial like a panel. Add a per-job "video type" hint the user can pick from a small dropdown; the orchestrator prepends a paragraph of structural guidance to the topic prompt so Claude knows what shape to bias toward. Five genres modeled today (each one paragraph of prompt): - Classroom lecture: intro β background β concept(s) β examples β wrap-up - Lab walkthrough: intro β theory β materials β procedure β results - How-to tutorial: intro β setup β steps β verification β troubleshooting - Interview / Q&A: boundaries at new questions or subject pivots - Talk / presentation: hook β thesis β supporting points β closer plus "General" which leaves the prompt untouched. Unknown values fall back to general so a stale form post can't break the call. Wired through extract_topics(video_type=...), LLMToggles .topic_video_type, the orchestrator, and both β¦6dd9ac4 -
Auto-enable topic segmentation when an API key is presentβ
ClaudeTopic segmentation is the standout feature for users with an API key but it sat as an opt-in toggle three groups deep in the form. Most people never discovered it and walked away from a per-slide guide thinking that was the tool's default story β when the JamWorks-style topical breakdown they actually wanted was one checkbox away. Auto-check the toggle the first time the API key field gains a value. Pair the flip with a soft maroon banner under the section heading ("Auto-enabled. Claude will group the slides into topical sectionsβ¦") so the change is visible and the opt-out path is named. The first manual toggle (in either direction) sets a userTouched flag on the checkbox; from then on we never auto-flip again, even if the key is later removed and re-entered β the user's explicit choice always wins. Removing the key with no manual touch unchecks again so the form's submitted state stays consistent with the key state.f759553