Commit History
-
chore: add attachments/ to gitignore
f6eb4b4 -
[2026-01-21] [Documentation] [COMPLETED] ACHIEVEMENT.md - Project Success Report
Created comprehensive achievement report with 7 strategic engineering decisions: - Design-first approach (8-level framework) - Tech stack selection (LangGraph, Gradio, multi-provider LLM) - Free-tier-first cost architecture (96% cost reduction) - UI-driven runtime configuration - Unified fallback pattern - Evidence-based state design - Dynamic planning via LLM Tech Stack Details: - LLM Chain: Gemini 2.0 Flash Exp → GPT-OSS 120B (HF) → GPT-OSS 120B (Groq) → Claude Sonnet 4.5 - Vision: Gemma-3-27B (HF) → Gemini 2.0 Flash → Claude Sonnet 4.5 - Search: Tavily → Exa - Audio: Whisper Small with ZeroGPU Note: Images referenced in ACHIEVEMENT.md are in local attachments/ folder (not tracked in git) Modified Files: - ACHIEVEMENT.md (401 lines) - Project success report with strategic decisions - CHANGELOG.md - Added ACHIEVEMENT.md entry with full tech stack details - PLAN.md, WORKSPACE.md - Session workspace updates - app.py - Application code updates Co-Authored-By: Claude <noreply@anthropi…3b2e582
-
fix: correct author name formatting in multiple files
e7b4937
-
Enhance YouTube video processing with transcript and frame analysis modes
- Updated logging standard to use Markdown format for session logs. - Modified `run_and_submit_all` to include a new `video_mode` parameter for selecting YouTube processing mode (Transcript or Frames). - Removed obsolete brainstorming document for YouTube transcript support. - Added OpenCV and other dependencies for frame extraction in `pyproject.toml` and `requirements.txt`. - Refactored `llm_client.py` to log session details in Markdown format. - Implemented `youtube.py` to support both transcript extraction and frame analysis, with appropriate logging and error handling. - Updated tool descriptions to reflect new functionality for analyzing video frames. - Added backward compatibility for the `youtube_transcript` function to respect the `YOUTUBE_MODE` environment variable.f1b095a
-
refactor: update folder structure and adjust output paths
3dcf523 -
refactor: implement 3-tier folder naming convention
Implement clear 3-tier naming convention to distinguish user-only, runtime, and application folders. 3-Tier Convention: 1. User-only (user_* prefix): Manual use, not app runtime - user_input/ - Testing files, manual data input - user_output/ - Downloaded results, exports - user_dev/ - Dev records, problem documentation - user_archive/ - Archived code/reference materials 2. Runtime/Internal (_ prefix): App creates, temporary - _cache/ - Runtime cache, served via app download - _log/ - Runtime logs, debugging traces 3. Application (no prefix): Permanent code - src/, test/, docs/ - Application code and tests Folders renamed: - _input/ → user_input/ - _output/ → user_output/ - dev/ → user_dev/ - archive/ → user_archive/ Updated files: - test/test_phase0_hf_vision_api.py - Path("_output") → Path("user_output") - .gitignore - Updated folder references and added comments - CHANGELOG.md - Added 3-tier convention entry - ~/.claude/CLAUDE.md - Updated Project Structure …bcc49f8 -
chore: ignore entire docs/ folder (not in git)
12ba14c -
refactor: move project template to ref/ folder
Move project_template_original/ to ref/ for better organization. Changes: - Moved project_template_original/ → ref/project_template_original/ - Updated CHANGELOG.md path references - Added ref/ to .gitignore (reference materials, not in git) - Added docs/*.pdf to .gitignore (HF doesn't allow large binaries) - Removed old project_template_original/ from git tracking Rationale: ref/ folder contains static reference materials, separate from active code. Co-Authored-By: Claude <noreply@anthropic.com>2e3c815 -
refactor: rename runtime folders with underscore prefix
Rename folders to follow Python convention for internal/private: - log/ → _log/ (runtime logs, debugging) - output/ → _output/ (runtime results, user downloads) - input/ → _input/ (user testing files, not app input) Rationale: _ prefix signals "internal, temporary, not part of public API" Updated files: - src/agent/llm_client.py - Path("log") → Path("_log") - src/tools/youtube.py - Path("log") → Path("_log") - test/test_phase0_hf_vision_api.py - Path("output") → Path("_output") - .gitignore - Updated folder references (log/, input/ → _log/, _input/, added _output/) - CHANGELOG.md - Added infrastructure entry Co-Authored-By: Claude <noreply@anthropic.com>04ece4c -
chore: folder rename and changelog formatting
- Rename _template_original/ to project_template_original/ for clarity - CHANGELOG.md formatting adjustments Co-Authored-By: Claude <noreply@anthropic.com>8eacd1b -
feat: consolidate LLM logs to single session file
Implement session-level log file to avoid polluting log/ folder with 20+ files per evaluation. Changes: - Added session log management (get_session_log_file, reset_session_log) - Changed from per-question logs to per-session logs - New format: log/llm_session_YYYYMMDD_HHMMSS.txt - All questions append to single file with QUESTION START/END markers Also added milestone entry: 30% target achieved! Phase 1 (YouTube + Audio) fixed 4 questions. Co-Authored-By: Claude <noreply@anthropic.com>a0fa418 -
feat: add Chain of Thought for LLM synthesis debugging
Implement CoT format to expose LLM reasoning process for debugging synthesis failures. Changes: - Updated system_prompt for all 3 providers (HF, Groq, Claude) to request REASONING + FINAL ANSWER format - Increased max_tokens from 256 to 1024 to accommodate reasoning - Added parsing logic to extract FINAL ANSWER from response - Enhanced log file format to save full response with reasoning - Updated CHANGELOG.md Result: Can now inspect LLM's thought process in log/llm_context_*.txt files Co-Authored-By: Claude <noreply@anthropic.com>c64957b -
fix: update logging reduction with actual measurement
16k tokens → 6.7k tokens (58% reduction, not 80% as estimated)9ce1b76 -
docs: add logging standard to CHANGELOG
Document console + file separation, 80% token reduction. Global rule update: log/ added to standard project structure. Co-Authored-By: Claude <noreply@anthropic.com>dfa0e6e -
fix: close parenthesis in execute_node error handling
Co-Authored-By: Claude <noreply@anthropic.com>9c1dff0 -
feat: standardize logging with console + file separation
- Console: Compressed status workflow ([plan] ✓, [execute] ✓, [answer] ✓) - Files: log/ folder with full LLM context and transcripts - Update CLAUDE.md with logging standard - Add log/ to .gitignore Separates status updates (console) from debugging details (log/ files). Co-Authored-By: Claude <noreply@anthropic.com>fa82033 -
refactor: compress console logs for clarity
Console logs now show status only: - [plan] ✓ 660 chars - [execute] 1 tool(s) selected - [1/1] youtube_transcript ✓ - [execute] 1 tools, 1 evidence - [answer] ✓ 3 Full context saved to _cache/llm_context_*.txt for debugging. Co-Authored-By: Claude <noreply@anthropic.com>fed2514 -
refactor: suppress noisy third-party logs
Set httpx, urllib3, huggingface_hub, gradio to WARNING level. Reduces log noise from HTTP requests while keeping errors visible. Co-Authored-By: Claude <noreply@anthropic.com>751698a -
feat: export LLM synthesis context to cache for debugging
Save full LLM context to _cache/llm_context_TIMESTAMP.txt: - System prompt - User prompt with evidence - Raw evidence items - LLM's final answer Helps debug why LLM fails despite having correct evidence. Co-Authored-By: Claude <noreply@anthropic.com>86bc1db -
debug: log full transcript content for debugging
Add comprehensive logging to see exact LLM context during synthesis: - Log full evidence content in answer_node (before synthesis) - Log full system + user prompts in synthesize_answer_hf - Helps debug why LLM says "Unable to answer" despite having transcript Co-Authored-By: Claude <noreply@anthropic.com>4496081 -
feat: add transcript caching and upgrade synthesis model
- Add save_transcript_to_cache() to save transcripts for debugging - Upgrade LLM: Qwen 2.5 → openai/gpt-oss-120b (Scaleway) - Document HF provider suffix behavior (auto-routing is bad practice) - Model iteration: Qwen 2.5 → Llama 3.3 → gpt-oss-120b Co-Authored-By: Claude <noreply@anthropic.com>b68b317 -
debug: log full transcript content for debugging
Added logging of full transcript text to inspect why AI fails to extract answer. Logs will show first 200 chars for API, full transcript for Whisper fallback. Co-Authored-By: Claude <noreply@anthropic.com>a1d5de5 -
fix: vision tool cannot process videos, clarify image-only capability
Removed misleading statement about video files. Vision tool ONLY processes image files (JPEG, PNG, GIF, BMP, WebP), not videos. Changes: - Changed "Analyze images or videos" to "Analyze image files" - Added explicit image format list (JPEG, PNG, GIF, BMP, WebP) - Removed "For video files...file must be downloaded first" - Added "Cannot process videos" NOTE - Added guidance to use youtube_transcript/transcribe_audio for videos Co-Authored-By: Claude <noreply@anthropic.com>3ee1e2c -
fix: clarify tool descriptions to prevent LLM selecting vision for YouTube
Problem: LLM was selecting vision tool for YouTube URLs, which cannot process them directly because vision description mentioned "YouTube links". Changes: - Vision: Removed "videos, YouTube links" from description - Vision: Added NOTE about using youtube_transcript for YouTube - YouTube: Enhanced description to be more explicit about being FIRST choice - YouTube: Added "ONLY tool that can process YouTube URLs directly" This should guide LLM to select youtube_transcript for YouTube URLs. Co-Authored-By: Claude <noreply@anthropic.com>61ecfdb -
fix: typo WISPER_MODEL -> WHISPER_MODEL in audio.py
Typo caused NameError during Whisper transcription. Fixed by correcting variable name reference. Co-Authored-By: Claude <noreply@anthropic.com>7bb1d7c -
fix: YouTubeTranscriptApi.fetch is instance method not class method
The API changed: fetch() requires an instance. Added: api = YouTubeTranscriptApi() before calling api.fetch() Co-Authored-By: Claude <noreply@anthropic.com>92346a5 -
fix: youtube transcript API and missing datasets dependency
- Changed YouTubeTranscriptApi.get_transcript to .fetch (API changed) - Added datasets>=4.4.0 to pyproject.toml (was missing) Co-Authored-By: Claude <noreply@anthropic.com>a170083 -
fix: remove duplicate huggingface-hub, update tenacity version
- Removed duplicate huggingface-hub==1.2.3 (already have >=0.26.0) - Updated tenacity from >=8.2.0 to >=9.1.2 (match pyproject.toml) Co-Authored-By: Claude <noreply@anthropic.com>c9e941b -
fix: correct google-generativeai package name
Wrong package name was causing ModuleNotFoundError. Changed: google-genai → google-generativeai Co-Authored-By: Claude <noreply@anthropic.com>fdd1cbb -
deps: add audio/video processing to pyproject.toml
Added Phase 1 dependencies to pyproject.toml for uv sync: - youtube-transcript-api>=0.6.0 - openai-whisper>=20231117 - yt-dlp>=2024.0.0 Co-Authored-By: Claude <noreply@anthropic.com>7a850db -
ui: move Test & Debug tab after Full Evaluation
Test & Debug is rarely used, so Full Evaluation is now the first tab. Tab order: - Tab 1: Full Evaluation (primary functionality) - Tab 2: Test & Debug (debugging/diagnostics) Co-Authored-By: Claude <noreply@anthropic.com>c8247b8 -
fix: add youtube_transcript and transcribe_audio to TOOL_FUNCTIONS
Bug: Tools were registered in TOOLS dict but missing from TOOL_FUNCTIONS mapping in graph.py, causing "Tool 'youtube_transcript' not found in TOOL_FUNCTIONS" error. Fixed: - Added imports: youtube_transcript, transcribe_audio - Updated TOOL_FUNCTIONS dict in execute_node (line 268) - Updated TOOL_FUNCTIONS dict in fallback section (line 410) Co-Authored-By: Claude <noreply@anthropic.com>40db96d -
docs: update CHANGELOG for Phase 1 completion
Added Phase 1 entry documenting: - YouTube transcript + Whisper audio transcription implementation - ZeroGPU @spaces.GPU requirement - Expected impact: 10% → 40% score Co-Authored-By: Claude <noreply@anthropic.com>7b6de93 -
feat: Phase 1 - YouTube transcript + Whisper audio transcription
Implementation complete for Phase 1 (YouTube support): New tools: - src/tools/audio.py: Whisper transcription with @spaces.GPU decorator * ZeroGPU acceleration (5-10x speedup vs CPU) * Supports MP3, WAV, M4A, OGG, FLAC, AAC * Model caching for efficient repeated use * Unified tool for Phase 1 (YouTube fallback) and Phase 2 (MP3 files) - src/tools/youtube.py: YouTube transcript extraction with audio fallback * Primary: youtube-transcript-api (instant, 1-3 seconds) * Fallback: yt-dlp audio extraction + Whisper (30s-2min) * Handles youtube.com, youtu.be, shorts URLs * Returns clean transcript for LLM analysis Updated files: - requirements.txt: Added youtube-transcript-api, openai-whisper, yt-dlp - src/tools/__init__.py: Registered youtube_transcript and transcribe_audio tools - brainstorming_phase1_youtube.md: Documented ZeroGPU requirement, validation results Architecture: YouTube URL → youtube-transcript-api (fast) → fallback: yt-dlp + Whisper (slow) Expected impact…38cc8e4 -
feat: phase1 planning and video processing research
- Rewrite PLAN.md: focus on system error fixes (current 10% → 30% target) - Add brainstorming_phase1_youtube.md: YouTube transcript approach research - Add _template_original/: static reference for comparison - Update CHANGELOG.md: course test setup analysis, 20 fixed questions documented - Research findings: * Transcript-first approach: 1-3s vs frame extraction 1-10min total * Frame extraction is fast (5-20s), vision processing is slow * Whisper open-source on ZeroGPU: 5-10x speedup, free * Unified Phase 1+2 architecture: single transcribe_audio() function Co-Authored-By: Claude <noreply@anthropic.com>0d77f39
-
Update README.md
Verified signatureUpdate Project Title2577d6f -
feat: system error field, calculator fix, target task IDs, course vs GAIA docs
Changes: - System error field: Changed to boolean yes/no with error_log - Calculator threading fix: Handle signal.alarm() failure in threads - Target task IDs: Debug feature to run specific questions - Course vs GAIA: Documentation distinguishing course API from official GAIA - Quick test script: test/test_quick_fixes.py for targeted testing Modified: - app.py: System error field, target task IDs UI, submission logic - src/tools/calculator.py: Thread-safe timeout handling - src/agent/graph.py: Evidence formatting for dict results - src/agent/llm_client.py: Fallback mechanism archived - CHANGELOG.md: All changes documented - README.md: Added submission guide reference Added: - docs/gaia_submission_guide.md: Complete submission guide - test/test_quick_fixes.py: Targeted question testing Co-Authored-By: Claude <noreply@anthropic.com>41ac444 -
Change UI default LLM provider from Groq to HuggingFace
Both Test Question and Full Evaluation tabs now default to HuggingFace. Co-Authored-By: Claude <noreply@anthropic.com>7d0cc73 -
Add LLM_PROVIDER to .env.example
Clarify where to set provider for vision tool routing. Co-Authored-By: Claude <noreply@anthropic.com>7e5d63e -
Phase 0-2: HF Vision Integration Complete - Google Gemma 3 Selected
Phase 0 (Extended Testing): - Tested 4 additional user-requested models - Validated: google/gemma-3-27b-it:scaleway (6s, RECOMMENDED) - Validated: zai-org/GLM-4.6V-Flash:zai-org (16s) - Validated: Qwen/Qwen3-VL-30B-A3B-Instruct:novita (14s) - Failed: GLM-4.7, gpt-oss-120b (text-only models) Phase 1 (Implementation): - Added analyze_image_hf() in src/tools/vision.py - Fixed analyze_image() routing to respect LLM_PROVIDER - Added HF_TOKEN, HF_VISION_MODEL to settings - Each provider fails independently (no fallback chains) Phase 2 (Smoke Tests): - Created test/test_smoke_hf_vision.py - Smoke test PASSED: red square image correctly identified - Fixed Settings.hf_token integration - Removed unsupported timeout parameter Modified Files: - src/tools/vision.py: +120 lines (HF vision function + routing fix) - src/config/settings.py: +5 lines (HF config) - .env.example: +7 lines (HF_TOKEN, HF_VISION_MODEL docs) - CHANGELOG.md: +93 lines (Phase 0-2 documentation) - PLAN.md: +20 lines (Phase 0…2bb2d3d
-
Remove binary test images and update .gitignore
c86df49 -
Add initial implementation for Phase 0 validation of HF Inference API with vision models
- Introduced a new test script `test_phase0_hf_vision_api.py` to validate multimodal support for vision models. - Implemented functions to encode images to base64, test models with base64 and file path inputs, and handle OCR models. - Configured logging for detailed output during testing. - Added a sample test image `test_image_red_square.jpg` for validation purposes. - Established a decision gate to proceed to Phase 1 based on test results.630f609
-
Phase 0 Prep: Cache restructure, Vision investigation, HF integration plan
Changes: - Unified export location to _cache/ folder (all environments) - Cleaned up obsolete exports/ files - PLAN.md: Comprehensive HF vision integration plan with Phase 0 API validation - CHANGELOG: Documented Stage 5 investigation findings (0% accuracy root cause) - README: Updated project status and workflow documentation Key findings: - Vision tool hardcoded to Gemini→Claude, ignores UI LLM selection - HF Inference API vision support needs validation before implementation - Added decision gates and backup strategies to PLAN.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>9fb579f
-
Update Dev
d93842c -
Update
87c4c82 -
Update
5731c0c -
Update
24396e3 -
Fix bug
1bf8fc5
-
Update Groq model to openai/gpt-oss-120b
44d1862 -
Add Groundtruth Answer Column
dc583a7