Claude commited on
Commit
7da7ce7
·
unverified ·
1 Parent(s): 7372959

docs: Update README and E2E tests for unified chatbot UI

Browse files

- Rewrite README with full feature documentation
- Add tech stack table and example interactions
- Include development setup instructions
- Update E2E tests for unified chatbot interface
- Remove outdated tab-based UI tests
- Add tests for welcome message, clear chat, responsive layout

Files changed (2) hide show
  1. README.md +82 -16
  2. tests/test_e2e.py +104 -62
README.md CHANGED
@@ -32,34 +32,100 @@ short_description: Download, transcribe, and chat with YouTube videos using AI
32
 
33
  # Video Analyzer
34
 
35
- Download, transcribe, and chat with YouTube videos using AI.
36
 
37
  ## Features
38
 
39
- - **YouTube Video Download**: Supports videos, playlists, and shorts
40
- - **Speech-to-Text**: Automatic transcription using OpenAI Whisper
 
41
  - **Visual Analysis**: Key frame extraction and captioning with BLIP
42
- - **Knowledge Base**: Vector storage with ChromaDB for semantic search
43
- - **RAG Chatbot**: Ask questions about your videos using Qwen2.5-72B
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
  ## How to Use
46
 
47
- 1. **Sign in** with your HuggingFace account
48
- 2. **Paste** a YouTube URL in the Analyze tab
49
- 3. **Wait** for processing (transcription + frame analysis)
50
- 4. **Chat** about the video content in the Chat tab
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
 
52
  ## Tech Stack
53
 
54
- - **Gradio**: Web UI framework
55
- - **Whisper**: Speech recognition
56
- - **BLIP**: Image captioning
57
- - **ChromaDB**: Vector database
58
- - **Sentence Transformers**: Text embeddings
59
- - **HuggingFace Inference API**: SOTA language model
 
 
 
 
60
 
61
  ## Limitations
62
 
63
  - Works best with videos under 10 minutes
64
  - Requires HuggingFace login for authentication
65
- - Knowledge base is session-based (resets on Space restart)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
 
33
  # Video Analyzer
34
 
35
+ A conversational AI assistant that analyzes YouTube videos and answers questions about their content.
36
 
37
  ## Features
38
 
39
+ ### Core Capabilities
40
+ - **YouTube Video Download**: Supports videos, playlists, and shorts via yt-dlp
41
+ - **Speech-to-Text**: Automatic transcription using OpenAI Whisper (whisper-base)
42
  - **Visual Analysis**: Key frame extraction and captioning with BLIP
43
+ - **Knowledge Base**: Per-session vector storage with ChromaDB for semantic search
44
+ - **RAG Chatbot**: Ask questions about your videos using Qwen2.5-72B-Instruct
45
+
46
+ ### User Experience
47
+ - **Unified Chat Interface**: Single chatbot handles both video analysis and Q&A
48
+ - **Auto URL Detection**: Just paste a YouTube URL and the assistant analyzes it
49
+ - **Conversational Flow**: The assistant guides you through the process
50
+ - **Per-Session Storage**: Your analyzed videos are private to your session
51
+ - **Persistent Sessions**: Your knowledge base persists across page reloads (tied to your HuggingFace profile)
52
+
53
+ ### Technical Features
54
+ - **ZeroGPU Support**: Leverages HuggingFace ZeroGPU for faster GPU-accelerated processing
55
+ - **Model Fallback**: Automatic fallback chain (Qwen2.5-72B → Llama-3.1-70B) for reliability
56
+ - **HuggingFace OAuth**: Secure authentication via HuggingFace login
57
+ - **Gradio 6**: Modern UI with the Soft theme
58
 
59
  ## How to Use
60
 
61
+ 1. **Sign in** with your HuggingFace account using the button in the top right
62
+ 2. **Paste** a YouTube URL directly in the chat (e.g., `https://youtube.com/watch?v=...`)
63
+ 3. **Wait** for processing - the assistant will transcribe audio and analyze key frames
64
+ 4. **Ask questions** about the video content in natural language
65
+
66
+ ### Example Interactions
67
+
68
+ ```
69
+ You: https://youtube.com/watch?v=dQw4w9WgXcQ
70
+ Bot: I'll analyze that video for you. This may take a few minutes...
71
+ Bot: Done! I've analyzed "Never Gonna Give You Up" and added it to my knowledge base.
72
+
73
+ You: What is this video about?
74
+ Bot: Based on the transcript, this video is a music video for Rick Astley's 1987 hit song...
75
+
76
+ You: What visual elements were shown?
77
+ Bot: The video shows a man dancing in various locations...
78
+ ```
79
 
80
  ## Tech Stack
81
 
82
+ | Component | Technology |
83
+ |-----------|------------|
84
+ | Web Framework | Gradio 6 with OAuth |
85
+ | Speech Recognition | OpenAI Whisper (whisper-base) |
86
+ | Image Captioning | Salesforce BLIP |
87
+ | Vector Database | ChromaDB (in-memory, per-session) |
88
+ | Text Embeddings | Sentence Transformers (all-MiniLM-L6-v2) |
89
+ | Language Model | HuggingFace Inference API (Qwen2.5-72B-Instruct) |
90
+ | Video Download | yt-dlp |
91
+ | GPU Acceleration | HuggingFace ZeroGPU (A10G) |
92
 
93
  ## Limitations
94
 
95
  - Works best with videos under 10 minutes
96
  - Requires HuggingFace login for authentication
97
+ - Knowledge base is session-based (stored in memory, not persistent across Space restarts)
98
+ - Audio extraction requires FFmpeg (pre-installed on HuggingFace Spaces)
99
+
100
+ ## Development
101
+
102
+ ### Prerequisites
103
+ - Python 3.11+
104
+ - uv package manager
105
+ - FFmpeg
106
+
107
+ ### Setup
108
+ ```bash
109
+ # Install dependencies
110
+ uv sync
111
+
112
+ # Install dev dependencies
113
+ uv sync --extra dev
114
+
115
+ # Run the app locally
116
+ uv run python app.py
117
+ ```
118
+
119
+ ### Testing
120
+ ```bash
121
+ # Run unit tests
122
+ uv run --extra dev pytest tests/test_app.py -v
123
+
124
+ # Run E2E tests (requires playwright browsers)
125
+ uv run --extra dev playwright install
126
+ uv run --extra dev pytest tests/test_e2e.py -v
127
+ ```
128
+
129
+ ## License
130
+
131
+ MIT
tests/test_e2e.py CHANGED
@@ -30,8 +30,8 @@ def app_url() -> Generator[str, None, None]:
30
  process.wait()
31
 
32
 
33
- class TestVideoAnalyzerUI:
34
- """E2E tests for the Video Analyzer UI."""
35
 
36
  def test_homepage_loads(self, page: Page, app_url: str):
37
  """Test that the homepage loads correctly."""
@@ -40,105 +40,147 @@ class TestVideoAnalyzerUI:
40
  # Check title is visible
41
  expect(page.locator("text=Video Analyzer")).to_be_visible()
42
 
43
- def test_app_description_visible(self, page: Page, app_url: str):
44
- """Test that the app description is visible."""
45
  page.goto(app_url)
46
 
47
- # Check description
48
- expect(page.locator("text=Download, transcribe, analyze")).to_be_visible()
49
 
50
  def test_login_button_visible(self, page: Page, app_url: str):
51
  """Test that the login button is visible."""
52
  page.goto(app_url)
53
 
54
- # Look for login button
55
  login_button = page.locator("button:has-text('Sign in')")
56
  expect(login_button).to_be_visible()
57
 
58
- def test_analyze_tab_visible(self, page: Page, app_url: str):
59
- """Test that the Analyze Videos tab is visible."""
60
  page.goto(app_url)
61
 
62
- # Check for Analyze Videos tab
63
- analyze_tab = page.locator("text=Analyze Videos")
64
- expect(analyze_tab).to_be_visible()
65
 
66
- def test_chat_tab_visible(self, page: Page, app_url: str):
67
- """Test that the Chat with Videos tab is visible."""
68
  page.goto(app_url)
69
 
70
- # Check for Chat tab
71
- chat_tab = page.locator("text=Chat with Videos")
72
- expect(chat_tab).to_be_visible()
73
 
74
- def test_youtube_url_input_exists(self, page: Page, app_url: str):
75
- """Test that the YouTube URL input field exists."""
 
 
 
 
 
 
 
 
 
 
 
76
  page.goto(app_url)
77
 
78
- # Check for URL input with placeholder
79
- url_input = page.locator("input[type='text'], textarea").first
80
- expect(url_input).to_be_visible()
81
 
82
- def test_analyze_button_exists(self, page: Page, app_url: str):
83
- """Test that the Analyze Video button exists."""
84
  page.goto(app_url)
85
 
86
- # Check for Analyze button
87
- analyze_btn = page.locator("button:has-text('Analyze Video')")
88
- expect(analyze_btn).to_be_visible()
 
 
 
 
 
 
 
 
 
 
89
 
90
- def test_frame_analysis_checkbox_exists(self, page: Page, app_url: str):
91
- """Test that the frame analysis checkbox exists."""
92
  page.goto(app_url)
93
 
94
- # Check for checkbox label
95
- checkbox = page.locator("text=Analyze video frames")
96
- expect(checkbox).to_be_visible()
97
 
98
- def test_frame_slider_exists(self, page: Page, app_url: str):
99
- """Test that the frame count slider exists."""
 
 
 
100
  page.goto(app_url)
101
 
102
- # Check for slider
103
- slider_label = page.locator("text=Number of frames")
104
- expect(slider_label).to_be_visible()
105
 
106
- def test_step_instructions_visible(self, page: Page, app_url: str):
107
- """Test that step instructions are visible."""
108
  page.goto(app_url)
109
 
110
- # Check for step labels
111
- expect(page.locator("text=Step 1")).to_be_visible()
112
- expect(page.locator("text=Step 2")).to_be_visible()
113
- expect(page.locator("text=Step 3")).to_be_visible()
 
114
 
115
- def test_can_switch_to_chat_tab(self, page: Page, app_url: str):
116
- """Test switching to the Chat tab."""
117
  page.goto(app_url)
118
 
119
- # Click Chat tab
120
- page.click("text=Chat with Videos")
 
 
 
 
 
 
 
 
121
 
122
- # Verify chat elements are visible
123
- expect(page.locator("text=Ask questions about videos")).to_be_visible()
124
 
125
- def test_ask_button_in_chat_tab(self, page: Page, app_url: str):
126
- """Test that Ask button exists in Chat tab."""
127
  page.goto(app_url)
128
 
129
- # Switch to Chat tab
130
- page.click("text=Chat with Videos")
131
 
132
- # Check for Ask button
133
- ask_btn = page.locator("button:has-text('Ask')")
134
- expect(ask_btn).to_be_visible()
135
 
136
- def test_knowledge_base_status_in_chat_tab(self, page: Page, app_url: str):
137
- """Test that knowledge base status is shown in Chat tab."""
138
  page.goto(app_url)
139
 
140
- # Switch to Chat tab
141
- page.click("text=Chat with Videos")
 
 
 
 
 
 
 
 
 
142
 
143
- # Should show empty knowledge base message
144
- expect(page.locator("text=Knowledge base")).to_be_visible()
 
 
30
  process.wait()
31
 
32
 
33
+ class TestUnifiedChatbotUI:
34
+ """E2E tests for the unified chatbot Video Analyzer UI."""
35
 
36
  def test_homepage_loads(self, page: Page, app_url: str):
37
  """Test that the homepage loads correctly."""
 
40
  # Check title is visible
41
  expect(page.locator("text=Video Analyzer")).to_be_visible()
42
 
43
+ def test_app_subtitle_visible(self, page: Page, app_url: str):
44
+ """Test that the app subtitle is visible."""
45
  page.goto(app_url)
46
 
47
+ # Check subtitle
48
+ expect(page.locator("text=Analyze YouTube videos")).to_be_visible()
49
 
50
  def test_login_button_visible(self, page: Page, app_url: str):
51
  """Test that the login button is visible."""
52
  page.goto(app_url)
53
 
54
+ # Look for login button (HuggingFace sign in)
55
  login_button = page.locator("button:has-text('Sign in')")
56
  expect(login_button).to_be_visible()
57
 
58
+ def test_chatbot_visible(self, page: Page, app_url: str):
59
+ """Test that the chatbot component is visible."""
60
  page.goto(app_url)
61
 
62
+ # Check for chatbot container
63
+ chatbot = page.locator("[data-testid='chatbot']")
64
+ expect(chatbot).to_be_visible()
65
 
66
+ def test_welcome_message_displayed(self, page: Page, app_url: str):
67
+ """Test that welcome message is shown on load."""
68
  page.goto(app_url)
69
 
70
+ # Wait for page to load
71
+ page.wait_for_timeout(2000)
 
72
 
73
+ # Check for welcome message content
74
+ expect(page.locator("text=Welcome to Video Analyzer")).to_be_visible()
75
+
76
+ def test_message_input_exists(self, page: Page, app_url: str):
77
+ """Test that the message input field exists."""
78
+ page.goto(app_url)
79
+
80
+ # Check for text input with placeholder
81
+ msg_input = page.locator("textarea[placeholder*='YouTube URL']")
82
+ expect(msg_input).to_be_visible()
83
+
84
+ def test_send_button_exists(self, page: Page, app_url: str):
85
+ """Test that the Send button exists."""
86
  page.goto(app_url)
87
 
88
+ # Check for Send button
89
+ send_btn = page.locator("button:has-text('Send')")
90
+ expect(send_btn).to_be_visible()
91
 
92
+ def test_clear_chat_button_exists(self, page: Page, app_url: str):
93
+ """Test that the Clear Chat button exists."""
94
  page.goto(app_url)
95
 
96
+ # Check for Clear Chat button
97
+ clear_btn = page.locator("button:has-text('Clear Chat')")
98
+ expect(clear_btn).to_be_visible()
99
+
100
+ def test_knowledge_base_status_visible(self, page: Page, app_url: str):
101
+ """Test that knowledge base status is displayed."""
102
+ page.goto(app_url)
103
+
104
+ # Wait for status to load
105
+ page.wait_for_timeout(2000)
106
+
107
+ # Check for knowledge base empty message
108
+ expect(page.locator("text=Knowledge base is empty")).to_be_visible()
109
 
110
+ def test_can_type_in_message_input(self, page: Page, app_url: str):
111
+ """Test that user can type in the message input."""
112
  page.goto(app_url)
113
 
114
+ # Find and fill the message input
115
+ msg_input = page.locator("textarea[placeholder*='YouTube URL']")
116
+ msg_input.fill("https://youtube.com/watch?v=test123")
117
 
118
+ # Verify the input has the text
119
+ expect(msg_input).to_have_value("https://youtube.com/watch?v=test123")
120
+
121
+ def test_send_button_is_primary(self, page: Page, app_url: str):
122
+ """Test that Send button has primary styling."""
123
  page.goto(app_url)
124
 
125
+ # Check for primary variant button
126
+ send_btn = page.locator("button:has-text('Send')").first
127
+ expect(send_btn).to_be_visible()
128
 
129
+ def test_login_prompt_for_unauthenticated_users(self, page: Page, app_url: str):
130
+ """Test that unauthenticated users see login prompt in welcome."""
131
  page.goto(app_url)
132
 
133
+ # Wait for welcome message
134
+ page.wait_for_timeout(2000)
135
+
136
+ # Check for sign in prompt
137
+ expect(page.locator("text=sign in with HuggingFace")).to_be_visible()
138
 
139
+ def test_clear_chat_works(self, page: Page, app_url: str):
140
+ """Test that Clear Chat button clears the chatbot."""
141
  page.goto(app_url)
142
 
143
+ # Wait for welcome message to appear
144
+ page.wait_for_timeout(2000)
145
+ expect(page.locator("text=Welcome to Video Analyzer")).to_be_visible()
146
+
147
+ # Click clear chat
148
+ clear_btn = page.locator("button:has-text('Clear Chat')")
149
+ clear_btn.click()
150
+
151
+ # Wait a moment for clear to process
152
+ page.wait_for_timeout(500)
153
 
154
+ # Welcome message should be gone (chat cleared)
155
+ expect(page.locator("text=Welcome to Video Analyzer")).not_to_be_visible()
156
 
157
+ def test_responsive_layout(self, page: Page, app_url: str):
158
+ """Test that the layout is responsive."""
159
  page.goto(app_url)
160
 
161
+ # Set mobile viewport
162
+ page.set_viewport_size({"width": 375, "height": 667})
163
 
164
+ # UI elements should still be visible
165
+ expect(page.locator("text=Video Analyzer")).to_be_visible()
166
+ expect(page.locator("button:has-text('Send')")).to_be_visible()
167
 
168
+ def test_chatbot_has_height(self, page: Page, app_url: str):
169
+ """Test that chatbot has appropriate height."""
170
  page.goto(app_url)
171
 
172
+ # Get chatbot element
173
+ chatbot = page.locator("[data-testid='chatbot']")
174
+ box = chatbot.bounding_box()
175
+
176
+ # Should have significant height (500px configured)
177
+ assert box is not None
178
+ assert box["height"] >= 400 # Allow some flexibility
179
+
180
+ def test_theme_applied(self, page: Page, app_url: str):
181
+ """Test that Soft theme is applied (lighter colors)."""
182
+ page.goto(app_url)
183
 
184
+ # The Soft theme should be applied - check body has gradio styling
185
+ body = page.locator("body")
186
+ expect(body).to_be_visible()