ndurner commited on
Commit
5f045d0
·
1 Parent(s): 12f6601

improve short desc

Browse files
Files changed (2) hide show
  1. .github/README.md +23 -92
  2. README.md +1 -1
.github/README.md CHANGED
@@ -118,96 +118,27 @@ When integrating this MCP into your own agent or client:
118
  - Set transport-level timeouts generously (10–20 minutes) and rely on the tools’ `wait_seconds` argument plus status polling for progress.
119
  - Ensure `GEMINI_API_KEY` (and any optional `AILEEN3_*` variables you use) are visible in the environment of the MCP server process, not just the client.
120
 
121
- ### 🛠️ MCP tools and definitions
122
- #### 🩺 Health and search
123
-
124
- - `health() -> { ok, detail, ffmpeg, gemini_api_key }`
125
- - Purpose: Lightweight health probe mirroring the Gradio demo’s health check. Confirms that `ffmpeg` is callable and `GEMINI_API_KEY` is present.
126
- - Usage: Call before running longer flows to surface missing runtime dependencies early.
127
-
128
- - `search_youtube(query: str, max_results: int = 10) -> { videos: [...] }`
129
- - Purpose: Fast YouTube search using `yt-dlp` (no downloads).
130
- - Arguments:
131
- - `query` (required): Free-form search terms (e.g. `"taler auditor bachelorthesis"`).
132
- - `max_results` (optional, default `10`, clamped to `1–50`).
133
- - Returns: `videos` list with `id`, `title`, `webpage_url`, `duration_seconds`, `channel`, `channel_id`.
134
- - Typical flow: Use from an agent to shortlist candidate videos before picking one `source` for retrieval.
135
-
136
- #### 📺 Media retrieval (entry point)
137
-
138
- - `start_media_retrieval(source: str, prefer_audio_only: bool = False, wait_seconds: int = 54) -> dict`
139
- - Purpose: Download long-form media (YouTube, podcasts, HTTP URLs) and normalize basic metadata.
140
- - Arguments:
141
- - `source`: YouTube URL/ID, podcast URL, or other `yt-dlp`-supported locator.
142
- - `prefer_audio_only`: When `true`, prefer audio-first formats; use when visuals are not needed.
143
- - `wait_seconds`: How long to block before returning; if the job is still running, you get status + reference.
144
- - Returns:
145
- - On success: `{ reference, status: "done", metadata: {...}, cached? }`
146
- - In progress: `{ reference, status: "pending"|"running", progress?, job_id }`
147
- - On error: `{ is_error: true, status, detail, reference }`
148
- - Typical flow: This is the first call once you have chosen a `source`. The `reference` token is required for all downstream tools.
149
-
150
- - `get_media_retrieval_status(reference: str, wait_seconds: int = 0) -> dict`
151
- - Purpose: Poll the retrieval job or fetch cached metadata.
152
- - Returns:
153
- - `{ status: "done", reference, metadata }` when cached or finished.
154
- - `{ status: "pending"|"running", ... }` while in flight.
155
- - `{ status: "not_found", reference }` if no job or cache exists.
156
-
157
- #### 🖼️ Slides: extraction and translation
158
-
159
- - `start_slide_extraction(reference: str, wait_seconds: int = 55) -> dict`
160
- - Purpose: Extract representative slide stills from a downloaded video.
161
- - Note: Full media analysis (`start_media_analysis`) automatically triggers slide extraction; call this explicitly only if you need slides on their own.
162
- - Returns: Standard job envelope with `slides` once done or `status` + `job_id` while running.
163
-
164
- - `get_extracted_slides(reference: str, wait_seconds: int = 0) -> dict`
165
- - Purpose: Fetch extracted slides or current extraction status.
166
- - Returns: `{ status: "done", reference, slides: [...] }` on success, otherwise a job status or `{ status: "not_found" }`. Slides include indices that are used by `translate_slide`.
167
-
168
- - `translate_slide(reference: str, slide_index: int, language: str) -> ImageContent`
169
- - Purpose: Translate a single slide image into another language using Gemini image-to-image.
170
- - Arguments:
171
- - `reference`: Token from `start_media_retrieval`.
172
- - `slide_index`: Zero-based index into `get_extracted_slides.slides[].index`.
173
- - `language`: Target language name (e.g. `"German"`, `"Spanish"`).
174
- - Returns: `ImageContent` with base64-encoded translated slide image. Responses are cached per `(reference, language, slide_index)`.
175
-
176
- #### ⛳️ Expectation-driven analysis
177
-
178
- - `start_media_analysis(reference: str, priors: object, wait_seconds: int = 55) -> dict`
179
- - Purpose: Run expectation-driven analysis over the media’s audio and slides, surfacing *surprises* and *new actors* instead of rehashing everything.
180
- - Arguments:
181
- - `reference`: Token produced by `start_media_retrieval`.
182
- - `priors`: Object with optional string fields:
183
- - `context`: Scene setting (participants, venue, goal, spelled names).
184
- - `expectations`: What the user already expects to hear.
185
- - `prior_knowledge`: What the user already knows from past work.
186
- - `questions`: Concrete questions to be answered.
187
- - Important: Only populate `priors` with information coming from the user or trusted tools (e.g. Memory Bank); do not invent priors in the agent.
188
- - Returns: Same job envelope pattern as retrieval. When `status: "done"`, the payload includes an `analysis` markdown briefing optimised for fast reading.
189
-
190
- - `get_media_analysis_result(reference: str, wait_seconds: int = 0) -> dict`
191
- - Purpose: Poll for completion or fetch cached analysis for a `reference`.
192
- - Returns:
193
- - `status: "done"` with `analysis` text on success.
194
- - `status: "pending"|"running"` during processing.
195
- - Errors include `is_error: true`, `detail`, `reference`.
196
-
197
- #### ✍️ Transcription
198
-
199
- - `start_media_transcription(reference: str, context: str = "", prefer_audio_only: bool = False, wait_seconds: int = 55) -> dict`
200
- - Purpose: Produce a diarized, speaker-labelled transcription of the media’s audio channel.
201
- - Arguments:
202
- - `reference`: From `start_media_retrieval`.
203
- - `context`: Optional grounding text with names, acronyms, or domain hints.
204
- - `prefer_audio_only`: When `true`, skip slide context for cheaper audio-only runs.
205
- - `wait_seconds`: Poll window before returning.
206
- - Returns: Job envelope, with `transcription` once `status: "done"`.
207
-
208
- - `get_media_transcription_result(reference: str, wait_seconds: int = 0) -> dict`
209
- - Purpose: Retrieve a previously computed transcription or current job status.
210
- - Returns: Same pattern as `get_media_analysis_result`, but with `transcription` instead of `analysis`.
211
 
212
  ## 🏆 Hackathon Context & Journey
213
  Aileen 3 Core was built for the [MCP's 1st Birthday - Hosted by Anthropic and Gradio](https://huggingface.co/MCP-1st-Birthday) and serves as the backbone for the [Aileen 3 Agent](https://ndurner.de/links/aileen3-kaggle-writeup) (developed for the [AI Agents Intensive Course with Google](https://www.kaggle.com/learn-guide/5-day-agents)).
@@ -231,8 +162,8 @@ docker run -it -p 7860:7860 aileen3-core
231
 
232
  ## 🚧 Limitations
233
  - `translate_slide` does currently not benefit from priors; translation quality could be improved that way
234
- - No AI safety guardrails (tone, style, anti prompt-injection, ...)
235
- - No cost control
236
  - Hallucination risk - Aileen may make mistakes.
237
  - Remote MCP operating mode not tested; would rely on external access protection
238
 
 
118
  - Set transport-level timeouts generously (10–20 minutes) and rely on the tools’ `wait_seconds` argument plus status polling for progress.
119
  - Ensure `GEMINI_API_KEY` (and any optional `AILEEN3_*` variables you use) are visible in the environment of the MCP server process, not just the client.
120
 
121
+ ### 🛠️ MCP tools overview
122
+
123
+ All tools are registered in `aileen3_mcp.server.make_app` and exposed via a stdio MCP server for use by the Gradio demo, Claude Desktop, and other clients.
124
+
125
+ In short, the public tools are:
126
+
127
+ - `health`
128
+ - `search_youtube`
129
+ - `start_media_retrieval` / `get_media_retrieval_status`
130
+ - `start_slide_extraction` / `get_extracted_slides`
131
+ - `translate_slide`
132
+ - `start_media_analysis` / `get_media_analysis_result`
133
+ - `start_media_transcription` / `get_media_transcription_result`
134
+
135
+ These tools are designed to be called from an agentic chat interface that:
136
+
137
+ - first chooses a media `source` (optionally using `search_youtube`)
138
+ - then calls `start_media_retrieval`
139
+ - and finally uses the `reference` token to drive analysis, transcription, or slide translation.
140
+
141
+ For detailed tool contracts (arguments, return payloads, and error shapes), see `mcp/README.md`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
142
 
143
  ## 🏆 Hackathon Context & Journey
144
  Aileen 3 Core was built for the [MCP's 1st Birthday - Hosted by Anthropic and Gradio](https://huggingface.co/MCP-1st-Birthday) and serves as the backbone for the [Aileen 3 Agent](https://ndurner.de/links/aileen3-kaggle-writeup) (developed for the [AI Agents Intensive Course with Google](https://www.kaggle.com/learn-guide/5-day-agents)).
 
162
 
163
  ## 🚧 Limitations
164
  - `translate_slide` does currently not benefit from priors; translation quality could be improved that way
165
+ - No AI safety guardrails (tone, style, anti prompt-injection, ...) included
166
+ - No cost control included
167
  - Hallucination risk - Aileen may make mistakes.
168
  - Remote MCP operating mode not tested; would rely on external access protection
169
 
README.md CHANGED
@@ -6,7 +6,7 @@ colorTo: blue
6
  sdk: docker
7
  pinned: false
8
  license: cc-by-4.0
9
- short_description: Expectation-driven briefs from talks, lectures, panels, ...
10
  tags:
11
  - building-mcp-track-enterprise
12
  - building-mcp-track-customer
 
6
  sdk: docker
7
  pinned: false
8
  license: cc-by-4.0
9
+ short_description: Use priors to surface novel insights in noisy communications
10
  tags:
11
  - building-mcp-track-enterprise
12
  - building-mcp-track-customer