Spaces:
Sleeping
Sleeping
feat: complete Ever-Brain engine, MCP server, CLI evb mcp command, Dockerfile and release workflow
61b1da1 | # Wiki: 02. API Specification & Implementation | |
| The **Ever-Brain API** is the content-access layer built on **TypeScript + Hono (Node.js)**. It serves as a reliable source layer that resolves GitHub repositories and streams their content. | |
| --- | |
| ## 1. Dual Ingestion Strategy | |
| To balance fast response times with strict GitHub API rate-limit conservation, the API utilizes a two-tier ingestion mechanism: | |
| ```text | |
| ┌────────────────────────────────────────────────┐ | |
| │ Ever-Brain API │ | |
| └────────┬──────────────────────────────┬────────┘ | |
| │ │ | |
| 1. Tree Discovery 2. Content Streaming | |
| (api.github.com) (raw.githubusercontent.com) | |
| │ │ | |
| ▼ ▼ | |
| GitHub Git Trees API Raw GitHub CDN | |
| (1 req / 60s cached per repo) (0 REST API quota cost) | |
| ``` | |
| 1. **Discovery Tier**: | |
| - Discovers project trees via `GET https://api.github.com/repos/{owner}/{repo}/git/trees/{branch}?recursive=1`. | |
| - Single HTTP request discovers entire repository hierarchies (up to 100,000 objects in 45–380ms). | |
| - Rate limit tracked via response headers (`x-ratelimit-remaining`, `x-ratelimit-limit`). | |
| 2. **Streaming Tier**: | |
| - Streams raw file contents directly from `https://raw.githubusercontent.com/{owner}/{repo}/{branch}/{path}`. | |
| - **Consumes 0 GitHub REST API quota**, allowing unlimited content reads for public repositories. | |
| --- | |
| ## 2. In-Memory Tree Cache Engine | |
| - **Implementation**: `packages/api/src/services/tree-cache.ts` | |
| - **TTL**: 60,000ms (60 seconds) by default (configurable via `CACHE_TTL_MS`). | |
| - **Key**: `${owner.toLowerCase()}/${repo.toLowerCase()}:${branch.toLowerCase()}`. | |
| - **Cache Invalidation**: | |
| - Automatically expires after TTL. | |
| - Can be bypassed on any request by passing query parameter `?fresh=true`. | |
| - In-memory only: Server restarts cleanly drop cache with zero data loss or corruption. | |
| --- | |
| ## 3. Path Normalization & Binary Filtering | |
| ### Case-Insensitive Path Normalization | |
| Many autonomous AI agents request uppercase filenames (e.g. `README.md`, `INDEX.md`, `LICENSE.txt`) even when the repository uses lowercase (`readme.md`). | |
| - The API builds a lowercase lookup index during tree discovery: | |
| `pathMap: Map<lowercasePath, exactCasedPath>`. | |
| - When an endpoint receives `/repos/acme/project/file/README.md`, the resolver looks up `readme.md`, discovers the true casing, and streams the correct raw CDN file without raising a 404. | |
| ### Binary and Noise Filter | |
| The API excludes non-textual files and `.git/` repository internals during discovery: | |
| - **Git internals**: `.git/`, `.git/HEAD`, `.git/objects/...` | |
| - **Images**: `.png`, `.jpg`, `.jpeg`, `.gif`, `.ico`, `.webp`, `.bmp`, `.tiff`, `.svg` | |
| - **Audio/Video**: `.mp3`, `.mp4`, `.wav`, `.mov`, `.avi`, `.flac`, `.mkv`, etc. | |
| - **Binaries & Archives**: `.zip`, `.tar`, `.gz`, `.7z`, `.rar`, `.exe`, `.dll`, `.so`, `.wasm`, `.bin` | |
| - **Lockfiles & DBs**: `package-lock.json`, `pnpm-lock.yaml`, `yarn.lock`, `Cargo.lock`, `.sqlite`, `.db` | |
| *(Note: The API does not enforce arbitrary framework ignore lists like `node_modules` or `dist` to avoid restricting genuine content repositories).* | |
| --- | |
| ## 4. Substring Search Engine | |
| - **Implementation**: `packages/api/src/services/search-service.ts` | |
| - **Design**: Exact substring matching across discovered text/Markdown files. | |
| - **Concurrency**: Streams candidate files concurrently in controlled batches of 10. | |
| - **Response**: Returns matching file paths, total match count, and 1-indexed line numbers where the query occurs. | |
| --- | |
| ## 5. Complete REST API Reference | |
| ### `GET /health` | |
| Operational probe returning service status, uptime, and timestamp. | |
| ```json | |
| { | |
| "status": "ok", | |
| "version": "1.0.0", | |
| "uptimeSeconds": 142, | |
| "timestamp": "2026-08-21T14:36:28.656Z" | |
| } | |
| ``` | |
| ### `GET /repos/:owner/:repo` | |
| Resolves repository metadata and detects the default branch (`main`, `master`, `develop`, etc.). | |
| ```json | |
| { | |
| "owner": "abrez-rizvi", | |
| "repo": "trial-markdown", | |
| "defaultBranch": "main" | |
| } | |
| ``` | |
| ### `GET /repos/:owner/:repo/files` | |
| Returns flat array of discovered project files. | |
| - **Query Parameters**: | |
| - `prefix` *(optional)*: Filter by path prefix (e.g. `?prefix=knowledge/`). | |
| - `fresh` *(optional)*: Pass `true` or `1` to bypass the 60s in-memory tree cache. | |
| - `branch` *(optional)*: Target a specific branch instead of the default branch. | |
| ```json | |
| { | |
| "repository": "abrez-rizvi/trial-markdown", | |
| "branch": "main", | |
| "totalFiles": 1, | |
| "files": [ | |
| { | |
| "path": "text.md", | |
| "type": "file", | |
| "sizeBytes": 92, | |
| "mimeType": "text/markdown" | |
| } | |
| ] | |
| } | |
| ``` | |
| ### `GET /repos/:owner/:repo/file/*path` | |
| Streams the exact raw text or Markdown content with appropriate headers. | |
| - **Response Headers**: | |
| - `Content-Type`: MIME type (e.g. `text/markdown; charset=utf-8`, `text/plain; charset=utf-8`). | |
| - `X-Ever-Brain-Path`: Exact cased path in the repository. | |
| - `X-Ever-Brain-Branch`: Target branch resolved. | |
| - **Aliases**: `GET /repos/:owner/:repo/files/*path` and `GET /projects/:owner/:repo/files/*path`. | |
| ### `GET /repos/:owner/:repo/search` | |
| Searches text/Markdown documents for a string query. | |
| - **Query Parameters**: | |
| - `q` *(required)*: The search string. | |
| - `prefix` *(optional)*: Search only within a specific folder prefix. | |
| - `branch` *(optional)*: Target branch. | |
| ```json | |
| { | |
| "query": "architecture", | |
| "totalMatches": 3, | |
| "results": [ | |
| { | |
| "path": "knowledge/architecture.md", | |
| "matches": 3, | |
| "lines": [1, 14, 42] | |
| } | |
| ] | |
| } | |
| ``` | |