Compost / BUILD.md
abc1181's picture
Add dynamic auth configs, connect page, Next.js frontend
7657e9f
|
Raw
History Blame Contribute Delete
5.39 kB
# Platform - Build & Test Guide
## Architecture
```
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Platform Platform β”‚
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ React UI │◄──►│ FastAPI │◄──►│ SQLite β”‚ β”‚
β”‚ β”‚ (Vite) β”‚ β”‚ Backend β”‚ β”‚ Database β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚ β”‚ β”‚
β”‚ β–Ό β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ Composio API β”‚ β”‚
β”‚ β”‚ backend.composio.dev β”‚ β”‚
β”‚ β”‚ /api/v3.1/toolkits β”‚ β”‚
β”‚ β”‚ /api/v3.1/tools β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚ β”‚ β”‚
β”‚ β–Ό β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ Local Cache β”‚ β”‚
β”‚ β”‚ SQLite + JSON backup β”‚ β”‚
β”‚ β”‚ (/data/) β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```
## Local DB System
The platform uses a **dual-layer caching system**:
1. **SQLite Database** (`/data/platform.db`) - Primary store for integrations and tools
2. **JSON Backup** (`/data/composio_cache.json`) - Raw Composio API response backup
### How it works:
1. **Startup**: Tries to fetch from Composio API
2. **If API succeeds**: Saves to SQLite + JSON backup
3. **If API fails**: Loads from JSON backup β†’ seeds SQLite
4. **If no cache**: Uses minimal fallback definitions (5 integrations)
### Cache TTL: 24 hours (configurable via `CACHE_TTL`)
## Tool Calls Quota
**Good news**: Fetching tools via `GET /api/v3.1/toolkits` and `GET /api/v3.1/tools` **does NOT consume your 20,000 free tool calls**. Only `POST /api/v3.1/tools/execute/{slug}` counts as a tool call.
## Build & Run
### Option 1: Docker (Recommended for HF Spaces)
```bash
cd /home/runner/workspace/platform
# Set your Composio API key
export COMPOSIO_API_KEY=sk_your_key_here
# Build
docker build -t platform .
# Run
docker run -p 7860:7860 \
-v $(pwd)/data:/data \
-e COMPOSIO_API_KEY=$COMPOSIO_API_KEY \
platform
```
### Option 2: Direct (Development)
```bash
cd /home/runner/workspace/platform
# Build frontend
cd frontend && npm install && npm run build && cd ..
# Install backend deps
cd backend && pip install -r requirements.txt && cd ..
# Set env vars
export COMPOSIO_API_KEY=sk_your_key_here
export DATA_DIR=$(pwd)/data
# Initialize and run
python -c "from backend.database import init_db; init_db()"
python backend/integrations/registry.py # Syncs tools
cd backend && uvicorn main:app --reload --port 7860
```
## Test
```bash
# After starting the server:
cd /home/runner/workspace/platform
chmod +x test.sh
./test.sh
```
## Hugging Face Spaces Deployment
1. Create a new Docker Space on HF
2. Push these files to the repo:
```
Dockerfile
entrypoint.sh
backend/
frontend/
.env.example
```
3. Add `COMPOSIO_API_KEY` as a Space secret
4. The Space will auto-build and start on port 7860
## API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| GET | `/health` | Health check |
| GET | `/api/docs` | Swagger UI |
| POST | `/api/auth/register` | Register user |
| POST | `/api/auth/login` | Login |
| GET | `/api/apps` | List integrations |
| POST | `/api/apps/sync` | Sync from Composio API |
| GET | `/api/tools` | List tools |
| POST | `/api/tools/{id}/execute` | Execute tool |
| POST | `/api/tools/sync` | Sync tools from Composio |
| GET | `/api/connections` | List connections |
| GET | `/api/apikeys` | List API keys |
| GET | `/api/analytics/overview` | Usage stats |
| GET | `/mcp/v1/discovery` | MCP discovery |
| GET | `/mcp/v1/tools` | MCP tools list |
## Environment Variables
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `COMPOSIO_API_KEY` | No | - | Composio API key for dynamic sync |
| `COMPOSIO_API_BASE` | No | `https://backend.composio.dev/api/v3.1` | Composio API base URL |
| `SECRET_KEY` | No | (default) | JWT signing key |
| `DATA_DIR` | No | `/data` | Data directory |
| `CACHE_TTL` | No | `86400` | Cache TTL in seconds (24h) |