Spaces:

shimen
/

API

Paused

App Files Files Community

API / kiro-gateway /docs /en /ARCHITECTURE.md

sshinmen

Clean deploy to HF Space

bf9e111 5 months ago

preview code

Raw

History Blame Contribute Delete

38.8 kB

Architectural Overview: Kiro Gateway

1. System Purpose and Goals

The project is a high-level proxy gateway implementing the "Adapter" structural design pattern.

The main goal of the system is to provide transparent compatibility between multiple heterogeneous interfaces:

Supported API Formats

API	Endpoints	Status
OpenAI	`/v1/models`, `/v1/chat/completions`	✅ Supported
Anthropic	`/v1/messages`	✅ Supported

Architectural Model

┌─────────────────────────────────────────────────────────────────┐
│                          Clients                                │
│  ┌─────────────────────┐       ┌─────────────────────┐         │
│  │  OpenAI SDK/Tools   │       │ Anthropic SDK/Tools │         │
│  │  (Cursor, Cline,    │       │ (Claude Code,       │         │
│  │   Continue, etc.)   │       │  Anthropic SDK)     │         │
│  └──────────┬──────────┘       └──────────┬──────────┘         │
└─────────────┼──────────────────────────────┼───────────────────┘
              │                              │
              ▼                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Kiro Gateway                               │
│  ┌─────────────────────┐       ┌─────────────────────┐         │
│  │  OpenAI Adapter     │       │  Anthropic Adapter  │         │
│  │  /v1/chat/...       │       │  /v1/messages       │         │
│  └──────────┬──────────┘       └──────────┬──────────┘         │
│             └──────────────┬───────────────┘                    │
│                            ▼                                    │
│             ┌─────────────────────────────┐                     │
│             │      Core Layer             │                     │
│             │  (Shared conversion logic)  │                     │
│             └──────────────┬──────────────┘                     │
└────────────────────────────┼────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                        Kiro API                                 │
│              (AWS CodeWhisperer Backend)                        │
└─────────────────────────────────────────────────────────────────┘

The system acts as a "translator", allowing the use of any tools, libraries, and IDE plugins developed for OpenAI and Anthropic ecosystems with Claude models through the Kiro API.

Both APIs work simultaneously on the same server without any configuration switching.

2. Project Structure

The project is organized as a modular Python package kiro/:

kiro-gateway/
├── main.py                    # Entry point, FastAPI application creation
├── requirements.txt           # Python dependencies
├── .env.example               # Environment configuration example
│
├── kiro/              # Main package
│   ├── __init__.py            # Package exports, version
│   │
│   │   # ═══════════════════════════════════════════════════════
│   │   # SHARED LAYER - Reused by all APIs
│   │   # ═══════════════════════════════════════════════════════
│   ├── config.py              # Configuration and constants
│   ├── auth.py                # KiroAuthManager - token management
│   ├── cache.py               # ModelInfoCache - model cache
│   ├── http_client.py         # HTTP client with retry logic
│   ├── parsers.py             # AWS SSE stream parsers
│   ├── utils.py               # Helper utilities
│   ├── tokenizer.py           # Token counting (tiktoken)
│   ├── debug_logger.py        # Debug request logging
│   ├── exceptions.py          # Exception handlers
│   ├── thinking_parser.py     # Thinking blocks parser
│   │
│   │   # ═══════════════════════════════════════════════════════
│   │   # CORE LAYER - Shared core for all APIs
│   │   # ═══════════════════════════════════════════════════════
│   ├── converters_core.py     # Shared Kiro payload building logic
│   ├── streaming_core.py      # Shared Kiro stream parsing logic
│   │
│   │   # ═══════════════════════════════════════════════════════
│   │   # OPENAI API LAYER
│   │   # ═══════════════════════════════════════════════════════
│   ├── models_openai.py       # Pydantic models for OpenAI API
│   ├── converters_openai.py   # OpenAI → Kiro adapter
│   ├── routes_openai.py       # FastAPI routes for OpenAI
│   ├── streaming_openai.py    # Kiro → OpenAI SSE formatter
│   │
│   │   # ═══════════════════════════════════════════════════════
│   │   # ANTHROPIC API LAYER
│   │   # ═══════════════════════════════════════════════════════
│   ├── models_anthropic.py    # Pydantic models for Anthropic API
│   ├── converters_anthropic.py # Anthropic → Kiro adapter
│   ├── routes_anthropic.py    # FastAPI routes for Anthropic
│   └── streaming_anthropic.py # Kiro → Anthropic SSE formatter
│
├── tests/                     # Tests
│   ├── conftest.py            # Pytest fixtures
│   ├── unit/                  # Unit tests
│   └── integration/           # Integration tests
│
├── docs/                      # Documentation
│   ├── ru/                    # Russian version
│   └── en/                    # English version
│
└── debug_logs/                # Debug logs (generated when DEBUG_LAST_REQUEST=true)

Organization Principle: Shared Core + Thin Adapters

The architecture is built on the principle of maximum code reuse:

Layer	Purpose	Files
Shared Layer	Infrastructure independent of API format	`auth.py`, `http_client.py`, `cache.py`, `parsers.py`, `tokenizer.py`
Core Layer	Shared business logic for conversion	`converters_core.py`, `streaming_core.py`
API Layer	Thin adapters for specific formats	`_openai.py`, `_anthropic.py`

3. Architectural Topology and Components

The system is built on the asynchronous FastAPI framework and uses an event-driven lifecycle management model (Lifespan Events).

3.1. Entry Point (`main.py`)

The main.py file is responsible for:

Logging configuration — Loguru setup with colored output
Configuration validation — validate_configuration() function checks:
- Presence of .env file
- Presence of credentials (REFRESH_TOKEN or KIRO_CREDS_FILE)
Lifespan Manager — creation and initialization of:
- KiroAuthManager for token management
- ModelInfoCache for model caching
Error handler registration — validation_exception_handler for 422 errors
Route connection — app.include_router(router)

3.2. Configuration Module (`kiro/config.py`)

Centralized storage of all settings:

Parameter	Description	Default Value
`PROXY_API_KEY`	API key for proxy access	`changeme_proxy_secret`
`REFRESH_TOKEN`	Kiro refresh token	from `.env`
`PROFILE_ARN`	AWS CodeWhisperer profile ARN	from `.env`
`REGION`	AWS region	`us-east-1`
`KIRO_CREDS_FILE`	Path to JSON credentials file	from `.env`
`TOKEN_REFRESH_THRESHOLD`	Time before token refresh	600 sec (10 min)
`MAX_RETRIES`	Max retry attempts	3
`BASE_RETRY_DELAY`	Base retry delay	1.0 sec
`MODEL_CACHE_TTL`	Model cache TTL	3600 sec (1 hour)
`DEFAULT_MAX_INPUT_TOKENS`	Default max input tokens	200000
`TOOL_DESCRIPTION_MAX_LENGTH`	Max tool description length	10000 characters
`DEBUG_LAST_REQUEST`	Enable debug logging	`false`
`DEBUG_DIR`	Debug logs directory	`debug_logs`
`APP_VERSION`	Application version	`0.0.0`

Helper functions:

get_kiro_refresh_url(region) — URL for token refresh
get_kiro_api_host(region) — main API host
get_kiro_q_host(region) — Q API host
get_internal_model_id(external_model) — model name conversion

3.3. Pydantic Models (`kiro/models_openai.py`)

Models for `/v1/models`

Model	Description
`OpenAIModel`	AI model description (id, object, created, owned_by)
`ModelList`	Model list for endpoint response

Models for `/v1/chat/completions`

Model	Description
`ChatMessage`	Chat message (role, content, tool_calls, tool_call_id)
`ToolFunction`	Tool function description (name, description, parameters)
`Tool`	OpenAI format tool (type, function)
`ChatCompletionRequest`	Generation request (model, messages, stream, tools, ...)

Response Models

Model	Description
`ChatCompletionChoice`	Single response variant
`ChatCompletionUsage`	Token information (prompt_tokens, completion_tokens, credits_used)
`ChatCompletionResponse`	Full response (non-streaming)
`ChatCompletionChunk`	Streaming chunk
`ChatCompletionChunkDelta`	Delta changes in chunk
`ChatCompletionChunkChoice`	Variant in streaming chunk

3.4. State Management Layer

KiroAuthManager (`kiro/auth.py`)

Role: Stateful singleton encapsulating Kiro token management logic.

Capabilities:

Loading credentials from .env or JSON file
Support for expiresAt to check token expiration time
Automatic token refresh 10 minutes before expiration
Saving updated tokens back to JSON file
Support for different AWS regions
Unique fingerprint generation for User-Agent

Concurrency Control: Uses asyncio.Lock to protect against race conditions.

Main methods:

get_access_token() — returns valid token, refreshing if necessary
force_refresh() — forced token refresh (on 403)
is_token_expiring_soon() — expiration time check

Properties:

profile_arn — profile ARN
region — AWS region
api_host — API host for region
q_host — Q API host for region
fingerprint — unique machine fingerprint

# Usage example
auth_manager = KiroAuthManager(
    refresh_token="your_token",
    region="us-east-1",
    creds_file="~/.aws/sso/cache/kiro-auth-token.json"
)
token = await auth_manager.get_access_token()

ModelInfoCache (`kiro/cache.py`)

Role: Thread-safe storage for model configurations.

Population Strategy:

Lazy Loading via /ListAvailableModels
Cache TTL: 1 hour
Fallback to static model list

Main methods:

update(models_data) — cache update
get(model_id) — get model information
get_max_input_tokens(model_id) — get token limit
is_empty() / is_stale() — cache state check
get_all_model_ids() — list of all model IDs

3.5. Helper Utilities (`kiro/utils.py`)

Function	Description
`get_machine_fingerprint()`	SHA256 hash of `{hostname}-{username}-kiro-gateway`
`get_kiro_headers(auth_manager, token)`	Form headers for Kiro API
`generate_completion_id()`	ID in format `chatcmpl-{uuid_hex}`
`generate_conversation_id()`	UUID for conversation
`generate_tool_call_id()`	ID in format `call_{uuid_hex[:8]}`

3.6. Conversion Layer (`kiro/converters_openai.py`)

Message Conversion

OpenAI messages are transformed into Kiro conversationState:

System prompt — added to the first user message
Message history — fully passed in history array
Adjacent message merging — messages with the same role are merged
Tool calls — OpenAI tools format support
Tool results — correct transmission of tool call results

Long Tool Description Handling

Problem: Kiro API returns error 400 for too long descriptions in toolSpecification.description.

Solution: Tool Documentation Reference Pattern

If description ≤ TOOL_DESCRIPTION_MAX_LENGTH → leave as is
If description > TOOL_DESCRIPTION_MAX_LENGTH:
- In toolSpecification.description → reference: "[Full documentation in system prompt under '## Tool: {name}']"
- In system prompt, section "## Tool: {name}" with full description is added

Function: process_tools_with_long_descriptions(tools) → (processed_tools, tool_documentation)

Main Functions

Function	Description
`extract_text_content(content)`	Extract text from various formats
`merge_adjacent_messages(messages)`	Merge adjacent messages with same role
`build_kiro_history(messages, model_id)`	Build history array for Kiro
`build_kiro_payload(request_data, conversation_id, profile_arn)`	Full payload for request

Model Mapping

External model names are converted to internal Kiro IDs:

External Name	Internal Kiro ID
`claude-opus-4-5`	`claude-opus-4.5`
`claude-opus-4-5-20251101`	`claude-opus-4.5`
`claude-haiku-4-5`	`claude-haiku-4.5`
`claude-haiku-4.5`	`claude-haiku-4.5` (direct passthrough)
`claude-sonnet-4-5`	`CLAUDE_SONNET_4_5_20250929_V1_0`
`claude-sonnet-4-5-20250929`	`CLAUDE_SONNET_4_5_20250929_V1_0`
`claude-sonnet-4`	`CLAUDE_SONNET_4_20250514_V1_0`
`claude-sonnet-4-20250514`	`CLAUDE_SONNET_4_20250514_V1_0`
`claude-3-7-sonnet-20250219`	`CLAUDE_3_7_SONNET_20250219_V1_0`
`auto`	`claude-sonnet-4.5` (alias)

3.7. Parsing Layer (`kiro/parsers.py`)

AwsEventStreamParser

Advanced AWS SSE format parser with support for:

Bracket counting — correct parsing of nested JSON objects
Content deduplication — filtering of duplicate events
Tool calls — parsing of structured and bracket-style tool calls
Escape sequences — decoding of \n and others

Event Types

Event	Description
`content`	Text content of the response
`tool_start`	Start of tool call (name, toolUseId)
`tool_input`	Continuation of input for tool call
`tool_stop`	End of tool call
`usage`	Credit consumption information
`context_usage`	Context usage percentage

Helper Functions

Function	Description
`find_matching_brace(text, start_pos)`	Find closing brace with nesting support
`parse_bracket_tool_calls(response_text)`	Parse `[Called func with args: {...}]`
`deduplicate_tool_calls(tool_calls)`	Remove duplicate tool calls

3.8. Streaming (`kiro/streaming_openai.py`)

stream_kiro_to_openai

Async generator for transforming Kiro stream to OpenAI format.

Functionality:

Parse AWS SSE stream via AwsEventStreamParser
Form OpenAI chat.completion.chunk
Handle tool calls (structured and bracket-style)
Calculate usage based on contextUsagePercentage
Debug logging via debug_logger

collect_stream_response

Collects full response from streaming for non-streaming mode.

3.9. HTTP Client (`kiro/http_client.py`)

KiroHttpClient

Automatic error handling with exponential backoff:

Error Code	Action
`403`	Token refresh via `force_refresh()` + retry
`429`	Exponential backoff: `BASE_RETRY_DELAY * (2 ** attempt)`
`5xx`	Exponential backoff (up to MAX_RETRIES attempts)
Timeout	Exponential backoff

Delay formula: 1s, 2s, 4s (with BASE_RETRY_DELAY=1.0)

Methods:

request_with_retry(method, url, json_data, stream) — request with retry
close() — close client

Supports async context manager (async with).

3.10. Routes (`kiro/routes_openai.py`)

Endpoint	Method	Description
`/`	GET	Health check (status, message, version)
`/health`	GET	Detailed health check (status, timestamp, version)
`/v1/models`	GET	List of available models (requires API key)
`/v1/chat/completions`	POST	Chat completions (requires API key)

Authentication: Bearer token in Authorization header

3.11. Exception Handling (`kiro/exceptions.py`)

Function	Description
`sanitize_validation_errors(errors)`	Convert bytes to strings for JSON serialization
`validation_exception_handler(request, exc)`	Pydantic validation error handler (422)

3.12. Debug Logging (`kiro/debug_logger.py`)

Class: DebugLogger (singleton)

Activation: DEBUG_LAST_REQUEST=true in .env

Methods:

Method	Description
`prepare_new_request()`	Clear directory for new request
`log_request_body(body)`	Save incoming request
`log_kiro_request_body(body)`	Save request to Kiro API
`log_raw_chunk(chunk)`	Append raw chunk from Kiro
`log_modified_chunk(chunk)`	Append transformed chunk

Files in debug_logs/:

request_body.json — incoming request (OpenAI format)
kiro_request_body.json — request to Kiro API
response_stream_raw.txt — raw stream from Kiro
response_stream_modified.txt — transformed stream (OpenAI format)

3.13. Tokenizer (`kiro/tokenizer.py`)

Problem: Kiro API does not return token counts directly. Instead, the API only provides context_usage_percentage — the percentage of model context usage.

Solution: Tokenizer module based on tiktoken (OpenAI's Rust library) for fast token counting.

Features:

Uses cl100k_base encoding (GPT-4), close to Claude tokenization
Correction factor CLAUDE_CORRECTION_FACTOR = 1.15 for improved accuracy
Lazy initialization for faster imports
Fallback to rough estimation if tiktoken is unavailable

Token calculation formula in response:

total_tokens = context_usage_percentage × max_input_tokens  (from Kiro API)
completion_tokens = tiktoken(response)                       (our calculation)
prompt_tokens = total_tokens - completion_tokens             (subtraction)

Main functions:

Function	Description
`count_tokens(text)`	Count tokens in text
`count_message_tokens(messages)`	Count tokens in message list
`count_tools_tokens(tools)`	Count tokens in tool definitions
`estimate_request_tokens(messages, tools)`	Full request token estimation

Debug log:

[Usage] claude-opus-4-5: prompt_tokens=142211 (subtraction), completion_tokens=769 (tiktoken), total_tokens=142980 (API Kiro)

Accuracy: ~97-99.7% compared to API data.

3.14. Kiro API Endpoints

All URLs are dynamically formed based on the region:

Token Refresh: POST https://prod.{region}.auth.desktop.kiro.dev/refreshToken
List Models: GET https://q.{region}.amazonaws.com/ListAvailableModels
Generate Response: POST https://codewhisperer.{region}.amazonaws.com/generateAssistantResponse

4. Detailed Data Flow

4.1 Multi-API Overview

┌─────────────────────────────────────────────────────────────────┐
│                          CLIENTS                                │
│  ┌─────────────────────┐       ┌─────────────────────┐         │
│  │  OpenAI Client      │       │  Anthropic Client   │         │
│  └──────────┬──────────┘       └──────────┬──────────┘         │
└─────────────┼──────────────────────────────┼───────────────────┘
              │                              │
              │ POST /v1/chat/completions    │ POST /v1/messages
              ▼                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                      API LAYER                                  │
│  ┌─────────────────────┐       ┌─────────────────────┐         │
│  │  routes_openai.py   │       │ routes_anthropic.py │         │
│  │  Security Gate      │       │ Security Gate       │         │
│  └──────────┬──────────┘       └──────────┬──────────┘         │
│             │                              │                    │
│             ▼                              ▼                    │
│  ┌─────────────────────┐       ┌─────────────────────┐         │
│  │converters_openai.py │       │converters_anthropic │         │
│  │ Extract system      │       │ System already      │         │
│  │ from messages       │       │ separate in request │         │
│  └──────────┬──────────┘       └──────────┬──────────┘         │
└─────────────┼──────────────────────────────┼───────────────────┘
              │                              │
              └──────────────┬───────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                      CORE LAYER                                 │
│             ┌─────────────────────────────┐                     │
│             │    converters_core.py       │                     │
│             │  build_kiro_payload()       │                     │
│             │  build_kiro_history()       │                     │
│             │  process_tools()            │                     │
│             └──────────────┬──────────────┘                     │
└────────────────────────────┼────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                     SHARED LAYER                                │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐ │
│  │ KiroAuthManager │  │ KiroHttpClient  │  │  ModelInfoCache │ │
│  │   (auth.py)     │  │(http_client.py) │  │   (cache.py)    │ │
│  └────────┬────────┘  └────────┬────────┘  └─────────────────┘ │
└───────────┼────────────────────┼────────────────────────────────┘
            │                    │
            │                    │ POST /generateAssistantResponse
            │                    ▼
            │         ┌─────────────────────────────────────┐
            │         │              Kiro API                   │
            │         └──────────────────┬──────────────────────┘
            │                            │
            │                            │ AWS SSE Stream
            │                            ▼
┌───────────┼────────────────────────────────────────────────────┐
│           │            CORE LAYER                              │
│           │  ┌─────────────────────────────┐                   │
│           │  │    streaming_core.py        │                   │
│           │  │  parse_kiro_stream()        │                   │
│           │  │  → KiroEvent objects        │                   │
│           │  └──────────────┬──────────────┘                   │
└────────────────────────────┼───────────────────────────────────┘
                             │
              ┌──────────────┴───────────────┐
              │                              │
              ▼                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                      OUTPUT LAYER                               │
│  ┌─────────────────────┐       ┌─────────────────────┐         │
│  │streaming_openai.py  │       │streaming_anthropic  │         │
│  │ format_openai_sse() │       │format_anthropic_sse │         │
│  │                     │       │                     │         │
│  │ data: {...}         │       │ event: type         │         │
│  │ data: [DONE]        │       │ data: {...}         │         │
│  └──────────┬──────────┘       └──────────┬──────────┘         │
└─────────────┼──────────────────────────────┼───────────────────┘
              │                              │
              ▼                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                          CLIENTS                                │
│  ┌─────────────────────┐       ┌─────────────────────┐         │
│  │  OpenAI Client      │       │  Anthropic Client   │         │
│  └─────────────────────┘       └─────────────────────┘         │
└─────────────────────────────────┘

4.2 OpenAI API Flow

OpenAI Client
     │ POST /v1/chat/completions
     ▼
routes_openai.py ──► converters_openai.py ──► converters_core.py
     │                                              │
     │                                              ▼
     │                                        Kiro Payload
     │                                              │
     ▼                                              ▼
KiroAuthManager ──────────────────────────► KiroHttpClient
                                                   │
                                                   ▼
                                              Kiro API
                                                   │
                                                   ▼
streaming_core.py ◄─────────────────────── AWS SSE Stream
     │
     ▼
streaming_openai.py
     │
     ▼
OpenAI SSE Format ──────────────────────► OpenAI Client

4.3 Anthropic API Flow

Anthropic Client
     │ POST /v1/messages
     ▼
routes_anthropic.py ──► converters_anthropic.py ──► converters_core.py
     │                                                    │
     │                                                    ▼
     │                                              Kiro Payload
     │                                                    │
     ▼                                                    ▼
KiroAuthManager ──────────────────────────────────► KiroHttpClient
                                                         │
                                                         ▼
                                                    Kiro API
                                                         │
                                                         ▼
streaming_core.py ◄─────────────────────────────── AWS SSE Stream
     │
     ▼
streaming_anthropic.py
     │
     ▼
Anthropic SSE Format ──────────────────────────► Anthropic Client

5. Available Models

Model	Description	Credits
`claude-opus-4-5`	Top-tier model	~2.2
`claude-opus-4-5-20251101`	Top-tier model (version)	~2.2
`claude-sonnet-4-5`	Enhanced model	~1.3
`claude-sonnet-4-5-20250929`	Enhanced model (version)	~1.3
`claude-sonnet-4`	Balanced model	~1.3
`claude-sonnet-4-20250514`	Balanced (version)	~1.3
`claude-haiku-4-5`	Fast model	~0.4
`claude-3-7-sonnet-20250219`	Legacy model	~1.0

6. Configuration

Environment Variables (.env)

# Required
REFRESH_TOKEN="your_kiro_refresh_token"
PROXY_API_KEY="your_proxy_secret"

# Optional
PROFILE_ARN="arn:aws:codewhisperer:..."
KIRO_REGION="us-east-1"
KIRO_CREDS_FILE="~/.aws/sso/cache/kiro-auth-token.json"

# Debug
DEBUG_LAST_REQUEST="false"
DEBUG_DIR="debug_logs"

# Limits
TOOL_DESCRIPTION_MAX_LENGTH="10000"

JSON Credentials File (optional)

{
  "accessToken": "eyJ...",
  "refreshToken": "eyJ...",
  "expiresAt": "2025-01-12T23:00:00.000Z",
  "profileArn": "arn:aws:codewhisperer:us-east-1:...",
  "region": "us-east-1"
}

7. API Endpoints

7.1 Common Endpoints

Endpoint	Method	Description
`/`	GET	Health check
`/health`	GET	Detailed health check

7.2 OpenAI-compatible Endpoints

Endpoint	Method	Description
`/v1/models`	GET	List of available models
`/v1/chat/completions`	POST	Chat completions (streaming/non-streaming)

Authentication: Authorization: Bearer {PROXY_API_KEY}

7.3 Anthropic-compatible Endpoints

Endpoint	Method	Description
`/v1/messages`	POST	Messages API (streaming/non-streaming)

Authentication: x-api-key: {PROXY_API_KEY} + anthropic-version: 2023-06-01

7.4 Format Comparison

Aspect	OpenAI	Anthropic
System prompt	In `messages` with `role: "system"`	Separate `system` field
Content	String or array	Always array of content blocks
Stop reason	`finish_reason: "stop"`	`stop_reason: "end_turn"`
Usage	`prompt_tokens`, `completion_tokens`	`input_tokens`, `output_tokens`
Streaming	`data: {...}\n\n` + `data: [DONE]`	`event: type\ndata: {...}\n\n`
Tool format	`{type: "function", function: {...}}`	`{name: "...", input_schema: {...}}`

8. Implementation Features

Tool Calling

Support for OpenAI-compatible tools format:

{
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get weather for a location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {"type": "string"}
        }
      }
    }
  }]
}

Streaming

Full SSE streaming support with correct OpenAI format:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...}

data: [DONE]

Debugging

When DEBUG_LAST_REQUEST=true, all requests and responses are logged in debug_logs/:

request_body.json — incoming request
kiro_request_body.json — request to Kiro API
response_stream_raw.txt — raw stream from Kiro
response_stream_modified.txt — transformed stream

9. Extensibility

Adding a New API Format

The modular architecture allows easy addition of support for other API formats. Thanks to the Core Layer, most of the logic is already implemented.

Steps to Add a New Format (e.g., Gemini)

Create models — models_gemini.py

class GeminiRequest(BaseModel):
    """Pydantic model for Gemini request."""
    contents: List[GeminiContent]
    ...

Create conversion adapter — converters_gemini.py

from kiro.converters_core import build_kiro_payload

def gemini_to_kiro(request: GeminiRequest, ...) -> dict:
    """Converts Gemini request to Kiro payload."""
    # Extract data from Gemini format
    system_prompt = extract_system_instruction(request)
    messages = convert_gemini_contents(request.contents)
    tools = convert_gemini_tools(request.tools)
    
    # Use shared core
    return build_kiro_payload(
        messages=messages,
        system_prompt=system_prompt,
        tools=tools,
        ...
    )

Create streaming formatter — streaming_gemini.py

from kiro.streaming_core import parse_kiro_stream

async def stream_to_gemini(response, ...) -> AsyncGenerator[str, None]:
    """Formats Kiro events to Gemini SSE."""
    async for event in parse_kiro_stream(response):
        yield format_gemini_chunk(event)

Create routes — routes_gemini.py

router = APIRouter()

@router.post("/v1beta/models/{model}:generateContent")
async def generate_content(request: GeminiRequest):
    ...

Connect in main.py

from kiro.routes_gemini import router as gemini_router
app.include_router(gemini_router)

What Gets Reused Automatically

When adding a new format, the following components work out of the box:

Component	Functionality
`auth.py`	Kiro token management
`http_client.py`	HTTP with retry logic
`cache.py`	Model cache
`parsers.py`	AWS SSE parsing
`tokenizer.py`	Token counting
`converters_core.py`	Kiro payload building
`streaming_core.py`	Kiro stream parsing

10. Dependencies

Main project dependencies (from requirements.txt):

Package	Purpose
`fastapi`	Asynchronous web framework
`uvicorn`	ASGI server
`httpx`	Asynchronous HTTP client
`pydantic`	Data validation and models
`python-dotenv`	Environment variable loading
`loguru`	Advanced logging
`tiktoken`	Fast token counting