Modelflare Blog

Modelflare API recipes: streaming, routing fallback, images, and async video

Request examples for streaming chat, gateway-side fallback with group failover and Smart API Keys, client-side model fallback, image generation, and the asynchronous video task flow.

Diagram: parallel routes converging with fallback paths

The recipes below use Modelflare's OpenAI-compatible base URL, https://modelflare.dev/v1, and a placeholder key. Each model ID must be enabled on the key's routing group; check GET /v1/models or Models & Pricing before sending requests.

from openai import OpenAI
client = OpenAI(api_key="YOUR_MODELFLARE_API_KEY", base_url="https://modelflare.dev/v1")

Streaming chat completions

The streaming request follows the Modelflare/api-examples dataset: it sets "stream": true on Chat Completions and consumes server-sent events the same way the OpenAI SDK does with stream=True. The Modelflare documentation does not include a separate streaming example.

stream = client.chat.completions.create(
    model="gpt-5.6-sol",
    messages=[{"role": "user", "content": "Say hi in one line"}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content if chunk.choices else None
    if delta:
        print(delta, end="", flush=True)

The equivalent curl request:

curl -N https://modelflare.dev/v1/chat/completions \
  -H "Authorization: Bearer YOUR_MODELFLARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Say hi in one line"}],"stream":true}'

Streaming interacts with failover in one specific way. Once a streamed response has written content bytes to the client, the gateway does not replay the request on another route; heartbeats alone do not count as content. A stream that fails after content has started must be retried by the client.

Fallback at the gateway

Modelflare implements fallback in the routing layer, so the request body does not change.

An ordinary key names a primary routing group and, optionally, fallback groups in priority order. The chain is configured in API Keys and can be reordered there. The gateway moves to the next group when the current group has no channel for the requested model, when the group is at its rate limit for the account, or when every channel priority in the group has returned a retryable error. It does not fail over on client errors (malformed JSON, an invalid parameter, context length exceeded), on a cancelled client connection, on a request pinned to one channel, or on a stream that has already sent content. The request is billed at the multiplier of the group that served it, and that group is recorded in the consume log. A key whose primary group is a limited-time campaign route must have at least one fallback group before it can be saved.

A Smart API Key is bound to auto. On each request it collects the groups the account can use, keeps those that serve the requested model, orders them by the key's strategy (Balanced, the default; Stability first; or Low price first), and tries them in order. Smart API Keys do not take a manual fallback list. Client code is unchanged: same base URL, same Authorization: Bearer header.

Diagram: primary group followed by two fallback groups, with the conditions that trigger and stop failover Group failover on an ordinary key: when the gateway moves to the next group, and when it stops.

Fallback across models in client code

Switching to a different model after a failure is application logic, not a gateway feature. A minimal version:

from openai import APIStatusError

MODELS = ["gpt-5.6-sol", "YOUR_SECOND_MODEL_ID"]  # exact IDs your key can access

def chat_with_fallback(messages):
    last = None
    for m in MODELS:
        try:
            return client.chat.completions.create(model=m, messages=messages)
        except APIStatusError as e:
            last = e
            if e.status_code in (400, 401, 403):   # bad request, bad key, model/group not allowed
                raise
    raise last

Status codes 400, 401, and 403 indicate a malformed request, an invalid key, or a model or group the key may not use. They are raised immediately rather than masked by a retry.

Image generation

curl https://modelflare.dev/v1/images/generations \
  -H "Authorization: Bearer YOUR_MODELFLARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-image-2",
    "prompt": "A white cat runs through a neon-lit street in the rain",
    "size": "1024x1024", "n": 1
  }'

gpt-image-2 returns Base64 image data in data[].b64_json. This endpoint serves GPT Images, Wan Image, and Seedream; the accepted fields vary by model and are listed on each image model page. Gemini image models use the native POST /v1/models/{model}:generateContent endpoint instead. Requests sent to the wrong family are rejected with 400 unsupported_endpoint. Use a Smart API Key or a key on the media group.

Asynchronous video

Diagram: video task created, then queued, in_progress, and completed or failed, followed by download Video task states returned by GET /v1/videos/{task_id}.

Video generation is a three-step task: create, poll, download.

# 1) create a task (Seedance example; the body differs per family)
curl https://modelflare.dev/v1/videos \
  -H "Authorization: Bearer YOUR_MODELFLARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "seedance-2.5",
    "content": [{"type": "text", "text": "A cat walks through a sunlit room"}],
    "resolution": "720p",
    "duration": 5,
    "ratio": "16:9"
  }'
# save "id" from the response (compatibility responses may use "task_id")

# 2) poll while status is "queued" or "in_progress"; stop at "completed" or "failed"
curl https://modelflare.dev/v1/videos/TASK_ID \
  -H "Authorization: Bearer YOUR_MODELFLARE_API_KEY"

# 3) download
curl --fail --location \
  https://modelflare.dev/v1/videos/TASK_ID/content \
  -H "Authorization: Bearer YOUR_MODELFLARE_API_KEY" \
  --output seedance-result.mp4

Task creation can return an ID before provider validation finishes, so a later poll may report failed; keep the request ID for investigation. The request body depends on the model family (Seedance, Wan, Kling, MiniMax, HappyHorse) and is documented on each video model page. Result URLs can expire, so store the file promptly.

Responses API

POST /v1/responses is the endpoint used by Codex and Grok Build. The documentation does not show a raw request body; the request below comes from the Modelflare/api-examples dataset.

curl https://modelflare.dev/v1/responses \
  -H "Authorization: Bearer YOUR_MODELFLARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-5.6-sol", "input": "Reply with one sentence confirming the model."}'

References

The Modelflare/api-recipes Space and the Modelflare/api-examples dataset contain these requests. Background reading: Which endpoint to call, Group failover, and Smart API Key. Higher limits or volume arrangements: business@modelflare.dev.

← All articles