README.md

metadata

title: InferencePort Relay
emoji: 🔁
colorFrom: indigo
colorTo: purple
sdk: docker
pinned: false

InferencePort Relay

FastAPI endpoint that accepts a system prompt + conversation history, trims to the last 3 message pairs, and forwards to InferencePort AI.

Setup

Add a secret in your HF Space settings:

Name	Value
INFERENCE_PORT_API_KEY	your InferencePort API key

POST /chat

{
  "system": "your system prompt here",
  "messages": [
    { "role": "user", "content": "hi" },
    { "role": "assistant", "content": "yo wsp" },
    { "role": "user", "content": "good hbu" }
  ]
}

Returns:

{
  "reply": "not bad lol",
  "trimmed_message_count": 3
}