retro / .env.example
sankalphs's picture
Add Modal GPU inference support for faster LLM responses
ad0ab13
Raw
History Blame Contribute Delete
426 Bytes
# LLM API for synthetic data generation (OpenAI-compatible)
API_KEY=your-api-key-here
BASE_URL=https://zenmux.ai/api/v1
MODEL=moonshotai/kimi-k2.7-code-free
# Hugging Face
HF_TOKEN=your-hf-token-here
# Modal
MODAL_TOKEN_ID=your-modal-token-id
MODAL_TOKEN_SECRET=your-modal-token-secret
# Set to use Modal GPU inference (leave empty for local llama.cpp)
# MODAL_INFERENCE_URL=https://your-modal-app--nemotron-chat.modal.run