Downloads/hackathon/meta/SETUP.md

🔧 Fix: Missing Environment Variables Error

Problem

Your inference.py is failing because it can't find these environment variables:

• API_BASE_URL - URL to your LLM API
• MODEL_NAME - Model identifier
• HF_TOKEN - Hugging Face authentication token

Solution

Step 1: Create/Edit the .env file

A .env file has been created in your project root. Edit it with your actual credentials:

# Open .env in your editor and fill in:
API_BASE_URL=https://api.together.xyz/v1     # Your LLM API endpoint
MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct  # Model you want to use
HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx            # Your HF token from https://huggingface.co/settings/tokens

Step 2: Verify Your Setup

Run the verification script to check if everything is configured:

python verify_env.py

Expected output:

✅ .env file found
✅ API_BASE_URL: https://api.together.xyz/v1
✅ MODEL_NAME: meta-llama/...
✅ HF_TOKEN: hf_xxxx...xxxxx
✅ All dependencies installed
✅ All checks passed! Ready to run inference.py

Step 3: Run Inference

Once verification passes, run your inference:

python inference.py

Environment Variable Options

API_BASE_URL (choose one):

Option 1: Together AI (Recommended for hackathons)

API_BASE_URL=https://api.together.xyz/v1

• Free tier available
• Sign up at: https://api.together.xyz

Option 2: Hugging Face Inference

API_BASE_URL=https://api-inference.huggingface.co/v1

• Use your HF token for authentication
• Limited free tier

Option 3: Local vLLM Server (Advanced)

API_BASE_URL=http://localhost:8000/v1

• Requires running vLLM locally
• Best for development

MODEL_NAME (examples):

• meta-llama/Llama-3.1-8B-Instruct ✅ Recommended
• meta-llama/Llama-3.2-70B-Instruct (more powerful)
• mistralai/Mistral-7B-Instruct-v0.3
• NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO

HF_TOKEN:

1. Go to https://huggingface.co/settings/tokens
2. Click "New token"
3. Select "Read" permission
4. Copy the token and paste in .env

Common Issues

❌ "curl: (6) Could not resolve host"

• Check your API_BASE_URL is correct and accessible
• Test: curl {API_BASE_URL}/models

❌ "401 Unauthorized" or "invalid_api_key"

• Your HF_TOKEN is wrong or expired
• Generate a new token at https://huggingface.co/settings/tokens

❌ "Model not found"

• Check your MODEL_NAME matches the provider's available models
• Visit the provider's documentation

How It Works

After the changes made to inference.py:

1. The script automatically loads variables from .env file (if exists)
2. Then it checks for required environment variables
3. If any are missing, it shows a helpful error message
4. If all are set, it proceeds with inference

This means you can:

• Use .env file (checked automatically)
• Export variables manually: export API_BASE_URL=...
• Mix both approaches (manual exports override .env)

Quick Commands (PowerShell)

# Edit .env file
code .env

# Verify environment
python verify_env.py

# Run inference (after .env is filled in)
python inference.py

Next Steps for Phase 2

1. ✅ Fill in .env with your credentials
2. ✅ Run python verify_env.py to confirm setup
3. ✅ Run python inference.py to complete phase 2
4. ✅ Check inference_results.txt for logs an results