README.md

metadata

title: Modular GAIA Level-1 Agent Source
colorFrom: indigo
colorTo: blue
sdk: static
app_file: index.html
pinned: false

This free static Space publishes the complete inspectable agent source. Run python app.py locally for the Gradio evaluation UI; no paid hosted hardware is required.

Modular GAIA Level-1 Agent

Production-oriented Hugging Face Agents Course final assignment. It preserves the official OAuth username and submission payload while separating checkpointed dry-run evaluation, review/reruns, and explicit submission.

Space configuration

Set HF_TOKEN as a Space secret. No credentials are stored in source code.

Supported variables:

• MODEL_ID or GAIA_MODEL_ID
• HF_PROVIDER or HF_INFERENCE_PROVIDER
• FALLBACK_MODEL_ID (defaults to openai/gpt-oss-20b) and HF_FALLBACK_PROVIDER
• GAIA_VISION_MODEL_ID and GAIA_ASR_MODEL_ID
• GAIA_API_URL, GAIA_CACHE_DIR, GAIA_RESULTS_PATH, GAIA_USE_CACHE
• GAIA_RETRIES, GAIA_BACKOFF_SECONDS, GAIA_REQUEST_TIMEOUT, GAIA_MAX_STEPS
• GAIA_WORKERS, GAIA_MODEL_REQUESTS_PER_MINUTE, GAIA_SEARCH_REQUESTS_PER_MINUTE
• STOCKFISH_PATH when Stockfish is not on PATH
• GAIA_AGENT_CODE_URL for local submission with a public GitHub/Hugging Face code URL
• GAIA_ALLOW_INLINE_AGENT_CODE=1 to explicitly send a local source bundle instead
• GAIA_LOCAL_MODEL_ID for an optional OpenAI-compatible local/Ollama fallback
• GAIA_LOCAL_MODEL_URL (defaults to http://127.0.0.1:11434/v1)
• GAIA_PREFER_LOCAL_MODEL=1 to avoid hosted inference calls for text tasks

packages.txt installs Stockfish in the Space. Chess evaluation fails closed if an engine is unavailable. Keep the Space public so its submitted agent_code is visible.

Local validation and dry-run

python -m pip install -r requirements-dev.txt
python -m pytest -q
python run_local_eval.py
python run_local_eval.py --force --task-id TASK_ID

Run and submit locally without paid Space hardware

The scoring API is public, so evaluation and explicit submission can run locally. Authenticate and expose the CLI token to the process (PowerShell example):

hf auth login
$env:HF_TOKEN = hf auth token --quiet
$env:GAIA_AGENT_CODE_URL = "https://github.com/YOUR_NAME/YOUR_PUBLIC_REPO"
python app.py

GAIA_AGENT_CODE_URL is recommended: it satisfies the course's inspectable-code intent without running a hosted Space. If no public repository is available, the current API schema also accepts Python source in agent_code; opt into that behavior explicitly:

$env:GAIA_ALLOW_INLINE_AGENT_CODE = "1"
python app.py

Inline mode bundles only the declared implementation .py files. It never reads .env, cached attachments, results, or credentials. In either mode, the app still requires a successful 20-question dry run and a separate click on the submission button.

The dry-run CLI cannot submit. The only /submit POST is encapsulated by GaiaClient.submit_answers() and reached from Submit Reviewed Complete Run. Submission requires exactly 20 unique, successful, non-empty answers.

Architecture

• GaiaClient exposes get_questions, get_random_question, download_task_file, and submit_answers, with typed errors and attachment metadata.
• TaskRouter selects deterministic file/media/logic/text specialists before inference.
• A managed ToolCallingAgent performs web, Wikipedia, HTML, and PDF research under a planning CodeAgent; only candidate answers leave the evidence record.
• YouTube supports captions and scene/interval frame sampling. Audio uses Whisper ASR.
• Excel arithmetic, Python stdout, Markdown algebra, text transforms, and Stockfish move selection are deterministic.
• results/results.json is keyed by task ID and records status, task type, duration, evidence, confidence, and errors after every task. Independent tasks run concurrently; shared model calls are rate-limited and serialized safely.