README.md

---
title: GAIA Agent - GPT-6 Sol Precision
emoji: 🧠
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.49.1
python_version: 3.10
app_file: app.py
pinned: false
hf_oauth: true
hf_oauth_expiration_minutes: 480
---

# GAIA course — controlled-cost OpenAI agent

This is an **original student agent**, not an official solution. It uses **GPT-6 Sol**, a limited sequence of model calls, public web search, local document extraction, a calculator, and a separate Hugging Face scoring submission button. **There is no guarantee of a particular score.**

## Installation in your EXISTING ZeroGPU Space

1. On Hugging Face, open your already duplicated `Final_Assignment_Template` Space (keep it PUBLIC and on ZeroGPU).
2. Open **Files**, and replace/add **`app.py`**, **`agent.py`**, **`requirements.txt`**, and **`README.md`** at the repository root. Upload the contents of this ZIP, **not the ZIP itself**. You can remove old Gemini agent files if any, but keep Hugging Face metadata.
3. Go to **Settings → Variables and secrets → Secrets → New secret**. Name `OPENAI_API_KEY` and paste your **OpenAI Platform API key**. Do **NOT** put it in a plain variable or in public code.
4. Allow the Space to rebuild. In **App**, log in with Hugging Face.
5. Click **1. Check setup (FREE)**. This checks only the GAIA scoring API and presence of your key; it does not make an OpenAI model call.
6. Click **2. Test ONE question**. Check the cached answer and the cost estimate, then check *actual* API spending on https://platform.openai.com/usage .
7. **BEFORE UPLOADING THIS UPDATE, download your existing answer backup** from the currently running Space. If it restarts, its /tmp answer cache can vanish. After deploying, restore the saved `.json` in the Backup section and click Show saved answers to verify your earlier answers survived.
8. To answer more, tick the paid-request consent checkbox. In **Maximum supported questions to solve in one batch**, choose **3, 5, 10 or 20**; choose **20** only after checking actual API usage/balance. Press **3. Run selected batch (OpenAI API)**. Previously cached and previously skipped questions are not repeated. A missing GAIA attachment (`/files/{task_id}` HTTP 404/410) is marked skipped **without OpenAI charges**, and the agent continues with the next available question. Audio/video attachments are skipped because transcription is disabled.
9. After the batch, click **Download answer backup** again. The batch stops on OpenAI API errors and the best-effort local budget guard. It does NOT submit to the leaderboard automatically.
10. Once you have as many answers as possible, press **Submit answers to GAIA scoring**. You can explicitly opt to submit blank answers for remaining unsupported questions. Scoring occurs only upon this click, and blank answers count as wrong. The Unit 4 target is 30% (6 correct out of 20).

**Only one model is enabled:** `gpt-6-sol`, list prices checked October 28, 2026: **$2.00/M input tokens and $10.00/M output tokens**. The agent uses the **Responses API** with `reasoning.effort=medium` for function calling; built-in OpenAI tools are disabled (public web search is performed locally). The OpenAI API key is NOT the same as a ChatGPT subscription.

## Budget controls and safety

- Default `GAIA_BUDGET_USD=2.00` local estimated TOTAL, `GAIA_QUESTION_BUDGET_USD=0.55` local estimated per question. You can change these in **Space Settings → Variables** later if needed; starting low is intentional. Never put secrets in Variables.
- Maximum **4 paid model requests/question**, 5,000 output tokens/request (including reasoning), **8 locally run tools/question**. No implicit retries; OpenAI SDK retries disabled.
- Before each request, the app temporarily reserves **$0.12** against the estimated budget; after a successful call, it reconciles using the model's reported input/output token usage. Failed calls keep their reservation to avoid undercounting. This is a conservative approximation, not an absolute per-call billing limit.
- This guard **cannot enforce a hard limit on your OpenAI account**. OpenAI bills based on actual requests; actual costs could differ because of multimodal pricing, missing usage metadata, failures, deployment restarts, other apps/keys or changes in price. Set a project budget/alerts if available on the OpenAI dashboard and monitor its actual usage. Disable auto-recharge if you don't want it.
- The application **can** run up to 20 eligible questions in one explicitly authorized batch, but defaults to 3 and does not automatically buy or renew credits.
- Answer and local estimated spend files use **`/data` only when persistence is available**, otherwise temporary `/tmp` storage. On a Space restart, the latter may reset! Use **Download answer backup** and restore it if required. The budget estimate also resets in /tmp, so always check provider usage separately.
- On purpose, **audio/video file transcription is disabled** to avoid extra, untracked transcription charges. Text, PDF, DOCX, spreadsheet and image tasks are partially supported. This means some GAIA tasks are unsupported until you choose to enable additional services.
- For web search, the app uses `ddgs` via the Space itself, **not OpenAI paid web-search tools**. Search quality, page availability, YouTube captions, rate limits and exact-match accuracy are not guaranteed.

## Important troubleshooting

- `429` / insufficient quota: look at your OpenAI project limits. No automatic retries.
- `401` / wrong API key: verify that `OPENAI_API_KEY` is in **Secrets**.
- `503`: temporary provider overload. Stop and decide whether a manual retry is worthwhile.
- `404` from the official scoring `GET /files/{task_id}`: known server-side bug (https://github.com/huggingface/agents-course/issues/624). The app skips that task before sending an OpenAI request, and continues. Do not invent an answer to the missing attachment.
- `Maximum model calls reached`: question is too complex for the conservative configuration. It is deliberately **not cached**. Decide whether to increase limits later.
- Existing cached answers are **not re-run** accidentally; to retry one, select its question number, tick **I confirm clearing**, and click **Clear selected Q# answer**. This deletes the cached response without making any OpenAI API call. The next test will be billed normally.
- Certificate and submission details: https://huggingface.co/learn/agents-course/unit4/hands-on

The current version was syntax-checked and offline-tested for 404 skip, cache preservation, continuation past unavailable tasks, no model calls on missing attachments, and the scoring payload. It has **not** been executed against your OpenAI key or deployed live on your Space.

## Data-sharing token incentive

If the organization is eligible **and enrolled**, GPT-6 Sol is in the **250K daily shared complimentary-token group for Build tier**, NOT the 2.5M group. The free quota is not guaranteed. Tokens exceeding the quota and separately priced API tools are billable. In particular, this app uses only local web functions; it does not enable the OpenAI built-in web-search tool. See https://help.openai.com/en/articles/10306912-sharing-feedback-evaluation-and-fine-tuning-data-and-api-inputs-and-outputs-with-openai . Monitor real billing in OpenAI Platform.

Keep the previous answer backup before redeploying. The in-memory and /tmp caches may be reset; use the built-in **Restore answers** button after you deploy. Previous wrong answers will NOT automatically be regenerated. Choose **Clear selected Q#** to retry selected questions after creating a backup.

GPT-6 Sol uses Responses API with medium reasoning and strict JSON final answers. If the model outputs no answer after 5,000 tokens, this app deliberately does not save a blank answer; the call can still consume tokens.

## Precision upgrade vs the Luna version

- New evidence-first system prompt: check time periods, precise entity matches, calculation conditions and tie-break rules; cross-check ambiguous claims with primary sources where available.
- Responses API `text.format` enforces one `final_answer` JSON string. Only that string is cached or submitted, never the JSON wrapper, web queries, debug traces, or narration.
- Conservative output validation rejects explanations, tool traces and unsupported guesses without automatically retrying expensive calls. It strips *outer* quotation marks and unnecessary units on numerical count answers.
- Sol is **20x the standard token price of Luna**: keep the default $2 local budget and run one question first. The $0.12 per-call reservation is an estimate, **not a billing cap**; provider charges can exceed it.
- The original GAIA attachment-404 problem is **not fixed** here: missing attachments still get skipped. Some tasks cannot be solved accurately without them.
- If a previous answer backup is restored, old answers remain cached. Use Clear selected Q# intentionally and re-run only selected questions. No hidden answer keys or prefilled benchmark solutions.

### Model configuration

`agent.py` defines `MODEL = "gpt-6-sol"`, `REASONING_EFFORT = "medium"`, and the updated token prices. The exact updated prompt is in `SYSTEM_PROMPT`.


## v3 correction: empty answers no longer stop the batch

The previous prompt made the model return an empty `final_answer` whenever
it could not fully verify a source. This version instead requests a
best-supported answer when the facts permit one; it does **not** fabricate
sources or answers. For an inherently unanswerable question, a blank answer
is marked **Unresolved/Skipped**, not cached, and the batch proceeds. A
failed tool is reported in the skip reason, without automatic paid retries.

Search first uses DDGS and falls back to Bing RSS if needed. These external
web requests are not OpenAI billed tool calls (but network reliability
remains outside our control). Existing answers are not overwritten.

**Test Q1 on its own first, check the model response, and then run a small
batch.** The $2 estimated guard is not a hard provider billing limit.