Spaces:
No application file
No application file
| # 🤖 GAIA Benchmark Agent (LangGraph) | |
| This is a LangGraph-powered agent for the [HuggingFace Agents Course](https://huggingface.co/learn/agents-course) Final Assignment. The agent is designed to solve GAIA benchmark questions and achieve a 30%+ score on Level 1 questions to earn the course certificate. | |
| ## 🎯 Goal | |
| **Score 30% or higher** (6+ correct out of 20 questions) on the GAIA Level 1 benchmark to earn your Certificate of Completion. | |
| ## Architecture | |
| ``` | |
| ┌─────────────────────────────────────────────────────────┐ | |
| │ LangGraph Workflow │ | |
| ├─────────────────────────────────────────────────────────┤ | |
| │ │ | |
| │ ┌─────────┐ ┌─────────┐ ┌──────────────┐ │ | |
| │ │ START │────▶│ Agent │────▶│ Should │ │ | |
| │ └─────────┘ │ Node │ │ Continue? │ │ | |
| │ └─────────┘ └──────────────┘ │ | |
| │ ▲ │ │ │ | |
| │ │ Yes │ │ No │ | |
| │ │ ▼ │ │ | |
| │ ┌─────────┐ ┌─────────▼────┐ │ | |
| │ │ Tool │◀────│ Extract │ │ | |
| │ │ Node │ │ Answer │ │ | |
| │ └─────────┘ └──────────────┘ │ | |
| │ │ │ | |
| │ ▼ │ | |
| │ ┌─────────┐ │ | |
| │ │ END │ │ | |
| │ └─────────┘ │ | |
| └─────────────────────────────────────────────────────────┘ | |
| ``` | |
| ## Available Tools | |
| | Tool | Description | Use Case | | |
| |------|-------------|----------| | |
| | 🔍 `web_search` | DuckDuckGo web search | Current information, recent events, facts | | |
| | 📚 `wikipedia_search` | Wikipedia API | Historical facts, biographies, definitions | | |
| | 🐍 `python_executor` | Python REPL | Calculations, data processing, analysis | | |
| | 📄 `read_file` | File reader | PDFs, text files, Excel spreadsheets | | |
| | 🔢 `calculator` | Math evaluator | Quick mathematical calculations | | |
| ## 🚀 Setup | |
| ### Option 1: HuggingFace Spaces (Recommended for Certification) | |
| 1. **Fork/Duplicate this Space** to your HuggingFace account | |
| - Go to the Space and click "Duplicate this Space" | |
| - Choose a name and make it **Public** (required for certification) | |
| 2. **Add API Key** | |
| - Go to Space Settings > Secrets | |
| - Add a new secret: `OPENAI_API_KEY` with your OpenAI API key value | |
| - Click "Save secrets" | |
| 3. **Deploy** | |
| - The Space will automatically build and deploy | |
| - Wait for the build to complete (usually 2-5 minutes) | |
| 4. **Test and Submit** | |
| - Open the Space and test with a single question | |
| - Run the full benchmark | |
| - Submit to the leaderboard | |
| ### Option 2: Local Development | |
| ```bash | |
| # Clone the repository | |
| git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE | |
| cd YOUR_SPACE | |
| # Create virtual environment (recommended) | |
| python -m venv venv | |
| source venv/bin/activate # On Windows: venv\Scripts\activate | |
| # Install dependencies | |
| pip install -r requirements.txt | |
| # Set environment variable | |
| export OPENAI_API_KEY="sk-..." # On Windows: set OPENAI_API_KEY=sk-... | |
| # Run the app | |
| python app.py | |
| ``` | |
| The app will be available at `http://localhost:7860` | |
| ## 📖 Usage | |
| ### 1. Test Single Question | |
| - Click "Fetch & Solve Random Question" to test the agent on one question | |
| - Review the answer and validation status | |
| - This helps verify the agent is working correctly before running the full benchmark | |
| ### 2. Run Full Benchmark | |
| - Click "Run Agent on All Questions" | |
| - The process takes approximately 10-15 minutes | |
| - Progress is shown in real-time | |
| - Results are displayed in a table | |
| - Answers are automatically formatted for submission | |
| ### 3. Submit to Leaderboard | |
| - After running the benchmark, go to the "Submit to Leaderboard" tab | |
| - Enter your HuggingFace username | |
| - Enter your Space URL (must be public and end with `/tree/main`) | |
| - Answers JSON is auto-filled | |
| - Click "Submit to Leaderboard" | |
| - View your score and ranking | |
| ## 🎓 Tips for Better Scores | |
| ### Answer Formatting (Critical!) | |
| The GAIA benchmark uses **exact string matching**. Your answers must match the ground truth character-for-character. | |
| **✅ DO:** | |
| - Give just the number: `"42"` | |
| - Use exact spelling: `"John Smith"` | |
| - Comma-separated lists with NO spaces: `"apple,banana,cherry"` | |
| - Just "Yes" or "No" (capitalized) | |
| - Follow the date format specified in the question | |
| **❌ DON'T:** | |
| - Include prefixes like "FINAL ANSWER:" or "The answer is:" | |
| - Add explanations or context | |
| - Use different capitalization or spelling | |
| - Add spaces in comma-separated lists | |
| - Include units unless specifically requested | |
| ### Agent Strategy | |
| 1. **File Priority**: If a file is available, the agent reads it first - answers are often in the file | |
| 2. **Tool Selection**: The agent automatically chooses the best tool for each task | |
| 3. **Iteration Limit**: The agent has up to 15 iterations to solve each question | |
| 4. **Error Handling**: The agent gracefully handles errors and tries alternative approaches | |
| ### Best Practices | |
| 1. **Test First**: Always test with a single question before running the full benchmark | |
| 2. **Review Answers**: Check the validation status for each answer | |
| 3. **Verify Format**: Ensure answers don't contain prefixes or explanations | |
| 4. **Public Space**: Keep your Space public so the code link works for verification | |
| 5. **API Key**: Ensure your OpenAI API key has sufficient credits | |
| ## ⚙️ Configuration | |
| ### Modifying the Agent | |
| The agent can be customized in `agent_enhanced.py`: | |
| - **Model**: Change `model_name` in `GAIAAgent.__init__()` (default: "gpt-4o") | |
| - **Temperature**: Adjust `temperature` (default: 0 for deterministic) | |
| - **Max Iterations**: Change `max_iterations` (default: 15) | |
| - **System Prompt**: Modify `SYSTEM_PROMPT` for different instructions | |
| - **Tools**: Add or remove tools from the `TOOLS` list | |
| ### Environment Variables | |
| - `OPENAI_API_KEY`: Required - Your OpenAI API key | |
| ## 🐛 Troubleshooting | |
| ### Common Issues | |
| **"Please provide your OpenAI API key"** | |
| - Ensure `OPENAI_API_KEY` is set in Space Secrets (for HF Spaces) or environment variables (for local) | |
| **"Failed to fetch questions from API"** | |
| - Check your internet connection | |
| - Verify the API URL is accessible: `https://agents-course-unit4-scoring.hf.space` | |
| - The API may be temporarily unavailable - try again later | |
| **"Agent error: ..."** | |
| - Check that your OpenAI API key is valid and has credits | |
| - Verify the model name is correct (e.g., "gpt-4o") | |
| - Review the error message for specific issues | |
| **"Submission error: ..."** | |
| - Ensure your Space URL is correct and public | |
| - Verify the URL ends with `/tree/main` (auto-added if missing) | |
| - Check that answers JSON is properly formatted | |
| - Ensure your HuggingFace username is correct | |
| **Low Scores (< 30%)** | |
| - Review answer formatting - exact matching is critical | |
| - Check that answers don't contain prefixes or explanations | |
| - Verify file reading is working (some questions require file analysis) | |
| - Consider increasing `max_iterations` for complex questions | |
| - Test with single questions to identify patterns | |
| ### Getting Help | |
| - Check the [Course Materials](https://huggingface.co/learn/agents-course/en/unit4/hands-on) | |
| - Review the [API Documentation](https://agents-course-unit4-scoring.hf.space/docs) | |
| - Check the [Student Leaderboard](https://huggingface.co/spaces/agents-course/Students_leaderboard) for examples | |
| - Review the [GAIA Benchmark Paper](https://huggingface.co/papers/2311.12983) | |
| ## 📁 Project Structure | |
| ``` | |
| certification/ | |
| ├── app.py # Gradio interface and main entry point | |
| ├── agent_enhanced.py # LangGraph agent implementation | |
| ├── requirements.txt # Python dependencies | |
| ├── README.md # This file | |
| └── .gitignore # Git ignore rules | |
| ``` | |
| ## 🔗 Important Links | |
| - [GAIA Benchmark Leaderboard](https://huggingface.co/spaces/gaia-benchmark/leaderboard) | |
| - [Student Leaderboard](https://huggingface.co/spaces/agents-course/Students_leaderboard) | |
| - [Course Unit 4 - Hands-On](https://huggingface.co/learn/agents-course/en/unit4/hands-on) | |
| - [API Documentation](https://agents-course-unit4-scoring.hf.space/docs) | |
| - [GAIA Paper](https://huggingface.co/papers/2311.12983) | |
| - [HuggingFace Agents Course](https://huggingface.co/learn/agents-course) | |
| ## 📊 Scoring | |
| - **Target**: 30%+ (6+ correct out of 20 questions) | |
| - **Evaluation**: Exact string matching | |
| - **Questions**: 20 Level 1 questions from GAIA validation set | |
| - **Submission**: Via the API endpoint `/submit` | |
| ## 🏆 Certification | |
| Once you achieve 30% or higher: | |
| 1. Your score will appear on the Student Leaderboard | |
| 2. You'll earn the Certificate of Completion | |
| 3. Share your achievement! | |
| ## 📝 License | |
| MIT License | |
| ## 🙏 Acknowledgments | |
| - Built for the [HuggingFace Agents Course](https://huggingface.co/learn/agents-course) | |
| - Uses [LangGraph](https://langchain-ai.github.io/langgraph/) for agent orchestration | |
| - Based on the [GAIA Benchmark](https://huggingface.co/papers/2311.12983) | |