Spaces:
Running
Running
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,10 +1,56 @@
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
colorTo: indigo
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
+
emoji: π‘
|
| 4 |
+
colorFrom: blue
|
| 5 |
colorTo: indigo
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
_Welcome to the official GSMA organization on Hugging Face!_
|
| 11 |
+
|
| 12 |
+
The [GSMA](https://www.gsma.com) represents mobile operators and organisations across the mobile ecosystem worldwide. We are building **open resources to advance AI in telecommunications** β making telecom-domain evaluation, benchmarking, and knowledge accessible to the global research community.
|
| 13 |
+
|
| 14 |
+
## Open Telco AI
|
| 15 |
+
|
| 16 |
+
[**Open Telco**](https://github.com/gsma-research/open_telco) is a comprehensive suite of telco-specific benchmarks built on the [Inspect AI](https://inspect.ai-safety-institute.org.uk/) framework, designed to ensure safe and optimal deployment of AI in telecommunications environments. A collaborative effort with major telecom providers, research institutions, and universities.
|
| 17 |
+
|
| 18 |
+
- **[ot-full](https://huggingface.co/datasets/GSMA/ot-full)** β 16,866 evaluation samples across 7 benchmarks β the complete evaluation suite
|
| 19 |
+
- **[ot-lite](https://huggingface.co/datasets/GSMA/ot-lite)** β 1,700 sample subset for fast iteration during model development
|
| 20 |
+
- **[Leaderboard Scores](https://huggingface.co/datasets/GSMA/leaderboard)** β Published benchmark scores with standard errors
|
| 21 |
+
|
| 22 |
+
### Benchmarks
|
| 23 |
+
|
| 24 |
+
The evaluation suite curates 7 telecom-domain benchmarks from academic and industry sources:
|
| 25 |
+
|
| 26 |
+
| Benchmark | Samples | Task |
|
| 27 |
+
|-----------|---------|------|
|
| 28 |
+
| **TeleQnA** | 10,000 | Multiple-choice Q&A on telecom standards |
|
| 29 |
+
| **TeleMath** | 1,500 | Mathematical reasoning in telecom contexts |
|
| 30 |
+
| **TeleTables** | 500 | Table interpretation from 3GPP specifications |
|
| 31 |
+
| **TeleLogs** | 586 | Log analysis and network troubleshooting |
|
| 32 |
+
| **3GPP TSG** | 3,780 | 3GPP Technical Specification Group document understanding |
|
| 33 |
+
| **ORANBench** | 200 | O-RAN architecture and specifications |
|
| 34 |
+
| **SRSRANBench** | 300 | srsRAN open-source network stack |
|
| 35 |
+
|
| 36 |
+
## Satellite β Eval Runner
|
| 37 |
+
|
| 38 |
+
[**Satellite**](https://github.com/gsma-labs/evals) provides telecom-focused evaluation operations built on Inspect AI. Run the full Open Telco benchmark suite locally within your own infrastructure with a single command.
|
| 39 |
+
|
| 40 |
+
## Telecom Simulation Sandboxes
|
| 41 |
+
|
| 42 |
+
Purpose-built sandbox environments that place AI agents inside live telecom network simulations β for evaluating whether models can *operate* networks, not just answer questions about them.
|
| 43 |
+
|
| 44 |
+
- **[inspect-kathara](https://github.com/gsma-labs/inspect-kathara)** β Run AI agent evaluations inside isolated network topologies. Integrates Inspect AI with Docker-based network sandboxes to evaluate agents' ability to diagnose and resolve network connectivity issues in reproducible environments.
|
| 45 |
+
|
| 46 |
+
- **[5gs-sandbox](https://github.com/gsma-labs/5gs-sandbox)** β Run AI agent evaluations inside a complete 5G Standalone network. A full 5G SA deployment with 15 Docker containers (Open5GS + UERANSIM), enabling agents to configure, diagnose, and optimize real 5G network functions with actual performance measurement.
|
| 47 |
+
|
| 48 |
+
## Datasets and Knowledge Resources
|
| 49 |
+
|
| 50 |
+
- **[telecom-kg-rel19](https://huggingface.co/datasets/GSMA/telecom-kg-rel19)** β Large-scale telecom knowledge graph built from 3GPP Release 19 specifications, with text chunks for retrieval-augmented generation (RAG) and LLM reasoning over standards
|
| 51 |
+
|
| 52 |
+
## Research and Community
|
| 53 |
+
|
| 54 |
+
- **GSMA AI Initiatives**: https://www.gsma.com/solutions-and-impact/technologies/artificial-intelligence/
|
| 55 |
+
- **Open Gateway**: https://www.gsma.com/solutions-and-impact/gsma-open-gateway/
|
| 56 |
+
- **MWC (Mobile World Congress)**: https://www.mwcbarcelona.com/
|