emolero commited on
Commit
b988434
Β·
verified Β·
1 Parent(s): 38bf8d3

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +49 -3
README.md CHANGED
@@ -1,10 +1,56 @@
1
  ---
2
  title: README
3
- emoji: πŸ“Š
4
- colorFrom: gray
5
  colorTo: indigo
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: README
3
+ emoji: πŸ“‘
4
+ colorFrom: blue
5
  colorTo: indigo
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
+ _Welcome to the official GSMA organization on Hugging Face!_
11
+
12
+ The [GSMA](https://www.gsma.com) represents mobile operators and organisations across the mobile ecosystem worldwide. We are building **open resources to advance AI in telecommunications** β€” making telecom-domain evaluation, benchmarking, and knowledge accessible to the global research community.
13
+
14
+ ## Open Telco AI
15
+
16
+ [**Open Telco**](https://github.com/gsma-research/open_telco) is a comprehensive suite of telco-specific benchmarks built on the [Inspect AI](https://inspect.ai-safety-institute.org.uk/) framework, designed to ensure safe and optimal deployment of AI in telecommunications environments. A collaborative effort with major telecom providers, research institutions, and universities.
17
+
18
+ - **[ot-full](https://huggingface.co/datasets/GSMA/ot-full)** β€” 16,866 evaluation samples across 7 benchmarks β€” the complete evaluation suite
19
+ - **[ot-lite](https://huggingface.co/datasets/GSMA/ot-lite)** β€” 1,700 sample subset for fast iteration during model development
20
+ - **[Leaderboard Scores](https://huggingface.co/datasets/GSMA/leaderboard)** β€” Published benchmark scores with standard errors
21
+
22
+ ### Benchmarks
23
+
24
+ The evaluation suite curates 7 telecom-domain benchmarks from academic and industry sources:
25
+
26
+ | Benchmark | Samples | Task |
27
+ |-----------|---------|------|
28
+ | **TeleQnA** | 10,000 | Multiple-choice Q&A on telecom standards |
29
+ | **TeleMath** | 1,500 | Mathematical reasoning in telecom contexts |
30
+ | **TeleTables** | 500 | Table interpretation from 3GPP specifications |
31
+ | **TeleLogs** | 586 | Log analysis and network troubleshooting |
32
+ | **3GPP TSG** | 3,780 | 3GPP Technical Specification Group document understanding |
33
+ | **ORANBench** | 200 | O-RAN architecture and specifications |
34
+ | **SRSRANBench** | 300 | srsRAN open-source network stack |
35
+
36
+ ## Satellite β€” Eval Runner
37
+
38
+ [**Satellite**](https://github.com/gsma-labs/evals) provides telecom-focused evaluation operations built on Inspect AI. Run the full Open Telco benchmark suite locally within your own infrastructure with a single command.
39
+
40
+ ## Telecom Simulation Sandboxes
41
+
42
+ Purpose-built sandbox environments that place AI agents inside live telecom network simulations β€” for evaluating whether models can *operate* networks, not just answer questions about them.
43
+
44
+ - **[inspect-kathara](https://github.com/gsma-labs/inspect-kathara)** β€” Run AI agent evaluations inside isolated network topologies. Integrates Inspect AI with Docker-based network sandboxes to evaluate agents' ability to diagnose and resolve network connectivity issues in reproducible environments.
45
+
46
+ - **[5gs-sandbox](https://github.com/gsma-labs/5gs-sandbox)** β€” Run AI agent evaluations inside a complete 5G Standalone network. A full 5G SA deployment with 15 Docker containers (Open5GS + UERANSIM), enabling agents to configure, diagnose, and optimize real 5G network functions with actual performance measurement.
47
+
48
+ ## Datasets and Knowledge Resources
49
+
50
+ - **[telecom-kg-rel19](https://huggingface.co/datasets/GSMA/telecom-kg-rel19)** β€” Large-scale telecom knowledge graph built from 3GPP Release 19 specifications, with text chunks for retrieval-augmented generation (RAG) and LLM reasoning over standards
51
+
52
+ ## Research and Community
53
+
54
+ - **GSMA AI Initiatives**: https://www.gsma.com/solutions-and-impact/technologies/artificial-intelligence/
55
+ - **Open Gateway**: https://www.gsma.com/solutions-and-impact/gsma-open-gateway/
56
+ - **MWC (Mobile World Congress)**: https://www.mwcbarcelona.com/