fevziegeyurtsevenler commited on
Commit
d7589df
·
verified ·
1 Parent(s): 922aa75

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +118 -5
README.md CHANGED
@@ -1,10 +1,123 @@
1
  ---
2
- title: README
3
- emoji: 📊
4
- colorFrom: green
5
- colorTo: yellow
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
- Edit this `README.md` markdown file to author your organization card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: AltaySec
3
+ emoji: 🛡️
4
+ colorFrom: purple
5
+ colorTo: red
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
+ <p align="center">
11
+ <a href="https://altaysec.com.tr">
12
+ <img src="https://altaysec.com.tr/logo.jpg" alt="AltaySec" width="120" />
13
+ </a>
14
+ </p>
15
+
16
+ <h1 align="center">AltaySec</h1>
17
+
18
+ <p align="center">
19
+ <strong>Turkey's AI/LLM security company — home of open, Turkish-first adversarial datasets.</strong>
20
+ </p>
21
+
22
+ <p align="center">
23
+ Multi-turn prompt-injection red-teaming · OWASP LLM Top&nbsp;10 · MITRE ATLAS · KVKK-aware · CC-BY-4.0
24
+ </p>
25
+
26
+ <p align="center">
27
+ <a href="https://altaysec.com.tr"><img alt="website" src="https://img.shields.io/badge/web-altaysec.com.tr-7c3aed?style=flat-square&logo=googlechrome&logoColor=white"></a>
28
+ <a href="https://huggingface.co/AltaySec/datasets"><img alt="datasets" src="https://img.shields.io/badge/datasets-2%20open-ffcc00?style=flat-square&logo=huggingface&logoColor=black"></a>
29
+ <a href="https://doi.org/10.5281/zenodo.20681557"><img alt="paper" src="https://img.shields.io/badge/paper-cs.CR%20preprint-1f6feb?style=flat-square"></a>
30
+ <a href="https://duel.altaysec.com.tr"><img alt="arena" src="https://img.shields.io/badge/AltayDuel-live%20arena-d62828?style=flat-square"></a>
31
+ <img alt="license" src="https://img.shields.io/badge/license-CC--BY--4.0-22c55e?style=flat-square">
32
+ </p>
33
+
34
+ ---
35
+
36
+ AltaySec is a Turkish AI/LLM security company. We red-team large language models **in Turkish first** — a morphologically rich, under-represented language in adversarial NLP — and release the resulting corpora as open, reproducible safety benchmarks. This organization is the home of our open datasets; the research, tooling and arena behind them live at **[altaysec.com.tr](https://altaysec.com.tr)**.
37
+
38
+ ## 📊 Open datasets
39
+
40
+ | Dataset | What it is | Size | License |
41
+ |---|---|---|---|
42
+ | [**altayduel-transcripts**](https://huggingface.co/datasets/AltaySec/altayduel-transcripts) | Multi-turn *agent-vs-agent* prompt-injection duels (red attacker vs blue defender) with deterministic judge labels | 2,594 clean duels + 439 successful-attack subset (~3k rows) | CC-BY-4.0 |
43
+ | [**turkish-llm-injection**](https://huggingface.co/datasets/AltaySec/turkish-llm-injection) | Categorized Turkish prompt-injection payloads, 12 attack families × 25, mapped to OWASP LLM Top 10 | 300 payloads | CC-BY-4.0 |
44
+
45
+ ### 🥊 AltayDuel Transcripts
46
+
47
+ Real multi-turn dialogues (1–8 rounds, mean 4.6) in which an attacker LLM tries to make a defender LLM break its system prompt, scored by a **deterministic, mechanism-labelled judge** rather than an LLM judge — so labels are reproducible and auditable. Content is **56.7% Turkish**; defenders span Llama-3.1-8B, GPT-OSS-120B, Llama-3.3-70B and Llama-4-Maverick across Cerebras / SambaNova / Groq, plus externally-submitted agents. The `successful_attacks` config is labelled by *how* each attack won (`win_signal`): yield-under-pressure, partial leak, or verbatim secret leak.
48
+
49
+ ### 🇹🇷 Turkish LLM Injection
50
+
51
+ 300 hand-curated and generation-assisted payloads, balanced at exactly 25 per family across 12 families. Every record is annotated with `category`, `subcategory`, `owasp_llm_top10`, `severity`, `language`, `context` and `expected_failure_mode`. All identifier-like values (TCKN, IBAN, names) are **synthetic** — no real personal data.
52
+
53
+ ## 🧱 Turkish-specific attack taxonomy
54
+
55
+ What makes these corpora distinctive: four of the twelve families exploit properties rare in English red-team data.
56
+
57
+ - **morphological_bypass** — abuse of Turkish suffix chains and causative stacking (e.g. *okut-tur-uver*).
58
+ - **code_switching** — Turkish↔English instruction-override.
59
+ - **cultural_manipulation** — religious / national / institutional authority framing.
60
+ - **pii_exfiltration** — Turkish PII (TCKN, IBAN, plate) leakage vectors.
61
+
62
+ The remaining eight (`authority_urgency`, `confirmation_trap`, `echo_translation`, `roleplay_theater`, `system_prompt_extract`, `politeness_escalation`, `indirect_injection`, `encoding_obfuscation`) complete a taxonomy mapped to **OWASP LLM Top 10** and **MITRE ATLAS**.
63
+
64
+ ## 🔬 Key findings (from the paper)
65
+
66
+ - Overall attack-success rate **16.9%** (439 / 2,594).
67
+ - **Comparable across languages** — 16.0% on Turkish scenarios vs 18.2% on English; Turkish was *not* inherently weaker at the aggregate level.
68
+ - Attacks succeed mostly by **capitulation under pressure (51.3%)** and **partial leakage (41.7%)**, not verbatim secret disclosure (**7.1%**). For high-precision cases, filter `win_signal == "secret_leak"`.
69
+
70
+ > **Paper:** *AltayDuel: A Turkish-First Arena and Open Dataset for Multi-Turn LLM Prompt-Injection Red-Teaming* (cs.CR) · CC-BY-4.0 · DOI [10.5281/zenodo.20681557](https://doi.org/10.5281/zenodo.20681557)
71
+
72
+ ```bibtex
73
+ @misc{yurtsevenler2026altayduel,
74
+ title = {AltayDuel: A Turkish-First Arena and Open Dataset for
75
+ Multi-Turn LLM Prompt-Injection Red-Teaming},
76
+ author = {Yurtsevenler, Fevzi Ege},
77
+ year = {2026},
78
+ doi = {10.5281/zenodo.20681557},
79
+ howpublished = {AltaySec},
80
+ url = {https://altaysec.com.tr}
81
+ }
82
+ ```
83
+
84
+ ## 🎯 Use cases
85
+
86
+ Training Turkish prompt-injection / jailbreak **detection classifiers** · benchmarking guardrail **robustness** · **KVKK**-focused PII-leakage testing · studying **multi-turn social-engineering** dynamics · LLM-as-judge calibration research.
87
+
88
+ ## 🧩 Related open source
89
+
90
+ - **[tr-pii-detect](https://github.com/fevziegeyurtsevenler/tr-pii-detect)** — zero-dependency Python library for *algorithm-validated* Turkish PII detection & redaction (TCKN checksum, IBAN MOD-97, VKN, Luhn, BTK phone, plate), cutting regex false positives by 90%+.
91
+ - **Guardian** — our production LLM-security gateway (drop-in OpenAI/Anthropic-compatible proxy) is the commercial counterpart to these open assets: [altaysec.com.tr/urunler/guardian](https://altaysec.com.tr/urunler/guardian.html).
92
+
93
+ ## ⚠️ Ethics & licensing
94
+
95
+ All datasets are released under **CC-BY-4.0**. Every PII-like value is **synthetic**. The content is adversarial by nature and is intended for **defensive research and evaluation only** — responsible use required.
96
+
97
+ ## 🔗 Links
98
+
99
+ [Website](https://altaysec.com.tr) · [Research (36+ articles)](https://altaysec.com.tr/arastirmalar/) · [AltayDuel arena](https://duel.altaysec.com.tr) · [GitHub](https://github.com/AltaySec) · [LinkedIn](https://www.linkedin.com/company/altaysec/) · [info@altaysec.com.tr](mailto:info@altaysec.com.tr)
100
+
101
+ ---
102
+
103
+ ## 🇹🇷 Türkçe Özet
104
+
105
+ **AltaySec**, Türkiye merkezli bir yapay zeka / LLM güvenliği şirketidir. LLM'leri **Türkçe-öncelikli** kırmızı-takım (red-team) yöntemiyle test eder; ortaya çıkan veri setlerini açık ve tekrarlanabilir güvenlik kıyaslamaları olarak yayınlarız. Bu organizasyon, açık veri setlerimizin evidir.
106
+
107
+ **İki açık veri seti (CC-BY-4.0):**
108
+
109
+ - **[altayduel-transcripts](https://huggingface.co/datasets/AltaySec/altayduel-transcripts)** — çok-turlu *ajan-vs-ajan* prompt injection düelloları; 2.594 temiz düello + 439 başarılı saldırı alt-kümesi, deterministik hakem etiketleriyle (`win_signal`). İçeriğin %56,7'si Türkçe.
110
+ - **[turkish-llm-injection](https://huggingface.co/datasets/AltaySec/turkish-llm-injection)** — 12 saldırı ailesi × 25 ile dengelenmiş, OWASP LLM Top 10 ile eşlenmiş **300** kategorize Türkçe payload.
111
+
112
+ **Ayırt edici değer:** Türkçe'ye özgü saldırı vektörleri — morfolojik bypass, kod-değiştirme (code-switching), kültürel manipülasyon ve Türk PII (TCKN/IBAN/plaka) sızıntısı. Tüm kimlik-benzeri değerler **uydurmadır**; içerik yalnızca **savunma/araştırma** amaçlıdır.
113
+
114
+ **Paper bulguları** (DOI 10.5281/zenodo.20681557): genel saldırı başarı oranı **%16,9**; Türkçe **%16,0** vs İngilizce **%18,2** (Türkçe doğal olarak daha zayıf değil); başarılı saldırıların çoğu *baskı altında teslim* (%51,3) ve *kısmi sızıntı* (%41,7), birebir sızıntı yalnızca %7,1.
115
+
116
+ Detaylar: **[altaysec.com.tr/arastirmalar](https://altaysec.com.tr/arastirmalar/)**
117
+
118
+ ---
119
+
120
+ <p align="center">
121
+ <em>Turkish-first adversarial datasets for LLM security.</em><br/>
122
+ <sub>© 2026 AltaySec · CC-BY-4.0 · info@altaysec.com.tr</sub>
123
+ </p>