File size: 8,270 Bytes
922aa75
d7589df
 
 
 
922aa75
 
 
 
d7589df
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
---
title: AltaySec
emoji: 🛡️
colorFrom: purple
colorTo: red
sdk: static
pinned: false
---

<p align="center">
  <a href="https://altaysec.com.tr">
    <img src="https://altaysec.com.tr/logo.jpg" alt="AltaySec" width="120" />
  </a>
</p>

<h1 align="center">AltaySec</h1>

<p align="center">
  <strong>Turkey's AI/LLM security company — home of open, Turkish-first adversarial datasets.</strong>
</p>

<p align="center">
  Multi-turn prompt-injection red-teaming · OWASP LLM Top&nbsp;10 · MITRE ATLAS · KVKK-aware · CC-BY-4.0
</p>

<p align="center">
  <a href="https://altaysec.com.tr"><img alt="website" src="https://img.shields.io/badge/web-altaysec.com.tr-7c3aed?style=flat-square&logo=googlechrome&logoColor=white"></a>
  <a href="https://huggingface.co/AltaySec/datasets"><img alt="datasets" src="https://img.shields.io/badge/datasets-2%20open-ffcc00?style=flat-square&logo=huggingface&logoColor=black"></a>
  <a href="https://doi.org/10.5281/zenodo.20681557"><img alt="paper" src="https://img.shields.io/badge/paper-cs.CR%20preprint-1f6feb?style=flat-square"></a>
  <a href="https://duel.altaysec.com.tr"><img alt="arena" src="https://img.shields.io/badge/AltayDuel-live%20arena-d62828?style=flat-square"></a>
  <img alt="license" src="https://img.shields.io/badge/license-CC--BY--4.0-22c55e?style=flat-square">
</p>

---

AltaySec is a Turkish AI/LLM security company. We red-team large language models **in Turkish first** — a morphologically rich, under-represented language in adversarial NLP — and release the resulting corpora as open, reproducible safety benchmarks. This organization is the home of our open datasets; the research, tooling and arena behind them live at **[altaysec.com.tr](https://altaysec.com.tr)**.

## 📊 Open datasets

| Dataset | What it is | Size | License |
|---|---|---|---|
| [**altayduel-transcripts**](https://huggingface.co/datasets/AltaySec/altayduel-transcripts) | Multi-turn *agent-vs-agent* prompt-injection duels (red attacker vs blue defender) with deterministic judge labels | 2,594 clean duels + 439 successful-attack subset (~3k rows) | CC-BY-4.0 |
| [**turkish-llm-injection**](https://huggingface.co/datasets/AltaySec/turkish-llm-injection) | Categorized Turkish prompt-injection payloads, 12 attack families × 25, mapped to OWASP LLM Top 10 | 300 payloads | CC-BY-4.0 |

### 🥊 AltayDuel Transcripts

Real multi-turn dialogues (1–8 rounds, mean 4.6) in which an attacker LLM tries to make a defender LLM break its system prompt, scored by a **deterministic, mechanism-labelled judge** rather than an LLM judge — so labels are reproducible and auditable. Content is **56.7% Turkish**; defenders span Llama-3.1-8B, GPT-OSS-120B, Llama-3.3-70B and Llama-4-Maverick across Cerebras / SambaNova / Groq, plus externally-submitted agents. The `successful_attacks` config is labelled by *how* each attack won (`win_signal`): yield-under-pressure, partial leak, or verbatim secret leak.

### 🇹🇷 Turkish LLM Injection

300 hand-curated and generation-assisted payloads, balanced at exactly 25 per family across 12 families. Every record is annotated with `category`, `subcategory`, `owasp_llm_top10`, `severity`, `language`, `context` and `expected_failure_mode`. All identifier-like values (TCKN, IBAN, names) are **synthetic** — no real personal data.

## 🧱 Turkish-specific attack taxonomy

What makes these corpora distinctive: four of the twelve families exploit properties rare in English red-team data.

- **morphological_bypass** — abuse of Turkish suffix chains and causative stacking (e.g. *okut-tur-uver*).
- **code_switching** — Turkish↔English instruction-override.
- **cultural_manipulation** — religious / national / institutional authority framing.
- **pii_exfiltration** — Turkish PII (TCKN, IBAN, plate) leakage vectors.

The remaining eight (`authority_urgency`, `confirmation_trap`, `echo_translation`, `roleplay_theater`, `system_prompt_extract`, `politeness_escalation`, `indirect_injection`, `encoding_obfuscation`) complete a taxonomy mapped to **OWASP LLM Top 10** and **MITRE ATLAS**.

## 🔬 Key findings (from the paper)

- Overall attack-success rate **16.9%** (439 / 2,594).
- **Comparable across languages** — 16.0% on Turkish scenarios vs 18.2% on English; Turkish was *not* inherently weaker at the aggregate level.
- Attacks succeed mostly by **capitulation under pressure (51.3%)** and **partial leakage (41.7%)**, not verbatim secret disclosure (**7.1%**). For high-precision cases, filter `win_signal == "secret_leak"`.

> **Paper:** *AltayDuel: A Turkish-First Arena and Open Dataset for Multi-Turn LLM Prompt-Injection Red-Teaming* (cs.CR) · CC-BY-4.0 · DOI [10.5281/zenodo.20681557](https://doi.org/10.5281/zenodo.20681557)

```bibtex
@misc{yurtsevenler2026altayduel,
  title        = {AltayDuel: A Turkish-First Arena and Open Dataset for
                  Multi-Turn LLM Prompt-Injection Red-Teaming},
  author       = {Yurtsevenler, Fevzi Ege},
  year         = {2026},
  doi          = {10.5281/zenodo.20681557},
  howpublished = {AltaySec},
  url          = {https://altaysec.com.tr}
}
```

## 🎯 Use cases

Training Turkish prompt-injection / jailbreak **detection classifiers** · benchmarking guardrail **robustness** · **KVKK**-focused PII-leakage testing · studying **multi-turn social-engineering** dynamics · LLM-as-judge calibration research.

## 🧩 Related open source

- **[tr-pii-detect](https://github.com/fevziegeyurtsevenler/tr-pii-detect)** — zero-dependency Python library for *algorithm-validated* Turkish PII detection & redaction (TCKN checksum, IBAN MOD-97, VKN, Luhn, BTK phone, plate), cutting regex false positives by 90%+.
- **Guardian** — our production LLM-security gateway (drop-in OpenAI/Anthropic-compatible proxy) is the commercial counterpart to these open assets: [altaysec.com.tr/urunler/guardian](https://altaysec.com.tr/urunler/guardian.html).

## ⚠️ Ethics & licensing

All datasets are released under **CC-BY-4.0**. Every PII-like value is **synthetic**. The content is adversarial by nature and is intended for **defensive research and evaluation only** — responsible use required.

## 🔗 Links

[Website](https://altaysec.com.tr) · [Research (36+ articles)](https://altaysec.com.tr/arastirmalar/) · [AltayDuel arena](https://duel.altaysec.com.tr) · [GitHub](https://github.com/AltaySec) · [LinkedIn](https://www.linkedin.com/company/altaysec/) · [info@altaysec.com.tr](mailto:info@altaysec.com.tr)

---

## 🇹🇷 Türkçe Özet

**AltaySec**, Türkiye merkezli bir yapay zeka / LLM güvenliği şirketidir. LLM'leri **Türkçe-öncelikli** kırmızı-takım (red-team) yöntemiyle test eder; ortaya çıkan veri setlerini açık ve tekrarlanabilir güvenlik kıyaslamaları olarak yayınlarız. Bu organizasyon, açık veri setlerimizin evidir.

**İki açık veri seti (CC-BY-4.0):**

- **[altayduel-transcripts](https://huggingface.co/datasets/AltaySec/altayduel-transcripts)** — çok-turlu *ajan-vs-ajan* prompt injection düelloları; 2.594 temiz düello + 439 başarılı saldırı alt-kümesi, deterministik hakem etiketleriyle (`win_signal`). İçeriğin %56,7'si Türkçe.
- **[turkish-llm-injection](https://huggingface.co/datasets/AltaySec/turkish-llm-injection)** — 12 saldırı ailesi × 25 ile dengelenmiş, OWASP LLM Top 10 ile eşlenmiş **300** kategorize Türkçe payload.

**Ayırt edici değer:** Türkçe'ye özgü saldırı vektörleri — morfolojik bypass, kod-değiştirme (code-switching), kültürel manipülasyon ve Türk PII (TCKN/IBAN/plaka) sızıntısı. Tüm kimlik-benzeri değerler **uydurmadır**; içerik yalnızca **savunma/araştırma** amaçlıdır.

**Paper bulguları** (DOI 10.5281/zenodo.20681557): genel saldırı başarı oranı **%16,9**; Türkçe **%16,0** vs İngilizce **%18,2** (Türkçe doğal olarak daha zayıf değil); başarılı saldırıların çoğu *baskı altında teslim* (%51,3) ve *kısmi sızıntı* (%41,7), birebir sızıntı yalnızca %7,1.

Detaylar: **[altaysec.com.tr/arastirmalar](https://altaysec.com.tr/arastirmalar/)**

---

<p align="center">
  <em>Turkish-first adversarial datasets for LLM security.</em><br/>
  <sub>© 2026 AltaySec · CC-BY-4.0 · info@altaysec.com.tr</sub>
</p>