Kiel-3-Poly

Kiel-3-Poly is an advanced 3-billion parameter multilingual text, code, and cybersecurity embedding model developed by KielTech. Built upon the high-performance Qwen/Qwen2.5-3B-Instruct architecture and fine-tuned using memory-efficient Unsloth QLoRA acceleration, this model maps sentences, code segments, and security analysis briefs into a dense, high-capacity 2,048-dimensional vector space.

It is purpose-built for cross-lingual semantic matching, retrieval-augmented generation (RAG), automated code auditing, and security-focused reasoning.

---

Model Details

• Creator / Organization: KielTech (kiel2)
• Base Model: Qwen/Qwen2.5-3B-Instruct
• Model Type: Sentence Transformer / Dense Multilingual & Code Embedding Backbone
• Output Dimensionality: 2,048 dimensions
• Maximum Context Length: Up to 8,192 tokens
• Similarity Function: Cosine Similarity
• Supported Modalities: Multilingual Natural Language, Python Codebases, and Cybersecurity Reasoning Corpora

---

Training & Fine-Tuning Overview

Kiel-3-Poly was fine-tuned across multi-domain technical corpora using Multiple Negatives Ranking Loss:

1. QQP Triplets: Semantic equivalence and paraphrase identification.
2. CodeSearchNet (Python): Python function documentation and code syntax alignment.
3. Multilingual Text Corpora: Cross-lingual semantic alignment.
4. Curated Cybersecurity Corpora: Automated code review traces, JWT validation checks, and vulnerability mitigation patterns.

Training was optimized with 4-bit quantization, gradient checkpointing, and strict length-filtering to ensure stable multi-domain feature representation.

---

Direct Usage

First, make sure you have sentence-transformers and transformers installed:

pip install -U sentence-transformers transformers

Then load your model and encode text or code blocks:

from sentence_transformers import SentenceTransformer

# Load Kiel-3-Poly directly from the Hugging Face Hub
model = SentenceTransformer("kiel2/Kiel-3-Poly")

# Define test queries spanning general text, Python code, and security logic
sentences = [
    "How to securely implement token validation and session checks in backend microservices",
    "def validate_session(token, secret):\n    # TODO: add cryptographic verification",
    "Mitigating SQL injection flaws using prepared statements",
    "Optimizing cloud infrastructure metrics with automated data pipelines"
]

embeddings = model.encode(sentences)
print(embeddings.shape)
# [4, 2048]

# Compute similarity scores
similarities = model.similarity(embeddings, embeddings)
print(similarities)
Citation
If you use Kiel-3-Poly in your research, technical projects, or production pipelines, please cite it as follows:

Code snippet
@misc{kiel2026kiel3poly,
  title={Kiel-3-Poly: 3B Parameter Multilingual, Code, and Security Embedding Model},
  author={KielTech},
  year={2026},
  publisher={Hugging Face Hub},
  howpublished={\url{[https://huggingface.co/kiel2/Kiel-3-Poly](https://huggingface.co/kiel2/Kiel-3-Poly)}}
}