v0.1

README.md

@@ -0,0 +1,81 @@
+---
+language:
+- hu
+license: mit
+tags:
+- hungarian
+- causal-lm
+- llama
+- mlx
+- apple-silicon
+- sentencepiece
+library_name: transformers
+pipeline_tag: text-generation
+model-index:
+- name: csermely
+  results: []
+---
+
+# Csermely 0.1B
+
+**The smallest coherent Hungarian language model.** Part of the [Emese](https://emese.tech) model family.
+
+Csermely is a 138M parameter decoder-only transformer trained exclusively on high-quality Hungarian text. It runs on edge devices and excels in summarization, grammar checking, and tone detection.
+
+## Model Details
+
+| | |
+|---|---|
+| **Parameters** | 137.8M |
+| **Context length** | 8,192 tokens (YaRN RoPE) |
+| **Architecture** | LLaMA-style (decoder-only transformer) |
+| **Training context** | 2,048 tokens |
+| **Training precision** | bfloat16 (MLX) |
+| **Published weights** | float16 |
+| **Vocabulary** | 32,000 (SentencePiece Unigram, Hungarian) |
+| **Training data** | ~1B tokens of Hungarian text |
+| **License** | MIT |
+
+## Architecture
+
+- 16 transformer layers
+- 768 hidden dimension
+- 12 attention heads
+- 2048 FFN intermediate size
+- RMSNorm pre-layer normalization
+- Rotary positional embeddings (RoPE) with YaRN extension
+- SwiGLU feed-forward activation
+- Tied input/output embeddings
+
+## Tokenizer
+
+Custom 32K vocabulary SentencePiece Unigram tokenizer trained on high-quality Hungarian corpora. ~30% more token-efficient than multilingual tokenizers for Hungarian text.
+
+Available separately: [emese-tech/emese-tokenizer-32k](https://huggingface.co/emese-tech/emese-tokenizer-32k)
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("emese-tech/csermely")
+model = AutoModelForCausalLM.from_pretrained("emese-tech/csermely")
+
+input_text = "A magyar nyelv"
+inputs = tokenizer(input_text, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=100)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+The default generation config uses `temperature=0.7`, `top_p=0.9`, and `repetition_penalty=1.2` to reduce repetitive output.
+
+## Citation
+
+```bibtex
+@misc{emese-csermely-2026,
+  title={Csermely: A Tiny Hungarian Language Model},
+  author={Emese Tech},
+  year={2026},
+  url={https://huggingface.co/emese-tech/csermely}
+}
+```

config.json

@@ -0,0 +1,26 @@
+{
+  "architectures": ["LlamaForCausalLM"],
+  "model_type": "llama",
+  "model_version": "0.1",
+  "vocab_size": 32000,
+  "hidden_size": 768,
+  "intermediate_size": 2048,
+  "num_hidden_layers": 16,
+  "num_attention_heads": 12,
+  "num_key_value_heads": 12,
+  "hidden_act": "silu",
+  "max_position_embeddings": 8192,
+  "rope_theta": 10000.0,
+  "rope_scaling": {
+    "type": "yarn",
+    "factor": 4.0,
+    "original_max_position_embeddings": 2048
+  },
+  "rms_norm_eps": 1e-5,
+  "tie_word_embeddings": true,
+  "torch_dtype": "float16",
+  "bos_token_id": 2,
+  "eos_token_id": 3,
+  "pad_token_id": 1,
+  "transformers_version": "4.45.0"
+}

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "max_new_tokens": 256,
+  "do_sample": true,
+  "temperature": 0.7,
+  "top_p": 0.9,
+  "repetition_penalty": 1.2,
+  "bos_token_id": 2,
+  "eos_token_id": 3,
+  "pad_token_id": 1,
+  "transformers_version": "4.45.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b71a2b93e76e0e24f49dee8e66ff08ced606a220c7e7676d2824d81d27bd4552
+size 324863416

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:244bb6facd57b3890990261b9932ab50d79630d5b35058c902a5c96c32fa2950
+size 845117

tokenizer_config.json

@@ -0,0 +1,12 @@
+{
+  "bos_token": "<bos>",
+  "eos_token": "<eos>",
+  "unk_token": "<unk>",
+  "pad_token": "<pad>",
+  "sp_model_kwargs": {},
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "model_max_length": 8192,
+  "tokenizer_class": "LlamaTokenizer",
+  "clean_up_tokenization_spaces": false
+}