Upload 4 files

README.md

@@ -1,3 +1,63 @@
----
-license: apache-2.0
----
+---
+language:
+- id
+license: mit
+tags:
+- text-generation
+- indonesian
+- pytorch
+- caca
+datasets:
+- Lyon28/Corpus-Indonesia
+- Lyon28/Caca-Behavior
+---
+
+# Caca-Tiny 🔥
+
+Caca-Tiny adalah language model berbahasa Indonesia yang dilatih menggunakan arsitektur transformer decoder.
+
+## Model Details
+
+- **Architecture**: Transformer Decoder
+- **Parameters**: ~4,156,928
+- **Vocabulary Size**: 8000
+- **Max Sequence Length**: 512
+- **Training Data**: Lyon28/Corpus-Indonesia
+- **Fine-tuning Data**: Lyon28/Caca-Behavior
+
+## Usage
+
+```python
+import torch
+from safetensors.torch import load_file
+
+state_dict = load_file("model.safetensors")
+
+prompt = "Indonesia adalah"
+generated = model.generate(prompt, max_new_tokens=50)
+print(generated)
+```
+
+## Training
+
+Model ini dilatih dengan:
+- Optimizer: AdamW
+- Learning Rate: 3e-4
+- Batch Size: 8
+- Epochs: 3 (pre-training) + 2 (fine-tuning)
+
+## License
+
+MIT License
+
+## Citation
+
+```bibtex
+@misc{caca-tiny,
+  author = {Lyon28},
+  title = {Caca-Tiny: Indonesian Language Model},
+  year = {2026},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/Lyon28/Caca-Tiny}
+}
+```

config.json

@@ -0,0 +1,13 @@
+{
+  "model_type": "caca",
+  "model_name": "Caca-Tiny",
+  "version": "1.0.0",
+  "vocab_size": 8000,
+  "embedding_dim": 256,
+  "num_layers": 4,
+  "num_heads": 4,
+  "ffn_hidden_dim": 512,
+  "max_seq_length": 512,
+  "dropout": 0.1,
+  "head_dim": 64
+}

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd840462a63ee165c22d99e6b295b18fc3da832afaad863005f1045c519627f9
+size 17171715