README and LICENSE

LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tarik Dedić
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,98 @@
+---
+tags:
+  - llm
+  - language-model
+  - causal-lm
+  - bosnian
+  - croatian
+  - serbian
+  - bcs
+  - balkan-languages
+  - llama
+  - 779m
+  - base-model
+  - undertrained
+  - multilingual
+language:
+  - bs
+  - hr
+  - sr
+  - en
+license: mit
+base_model: none
+pipeline_tag: text-generation
+---
+
+## predBor-v0.5
+
+A preview of predBor, a small language model built from the ground up to natively understand Bosnian/Croatian/Serbian while supporting English.
+
+**NOTE THAT THIS IS A BASE LANGUAGE MODEL. IT DOES NOT POSSESS THE ABILITY TO CHAT OR ANSWER QUESTIONS, ONLY CONTINUE TEXT.**
+
+### Architecture Details
+- **Type:** Causal Language Model
+- **Parameters:** 779M
+- **Architecture:** LLaMA
+- **Context window:** 4096 tokens
+- **Tokenizer:** Bor-v2
+- **Dataset:** Bor-CORPUS-22B
+
+This repo contains an undertrained checkpoint of the predBor language model which has only seen 11B tokens of data, yet still shows promising performance on all four supported languages.
+
+### Emerging Capabilities
+Even undertrained, predBor is starting to display some emerging skills in different fields, only serving to show the quality of its data and architecture. Notable achievements:
+
+- **Translation:** The model has developed semantic mapping between BCS and English, being able to translate words (and sometimes phrases) with the right prompt.
+- **Fact memorization:** The model has seen enough data to show surprising amounts of general world knowledge, such as viable medical advice, historical facts, recipes with logically correct steps and ingredients, and the capital city to every country there is... for some reason.
+- **Question answering:** When prompted with a question and the beginning of an answer (e.g., "The meaning of life is"), it delivers a somewhat viable answer depending on the topic. The model's current state impacts its overall intelligence, meaning some answers will be hallucinated. This does not represent the final state of the project.
+- **And more.** Feel free to download and test the model yourself.
+
+### How to run
+
+#### Hugging Face Transformers (for quick testing)
+
+```bash
+pip install transformers torch accelerate
+```
+
+Then,
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_name = "absltnull/predBor-v0.5"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True
+)
+
+prompt = "Glavni grad Bosne i Hercegovine je"
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+output = model.generate(
+    **inputs,
+    max_new_tokens=256,
+    temperature=0.7,
+    top_p=0.9,
+    repetition_penalty=1.1,
+    do_sample=True
+)
+
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+**System requirements**
+- 3.2 GB storage (full model)
+- 2+ GB VRAM/RAM for full bf16/fp16
+
+---
+
+The model is deliberately released in this raw base state so the community can see the real quality of the training data before any continued pretraining, post-training or alignment is applied.
+
+Feel free to fine-tune. The finished version of predBor will be out soon.