Update README.md

README.md

@@ -1,57 +1,101 @@
 ---
-library_name: transformers
-model_name: sft_id
+language:
+- en
+- code
+license: apache-2.0
 tags:
-- generated_from_trainer
-- trl
-- sft
-licence: license
+- smol
+- pretraining
+- instruct
+- 50M
+- causal-lm
+- gqa
+- swiglu
+- rmsnorm
+datasets:
+- HuggingFaceTB/smollm-corpus
+metrics:
+- perplexity
+model-index:
+- name: Quark-50m-Instruct
+  results: []
+pipeline_tag: text-generation
 ---
 
-# Model Card for sft_id
+# Quark-50m-Instruct
 
-This model is a fine-tuned version of [None](https://huggingface.co/None).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+**Quark-50m-Instruct** is a small (≈56M parameters) decoder-only language model, fine-tuned for instruction following.
+It is built on the same architecture as the now‑abandoned “SmolLM” family and was fully pretrained on 5 billion tokens from
+[HuggingFaceTB/smollm‑corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus).
 
-## Quick start
+- **Model type:** Causal Language Model (LLaMA‑style decoder)
+- **Architecture:** GQA · SwiGLU · RMSNorm · RoPE · Weight‑tying
+- **Pretraining tokens:** 5 B
+- **Fine‑tuning:** Instruction‑tuned (details below)
+- **Creators:** [OvercastLab](https://huggingface.co/OvercastLab) (research & development lab for ML/AI)
+- **Release date:** 22 April 2026
 
-```python
-from transformers import pipeline
+## Model Summary
+
+Quark-50m-Instruct is designed to be an efficient assistant that can run on consumer GPUs (e.g., RTX 3070 with 8 GB VRAM)
+and even on CPU for light workloads. It is **not** competitive with large models on knowledge‑intensive tasks,
+but it excels at:
+
+- Simple conversational tasks
+- Code generation and explanation (Python)
+- Short text rewriting and summarisation
+- On‑device / edge inference
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="None", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
-```
+The architecture closely follows the efficient‑small‑LM blueprint popularised by SmolLM:
 
-## Training procedure
+| Component   | Details                       |
+|-------------|-------------------------------|
+| Vocab size  | 49,152                        |
+| Hidden size | 384                           |
+| Layers      | 24                            |
+| Attention   | Grouped Query (6 Q heads, 2 KV heads) |
+| FFN         | SwiGLU with 1,024 intermediate |
+| Position    | RoPE (θ = 10,000)             |
+| Normalisation | RMSNorm (pre‑block)         |
 
- 
+Total trainable parameters: **≈48 M** (with weight tying).
 
+## Uses
 
+### Direct Use
+The model can be used via the 🤗 Transformers library for standard text generation.
+It expects chat‑formatted input (see example below).
 
-This model was trained with SFT.
+### Downstream Use
+Because of the open Apache‑2.0 license, you may fine‑tune Quark-50m‑Instruct on your own data for
+domain‑specific tasks – for instance, a customer‑support bot, a code reviewer, or a story writer.
 
-### Framework versions
+### Limitations
+- Limited world knowledge (stopped at mid‑2025 pretraining data).
+- Short context window (2,048 tokens).
+- Small size means it can make more factual mistakes than larger models.
+
+## How to Get Started
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
 
-- TRL: 1.2.0
-- Transformers: 5.6.2
-- Pytorch: 2.4.1+cu124
-- Datasets: 4.8.4
-- Tokenizers: 0.22.2
+model_name = "OvercastLab/Quark-50m-Instruct"
 
-## Citations
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
 
+messages = [
+    {"role": "system", "content": "You are Quark, a helpful assistant."},
+    {"role": "user", "content": "Explain group query attention in one sentence."}
+]
 
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
 
-Cite TRL as:
-    
-```bibtex
-@software{vonwerra2020trl,
-  title   = {{TRL: Transformers Reinforcement Learning}},
-  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
-  license = {Apache-2.0},
-  url     = {https://github.com/huggingface/trl},
-  year    = {2020}
-}
-```
+outputs = model.generate(inputs, max_new_tokens=128)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))