Create README.md

README.md

@@ -0,0 +1,103 @@
+---
+license: apache-2.0
+base_model: unsloth/Qwen3.5-0.8B
+tags:
+  - code-generation
+  - math-reasoning
+  - qwen3.5
+  - lora
+  - sft
+datasets:
+  - ise-uiuc/Magicoder-Evol-Instruct-110K
+  - meta-math/MetaMathQA
+  - AI-MO/NuminaMath-CoT
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# CMLM-0.8B
+
+**C**oding + **M**ath **L**anguage **M**odel — a Qwen3.5-0.8B fine-tuned for code generation and mathematical reasoning.
+
+## Model Details
+
+| Property | Value |
+| :--- | :--- |
+| Base Model | [unsloth/Qwen3.5-0.8B](https://huggingface.co/unsloth/Qwen3.5-0.8B) |
+| Architecture | Qwen3.5 (Gated DeltaNet + Full Attention hybrid) |
+| Parameters | 0.8B |
+| Training Method | LoRA (r=16, α=32) |
+| Precision | float32 (no quantization) |
+| Max Context | 2048 tokens |
+| Framework | Unsloth + TRL SFTTrainer |
+| Hardware | NVIDIA Tesla T4 (16 GB VRAM) |
+
+## Training Data
+
+| Dataset | Samples | Domain |
+| :--- | :--- | :--- |
+| [Magicoder-Evol-Instruct-110K](https://huggingface.co/datasets/ise-uiuc/Magicoder-Evol-Instruct-110K) | 25,000 | Code instruction following |
+| [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA) | 25,000 | Mathematical reasoning |
+| [NuminaMath-CoT](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT) | 15,000 | Math chain-of-thought |
+| **Total** | **65,000** | |
+
+## Training Hyperparameters
+
+```yaml
+learning_rate: 2e-4
+max_steps: 500
+per_device_train_batch_size: 2
+gradient_accumulation_steps: 8
+effective_batch_size: 16
+warmup_steps: 100
+optimizer: adamw_8bit
+gradient_checkpointing: unsloth
+lora_r: 16
+lora_alpha: 32
+lora_dropout: 0
+target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
+packing: true
+max_seq_length: 2048
+```
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("Natarizki/CMLM-0.8B", device_map="auto")
+tokenizer = AutoTokenizer.from_pretrained("Natarizki/CMLM-0.8B")
+
+messages = [{"role": "user", "content": "Write a Python function to check if a number is prime."}]
+inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(model.device)
+
+outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Benchmarks
+
+| Domain | CMLM-0.8B (tok/s) | Base Qwen3.5-0.8B (tok/s) | Avg Latency (CMLM) |
+| :--- | :--- | :--- | :--- |
+| Coding | 12.1 | 15.2 | 39.0s |
+| Math | 14.8 | 15.1 | 15.9s |
+| General | 15.0 | 15.1 | 17.0s |
+
+> **Note:** CMLM generates longer, more detailed responses for coding tasks (hence lower tok/s but higher quality). Math and general domains show near-parity with base model throughput. Benchmarked on NVIDIA T4 with float32 inference via Unsloth.
+
+## Limitations
+
+- Trained on 65K samples; may underperform on niche domains
+- 2048 token context limit; not suitable for long-document tasks
+- float32 training preserves accuracy but increases inference memory vs. quantized variants
+- No vision capabilities despite Qwen3.5's native multimodal architecture
+
+## License
+
+Apache 2.0 (inherits from Qwen3.5)
+
+## Acknowledgments
+
+- [Qwen Team](https://huggingface.co/Qwen) for the base model
+- [Unsloth](https://unsloth.ai/) for efficient T4-compatible training
+- Dataset authors: Magicoder, MetaMath, NuminaMath teams