Upload README.md

README.md

@@ -0,0 +1,101 @@
+---
+language:
+- si
+- ta
+- en
+license: other
+tags:
+- qwen
+- qlora
+- cpt
+- sri-lanka
+- sinhala
+- tamil
+model_type: causal-lm
+pipeline_tag: text-generation
+base_model: qwen3.5-4b
+---
+
+# Chat2Find-CPT: Qwen 3.5 4B (Sri Lankan Continued Pre-Training)
+
+Chat2Find-CPT is a specialized version of the Qwen 3.5 4B model, enhanced via **Continued Pre-Training (CPT)** using **QLoRA (4-bit)** to excel in Sri Lankan linguistic and cultural contexts. It features robust proficiency in **Sinhala**, **Tamil**, and English.
+
+## Model Details
+
+- **Developed by:** Sentient (Chat2Find)
+- **Base Model:** Qwen 3.5-4B
+- **Training Method:** Continued Pre-Training (CPT) with QLoRA
+- **Languages:** Sinhala (si), Tamil (ta), English (en)
+- **Quantization:** 4-bit (bitsandbytes)
+
+## Technical Specifications
+
+### Training Hardware
+- **GPU:** NVIDIA GeForce RTX 3060 (12GB VRAM)
+- **Frameworks:** Unsloth, Hugging Face Transformers, PEFT
+
+### Training Hyperparameters
+- **Method:** QLoRA (Rank 32, Alpha 32)
+- **Learning Rate:** 5e-5 (Cosine Scheduler)
+- **Optimizer:** AdamW (8-bit)
+- **Epochs:** 1.0
+- **Sequence Length:** 2048 tokens
+- **Batch Size:** 2 (local) / 8 (global with Gradient Accumulation)
+
+### Dataset
+The model was trained on a curated corpus of ~270,000 sequences focusing on:
+- **Sri Lankan News & Media:** Current events and reporting styles.
+- **Business & Logistics:** Domain-specific data from Lankabizz and local commerce.
+- **Cultural Context:** General web-scraped data reflecting local nuances.
+
+## Capabilities
+
+Chat2Find-CPT excels at:
+1. **Sinhala & Tamil Generation:** Fluent and contextually relevant text generation.
+2. **Code-Switching:** Handling natural language mixes common in Sri Lankan communication.
+3. **Local Knowledge:** Understanding entities, locations, and cultural references specific to Sri Lanka.
+
+## Usage
+
+### Using Unsloth (Recommended for Speed)
+
+```python
+from unsloth import FastLanguageModel
+import torch
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = "SENTIENT_ID/Chat2Find-CPT",
+    max_seq_length = 2048,
+    load_in_4bit = True,
+)
+FastLanguageModel.for_inference(model)
+
+inputs = tokenizer(
+    ["ශ්‍රී ලංකාව ගැන කෙටි විස්තරයක්:"],
+    return_tensors = "pt"
+).to("cuda")
+
+outputs = model.generate(**inputs, max_new_tokens = 256)
+print(tokenizer.batch_decode(outputs))
+```
+
+### Using Standard Transformers
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "SENTIENT_ID/Chat2Find-CPT"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
+
+# Note: The model is a merged 16-bit weight set. 
+# You can load it in 4-bit/8-bit using BitsAndBytes.
+```
+
+## Limitations & Bias
+
+While Chat2Find-CPT is significantly better at local languages than the base Qwen model, it may still exhibit biases present in the training data or the base model's internal knowledge. Users are encouraged to perform their own safety checks for specific deployment scenarios.
+
+## License
+
+This model is subject to the original **Qwen License Agreement**.