Upload README.md with huggingface_hub

README.md

@@ -1,60 +1,149 @@
 ---
-library_name: peft
-license: other
-base_model: Qwen/Qwen2.5-3B-Instruct
+language:
+- en
+license: apache-2.0
+library_name: transformers
 tags:
-- base_model:adapter:Qwen/Qwen2.5-3B-Instruct
+- legal
+- immigration
+- assistant
+- qwen2
+- qwen2.5
+- fine-tuned
 - lora
-- transformers
+base_model: Qwen/Qwen2.5-3B-Instruct
+model_type: qwen2
 pipeline_tag: text-generation
-model-index:
-- name: DoloresAI-Qwen25
-  results: []
+widget:
+- text: "What is an H-1B visa?"
+  example_title: "H-1B Visa Question"
+- text: "How do I apply for a green card?"
+  example_title: "Green Card Process"
+- text: "What documents do I need for an O-1 visa?"
+  example_title: "O-1 Visa Requirements"
+datasets:
+- busybisi/dolores-training-data
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+# Dolores AI - Immigration Case Manager (Qwen 2.5 LoRA)
+
+Dolores is a specialized AI Immigration Case Manager fine-tuned using LoRA (Low-Rank Adaptation) on Qwen 2.5-3B-Instruct. Her mission is to de-mystify the complex immigration journey, breaking it down into manageable, actionable steps with high empathy and precision.
+
+## Model Details
+
+- **Base Model**: [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)
+- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
+- **LoRA Rank**: 16
+- **LoRA Alpha**: 32
+- **Training Data**: Immigration law documents, case examples, and expert guidance
+- **Use Case**: Immigration consultation, visa guidance, document preparation
+- **Model Size**: ~3B parameters (LoRA adapter only)
+
+## Training Details
+
+### Training Configuration
+- **Epochs**: 3
+- **Batch Size**: 4 (effective: 16 with gradient accumulation)
+- **Learning Rate**: 2e-4
+- **Quantization**: 4-bit (QLoRA) during training
+- **Max Sequence Length**: 2048 tokens
+- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
+
+### Training Data
+Fine-tuned on curated immigration law datasets including:
+- U.S. immigration policies and procedures
+- Visa types and requirements (H-1B, O-1, EB-1, etc.)
+- Green card processes
+- Case examples and expert guidance
+- Document preparation instructions
+
+## Usage
+
+This is a **LoRA adapter** that needs to be loaded with the base model. For production use, see the merged version: [JustiGuide/DoloresAI-Qwen25-Merged](https://huggingface.co/JustiGuide/DoloresAI-Qwen25-Merged)
+
+### Loading the LoRA Adapter
 
-# DoloresAI-Qwen25
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+import torch
 
-This model is a fine-tuned version of [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) on an unknown dataset.
+base_model_id = "Qwen/Qwen2.5-3B-Instruct"
+lora_adapter_id = "JustiGuide/DoloresAI-Qwen25"
 
-## Model description
+# Load base model
+model = AutoModelForCausalLM.from_pretrained(
+    base_model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
 
-More information needed
+# Load LoRA adapter
+model = PeftModel.from_pretrained(model, lora_adapter_id)
 
-## Intended uses & limitations
+tokenizer = AutoTokenizer.from_pretrained(base_model_id)
+```
 
-More information needed
+### Inference Example
 
-## Training and evaluation data
+```python
+system_prompt = "You are Dolores, a specialized AI Immigration Case Manager."
 
-More information needed
+question = "What is an H-1B visa and who qualifies for it?"
 
-## Training procedure
+prompt = f'''<|im_start|>system
+{system_prompt}<|im_end|>
+<|im_start|>user
+{question}<|im_end|>
+<|im_start|>assistant
+'''
 
-### Training hyperparameters
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
 
-The following hyperparameters were used during training:
-- learning_rate: 0.0002
-- train_batch_size: 4
-- eval_batch_size: 4
-- seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 16
-- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_steps: 100
-- num_epochs: 3
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    temperature=0.7,
+    top_p=0.9,
+    do_sample=True,
+    repetition_penalty=1.1,
+)
 
-### Training results
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(response)
+```
 
+## Deployment
 
+For production deployment, use the merged model: [JustiGuide/DoloresAI-Qwen25-Merged](https://huggingface.co/JustiGuide/DoloresAI-Qwen25-Merged)
 
-### Framework versions
+### HuggingFace Inference Endpoint
+- GPU: Nvidia L4 (24GB VRAM)
+- Scale to Zero: Enabled
+- Region: us-east-1
+
+## Performance
+
+- **Inference Speed**: ~10-20 tokens/second (on L4 GPU)
+- **Context Length**: Up to 2048 tokens
+- **Quality**: High accuracy on immigration-specific questions
+
+## Limitations
+
+- Provides general immigration guidance, **not legal advice**
+- Always consult with a licensed immigration attorney for specific cases
+- Trained primarily on U.S. immigration law
+- May not have information on very recent policy changes
+
+## License
+
+Apache 2.0 License (following base model's license)
+
+## Contact
+
+- **Organization**: JustiGuide
+- **Website**: https://justi.guide
+
+---
 
-- PEFT 0.18.0
-- Transformers 4.57.3
-- Pytorch 2.9.0+cu126
-- Datasets 4.0.0
-- Tokenizers 0.22.2
+**Built with ❤️ by JustiGuide to make immigration more accessible**