README.md

---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- legal
- immigration
- assistant
- qwen2
- qwen2.5
- fine-tuned
- lora
base_model: Qwen/Qwen2.5-3B-Instruct
model_type: qwen2
pipeline_tag: text-generation
widget:
- text: "What is an H-1B visa?"
  example_title: "H-1B Visa Question"
- text: "How do I apply for a green card?"
  example_title: "Green Card Process"
- text: "What documents do I need for an O-1 visa?"
  example_title: "O-1 Visa Requirements"
datasets:
- busybisi/dolores-training-data
---

# Dolores AI - Immigration Case Manager (Qwen 2.5 LoRA)

Dolores is a specialized AI Immigration Case Manager fine-tuned using LoRA (Low-Rank Adaptation) on Qwen 2.5-3B-Instruct. Her mission is to de-mystify the complex immigration journey, breaking it down into manageable, actionable steps with high empathy and precision.

## Model Details

- **Base Model**: [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **LoRA Rank**: 16
- **LoRA Alpha**: 32
- **Training Data**: Immigration law documents, case examples, and expert guidance
- **Use Case**: Immigration consultation, visa guidance, document preparation
- **Model Size**: ~3B parameters (LoRA adapter only)

## Training Details

### Training Configuration
- **Epochs**: 3
- **Batch Size**: 4 (effective: 16 with gradient accumulation)
- **Learning Rate**: 2e-4
- **Quantization**: 4-bit (QLoRA) during training
- **Max Sequence Length**: 2048 tokens
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

### Training Data
Fine-tuned on curated immigration law datasets including:
- U.S. immigration policies and procedures
- Visa types and requirements (H-1B, O-1, EB-1, etc.)
- Green card processes
- Case examples and expert guidance
- Document preparation instructions

## Usage

This is a **LoRA adapter** that needs to be loaded with the base model. For production use, see the merged version: [JustiGuide/DoloresAI-Qwen25-Merged](https://huggingface.co/JustiGuide/DoloresAI-Qwen25-Merged)

### Loading the LoRA Adapter

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen2.5-3B-Instruct"
lora_adapter_id = "JustiGuide/DoloresAI-Qwen25"

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, lora_adapter_id)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
```

### Inference Example

```python
system_prompt = "You are Dolores, a specialized AI Immigration Case Manager."

question = "What is an H-1B visa and who qualifies for it?"

prompt = f'''<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{question}<|im_end|>
<|im_start|>assistant
'''

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    repetition_penalty=1.1,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

## Deployment

For production deployment, use the merged model: [JustiGuide/DoloresAI-Qwen25-Merged](https://huggingface.co/JustiGuide/DoloresAI-Qwen25-Merged)

### HuggingFace Inference Endpoint
- GPU: Nvidia L4 (24GB VRAM)
- Scale to Zero: Enabled
- Region: us-east-1

## Performance

- **Inference Speed**: ~10-20 tokens/second (on L4 GPU)
- **Context Length**: Up to 2048 tokens
- **Quality**: High accuracy on immigration-specific questions

## Limitations

- Provides general immigration guidance, **not legal advice**
- Always consult with a licensed immigration attorney for specific cases
- Trained primarily on U.S. immigration law
- May not have information on very recent policy changes

## License

Apache 2.0 License (following base model's license)

## Contact

- **Organization**: JustiGuide
- **Website**: https://justi.guide

---

**Built with ❤️ by JustiGuide to make immigration more accessible**