Instructions to use Aryaman02/InLawMate-peft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Aryaman02/InLawMate-peft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Aryaman02/InLawMate-peft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Aryaman02/InLawMate-peft")
model = AutoModelForCausalLM.from_pretrained("Aryaman02/InLawMate-peft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use Aryaman02/InLawMate-peft with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Aryaman02/InLawMate-peft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Aryaman02/InLawMate-peft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aryaman02/InLawMate-peft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Aryaman02/InLawMate-peft

SGLang

How to use Aryaman02/InLawMate-peft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Aryaman02/InLawMate-peft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aryaman02/InLawMate-peft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Aryaman02/InLawMate-peft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aryaman02/InLawMate-peft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Aryaman02/InLawMate-peft with Docker Model Runner:
```
docker model run hf.co/Aryaman02/InLawMate-peft
```

InLawMate-peft: Indian Legal Domain PEFT Model

Model Description

InLawMate-peft is a Parameter-Efficient Fine-Tuned (PEFT) language model specifically optimized for understanding and reasoning about Indian legal documentation. The model was trained on a carefully curated dataset of nearly 7,000 question-answer pairs derived from Indian criminal law documentation, making it particularly adept at legal comprehension and explanation tasks.

Training Data

The training data consists of nearly 7,000 high-quality legal Q&A pairs that were systematically generated using a sophisticated two-stage process:

Question Generation: Questions were extracted to cover key legal concepts, definitions, procedures, and roles, ensuring comprehensive coverage of:
- Legal terminology and definitions
- Procedural rules and steps
- Rights and penalties
- Jurisdictional aspects
- Roles of legal entities (judges, lawyers, law enforcement)
Answer Generation: Answers were crafted following a structured legal reasoning approach, ensuring:
- Legal precision and accuracy
- Comprehensive coverage of relevant points
- Clear explanation of legal concepts
- Professional legal discourse style

Training Details

Base Model: allenai/Llama-3.1-Tulu-3-8B
Architecture: PEFT (Parameter-Efficient Fine-Tuning)
Training Epochs: 3
Batch Size: 2 (with gradient accumulation steps of 4)
Learning Rate: 3e-05 with cosine scheduler
Sequence Length: 1024 tokens
Mixed Precision: BF16
Optimization: AdamW with β1=0.9, β2=0.999

Use Cases

This model is particularly suited for:

Legal document analysis and comprehension
Answering questions about Indian criminal law
Understanding legal procedures and requirements
Explaining legal concepts and terminology
Assisting in legal research and education

Limitations

The model is specifically trained on Indian legal documentation
Responses should be verified by legal professionals for critical applications
The model should not be used as a substitute for professional legal advice

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "aryaman/legalpara-lm",
    device_map="auto",
    torch_dtype='auto'
).eval()

tokenizer = AutoTokenizer.from_pretrained("Aryaman02/InLawMate-peft")

# Example legal query
messages = [
    {"role": "user", "content": "What are the requirements for cross-examination according to Indian law?"}
]

input_ids = tokenizer.apply_chat_template(
    conversation=messages, 
    tokenize=True, 
    add_generation_prompt=True, 
    return_tensors='pt'
)
output_ids = model.generate(input_ids.to('cuda'))
response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Citation

If you use this model in your research, please cite:

@misc{legalpara-lm,
  title={InLawMate: A PEFT Model for Indian Legal Domain Understanding},
  year={2024},
  publisher={Aryaman},
  note={Model trained on Indian legal documentation}
}

Our training data and procedure for synth data creation is outlined in https://github.com/DarryCrucian/law-llm