metadata
license: llama3
datasets:
- ai-eldorado/Brazilian_CLT_preferences
language:
- en
- pt
base_model:
- meta-llama/Llama-3.1-8B-Instruct
tags:
- legal
Model Description
This repository provides a LoRA adapter trained on the same human-preference dataset used for the DPO model. It enables efficient fine-tuning of LLaMA-3 8B Instruct (4-bit) for Brazilian labor law applications without requiring full model retraining.
Intended Use
The LoRA adapter is intended for developers and researchers who want to adapt LLaMA-3 models for CLT-related tasks while minimizing computational costs. It is particularly useful for resource-constrained environments or for integrating into multi-agent legal assistant systems.
Training Details
- Base Model: LLaMA-3 8B (4-bit quantized)
- Method: LoRA fine-tuning with DPO-aligned dataset
- Dataset: 736 human-preference entries on CLT-related questions
- Hyperparameters: Same as the DPO model
Performance Summary
When merged with the base model, the LoRA adapter reproduces the improvements observed in the full DPO model:
- Increased factual accuracy
- Better semantic alignment with CLT regulations
Ethical Considerations
- Legal Disclaimer: Not a substitute for professional legal advice.
- Risk of Misuse: Incorrect merging or use outside intended domain may lead to inaccurate outputs.
- Data Privacy: No personal data was used in training.
Bias and Fairness
- Same considerations as the DPO model:
- Regional and interpretation biases may exist.
- Limited dataset size could affect fairness in edge cases.
Limitations
- Requires merging with the base LLaMA-3 model for inference.
- Domain-specific; not suitable for general-purpose legal reasoning.
Citation
soon