metadata
base_model: unsloth/Meta-Llama-3.1-8B-bnb-4bit
library_name: peft
tags:
- base_model:adapter:unsloth/Meta-Llama-3.1-8B-bnb-4bit
- lora
- transformers
language:
- en
metrics:
- bleu
- bertscore
- rouge
pipeline_tag: summarization
Model Card for Model ID
Basically paired the unsloth/Meta-Llama-3.1-8B-bnb-4bit base model with the fine-tuned Chilliwiddit/Openi-llama3.1-8B-WeightedLoss-small2 adapter.
Training Details
Training Data
I used the Open-i dataset
Training Hyperparameters
Training regime: [More Information Needed]
16 Mixed Precision
LR of 0.0-1
5 Epochs
lambda medical weight of 20 and lambda negation weight of 20
Used 2nd iteration of summary medical concepts file