| --- |
| base_model: unsloth/Meta-Llama-3.1-8B-bnb-4bit |
| library_name: peft |
| tags: |
| - base_model:adapter:unsloth/Meta-Llama-3.1-8B-bnb-4bit |
| - lora |
| - transformers |
| language: |
| - en |
| metrics: |
| - bleu |
| - bertscore |
| - rouge |
| pipeline_tag: summarization |
| --- |
| |
| # Model Card for Model ID |
|
|
| Basically paired the [unsloth/Meta-Llama-3.1-8B-bnb-4bit](https://huggingface.co/unsloth/Meta-Llama-3.1-8B-bnb-4bit) base model with the fine-tuned [Chilliwiddit/Openi-llama3.1-8B-WeightedLoss-small2](https://huggingface.co/Chilliwiddit/Openi-llama3.1-8B-WeightedLoss-small2) adapter. |
|
|
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| I used the Open-i dataset |
|
|
|
|
| #### Training Hyperparameters |
|
|
| - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision --> |
|
|
| - 16 Mixed Precision |
| - LR of 0.0-1 |
| - 5 Epochs |
| - lambda medical weight of 20 and lambda negation weight of 20 |
| - Used 2nd iteration of summary medical concepts file |
|
|
