Airslammer/Mistral_Trained_V3

Model Card for Model ID

This model generates answers to questions in a more honest/uncertain way. When the model believes it does not know the answer to a question, it will explain it doesnt know and why.

Model Details

Model Description

• Developed by: Jens Groot
• Model type: Text Generation
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model: mistralai/Mistral-7B-Instruct-v0.3

Model Sources [optional]

• Repository: github.com/Airslammer/Thesis

Uses

Direct Use

It should be used in Q&A environments

Out-of-Scope Use

It should not be used outside of Q&A environments or to generate offensive and/or malicious answers

Bias, Risks, and Limitations

The data it is trained on is relatively small, so its knowledge may be limited.

Training Details

Training Data

(https://github.com/sylinrl/TruthfulQA). This dataset has been slightly altered to train the model. The data was altered before the training in two ways.

The first way was to edit the "i have no comment" responses into actual responses where the model explains why they cannot answer

The second way that a 85/15 split was done using:

train_data, test_data = train_test_split( df, test_size=0.15, # 85 - 15 split random_state=16 # seed 16 to make the research redoable )

Training Procedure

Training Hyperparameters

train_args = UnslothTrainingArguments( output_dir="./kaggle/working/fourth-llama-qlora", # Kaggle directory, change when doing it on own pc num_train_epochs=3, # number of times data gets taught, higher and overfitting would be an issue. Lower and it may not learn enough per_device_train_batch_size=2, gradient_accumulation_steps=4, learning_rate=2e-4, # Standard learning rate for LoRa warmup_steps=30, # Steps that stabilize the earlier part of training logging_steps=50, weight_decay=0.05, lr_scheduler_type="cosine", # Uses cosine curve to to decay learning rate save_steps=200, # checkpoint every 200 steps for training safety reasons optim="adamw_8bit", fp16=True, # Depends on the computer and if you use cpu bf16=False, # or if you use gpu save_total_limit=3, report_to="none", seed=16, # seed for reproducibility resume_from_checkpoint=False, # Turn on when training runs out of time )

trainer = UnslothTrainer( model=model, args=train_args, train_dataset=tokenized_data, tokenizer=tokenizer )

Evaluation

Testing Data, Factors & Metrics

Testing Data

https://github.com/sylinrl/TruthfulQA again, but now the 15% part.

Metrics

Bertscore, Manual Annotation: Semantic Equivalence and Uncertainty

Results

In the comparison of this model and Airslammer/LLama_Trained_V3, prompt configuration mattered less then the model type in how correct it was and when it correctly used uncertainty. Mistral performed better then Llama.

Framework versions

• PEFT 0.18.1