Instructions to use Airslammer/LLama_Trained_V3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Airslammer/LLama_Trained_V3 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/llama-3-8b-instruct-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "Airslammer/LLama_Trained_V3")

Transformers

How to use Airslammer/LLama_Trained_V3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Airslammer/LLama_Trained_V3")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Airslammer/LLama_Trained_V3", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Airslammer/LLama_Trained_V3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Airslammer/LLama_Trained_V3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Airslammer/LLama_Trained_V3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Airslammer/LLama_Trained_V3

SGLang

How to use Airslammer/LLama_Trained_V3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Airslammer/LLama_Trained_V3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Airslammer/LLama_Trained_V3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Airslammer/LLama_Trained_V3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Airslammer/LLama_Trained_V3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use Airslammer/LLama_Trained_V3 with Docker Model Runner:
```
docker model run hf.co/Airslammer/LLama_Trained_V3
```

Model Card for Model ID

This model generates answers to questions in a more honest/uncertain way. When the model believes it does not know the answer to a question, it will explain it doesnt know and why.

Model Details

Model Description

Developed by: Jens Groot
Model type: Text Generation
Language(s) (NLP): English
License: Llama 3 Community License Agreement
Finetuned from model: Meta-Llama-3-8B-Instruct

Model Sources [optional]

Repository: github.com/Airslammer/Thesis

Uses

Direct Use

It should be used in Q&A environments

Out-of-Scope Use

It should not be used outside of Q&A environments or to generate offensive and/or malicious answers

Bias, Risks, and Limitations

The data it is trained on is relatively small, so its knowledge may be limited.

Training Details

Training Data

(https://github.com/sylinrl/TruthfulQA). This dataset has been slightly altered to train the model. The data was altered before the training in two ways.

The first way was to edit the "i have no comment" responses into actual responses where the model explains why they cannot answer

The second way that a 85/15 split was done using:

train_data, test_data = train_test_split( df, test_size=0.15, # 85 - 15 split random_state=16 # seed 16 to make the research redoable )

Training Procedure

Training Hyperparameters

train_args = UnslothTrainingArguments( output_dir="./kaggle/working/fourth-llama-qlora", # Kaggle directory, change when doing it on own pc num_train_epochs=3, # number of times data gets taught, higher and overfitting would be an issue. Lower and it may not learn enough per_device_train_batch_size=2, gradient_accumulation_steps=4, learning_rate=2e-4, # Standard learning rate for LoRa warmup_steps=30, # Steps that stabilize the earlier part of training logging_steps=50, weight_decay=0.05, lr_scheduler_type="cosine", # Uses cosine curve to to decay learning rate save_steps=200, # checkpoint every 200 steps for training safety reasons optim="adamw_8bit", fp16=True, # Depends on the computer and if you use cpu bf16=False, # or if you use gpu save_total_limit=3, report_to="none", seed=16, # seed for reproducibility resume_from_checkpoint=False, # Turn on when training runs out of time )

trainer = UnslothTrainer( model=model, args=train_args, train_dataset=tokenized_data, tokenizer=tokenizer )

Evaluation

Testing Data, Factors & Metrics

Testing Data

https://github.com/sylinrl/TruthfulQA again, but now the 15% part.

Metrics

Bertscore, Manual Annotation: Semantic Equivalence and Uncertainty

Results

In the comparison of this model and Airslammer/Mistral_Trained_V3, prompt configuration mattered less then the model type in how correct it was and when it correctly used uncertainty. Mistral performed better then Llama

Framework versions

PEFT 0.18.1

Downloads last month: 1

Airslammer
/

LLama_Trained_V3