SmolLlama3

SmolLlama3 is a LoRA adapter fine-tuned on meta-llama/Llama-3.1-8B using TRL and PEFT. It was trained on the maharnab/smol-smoltalk-10k dataset (a 10,000-sample conversational dataset derived from smoltalk) to enable light, general-purpose conversational capabilities without post-processing like DPO.

Model Details

Model Description

SmolLlama3 is an 8B-parameter language model adapter built as part of an experiment with fine-tuning LLMs. It is based on Llama 3.1 8B and was fine-tuned using Supervised Fine-Tuning (SFT) on a custom dataset, smol-smoltalk-10k, containing 10,000 conversational samples. The model is designed for simple conversational tasks; however, its responses may be less refined as Direct Preference Optimization (DPO) was not applied.

• Developed by: Maharnab Saikia
• Model type: PEFT Adapter (LoRA for Causal LM)
• Language(s) (NLP): English
• License: Llama 3.1 Community License
• Finetuned from model: meta-llama/Llama-3.1-8B

Uses

Direct Use

The model is intended for light multi-turn English chat, simple instruction-following, and experimentation with small-scale conversational SFT adapters on top of Llama 3.1 8B.

Downstream Use

Can be merged with meta-llama/Llama-3.1-8B base weights for deployment or served directly using peft and transformers.

Out-of-Scope Use

This model should not be used for high-stakes decision-making, medical, legal, or safety-critical applications. Because DPO or RLHF alignment steps were omitted, the model may generate unrefined, hallucinated, or unsafe content if prompted adversarial.

Bias, Risks, and Limitations

• Lack of Preference Alignment: Omitting DPO/RLHF means outputs may be verbose, redundant, or inconsistent in safety boundaries.
• Inherited Base Model Biases: Inherits all limitations, knowledge cutoffs, and potential biases present in meta-llama/Llama-3.1-8B.

Recommendations

Users should implement guardrails and system prompts when serving this model in interactive environments.

How to Get Started with the Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "meta-llama/Llama-3.1-8B"
adapter_id = "maharnab/SmolLlama3"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, adapter_id)

messages = [{"role": "user", "content": "Hello! How are you today?"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

Trained on maharnab/smol-smoltalk-10k, a 10k subset of the smoltalk dataset containing multi-turn conversational interactions, instruction-following tasks, and general QA.

Training Procedure

Training Hyperparameters

• Training regime: bf16 mixed precision
• Fine-tuning technique: LoRA via PEFT & SFTTrainer (trl)

Speeds, Sizes, Times

• Checkpoint Size: ~84MB (LoRA adapter weights)

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: NVIDIA A100 40GB
• Hours used: 5
• Cloud Provider: Modal
• Compute Region: South asia
• Carbon Emitted: 1.15

Technical Specifications

Model Architecture and Objective

Causal Language Modeling (CLM) fine-tuned with Low-Rank Adaptation (LoRA) over the query/value projections of Llama 3.1 8B.

Compute Infrastructure

Hardware

• GPU: NVIDIA A100 40GB / Modal

Software

• Python: 3.10+
• Transformers: 4.x
• PEFT: 0.19.1
• TRL: latest