syaffers/tiny-random-llama-lora

Tiny Random LLaMA LoRA

A minimal LoRA adapter for hmellor/tiny-random-LlamaForCausalLM, useful for smoke testing deployments.

Model Details

Model Description

This is a LoRA (Low-Rank Adaptation) adapter trained on a tiny random LLaMA model. The model and adapter are intentionally small and produce random outputs—they are not meant for any real inference tasks. The primary purpose is to provide a lightweight adapter for testing deployment pipelines, inference servers, and LoRA loading mechanisms.

• Model type: LoRA adapter for causal language modeling
• Language(s): English
• License: MIT
• Finetuned from: hmellor/tiny-random-LlamaForCausalLM

Model Sources

• Repository: syaffers/tiny-random-llama-lora (training code)

Uses

Direct Use

This adapter is intended for:

• Smoke testing LoRA adapter loading in inference pipelines
• Testing deployment configurations with minimal resource usage
• Validating HuggingFace PEFT integration in your infrastructure

Out-of-Scope Use

This model should not be used for:

• Any real text generation or NLP tasks
• Production applications
• Any use case requiring meaningful outputs

How to Get Started with the Model

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("hmellor/tiny-random-LlamaForCausalLM")
model = PeftModel.from_pretrained(base_model, "syaffers/tiny-random-llama-lora")
tokenizer = AutoTokenizer.from_pretrained("syaffers/tiny-random-llama-lora")

# Generate (output will be random/meaningless)
inputs = tokenizer("Hello world", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=10)
print(tokenizer.decode(outputs[0]))

Training Details

Training Data

iamholmes/tiny-imdb - A tiny subset of IMDB reviews used for quick training iterations.

Training Procedure

Training Hyperparameters

• Training regime: fp32
• Batch size: 4
• Learning rate: 1e-4
• Epochs: 3
• Warmup steps: 10
• Max sequence length: 128

LoRA Configuration

Parameter	Value
r (rank)	8
lora_alpha	16
target_modules	q_proj, v_proj
lora_dropout	0.05
bias	none
task_type	CAUSAL_LM

Technical Specifications

Model Architecture and Objective

LoRA adapter applied to the query and value projection layers of a tiny random LLaMA architecture for causal language modeling.

Compute Infrastructure

Hardware

• Apple M3 Pro (36GB unified memory)
• macOS Sequoia 15.6.1

Software

• Transformers
• PEFT 0.18.0
• PyTorch
• Datasets

Framework versions

• PEFT 0.18.0