Model Card for llama-3.2-1b-it-brainrot

Model Details

Model Description

• Model Name: CallmeKaito/llama-3.2-1b-it-brainrot
• Base Model: LLaMA-3.2-1B
• Fine-Tuned On: ShreeshaBhat1004/Brain-rot dataset
• Task: Text generation with a focus on "brainrot" content (humorous, absurd, or nonsensical text).
• Fine-Tuning Dataset Size: 32 rows (small dataset for experimental purposes).

Quick start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("unsloth/Llama-3.2-1B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Llama-3.2-1B-Instruct")
model = PeftModel.from_pretrained(base_model, "CallmeKaito/llama-3.2-1b-it-brainrot")

# Create chat template
messages = [
    {"role": "system", "content": "ayoooo, you be Llama, big brain bot built by dem Meta wizards, no cap. Now, spit out mega chonky, hyper-thicc explain-o answers like some ultimate galaxy-brain encyclopedia. If peeps want that yummy deep knowledge buffet, you drop that big brain bomb and make it so they’re stuffed with juicy details, aight? If they just chattin’ small fries, keep it chill and normal vibes, but if they hunger for dat prime prime think-juices, show ’em all them hidden crevices of know-how, bruh."},
    {"role": "user", "content": "homie tell me a lil more about the bronx situation and the wild stuff happening in nyc?"}
]

# Generate prompt
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

# Tokenize inputs
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

# Generate response
outputs = model.generate(
    **inputs,
    max_new_tokens=150,
    eos_token_id=tokenizer.eos_token_id,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)

# Decode and format output
full_response = tokenizer.decode(outputs[0], skip_special_tokens=True)
response = full_response.split("assistant\n")[-1].strip()
print(response)

Intended Use

This model is intended for experimental and entertainment purposes. It is fine-tuned on a small dataset of "brainrot" content. Use cases include:

• Generating funny or absurd text for entertainment.
• Exploring the effects of fine-tuning on small, niche datasets.
• Testing the limits of language models with minimal data.

Limitations

• Overfitting: Due to the extremely small dataset (32 rows), the model may have overfitted to the training data, leading to poor generalization on unseen data.
• Validation Loss: The validation loss increased during training, indicating potential overfitting or lack of generalization.
• Niche Use Case: The model is specialized for "brainrot" content and may not perform well on general text generation tasks.
• Ethical Considerations: The model may generate nonsensical or inappropriate content. Use with caution and ensure outputs are reviewed before sharing.

Training procedure

This model was trained with SFT.

Framework versions

• TRL: 0.13.0
• Transformers: 4.48.1
• Pytorch: 2.0.0
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citations

Cite TRL as:

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}