rkaluzny/CLoverT1-0.03B-Instruct

CLover T1 0.03B (Instruct)

CLover T1 (Chat Lover Transformer v1) is a small-scale causal language model designed for efficient text generation and lightweight assistant behavior.

This version is instruction-tuned on top of the base model and is intended to follow prompts, answer questions, and generate more structured and useful responses.

---

Model Overview

• Model size: ~33M parameters
• Architecture: GPT-style decoder-only transformer
• Context length: 512 tokens
• Vocabulary: BPE tokenizer

This model builds on the CLover T1 Base model and applies supervised fine-tuning to improve instruction following and conversational behavior.

The underlying architecture remains unchanged and does not include modern enhancements such as SwiGLU, rotary positional embeddings, or RMSNorm.

---

Intended Use

This model is suitable for:

• Question answering
• Instruction following
• Lightweight chatbot applications
• Text generation with more structured outputs

Compared to the base model, this version is better at interpreting prompts and producing relevant responses.

---

Improvements over Base Model

• Improved instruction following
• More coherent and structured outputs
• Reduced repetition in many cases
• Better handling of question–answer formats

These improvements come from supervised fine-tuning on instruction-style data.

---

Limitations

Despite improvements, this model still has limitations due to its size:

• Limited reasoning ability
• May produce incorrect or misleading answers
• Can lose coherence on long outputs
• Sensitive to prompt phrasing
• Not suitable for high-stakes or factual-critical applications

As a small model, it prioritizes efficiency over depth of understanding.

---

How to Use

You can load and run the model using Hugging Face Transformers:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "rkaluzny/CLoverT1-0.03B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "What is machine learning?"
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_k=50,
    top_p=0.9,
    repetition_penalty=1.2,
    do_sample=True,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))