cijov/Sienna-v1.0

Sienna v1.0

Sienna is a children's story generation model that creates age-appropriate stories for babies and toddlers across four genres:

• Bedtime & Lullaby — Gentle, calming stories to help little ones fall asleep
• Animal Adventures — Fun stories featuring animals toddlers love
• Friendship & Sharing — Stories that teach kindness, sharing, and social skills
• Fantasy & Magic — Magical fairy-tale adventures with simple concepts

Model Details

Property	Value
Base Model	Qwen/Qwen2.5-0.5B-Instruct
Parameters	~648M
Fine-tuning	LoRA (r=32, alpha=32)
Merge Strategy	Average of 4 genre-specialized LoRA adapters
Model Size	~2.5 GB
Format	PyTorch state dict

Quick Start

Requirements

pip install torch transformers peft

Inference

import torch
import json
from model import SiennaConfig, SiennaModel, load_sienna_weights
from generate import sample
from transformers import AutoTokenizer

# Load model
config = SiennaConfig(**json.load(open("config.json")))
model = SiennaModel(config)
load_sienna_weights(model, hf_model_name=config.hf_model_name)
model.apply_lora()

state = torch.load("model.pt", map_location="cpu")
model.load_state_dict(state, strict=False)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
model.eval()

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("tokenizer")

# Generate a story
instruction = "Write a gentle bedtime story about a sleepy bunny"
prompt = f"<instruction>{instruction}</instruction><story>"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)

output_ids = sample(
    model,
    input_ids,
    max_new_tokens=512,
    temperature=0.8,
    top_k=50,
    top_p=0.9,
    repetition_penalty=1.15,
    eos_token_id=tokenizer.eos_token_id,
)

story = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
print(story)

Command-Line Generation

# Generate with a specific genre
python generate.py --model . --genre bedtime --max_new_tokens 512

# Generate with a custom instruction
python generate.py --model . --instruction_text "Write a story about a cat who makes new friends" --max_new_tokens 512

# Adjust generation parameters
python generate.py --model . --genre fantasy --temperature 0.9 --top_k 40 --top_p 0.92 --max_new_tokens 1024

Available Genres

Genre	Flag	Description
Bedtime	--genre bedtime	Gentle, calming bedtime and lullaby stories
Animals	--genre animals	Simple animal adventure stories
Friendship	--genre friendship	Stories about sharing, kindness, and making friends
Fantasy	--genre fantasy	Magical fairy-tale adventures

Generation Parameters

Parameter	Default	Description
--temperature	0.8	Controls randomness (lower = more focused)
--top_k	50	Top-K sampling
--top_p	0.9	Nucleus sampling threshold
--repetition_penalty	1.15	Penalizes repeated tokens
--max_new_tokens	200	Maximum tokens to generate

File Structure

├── README.md           # This file
├── config.json         # Model configuration
├── model.pt            # Merged model weights
├── model.py            # Model architecture definition
├── generate.py         # Inference/generation script
├── genres.py           # Genre definitions
└── tokenizer/          # Tokenizer files
    ├── config.json
    ├── tokenizer.json
    ├── tokenizer_config.json
    └── chat_template.jinja

License

Apache 2.0