Hwiiiiiiii/gemby-130-tiny

GembyTiny 🤖

GembyTiny is a lightweight ~120M parameter causal language model built for chat. Custom transformer with RoPE embeddings, SwiGLU activations, and RMSNorm.

Quick Start

from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="Hwiiiiiiii/gemby-130-tiny",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "Hello!"}]
result = pipe(messages, max_new_tokens=100, do_sample=True, temperature=0.7)
print(result[0]["generated_text"][-1]["content"])

License

Apache 2.0