Gonyai-v1: A Poetic Konkani Language Model

Gonyai-v1 is a 160M parameter transformer model specifically designed for Konkani text generation. It features a custom architecture (KonkanGPT) utilizing Rotary Positional Embeddings (RoPE), RMSNorm, and SwiGLU activation functions.

Model Details

• Architecture: KonkanGPT (Custom Transformer)
• Parameters: ~160 Million
• Tokenizer: Custom 32k Byte-Level BPE (trained on Konkani corpora)
• Training Data: A curated collection of Konkani literature, news, and poetic works.

How to Use

Since Gonyai-v1 is a 160M model built from scratch, it performs best with specific inference settings. For poetic and coherent results, we recommend using a low temperature and a repetition penalty.

Note: You must set trust_remote_code=True to load the custom KonkanGPT architecture.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "omdeep22/Gonyai-v1"

device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    trust_remote_code=True,
    torch_dtype=dtype
).to(device)

messages = [{"role": "user", "content": "गोंयच्या पावसाचेर एक कविता बरोव."}]
inputs = tokenizer.apply_chat_template(
    messages, 
    tokenize=True, 
    add_generation_prompt=True, 
    return_tensors="pt",
    return_dict=True
).to(device)

with torch.inference_mode():
    with torch.autocast(device_type=device, dtype=dtype):
        outputs = model.generate(
            input_ids=inputs["input_ids"],
            attention_mask=inputs["attention_mask"],
            max_new_tokens=100,
            temperature=0.3,
            repetition_penalty=1.2,
            do_sample=True,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.eos_token_id
        )

generated_tokens = outputs[0][inputs["input_ids"].shape[-1]:]
response = tokenizer.decode(generated_tokens, skip_special_tokens=True).strip()

print(f"\nAssistant: {response}")