--- language: [bn, en] license: cc-by-sa-4.0 pipeline_tag: text-generation library_name: transformers tags: [llama, bengali, tagore, poetry, small-model, gguf] --- # TagoreGPT 29M A small (29 million parameter) model that writes **poems, songs, stories, letters, essays and plays in the style of Rabindranath Tagore (1861–1941)**, in **Bengali or English**. Trained from scratch on a laptop. Sir Rabindranath Tagore (Gurudev) was the first non-European to win the Nobel Prize in Literature (1913, for *Gitanjali*) and wrote the national anthems of India and Bangladesh. ## How to ask The model writes from **a topic + hidden labels** (language, type, length, rhyme); it does not read free-form requests. The chat template in the GGUF does that step for you, like the TagoreGPT app: it finds the language (Bengali letters or not), the type (poem / song / story / letter / essay / play / novel words) and the topic (the request minus "write a poem about ..."). Examples that work: - `write a poem about the river` · `write a song about the rain` · `Write a story about a boatman` - `নদী নিয়ে একটা কবিতা লেখো` · `বৃষ্টি নিয়ে একটা গান লেখো` Settings that work well (= the app): temperature 0.75 (poem) / 0.7 (song) / 0.8 (story), top-k 40, top-p 0.95, min-p 0.02, **repeat penalty 1.3** (last 256 tokens). ### Ollama: type just the topic Four ready recipes, one per language + type: ``` ollama create tagoregpt-poem-en -f Modelfile.poem-en # then: ollama run tagoregpt-poem-en "The River" ollama create tagoregpt-story-en -f Modelfile.story-en # ollama run tagoregpt-story-en "The old boatman" ollama create tagoregpt-poem-bn -f Modelfile.poem-bn # ollama run tagoregpt-poem-bn "নদী" ollama create tagoregpt-song-bn -f Modelfile.song-bn # ollama run tagoregpt-song-bn "বৃষ্টি" ``` ### LM Studio / llama.cpp - File: `tagoregpt-29m-f16.gguf` (Llama layout, 85 MB). Its chat template reads the request as described above. - llama.cpp: `llama-cli -m tagoregpt-29m-f16.gguf -cnv --jinja --temp 0.75 --top-k 40 --repeat-penalty 1.3` - English is weaker than Bengali in this version; the TagoreGPT app adds topic guidance and line rules that plain llama.cpp / Ollama / LM Studio do not have. ### Transformers ```python from transformers import AutoTokenizer, AutoModelForCausalLM tok = AutoTokenizer.from_pretrained("PATH_OR_REPO") model = AutoModelForCausalLM.from_pretrained("PATH_OR_REPO") ids = tok.apply_chat_template([{"role": "user", "content": "write a poem about the river"}], return_tensors="pt") out = model.generate(ids, max_new_tokens=300) print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)) ``` ## What it cannot do - It is not a chatbot: it does not answer questions, explain, or translate. It only writes. - Topics given in English for a Bengali piece are not translated outside the TagoreGPT app. - It sees about 1,024 word pieces (~3,300 letters) at a time; longer pieces can wander. (The TagoreGPT app also keeps the title and opening paragraph in view; plain llama.cpp / LM Studio do not.) - It is small: lines can be beautiful or nonsensical. It is a style imitation, never Tagore's real words. ## Training - Llama layout: 8 layers, width 512, 8 heads, SwiGLU 1344, rotary positions, tied embeddings. 8,000 SentencePiece word pieces (Bengali + English). - Phase 1 (warm-up): simple practice stories (TinyStories, CDLA-Sharing-1.0; Regional-TinyStories Bengali, Apache-2.0), other Bengali writers (Bengali Wikisource) + 19th-century English (Project Gutenberg) + Tagore. - Phase 2: Tagore only (Bengali scans from the Digital Library of India / Internet Archive, English from Project Gutenberg). - Every training piece starts with hidden labels (language, type, author); its title and opening paragraph stay in view, which keeps long pieces on their hero and topic. - Weights in this release: tagore step -1. Unseen-text loss per word piece: tagore_en: 3.278, tagore_bn: 3.674. ## Data and licence Tagore's works are in the public domain. The Bengali warm-up text comes from **Bengali Wikisource** (CC BY-SA 4.0), so this model is released under **CC BY-SA 4.0**. English text from Project Gutenberg (public domain).