Update README.md

README.md

@@ -3,4 +3,143 @@ license: mit
 language:
 - pt
 pipeline_tag: text-generation
----
+tags:
+- base
+- pretrain
+- pretrained
+- nano
+- mini
+- chatbot
+---
+
+🧠 MiniBot-0.9M-Base
+
+Ultra-lightweight GPT-2 style language model (~900K parameters) specialized in Portuguese conversational text.
+
+📌 Model Overview
+
+MiniBot-0.9M-Base is a tiny decoder-only Transformer (~0.9M parameters) based on the GPT-2 architecture, designed for efficient text generation in Portuguese.
+
+This model is a base (pretrained) model, meaning it was trained for next-token prediction without instruction tuning or alignment.
+
+It is intended primarily for:
+
+🧪 Fine-tuning experiments
+🎮 Playground usage
+⚡ Ultra-fast local inference
+🧠 Research on small-scale language models
+🎯 Key Characteristics
+🇧🇷 Language: Portuguese (primary)
+🧠 Architecture: GPT-2 style (decoder-only Transformer)
+🔤 Embeddings: GPT-2 compatible embeddings
+📉 Parameters: ~900,000
+⚙️ Objective: Causal Language Modeling (next-token prediction)
+🚫 Alignment: None (base model)
+🏗️ Architecture Details
+
+MiniBot-0.9M follows a scaled-down GPT-2 design, including:
+
+Token + positional embeddings
+Multi-head self-attention
+Feed-forward (MLP) layers
+Autoregressive decoding
+
+Despite its small size, it preserves the core inductive biases of GPT-2, making it ideal for experimentation and educational purposes.
+
+📚 Training
+Dataset
+
+The model was trained on a Portuguese conversational dataset, including:
+
+Dialogues (Usuário ↔ Bot)
+Perguntas e respostas simples
+Conversas casuais
+Estruturas de linguagem natural
+Format
+User: Oi!
+Bot: Olá! Como posso te ajudar?
+Training Notes
+Focused on language pattern learning, not reasoning
+No instruction tuning (no RLHF, no alignment)
+Lightweight training pipeline
+Optimized for small-scale experimentation
+💡 Capabilities
+
+✅ Strengths:
+
+Geração de texto em português
+Estrutura básica de diálogo
+Continuação de prompts simples
+Aprendizado de padrões linguísticos
+
+❌ Limitations:
+
+Raciocínio muito limitado
+Perda de contexto em conversas longas
+Respostas inconsistentes
+Possível repetição ou incoerência
+
+👉 This model behaves as a statistical language generator, not a reasoning system.
+
+🚀 Usage
+Hugging Face Transformers
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "AxionLab-official/MiniBot-0.9M-Base"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
+
+prompt = "User: Oi\nBot:"
+inputs = tokenizer(prompt, return_tensors="pt")
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=50,
+    temperature=0.8,
+    top_p=0.95,
+    do_sample=True
+)
+```
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+⚙️ Recommended Generation Settings
+
+For better results:
+
+temperature: 0.7 – 1.0
+top_p: 0.9 – 0.95
+do_sample: True
+max_new_tokens: 30 – 80
+🧪 Intended Use
+
+This is a foundation model, ideal for:
+
+🧠 Fine-tuning (chat, instruction, roleplay, tools)
+🎮 Prompt playground experimentation
+🔬 Research in tiny LLMs
+📉 Benchmarking small architectures
+⚠️ Limitations
+
+Due to its extremely small size:
+
+Limited world knowledge
+Weak generalization
+No safety alignment
+Not suitable for production use
+🔮 Future Work
+
+Planned directions:
+
+🧠 Instruction-tuned version (MiniBot-Instruct)
+📚 Larger dataset scaling
+🔤 Tokenizer improvements
+📈 Larger variants (1M–10M params)
+🤖 Experimental reasoning fine-tuning
+📜 License
+
+MIT
+
+👤 Author
+
+Developed by AxionLab