glubbot1.1-7.5m

Model Description

glubbot1.1-7.5m is an experimental, ultra-lightweight text generation model developed by the Glubbot-Team.

With only 7.5 million parameters, this model is designed for experimental testing, low-resource inference benchmarking, and studying language acquisition in small-scale models. It was trained specifically to generate short, grammatically correct narratives.

• Developed by: Glubbot-Team
• Model Type: Transformer-based Causal Language Model
• Language(s): English
• License: Apache 2.0
• Status: Experimental / Testing

Intended Uses & Limitations

This model is intended solely for research, architectural testing, and experimental evaluation.

Intended Use

• Synthetic text generation for short, child-like stories.
• Benchmarking inference speeds on edge devices or CPUs.
• Testing tokenization strategies and pipeline integration.

Limitations & Biases

• Domain Specificity: Optimized purely for short stories; it cannot handle reasoning, coding, or factual factual queries.
• Limited Capacity: Due to the 7.5M parameter size, the model may produce repetitive structures or drift logically during long generations.
• Not for Production: Do not deploy this model for general-purpose assistant tasks.

Training Details

Training Data

The model was trained on the TinyStories dataset (roneneldan/TinyStories). This dataset consists of short stories containing words typically understood by 3- to 4-year-olds, optimized to teach models basic grammar and reasoning.

Hyperparameters

• Context Length: [e.g., 512 / 1024]
• Batch Size: [Insert Batch Size]
• Learning Rate: [Insert Learning Rate]
• Optimizer: AdamW

Evaluation

Evaluation metrics are monitored primarily to test convergence stability on the TinyStories split. The model is evaluated on its ability to maintain basic grammatical coherence.