MoroLMai V1.2 — A Moroccan Darija Language Model

MoroLMai V1.2 is an experimental language model developed from scratch, with a focus on Moroccan Darija and English. The project explores building compact language models that can be run locally, supporting experimentation, research, and offline inference.

The goal of MoroLMai is to contribute to the development of language technology for Moroccan Darija and make experimentation with locally hosted language models more accessible.

This is an experimental research project, not a claim of state-of-the-art performance.

Model Details

• Developed by: Hamza Daoud
• Project: MoroLMai
• Version: V1.2
• Model type: Autoregressive language model (causal language modeling)
• Model size: Approximately 136 million parameters
• Languages: Moroccan Darija and English
• Training approach: Training from scratch, followed by supervised fine-tuning
• Model format: GGUF, with variants included in the repository
• License: Apache-2.0, subject to the license terms and applicable rights associated with the training data

Model Sources

• Hugging Face repository: This repository
• Project website: https://morolmai.com
• Source code: https://github.com/hamzadaoud/
• Paper: Not currently available
• Demo: A local chat interface is under development.

Intended Uses

Direct Use

MoroLMai V1.2 is intended for:

• Experimenting with Moroccan Darija text generation.
• Exploring small language models trained from scratch.
• Testing local and offline inference.
• Educational demonstrations of language-model development.
• Research into Moroccan Arabic and multilingual language technologies.

Downstream Use

Developers and researchers may explore integrating the model into local applications, prototypes, conversational interfaces, and educational projects, subject to the applicable license and training-data restrictions.

Out-of-Scope Use

This model is not intended to serve as the sole source of information for medical, legal, financial, or other high-stakes decisions. It should not be treated as an authoritative source, a verified knowledge base, or a substitute for professional advice.

Bias, Risks, and Limitations

MoroLMai V1.2 is a relatively small experimental model. Its capabilities are limited by its size, training data, and training procedure.

Potential limitations include:

• Incorrect, incomplete, or fabricated information.
• Inconsistent Moroccan Darija spelling, grammar, and dialect usage.
• Repetition, incoherent continuations, and weak long-form reasoning.
• Uneven performance across Darija written in Arabic script and Latin transliteration.
• Variable performance in English and code-switched conversations.
• Limited ability to follow complex instructions reliably.

The model has not been established as suitable for production-critical applications.

Recommendations

Users should evaluate outputs before relying on them. Applications should provide appropriate safeguards, especially when model outputs affect people or inform consequential decisions.

How to Get Started

GGUF Inference

If this repository contains a compatible GGUF file, you can run it locally using llama.cpp.

Example:

llama-cli \
  -m ./MoroLMai-V1.2-Q4_K_M.gguf \
  -cnv \
  -c 2048

Replace the filename with the exact GGUF file you download. The context length and other settings may need adjustment for the model's actual configuration.

For Windows, the same command can be adapted to use llama-cli.exe.

Note: The example assumes that the published GGUF file is compatible with your installed llama.cpp version. Check the repository's files and instructions before running it.

Training Details

Training Data

The model was developed using a mixture of text intended to support Moroccan Darija and English generation.

The development workflow included data preparation, pretraining, and supervised fine-tuning.

The complete dataset inventory, source attribution, filtering procedures, and applicable licenses should be documented separately. Third-party datasets are not included or redistributed by this model card.

Users should consult the original dataset terms before using or redistributing training data.

Training Procedure

The model was trained from randomly initialized weights rather than fine-tuned from an existing pretrained language model.

The development process included:

1. Text collection and preparation.
2. Tokenization and training-data preparation.
3. Autoregressive language-model pretraining.
4. Supervised fine-tuning.
5. Model conversion and local inference testing.

Exact training-step counts, optimizer settings, learning-rate schedules, and dataset statistics should be added once verified against the V1.2 training logs.

Training Hyperparameters

• Training regime: Mixed-precision training was used in the development workflow; the exact V1.2 precision configuration should be verified against the training logs.
• Optimizer: To be documented.
• Learning rate: To be documented.
• Batch size: To be documented.
• Training steps: To be documented.
• Tokenizer: Project-specific tokenizer; exact vocabulary size to be confirmed for V1.2.

Speeds, Sizes, and Times

The model has been exported to GGUF for local inference.

Exact training duration, inference throughput, checkpoint sizes, and benchmark results are not reported here because they have not been independently verified for this release.

Evaluation

Testing Data, Factors, and Metrics

A reproducible benchmark suite and verified quantitative evaluation results have not yet been documented for this release.

Recommended future evaluations include:

• Moroccan Darija text-generation quality.
• Arabic-script and Latin-script Darija consistency.
• English text-generation quality.
• Code-switching between Darija, Arabic, French, and English.
• Repetition and coherence.
• Performance across different prompts and context lengths.

Results

No formal benchmark scores are claimed in this model card.

Summary

MoroLMai V1.2 is an experimental step toward building compact language models with Moroccan Darija support. Further evaluation is needed to establish its strengths, limitations, and suitability for particular applications.

Environmental Impact

Carbon emissions have not been measured for this training run.

• Hardware: Local workstation with an NVIDIA GPU.
• Cloud provider: None reported for local training.
• Compute region: Not applicable to local training.
• Training duration: To be documented.
• Carbon emissions: Not measured.

Technical Specifications

Model Architecture and Objective

MoroLMai V1.2 is an autoregressive language model trained to predict subsequent tokens from preceding context.

The model was developed from scratch and subsequently fine-tuned using supervised examples.

The exact number of Transformer layers, hidden dimension, attention configuration, context length, and tokenizer vocabulary should be confirmed against the V1.2 configuration files before publication.

Compute Infrastructure

Hardware

Development and training were performed on a local workstation.

The precise hardware allocation and utilization for the V1.2 training run should be verified against the relevant logs.

Software

The project development environment includes Python, PyTorch, CUDA, and llama.cpp for model conversion and local inference. Exact version numbers should be added for the release.

Citation

If you use or reference MoroLMai V1.2 in research, please cite the project and include the repository URL and version.

A formal academic publication is not currently available.

Glossary

• Darija: Moroccan Arabic as used in everyday communication.
• Pretraining: Training a language model to predict tokens from text.
• Supervised fine-tuning (SFT): Further training on curated examples to influence model responses.
• GGUF: A model file format commonly used for local inference with llama.cpp.
• Quantization: Reducing model-weight precision to reduce memory requirements, often with trade-offs in output quality.

More Information

MoroLMai is an independent project focused on experimenting with language-model development for Moroccan Darija.

Project website: https://morolmai.com

Source code: https://github.com/hamzadaoud/

Model Card Authors

Hamza Daoud — MoroLMai project.

Model Card Contact

Please use the project's repository or website for project-related questions.