---
library_name: transformers
tags:
- creole
- haitian
license: mit
language:
- ht
base_model:
- jsbeaudry/makandal-pre-trained
pipeline_tag: text-generation
---
# Makandal  Continue Pre-trained from qwen3-0.6b


## Model Details

This model has been continued pre-trained from qwen3-0.6b by Palmis Labs AI. .


### Model Description

- **Developed by:** Palmis Labs AI
- **Funded by:** Jean Sauvenel Beaudry
- **Model type:** GPT (Generative Pre-trained Transformer)
- **Language(s) (NLP):** Haitian Creole
- **License:** MIT
- **Model size:** 0.6B parameters
- **Architecture:** qwen3


### Direct Use

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

def generate(model, tokenizer, prompt, device):
    inputs = tokenizer(prompt, return_tensors="pt", padding=True).to(device)
    output = model.generate(
        **inputs,
        max_new_tokens=100,
        do_sample=True,
        repetition_penalty=1.2,
        no_repeat_ngram_size=3,
        temperature=0.9,
        top_k=40,
        top_p=0.85,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id
    )
    return tokenizer.decode(output[0], skip_special_tokens=True)

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("jsbeaudry/makandal-v2")
model = AutoModelForCausalLM.from_pretrained("jsbeaudry/makandal-v2")

# Set device
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

# Generate text
prompt = "matematik"
response = generate(model, tokenizer, prompt, device)
print(response)

# Answer:
# Matematik se yon disiplin matematik ki konsantre sou kalkil, estatistik, ak analiz matematik.
# Li pèmèt nou konprann enfòmasyon ak fòmèlman analize done pou jwenn pwopriyete oswa fòmèlman verifye yon konpreyansyon.
```

### Out-of-Scope Use

This model should **NOT** be used for:
- Critical decision-making systems
- Any application requiring reliable or factual outputs
- Commercial deployment without significant additional training

## Bias, Risks, and Limitations

- **Insufficient training data**: Only 4.7 MB of training data used
- **Limited training time**: Only 4.5 hours of training
- **High hallucination rate**: Model frequently generates inaccurate or nonsensical content
- **Language coverage**: Limited Haitian Creole language understanding due to minimal dataset
- **Bias**: May reflect biases present in the small training dataset

### Recommendations

- Do not rely on outputs for factual information
- Supervise usage in educational settings


### Training Infrastructure

- **GPU:** Tesla T4 (15GB)
- **Framework:** Transformers/PyTorch


## Citation

```bibtex
@misc{makandal2025,
  title={Makandal-pretrain: An Educational Haitian Creole Language Model},
  author={Jean Sauvenel Beaudry},
  year={2025},
  howpublished={\url{https://huggingface.co/jsbeaudry/makandal-pre-trained}},
  note={Educational demonstration model}
}
```

## Glossary

**Makandal**: Named after François Makandal, an 18th-century Haitian revolutionary leader, symbolizing the model's connection to Haitian culture and education.