Instructions to use Ephab/Arabic-Masked-Diffusion-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Ephab/Arabic-Masked-Diffusion-0.5B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Ephab/Arabic-Masked-Diffusion-0.5B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Ephab/Arabic-Masked-Diffusion-0.5B") model = AutoModelForCausalLM.from_pretrained("Ephab/Arabic-Masked-Diffusion-0.5B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Ephab/Arabic-Masked-Diffusion-0.5B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Ephab/Arabic-Masked-Diffusion-0.5B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ephab/Arabic-Masked-Diffusion-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Ephab/Arabic-Masked-Diffusion-0.5B
- SGLang
How to use Ephab/Arabic-Masked-Diffusion-0.5B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Ephab/Arabic-Masked-Diffusion-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ephab/Arabic-Masked-Diffusion-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Ephab/Arabic-Masked-Diffusion-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ephab/Arabic-Masked-Diffusion-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Ephab/Arabic-Masked-Diffusion-0.5B with Docker Model Runner:
docker model run hf.co/Ephab/Arabic-Masked-Diffusion-0.5B
Arabic Masked Diffusion 0.5B
A 518M-parameter Arabic masked diffusion language model. It was converted from the autoregressive model RightNow-Arabic-0.5B-Turbo (itself built on Qwen2.5-0.5B). We removed the causal attention mask, added a mask token, and continued training with a masked diffusion objective on cleaned modern Arabic (web, Wikipedia and news). The model reads context on both sides, so it can fill gaps inside a text as well as continue one.
This is the model from the paper Converting an Autoregressive Arabic Language Model into a Masked Diffusion Model (Mohammed Feras Sawab, Qasim Al Mahfoodh, Rashed Alkhathlan and Hassan Bader Alhumaidi).
Results
Bits per character (BPC), lower is better. For this model the number is an upper bound on its likelihood; for the parent it is the exact left-to-right likelihood.
| Test set | This model | Parent |
|---|---|---|
| 1,000 unseen web documents | 1.372 | 1.415 |
| 64 documents (web, news, Wikipedia) | 1.325 | 1.294 |
It is better than its parent on news and web text and about 25% worse on Wikipedia, the parent's own training domain. On four zero-shot Arabic benchmarks it averages 42.5% against the parent's 44.3%. Seeing the text after a hidden 32-token span lowers its BPC on that span by 13%.
Limitations. Generation is the weak point. Revealing tokens in confidence order leads to repetition loops, and no decoding setting we tried matches the parent's fluency. The model is not instruction-tuned and should not be used as a chat model. Classical and religious text was filtered out of the training data, so the model is aimed at contemporary written Arabic.
Usage
The weights load as a normal Qwen2 model, but attention has to be made bidirectional and the model
is not used through generate(). The mask token is <|mask|> (id 178697). The weights are stored in
float16.
import math
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "Ephab/Arabic-Masked-Diffusion-0.5B"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, torch_dtype=torch.float32, attn_implementation="sdpa").eval()
for module in model.modules():
if hasattr(module, "is_causal"):
module.is_causal = False
mask_id = tokenizer.mask_token_id
NO_MASK = {"full_attention": None, "sliding_attention": None} # no causal mask, every position sees all others
@torch.no_grad()
def fill(token_ids, start, end, temperature=0.8, top_p=0.9, repetition_penalty=1.2):
"""Fills the masked positions in token_ids[start:end], one token per pass, most confident first."""
for step in range(end - start):
is_masked = token_ids[0, start:end] == mask_id
if not is_masked.any():
break
features = model.model(input_ids=token_ids, attention_mask=NO_MASK, use_cache=False).last_hidden_state
logits = model.lm_head(features).float()[0, start:end]
logits[:, mask_id] = -float("inf")
used = token_ids[0][token_ids[0] != mask_id].unique()
logits[:, used] = torch.where(logits[:, used] > 0, logits[:, used] / repetition_penalty, logits[:, used] * repetition_penalty)
probabilities = torch.softmax(logits / temperature, dim=-1)
sorted_p, order = probabilities.sort(dim=-1, descending=True)
sorted_p[(sorted_p.cumsum(dim=-1) - sorted_p) > top_p] = 0
probabilities = torch.zeros_like(probabilities).scatter_(-1, order, sorted_p)
candidates = torch.multinomial(probabilities, 1).squeeze(-1)
confidence = probabilities.gather(-1, candidates[:, None]).squeeze(-1)
count = math.ceil(int(is_masked.sum()) / (end - start - step))
positions = confidence.masked_fill(~is_masked, -float("inf")).topk(count).indices
token_ids[0, start + positions] = candidates[positions]
# Fill a gap of 16 tokens between two pieces of text.
before = tokenizer("ุชูุน ู
ุฏููุฉ ุงูุฑูุงุถ ูู", add_special_tokens=False).input_ids
after = tokenizer("ููู ุนุงุตู
ุฉ ุงูู
ู
ููุฉ ุงูุนุฑุจูุฉ ุงูุณุนูุฏูุฉ.", add_special_tokens=False).input_ids
token_ids = torch.tensor([before + [mask_id] * 16 + after])
fill(token_ids, len(before), len(before) + 16)
print(tokenizer.decode(token_ids[0], skip_special_tokens=True))
To continue a text, append masks after the prompt and fill them in blocks of 32 tokens, left to right, each block using the finished ones as context. This is how the paper generates text.
Training
Training ran in two phases on sequences of 512 tokens: 23 hours on two Kaggle T4 GPUs (fp16), then 72 hours on one RTX 4080 Super (bf16), about 4.3B tokens in total. The released weights are the exponential moving average of the parameters (decay 0.998) at the end of training. The training and test data are in Ephab/Arabic-Masked-Diffusion-0.5B-Data.
License
Apache 2.0, the same as the parent model. The weights are a derivative of RightNow-Arabic-0.5B-Turbo by RightNow AI and Qwen2.5-0.5B by the Qwen team, both released under Apache 2.0.
- Downloads last month
- -
Model tree for Ephab/Arabic-Masked-Diffusion-0.5B
Base model
Qwen/Qwen2.5-0.5B
docker model run hf.co/Ephab/Arabic-Masked-Diffusion-0.5B