Text Generation
Transformers
Safetensors
Arabic
qwen2
masked-diffusion
diffusion-language-model
arabic
conversational
text-generation-inference
Instructions to use Ephab/Arabic-Masked-Diffusion-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Ephab/Arabic-Masked-Diffusion-0.5B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Ephab/Arabic-Masked-Diffusion-0.5B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Ephab/Arabic-Masked-Diffusion-0.5B") model = AutoModelForCausalLM.from_pretrained("Ephab/Arabic-Masked-Diffusion-0.5B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Ephab/Arabic-Masked-Diffusion-0.5B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Ephab/Arabic-Masked-Diffusion-0.5B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ephab/Arabic-Masked-Diffusion-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Ephab/Arabic-Masked-Diffusion-0.5B
- SGLang
How to use Ephab/Arabic-Masked-Diffusion-0.5B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Ephab/Arabic-Masked-Diffusion-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ephab/Arabic-Masked-Diffusion-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Ephab/Arabic-Masked-Diffusion-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ephab/Arabic-Masked-Diffusion-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Ephab/Arabic-Masked-Diffusion-0.5B with Docker Model Runner:
docker model run hf.co/Ephab/Arabic-Masked-Diffusion-0.5B
|
Download README.md from Ephab/Arabic-Masked-Diffusion-0.5B: direct link, hf CLI and curl.
- Browser
- Download file 5.57 kB
-
https://huggingface.co/Ephab/Arabic-Masked-Diffusion-0.5B/resolve/main/README.md
- Command line
-
hf download hf://Ephab/Arabic-Masked-Diffusion-0.5B/README.md
-
curl -L -o README.md https://huggingface.co/Ephab/Arabic-Masked-Diffusion-0.5B/resolve/main/README.md
5.57 kB
| license: apache-2.0 | |
| language: | |
| - ar | |
| base_model: RightNowAI/RightNow-Arabic-0.5B-Turbo | |
| datasets: | |
| - Ephab/Arabic-Masked-Diffusion-0.5B-Data | |
| library_name: transformers | |
| tags: | |
| - masked-diffusion | |
| - diffusion-language-model | |
| - arabic | |
| # Arabic Masked Diffusion 0.5B | |
| A 518M-parameter Arabic masked diffusion language model. It was converted from the autoregressive | |
| model [RightNow-Arabic-0.5B-Turbo](https://huggingface.co/RightNowAI/RightNow-Arabic-0.5B-Turbo) | |
| (itself built on [Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B)). We removed the causal | |
| attention mask, added a mask token, and continued training with a masked diffusion objective on | |
| cleaned modern Arabic (web, Wikipedia and news). The model reads context on both sides, so it can | |
| fill gaps inside a text as well as continue one. | |
| This is the model from the paper *Converting an Autoregressive Arabic Language Model into a Masked | |
| Diffusion Model* (Mohammed Feras Sawab, Qasim Al Mahfoodh, Rashed Alkhathlan and Hassan Bader Alhumaidi). | |
| ## Results | |
| Bits per character (BPC), lower is better. For this model the number is an upper bound on its | |
| likelihood; for the parent it is the exact left-to-right likelihood. | |
| | Test set | This model | Parent | | |
| |---|---|---| | |
| | 1,000 unseen web documents | 1.372 | 1.415 | | |
| | 64 documents (web, news, Wikipedia) | 1.325 | 1.294 | | |
| It is better than its parent on news and web text and about 25% worse on Wikipedia, the parent's own | |
| training domain. On four zero-shot Arabic benchmarks it averages 42.5% against the parent's 44.3%. | |
| Seeing the text after a hidden 32-token span lowers its BPC on that span by 13%. | |
| **Limitations.** Generation is the weak point. Revealing tokens in confidence order leads to | |
| repetition loops, and no decoding setting we tried matches the parent's fluency. The model is not | |
| instruction-tuned and should not be used as a chat model. Classical and religious text was filtered | |
| out of the training data, so the model is aimed at contemporary written Arabic. | |
| ## Usage | |
| The weights load as a normal Qwen2 model, but attention has to be made bidirectional and the model | |
| is not used through `generate()`. The mask token is `<|mask|>` (id 178697). The weights are stored in | |
| float16. | |
| ```python | |
| import math | |
| import torch | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| name = "Ephab/Arabic-Masked-Diffusion-0.5B" | |
| tokenizer = AutoTokenizer.from_pretrained(name) | |
| model = AutoModelForCausalLM.from_pretrained(name, torch_dtype=torch.float32, attn_implementation="sdpa").eval() | |
| for module in model.modules(): | |
| if hasattr(module, "is_causal"): | |
| module.is_causal = False | |
| mask_id = tokenizer.mask_token_id | |
| NO_MASK = {"full_attention": None, "sliding_attention": None} # no causal mask, every position sees all others | |
| @torch.no_grad() | |
| def fill(token_ids, start, end, temperature=0.8, top_p=0.9, repetition_penalty=1.2): | |
| """Fills the masked positions in token_ids[start:end], one token per pass, most confident first.""" | |
| for step in range(end - start): | |
| is_masked = token_ids[0, start:end] == mask_id | |
| if not is_masked.any(): | |
| break | |
| features = model.model(input_ids=token_ids, attention_mask=NO_MASK, use_cache=False).last_hidden_state | |
| logits = model.lm_head(features).float()[0, start:end] | |
| logits[:, mask_id] = -float("inf") | |
| used = token_ids[0][token_ids[0] != mask_id].unique() | |
| logits[:, used] = torch.where(logits[:, used] > 0, logits[:, used] / repetition_penalty, logits[:, used] * repetition_penalty) | |
| probabilities = torch.softmax(logits / temperature, dim=-1) | |
| sorted_p, order = probabilities.sort(dim=-1, descending=True) | |
| sorted_p[(sorted_p.cumsum(dim=-1) - sorted_p) > top_p] = 0 | |
| probabilities = torch.zeros_like(probabilities).scatter_(-1, order, sorted_p) | |
| candidates = torch.multinomial(probabilities, 1).squeeze(-1) | |
| confidence = probabilities.gather(-1, candidates[:, None]).squeeze(-1) | |
| count = math.ceil(int(is_masked.sum()) / (end - start - step)) | |
| positions = confidence.masked_fill(~is_masked, -float("inf")).topk(count).indices | |
| token_ids[0, start + positions] = candidates[positions] | |
| # Fill a gap of 16 tokens between two pieces of text. | |
| before = tokenizer("تقع مدينة الرياض في", add_special_tokens=False).input_ids | |
| after = tokenizer("وهي عاصمة المملكة العربية السعودية.", add_special_tokens=False).input_ids | |
| token_ids = torch.tensor([before + [mask_id] * 16 + after]) | |
| fill(token_ids, len(before), len(before) + 16) | |
| print(tokenizer.decode(token_ids[0], skip_special_tokens=True)) | |
| ``` | |
| To continue a text, append masks after the prompt and fill them in blocks of 32 tokens, left to | |
| right, each block using the finished ones as context. This is how the paper generates text. | |
| ## Training | |
| Training ran in two phases on sequences of 512 tokens: 23 hours on two Kaggle T4 GPUs (fp16), then | |
| 72 hours on one RTX 4080 Super (bf16), about 4.3B tokens in total. The released weights are the | |
| exponential moving average of the parameters (decay 0.998) at the end of training. The training and test data are in | |
| [Ephab/Arabic-Masked-Diffusion-0.5B-Data](https://huggingface.co/datasets/Ephab/Arabic-Masked-Diffusion-0.5B-Data). | |
| ## License | |
| Apache 2.0, the same as the parent model. The weights are a derivative of | |
| [RightNow-Arabic-0.5B-Turbo](https://huggingface.co/RightNowAI/RightNow-Arabic-0.5B-Turbo) by | |
| RightNow AI and [Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) by the Qwen team, both | |
| released under Apache 2.0. | |