Update README.md

README.md

@@ -1,3 +1,140 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language:
+- ar
+- en
+pipeline_tag: text-generation
+tags:
+  - pytorch
+library_name: transformers
+---
+# Fanar-1-9B-Instruct
+
+**Fanar-1-9B-Instruct** is a powerful Arabic-English LLM developed by [Qatar Computing Research Institute (QCRI)](https://www.hbku.edu.qa/en/qcri) and [Hamad Bin Khalifa University (HBKU)](https://www.hbku.edu.qa/). It is the instruction-tuned version of [Fanar-1-9B](). Built on top of `google/gemma-2-9b`, Fanar is further pretrained on 1T Arabic and English tokens. Fanar pays particular attention to the richness of the Arabic language by supporting both a diverse set of Arabic dialects including Modern Standard Arabic (MSA), Levantine, and Egyptian. Fanar, through meticulous curation of the pretraining and instruction-tuning data, is aligned with Arab cultural values.
+
+---
+
+## Model Details
+
+| Attribute                  | Value                              |
+|---------------------------|------------------------------------|
+| Developed by              | [QCRI](https://www.hbku.edu.qa/en/qcri) and [HBKU](https://www.hbku.edu.qa/)                      |
+| Model Type                | Autoregressive Transformer         |
+| Parameter Count           | 8.7 Billion                          |
+| Context Length            | 4096 Tokens                        |
+| Precision                 | bfloat16                           |
+| Input                     | text only                          |
+| Output                    | text only                          |
+| Training Framework        | [LitGPT](https://github.com/Lightning-AI/litgpt) |
+| Pretraining Token Count   | 1 Trillion (ar + en) |
+| SFT Instructions          | 4.5M                               |
+| DPO Preference Pairs      | 250K                               |
+| Languages                 | Arabic, English                    |
+| License                   | Apache 2.0                         |
+
+---
+
+## Model Training
+
+Additional dataset and training details can be found in our [report](https://arxiv.org/pdf/2501.13944).
+
+### Pretraining
+Fanar was continually pretrained on 1T tokens, with a balanced focus on Arabic and English: 450B English tokens from a carefully curated subset of the [Dolma](https://huggingface.co/datasets/allenai/dolma) dataset, 450B Arabic tokens that we collected, parsed, and flitered from a variety of sources, 100B code tokens curated from [The Stack](https://github.com/bigcode-project/the-stack-v2) dataset. Our codebase used the [LitGPT](https://github.com/Lightning-AI/litgpt) framework. 
+
+### Post-training
+Fanar underwent a two-phase post-training pipeline:
+
+| Phase | Method | Size |
+|-------|--------|------|
+| SFT (Supervised Fine-tuning) | 4.5M Instructions |
+| DPO (Direct Preference Optimization) | 250K Preference Pairs |
+
+---
+
+
+## Getting Started
+
+Fanar is compatible with the Hugging Face `transformers` library (≥ v4.40.0). Here's how to load and use the model:
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "your-org/Fanar-1-9B-Instruct"  # replace with actual HF repo
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
+
+messages = [
+    {"role": "user", "content": "ما هي عاصمة قطر؟"},
+]
+
+inputs = tokenizer.apply_chat_template(messages, tokenize=False, return_tensors="pt")
+outputs = model.generate(**tokenizer(inputs, return_tensors="pt", return_token_type_ids=False), max_new_tokens=256)
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+---
+
+## Intended Use
+
+Fanar-1-9B-Instruct is built for:
+
+- Conversational agents (Arabic only or bilingual)
+- Cultural and dialectal question answering in Arabic
+- Educational, governmental, and civic NLP applications in the Arab world
+- Research on Arabic instruction tuning and alignment
+
+Fanar is intended to be deployed as part of a broader AI system. Developers are encouraged to implement proper safeguards to ensure culturally respectful, accurate, and safe deployment. It should not be used to generate or spread **harmful, illegal, or misleading content**
+
+`A version of this model is currently deployed as part of a real-world system at (chat.fanar.qa).`
+
+--- 
+
+## Ethical Considerations & Limitations
+
+Fanar is capable of generating fluent and contextually appropriate responses, but as with any generative model there are uncertainities. The model may produce **biased, offensive, or incorrect outputs**. The model should It is **not suitable for high-stakes decision making** (e.g., legal, medical, or financial advice). Though we have extensively tested Fanar and attempted to mitigate these issues, we cannot redress every possible scenario. Thus, we advise developers to implement safety checks and perform domain-specific fine-tuning for sensitive use cases.
+
+The output generated by this model is not considered a statement of QCRI, HBKU, or any other organization or individual.
+ 
+---
+
+## Evaluation
+
+Evaluation results for Fanar-1-9B-Instruct will be released soon across Arabic and English benchmarks including:
+
+- Arabic MMLU
+- Ar-IFEval
+- TruthfulQA
+- HellaSwag
+- ARC (Easy/Challenge)
+- OpenBookQA
+
+Evaluation was conducted using a modified version of the LM Evaluation Harness and internal cultural alignment benchmarks.
+
+---
+
+## Citation
+
+If you use Fanar in your research or applications, please cite:
+
+```bibtex
+@misc{fanarllm2025,
+      title={Fanar: An Arabic-Centric Multimodal Generative AI Platform}, 
+      author={Fanar Team},
+      year={2025},
+      url={https://arxiv.org/abs/2501.13944}, 
+}
+```
+
+---
+
+## Acknowledgements
+
+This project is a collaboration between [Qatar Computing Research Institute (QCRI)](https://qcri.org) and [Hamad Bin Khalifa University (HBKU)](https://hbku.edu.qa). We thank our engineering and research teams for their efforts in advancing Arabic-centric large language models.
+
+---
+
+## License
+
+This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).