Upload folder using huggingface_hub

README.md

@@ -0,0 +1,146 @@
+---
+license: apache-2.0
+language:
+- fa
+- en
+library_name: transformers
+tags:
+- llama
+- persian
+- farsi
+- question-answering
+- scientific-qa
+- text-generation
+- instruction-following
+- LoRA
+---
+
+# PersianSciQA-LLaMA-13B
+## A Context-Adherent Question Answering Model for Persian Scientific Texts
+
+This model is a fine-tuned version of `ViraIntelligentDataMining/PersianLLaMA-13B`, specifically re-aligned to perform reliable, context-bound Question Answering on Persian scientific documents. Its key feature is its ability to **mitigate hallucination** by refusing to answer when the context does not contain the required information. This work was developed by Safora Jolfaei.
+
+این مدل یک نسخه فاین-تیون شده از `ViraIntelligentDataMining/PersianLLaMA-13B` است که به طور ویژه برای پاسخگویی به سوالات بر اساس متن (Question Answering) در حوزه متون علمی فارسی تنظیم شده است. ویژگی اصلی این مدل، **کاهش توهم (Hallucination)** از طریق خودداری از پاسخگویی در مواقعی است که اطلاعات لازم در متن زمینه وجود ندارد. این مدل توسط صفورا جلفائی توسعه داده شده است.
+
+## Model Description
+
+This model is the result of a two-stage fine-tuning process designed to correct "task-model misalignment."
+
+1.  **Initial Fine-tuning (`safora/PersianSciQA-LoRA`)**: The base model was first fine-tuned to identify salient information in scientific abstracts, creating an effective "relevance detector" (`safora/PersianSciQA-LoRA`). However, this initial model suffered from hallucination in a RAG pipeline.
+2.  **Corrective Fine-tuning (This Model)**: Using a "Teacher/Editor" methodology, the `safora/PersianSciQA-LoRA` adapter was further fine-tuned on a new, high-quality, context-bound instruction dataset. This process "edited" the model's behavior, explicitly teaching it to adhere strictly to the provided context and to output `CANNOT_ANSWER` when the information is absent.
+
+This process makes the model a reliable tool for applications requiring high-fidelity, grounded generation.
+
+## Intended Use & How to Use
+
+This model is intended for generative question answering where the answer must be derived solely from a given context. It follows a specific instruction format.
+
+**Installation:**
+```bash
+pip install transformers torch sentencepiece accelerate
+
+Usage Example:
+
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_id = "safora/PersianSciQA-LLaMA-13B"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+def get_response(context, question):
+    prompt = f"""<s>[INST] با توجه به متن زیر:
+{context}
+
+به این سوال پاسخ بده:
+{question} [/INST]"""
+
+    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+    outputs = model.generate(**inputs, max_new_tokens=256, pad_token_id=tokenizer.eos_token_id)
+    
+    response = tokenizer.decode(outputs[0, inputs.input_ids.shape[1]:], skip_special_tokens=True)
+    return response
+
+# --- Example 1: Answer is in the context ---
+context1 = "در این پژوهش، یک الگوریتم جدید برای بهینه‌سازی مصرف انرژی در شبکه‌های حسگر بی‌سیم ارائه شده است که منجر به افزایش ۳۰ درصدی طول عمر شبکه می‌شود."
+question1 = "الگوریتم جدید چه تاثیری بر شبکه دارد؟"
+
+print(f"Question 1: {question1}")
+print(f"Answer 1: {get_response(context1, question1)}\n")
+# Expected Output: این الگوریتم باعث افزایش ۳۰ درصدی طول عمر شبکه می‌شود.
+
+# --- Example 2: Answer is NOT in the context ---
+context2 = "این مقاله به بررسی تاثیر ورزش بر سلامت روان در نوجوانان می‌پردازد."
+question2 = "هزینه انجام این تحقیق چقدر بوده است؟"
+
+print(f"Question 2: {question2}")
+print(f"Answer 2: {get_response(context2, question2)}")
+# Expected Output: CANNOT_ANSWER
+
+Limitations and Bias
+The primary feature of this model is its designed limitation:
+
+Refusal to Answer: The model was explicitly trained to output the literal string CANNOT_ANSWER when the provided context does not contain the necessary information to answer a question. This is not an error, but the intended behavior to prevent factual invention.
+
+Domain Specificity: The model's expertise is in the domain of scientific and academic Persian text. Its performance on other domains (e.g., conversational or literary text) may be suboptimal.
+
+Bias: As the model is based on PersianLLaMA-13B, it may inherit any biases present in the original pre-training data.
+
+Fine-tuning Details
+The corrective fine-tuning was performed using LoRA on a single NVIDIA A100 GPU.
+
+Base Model: ViraIntelligentDataMining/PersianLLaMA-13B merged with the initial adapter safora/PersianSciQA-LoRA.
+
+Dataset: safora/PersianSciQA-Extractive (8,232 instruction-answer pairs).
+
+LoRA Rank (r): 16
+
+LoRA Alpha (alpha): 32
+
+Learning Rate: 5e-6 with a cosine scheduler
+
+Epochs: 3
+
+Effective Batch Size: 8
+
+Precision: bfloat16
+
+Citation
+If you use this model or dataset in your research, please consider citing the following works.
+
+This Model:
+
+@misc{jolfaei2024persiansciqa,
+  author = {Jolfaei, Safora},
+  title = {PersianSciQA-LLaMA-13B: A Context-Adherent QA Model for Persian Scientific Texts},
+  year = {2024},
+  publisher = {Hugging Face},
+  journal = {Hugging Face repository},
+  howpublished = {\url{[https://huggingface.co/safora/PersianSciQA-LLaMA-13B](https://huggingface.co/safora/PersianSciQA-LLaMA-13B)}}
+}
+
+Dataset:
+
+@misc{jolfaei2024persiansciqa_extractive,
+  author = {Jolfaei, Safora},
+  title = {PersianSciQA-Extractive: A Context-Bound Instruction Dataset for Persian Scientific QA},
+  year = {2024},
+  publisher = {Hugging Face},
+  journal = {Hugging Face repository},
+  howpublished = {\url{[https://huggingface.co/datasets/safora/PersianSciQA-Extractive](https://huggingface.co/datasets/safora/PersianSciQA-Extractive)}}
+}
+
+Base Model:
+
+@misc{persianllama,
+  author = {Vira Intelligent Data Mining},
+  title = {PersianLLaMA-13B},
+  year = {2023},
+  publisher = {Hugging Face},
+  journal = {Hugging Face repository},
+  howpublished = {\url{[https://huggingface.co/ViraIntelligentDataMining/PersianLLaMA-13B](https://huggingface.co/ViraIntelligentDataMining/PersianLLaMA-13B)}}
+}