Upload README.md with huggingface_hub

README.md

@@ -1,3 +1,82 @@
 ---
 license: mit
+language:
+  - en
+pipeline_tag: text-generation
+tags:
+  - mental-health
+  - counseling
+  - lora
+  - peft
+  - diffusion-language-model
+  - LLaDA
 ---
+
+# BiGraph-Diffuse
+
+LoRA adapter for **BiGraph-Diffuse**, a retrieval-augmented diffusion language model for empathetic mental health counseling.
+
+## Model Overview
+
+This is a [LoRA](https://arxiv.org/abs/2106.09685) adapter fine-tuned on **LLaDA-8B-Instruct**, a discrete diffusion language model. The adapter is trained on counseling dialogues to generate empathetic, psychologically grounded counselor responses.
+
+### Architecture
+
+- **Base Model**: LLaDA-8B-Instruct (discrete diffusion LM)
+- **Adapter**: LoRA (rank=32, alpha=64, dropout=0.1)
+- **Target Modules**: `q_proj`, `k_proj`, `v_proj`, `o_proj`
+- **Task**: Causal language modeling with masked diffusion loss
+
+Full architecture includes **BiGraph-RAG**, a bipartite graph retrieval system that augments generation with relevant psychological knowledge. Code available at the [GitHub repo](https://github.com/Chekhov0919/BiGraph-Diffuse).
+
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+# Load base model
+base_model_path = "path/to/LLaDA-8B-Instruct"
+tokenizer = AutoTokenizer.from_pretrained(base_model_path, trust_remote_code=True)
+tokenizer.padding_side = "left"
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_path,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True
+)
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(base_model, "Chekhov0919/BiGraph-Diffuse")
+model.eval()
+
+# Generate with diffusion
+# See GitHub repo for full inference code with BiGraph-RAG integration
+```
+
+For the complete inference pipeline with BiGraph-RAG retrieval, refer to the [GitHub repository](https://github.com/Chekhov0919/BiGraph-Diffuse).
+
+## Training
+
+| Setting | Value |
+|---------|-------|
+| Base Model | LLaDA-8B-Instruct |
+| Dataset | CPsyCounD (counseling dialogues) |
+| LoRA rank | 32 |
+| LoRA alpha | 64 |
+| LoRA dropout | 0.1 |
+| Batch size | 2 × 32 (gradient accumulation) |
+| Learning rate | 3e-5 |
+| Epochs | 5 |
+| LR scheduler | Cosine |
+| Mask token ID | 126336 |
+
+## Citation
+
+Please stay tuned — citation information will be added upon publication.
+
+## License
+
+MIT