Updated Model Card

README.md

@@ -0,0 +1,102 @@
+---
+base_model:
+- deepseek-ai/deepseek-coder-1.3b-base
+finetuned version: aiswaryards/deepseek1.3B-coder-dora-finetuned
+
+tags:
+- '`DoRA`'
+- '`question-generation`'
+- '`data-science`'
+- '`knowledge-transfer`'
+- '`rag`'
+- '`llm`'
+- '`agents`'
+- '`deepseek`'
+
+model_type: causal-lm
+---
+## Quick Overview:
+
+The deployed model `aiswaryards/deepseek1.3B-coder-dora-finetuned` is a fine-tuned version of [DeepSeek-Coder 1.3B](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base), adapted using the **DoRA (Decoupled Low-Rank Adaptation)** method.
+
+It was trained as part of a **graduate-level academic project** focused on simulating expert question generation from domain-specific Knowledge Transfer (KT) documents.
+
+To fine-tune a compact and capable LLM to generate **high-quality, context-specific technical questions** based on internal handover documents in a Retrieval-Augmented Generation (RAG) pipeline.
+
+
+  
+## Use Case
+- Designed to simulate domain expert handover, where the model generates precise and context-aware questions to extract undocumented insights from KT documents.
+- Create prompts that makes the 
+
+## Model Information:
+Base: deepseek-ai/deepseek-coder-1.3b-base
+
+Fine-tuning: (DoRA) Dynamically Optimized Rank Adaptation
+
+Format: Instruction → Input (context) → Response (question)
+
+## Fine-Tuning Details
+
+- **Base Model**: `deepseek-ai/deepseek-coder-1.3b-base`
+
+- **Adapter Method**: DoRA using `AdaLoraConfig` from PEFT
+  - `r`: 8
+  - `lora_alpha`: 16
+  - `lora_dropout`: 0.05
+  - `bias`: none
+  - `target_modules`: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
+  - use_dora=True
+
+- **Quantization**: bfloat16
+
+- **Data Source**: Webscrapped dataset (~100+ high-quality Q&A pairs from Knowledge Transfer docs)
+
+- **Training**:
+  - Epochs: 5
+  - Batch Size: 1 (gradient accumulation: 8)
+  - Optimized using Hugging Face `Trainer`
+  - Platform: Google Colab (A100)
+  - Quantization: 16-bit (bfloat16)
+  - Hardware: Google Colab - (A100 GPU) - cuda
+  - Frameworks: HuggingFace Transformers, PEFT, Datasets, Trainer
+  - Tokenization length: 1024
+  - Trained on :  high-quality Q&A pairs collected from various branches of data science domain KT handovers.
+
+
+## Training Performance:
+Step	Train Loss	Validation Loss
+400	      1.780	        1.795
+
+- This exhibits a strong convergence for a fine-tuned instruction. 
+- The validation loss curve flattens under 2.0, which is a great sign for generative quality.
+
+
+## Use Cases
+
+- Domain Expert Simulation (Agentic RAG)
+- Knowledge Transfer Automation
+- Multiple roles - currently experimented on datascience domain (for eg. BI / Data Engineering role transitions)
+- Question synthesis for downstream QA chains
+
+## Challenges:
+
+- It requires instruction-style prompting. 
+- Might hallucinate if context is vague or unrelated  
+- It is best suited within structured RAG pipelines
+
+## License
+
+---
+license: deepseek
+---
+
+### DeepSeek-Coder 1.3B - DoRA Fine-tuned Version
+
+This model is a fine-tuned version of `deepseek-ai/deepseek-coder-1.3b-base` using the DoRA method on academic Q&A data from Knowledge Transfer documents.
+
+...
+
+This model inherits the original license from [DeepSeek-AI](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base), which can be found [here](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base/blob/main/LICENSE).
+
+For academic use only. Please refer to the original license terms for reuse, distribution, or commercial applications.