Update README.md

README.md

@@ -8,6 +8,12 @@ tags:
 - peft
 - presentation-templates
 - information-retrieval
+- gemma
+base_model: unsloth/gemma-3-4b-it
+datasets:
+- cyberagent/crello
+language:
+- en
 ---
 
 # Field-adaptive-query-generator
@@ -15,7 +21,7 @@ tags:
 ## Model Details
 
 ### Model Description
-A fine-tuned text generation model for query generation from presentation template metadata. This model uses LoRA adapters to efficiently fine-tune Microsoft Phi-2 for generating diverse and relevant content.
+A fine-tuned text generation model for query generation from presentation template metadata. This model uses LoRA adapters to efficiently fine-tune Google Gemma-3-4B-IT for generating diverse and relevant search queries as part of the Field-Adaptive Dense Retrieval framework.
 
 **Developed by:** Mudasir Syed (mudasir13cs)
 
@@ -25,35 +31,42 @@ A fine-tuned text generation model for query generation from presentation templa
 
 **License:** Apache 2.0
 
-**Finetuned from model:** microsoft/Phi-2
+**Finetuned from model:** unsloth/gemma-3-4b-it
+
+**Paper:** [Field-Adaptive Dense Retrieval of Structured Documents](https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544)
 
 ### Model Sources
-**Repository:** https://github.com/mudasir13cs/hybrid-search
+- **Repository:** https://github.com/mudasir13cs/hybrid-search
+- **Paper:** https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544
+- **Base Model:** https://huggingface.co/unsloth/gemma-3-4b-it
 
 ## Uses
 
 ### Direct Use
-This model is designed for generating query generation from presentation template metadata including titles, descriptions, industries, categories, and tags.
+This model is designed for generating search queries from presentation template metadata including titles, descriptions, industries, categories, and tags. It serves as a key component in the Field-Adaptive Dense Retrieval system for structured documents.
 
 ### Downstream Use
 - Content generation systems
 - SEO optimization tools
 - Template recommendation engines
 - Automated content creation
+- Field-adaptive search query generation
+- Dense retrieval systems for structured documents
+- Query expansion and reformulation
 
 ### Out-of-Scope Use
 - Factual information generation
 - Medical or legal advice
 - Harmful content generation
-- Tasks unrelated to presentation templates
+- Tasks unrelated to presentation templates or structured document retrieval
 
 ## Bias, Risks, and Limitations
 - The model may generate biased or stereotypical content based on training data
 - Generated content should be reviewed for accuracy and appropriateness
 - Performance depends on input quality and relevance
+- Model outputs are optimized for presentation template domain
 
 ## How to Get Started with the Model
-
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 import torch
@@ -75,65 +88,85 @@ print(generated_text)
 
 ### Training Data
 - **Dataset:** Presentation template dataset with metadata
-- **Size:** Custom dataset with template-description pairs
-- **Source:** Curated presentation template collection
+- **Size:** Custom dataset with template-query pairs
+- **Source:** Curated presentation template collection from structured documents
+- **Domain:** Presentation templates with field-adaptive metadata
 
 ### Training Procedure
-- **Architecture:** Microsoft Phi-2 with LoRA adapters
+- **Architecture:** Google Gemma-3-4B-IT with LoRA adapters
+- **Base Model:** unsloth/gemma-3-4b-it
 - **Loss Function:** Cross-entropy loss
 - **Optimizer:** AdamW
 - **Learning Rate:** 2e-4
 - **Batch Size:** 4
 - **Epochs:** 3
+- **Framework:** Unsloth for efficient fine-tuning
 
 ### Training Hyperparameters
-- **Training regime:** Supervised fine-tuning with LoRA
+- **Training regime:** Supervised fine-tuning with LoRA (PEFT)
 - **LoRA Rank:** 16
 - **LoRA Alpha:** 32
 - **Hardware:** GPU (NVIDIA)
 - **Training time:** ~3 hours
+- **Fine-tuning method:** Parameter-Efficient Fine-Tuning (PEFT)
 
 ## Evaluation
 
 ### Testing Data, Factors & Metrics
 - **Testing Data:** Validation split from template dataset
-- **Factors:** Content quality, relevance, diversity
+- **Factors:** Content quality, relevance, diversity, field-adaptive retrieval performance
 - **Metrics:** 
   - BLEU score
   - ROUGE score
   - Human evaluation scores
+  - Query relevance metrics
+  - Retrieval accuracy metrics
 
 ### Results
 - **BLEU Score:** ~0.75
 - **ROUGE Score:** ~0.80
-- **Performance:** Optimized for query generation quality
+- **Performance:** Optimized for query generation quality in structured document retrieval
+- **Domain:** High performance on presentation template metadata
 
 ## Environmental Impact
 - **Hardware Type:** NVIDIA GPU
 - **Hours used:** ~3 hours
 - **Cloud Provider:** Local/Cloud
-- **Carbon Emitted:** Minimal (LoRA training)
+- **Carbon Emitted:** Minimal (LoRA training with efficient Unsloth framework)
 
 ## Technical Specifications
 
 ### Model Architecture and Objective
-- **Architecture:** Transformer decoder with LoRA adapters
-- **Objective:** Generate relevant query generation from template metadata
-- **Input:** Template metadata (title, description, industries, etc.)
-- **Output:** Generated text (queries or descriptions)
+- **Base Architecture:** Google Gemma-3-4B-IT transformer decoder
+- **Adaptation:** LoRA adapters for parameter-efficient fine-tuning
+- **Objective:** Generate relevant search queries from template metadata for field-adaptive dense retrieval
+- **Input:** Template metadata (title, description, industries, categories, tags)
+- **Output:** Generated search queries for structured document retrieval
 
 ### Compute Infrastructure
 - **Hardware:** NVIDIA GPU
-- **Software:** PyTorch, Transformers, PEFT
+- **Software:** PyTorch, Transformers, PEFT, Unsloth
 
 ## Citation
 
-**BibTeX:**
+**Paper:**
+```bibtex
+@article{field_adaptive_dense_retrieval,
+  title={Field-Adaptive Dense Retrieval of Structured Documents},
+  author={Mudasir Syed},
+  journal={DBPIA},
+  year={2024},
+  url={https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544}
+}
+```
+
+**Model:**
 ```bibtex
 @misc{field_adaptive_query_generator,
   title={Field-adaptive-query-generator for Presentation Template Query Generation},
   author={Mudasir Syed},
   year={2024},
+  howpublished={Hugging Face},
   url={https://huggingface.co/mudasir13cs/Field-adaptive-query-generator}
 }
 ```
@@ -147,8 +180,10 @@ Mudasir Syed (mudasir13cs)
 ## Model Card Contact
 - **GitHub:** https://github.com/mudasir13cs
 - **Hugging Face:** https://huggingface.co/mudasir13cs
+- **LinkedIn:** https://pk.linkedin.com/in/mudasir-sayed
 
 ## Framework versions
-- Transformers: 4.35.0
-- PEFT: 0.16.0
-- PyTorch: 2.0.0
+- Transformers: 4.35.0+
+- PEFT: 0.16.0+
+- PyTorch: 2.0.0+
+- Unsloth: Latest