Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,79 @@
+# FLAN-T5-Small Fine-Tuned on Red Hat Documentation
+
+## Overview
+This repository hosts a fine-tuned **FLAN-T5-Small** model for question-answering tasks on Red Hat documentation. The model was fine-tuned using Low-Rank Adaptation (LoRA) and 4-bit quantization on a Google Colab T4 GPU (~15 GB VRAM, CUDA 11.8). The training dataset, `redhat-docs_dataset`, contains 55,741 rows in JSONL format with fields: `title`, `content`, `command`, and `url`. The model excels at extracting commands (e.g., `yum install X`) and summarizing procedures.
+
+Project details are available on GitHub: [mtptisid/FLAN-T5-Small_finetuning_LoRA](https://github.com/mtptisid/FLAN-T5-Small_finetuning_LoRA).
+
+## Model Details
+- **Base Model**: `google/flan-t5-small` (77M parameters).
+- **Fine-Tuning**: LoRA (r=8, alpha=32, target_modules=["q", "v"], dropout=0.1).
+- **Quantization**: 4-bit NormalFloat (nf4) with bfloat16 compute dtype (~6-8 GB VRAM).
+- **Task**: Question-answering on Red Hat documentation.
+
+## Dataset
+The `redhat-docs_dataset` contains 55,741 entries:
+- **title**: Documentation section title.
+- **content**: Detailed procedure or concept.
+- **command**: Associated command (may be `null`).
+- **url**: Reference URL.
+
+### Preprocessing
+- Null `command` fields set to `""`; missing `title`/`content` set to `"Untitled"`/`""`.
+- Formatted into `text` field: "Title: {title}
+Content: {content}
+Command: {command}".
+- Tokenized with max_length=512, truncation, and padding.
+
+### Artifacts
+- `data/redhat-docs_dataset.jsonl`: Original dataset.
+- `data/formatted_dataset.jsonl`: Preprocessed dataset.
+- `data/tokenized_dataset.jsonl`: Tokenized dataset.
+
+## Training
+- **Hardware**: NVIDIA T4 GPU, CUDA 11.8.
+- **Epochs**: 2 (~4-8 hours).
+- **Batch Size**: Effective 32 (4 per-device, 8 gradient accumulation steps).
+- **Optimizer**: Paged AdamW 8-bit.
+- **Mixed Precision**: FP16.
+- **Dependencies**: PyTorch 2.3.1, Transformers 4.46.0, BitsAndBytes 0.43.3, Triton 2.0.0, Datasets 3.0.2, PEFT 0.13.2.
+
+## Repository Structure
+- `model/`: Model weights and tokenizer.
+- `data/`: Dataset files.
+- `finetune_script.py`: Training script.
+- `README.md`: This file.
+
+## Usage
+Load the model for inference on Red Hat documentation queries. Example:
+- **Question**: "How do I install Package X?"
+- **Context**: "Title: Installing Package X
+Content: To install Package X, use the package manager yum.
+Command: yum install X"
+- **Output**: "Run `yum install X`"
+
+## Installation
+Requires PyTorch 2.3.1, Transformers 4.46.0, and a GPU for 4-bit quantization.
+
+## Limitations
+- Dataset may have `null` commands, affecting some queries.
+- Trained for 2 epochs; more epochs may improve performance.
+- Specialized for Red Hat documentation.
+
+## Future Work
+- Add synthetic Q&A data.
+- Implement retrieval for dynamic context.
+- Evaluate with BLEU/ROUGE metrics.
+
+## License
+MIT License. Verify `redhat-docs_dataset` licensing separately.
+
+## Acknowledgments
+- Google FLAN-T5 Team
+- Hugging Face
+- Red Hat Documentation Team
+
+## Contact
+Open issues on [GitHub](https://github.com/mtptisid/FLAN-T5-Small_finetuning_LoRA) or contact `mtpti5iD` via Hugging Face.
+
+*Last Updated: June 17, 2025*