Upload 10 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,283 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+license: other
+license_name: proprietary
+license_link: LICENSE
+tags:
+  - base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+  - lora
+  - transformers
+  - compliance
+  - nist
+  - control-extraction
+  - regulatory
+---
+
+# NIST Control Extraction LoRA Adapter
+
+A fine-tuned LoRA adapter for **Qwen2.5-7B-Instruct** specifically designed for accurate extraction of security controls from NIST framework documents. This model eliminates hallucination issues present in the base model, ensuring precise identification of controls without mistaking control enhancements or related text as valid controls.
+
+## Key Features
+
+- **Accurate Control Extraction**: Precisely identifies control IDs, titles, and descriptions from framework documents
+- **Reduced Hallucination**: Trained to distinguish between actual controls and control enhancements/related content
+- **Fast Inference**: Processes 492-page NIST documents in ~15 minutes (vs. 27 minutes with base model)
+- **Structured Output**: Returns controls in clean JSON format with `<END>` token for reliable parsing
+
+---
+
+## Model Details
+
+### Model Description
+
+This LoRA adapter enhances the Qwen2.5-7B-Instruct model for the specialized task of extracting security controls from compliance framework documents. The adapter was trained using a custom weighted loss function that penalizes false positives more heavily than false negatives, addressing the critical requirement in compliance auditing where incorrectly identifying a control is more problematic than missing one.
+
+| Property | Value |
+|----------|-------|
+| **Developed by** | Rishit Sharma |
+| **Model Type** | LoRA Adapter (PEFT) |
+| **Base Model** | [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
+| **Language** | English |
+| **Domain** | Compliance & Regulatory Frameworks |
+| **License** | Proprietary - No use allowed without prior permission |
+
+---
+
+## Model Architecture
+
+### LoRA Configuration
+
+| Parameter | Value |
+|-----------|-------|
+| **Rank (r)** | 16 |
+| **Alpha** | 32 |
+| **Dropout** | 0.05 |
+| **Target Modules** | `q_proj`, `k_proj`, `v_proj`, `o_proj` |
+| **Bias** | None |
+| **Task Type** | CAUSAL_LM |
+
+### Quantization (Training)
+
+| Parameter | Value |
+|-----------|-------|
+| **Quantization** | 4-bit (QLoRA) |
+| **Quant Type** | NF4 |
+| **Double Quantization** | Enabled |
+| **Compute Dtype** | bfloat16 |
+
+### Special Tokens
+
+- **`<END>`**: Custom stop token appended to outputs for reliable generation termination
+
+---
+
+## Intended Use
+
+### Primary Use Case
+
+Building autonomous compliance auditing agents that can:
+- Parse and analyze framework documents (PDF/text)
+- Extract structured control information automatically
+- Verify deployment status of controls within an organization
+
+### Target Users
+
+- Compliance Officers & Auditors
+- GRC (Governance, Risk, Compliance) Teams
+- Security Analysts
+- Organizations undergoing NIST compliance assessments
+
+---
+
+## Quick Start
+
+### Installation
+
+```bash
+pip install transformers peft torch accelerate bitsandbytes
+```
+
+### Loading the Model
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from peft import PeftModel
+import torch
+
+# Quantization config (optional, for memory efficiency)
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_use_double_quant=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16
+)
+
+# Load base model
+base_model = AutoModelForCausalLM.from_pretrained(
+    "Qwen/Qwen2.5-7B-Instruct",
+    quantization_config=bnb_config,
+    device_map="auto",
+    trust_remote_code=True
+)
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained("path/to/final_adapter")
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(base_model, "path/to/final_adapter")
+```
+
+### Inference Example
+
+```python
+system_prompt = """You are a senior Compliance Auditor and Regulatory Analyst specialized in ISO, NIST, and statutory frameworks."""
+
+messages = [
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": f"Analyze this text:\n\n{page_text}"}
+]
+
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+outputs = model.generate(
+    input_ids,
+    max_new_tokens=512,
+    temperature=0.1,
+    do_sample=False
+)
+
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+```
+
+### Expected Output Format
+
+```json
+[
+    {
+        "control_id": "AC-1",
+        "control_title": "Access Control Policy and Procedures",
+        "control_desc": "Description of the control..."
+    }
+]
+<END>
+```
+
+---
+
+## Training Details
+
+### Training Configuration
+
+| Parameter | Value |
+|-----------|-------|
+| **Hardware** | NVIDIA RTX 5070 Ti |
+| **Training Time** | ~1 hour |
+| **Epochs** | 14 (with early stopping) |
+| **Batch Size** | 1 (effective: 8 with gradient accumulation) |
+| **Learning Rate** | 1e-5 |
+| **Optimizer** | Paged AdamW 8-bit |
+| **LR Scheduler** | Cosine |
+| **Warmup Ratio** | 0.05 |
+| **Max Gradient Norm** | 0.3 |
+| **Precision** | FP16 |
+
+### Training Data
+
+- **Source**: NIST SP 800-53 Framework (492 pages)
+- **Dataset Creation**: Custom pipeline using Gemini Pro for initial extraction, followed by manual verification
+- **Data Balance**: ~60% positive samples (pages with controls), ~40% negative samples (pages without controls)
+- **Format**: JSONL with chat template structure
+
+### Custom Loss Function
+
+A **Weighted Loss Trainer** was implemented to address the asymmetric cost of errors in compliance:
+
+- **Positive Weight**: 2.0x (penalizes missing actual controls less than falsely identifying controls)
+- **Rationale**: In compliance auditing, falsely identifying a control (hallucination) is more problematic than missing one, as it can lead to incorrect compliance assessments
+
+```python
+# Samples with controls are weighted 2x during loss computation
+weights = torch.where(has_control, 2.0, 1.0)
+weighted_loss = (sample_loss * weights).mean()
+```
+
+---
+
+## 📈 Evaluation & Performance
+
+| Metric | Base Qwen2.5-7B | This Adapter |
+|--------|-----------------|--------------|
+| **Processing Time (492 pages)** | ~27 minutes | ~15 minutes |
+| **Hallucination Rate** | High | Minimal |
+| **Control Enhancement Confusion** | Frequent | Resolved |
+
+---
+
+## ⚠️ Limitations & Risks
+
+### Current Limitations
+
+- **Framework Specificity**: Optimized primarily for NIST SP 800-53; performance on other frameworks (ISO 27001, SOC 2, etc.) may vary
+- **Language**: Trained on English documents only
+- **Document Format**: Best performance on well-structured PDF documents
+
+### Known Risks
+
+- May require additional fine-tuning for non-NIST frameworks
+- Performance depends on input text quality and preprocessing
+- Should be validated by human auditors for critical compliance decisions
+
+### Future Improvements
+
+- [ ] Training on ISO 27001/27002 frameworks
+- [ ] Multi-framework support (SOC 2, HIPAA, PCI-DSS)
+- [ ] Improved handling of complex document layouts
+- [ ] Longer training with expanded dataset
+
+---
+
+## 📄 License
+
+**Proprietary License** - This model is not available for public use without explicit prior permission from the developer.
+
+For licensing inquiries, please contact via the channels below.
+
+---
+
+## 📬 Contact
+
+| Channel | Link |
+|---------|------|
+| **Email** | [rishitshar36@gmail.com](mailto:rishitshar36@gmail.com) |
+| **GitHub** | [github.com/rishit836](https://github.com/rishit836) |
+| **Project Repository** | [control-extraction-using-llm-finetuned](https://github.com/rishit836/control-extraction-using-llm-finetuned) |
+
+---
+
+## 🙏 Acknowledgments
+
+- [Qwen Team](https://github.com/QwenLM/Qwen2.5) for the excellent base model
+- [Hugging Face](https://huggingface.co/) for the Transformers and PEFT libraries
+- NIST for the publicly available SP 800-53 framework documentation
+
+---
+
+## 📚 Citation
+
+If you use this model in your research or project, please cite:
+
+```bibtex
+@misc{sharma2026nist-control-extraction,
+    title={NIST Control Extraction LoRA Adapter for Qwen2.5-7B},
+    author={Sharma, Rishit},
+    year={2026},
+    publisher={GitHub},
+    howpublished={\url{https://github.com/rishit836/control-extraction-using-llm-finetuned}}
+}
+```

adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "o_proj",
+    "k_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bf4dc666d62aa595c65f300110b7411d50eeeb252e371ab8c6ab6f6a8d54c8a
+size 4388968992

added_tokens.json

@@ -0,0 +1,25 @@
+{
+  "</tool_call>": 151658,
+  "<END>": 151665,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

special_tokens_map.json

@@ -0,0 +1,25 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "<END>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fbe196efd3fc4b085e353a7b9c2000540f40f4fd7e8926ad3eba825cfea4597
+size 11422078

tokenizer_config.json

@@ -0,0 +1,203 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<END>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<END>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}