Update to latest weights (run LORA-MISTRAL_7B-20260709-232328); refresh model card with real config, dataset links, Apache-2.0

MODELCARD.md

@@ -3,6 +3,11 @@ language:
 - en
 license: apache-2.0
 base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+pipeline_tag: text-generation
+datasets:
+- dpevzner/Cybersecurity_Reasoning_Dataset_MistralFamily_7b
+- dpevzner/Cybersecurity_Reasoning_Dataset
 tags:
 - cybersecurity
 - lora
@@ -12,19 +17,23 @@ tags:
 - threat-analysis
 - dual-use-detection
 - it-operations
-pipeline_tag: text-generation
-library_name: peft
 model-index:
-- name: CyberOps-Mistral-7B-LoRA
+- name: CyberOps_Mistral_7B_LoRA
   results: []
 ---
 
 # CyberOps-Mistral-7B-LoRA
 
-A LoRA fine-tuned adapter for **Mistral 7B Instruct v0.3**, specialized for cybersecurity IT operations reasoning — dual-use command intent analysis, failure diagnosis, security scope assessment, and cross-shell translation.
+A LoRA fine-tuned adapter for **Mistral 7B Instruct v0.3**, specialized for cybersecurity
+IT-operations reasoning — dual-use command intent analysis, failure diagnosis, security
+scope assessment, tool correctness, and cross-shell translation.
 
 > **Adapter only** — requires `mistralai/Mistral-7B-Instruct-v0.3` as the base model.
 
+**Weights provenance:** `final_adapter` from training run
+`LORA-MISTRAL_7B-20260709-232328` (latest). Best confirmed benchmark across the iterative
+training loop: **88.1** (50-item rubric).
+
 ---
 
 ## Model Details
@@ -32,71 +41,106 @@ A LoRA fine-tuned adapter for **Mistral 7B Instruct v0.3**, specialized for cybe
 | Field | Value |
 |---|---|
 | **Base model** | `mistralai/Mistral-7B-Instruct-v0.3` |
-| **Adapter type** | LoRA (PEFT) |
-| **Trainable parameters** | 41,943,040 (1.10% of base) |
-| **LoRA rank** | 16 |
-| **Training epochs** | 3 |
-| **Training records** | 3,329 |
-| **Precision** | fp16 (base) / fp32 (training) |
-| **Framework** | HuggingFace Transformers + PEFT + TRL |
-| **Hardware** | NVIDIA RTX 4070 Laptop GPU (8GB VRAM) |
-| **Training time** | ~8.5 hours per run |
-| **Training dataset** | [dpevzner/Cybersecurity_Reasoning_Dataset](https://huggingface.co/datasets/dpevzner/Cybersecurity_Reasoning_Dataset) |
+| **Adapter type** | LoRA (PEFT 0.19.1) |
+| **LoRA rank `r`** | 16 |
+| **`lora_alpha`** | 32 |
+| **`lora_dropout`** | 0.05 |
+| **`bias`** | none |
+| **Target modules** | `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` (all linear) |
+| **Trainable parameters** | ~41,943,040 (~1.10% of base); 448 LoRA tensors |
+| **Task type** | `CAUSAL_LM` |
+| **Training method** | Supervised fine-tuning (SFT) via TRL; iterative single-epoch repair on prior best |
+| **Effective batch** | 8 (per-device 1 × grad-accum 8) |
+| **Learning rate** | 2e-4 |
+| **Max sequence length** | 1024 |
+| **Training records** | ~4,615 (123 curriculum files) |
+| **Frameworks** | TRL 1.2.0 · Transformers 5.6.2 · PEFT 0.19.1 · PyTorch 2.5.1+cu121 |
+| **Hardware** | NVIDIA RTX 4070 Laptop GPU (8 GB) · 64 GB RAM · ~5.56 GB peak VRAM |
+| **Best benchmark** | **88.1** avg (50-item rubric) |
 
 ---
 
-## Intended Use
+## Model-Family Compatibility
 
-This model is designed for **offline, air-gapped cybersecurity operations environments**. It is a domain-specific reasoning assistant for IT security testing.
+Optimized for the **Mistral / Llama instruction-tuned family**. The training data uses the
+Alpaca-style instruct format:
 
-### Primary capabilities
+```
+### Instruction:
+[prompt]
+
+### Response:
+[answer]
+```
 
-- **Dual-use command intent analysis** — Given an ambiguous security command (e.g., `Get-Process lsass`, `net user /domain`, `Get-ADUser`), the model produces structured benign and malicious interpretations with specific contextual reasoning.
-- **Failure diagnosis** — Classifies shell and tool failures by type: `privilege_failure`, `syntax_failure`, `environment_mismatch`, `dependency_absence`, `stale_documentation`.
-- **Security scope violation identification** — Flags commands that exceed authorized scope and recommends compliant alternatives.
-- **Tool correctness** — Recommends correct PowerShell, CMD, and bash commands for security-relevant tasks including registry queries, event log analysis, firewall management, and process memory analysis.
-- **Cross-shell translation** — Converts security commands between PowerShell, bash, and CMD with behavioral parity notes.
+**The binding is to this prompt *format*, not the architecture.** The same dataset trained
+on three bases shows the format mismatch — not capability — drives the gap:
 
-### Out of scope
+| Base model | Prompt format | Best benchmark |
+|---|---|---|
+| `mistralai/Mistral-7B-Instruct-v0.3` | native (Alpaca) | **88.1** |
+| `deepseek-ai/deepseek-llm-7b-chat` | mismatched | **47.38** |
+| `microsoft/Phi-3.5-mini-instruct` | mismatched | early experiments |
 
-- General threat intelligence, malware analysis, CVE assessment, and network forensics are not covered in the current training corpus (planned for Phase 3 expansion).
-- Not suitable for use as a general-purpose assistant.
-- Not designed for or tested on offensive security automation.
+Recommended base models for **this** adapter: `mistralai/Mistral-7B-Instruct-v0.3`
+(primary), other Mistral 7B Instruct point releases, or Llama-3/3.1 8B Instruct (compatible
+format; requires retraining on that base). **For DeepSeek/Gemma/ChatML/Qwen, use the
+model-agnostic dataset** (below) rendered in each family's native format.
 
 ---
 
-## Training Data
+## Datasets
 
-The training corpus contains **3,329 synthetic records** across three curriculum epochs:
+- **Training data (format-specific):**
+  [`dpevzner/Cybersecurity_Reasoning_Dataset_MistralFamily_7b`](https://huggingface.co/datasets/dpevzner/Cybersecurity_Reasoning_Dataset_MistralFamily_7b)
+  — ~4,615 records in the Mistral/Llama `### Instruction:`/`### Response:` format.
+- **Model-agnostic variant:**
+  [`dpevzner/Cybersecurity_Reasoning_Dataset`](https://huggingface.co/datasets/dpevzner/Cybersecurity_Reasoning_Dataset)
+  — neutral canonical corpus + Mistral/DeepSeek/ChatML/Gemma rendered variants (to train
+  other families without the format-mismatch penalty).
 
-| Epoch | Folders | Records | Content |
-|---|---|---|---|
-| 1 | `03_toolknowledge`, `05_detection_rules` | 1,070 | Tool syntax, SIGMA rules, network knowledge |
-| 2 | `04_goldens`, `05_seedcases` | 1,331 | Golden responses, execution traces, seedcases |
-| 3 | `06_contrast_pairs` | 928 | Contrast pairs, terminology, format-repair, gap-fill records |
+### Curriculum (hard-ordered, 3-epoch)
 
-No real incident data, PII, or proprietary content is included.
+| Epoch | Folders | Records |
+|---|---|---|
+| 1 | `03_toolknowledge`, `05_detection_rules` | ~1,077 |
+| 2 | `04_goldens`, `05_seedcases` | ~1,508 |
+| 3 | `06_contrast_pairs` | ~2,030 |
 
-The training corpus is derived from and continues to be updated alongside the **[dpevzner/Cybersecurity_Reasoning_Dataset](https://huggingface.co/datasets/dpevzner/Cybersecurity_Reasoning_Dataset)** on HuggingFace, which is maintained as an active part of this project.
+New synthesis/repair records go in Epoch 3; placing them in Epoch 2 previously caused sharp
+regression in ambiguity reasoning (hard constraint). The benchmark is held out with zero
+overlap with training data.
 
 ---
 
-## Benchmark Performance
+## Intended Use
+
+For **offline / air-gapped cybersecurity operations** — a domain-specific reasoning
+assistant for authorized security testing and SOC support.
 
-Evaluated on a 50-item rubric-scored benchmark across 5 families:
+**Capabilities:** dual-use command intent analysis; failure diagnosis (`privilege_failure`,
+`syntax_failure`, `environment_mismatch`, `dependency_absence`, `stale_documentation`);
+security scope assessment; tool correctness (PowerShell/CMD/bash); cross-shell translation.
+**Out of scope:** general threat intel, malware RE, CVE assessment, offensive automation,
+general-purpose assistant use.
 
-| Family | Items | Baseline (Run 6) | Run 13 | Delta |
-|---|---|---|---|---|
-| tool_correctness | 10 | ~73 avg | ~81 avg | +8 |
-| cross_shell_translation | 10 | ~71 avg | ~72 avg | +1 |
-| failure_diagnosis | 10 | ~30 avg | ~70 avg | +40 |
-| ambiguity_reasoning | 10 | ~93 avg | ~84 avg | -9 |
-| safety_scope | 10 | ~10 avg | ~30 avg | +20 |
-| **Overall** | **50** | **56.28** | **69.43** | **+13.15** |
+---
+
+## Benchmark
+
+50-item rubric across five families. Best **88.1**; stable **88.0** most recent.
 
-Baseline (untuned Mistral 7B Instruct v0.3): **27.43 avg**
+| Family | Items | Baseline | Best |
+|---|---|---|---|
+| tool_correctness | 10 | ~73 | ~91.7 |
+| cross_shell_translation | 10 | ~71 | ~97 |
+| failure_diagnosis | 10 | ~30 | ~85 |
+| ambiguity_reasoning | 10 | ~100 | ~95.8 |
+| safety_scope | 10 | ~10 | ~70 |
+| **Overall** | **50** | **56.28** | **88.1** |
 
-> Trained on 3,329 records across a structured three-epoch curriculum. Run 13 represents the best-performing checkpoint, trained from a clean base adapter to prevent format interference from prior synthesis batches.
+Reached a **capacity ceiling at ~88 (rank 16)**: further repairs fix targeted items but
+displace others ~1:1. Progressing requires higher LoRA rank, not more data.
 
 ---
 
@@ -108,15 +152,12 @@ from peft import PeftModel
 import torch
 
 BASE_MODEL = "mistralai/Mistral-7B-Instruct-v0.3"
-ADAPTER    = "your-username/CyberOps-Mistral-7B-LoRA"
+ADAPTER    = "dpevzner/CyberOps_Mistral_7B_LoRA"
 
 tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
 model = AutoModelForCausalLM.from_pretrained(
-    BASE_MODEL,
-    dtype=torch.float16,
-    device_map="auto",
-    max_memory={0: "5GiB", "cpu": "20GiB"}
-)
+    BASE_MODEL, dtype=torch.float16, device_map="auto",
+    max_memory={0: "5GiB", "cpu": "20GiB"})
 model = PeftModel.from_pretrained(model, ADAPTER)
 model.eval()
 
@@ -131,51 +172,43 @@ with torch.no_grad():
 print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
 ```
 
----
-
-## Training Curriculum Design
-
-The three-epoch curriculum is carefully ordered to prevent interference between learning phases:
-
-1. **Epoch 1** teaches foundational tool knowledge and detection rules — syntax, correct commands, platform-specific behavior
-2. **Epoch 2** teaches execution context — golden responses, ambiguity reasoning, security scope awareness
-3. **Epoch 3** teaches contrast and refinement — dual-use contrast pairs, terminology precision, gap repair
-
-> **Critical:** New synthesis batches must always be placed in Epoch 3 (`06_contrast_pairs`). Placing new records in Epoch 2 caused catastrophic regression in ambiguity reasoning (Run 7: 8/10 PASS → 0/10 PASS). The curriculum order is a hard constraint, not a preference.
+This adapter expects the `### Instruction:` / `### Response:` format; other formats degrade
+output quality due to the format binding described above.
 
 ---
 
 ## Limitations
 
-- Ambiguity reasoning scores vary across items — ar_009 and ar_010 consistently score below baseline, indicating the model produces the correct structure but misses specific required keyword anchors for those scenarios.
-- Safety scope detection is improving but inconsistent — ss_001, ss_004, ss_006, ss_007, ss_010 remain at zero across runs.
-- GPU telemetry (util%, temp, power) reads zero due to an NVML polling bug in the training environment — training functioned correctly despite this.
-- Evaluated only on synthetic benchmarks. Real-world performance on live security operations tasks has not been measured.
+- **Adapter-capacity ceiling at ~88 (rank 16)** — repairs fix items but regress others ~1:1.
+- **Safety scope is the weakest family (~70)** — items ss_001/002/006/007/010 inconsistent.
+- **Ambiguity reasoning oscillates** (ar_008/009/010) under heavy contrast-pair synthesis.
+- **Substring-based scoring** — a correctly paraphrased answer can be scored a miss.
+- **GPU telemetry reads zero** (NVML polling gap) — cosmetic; training is correct.
+- **Synthetic evaluation only** — live-operations performance not measured.
 
 ---
 
 ## Ethical Considerations
 
-This model is trained to analyze dual-use commands — commands that have both legitimate administrative uses and potential malicious applications. It is designed for **defensive security operations** — helping analysts understand whether observed commands are likely benign or malicious in context.
-
-The model should not be used to generate offensive security tooling, attack automation, or to assist in unauthorized access to systems. All training data was synthesized under operator oversight with explicit content governance controls.
+Trained to analyze **dual-use** commands for **defensive** security operations. Not for
+generating offensive tooling, attack automation, or unauthorized access. All training data
+was synthesized under operator oversight with explicit content-governance controls.
 
 ---
 
 ## Citation
 
-```
-@misc{cyberops-mistral-7b-lora-2026,
-  title  = {CyberOps-Mistral-7B-LoRA: A LoRA Fine-Tuned Adapter for Cybersecurity IT Operations Reasoning},
+```bibtex
+@misc{cyberops_mistral_7b_lora_2026,
+  title  = {CyberOps-Mistral-7B-LoRA: A LoRA Adapter for Cybersecurity IT-Operations Reasoning},
+  author = {Pevzner, D.},
   year   = {2026},
-  note   = {LoRA adapter for Mistral 7B Instruct v0.3, trained on 3329 synthetic cybersecurity records
-            across a three-epoch curriculum covering tool correctness, ambiguity reasoning,
-            failure diagnosis, security scope analysis, and cross-shell translation.}
+  note   = {LoRA (r=16, alpha=32) adapter for Mistral 7B Instruct v0.3, trained on ~4,615
+            synthetic cybersecurity reasoning records; best benchmark 88.1 on a 50-item rubric.}
 }
 ```
 
 ---
 
-## Model Card Contact
-
-Training environment: Alienware M16R2 / RTX 4070 Laptop GPU / Windows 11.
+*Training environment: Alienware M16R2 / RTX 4070 Laptop GPU / Windows 11. Weights from run
+`LORA-MISTRAL_7B-20260709-232328` (script v4.7).*

README.md

@@ -3,6 +3,11 @@ language:
 - en
 license: apache-2.0
 base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+pipeline_tag: text-generation
+datasets:
+- dpevzner/Cybersecurity_Reasoning_Dataset_MistralFamily_7b
+- dpevzner/Cybersecurity_Reasoning_Dataset
 tags:
 - cybersecurity
 - lora
@@ -12,19 +17,23 @@ tags:
 - threat-analysis
 - dual-use-detection
 - it-operations
-pipeline_tag: text-generation
-library_name: peft
 model-index:
-- name: CyberOps-Mistral-7B-LoRA
+- name: CyberOps_Mistral_7B_LoRA
   results: []
 ---
 
 # CyberOps-Mistral-7B-LoRA
 
-A LoRA fine-tuned adapter for **Mistral 7B Instruct v0.3**, specialized for cybersecurity IT operations reasoning — dual-use command intent analysis, failure diagnosis, security scope assessment, and cross-shell translation.
+A LoRA fine-tuned adapter for **Mistral 7B Instruct v0.3**, specialized for cybersecurity
+IT-operations reasoning — dual-use command intent analysis, failure diagnosis, security
+scope assessment, tool correctness, and cross-shell translation.
 
 > **Adapter only** — requires `mistralai/Mistral-7B-Instruct-v0.3` as the base model.
 
+**Weights provenance:** `final_adapter` from training run
+`LORA-MISTRAL_7B-20260709-232328` (latest). Best confirmed benchmark across the iterative
+training loop: **88.1** (50-item rubric).
+
 ---
 
 ## Model Details
@@ -32,71 +41,106 @@ A LoRA fine-tuned adapter for **Mistral 7B Instruct v0.3**, specialized for cybe
 | Field | Value |
 |---|---|
 | **Base model** | `mistralai/Mistral-7B-Instruct-v0.3` |
-| **Adapter type** | LoRA (PEFT) |
-| **Trainable parameters** | 41,943,040 (1.10% of base) |
-| **LoRA rank** | 16 |
-| **Training epochs** | 3 |
-| **Training records** | 3,329 |
-| **Precision** | fp16 (base) / fp32 (training) |
-| **Framework** | HuggingFace Transformers + PEFT + TRL |
-| **Hardware** | NVIDIA RTX 4070 Laptop GPU (8GB VRAM) |
-| **Training time** | ~8.5 hours per run |
-| **Training dataset** | [dpevzner/Cybersecurity_Reasoning_Dataset](https://huggingface.co/datasets/dpevzner/Cybersecurity_Reasoning_Dataset) |
+| **Adapter type** | LoRA (PEFT 0.19.1) |
+| **LoRA rank `r`** | 16 |
+| **`lora_alpha`** | 32 |
+| **`lora_dropout`** | 0.05 |
+| **`bias`** | none |
+| **Target modules** | `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` (all linear) |
+| **Trainable parameters** | ~41,943,040 (~1.10% of base); 448 LoRA tensors |
+| **Task type** | `CAUSAL_LM` |
+| **Training method** | Supervised fine-tuning (SFT) via TRL; iterative single-epoch repair on prior best |
+| **Effective batch** | 8 (per-device 1 × grad-accum 8) |
+| **Learning rate** | 2e-4 |
+| **Max sequence length** | 1024 |
+| **Training records** | ~4,615 (123 curriculum files) |
+| **Frameworks** | TRL 1.2.0 · Transformers 5.6.2 · PEFT 0.19.1 · PyTorch 2.5.1+cu121 |
+| **Hardware** | NVIDIA RTX 4070 Laptop GPU (8 GB) · 64 GB RAM · ~5.56 GB peak VRAM |
+| **Best benchmark** | **88.1** avg (50-item rubric) |
 
 ---
 
-## Intended Use
+## Model-Family Compatibility
 
-This model is designed for **offline, air-gapped cybersecurity operations environments**. It is a domain-specific reasoning assistant for IT security testing.
+Optimized for the **Mistral / Llama instruction-tuned family**. The training data uses the
+Alpaca-style instruct format:
 
-### Primary capabilities
+```
+### Instruction:
+[prompt]
 
-- **Dual-use command intent analysis** — Given an ambiguous security command (e.g., `Get-Process lsass`, `net user /domain`, `Get-ADUser`), the model produces structured benign and malicious interpretations with specific contextual reasoning.
-- **Failure diagnosis** — Classifies shell and tool failures by type: `privilege_failure`, `syntax_failure`, `environment_mismatch`, `dependency_absence`, `stale_documentation`.
-- **Security scope violation identification** — Flags commands that exceed authorized scope and recommends compliant alternatives.
-- **Tool correctness** — Recommends correct PowerShell, CMD, and bash commands for security-relevant tasks including registry queries, event log analysis, firewall management, and process memory analysis.
-- **Cross-shell translation** — Converts security commands between PowerShell, bash, and CMD with behavioral parity notes.
+### Response:
+[answer]
+```
+
+**The binding is to this prompt *format*, not the architecture.** The same dataset trained
+on three bases shows the format mismatch — not capability — drives the gap:
 
-### Out of scope
+| Base model | Prompt format | Best benchmark |
+|---|---|---|
+| `mistralai/Mistral-7B-Instruct-v0.3` | native (Alpaca) | **88.1** |
+| `deepseek-ai/deepseek-llm-7b-chat` | mismatched | **47.38** |
+| `microsoft/Phi-3.5-mini-instruct` | mismatched | early experiments |
 
-- General threat intelligence, malware analysis, CVE assessment, and network forensics are not covered in the current training corpus (planned for Phase 3 expansion).
-- Not suitable for use as a general-purpose assistant.
-- Not designed for or tested on offensive security automation.
+Recommended base models for **this** adapter: `mistralai/Mistral-7B-Instruct-v0.3`
+(primary), other Mistral 7B Instruct point releases, or Llama-3/3.1 8B Instruct (compatible
+format; requires retraining on that base). **For DeepSeek/Gemma/ChatML/Qwen, use the
+model-agnostic dataset** (below) rendered in each family's native format.
 
 ---
 
-## Training Data
+## Datasets
 
-The training corpus contains **3,329 synthetic records** across three curriculum epochs:
+- **Training data (format-specific):**
+  [`dpevzner/Cybersecurity_Reasoning_Dataset_MistralFamily_7b`](https://huggingface.co/datasets/dpevzner/Cybersecurity_Reasoning_Dataset_MistralFamily_7b)
+  — ~4,615 records in the Mistral/Llama `### Instruction:`/`### Response:` format.
+- **Model-agnostic variant:**
+  [`dpevzner/Cybersecurity_Reasoning_Dataset`](https://huggingface.co/datasets/dpevzner/Cybersecurity_Reasoning_Dataset)
+  — neutral canonical corpus + Mistral/DeepSeek/ChatML/Gemma rendered variants (to train
+  other families without the format-mismatch penalty).
 
-| Epoch | Folders | Records | Content |
-|---|---|---|---|
-| 1 | `03_toolknowledge`, `05_detection_rules` | 1,070 | Tool syntax, SIGMA rules, network knowledge |
-| 2 | `04_goldens`, `05_seedcases` | 1,331 | Golden responses, execution traces, seedcases |
-| 3 | `06_contrast_pairs` | 928 | Contrast pairs, terminology, format-repair, gap-fill records |
+### Curriculum (hard-ordered, 3-epoch)
 
-No real incident data, PII, or proprietary content is included.
+| Epoch | Folders | Records |
+|---|---|---|
+| 1 | `03_toolknowledge`, `05_detection_rules` | ~1,077 |
+| 2 | `04_goldens`, `05_seedcases` | ~1,508 |
+| 3 | `06_contrast_pairs` | ~2,030 |
 
-The training corpus is derived from and continues to be updated alongside the **[dpevzner/Cybersecurity_Reasoning_Dataset](https://huggingface.co/datasets/dpevzner/Cybersecurity_Reasoning_Dataset)** on HuggingFace, which is maintained as an active part of this project.
+New synthesis/repair records go in Epoch 3; placing them in Epoch 2 previously caused sharp
+regression in ambiguity reasoning (hard constraint). The benchmark is held out with zero
+overlap with training data.
 
 ---
 
-## Benchmark Performance
+## Intended Use
+
+For **offline / air-gapped cybersecurity operations** — a domain-specific reasoning
+assistant for authorized security testing and SOC support.
 
-Evaluated on a 50-item rubric-scored benchmark across 5 families:
+**Capabilities:** dual-use command intent analysis; failure diagnosis (`privilege_failure`,
+`syntax_failure`, `environment_mismatch`, `dependency_absence`, `stale_documentation`);
+security scope assessment; tool correctness (PowerShell/CMD/bash); cross-shell translation.
+**Out of scope:** general threat intel, malware RE, CVE assessment, offensive automation,
+general-purpose assistant use.
 
-| Family | Items | Baseline (Run 6) | Run 13 | Delta |
-|---|---|---|---|---|
-| tool_correctness | 10 | ~73 avg | ~81 avg | +8 |
-| cross_shell_translation | 10 | ~71 avg | ~72 avg | +1 |
-| failure_diagnosis | 10 | ~30 avg | ~70 avg | +40 |
-| ambiguity_reasoning | 10 | ~93 avg | ~84 avg | -9 |
-| safety_scope | 10 | ~10 avg | ~30 avg | +20 |
-| **Overall** | **50** | **56.28** | **69.43** | **+13.15** |
+---
 
-Baseline (untuned Mistral 7B Instruct v0.3): **27.43 avg**
+## Benchmark
 
-> Trained on 3,329 records across a structured three-epoch curriculum. Run 13 represents the best-performing checkpoint, trained from a clean base adapter to prevent format interference from prior synthesis batches.
+50-item rubric across five families. Best **88.1**; stable **88.0** most recent.
+
+| Family | Items | Baseline | Best |
+|---|---|---|---|
+| tool_correctness | 10 | ~73 | ~91.7 |
+| cross_shell_translation | 10 | ~71 | ~97 |
+| failure_diagnosis | 10 | ~30 | ~85 |
+| ambiguity_reasoning | 10 | ~100 | ~95.8 |
+| safety_scope | 10 | ~10 | ~70 |
+| **Overall** | **50** | **56.28** | **88.1** |
+
+Reached a **capacity ceiling at ~88 (rank 16)**: further repairs fix targeted items but
+displace others ~1:1. Progressing requires higher LoRA rank, not more data.
 
 ---
 
@@ -108,15 +152,12 @@ from peft import PeftModel
 import torch
 
 BASE_MODEL = "mistralai/Mistral-7B-Instruct-v0.3"
-ADAPTER    = "your-username/CyberOps-Mistral-7B-LoRA"
+ADAPTER    = "dpevzner/CyberOps_Mistral_7B_LoRA"
 
 tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
 model = AutoModelForCausalLM.from_pretrained(
-    BASE_MODEL,
-    dtype=torch.float16,
-    device_map="auto",
-    max_memory={0: "5GiB", "cpu": "20GiB"}
-)
+    BASE_MODEL, dtype=torch.float16, device_map="auto",
+    max_memory={0: "5GiB", "cpu": "20GiB"})
 model = PeftModel.from_pretrained(model, ADAPTER)
 model.eval()
 
@@ -131,50 +172,43 @@ with torch.no_grad():
 print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
 ```
 
----
-
-## Training Curriculum Design
-
-The three-epoch curriculum is carefully ordered to prevent interference between learning phases:
-
-1. **Epoch 1** teaches foundational tool knowledge and detection rules — syntax, correct commands, platform-specific behavior
-2. **Epoch 2** teaches execution context — golden responses, ambiguity reasoning, security scope awareness
-3. **Epoch 3** teaches contrast and refinement — dual-use contrast pairs, terminology precision, gap repair
-
-> **Critical:** New synthesis batches must always be placed in Epoch 3 (`06_contrast_pairs`). Placing new records in Epoch 2 caused catastrophic regression in ambiguity reasoning (Run 7: 8/10 PASS → 0/10 PASS). The curriculum order is a hard constraint, not a preference.
+This adapter expects the `### Instruction:` / `### Response:` format; other formats degrade
+output quality due to the format binding described above.
 
 ---
 
 ## Limitations
 
-- Ambiguity reasoning scores vary across items — ar_009 and ar_010 consistently score below baseline, indicating the model produces the correct structure but misses specific required keyword anchors for those scenarios.
-- Safety scope detection is improving but inconsistent — ss_001, ss_004, ss_006, ss_007, ss_010 remain at zero across runs.
-- GPU telemetry (util%, temp, power) reads zero due to an NVML polling bug in the training environment — training functioned correctly despite this.
-- Evaluated only on synthetic benchmarks. Real-world performance on live security operations tasks has not been measured.
+- **Adapter-capacity ceiling at ~88 (rank 16)** — repairs fix items but regress others ~1:1.
+- **Safety scope is the weakest family (~70)** — items ss_001/002/006/007/010 inconsistent.
+- **Ambiguity reasoning oscillates** (ar_008/009/010) under heavy contrast-pair synthesis.
+- **Substring-based scoring** — a correctly paraphrased answer can be scored a miss.
+- **GPU telemetry reads zero** (NVML polling gap) — cosmetic; training is correct.
+- **Synthetic evaluation only** — live-operations performance not measured.
 
 ---
 
 ## Ethical Considerations
 
-This model is trained to analyze dual-use commands — commands that have both legitimate administrative uses and potential malicious applications. It is designed for **defensive security operations** — helping analysts understand whether observed commands are likely benign or malicious in context.
-
-The model should not be used to generate offensive security tooling, attack automation, or to assist in unauthorized access to systems. All training data was synthesized under operator oversight with explicit content governance controls.
+Trained to analyze **dual-use** commands for **defensive** security operations. Not for
+generating offensive tooling, attack automation, or unauthorized access. All training data
+was synthesized under operator oversight with explicit content-governance controls.
 
 ---
 
 ## Citation
 
-```
-@misc{cyberops-mistral-7b-lora-2026,
-  title  = {CyberOps-Mistral-7B-LoRA: A LoRA Fine-Tuned Adapter for Cybersecurity IT Operations Reasoning},
+```bibtex
+@misc{cyberops_mistral_7b_lora_2026,
+  title  = {CyberOps-Mistral-7B-LoRA: A LoRA Adapter for Cybersecurity IT-Operations Reasoning},
+  author = {Pevzner, D.},
   year   = {2026},
-  note   = {LoRA adapter for Mistral 7B Instruct v0.3, trained on 3329 synthetic cybersecurity records
-            across a three-epoch curriculum covering tool correctness, ambiguity reasoning,
-            failure diagnosis, security scope analysis, and cross-shell translation.}
+  note   = {LoRA (r=16, alpha=32) adapter for Mistral 7B Instruct v0.3, trained on ~4,615
+            synthetic cybersecurity reasoning records; best benchmark 88.1 on a 50-item rubric.}
 }
 ```
 
+---
 
-## Model Card Contact
-
-Training environment: Alienware M16R2 / RTX 4070 Laptop GPU / Windows 11.
+*Training environment: Alienware M16R2 / RTX 4070 Laptop GPU / Windows 11. Weights from run
+`LORA-MISTRAL_7B-20260709-232328` (script v4.7).*

adapter_config.json

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
-    "down_proj",
-    "v_proj",
+    "o_proj",
+    "q_proj",
     "up_proj",
+    "v_proj",
     "gate_proj",
-    "o_proj",
-    "q_proj"
+    "k_proj",
+    "down_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3634aff20134bb16c29a0cb2d2e82cf81727d968b66ffe27424472b6a0c1679f
+oid sha256:061bc08f9913730b2c60a761df90fcd98fa58d314832e0b37780a7d42f288766
 size 83946192

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:183da78f2b91f0f88eb543a917e7707ca1073512f1b342aaba09d130f8af0486
+oid sha256:1f45c274daf6e462bc3cf83e22652a45487780ef4e7bd4d16243b7566b8f855c
 size 5496