gemechisw/Tenacious-DPO-LoRA-v0.1

Model Card for gemechisw/tenacious-pathb-dpo-lora-v0.1

PEFT LoRA adapter trained with DPO for Tenacious-Bench sales-agent intervention experiments, focused on improving policy/reliability behavior over the Week 10 baseline while preserving reproducibility and explicit failure reporting.

Model Details

Model Description

This artifact is an adapter-only checkpoint (not a full merged foundation model). It was trained on preference pairs derived from Tenacious-Bench v0.1 train/dev splits, where chosen outputs pass benchmark constraints and rejected outputs represent policy/reliability failures.

• Developed by: Gemechis Worku
• Shared by [optional]: Gemechis Worku
• Model type: PEFT LoRA adapter trained with DPO (Path B)
• Language(s) (NLP): English
• License: MIT
• Finetuned from model [optional]: unsloth/Qwen2.5-3B-Instruct-bnb-4bit

Model Sources [optional]

• Repository: https://github.com/gemechisworku/tenacious_bench_v01

Uses

Direct Use

1. Benchmark-time intervention experiments on Tenacious-Bench v0.1 tasks.
2. Reproducing comparisons between baseline, prompt-only, and trained variants.
3. Evaluating preference-tuned behavior on sales-policy-sensitive drafting tasks.

Downstream Use [optional]

1. As an adapter component inside a larger sales-assistant pipeline where outputs are additionally guarded by deterministic policy checks.
2. As a starting point for further preference optimization on expanded hard-policy slices.

Out-of-Scope Use

1. Fully autonomous production outreach without additional policy guardrails.
2. Legal/compliance-sensitive quoting decisions without human review.
3. General-purpose conversational deployment outside the benchmark scope.

Bias, Risks, and Limitations

1. The adapter can produce fluent outputs that still violate hard policy constraints.
2. Remaining known failure clusters include capacity over-commitment, specific TCV quoting, and discount/promo language.
3. Evaluation performance is benchmark-specific and should not be interpreted as universal sales competence.
4. Delta B did not beat a strong prompt-only intervention in this run.

Recommendations

1. Keep a deterministic policy layer in front of any send action.
2. Require human approval for capacity, pricing, and discount claims.
3. Use this model as an experimental component, not as a standalone policy system.
4. Monitor failure-family slices separately, not only aggregate score.

How to Get Started with the Model

Use the code below to load the base model and attach the adapter.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/Qwen2.5-3B-Instruct-bnb-4bit"
adapter_id = "gemechisw/tenacious-pathb-dpo-lora-v0.1"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base, adapter_id)

Training Details

Training Data

Training data comes from Tenacious-Bench v0.1 preference pairs:

1. training_data/path_b/preferences_train_dpo.jsonl (125 pairs)
2. training_data/path_b/preferences_dev_dpo.jsonl (75 pairs)

Dataset reference:

1. https://huggingface.co/datasets/gemechisw/tenacious_bench_v0.1

Preprocessing and controls:

1. Held-out split excluded from preference construction.
2. Chosen/rejected pairs derived under benchmark rubric constraints.
3. Leakage controls documented in methodology_rationale.md

Training Procedure

Preprocessing [optional]

1. Preference pairs were built from train/dev tasks only.
2. Rejected outputs include deterministic-rubric and hard-policy failures.
3. Chosen outputs are corrected outputs that pass threshold criteria.

Training Hyperparameters

• Training regime: Mixed precision (Colab/Unsloth workflow; exact mode to confirm from notebook runtime)
• Method: DPO + LoRA
• Seed: 42
• Max sequence length: 1024
• Optimizer: adamw_8bit
• Learning rate: 1e-5
• Epochs: configured 2 in run config
• Per-device batch size: 2
• Gradient accumulation steps: 4
• DPO beta (selected run): 0.1 (run args include beta: 0.05, selected run metadata records 0.1)
• LoRA config (project runbook): r=16, alpha=32, dropout=0.05
• LoRA target modules: q_proj,k_proj,v_proj,o_proj,up_proj,down_proj,gate_proj

Speeds, Sizes, Times [optional]

1. Train runtime: 994.5271 seconds (~16.6 minutes).
2. Train samples/sec: 0.251.
3. Train steps/sec: 0.032.
4. Reported train loss: 0.03.
5. Artifacts tracked in:
   • training/config.yaml
   • training/metrics.json
   • training/training_run.log

Evaluation

Testing Data, Factors & Metrics

Testing Data

Held-out evaluation used Tenacious-Bench v0.1 held-out split:

1. tenacious_bench_v0.1/held_out/tasks.jsonl (n=50)

Supporting outputs and traces:

1. ablation_results.json
2. held_out_traces.jsonl

Factors

Evaluation disaggregates by variant and policy behavior:

1. Baseline vs prompt-only vs trained intervention.
2. Failure families including signal honesty, bench capacity, pricing scope, and tone constraints.

Metrics

1. Mean score percentage.
2. Pass rate.
3. Paired bootstrap 95% confidence interval for mean differences.
4. One-sided and two-sided p-values for lift claims.

Results

Held-out aggregate (n=50):

1. Baseline mean score: 93.44, pass rate 0.86.
2. Prompt-only mean score: 100.0, pass rate 1.0.
3. Trained mean score: 97.92, pass rate 0.82.

Delta A (trained vs baseline):

1. Mean diff: +4.48.
2. 95% CI: [3.68, 5.44].
3. One-sided p-value: 0.0002.
4. Claim positive with significance: true.

Delta B (trained vs prompt-only):

1. Mean diff: -2.08.
2. 95% CI: [-3.36, -0.96].
3. One-sided p-value: 1.0.
4. Claim training beats prompt-only: false.

Summary

The adapter produced statistically significant lift over the baseline comparator (Delta A), but did not outperform the prompt-only intervention (Delta B). Remaining failures are concentrated in hard-policy honesty slices.

Model Examination [optional]

No dedicated interpretability study was run for this release. Error analysis was performed at trace level via held_out_traces.jsonl and summarized in project documentation.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: NVIDIA T4 (Colab workflow)
• Hours used: ~0.28 hours for the reported training run
• Cloud Provider: Google Colab
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

1. Base architecture: Qwen2.5-3B Instruct variant (4-bit base loading workflow via Unsloth).
2. Adapter method: LoRA on attention + MLP projection modules.
3. Objective: Direct Preference Optimization over chosen/rejected response pairs.
4. Training path: ACT III Path B.

Compute Infrastructure

Colab-based training workflow with adapter-only output artifacts.

Hardware

1. GPU class: T4 (16GB class runtime target)
2. Local development artifacts synchronized into repo for reproducibility.

Software

1. Unsloth training notebook workflow.
2. TRL-based DPO training pipeline.
3. Transformers + PEFT adapter loading stack.

Citation [optional]

BibTeX:

@misc{worku2026tenaciouspathb,
  title        = {Tenacious DPO LoRA v0.1},
  author       = {Gemechis Worku},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  note         = {Adapter model for Tenacious-Bench v0.1}
}

APA:

Worku, G. (2026). Tenacious DPO LoRA v0.1 [Model adapter]. Hugging Face.

Glossary [optional]

1. Delta A: Trained minus baseline held-out mean score difference.
2. Delta B: Trained minus prompt-only held-out mean score difference.
3. DPO: Direct Preference Optimization.
4. LoRA: Low-Rank Adaptation for parameter-efficient fine-tuning.

More Information [optional]

Related project artifacts:

1. Dataset card: https://huggingface.co/datasets/gemechisw/tenacious_bench_v0.1
2. Ablation JSON: https://github.com/gemechisworku/tenacious_bench_v01/blob/main/ablation_results.json
3. Traces JSONL: https://github.com/gemechisworku/tenacious_bench_v01/blob/main/held_out_traces.jsonl
4. Methodology rationale: https://github.com/gemechisworku/tenacious_bench_v01/blob/main/methodology_rationale.md

Model Card Authors [optional]

Gemechis Worku

Model Card Contact

[More Information Needed]

Framework versions

• PEFT 0.19.1