README.md

# ReCon: Contrastive Explanation Model for OWL Reasoning (LoRA)

## Model Details

### Model Description

ReCon is a LoRA fine-tuned language model based on **meta-llama/Llama-3.1-8B-Instruct**, designed to generate **contrastive natural language explanations** for OWL reasoning tasks.

The model translates structured outputs from a Description Logic (DL) reasoner into human-readable explanations that answer:

> *“Why does fact A satisfy a class expression while foil B does not?”*

Unlike standard explanations, ReCon focuses on **contrastive reasoning**, highlighting:

* **Common evidence** (shared properties)
* **Differences** (missing or distinguishing properties)
* **Conflicts** (inconsistencies)

This work is part of a system combining:

* Symbolic reasoning (OWL reasoner)
* Natural language explanation (LLM)
* Graph-based visualization



---

### Key Contributions

* Converts **formal OWL reasoning outputs (JSON)** into natural language
* Reduces hallucination via **strict evidence-grounded prompting**
* Supports **contrastive explanations (fact vs foil)**
* Integrated into:

  * Flask-based UI

---

### Model Information

* **Developed by:** Akash, Ashik and Sam (DICE Group, Paderborn University)
* **Model type:** Causal Language Model (LoRA adapted)
* **Base model:** meta-llama/Llama-3.1-8B-Instruct
* **Language(s):** English (primary), German (supported)
* **License:** Same as base model (Llama 3.1 license)

---

## Intended Use

### Direct Use

* OWL reasoning explanation generation
* Contrastive reasoning (fact vs foil)
* Explainable AI (XAI) workflows
* Knowledge graph interpretation

### Downstream Use

* Integration with reasoning pipelines
* Explainability layer for symbolic + neural systems
* Educational tools for ontology reasoning

### Out-of-Scope Use

* General chatbot usage
* Open-ended reasoning without structured input
* Tasks requiring external world knowledge

---

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "meta-llama/Llama-3.1-8B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(base_model)
base = AutoModelForCausalLM.from_pretrained(base_model)

model = PeftModel.from_pretrained(base, "vanishingMonk/Contrastive_Explainer")
```

---

## Training Details

### Training Objective

The model is fine-tuned to generate **faithful, structured explanations grounded strictly in reasoning evidence**, rather than free-form generation.

---

### Training Data

Training data was constructed from OWL reasoning tasks using datasets such as:

* Family ontology
* Carcinogenesis
* Lymphography

Each sample includes:

* Class expression
* Fact–foil pair
* Structured reasoning output:

  * `common`
  * `different`
  * `conflicts`

Both formats were used:

* Instruction-based samples
* Multi-turn chat-based samples



---

### Prompt Engineering

A strict **three-layer prompt structure** was used:

1. **System Prompt**

   * Enforces evidence-only reasoning
   * Controls output format
   * Language constraints

2. **Role Prompt**

   * Defines model as:

     > "Formal logician + OWL reasoner debugger"

3. **Context Prompt**

   * Defines semantics of:

     * common / different / conflicts
   * Enforces exact reasoning patterns
   * Fixes verdict:

     * FACT = satisfies
     * FOIL = fails

This significantly reduces hallucinations and improves logical consistency.



---

### Training Setup

* **Method:** LoRA (PEFT)
* **Rank (r):** 8
* **Alpha:** 16
* **Target modules:**

  * q_proj, v_proj, o_proj
  * gate_proj, up_proj, down_proj
* **Precision:** bf16
* **Framework:** Torchtune

---

## Evaluation

### General Reasoning Benchmarks

| Benchmark      | Metric   | Fine-tuned | Base  |
| -------------- | -------- | ---------- | ----- |
| ARC-Challenge  | Accuracy | 0.544      | 0.553 |
| HellaSwag (EN) | Accuracy | 0.790      | 0.796 |
| HellaSwag (DE) | Accuracy | 0.615      | 0.615 |
| TruthfulQA     | Accuracy | 0.533      | 0.545 |

→ Minimal degradation in general reasoning performance

---

### Explanation Quality

| Metric    | Fine-tuned | Base   |
| --------- | ---------- | ------ |
| ROUGE-1   | 0.4066     | 0.5845 |
| ROUGE-2   | 0.1424     | 0.3443 |
| ROUGE-L   | 0.2981     | 0.4013 |
| BERTScore | 0.8475     | 0.8705 |

**Important:**
Lower ROUGE/BERTScore reflects **stylistic differences**, not worse reasoning.
Human evaluation shows **higher logical fidelity and stability**.



---

## System Integration

ReCon is part of a full pipeline:

1. OWL reasoner generates contrastive explanation (JSON)
2. ReCon converts it into natural language
3. Graphviz generates a visual explanation
4. UI presents:

   * Explanation
   * Graph
   * Chat interface

Also integrated into:

* Flask web interface
* Protégé plugin



---

## Limitations

* Requires structured reasoning input
* Cannot perform standalone logical inference
* Sensitive to prompt format
* May hallucinate if constraints are relaxed

---

## Future Work

* Scaling to larger ontologies
* Support for more DL constructs

---

## Citation

This model is part of an academic project at Paderborn University.
If you use this model, please reference this repository.


---

## Contact

For questions, collaborations, or research discussions, feel free to reach out to Akash. Email: akbaum@mail.uni-paderborn.de