README.md

---
license: cc-by-4.0
language:
  - en
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
pipeline_tag: text-generation
library_name: peft
tags:
  - agriculture
  - peft
  - lora
  - instruction-tuning
  - adaption-labs
  - autoscientist
  - question-answering
datasets:
  - Charley890/AgroAdapt-Dataset
---

#  AgroAdapt-Mixtral-8x7B

> A PEFT (LoRA) fine-tuned Mixtral-8x7B-Instruct-v0.1 model for agricultural question answering, developed using the **Adaption Labs AutoScientist** workflow.

---![](Adaption_image.png)

#  Overview

AgroAdapt-Mixtral-8x7B is a domain-adapted language model designed to improve agricultural instruction following and question answering. The model was fine-tuned using the Adaption Labs AutoScientist workflow on a curated agricultural instruction dataset covering practical farming knowledge.

##  Quick Facts

| Item | Value |
|------|-------|
| **Domain** | Agriculture |
| **Task** | Agricultural Question Answering |
| **Base Model** | Mixtral-8x7B-Instruct-v0.1 |
| **Fine-Tuning Method** | PEFT (LoRA) |
| **Training Framework** | Adaption Labs AutoScientist |
| **Language** | English |
| **Dataset** | https://huggingface.co/datasets/Charley890/adaption-agricultural-qa-pairs |
| **License** | CC BY 4.0 |
---

#  Key Features

-  Agricultural Question Answering
-  Crop Production Guidance
-  Soil Science
-  Pest & Disease Identification
-  Irrigation Recommendations
-  Livestock Management
-  Climate-Smart Agriculture
-  Sustainable Farming
-  Agricultural Education

---

# Base Model

- **Base Model:** Mixtral-8x7B-Instruct-v0.1
- **Fine-tuning Method:** PEFT (LoRA)
- **Training Framework:** Adaption Labs AutoScientist
- **Language:** English

---

#  Training Dataset

The model was trained using an adapted agricultural instruction dataset containing high-quality question-answer pairs covering:

- Crop Production
- Soil Science
- Fertilizer Recommendations
- Pest Management
- Plant Disease Diagnosis
- Irrigation
- Livestock
- Poultry
- Sustainable Agriculture
- Climate-Smart Agriculture
- Farm Business
- Agricultural Extension

---

## Quick Start

The following example demonstrates how to load the LoRA adapter and perform inference using the Hugging Face Transformers ecosystem. The adapter is automatically merged with the base Mixtral model during inference.

### Installation

```bash
pip install torch transformers peft accelerate
```

### Load the Model

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "Mixtral-8x7B-Instruct-v0.1"
ADAPTER = "Charley890/AgroAdapt-Mixtral-8x7B"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    device_map="auto",
    torch_dtype="auto"
)

model = PeftModel.from_pretrained(model, ADAPTER)
```

### Inference

```python
prompt = """
Farmer:
My tomato leaves are turning yellow with brown spots.
What could be the cause and how can I treat it?
"""

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.7,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
Expected Output

The model returns a practical, step-by-step agricultural recommendation based on the farmer's question, including possible causes, preventive measures, and treatment options.
---

##  Training Configuration

| Parameter | Value |
|-----------|-------|
| Fine-tuning Method | LoRA |
| Base Model | Mixtral-8x7B-Instruct |
| Epochs | 5 |
| LoRA Rank (r) | 64 |
| LoRA Alpha | 128 |
| Warmup Ratio | 0.03 |
| Optimizer | Cosine |
| Gradient Clipping | 1.0 |
| Target Modules | q_proj, k_proj, v_proj, o_proj |

---

## LoRA Equation

The adapter follows the standard LoRA formulation:

\[
W' = W + \frac{\alpha}{r}BA
\]

Where:

- **r = 64**
- **α = 128**

Scaling factor:

\[
\frac{\alpha}{r} = \frac{128}{64} = 2
\]

This scaling improves learning efficiency while keeping the number of trainable parameters small.

---

##  Training Summary

- Fine-tuned using **Adaption Labs AutoScientist**
- **5 training epochs**
- Win rate improved from **24% → 76%**
- Agriculture benchmark improved from **26% → 75%**
- Training loss decreased consistently throughout optimization.
- Cosine learning-rate scheduling enabled smooth convergence.
- Gradient clipping stabilized training and reduced optimization spikes.

---

## Example

**Input**

> How can I prevent maize leaf blight?

**Output**

> Maize leaf blight can be reduced by planting resistant varieties, practicing crop rotation, avoiding overhead irrigation, removing infected crop residues, and applying recommended fungicides when disease pressure is high.


# Evaluation Results

> **Evaluation performed using Adaption Labs AutoScientist**

| Metric | Base Model | Adapted Model | Improvement |
|:-------|-----------:|--------------:|------------:|
|  Overall Win Rate | **24%** | **76%** | **+52 pts** |
|  Agriculture Win Rate | **26%** | **75%** | **+49 pts** |
|  Relative Improvement | — | — | **216.7%** |
|  Agriculture Improvement | — | — | **188.5%** |

---

## Performance Summary

```text
✓ Training Completed Successfully
✓ Stable Optimization
✓ Domain Adaptation Successful
✓ Agricultural Performance Improved
✓ Overall Win Rate Increased from 24% → 76%
✓ Agriculture Win Rate Increased from 26% → 75%
```

---

## Evaluation Figures

### Overall Training Win Rate

![Overall Training Win Rate](overall_training_win_rate.png)

---

### Agriculture Domain Win Rate

![Agriculture Domain Win Rate](agriculture_domain_win_rate.png)

---

### Evaluation Summary

![Evaluation Summary](evaluation_summary.png)

---
## Training Curves

### Training Loss

![Training Loss](graph_1_enhanced.png)

---

### Learning Rate

![Learning Rate](graph_2_enhanced.png)

---

### Gradient Norm

![Gradient Norm](graph_3_enhanced.png)


## Interpretation

The fine-tuned model demonstrates a substantial improvement over the baseline model after domain adaptation using **Adaption Labs AutoScientist**. The evaluation shows a significant increase in both overall and agriculture-specific performance while maintaining stable optimization throughout training.

These results indicate that the model effectively learned agricultural reasoning and instruction-following capabilities from the domain-specific dataset.

---
# Conclusion 

##  Training Dynamics

```text
Training Loss        ↘ steadily decreased
Validation Loss      → remained stable
Learning Rate        ↘ cosine decay schedule
Gradient Norm        → controlled through clipping
Adaptation Strategy  → LoRA parameter-efficient fine-tuning
```

### Adaptive Optimization

```python
while training:
    loss ↓
    learning_rate = cosine_decay(step)
    gradients = clip_norm(max_norm=1.0)
    weights = weights + LoRA_update()
```

### Model Equation

```text
Knowledge = Base Model + Domain Adaptation

M_adapted = M_base + LoRA(Agriculture)
```

### Final Result

```yaml
model: AgroAdapt-Mixtral-8x7B
framework: Adaption Labs AutoScientist
domain: Agriculture
training: Stable
convergence: Successful
adaptation: Optimized
status: Ready for Inference
```

> **AgroAdapt-Mixtral-8x7B** successfully adapts the Mixtral foundation model into a specialized agricultural assistant through efficient LoRA fine-tuning, stable optimization, and domain-specific instruction learning, enabling practical, context-aware support for modern farming applications.

# Intended Uses

This model is suitable for:

- AI Farming Assistants
- Agricultural Chatbots
- Farm Advisory Systems
- Agricultural Education
- Agricultural Research
- Smart Farming Applications
- Decision Support Systems

---

#  Limitations

Although the model performs well on agricultural instruction tasks, responses should be verified before being used for real-world farming decisions. Local farming practices and expert guidance should always take precedence.

---

# License

This model is released under the **CC BY 4.0** License.

---

#  Acknowledgements

This project was developed as part of the **Adaption Labs AutoScientist Challenge** using the Mixtral-8x7B-Instruct-v0.1 base model.

---

#  Citation

If you use this model in your research or applications, please cite:

```bibtex
@misc{charlie2026agroadapt,
  title={AgroAdapt-Mixtral-8x7B},
  author={Edidiong Charlie},
  year={2026},
  publisher={Hugging Face},
  note={Adaption Labs AutoScientist Challenge}
}
```