README.md

---
language: en
license: apache-2.0
library_name: transformers
tags:
- commonsense-reasoning
- winoGrande
- fine-tuned
- llama
- reasoning
datasets:
- allenai/winogrande
metrics:
- accuracy
- loss
base_model:
- PleIAs/Monad
---

## Model Details

### Model Description

The model has been trained on the WinoGrande dataset which tests the ability to resolve pronouns
and make logical inferences in everyday scenarios.


### Model Sources

- **Base Model:** https://huggingface.co/PleIAs/Monad

### Training Data

Dataset: WinoGrande (allenai/winogrande)
- Size: 9,248 training examples, 1,267 validation examples
- Task: Commonsense reasoning with pronoun resolution
- Format: Multiple choice questions requiring logical reasoning

## Training Hyperparameters
| Epochs | Batch Size | Learning Rate | Warmup Ratio | Warmup Steps | Weight Decay | Max Gradient Norm | Evaluation Steps | Save Steps | Early Stopping Patience |
|--------|------------|---------------|--------------|--------------|--------------|-------------------|------------------|------------|-------------------------|
| 5      | 16         | 1e-05         | 0.05         | 144          | 0.01         | 1.0               | 150              | 150        | 7                       |

## Training Results
| Metric | Value |
|--------|-------|
| Final Training Loss | 0.9143 |
| Training Time | 1,526.9s |

## Validation Performance
Validation loss stabilized between 0.83-0.86 throughout the training

#### Summary

The model achieved strong convergence during training:
- **Final training loss:** 0.9143
- **Evaluation loss:** ~0.834 (final checkpoint)
- **Training completed:** All 5 epochs with early stopping monitoring

### Compute Infrastructure

#### Hardware

- **GPU:** Single NVIDIA A10G (24GB VRAM)
- **Platform:** Modal.com