# ModernBERT Reward Model (CoT SQL/NL Alignment)

Finetuned `answerdotai/ModernBERT-base` to score how well a generated natural-language description (NL) and chain-of-thought reasoning align with a SQL query. The model is trained as a regression head (sigmoid output in `[0, 1]`) to predict `similarity_with_penalty` scores derived from human preference data plus corruption heuristics.


## Usage

```python
import torch
from transformers import AutoTokenizer
from modeling_reward import BERTRewardModel

model_name = "DarianNLP/modernbert-nl-sql"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = BERTRewardModel(model_name=model_name)
state_dict = torch.load("model.safetensors")  # or use safetensors.torch.load_file
model.load_state_dict(state_dict)
model.eval()

sql = "SELECT COUNT(*) FROM orders WHERE status = 'complete';"
reasoning = "think: Count rows in orders filtered by status 'complete'."
nl = "How many completed orders exist?"
text = f"SQL: {sql}\nReasoning: {reasoning}\nNL: {nl}"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=2048)
score = model(**inputs)["scores"].item()
print(f"Reward: {score:.3f}")
```

For convenience, `modeling_reward.py` exposes `load_finetuned_model(model_dir)` which handles loading `model.safetensors` or `pytorch_model.bin` and moves the module to GPU if available (falling back to CPU on OOM).


## Notes

- The reward target is bounded `[0, 1]` and already penalizes copied NL or incorrect reasoning.
- The model uses mean pooling instead of CLS to better leverage long ModernBERT contexts.
- Tokenizer files are saved from the finetuned run; no extra special tokens were introduced.