LogicPalette's picture
card: links, team, usage
8e29ed9 verified
|
Raw History Blame Contribute Delete
3.32 kB
---
base_model: CohereLabs/tiny-aya-global
library_name: peft
license: cc-by-nc-4.0
license_link: https://cohere.com/c4ai-cc-by-nc-license
tags: [text-to-sql, grpo, lora, agenttune, counterfactual-rewards, lexsi-hackathon]
datasets: [sqlcf-lexsi/counterfactual-sql-data]
---
# tiny-aya-global-sqlcf-grpo-c
LoRA adapter (r=16) for [CohereLabs/tiny-aya-global](https://huggingface.co/CohereLabs/tiny-aya-global): GRPO, 60 steps, counterfactual-DB reward `R_cf_mean` (condition C).
**Lexsi Labs AI Safety & Alignment Hackathon, Track 3 (PS #6 data and database agents + #9 reward design and
reward-hacking defences), team Teen_khapeetars:** Shreshth Rai (lead, [@hi-phi-r25i](https://huggingface.co/hi-phi-r25i)),
Rishii Kumar Singh ([@LogicPalette](https://huggingface.co/LogicPalette)), Aksh Aggarwal ([@AkshAgg](https://huggingface.co/AkshAgg)).
Collections: [models](https://huggingface.co/collections/sqlcf-lexsi/counterfactual-db-rewards-for-a-small-sql-agent-6af174f77dbae0bf0a54c3c4), [data](https://huggingface.co/collections/sqlcf-lexsi/counterfactual-sql-data-rollouts-and-traces-6af479430243b7731f85d144) 路 [Demo](https://huggingface.co/spaces/sqlcf-lexsi/counterfactual-sql-demo) 路
[Data](https://huggingface.co/datasets/sqlcf-lexsi/counterfactual-sql-data) 路
Adapters: [SFT](https://huggingface.co/sqlcf-lexsi/tiny-aya-global-sqlcf-sft),
[GRPO-B](https://huggingface.co/sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-b),
[GRPO-C](https://huggingface.co/sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c) 路
[Code](https://github.com/RishiiGamer2201/sexsi_hack) 路 [Write-up](https://github.com/RishiiGamer2201/sexsi_hack/blob/main/ARTICLE.md)
**Status (gate G3): null: no gain over SFT (p = 0.21) and below GRPO-B; reported as a null.**
Test set: 331 items x 8 samples, T=0.7, fresh databases from a different generator; "verified" = correct on the
original and all 4 counterfactual DBs.
| Metric | Value (95% CI) |
|---|---|
| Verified accuracy | 9.0% [7.9, 10.1] |
| Single-DB accuracy | 14.0% [12.8, 15.4] |
| Silent-error rate | 61.2% [59.3, 63.0] |
| Held-out family F4, verified | 8.8% [7.5, 10.2] |
## Use
Accept the [base model's licence](https://huggingface.co/CohereLabs/tiny-aya-global) first, then:
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c")
tok = AutoTokenizer.from_pretrained("sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c")
```
It is a multi-turn SQL agent with one `run_sql` tool, prompted and parsed the way AgentTune renders Tiny Aya
(see `sqlcf/agent.py` and `evaluate.py gen` in the repo). vLLM LoRA fails on Tiny Aya's tied embeddings, so for vLLM
merge first: `python train.py merge --init <this adapter> --out merged` (fp32 merge, saved fp16).
## Licence
This adapter is a derivative of CohereLabs/tiny-aya-global and is released under the same terms:
- CC-BY-NC-4.0 (full text in LICENSE) with Cohere's acceptable-use addendum: https://cohere.com/c4ai-cc-by-nc-license
- Cohere Labs Acceptable Use Policy: https://docs.cohere.com/docs/c4ai-acceptable-use-policy
Non-commercial use only. You also need access to the gated base model to use it.