LogicPalette's picture
card: links, team, usage
1b8f27f verified
|
Raw History Blame Contribute Delete
3.3 kB
---
base_model: CohereLabs/tiny-aya-global
library_name: peft
license: cc-by-nc-4.0
license_link: https://cohere.com/c4ai-cc-by-nc-license
tags: [text-to-sql, grpo, lora, agenttune, counterfactual-rewards, lexsi-hackathon]
datasets: [sqlcf-lexsi/counterfactual-sql-data]
---
# tiny-aya-global-sqlcf-sft
LoRA adapter (r=16) for [CohereLabs/tiny-aya-global](https://huggingface.co/CohereLabs/tiny-aya-global): SFT warm start on scripted oracle trajectories (3 train templates held out for GRPO); the init of GRPO-B and GRPO-C.
**Lexsi Labs AI Safety & Alignment Hackathon, Track 3 (PS #6 data and database agents + #9 reward design and
reward-hacking defences), team Teen_khapeetars:** Shreshth Rai (lead, [@hi-phi-r25i](https://huggingface.co/hi-phi-r25i)),
Rishii Kumar Singh ([@LogicPalette](https://huggingface.co/LogicPalette)), Aksh Aggarwal ([@AkshAgg](https://huggingface.co/AkshAgg)).
Collections: [models](https://huggingface.co/collections/sqlcf-lexsi/counterfactual-db-rewards-for-a-small-sql-agent-6af174f77dbae0bf0a54c3c4), [data](https://huggingface.co/collections/sqlcf-lexsi/counterfactual-sql-data-rollouts-and-traces-6af479430243b7731f85d144) 路 [Demo](https://huggingface.co/spaces/sqlcf-lexsi/counterfactual-sql-demo) 路
[Data](https://huggingface.co/datasets/sqlcf-lexsi/counterfactual-sql-data) 路
Adapters: [SFT](https://huggingface.co/sqlcf-lexsi/tiny-aya-global-sqlcf-sft),
[GRPO-B](https://huggingface.co/sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-b),
[GRPO-C](https://huggingface.co/sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c) 路
[Code](https://github.com/RishiiGamer2201/sexsi_hack) 路 [Write-up](https://github.com/RishiiGamer2201/sexsi_hack/blob/main/ARTICLE.md)
**Status (gate G3): baseline.**
Test set: 331 items x 8 samples, T=0.7, fresh databases from a different generator; "verified" = correct on the
original and all 4 counterfactual DBs.
| Metric | Value (95% CI) |
|---|---|
| Verified accuracy | 9.4% [8.4, 10.6] |
| Single-DB accuracy | 14.5% [13.2, 15.9] |
| Silent-error rate | 65.4% [63.6, 67.2] |
| Held-out family F4, verified | 9.2% [7.9, 10.7] |
## Use
Accept the [base model's licence](https://huggingface.co/CohereLabs/tiny-aya-global) first, then:
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "sqlcf-lexsi/tiny-aya-global-sqlcf-sft")
tok = AutoTokenizer.from_pretrained("sqlcf-lexsi/tiny-aya-global-sqlcf-sft")
```
It is a multi-turn SQL agent with one `run_sql` tool, prompted and parsed the way AgentTune renders Tiny Aya
(see `sqlcf/agent.py` and `evaluate.py gen` in the repo). vLLM LoRA fails on Tiny Aya's tied embeddings, so for vLLM
merge first: `python train.py merge --init <this adapter> --out merged` (fp32 merge, saved fp16).
## Licence
This adapter is a derivative of CohereLabs/tiny-aya-global and is released under the same terms:
- CC-BY-NC-4.0 (full text in LICENSE) with Cohere's acceptable-use addendum: https://cohere.com/c4ai-cc-by-nc-license
- Cohere Labs Acceptable Use Policy: https://docs.cohere.com/docs/c4ai-acceptable-use-policy
Non-commercial use only. You also need access to the gated base model to use it.