LogicPalette's picture
card: links, team, usage
8e29ed9 verified
|
Raw History Blame Contribute Delete
3.32 kB
metadata
base_model: CohereLabs/tiny-aya-global
library_name: peft
license: cc-by-nc-4.0
license_link: https://cohere.com/c4ai-cc-by-nc-license
tags:
  - text-to-sql
  - grpo
  - lora
  - agenttune
  - counterfactual-rewards
  - lexsi-hackathon
datasets:
  - sqlcf-lexsi/counterfactual-sql-data

tiny-aya-global-sqlcf-grpo-c

LoRA adapter (r=16) for CohereLabs/tiny-aya-global: GRPO, 60 steps, counterfactual-DB reward R_cf_mean (condition C).

Lexsi Labs AI Safety & Alignment Hackathon, Track 3 (PS #6 data and database agents + #9 reward design and reward-hacking defences), team Teen_khapeetars: Shreshth Rai (lead, @hi-phi-r25i), Rishii Kumar Singh (@LogicPalette), Aksh Aggarwal (@AkshAgg).

Collections: models, data 路 Demo 路 Data 路 Adapters: SFT, GRPO-B, GRPO-C 路 Code 路 Write-up

Status (gate G3): null: no gain over SFT (p = 0.21) and below GRPO-B; reported as a null.

Test set: 331 items x 8 samples, T=0.7, fresh databases from a different generator; "verified" = correct on the original and all 4 counterfactual DBs.

Metric Value (95% CI)
Verified accuracy 9.0% [7.9, 10.1]
Single-DB accuracy 14.0% [12.8, 15.4]
Silent-error rate 61.2% [59.3, 63.0]
Held-out family F4, verified 8.8% [7.5, 10.2]

Use

Accept the base model's licence first, then:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c")
tok = AutoTokenizer.from_pretrained("sqlcf-lexsi/tiny-aya-global-sqlcf-grpo-c")

It is a multi-turn SQL agent with one run_sql tool, prompted and parsed the way AgentTune renders Tiny Aya (see sqlcf/agent.py and evaluate.py gen in the repo). vLLM LoRA fails on Tiny Aya's tied embeddings, so for vLLM merge first: python train.py merge --init <this adapter> --out merged (fp32 merge, saved fp16).

Licence

This adapter is a derivative of CohereLabs/tiny-aya-global and is released under the same terms: