convaiinnovations

/

laya

Like 5.89k

Hi! I built laya-guard, a local guardrail for AI coding agents (Claude Code, Codex, Cursor, Gemini CLI and 7 more) that judges each shell command before it runs, with Laya's multilingual checkpoint as the judge behind a layer of regex policies and a safe-command lane. Sharing it here in case it's useful, along with a few field notes that might interest you.

Try it: https://huggingface.co/spaces/Ayaannnnnnn/laya-guard-trial (paste a command, see allow / review / block and what decided it). It runs on CPU: about 0.1–0.3 s per judgment.

How Laya is used: one choice question (allow / review / block), a score and a noul for maliciousness, on the command text. Risk = p(block) + 0.5·p(review), raised to the noul value; ≥0.8 blocks, ≥0.45 reviews. Rules decide what they can without the model; Laya judges the rest.

What worked: on replays of 7 public agent incidents (Claude Code wiping a home directory, the Nx s1ngularity prompts, the Amazon Q wiper, PocketOS's Railway delete…) the pipeline holds 13/13. On 30 shell rewrites of the same harms it holds 30/30, including forms no pattern catches, like r''m -rf ~/, a perl rmtree or find ~ -delete, which Laya reviewed on its own. Korean requests are judged well (30/30 on our small set).

Where the model needed help (field notes):

• On a day of real Claude Code commands, Laya alone reviewed or blocked many ordinary English shell commands: sed -n 1,80p f at 0.99, echo $f at 0.75, a bare done at 0.66, git fetch at 0.65. We fixed most of it on our side (a safe lane for read-only commands, splitting scripts per line, reading shell variables), which took reviews from 53% to 45% on the same 1,840 commands. The rest is code the agent runs and outward actions, which should be reviewed anyway, plus English shell the model scores high.
• We tried giving the model context, prefixing the input with "a shell command a coding agent is about to run in the user's project". It cut false reviews (58 of 200 cleared) but lost recall (9 instead of 11 incident commands held, and cd ~/.ssh && cat id_rsa got through), so we dropped it.

A question, if you have a moment: is there a better way to phrase the questions/criteria for short English shell commands, or would the typed-decisions checkpoint or calibration (load_calibration) be the right tool here? The criteria are currently Korean (읽기 전용 조회, 안전한 일상 작업 / 결제, 삭제, 외부전송, 권한변경 / 파괴, 유출, 인젝션, 시크릿노출, 인증우회).

Thanks for releasing Laya under Apache-2.0. It made an on-device judge possible.