| --- |
| license: apache-2.0 |
| language: |
| - en |
| - es |
| library_name: laya |
| base_model: |
| - convaiinnovations/laya |
| pipeline_tag: text-classification |
| tags: |
| - CRUD |
| - System1 |
| --- |
| |
| # ChopGuard CLI CRUD Classifier |
|
|
| Fine-tuned laya-Agent model for classifying CLI commands into CRUD (Create/Read/Update/Delete) operations. |
|
|
| ## 🎯 Purpose: RBAC for LLM Agents |
|
|
| This model was created to provide **RBAC-like protection** for LLM agents executing CLI commands. In high-context situations, LLMs often forget about specialized tools and resort to CLI commands, which can be risky. This model helps detect and gate potentially dangerous operations. |
|
|
| Use cases: |
| - Harness (Claude Mods) |
| - Agent Loops (Create cli Rules on top of CRUD) |
| - Cli GuardRails |
| - Friendly User Interface (LLM 1 liners can get too long for manual inspection) |
|
|
| ## 🚀 Why CRUD Classification? |
|
|
| Running full inference on every command to understand user intent would be too slow. Instead, we use a **CRUD classification approach**: |
|
|
| - **READ** - inspects, lists, queries, or views existing state (safe for automation) |
| - **CREATE** - creates new resources (generally safe) |
| - **UPDATE** - modifies, transforms, or reconfigures existing resources (needs review) |
| - **DELETE** - removes, destroys, or irreversibly discards resources (always review) |
|
|
| This allows users to quickly identify which 1-liner commands need careful inspection before execution. |
|
|
| ## Model Details |
|
|
| - **Base:** ModernBert (jhu-clsp/mmBERT-base) |
| - **Framework:** laya |
| - **Task:** CLI command classification into CRUD operations |
|
|
| ## 📊 Model Performance |
|
|
| Evaluated on the held-out test set. Overall accuracy is **~83.7%**, with the model showing particularly strong performance on **delete** (99.5%) and **read** (98.9%). The few errors are concentrated in the create ↔ update directions, which is expected given the semantic overlap between "creating" and "modifying" resources. |
|
|
| Its trained on the most common agent looping commands and extended to other cli families. (Any cli family requests feel free to reach out, currently improving the dataset.) |
|
|
|  |
|
|
| ## Citations |
|
|
| If you use this model, please cite: |
|
|
| ```bibtex |
| @misc{chopguard2026crud, |
| title={ChopGuard CLI CRUD Classifier}, |
| author={Daniel Flores}, |
| year={2026}, |
| howpublished={\url{https://huggingface.co/ChopGuard/ChopGuard-CRUD}}, |
| note={CRUD guardrail for LLM CLI execution} |
| } |
| @software{laya2024, |
| author = {Nandha Kishor}, |
| title = {{Laya}: Fast, non-autoregressive System 1 decision engine}, |
| year = {2024}, |
| url = {https://huggingface.co/convaiinnovations/laya} |
| } |
| ``` |
|
|
| ## Usage |
|
|
| ```python |
| from laya import Agent |
| agent = Agent("ChopGuard_CRUD_20261018.safetensors", device="cuda") |
| crud_question = { |
| 'type': 'choice', |
| 'instructions': 'What CRUD operation does this CLI command perform?', |
| 'criteria': { |
| 'create': 'creates a new resource', |
| 'read': 'inspects, lists, queries, or views existing state', |
| 'update': 'modifies, transforms, or reconfigures', |
| 'delete': 'removes, destroys, or discards' |
| } |
| } |
| result = agent.predict({"command": "git log"}, {"crud": crud_question}) |
| label = result["answers"]["crud"]["choice"] |
| ``` |
|
|
| ## Example: |
| [](https://www.youtube.com/watch?v=kTXpCUOJoVs) |
|
|
| ## License |
|
|
| Apache 2.0 |
|
|
| ### Special Thanks to my coworkers: |
| I wouldnt be able to accomplish all this without you: |
