Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <meta name="description" content="SLM JSON Cleaner docs: Repair malformed, incomplete, or truncated JSON outputs locally on CPU. Strict schema adherence guides, environment variables, and API specs."> | |
| <meta name="keywords" content="SLM JSON Cleaner, JSON repair, schema enforcement, local AI, CPU inference, ONNX Runtime, Python, pip, offline parsing"> | |
| <meta name="author" content="SLM Agents Community"> | |
| <meta name="robots" content="index, follow"> | |
| <!-- Open Graph / Facebook --> | |
| <meta property="og:type" content="article"> | |
| <meta property="og:url" content="https://www.slmagents.ai/json_cleaner.html"> | |
| <meta property="og:title" content="SLM JSON Cleaner | Local CPU JSON Repair — SLM Agents"> | |
| <meta property="og:description" content="Repair malformed, incomplete, or truncated JSON outputs locally on CPU. Strict schema enforcement, fallback parsers, and quick start guides."> | |
| <meta property="og:image" content="https://www.slmagents.ai/assets/preview.png"> | |
| <meta property="og:site_name" content="SLM Agents"> | |
| <!-- Twitter Card --> | |
| <meta name="twitter:card" content="summary_large_image"> | |
| <meta name="twitter:url" content="https://www.slmagents.ai/json_cleaner.html"> | |
| <meta name="twitter:title" content="SLM JSON Cleaner | Local CPU JSON Repair — SLM Agents"> | |
| <meta name="twitter:description" content="Repair malformed, incomplete, or truncated JSON outputs locally on CPU. Strict schema enforcement, fallback parsers, and quick start guides."> | |
| <meta name="twitter:image" content="https://www.slmagents.ai/assets/preview.png"> | |
| <!-- Canonical --> | |
| <link rel="canonical" href="https://www.slmagents.ai/json_cleaner.html"> | |
| <title>SLM JSON Cleaner | Documentation</title> | |
| <link rel="stylesheet" href="style.css"> | |
| <!-- Structured Data --> | |
| <script type="application/ld+json"> | |
| { | |
| "@context": "https://schema.org", | |
| "@type": "TechArticle", | |
| "headline": "SLM JSON Cleaner — Local CPU JSON Repair Documentation", | |
| "description": "Complete documentation for SLM JSON Cleaner: a local CPU-powered JSON sanitizer. Covers malformed repair, schema alignment, and fallback brace parsers.", | |
| "url": "https://www.slmagents.ai/json_cleaner.html", | |
| "author": { "@type": "Organization", "name": "SLM Agents Community" }, | |
| "publisher": { "@type": "Organization", "name": "SLM Agents", "url": "https://www.slmagents.ai" }, | |
| "dateModified": "2026-08-11", | |
| "keywords": "json repair, schema matching, parser fallback, CPU inference, ONNX" | |
| } | |
| </script> | |
| </head> | |
| <body> | |
| <div class="bg-glow-1"></div> | |
| <div class="bg-glow-2"></div> | |
| <!-- NAVIGATION HEADER --> | |
| <header> | |
| <div class="container nav-container"> | |
| <div style="display: flex; align-items: center; gap: 10px;"> | |
| <button class="sidebar-toggle" onclick="toggleSidebar()"> | |
| <svg width="24" height="24" fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24"> | |
| <path stroke-linecap="round" stroke-linejoin="round" d="M4 6h16M4 12h16M4 18h16"/> | |
| </svg> | |
| </button> | |
| <a href="index.html" class="logo"> | |
| <div class="logo-icon"> | |
| <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round"> | |
| <path d="M12 2L2 7l10 5 10-5-10-5zM2 17l10 5 10-5M2 12l10 5 10-5"/> | |
| </svg> | |
| </div> | |
| <span>SLM Agents</span> | |
| </a> | |
| </div> | |
| <nav> | |
| <ul> | |
| <li><a href="index.html">Home</a></li> | |
| <li class="dropdown"> | |
| <a class="dropdown-trigger" style="color:var(--primary)"> | |
| Frameworks | |
| <svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"><path d="M6 9l6 6 6-6"/></svg> | |
| </a> | |
| <div class="dropdown-content"> | |
| <a href="orchestrator.html">Orchestrator Docs</a> | |
| <a href="rag.html">RAG Docs</a> | |
| <a href="summarizer.html">Summarizer Docs</a> | |
| <a href="sql.html">Text-to-SQL Docs</a> | |
| <a href="cli.html">CLI Agent Docs</a> | |
| <a href="code_interpreter.html">Code Interpreter Docs</a> | |
| <a href="git_repo_manager.html">Git Repo Manager Docs</a> | |
| <a href="json_cleaner.html">JSON Cleaner Docs</a> | |
| <a href="document_parser.html">Document Parser Docs</a> | |
| <a href="vision_parser.html">Vision Parser Docs</a> | |
| <a href="web_agent.html">Web Agent Docs</a> | |
| <a href="web_scraper.html">Web Scraper Docs</a> | |
| <a href="search_orchestrator.html">Search Orchestrator Docs</a> | |
| </div> | |
| </li> | |
| <li><a href="playground.html">Playground</a></li> | |
| <li><a href="index.html#upcoming">Roadmap</a></li> | |
| <li><a href="https://github.com/t00114218-stack/SLMAgents" class="btn-github-nav" target="_blank"> | |
| <svg width="16" height="16" fill="currentColor" viewBox="0 0 16 16"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27s1.36.09 2 .27c1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.01 8.01 0 0 0 16 8c0-4.42-3.58-8-8-8"/></svg> | |
| GitHub | |
| </a></li> | |
| <li><a href="https://huggingface.co/spaces/spcv/slm-agents" class="btn-hf-nav" target="_blank" style="display: flex; align-items: center; gap: 6px; padding: 7px 14px; border-radius: 8px; background: #fef3c7; border: 1px solid #f59e0b; color: #b45309; font-weight: 700; text-decoration: none; font-size: 0.85rem; transition: all 0.2s;">🤗 Hugging Face Space</a></li> | |
| </ul> | |
| </nav> | |
| </div> | |
| </header> | |
| <div class="layout-container"> | |
| <aside class="sidebar" id="sidebar"> | |
| <div class="sidebar-group"> | |
| <div class="sidebar-group-title">Active Libraries</div> | |
| <ul class="sidebar-list" id="sidebar-active-list"></ul> | |
| </div> | |
| </aside> | |
| <main class="main-content"> | |
| <div class="doc-content-wrapper"> | |
| <section class="doc-section" style="margin-top: 0;"> | |
| <div class="breadcrumb" style="font-size: 0.85rem; color: #475569; margin-bottom: 1rem; font-weight: 600;"><a href="index.html" style="color: #4f46e5; text-decoration: none;">Home</a> <span style="margin: 0 6px;">›</span> <span style="color: #0f172a;">SLM JSON Cleaner</span></div> | |
| <div class="badge-pill">🧹 Data Sanitization Agent</div> | |
| <h1 style="font-size: 2.25rem; color: #0f172a; font-weight: 800; margin-bottom: 0.75rem;">SLM JSON Cleaner</h1> | |
| <p style="font-size: 1.05rem; color: #334155; line-height: 1.65; font-weight: 500; margin: 0;">Lightweight local JSON text sanitizer and structural repair tool that formats malformed, truncated, or unquoted LLM outputs offline on standard CPU.</p> | |
| </section> | |
| <nav class="doc-nav"> | |
| <a href="#overview">Overview</a> | |
| <a href="#install"> | |
| <a href="#git">Git Checkout</a>Installation</a> | |
| <a href="#config">Configuration API</a> | |
| <a href="#performance">Performance</a> | |
| </nav> | |
| <section class="doc-section" id="install"> | |
| <h2>💻 Installation</h2> | |
| <div class="code-panel" style="max-width:100%"> | |
| <div class="code-header"><div class="code-dots"><div class="code-dot"></div><div class="code-dot"></div><div class="code-dot"></div></div><div class="code-title">Terminal</div></div> | |
| <div class="code-content" style="display:block;padding:1.5rem"> | |
| <pre><code><span class="comment"># Install in editable mode locally</span> | |
| pip install -e ./slm_json_cleaner | |
| <span class="comment"># Set performance threads</span> | |
| export SLM_JSON_CLEANER_N_THREADS=4</code></pre> | |
| </div> | |
| </div> | |
| </section> | |
| <!-- GIT CHECKOUT --> | |
| <section class="doc-section" id="git"> | |
| <h2>🐙 Checkout from GitHub</h2> | |
| <p>Clone only this agent's folder from the monorepo using Git sparse-checkout — no need to download the full repository:</p> | |
| <h3 style="font-size: 1.05rem; color: #0f172a; font-weight: 700; margin-top: 1.5rem; margin-bottom: 0.75rem;">Option 1 — Sparse Checkout (Recommended)</h3> | |
| <div class="code-panel" style="max-width:100%; background: #0f172a; border: 1px solid #1e293b; border-radius: 14px; overflow: hidden; margin: 1rem 0; box-shadow: 0 16px 40px rgba(15, 23, 42, 0.12);"> | |
| <div class="code-header" style="background: #1e293b; padding: 10px 16px; display: flex; align-items: center; justify-content: space-between; border-bottom: 1px solid #334155;"> | |
| <div class="code-dots"><div class="code-dot"></div><div class="code-dot"></div><div class="code-dot"></div></div> | |
| <div class="code-title" style="color: #94a3b8; font-weight: 700; font-size: 0.8rem; font-family: 'JetBrains Mono', monospace;">Terminal — Git Sparse Checkout</div> | |
| </div> | |
| <div class="code-content" style="display:block; padding: 1.25rem 1.5rem; background: #0f172a;"> | |
| <pre style="margin:0; background:#0f172a; color:#f8fafc; font-family:'JetBrains Mono',monospace; font-size:0.88rem; border:none; box-shadow:none; padding:0; line-height: 1.75;"><span style="color:#64748b;"># 1. Create and enter a new directory</span> | |
| <span style="color:#34d399;">$</span> <span style="color:#c084fc; font-weight:700;">mkdir</span> <span style="color:#38bdf8;">slm_json_cleaner</span> <span style="color:#94a3b8;">&&</span> <span style="color:#c084fc; font-weight:700;">cd</span> <span style="color:#38bdf8;">slm_json_cleaner</span> | |
| <span style="color:#64748b;"># 2. Initialise empty git repo and add remote</span> | |
| <span style="color:#34d399;">$</span> <span style="color:#c084fc; font-weight:700;">git init</span> | |
| <span style="color:#34d399;">$</span> <span style="color:#c084fc; font-weight:700;">git remote add origin</span> <span style="color:#38bdf8;">https://github.com/t00114218-stack/SLMAgents.git</span> | |
| <span style="color:#64748b;"># 3. Enable sparse-checkout and set target folder</span> | |
| <span style="color:#34d399;">$</span> <span style="color:#c084fc; font-weight:700;">git sparse-checkout init</span> <span style="color:#94a3b8;">--cone</span> | |
| <span style="color:#34d399;">$</span> <span style="color:#c084fc; font-weight:700;">git sparse-checkout set</span> <span style="color:#38bdf8;">slm_json_cleaner</span> | |
| <span style="color:#64748b;"># 4. Pull only that agent's source</span> | |
| <span style="color:#34d399;">$</span> <span style="color:#c084fc; font-weight:700;">git pull origin</span> <span style="color:#38bdf8;">main</span></pre> | |
| </div> | |
| </div> | |
| <h3 style="font-size: 1.05rem; color: #0f172a; font-weight: 700; margin-top: 2rem; margin-bottom: 0.75rem;">Option 2 — Full Repository Clone</h3> | |
| <div class="code-panel" style="max-width:100%; background: #0f172a; border: 1px solid #1e293b; border-radius: 14px; overflow: hidden; margin: 1rem 0;"> | |
| <div class="code-header" style="background: #1e293b; padding: 10px 16px; display: flex; align-items: center; justify-content: space-between; border-bottom: 1px solid #334155;"> | |
| <div class="code-dots"><div class="code-dot"></div><div class="code-dot"></div><div class="code-dot"></div></div> | |
| <div class="code-title" style="color: #94a3b8; font-weight: 700; font-size: 0.8rem; font-family: 'JetBrains Mono', monospace;">Terminal — Full Clone</div> | |
| </div> | |
| <div class="code-content" style="display:block; padding: 1.25rem 1.5rem; background: #0f172a;"> | |
| <pre style="margin:0; background:#0f172a; color:#f8fafc; font-family:'JetBrains Mono',monospace; font-size:0.88rem; border:none; box-shadow:none; padding:0; line-height: 1.75;"><span style="color:#34d399;">$</span> <span style="color:#c084fc; font-weight:700;">git clone</span> <span style="color:#38bdf8;">https://github.com/t00114218-stack/SLMAgents.git</span> | |
| <span style="color:#34d399;">$</span> <span style="color:#c084fc; font-weight:700;">cd</span> <span style="color:#38bdf8;">SLMAgents/slm_json_cleaner</span></pre> | |
| </div> | |
| </div> | |
| <p style="margin-top: 1.25rem; font-size: 0.9rem; color: #475569; background: #f8fafc; border: 1px solid #cbd5e1; border-radius: 10px; padding: 1rem 1.25rem;"> | |
| 💡 <strong>Tip:</strong> After checkout, install the package locally with <code style="background: #eef2ff; color: #4f46e5; border: 1px solid #c7d2fe; padding: 2px 8px; border-radius: 5px; font-weight: 700;">pip install -e ./slm_json_cleaner</code> to run in editable mode without publishing to PyPI. | |
| </p> | |
| </section> | |
| <section class="doc-section" id="config"> | |
| <h2>⚙️ Configuration API</h2> | |
| <h3>Constructor Parameters</h3> | |
| <table class="param-table"> | |
| <thead><tr><th>Parameter</th><th>Type / Default</th><th>Description</th></tr></thead> | |
| <tbody> | |
| <tr><td>model_path</td><td>str | None</td><td>Explicit path to the ONNX model directory. Defaults to caching/sharing the main model in the monorepo.</td></tr> | |
| <tr><td>cache_dir</td><td>str | None</td><td>HF model directory path. Also settable via <span class="env-tag">SLM_JSON_CLEANER_CACHE_DIR</span>.</td></tr> | |
| <tr><td>n_ctx</td><td>int | 2048</td><td>Context window size in tokens. Also settable via <span class="env-tag">SLM_JSON_CLEANER_N_CTX</span>.</td></tr> | |
| <tr><td>n_threads</td><td>int | 4</td><td>CPU threads for ONNX Runtime. Also settable via <span class="env-tag">SLM_JSON_CLEANER_N_THREADS</span>.</td></tr> | |
| <tr><td>system_prompt</td><td>str | None</td><td>Optional custom system prompt instructions overriding the default template.</td></tr> | |
| <tr><td>user_input</td><td>str | None</td><td>Optional additional user-supplied target parameters or variables.</td></tr> | |
| </tbody> | |
| </table> | |
| <h3>clean_json() Parameters</h3> | |
| <table class="param-table"> | |
| <thead><tr><th>Parameter</th><th>Type / Default</th><th>Description</th></tr></thead> | |
| <tbody> | |
| <tr><td>malformed_text</td><td>str</td><td><strong>Required.</strong> Unstructured or broken JSON input string to repair.</td></tr> | |
| <tr><td>schema_dict</td><td>dict</td><td><strong>Required.</strong> A dictionary mapping keys and types required in the final output (e.g. {"name": "string", "age": "number"}).</td></tr> | |
| <tr><td>system_prompt</td><td>str | None</td><td>Optional custom system prompt instructions overriding the default template.</td></tr> | |
| <tr><td>user_input</td><td>str | None</td><td>Optional additional user-supplied target parameters or variables.</td></tr> | |
| </tbody> | |
| </table> | |
| <div class="code-panel" style="max-width:100%"> | |
| <div class="code-header"><div class="code-dots"><div class="code-dot"></div><div class="code-dot"></div><div class="code-dot"></div></div><div class="code-title">Python Quick Start</div></div> | |
| <div class="code-content" style="display:block;padding:1.5rem"> | |
| <pre><code><span class="keyword">from</span> slm_json_cleaner.json_cleaner <span class="keyword">import</span> SLMJSONCleaner | |
| cleaner = <span class="function">SLMJSONCleaner</span>() | |
| <span class="comment"># Truncated, nested, and conversational malformed input log:</span> | |
| broken_input = <span class="string">""" | |
| Raw logs: | |
| { | |
| "project_name": "Antigravity Pipeline", | |
| "build_status": "success", | |
| "metrics": { | |
| "duration_seconds": 124.5, | |
| "test_count": 48, | |
| "failed_tests": 0, | |
| "coverage": "98.5% | |
| }, | |
| "contributors": [ | |
| {"name": "Alice", "role": "lead"}, | |
| {"name": "Bob", "role": "reviewer" | |
| ], | |
| "releases": [ | |
| "v1.0", "v1.1", | |
| [LOG EXPIRED MID-TOKEN] | |
| """</span> | |
| schema = { | |
| <span class="string">"project_name"</span>: <span class="string">"string"</span>, | |
| <span class="string">"build_status"</span>: <span class="string">"string"</span>, | |
| <span class="string">"metrics"</span>: { | |
| <span class="string">"duration_seconds"</span>: <span class="string">"number"</span>, | |
| <span class="string">"test_count"</span>: <span class="string">"number"</span>, | |
| <span class="string">"failed_tests"</span>: <span class="string">"number"</span>, | |
| <span class="string">"coverage"</span>: <span class="string">"string"</span> | |
| }, | |
| <span class="string">"contributors"</span>: [ | |
| {<span class="string">"name"</span>: <span class="string">"string"</span>, <span class="string">"role"</span>: <span class="string">"string"</span>} | |
| ], | |
| <span class="string">"releases"</span>: [<span class="string">"string"</span>] | |
| } | |
| parsed, success = cleaner.<span class="function">clean_json</span>(broken_input, schema) | |
| <span class="function">print</span>(f<span class="string">"Success: {success}"</span>) | |
| <span class="function">print</span>(parsed)</code></pre> | |
| </div> | |
| </div> | |
| </section> | |
| <section class="doc-section" id="repair"> | |
| <h2>JSON Recovery & Repair Capabilities</h2> | |
| <p>The sanitization model operates as a high-speed pattern matching engine. It cleans common structural errors caused by early tokens cutoff or memory limits:</p> | |
| <ul> | |
| <li><strong>Auto-closure:</strong> Repairs unclosed brackets <code>}</code> or list braces <code>]</code>.</li> | |
| <li><strong>Quote repair:</strong> Fixes missing double-quotes on JSON keys and string values.</li> | |
| <li><strong>Regex brace fallback:</strong> Includes a backup regex matcher to extract matching curly braces if standard JSON parsing triggers an exception.</li> | |
| </ul> | |
| </section> | |
| <section class="doc-section" id="schema"> | |
| <h2>Target Schema Enforcement</h2> | |
| <p>By supplying a target schema dictionary, you guarantee that the returned attributes match the expected data types. The repaired response returns exactly structured JSON data:</p> | |
| <pre><code class="language-json">{ | |
| "project_name": "Antigravity Pipeline", | |
| "build_status": "success", | |
| "metrics": { | |
| "duration_seconds": 124.5, | |
| "test_count": 48, | |
| "failed_tests": 0, | |
| "coverage": "98.5%" | |
| }, | |
| "contributors": [ | |
| { "name": "Alice", "role": "lead" }, | |
| { "name": "Bob", "role": "reviewer" } | |
| ], | |
| "releases": [ "v1.0", "v1.1" ] | |
| }</code></pre> | |
| </section> | |
| <footer style="margin-top: 3rem; text-align: center; border-top: 1px solid #cbd5e1; padding-top: 2rem; color: #475569; font-size: 0.9rem; font-weight: 500;"> | |
| <p>© 2026 SLM Agents. Built with Apache 2.0 Permissive Open Source License.</p> | |
| </footer> | |
| </div> | |
| </main> | |
| </div> | |
| <script src="app.js"></script> | |
| </body> | |
| </html> | |