Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -19,20 +19,21 @@ tags:
|
|
| 19 |
pipeline_tag: text-classification
|
| 20 |
library_name: pico-type
|
| 21 |
---
|
| 22 |
-
|
| 23 |
<div align="center">
|
| 24 |
|
| 25 |
# pico-type 🔍
|
| 26 |
|
| 27 |
-
**A tiny byte-level multi-head content classifier** — ~1.5M params, ~
|
| 28 |
|
| 29 |
-
Classifies any content
|
| 30 |
|
| 31 |
[](LICENSE)
|
| 32 |
[]()
|
| 33 |
-
[](https://huggingface.co/spaces/eulogik/pico-type)
|
| 35 |
-
[ |
|
| 113 |
-
| HuggingFace Model
|
| 114 |
-
|
|
| 115 |
-
|
|
| 116 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
|
| 118 |
## 📄 License
|
| 119 |
|
| 120 |
Apache 2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
pipeline_tag: text-classification
|
| 20 |
library_name: pico-type
|
| 21 |
---
|
|
|
|
| 22 |
<div align="center">
|
| 23 |
|
| 24 |
# pico-type 🔍
|
| 25 |
|
| 26 |
+
**A tiny byte-level multi-head content classifier** — ~1.5M params, ~9MB single-file ONNX (FP32), ~18ms CPU inference.
|
| 27 |
|
| 28 |
+
Classifies any content from raw bytes: **coarse type · modality · subtype · code language · text language · file MIME · risk flags**
|
| 29 |
|
| 30 |
[](LICENSE)
|
| 31 |
[]()
|
| 32 |
+
[](https://pypi.org/project/pico-type/)
|
| 33 |
+
[]()
|
| 34 |
+
[](https://github.com/eulogik/pico-type/actions/workflows/ci.yml)
|
| 35 |
[](https://huggingface.co/spaces/eulogik/pico-type)
|
| 36 |
+
[](https://huggingface.co/eulogik/pico-type)
|
| 37 |
|
| 38 |
</div>
|
| 39 |
|
|
|
|
| 41 |
|
| 42 |
## ✨ Features
|
| 43 |
|
| 44 |
+
- **No tokenizer** — operates directly on raw UTF-8 bytes (supports all languages, no preprocessing)
|
| 45 |
+
- **7 heads, one forward pass** — coarse type, modality, subtype, code language, text language, file MIME, risk flags
|
| 46 |
+
- **4 Matryoshka tiers** — tiny (16d) → small (64d) → base (192d) → pro (576d) — same trunk, accuracy scales with dim
|
| 47 |
+
- **~9MB single-file ONNX (FP32)** — deploy on edge devices, serverless, browser (WebAssembly/ONNX Runtime Web)
|
| 48 |
+
- **~18ms inference** on CPU via ONNX Runtime
|
| 49 |
+
- **CLI, Python API, Gradio Space, MCP server** — ready to use
|
| 50 |
+
|
| 51 |
+
## 📊 Evaluation
|
| 52 |
+
|
| 53 |
+
### Overall Accuracy (v2 — trained on real data)
|
| 54 |
|
| 55 |
+
| Head | Classes | Accuracy | Dataset |
|
| 56 |
+
|------|---------|----------|---------|
|
| 57 |
+
| coarse | 12 | **100%** | Synthetic eval |
|
| 58 |
+
| modality | 8 | **100%** | Synthetic eval |
|
| 59 |
+
| subtype | 24 | **93.8%** | Synthetic eval |
|
| 60 |
+
| **code_lang** | **62** | **60.3%** | [The Heap](https://huggingface.co/datasets/AISE-TUDelft/the-heap) — 24 real-world langs, 1,200 samples |
|
| 61 |
+
| **text_lang** | **30** | **98.3%** | [Wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) — 30 langs, 1,500 samples |
|
| 62 |
+
| file_mime | 90 | **100%** | Synthetic eval |
|
| 63 |
+
| risk (mAP) | 6 | **100%** | Synthetic eval |
|
| 64 |
|
| 65 |
+
*v0.1 baseline (synthetic-only): code_lang 3%, text_lang 19%. Real-data training in v2 improves code by **57pp** and text by **79pp**.*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
|
| 67 |
+
### Code Language — Per-Language Accuracy
|
| 68 |
|
| 69 |
+
| Excellent (90%+) | Good (70–89%) | Needs Work (<50%) |
|
| 70 |
+
|---|---|---|
|
| 71 |
+
| cpp 96%, dart 98%, erlang 98%, rust 98%, r 94%, swift 92%, python 88%, lua 88% | go 86%, ruby 86%, ocaml 84%, php 78%, csharp 76%, java 76%, kotlin 76%, c 62% | perl 50%, haskell 24%, scala 4%, javascript 2%, clojure 0%, elixir 0%, julia 0%, sql 0% |
|
| 72 |
+
|
| 73 |
+
> **Note**: Low-accuracy languages have fewer real training samples. More data will improve them.
|
| 74 |
|
| 75 |
## 🚀 Quick Start
|
| 76 |
|
| 77 |
+
### Install
|
| 78 |
+
|
| 79 |
+
```bash
|
| 80 |
+
pip install picotype
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
### CLI
|
| 84 |
+
|
| 85 |
+
```bash
|
| 86 |
+
# Classify from stdin
|
| 87 |
+
echo "def hello(name):\n return f'Hi {name}'" | picotype --pretty
|
| 88 |
+
|
| 89 |
+
# Classify a file
|
| 90 |
+
picotype --file document.txt
|
| 91 |
+
|
| 92 |
+
# Classify clipboard content
|
| 93 |
+
picotype --clip
|
| 94 |
+
|
| 95 |
+
# All 4 tiers available
|
| 96 |
+
echo "..." | picotype --tier pro
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
### Python API
|
| 100 |
+
|
| 101 |
+
```python
|
| 102 |
+
from picotype import load_onnx_model, run_onnx
|
| 103 |
+
|
| 104 |
+
session = load_onnx_model("base")
|
| 105 |
+
result = run_onnx(session, "def hello(): pass")
|
| 106 |
+
print(result)
|
| 107 |
+
# {
|
| 108 |
+
# "coarse": "code",
|
| 109 |
+
# "code_language": "python",
|
| 110 |
+
# "modality": "textual",
|
| 111 |
+
# "confidence": 0.98,
|
| 112 |
+
# ...
|
| 113 |
+
# }
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
### MCP Server (for Claude Desktop, Cursor, etc.)
|
| 117 |
+
|
| 118 |
+
```bash
|
| 119 |
+
pip install picotype
|
| 120 |
+
PICOTYPE_MODEL_DIR=./checkpoints python -m model.pico_type.mcp_server
|
| 121 |
+
```
|
| 122 |
+
|
| 123 |
+
Then add to your MCP config:
|
| 124 |
+
```json
|
| 125 |
+
{
|
| 126 |
+
"mcpServers": {
|
| 127 |
+
"pico-type": {
|
| 128 |
+
"command": "python",
|
| 129 |
+
"args": ["-m", "model.pico_type.mcp_server"],
|
| 130 |
+
"env": { "PICOTYPE_MODEL_DIR": "./checkpoints" }
|
| 131 |
+
}
|
| 132 |
+
}
|
| 133 |
+
}
|
| 134 |
+
```
|
| 135 |
+
|
| 136 |
+
### Gradio Web UI
|
| 137 |
+
|
| 138 |
+
Try it live: [huggingface.co/spaces/eulogik/pico-type](https://huggingface.co/spaces/eulogik/pico-type)
|
| 139 |
+
|
| 140 |
## 🏗 Architecture
|
| 141 |
|
| 142 |
+
```
|
| 143 |
+
Bytes ─▶ ByteEmbed(256→96d) ─▶ 3×Conv1D(k=3,5,7) ─▶ 2×BiAttention(RoPE) ─▶ Pool ─▶ 7×Matryoshka Heads
|
| 144 |
+
```
|
| 145 |
+
|
| 146 |
+
| Component | Detail |
|
| 147 |
+
|-----------|--------|
|
| 148 |
+
| **ByteEmbed** | Lookup-free embedding — each byte value (0–255) maps to a learned 96-dim vector |
|
| 149 |
+
| **Conv1D** | 3 parallel depthwise convolutions (kernel widths 3, 5, 7) with residual + layer norm |
|
| 150 |
+
| **BiAttention** | Bidirectional self-attention with Rotary Position Embeddings (RoPE), 4 heads |
|
| 151 |
+
| **Pool** | Mean + max + std deviation concatenation → fixed-size representation |
|
| 152 |
+
| **Heads** | Matryoshka-style: slice pool dim to 16/64/192/576, project to 7 linear classifiers |
|
| 153 |
|
| 154 |
Total parameters: **1.43M** (tiny) / **1.45M** (small) / **1.48M** (base) / **1.56M** (pro)
|
| 155 |
|
| 156 |
## 🔧 Model Tiers
|
| 157 |
|
| 158 |
+
| Tier | Dim | Params | ONNX Size | Accuracy Multiplier |
|
| 159 |
+
|------|-----|--------|-----------|-------------------|
|
| 160 |
+
| tiny | 16 | 1.43M | 9.09 MB | 0.65× |
|
| 161 |
+
| small | 64 | 1.45M | 9.13 MB | 0.82× |
|
| 162 |
+
| base | 192 | 1.48M | 9.25 MB | 1.0× (reference) |
|
| 163 |
+
| pro | 576 | 1.56M | 9.61 MB | 1.05× |
|
| 164 |
|
| 165 |
+
ONNX sizes are single-file FP32 exports (graph-only files are 203–206 KB).
|
| 166 |
+
|
| 167 |
+
All tiers share the same backbone; only the final linear projection layers differ. Higher-tier models use more dimensions for finer-grained classification.
|
| 168 |
|
| 169 |
## 🧪 Classification Heads
|
| 170 |
|
| 171 |
+
| Head | Classes | What It Detects |
|
| 172 |
+
|------|---------|-----------------|
|
| 173 |
| **coarse** | 12 | text, code, link, image, file, config, markup, data, error, secret, archive, binary |
|
| 174 |
+
| **modality** | 8 | textual, binary_image, binary_archive, binary_executable, binary_document, etc. |
|
| 175 |
+
| **subtype** | 24 | json, yaml, toml, csv, html, markdown, sql, log, dockerfile, makefile, etc. |
|
| 176 |
+
| **code_lang** | **62** | python, javascript, typescript, java, c, cpp, go, rust, ruby, php, swift, kotlin, and 50 more |
|
| 177 |
+
| **text_lang** | 30 | en, es, fr, de, it, pt, nl, ru, zh, ja, ko, vi, th, id, and 15 more |
|
| 178 |
+
| **file_mime** | **90** | application/json, image/png, video/mp4, font/ttf, application/wasm, and 84 more |
|
| 179 |
| **risk** | 6 | api_key, jwt, password, email, phone, ssh_key |
|
| 180 |
|
| 181 |
## 🌐 Deployment
|
| 182 |
|
| 183 |
+
| Platform | Link | Notes |
|
| 184 |
+
|----------|------|-------|
|
| 185 |
+
| **HuggingFace Space** | [eulogik/pico-type](https://huggingface.co/spaces/eulogik/pico-type) | Gradio web UI, no GPU needed |
|
| 186 |
+
| **HuggingFace Model** | [eulogik/pico-type](https://huggingface.co/eulogik/pico-type) | ONNX models + export metadata |
|
| 187 |
+
| **GitHub** | [eulogik/pico-type](https://github.com/eulogik/pico-type) | Source code, training, paper |
|
| 188 |
+
| **PyPI** | `pip install picotype` | Python package |
|
| 189 |
+
| **ONNX Runtime** | Use with onnxruntime.js | Browser/Node.js deployment |
|
| 190 |
+
|
| 191 |
+
## 📚 Resources
|
| 192 |
+
|
| 193 |
+
- [Paper](paper/main.pdf) — Architecture, training, and evaluation details
|
| 194 |
+
- [Model Card](MODEL_CARD.md) — Detailed architecture and training configuration
|
| 195 |
+
- [Walkthrough](walkthrough.md) — Development log and decisions
|
| 196 |
+
- [Architecture Plan](docs/PLAN.md) — Original design document
|
| 197 |
|
| 198 |
## 📄 License
|
| 199 |
|
| 200 |
Apache 2.0
|
| 201 |
+
|
| 202 |
+
---
|
| 203 |
+
|
| 204 |
+
<div align="center">
|
| 205 |
+
<sub>Built with PyTorch · ONNX · Gradio · HuggingFace</sub>
|
| 206 |
+
</div>
|