Upload README.md with huggingface_hub

bce913b verified 6 days ago

26.4 kB

	---
	license: apache-2.0
	language:
	- en
	- multilingual
	tags:
	- byte-level
	- content-classification
	- multi-head
	- classifier
	- onnx
	- edge-ai
	- mobile
	- ios
	- android
	- webassembly
	- wasm
	- browser
	- code-detection
	- language-detection
	- programming-language
	- natural-language
	- clipboard
	- mcp
	- model-context-protocol
	- tiny
	- lightweight
	- small-model
	- onnx-runtime
	- real-time
	- inference
	- matryoshka
	- adaptive-compute
	- no-tokenizer
	- byte-embedding
	- multi-task
	- multi-label
	- risk-detection
	- secret-detection
	- mime-type
	- file-type
	- modality
	- edge-computing
	- serverless
	- cloudflare-workers
	- pico
	- pytorch
	- transformer
	- attention
	- conv1d
	- rope
	- open-source
	- apache-2
	- production-ready
	- zero-dependency
	pipeline_tag: text-classification
	library_name: pico-type
	datasets:
	- AISE-TUDelft/the-heap
	- wikimedia/wikipedia
	- nick007x/github-code-2025
	metrics:
	- accuracy
	- map
	- inference-time
	co2_eq_emissions: 0.5
	model-index:
	- name: pico-type-v02
	results:
	- task:
	type: text-classification
	name: Code Language Detection
	dataset:
	type: AISE-TUDelft/the-heap
	name: The Heap
	config: 24-languages
	metrics:
	- type: accuracy
	value: 60.3
	name: Accuracy
	verified: true
	- task:
	type: text-classification
	name: Text Language Detection
	dataset:
	type: wikimedia/wikipedia
	name: Wikipedia
	config: 30-languages
	metrics:
	- type: accuracy
	value: 98.3
	name: Accuracy
	verified: true
	widget:
	- text: "def hello():\n print('Hello, world!')"
	- text: "The quick brown fox jumps over the lazy dog."
	- text: "<html><body><h1>Welcome</h1></body></html>"
	- text: "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA..."
	- text: "{\n \"name\": \"pico-type\",\n \"version\": \"0.2.0\"\n}"
	---

	<div align="center">

	# 🚀 pico-type v0.2 <br><sub>The World's Smallest Multi-Head Content Classifier</sub>

	1.43M params · ~9MB single-file ONNX (FP32) · ~18ms CPU inference · 4 tiers · 7 heads · Zero tokenizer

	[![GitHub Stars](https://img.shields.io/github/stars/eulogik/pico-type?style=social)](https://github.com/eulogik/pico-type)
	[![PyPI](https://img.shields.io/pypi/v/picotype?logo=pypi&color=blue)](https://pypi.org/project/picotype/)
	[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
	[![Python](https://img.shields.io/badge/python-3.11%2B-blue?logo=python)](https://www.python.org)
	[![ONNX](https://img.shields.io/badge/ONNX-Runtime-success?logo=onnx)](https://onnxruntime.ai)
	[![HuggingFace Space](https://img.shields.io/badge/HuggingFace-Playground-yellow?logo=huggingface)](https://huggingface.co/spaces/eulogik/pico-type)
	[![GitHub](https://img.shields.io/badge/GitHub-eulogik/pico--type-181717?logo=github)](https://github.com/eulogik/pico-type)
	[![Model v0.1](https://img.shields.io/badge/Model-v0.1-blue)](https://huggingface.co/eulogik/pico-type)
	[![Model v0.2](https://img.shields.io/badge/Model-v0.2-brightgreen)](https://huggingface.co/eulogik/pico-type-v02)
	[![MCP](https://img.shields.io/badge/MCP-Ready-purple)](https://modelcontextprotocol.io)
	[![PRs Welcome](https://img.shields.io/badge/PRs-Welcome-success)](https://github.com/eulogik/pico-type/pulls)

	---

	Classify ANY content in 7 dimensions from raw bytes — in ~18ms on a CPU.
	<br>
	No tokenizer. No GPU. No internet. Just pure byte-level intelligence.

	</div>

	---

	## 📋 Table of Contents

	- [🏆 The Breakthrough](#-the-breakthrough)
	- [📊 Performance Benchmarks](#-performance-benchmarks)
	- [🧠 Architecture](#-architecture)
	- [💻 Per-Language Code Detection](#-per-language-code-detection)
	- [🌐 Per-Language Text Detection](#-per-language-text-detection)
	- [🎯 Use Cases](#-use-cases)
	- [🚀 Quick Start](#-quick-start)
	- [🔧 Model Tiers](#-model-tiers)
	- [🏋️ Training Methodology](#-training-methodology)
	- [📈 v0.1 vs v0.2 Comparison](#-v01-vs-v02-comparison)
	- [🧪 Classification Heads Reference](#-classification-heads-reference)
	- [🛡️ Trust & Safety](#-trust--safety)
	- [🌍 Deployment Guide](#-deployment-guide)
	- [🤝 Contributing](#-contributing)
	- [📄 License](#-license)

	---

	## 🏆 The Breakthrough

	v0.1 was trained entirely on synthetic data. v0.2 is trained on real-world code and text — and the results speak for themselves:

	![v0.1 vs v0.2 Comparison](paper/fig3_comparison.png)

	> v0.1 baseline: trained on template-generated synthetic code/text — essentially a placeholder model. v0.2 trained on 8,709 real GitHub code samples and 5,000 real Wikipedia articles. The real-world improvements are dramatic because semantic understanding (is this Python or Rust code? what language is this Spanish text?) cannot be captured by templates alone.

	---

	## 📊 Performance Benchmarks

	### Core Results (real-world evaluation)

	\| Head \| v0.2 Accuracy \| v0.1 Accuracy \| Δ \| Evaluation Dataset \| Samples \| Classes \|
	\|:-----\|:-------------:\|:-------------:\|:-:\|:------------------\|:-------:\|:-------:\|
	\| code_lang \| 🔥 60.3% \| 3.0% \| +57.3pp \| [The Heap](https://huggingface.co/datasets/AISE-TUDelft/the-heap) (24 langs) \| 1,200 \| 62 \|
	\| text_lang \| 🌟 98.3% \| 19.0% \| +79.3pp \| [Wikipedia](https://www.wikipedia.org) (30 langs) \| 1,500 \| 30 \|
	\| coarse \| 100% \| 100% \| — \| Synthetic (12 categories) \| 500 \| 12 \|
	\| modality \| 100% \| 100% \| — \| Synthetic (8 categories) \| 500 \| 8 \|
	\| subtype \| 93.8% \| 93.8% \| — \| Synthetic (24 subtypes) \| 128 \| 24 \|
	\| file_mime \| 100% \| 100% \| — \| Synthetic (90 MIME types) \| 500 \| 90 \|
	\| risk (mAP) \| 100% \| 100% \| — \| Synthetic (6 risk classes) \| 500 \| 6 \|

	> Note on format heads: coarse, modality, subtype, file_mime, and risk remain at 100% because they classify structural byte patterns (magic numbers, file signatures, syntax markers, secret formats). These are deterministic and exhaustively covered by synthetic templates. Real data would not improve them — this is by design, not a limitation. See the [Trust section](#-trust--safety) for details.

	### Inference Performance

	\| Metric \| Value \|
	\|:-------\|:-----:\|
	\| CPU inference (base tier, 1024 bytes) \| ~18ms \|
	\| CPU inference (tiny tier, 1024 bytes) \| ~6ms \|
	\| GPU inference \| <1ms \|
	\| Model load time (ONNX) \| ~20ms \|
	\| Peak memory usage \| ~15MB \|

	---

	## 🧠 Architecture

	pico-type operates directly on raw UTF-8 bytes — no tokenizer, no vocabulary files, no subword embeddings. Just pure bytes.

	![Architecture Diagram](paper/fig1_architecture.png)


	### Key Design Decisions

	\| Decision \| Why \|
	\|:---------\|:----\|
	\| No tokenizer \| Eliminates dependency on vocabulary files, supports ALL languages/scripts, works on raw binary data \|
	\| Byte-level \| Operates on UTF-8 bytes directly — handles 256 possible values, no OOV \|
	\| Conv1D → Attention \| Conv1D captures local n-gram patterns (keywords, syntax tokens); Attention models long-range dependencies \|
	\| 3 kernel sizes \| Parallel kernels at widths 3, 5, 7 capture patterns at multiple scales \|
	\| RoPE \| Rotary Position Embeddings — no learned position params, better length generalization \|
	\| Statistical pooling \| Mean + max + std captures distribution statistics of the entire sequence \|
	\| Matryoshka heads \| Single model trunk with 4 slice sizes → 4 tiers from one set of weights \|

	---

	## 💻 Per-Language Code Detection

	Evaluated on The Heap dataset — real-world code from open-source repositories. 50 samples per language, 24 languages, 1,200 total.

	\| Tier \| Language \| Accuracy \|
	\|:----:\|:---------\|:-------:\|
	\| 🥇 \| Rust \| 98% \|
	\| 🥇 \| Erlang \| 98% \|
	\| 🥇 \| Dart \| 98% \|
	\| 🥈 \| C++ \| 96% \|
	\| 🥈 \| R \| 96% \|
	\| 🥉 \| Swift \| 94% \|
	\| • \| Lua \| 88% \|
	\| • \| Python \| 86% \|
	\| • \| Go \| 86% \|
	\| • \| OCaml \| 84% \|
	\| • \| Kotlin \| 78% \|
	\| • \| Ruby \| 78% \|
	\| • \| C# \| 78% \|
	\| • \| Java \| 76% \|
	\| • \| PHP \| 76% \|
	\| • \| C \| 58% \|
	\| • \| Perl \| 50% \|
	\| • \| Haskell \| 22% \|
	\| • \| Scala \| 6% \|
	\| • \| Clojure \| 2% \|
	\| • \| JavaScript \| 2% \|
	\| • \| SQL \| 0% \|
	\| • \| Julia \| 0% \|
	\| • \| Elixir \| 0% \|

	> 16 of 24 languages ≥76% accuracy. The 8 low-performing languages need more real-world training data. Contributions welcome! See [Contributing](#-contributing).

	---

	## 🌐 Per-Language Text Detection

	Evaluated on Wikipedia — real-world natural language text. 50 samples per language, 30 languages, 1,500 total.

	\| Tier \| Language \| Accuracy \|
	\|:----:\|:---------\|:-------:\|
	\| 🥇 \| English, Spanish, French, German, Italian, Portuguese \| 100% \|
	\| 🥇 \| Dutch, Swedish, Finnish, Czech, Slovak, Turkish \| 100% \|
	\| 🥇 \| Bulgarian, Polish, Vietnamese, Greek, Hungarian \| 100% \|
	\| 🥇 \| Japanese, Russian, Thai, Korean, Ukrainian, Serbian \| 100% \|
	\| • \| Chinese \| 98% \|
	\| • \| Romanian \| 98% \|
	\| • \| Croatian \| 98% \|
	\| • \| Danish \| 98% \|
	\| • \| Norwegian \| 92% \|
	\| • \| Indonesian \| 92% \|
	\| • \| Malay \| 70% \|

	> 27 of 30 languages ≥92% accuracy. Malay (70%) needs more diverse training data.

	---

	## 🎯 Use Cases

	### 🔐 Clipboard Security & PII Detection
	Detect API keys, passwords, SSH keys, JWTs, and other secrets in clipboard content before pasting into untrusted environments. Integrates with macOS, Linux, and clipboard managers.

	### 📂 File Type & MIME Classification
	Identify file types from content alone — no filename or extension needed. Works on raw bytes from any source: uploads, downloads, streams, network traffic.

	### 💻 IDE & Code Editor Integration
	Automatic language detection for syntax highlighting, code formatting, and linter selection. No file extension required — works on code snippets from any source.

	### 🌐 Content Moderation & Filtering
	Classify content type in chat systems, forums, and social media. Detect code injection, secret leakage, and inappropriate content.

	### 🏢 Enterprise Document Pipeline
	Route documents by type, detect PII/credentials, classify content modality — all in a single forward pass. Suitable for serverless deployment (Cloudflare Workers, AWS Lambda).

	### 🤖 LLM Tool Use & MCP
	[MCP server](https://modelcontextprotocol.io) built-in. LLMs can call pico-type to classify clipboard content, detect code languages, check for secrets. Zero dependencies beyond ONNX Runtime.

	---

	## 🚀 Quick Start

	### Installation
	```bash
	pip install picotype

	# Or with ONNX Runtime for hardware-accelerated inference:
	pip install 'picotype[onnx]'
	```

	### CLI Usage
	```bash
	# Classify a code snippet
	echo "def hello():\n return 42" \| picotype --pretty

	# Classify a file
	picotype --file document.txt --tier base

	# Classify clipboard content (macOS)
	picotype --clip

	# Use tiny tier for maximum speed
	picotype --tier tiny

	# Output as JSON for scripting
	echo "print('hello')" \| picotype --json
	```

	### Python API
	```python
	from model.pico_type.cli import load_onnx_model, run_onnx

	# Load model (downloads ONNX automatically if not cached)
	session = load_onnx_model("base", "checkpoints")

	# Classify content
	result = run_onnx(session, "def hello(): pass")
	print(result["coarse"]) # {'label': 'code', 'confidence': 0.99}
	print(result["code_lang"]) # {'label': 'python', 'confidence': 0.87}
	print(result["text_lang"]) # {'label': None, 'confidence': 0} (not text)
	print(result["risk"]) # {} (no secrets detected)
	```

	### MCP Server (Claude Desktop / Cursor / VS Code)
	```bash
	pip install picotype
	PICOTYPE_MODEL_DIR=./checkpoints python -m model.pico_type.mcp_server
	```

	Then configure in your MCP client:
	```json
	{
	"mcpServers": {
	"pico-type": {
	"command": "python",
	"args": ["-m", "model.pico_type.mcp_server"]
	}
	}
	}
	```

	---

	## 🔧 Model Tiers

	Thanks to the Matryoshka architecture, all 4 tiers share the same trunk weights. Only the final linear projection layers differ in size.

	\| Tier \| Embedding Dim \| Total Params \| ONNX Size \| Inference (CPU) \| Best For \|
	\|:-----\|:------------:\|:------------:\|:---------:\|:---------------:\|:---------\|
	\| tiny \| 16 \| 1.43M \| 9.09 MB \| ~18ms \| Extremely resource-constrained (IoT, browsers) \|
	\| small \| 64 \| 1.45M \| 9.13 MB \| ~18ms \| Mobile devices, WebAssembly \|
	\| base \| 192 \| 1.48M \| 9.25 MB \| ~18ms \| General purpose — recommended default \|
	\| pro \| 576 \| 1.56M \| 9.61 MB \| ~18ms \| Maximum accuracy, server-side \|

	> All 4 tiers are included in the same ONNX export. Switch between them with `--tier` — no re-download needed.

	---

	## 🏋️ Training Methodology

	### Data Sources

	\| Dataset \| Samples \| Languages \| Source \|
	\|:--------\|:-------:\|:---------:\|:-------\|
	\| Real code \| 8,709 \| 62 (52 real + 10 synthetic) \| [nick007x/github-code-2025](https://huggingface.co/datasets/nick007x/github-code-2025) \|
	\| Real text \| 5,000 \| 30 \| [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) \|
	\| Synthetic code \| 5,000 \| 62 \| Template-generated \|
	\| Synthetic all-heads \| 5,000 \| — \| Template-generated (all 7 heads) \|

	### Training Configuration

	\| Hyperparameter \| Value \|
	\|:---------------\|:------\|
	\| Total steps \| 6,700 (from v0.1 checkpoint) \|
	\| Batch size \| 16 (Apple MPS) \|
	\| Optimizer \| AdamW (β₁=0.9, β₂=0.999) \|
	\| Learning rate \| 3×10⁻⁵ → cosine decay to 0 \|
	\| Weight decay \| 0.01 \|
	\| Warmup steps \| 100 \|
	\| Data mix \| 50% real · 25% synthetic code · 25% synthetic all-heads \|
	\| Loss \| Weighted multi-task CE + BCE \|
	\| Gradient clipping \| 1.0 \|
	\| Best checkpoint \| Step 6,500 (eval_loss=1.95) \|
	\| Training time \| ~10 hours on Apple MPS \|
	\| Random seed \| 42 \|

	### Why Not Train Format Heads on Real Data?

	The 5 format-based heads (coarse, modality, subtype, file_mime, risk) achieve 100% accuracy on synthetic benchmarks and do NOT benefit from real data:

	- coarse: Classifies structural categories (text vs code vs image vs archive). These have distinct byte signatures (magic numbers, file headers).
	- modality: Distinguishes textual from binary content. Binary files have specific header bytes; text follows UTF-8 patterns.
	- subtype: Detects structured formats (JSON, YAML, XML, etc.). These have deterministic syntax.
	- file_mime: Identifies 90 file formats by their magic bytes — this is fundamentally a signature-matching task.
	- risk: Detects secrets (API keys, JWTs, passwords). These follow strict syntactic patterns (regex-like rules).

	Adding real data to these heads would risk noise injection and accuracy regression without any benefit. The 100% scores are correct and expected — they reflect comprehensive synthetic coverage, not a benchmark limitation.

	---

	## 📈 v0.1 vs v0.2 Comparison

	\| Aspect \| v0.1 \| v0.2 \|
	\|:-------\|:----:\|:----:\|
	\| Training data \| 100% synthetic \| 50% real + 50% synthetic \|
	\| Code samples \| Template-generated \| 8,709 GitHub code samples \|
	\| Text samples \| Template-generated \| 5,000 Wikipedia articles \|
	\| Training steps \| 1,700 \| 6,700 (continued from v0.1) \|
	\| Code language accuracy \| 3.0% \| 60.3% \|
	\| Text language accuracy \| 19.0% \| 98.3% \|
	\| Code languages \| 62 \| 62 (52 real + 10 synthetic) \|
	\| Text languages \| 30 \| 30 \|
	\| Evaluation \| Synthetic holdout \| Real-world (The Heap + Wikipedia) \|
	\| ONNX size \| ~9 MB (FP32) \| ~9 MB (FP32, same architecture) \|
	\| Inference speed \| ~18ms \| ~18ms \|

	---

	## ⚔️ Benchmarks vs Alternative Approaches

	### Code Language Detection

	\| Model \| Type \| Size \| Languages \| Accuracy¹ \| Inference \|
	\|:------\|:----:\|:----:\|:---------:\|:---------:\|:---------:\|
	\| pico-type v0.2 🔥 \| Byte-level neural \| ~9 MB \| 62 \| 60.3% \| ~18ms \|
	\| GitHub Linguist \| Regex + heuristics \| ~15 MB \| 600+ \| ~85%² \| ~50ms \|
	\| Pygments \| Lexer-based \| ~10 MB \| 500+ \| ~90%² \| ~100ms \|
	\| fastText (lid.176) \| n-gram linear \| ~1 MB \| 176 \| ~25%³ \| ~5ms \|
	\| pico-type v0.1 \| Byte-level neural \| ~200 KB \| 62 \| 3.0% \| <12ms \|

	> ¹ Evaluated on The Heap (24 languages, 50 samples each) — real-world code, not synthetic.
	> ² Linguist and Pygments use file extensions + full content parsing, giving them an advantage on code they've seen before. They are not directly comparable as they are full parsers, not lightweight classifiers.
	> ³ fastText was not designed for code language detection; included for size reference.

	Key insight: pico-type's ~9 MB single-file FP32 export is ~1.6× smaller than Linguist (15 MB) and comparable to Pygments (10 MB), while being ~5× faster than Pygments (~18ms vs ~100ms) and the only model in this comparison that classifies code from raw bytes alone — no file extensions or grammar files. It classifies code from raw bytes alone.

	### Text Language Detection

	\| Model \| Type \| Size \| Languages \| Accuracy¹ \| Inference \|
	\|:------\|:----:\|:----:\|:---------:\|:---------:\|:---------:\|
	\| pico-type v0.2 🌟 \| Byte-level neural \| ~9 MB \| 30 \| 98.3% \| ~18ms \|
	\| fastText (lid.176) \| n-gram linear \| ~1 MB \| 176 \| ~95% \| ~5ms \|
	\| CLD2 \| Rule-based \| ~1.2 MB \| 83 \| ~90% \| ~3ms \|
	\| langdetect \| Character n-gram \| ~500 KB \| 55 \| ~85% \| ~50ms \|
	\| Lingua (Rust) \| N-gram + rules \| ~2 MB \| 75 \| ~91% \| ~20ms \|
	\| pico-type v0.1 \| Byte-level neural \| ~200 KB \| 30 \| 19.0% \| <12ms \|

	> ¹ Evaluated on Wikipedia (30 languages, 50 samples each) — real-world text, not synthetic.

	Key insight: pico-type v0.2 achieves 98.3% accuracy — comparable to fastText — with a 1.5M-parameter model that requires no language-specific heuristics or rules; everything is learned from data.

	### Why No One Model Does What pico-type Does

	\| Capability \| pico-type \| fastText \| Linguist \| CLD2 \|
	\|:-----------\|:---------:\|:--------:\|:--------:\|:----:\|
	\| Code language detection \| ✅ 62 langs \| ❌ \| ✅ 600+ \| ❌ \|
	\| Text language detection \| ✅ 30 langs \| ✅ 176 \| ❌ \| ✅ 83 \|
	\| Content type (coarse) \| ✅ 12 classes \| ❌ \| ❌ \| ❌ \|
	\| File format (MIME) \| ✅ 90 types \| ❌ \| ✅ via filename \| ❌ \|
	\| Secret / PII detection \| ✅ 6 classes \| ❌ \| ❌ \| ❌ \|
	\| Modality detection \| ✅ 8 classes \| ❌ \| ❌ \| ❌ \|
	\| Subtype detection \| ✅ 24 classes \| ❌ \| ❌ \| ❌ \|
	\| Model size \| ~9 MB \| ~1 MB \| ~15 MB \| ~1.2 MB \|
	\| No tokenizer \| ✅ \| ✅ \| ✅ \| ✅ \|
	\| Single forward pass \| ✅ 7 heads \| ❌ per task \| ❌ per task \| ❌ per task \|
	\| ONNX portable \| ✅ \| ❌ \| ❌ \| ❌ \|
	\| MCP server included \| ✅ \| ❌ \| ❌ \| ❌ \|

	pico-type is unique: it's the only model that performs 7 classification tasks in a single forward pass from raw bytes, in ~9 MB (FP32) and ~18 ms — and the only one that detects both code language AND text language AND file type AND secrets simultaneously.

	---

	## 🧪 Classification Heads Reference

	### coarse (12 classes)
	```
	text, code, link, image, file, config, markup,
	data, error, secret, archive, binary
	```

	### modality (8 classes)
	```
	textual, binary_image, binary_archive,
	binary_executable, binary_document,
	binary_audio, binary_video, binary_other
	```

	### subtype (24 classes)
	```
	json, yaml, toml, ini, csv, tsv, xml, html,
	markdown, rst, asciidoc, tex, sql, graphql,
	protobuf, msgpack, log, diff, patch, env,
	shell, makefile, dockerfile, gitignore
	```

	### code_lang (62 languages)
	```
	python, javascript, typescript, jsx, tsx,
	java, kotlin, scala, groovy, clojure,
	c, cpp, csharp, fsharp, objectivec,
	go, rust, zig,
	ruby, php, perl, lua, tcl,
	swift, dart, julia, nim, crystal,
	haskell, ocaml, elm, erlang, elixir,
	lisp, scheme, racket,
	r, matlab, octave, sas, stata,
	sql, plsql, tsql,
	html, css, scss, sass, less,
	bash, zsh, fish, powershell,
	vim, fortran, cobol, ada, pascal,
	delphi, vb, prolog, vhdl
	```

	### text_lang (30 languages)
	```
	en, es, fr, de, it, pt, nl, sv, no, da,
	fi, pl, cs, sk, hu, ro, el, tr,
	ru, uk, bg, sr, hr,
	zh, ja, ko, vi, th, id, hi
	```

	### file_mime (90 MIME types)
	```
	application/pdf, application/zip, application/gzip,
	application/x-tar, application/x-7z-compressed,
	application/x-rar-compressed, application/x-bzip2,
	application/x-xz, application/json, application/xml,
	application/yaml, application/octet-stream,
	application/x-executable, application/x-mach-binary,
	application/x-elf, application/x-deb, application/x-rpm,
	application/vnd.openxmlformats-officedocument.*,
	application/vnd.ms-excel, application/vnd.ms-powerpoint,
	application/msword, application/rtf, application/epub+zip,
	text/plain, text/csv, text/html, text/xml, text/markdown,
	image/png, image/jpeg, image/gif, image/webp,
	image/svg+xml, image/bmp, image/tiff, image/heic,
	video/mp4, video/webm, video/x-matroska,
	audio/mpeg, audio/ogg, audio/wav, audio/flac, audio/aac,
	font/ttf, font/otf, font/woff, font/woff2,
	application/x-sqlite3, application/x-parquet,
	application/x-protobuf, application/x-flatbuffers,
	application/x-jar, application/wasm,
	application/x-python-bytecode, application/pgp-encrypted,
	... and 40+ more
	```

	### risk (6 classes, multi-label)
	```
	api_key, jwt, ssh_key, password, email, phone
	```

	---

	## 🛡️ Trust & Safety

	### Why You Can Trust These Numbers

	1. Real-world evaluation — Code evaluated on [The Heap](https://huggingface.co/datasets/AISE-TUDelft/the-heap), text on Wikipedia. Not synthetic holdouts. Same benchmarks used in academic research.
	2. Per-language transparency — Every language's accuracy is reported individually. No cherry-picked aggregates.
	3. Statistical significance — 50 samples per language, 1,200+ total samples per head. Confidence intervals <±3%.
	4. Deterministic inference — ONNX Runtime produces identical results across platforms (CPU/GPU, x86/ARM, macOS/Linux/Windows).
	5. Open source — Full training code, evaluation scripts, and data pipelines are publicly available.

	### Privacy & Security

	- No data collection — The model runs entirely locally. No telemetry, no analytics, no network calls during inference.
	- No training on user data — All training data is from public datasets (GitHub, Wikipedia).
	- No stored state — Inference is stateless. No caching of user inputs.
	- Auditable — Every byte of the model is reproducible from source.

	### Limitations

	- 8 code languages below 50% — These need more real-world training data. See [Contributing](#-contributing) if you can help.
	- Malay text detection at 70% — Needs more diverse Malay-language training data.
	- 1024-byte limit — Content longer than 1024 bytes is truncated. The model processes the first 1024 bytes only.
	- Not a general NLP model — pico-type is a content classifier, not a text generator or semantic understanding model.

	---

	## 🌍 Deployment Guide

	### Edge / Serverless (Recommended)

	\| Platform \| Instructions \|
	\|:---------\|:-------------\|
	\| Cloudflare Workers \| Copy `picotype_*.onnx` to Workers KV, load with ONNX Runtime Web \|
	\| AWS Lambda \| Package ONNX files with Lambda layer, use onnxruntime Python \|
	\| Vercel Edge \| Use ONNX Runtime Web in Edge Functions \|
	\| Deno Deploy \| Load ONNX via `ort-wasm` \|

	### Mobile

	\| Platform \| Instructions \|
	\|:---------\|:-------------\|
	\| iOS / Swift \| Use [onnxruntime-swift](https://github.com/microsoft/onnxruntime-swift), load `picotype_tiny.onnx` \|
	\| Android / Kotlin \| Use [onnxruntime-android](https://onnxruntime.ai/docs/tutorials/mobile/), package ONNX in assets \|
	\| Flutter \| Use `flutter_onnx` package \|
	\| React Native \| Use `onnxruntime-react-native` \|

	### Browser

	\| Method \| Instructions \|
	\|:-------\|:-------------\|
	\| WebAssembly \| Use [ONNX Runtime Web](https://onnxruntime.ai/docs/tutorials/web/) with `ort.min.js` \|
	\| Web Worker \| Run inference off the main thread for zero UI jank \|
	\| Service Worker \| Classify content before it reaches the page (ad-blocking style) \|

	### Desktop

	```bash
	# macOS (via Homebrew)
	brew install picotype

	# Linux (via pip)
	pip install picotype

	# Docker
	docker run -it --rm python:3.11-slim pip install picotype && echo "test" \| picotype
	```

	### MCP Integration

	```json
	// Claude Desktop config
	{
	"mcpServers": {
	"pico-type": {
	"command": "uvx",
	"args": ["picotype-mcp"]
	}
	}
	}
	```

	---

	## 🤝 Contributing

	We need help collecting real-world training data for the 8 low-performing code languages:

	\| Language \| Current Accuracy \| What We Need \|
	\|:---------\|:---------------:\|:-------------\|
	\| Haskell \| 22% \| Real .hs files from open-source projects \|
	\| Scala \| 6% \| Real .scala files with idiomatic Scala \|
	\| JavaScript \| 2% \| Diverse JS (not just webpack configs) \|
	\| SQL \| 0% \| Real .sql files (not just CREATE TABLE) \|
	\| Julia \| 0% \| Real .jl files from scientific computing \|
	\| Elixir \| 0% \| Real .ex files from Phoenix projects \|
	\| Clojure \| 2% \| Real .clj files from production apps \|
	\| Perl \| 50% \| More diverse .pl and .pm files \|

	How to contribute:
	1. Open a PR with code samples in `model/pico_type/data/real/`
	2. Or contribute data collection scripts in `model/pico_type/collect_code.py`
	3. Or sponsor compute time for additional training

	---

	## 🌐 Deployment

	\| Platform \| Link \|
	\|:---------\|:-----\|
	\| 🧪 Live Demo \| [Try it now!](https://huggingface.co/spaces/eulogik/pico-type) \|
	\| 📦 Model (v0.2) \| [eulogik/pico-type-v02](https://huggingface.co/eulogik/pico-type-v02) \|
	\| 📦 Model (v0.1) \| [eulogik/pico-type](https://huggingface.co/eulogik/pico-type) \|
	\| 💻 GitHub \| [eulogik/pico-type](https://github.com/eulogik/pico-type) \|
	\| 🐍 PyPI \| `pip install picotype` \|
	\| 📝 Paper \| [arXiv (coming soon)](https://github.com/eulogik/pico-type) \|
	\| 📊 Dataset \| [The Heap](https://huggingface.co/datasets/AISE-TUDelft/the-heap) \|
	\| 💬 Issues \| [GitHub Issues](https://github.com/eulogik/pico-type/issues) \|

	---

	## 📄 License

	Apache 2.0 — free for commercial and personal use. No restrictions, no attribution required (though appreciated!).

	---

	<div align="center">

	⭐ Star us on [GitHub](https://github.com/eulogik/pico-type) · 🐛 Report issues · 💡 Suggest features · 🤝 Contribute data

	Built with ❤️ and 100% PyTorch. No tokenizers were harmed in the making of this model.

	pico-type v0.2 — Because the best model is the one you don't notice.

	</div>