riywo's picture
Retrained on synthetic Apache-2.0 gold; history squashed
4babb9a
|
Raw
History Blame Contribute Delete
2.37 kB
---
license: apache-2.0
library_name: transformers.js
pipeline_tag: token-classification
base_model: google/electra-small-discriminator
base_model_relation: finetune
language:
- en
tags:
- eye-grep
- log-analysis
- onnx
---
# eye-grep tagger β€” electra-small (distilled)
The small, browser-friendly tagger for **eye-grep**, a log colorizer that highlights
ids, timestamps, IPs and repeated strings in server logs. It is a **token classifier**
that labels each content token of a log line with one of 11 tags.
A **google/electra-small-discriminator** student (WordPiece), **knowledge-distilled**
from the [opsbr/eye-grep-deberta-v3-small](https://huggingface.co/opsbr/eye-grep-deberta-v3-small)
teacher (the two have different tokenizers, so distillation is done on aligned
per-content-token logits). At ~14 MB int8 it is ideal for an in-browser WebGPU/WASM
build β€” the eye-grep web demo and `eye-grep/react` use it by default. For maximum
accuracy (CLI / server) use the deberta teacher.
## Tag schema (11 classes)
`PUNCT WORD NUM RAND IP DURATION SIZE TIMESTAMP LEVEL URL PATH`
`RAND` is a high-entropy id (uuid / hash / token); `NUM`, `SIZE`, `DURATION` are
numeric values; `TIMESTAMP`, `IP`, `URL`, `PATH`, `LEVEL` are self-explanatory;
`WORD`/`PUNCT` are ordinary text.
## Files
ONNX in the transformers.js layout β€” `onnx/model.onnx` (fp32, for WebGPU) and
`onnx/model_quantized.onnx` (int8, for WASM) β€” plus the WordPiece tokenizer.
## Usage
eye-grep React component:
```tsx
import { LogView } from 'eye-grep/react';
<LogView text={logs} model="opsbr/eye-grep-electra-small" device="webgpu" />
```
transformers.js (browser):
```js
import { AutoTokenizer, AutoModelForTokenClassification } from '@huggingface/transformers';
const tokenizer = await AutoTokenizer.from_pretrained('opsbr/eye-grep-electra-small');
const model = await AutoModelForTokenClassification.from_pretrained(
'opsbr/eye-grep-electra-small', { dtype: 'q8', device: 'webgpu' });
```
## Training data
Distilled from the deberta teacher on the synthetic, fully-owned
[opsbr/eye-grep](https://huggingface.co/datasets/opsbr/eye-grep) gold set
(Apache-2.0) β€” no third-party log data.
## Notes
- A distilled student: smaller and faster than the deberta teacher at some accuracy cost.
- Same 11-tag schema and tokenizer-alignment contract as the teacher.