goryodog/tokihisu-correction-models

TOKIHISU Local Caption Correction Models

Browser-ready INT8 ONNX models used by TOKIHISU for local Japanese caption correction.

This repository contains converted and quantized derivatives of:

• Miwa-Keita/zenz-v3.1-xsmall
• izumi-lab/electra-small-japanese-discriminator

Purpose

• zenz generates Japanese orthography and homophone correction candidates.
• Japanese ELECTRA detects anomalous candidates and provides a KEEP/correct gate.
• URLs, identifiers, numbers, and readings are validated before a candidate is accepted.
• ELECTRA is not used to rewrite captions by itself.

Modifications

The upstream PyTorch checkpoints were:

• exported to ONNX opset 17
• dynamically quantized to INT8
• reorganized for browser loading through Transformers.js and ONNX Runtime Web
• accompanied by SHA-256 file-verification metadata

These are modified versions of the upstream models.

Upstream revisions

zenz

• Model: Miwa-Keita/zenz-v3.1-xsmall
• Revision: a728931c8b4867a53ca33d1ff3fe7360b0f15cd5
• License: CC BY-SA 4.0

Japanese ELECTRA

• Model: izumi-lab/electra-small-japanese-discriminator
• Revision: 990fd69669ecae8d35e6321a07601c163cf56890
• License: CC BY-SA 4.0

Runtime

The models are intended for local browser inference using:

• Transformers.js 4.2.0
• ONNX Runtime Web
• WASM
• Kuromoji/IPADIC-compatible Japanese preprocessing

File layout

• zenz/onnx/model_int8.onnx: quantized zenz model
• electra/onnx/model_int8.onnx: quantized Japanese ELECTRA model
• browser-manifest.json: pinned revisions, paths, byte sizes, and SHA-256 hashes
• THIRD_PARTY_NOTICES.md: upstream attribution and license notices

Limitations

These models do not guarantee that generated captions are correct.

TOKIHISU applies additional safeguards:

• reading preservation
• protected URL and identifier preservation
• numeric preservation
• maximum edit-ratio validation
• ELECTRA candidate comparison
• fallback to the original ASR result

See THIRD_PARTY_NOTICES.md for attribution details.