Upload 10 files

52960ea verified about 1 month ago

9.63 kB

language:
  - zh
  - en
tags:
  - translation
  - ocr
  - screen-capture
  - chinese
  - easyocr
  - marianmt
license: mit
library_name: transformers
pipeline_tag: translation
model-index:
  - name: ChineseScreenTranslator
    results:
      - task:
          type: translation
          name: Chinese to English Screen OCR Translation
        metrics:
          - type: bleu
            value: N/A
            name: BLEU

ChineseScreenTranslator

A screen region selection tool that automatically detects and translates Chinese text (Simplified, Traditional, Cantonese, Classical) to English in real time, powered by EasyOCR and neural machine translation.

Features

Feature	Details
Screen Selection	Click-and-drag rubber-band region selector
OCR Engine	EasyOCR - Simplified + Traditional Chinese
Translation	Google Translate, Microsoft, or offline Helsinki-NLP
Chinese Variants	Simplified, Traditional, Cantonese, Classical, Mixed
Hotkey Trigger	Global configurable hotkey (default: Ctrl+Shift+S)
Image Preprocessing	Upscale, contrast enhance, sharpen for accuracy
History	Persistent JSON history with search and export
Clipboard	Auto-copy or one-click copy
Offline Mode	Helsinki-NLP MarianMT (opus-mt-zh-en)
GPU Support	CUDA acceleration for OCR and offline translation
Settings UI	Built-in settings window

Screenshots

+--------------------------------------+
|  ChineseScreenTranslator  v1.0.0     |
|  Detected Chinese Text               |
|  +----------------------------------+|
|  | 你好，欢迎使用本翻译工具。         ||
|  +----------------------------------+|
|  English Translation                 |
|  +----------------------------------+|
|  | Hello, welcome to this           ||
|  | translation tool.                ||
|  +----------------------------------+|
|  [Copy Translation] [Copy Source]    |
+--------------------------------------+

Requirements

System Requirements

Component	Minimum	Recommended
Python	3.9	3.11+
RAM	4 GB	8 GB+
Disk Space	1 GB	2 GB (models + deps)
GPU (optional)	CUDA 11.8	CUDA 12.1
OS	Windows 10 / Ubuntu 20.04 / macOS 12	Windows 11 / Ubuntu 22.04

First Run Downloads

Component	Size	Purpose
EasyOCR detection model	~50 MB	Text region detection
EasyOCR ch_sim model	~90 MB	Simplified Chinese recognition
EasyOCR ch_tra model	~90 MB	Traditional Chinese recognition
Helsinki-NLP opus-mt-zh-en	~300 MB	Offline translation (optional)

Installation

Step 1 - Clone the repository

git clone https://github.com/algorembrant/ChineseScreenTranslator.git
cd ChineseScreenTranslator

Step 2 - Create a virtual environment (recommended)

python -m venv venv

# Windows
venv\Scripts\activate

# Linux / macOS
source venv/bin/activate

Step 3 - Install PyTorch

Choose the command that matches your hardware:

Hardware	Command
CPU only (any OS)	`pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu`
NVIDIA GPU CUDA 11.8	`pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118`
NVIDIA GPU CUDA 12.1	`pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121`
macOS (Apple Silicon)	`pip install torch torchvision`

Step 4 - Install remaining dependencies

pip install -r requirements.txt

Step 5 - Run the application

python translator.py

The first launch downloads OCR models (~230 MB total). This only happens once.

Usage

Basic Usage

Action	How
Start application	`python translator.py`
Trigger screen capture	Press `Ctrl+Shift+S`
Select region	Click and drag on screen
View translation	Result window appears automatically
Copy translation	Click "Copy Translation" button
View history	Press `Ctrl+Shift+H`
Quit	Press `Ctrl+Shift+Q` or close window

Keyboard Shortcuts

Shortcut	Action
`Ctrl+Shift+S`	Open screen region selector
`Ctrl+Shift+C`	Copy last translation to clipboard
`Ctrl+Shift+H`	Show/hide translation history
`Ctrl+Shift+Q`	Quit application
`Escape`	Cancel selection overlay

Command Line Options

Option	Example	Description
`--hotkey`	`--hotkey "ctrl+alt+x"`	Set custom global hotkey
`--lang`	`--lang traditional`	Force OCR language mode
`--backend`	`--backend offline`	Set translation backend
`--offline`	`--offline`	Use offline MarianMT model
`--gpu`	`--gpu`	Enable CUDA GPU acceleration
`--upscale`	`--upscale 3`	Upscale factor for small text
`--confidence`	`--confidence 0.5`	OCR confidence threshold
`--no-preprocess`	`--no-preprocess`	Disable image preprocessing
`--export-history`	`--export-history out.json`	Export history on exit
`--verbose`	`--verbose`	Enable DEBUG logging
`--version`	`--version`	Show version and exit
`--help`	`--help`	Show full help

Translation Backends

Backend	Speed	Requires	Quality
`google` (default)	Fast	Internet	Excellent
`microsoft`	Fast	Internet + API key	Excellent
`offline`	Moderate	~300 MB model	Good

For Microsoft Translator, add your Azure Cognitive Services key to ~/.chinese_screen_translator/config.json:

{
  "microsoft_api_key": "YOUR_KEY_HERE",
  "microsoft_region": "eastus"
}

Supported Chinese Variants

Variant	Mode	Notes
Simplified Chinese (普通话 / 简体)	`--lang simplified`	Mainland China standard
Traditional Chinese (繁體中文)	`--lang traditional`	Taiwan, Hong Kong
Cantonese / Yue (廣東話)	`--lang traditional`	Use traditional mode
Classical Chinese (文言文)	`--lang auto`	Auto-detected
Mixed Chinese-English	`--lang auto`	Handled by both models
Vertical Chinese text	`--lang auto`	EasyOCR handles automatically

Configuration

Config file location: ~/.chinese_screen_translator/config.json

{
  "hotkey": "ctrl+shift+s",
  "lang": "auto",
  "backend": "google",
  "use_gpu": false,
  "upscale_factor": 2,
  "confidence_threshold": 0.30,
  "preprocess": true,
  "auto_copy": false,
  "show_confidence": true,
  "max_history": 500,
  "microsoft_api_key": "",
  "microsoft_region": "eastus",
  "offline_model_dir": "~/.chinese_screen_translator/models"
}

Troubleshooting

Problem	Solution
`easyocr` not found	`pip install easyocr` then install PyTorch
Models not downloading	Check internet connection; proxy may block HuggingFace CDN
Hotkeys not working on Linux	Run as root: `sudo python translator.py`
Low OCR accuracy	Increase `--upscale 3`, zoom in on text, improve lighting
`keyboard` error on macOS	Grant Accessibility permissions in System Preferences
Out of memory	Use CPU mode (no `--gpu`), reduce `--upscale`
No text detected	Lower `--confidence 0.2`, ensure text is clear and large enough
Google Translate blocked	Use `--backend offline` or set up Microsoft API key

Project Structure

ChineseScreenTranslator/
  translator.py          Main application (single file)
  requirements.txt       Python dependencies
  README.md              This file
  .gitignore             Git ignore rules
  .gitattributes         Git / HuggingFace LFS config
  LICENSE                MIT License

Architecture

Hotkey / Button
     |
     v
ScreenOverlay (tkinter fullscreen transparent)
     |  click + drag
     v
_capture_screenshot (mss / PIL)
     |
     v
ImagePreprocessor
  - Upscale (Lanczos)
  - Contrast enhance
  - Sharpen
     |
     v
OCREngine (EasyOCR)
  - ch_sim reader
  - ch_tra reader
  - Auto-detect or forced mode
     |
     v
TranslationEngine
  - Google Translate (deep-translator)
  - Microsoft Translator
  - Helsinki-NLP MarianMT (offline)
     |
     v
ResultWindow (tkinter)
HistoryManager (JSON persistence)

License

MIT License. See LICENSE for details.

Author

algorembrant

Acknowledgments

EasyOCR - OCR engine by JaidedAI
Helsinki-NLP - MarianMT translation models
deep-translator - Translation backends
mss - Fast screen capture