How to use from the
Use from the
LiteRT library
# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

MediaPipe Language Detector

Ultra-lightweight UTF-8 text language identification model (language_detector.tflite, 315 KB, FP32) recognizing 110 languages via ISO 639-1 / BCP-47 script codes (af through zu, plus unknown) in the MediaPipe Language Detector (LanguageDetector) task.

Documentation: MediaPipe Language Detector Docs

Samples: Android · iOS · Python Colab · Web · Flutter

Try live: Web Demo

Embedding-based neural classification network optimized with feature hashing and compact embedding lookup tables. Takes a single UTF-8 string (Placeholder [1]) and outputs a 111-way probability distribution (pod/Softmax [1, 111]), bundled with labels.txt (unknown + 110 ISO 639-1 / -Latn transliterated language codes).

Files

File Precision Size SHA-256
language_detector.tflite float32 315.3 KB (315,294 B) 7db4f23dfe1ad8966b050b419a865da451143fd43eb6b606a256aadeeb1e5417

Companion Files in Repo Root

File Role Size SHA-256
LanguageDetector Model Card.pdf Upstream Google model card (PDF) 250.2 KB (250,203 B) b383b7d189279949e1cc27aa04d09d967989eacd784cf91188a19c8811b69152
labels.txt Label map 0.4 KB (376 B) 938a527f1ed52b8a10cfb5491ac3229e60baae5befbbc7d3a60863917d0a0841

Download

pip install -U huggingface_hub
hf download litert-community/MediaPipe-LanguageDetector language_detector.tflite --local-dir .

Inputs, Outputs, and Model Specifications

language_detector.tflite

  • Embedded TFLITE_METADATA: name LangID · version 1 · author MediaPipe · packed associated files: labels.txt
Direction Tensor Shape Type Notes
Input Placeholder [1] string Placeholder
Output pod/Softmax [1, 111] float32 score — Prediction scores for each language. — Labels: labels.txt

Use with MediaPipe Tasks

Python Quickstart

pip install -U mediapipe huggingface_hub
import mediapipe as mp
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="litert-community/MediaPipe-LanguageDetector", filename="language_detector.tflite")

BaseOptions = mp.tasks.BaseOptions
LanguageDetector = mp.tasks.text.LanguageDetector
LanguageDetectorOptions = mp.tasks.text.LanguageDetectorOptions

options = LanguageDetectorOptions(
    base_options=BaseOptions(model_asset_path=model_path),
    score_threshold=0.5,
)
with LanguageDetector.create_from_options(options) as detector:
    result = detector.detect("Bonjour tout le monde, comment allez-vous ?")
    for det in result.detections:
        print(f"{det.language_code}: {det.probability:.3f}")

Cross-Platform Guides & Samples

Latency Benchmarks

End-to-end MediaPipe Task pipeline average latency reported by the Language Detector documentation:

Model Variant Device CPU Latency GPU Latency
Language Detector Pixel 6 0.31 ms —

Additionally, the bundled PDF model card reports 90 µs average latency on Pixel 2 (350-token text) with F1 = 0.9781 (Precision 0.9893, Recall 0.9673) on Creative Commons v2.5 en.wikinews.org multilingual samples.

Intended Use, Limitations, and Fairness

Summarized from the bundled LanguageDetector Model Card.pdf (Google Research Learn2Compress & MediaPipe Teams, May 10, 2023):

  • Intended applications: Predicting the language of input text (110 supported languages) as preprocessing for Smart Reply, OCR, and downstream NLP pipelines.
  • Training data & bias: Trained on PII-stripped translated anonymized web data covering languages across all 17 UN geographic subregions; has more training data for popular languages such as English and French.
  • Out-of-scope uses:
    • Not appropriate for human life-critical decisions, including financial, contractual, legal, or medical decisions.
    • Predicting languages outside the 110 preset codes in labels.txt.
    • Mixed multilingual strings (no guarantee that all languages or even the dominant language will be ranked first).

Provenance

Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded TFLITE_METADATA bytes) to the official artifact served by Google from storage.googleapis.com/mediapipe-models.

Repo File Upstream URL GCS Generation Upstream Last-Modified Pinned Bucket Version
language_detector.tflite language_detector.tflite 1682624733160468 Thu, 27 Apr 2023 19:45:33 GMT —

License and Citation

  • License: apache-2.0 — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF.
Downloads last month
68
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including litert-community/MediaPipe-LanguageDetector