Text Classification
LiteRT
LiteRT
mediapipe
on-device
android
ios
web
python
flutter
nlp
language-identification
multilingual
fp32
Instructions to use litert-community/MediaPipe-LanguageDetector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/MediaPipe-LanguageDetector with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
|
Download README.md from litert-community/MediaPipe-LanguageDetector: direct link, hf CLI and curl.
- Browser
- Download file 7.99 kB
-
https://huggingface.co/litert-community/MediaPipe-LanguageDetector/resolve/main/README.md
- Command line
-
hf download hf://litert-community/MediaPipe-LanguageDetector/README.md
-
curl -L -o README.md https://huggingface.co/litert-community/MediaPipe-LanguageDetector/resolve/main/README.md
7.99 kB
| license: apache-2.0 | |
| library_name: litert | |
| pipeline_tag: text-classification | |
| tags: | |
| - mediapipe | |
| - litert | |
| - tflite | |
| - text-classification | |
| - on-device | |
| - android | |
| - ios | |
| - web | |
| - python | |
| - flutter | |
| - nlp | |
| - language-identification | |
| - multilingual | |
| - fp32 | |
| # MediaPipe Language Detector | |
| Ultra-lightweight UTF-8 text language identification model (`language_detector.tflite`, 315 KB, FP32) recognizing **110 languages** via ISO 639-1 / BCP-47 script codes (`af` through `zu`, plus `unknown`) in the [MediaPipe Language Detector (`LanguageDetector`)](https://developers.google.com/edge/mediapipe/solutions/text/language_detector) task. | |
| <div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;"> | |
| <p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/text/language_detector">MediaPipe Language Detector Docs</a></p> | |
| <p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/language_detector/python/%5BMediaPipe_Python_Tasks%5D_Language_Detector.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/language-detector.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/text/language_detector">Flutter</a></p> | |
| <p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/text/language_detector">Web Demo</a></p> | |
| </div> | |
| Embedding-based neural classification network optimized with feature hashing and compact embedding lookup tables. Takes a single UTF-8 string (`Placeholder` `[1]`) and outputs a 111-way probability distribution (`pod/Softmax` `[1, 111]`), bundled with [`labels.txt`](./labels.txt) (`unknown` + 110 ISO 639-1 / `-Latn` transliterated language codes). | |
| ## Files | |
| | File | Precision | Size | SHA-256 | | |
| |---|---|---:|---| | |
| | [`language_detector.tflite`](https://huggingface.co/litert-community/MediaPipe-LanguageDetector/blob/main/language_detector.tflite) | `float32` | 315.3 KB (315,294 B) | `7db4f23dfe1ad8966b050b419a865da451143fd43eb6b606a256aadeeb1e5417` | | |
| ### Companion Files in Repo Root | |
| | File | Role | Size | SHA-256 | | |
| |---|---|---:|---| | |
| | [`LanguageDetector Model Card.pdf`](./LanguageDetector%20Model%20Card.pdf) | Upstream Google model card (PDF) | 250.2 KB (250,203 B) | `b383b7d189279949e1cc27aa04d09d967989eacd784cf91188a19c8811b69152` | | |
| | [`labels.txt`](./labels.txt) | Label map | 0.4 KB (376 B) | `938a527f1ed52b8a10cfb5491ac3229e60baae5befbbc7d3a60863917d0a0841` | | |
| ### Download | |
| ```bash | |
| pip install -U huggingface_hub | |
| hf download litert-community/MediaPipe-LanguageDetector language_detector.tflite --local-dir . | |
| ``` | |
| ## Inputs, Outputs, and Model Specifications | |
| ### `language_detector.tflite` | |
| - Embedded `TFLITE_METADATA`: name `LangID` · version `1` · author `MediaPipe` · packed associated files: `labels.txt` | |
| | Direction | Tensor | Shape | Type | Notes | | |
| |---|---|---|---|---| | |
| | Input | `Placeholder` | `[1]` | `string` | Placeholder | | |
| | Output | `pod/Softmax` | `[1, 111]` | `float32` | score — Prediction scores for each language. — Labels: `labels.txt` | | |
| ## Use with MediaPipe Tasks | |
| ### Python Quickstart | |
| ```bash | |
| pip install -U mediapipe huggingface_hub | |
| ``` | |
| ```python | |
| import mediapipe as mp | |
| from huggingface_hub import hf_hub_download | |
| model_path = hf_hub_download(repo_id="litert-community/MediaPipe-LanguageDetector", filename="language_detector.tflite") | |
| BaseOptions = mp.tasks.BaseOptions | |
| LanguageDetector = mp.tasks.text.LanguageDetector | |
| LanguageDetectorOptions = mp.tasks.text.LanguageDetectorOptions | |
| options = LanguageDetectorOptions( | |
| base_options=BaseOptions(model_asset_path=model_path), | |
| score_threshold=0.5, | |
| ) | |
| with LanguageDetector.create_from_options(options) as detector: | |
| result = detector.detect("Bonjour tout le monde, comment allez-vous ?") | |
| for det in result.detections: | |
| print(f"{det.language_code}: {det.probability:.3f}") | |
| ``` | |
| ### Cross-Platform Guides & Samples | |
| - **Task overview & options:** [Language Detector documentation](https://developers.google.com/edge/mediapipe/solutions/text/language_detector) | |
| - **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter) | |
| - **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/text/language_detector) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/language-detector.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/language_detector/python/%5BMediaPipe_Python_Tasks%5D_Language_Detector.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/text/language_detector) | |
| ## Latency Benchmarks | |
| End-to-end MediaPipe Task pipeline average latency reported by the [Language Detector documentation](https://developers.google.com/edge/mediapipe/solutions/text/language_detector#benchmarks): | |
| | Model Variant | Device | CPU Latency | GPU Latency | | |
| |---|---|---:|---:| | |
| | Language Detector | Pixel 6 | 0.31 ms | — | | |
| Additionally, the bundled PDF model card reports **90 µs** average latency on Pixel 2 (350-token text) with **F1 = 0.9781** (Precision 0.9893, Recall 0.9673) on Creative Commons v2.5 `en.wikinews.org` multilingual samples. | |
| ## Intended Use, Limitations, and Fairness | |
| Summarized from the bundled [`LanguageDetector Model Card.pdf`](./LanguageDetector%20Model%20Card.pdf) (Google Research Learn2Compress & MediaPipe Teams, May 10, 2023): | |
| - **Intended applications:** Predicting the language of input text (110 supported languages) as preprocessing for Smart Reply, OCR, and downstream NLP pipelines. | |
| - **Training data & bias:** Trained on PII-stripped translated anonymized web data covering languages across all 17 UN geographic subregions; has more training data for popular languages such as English and French. | |
| - **Out-of-scope uses:** | |
| - Not appropriate for **human life-critical decisions**, including financial, contractual, legal, or medical decisions. | |
| - Predicting languages outside the 110 preset codes in `labels.txt`. | |
| - Mixed multilingual strings (no guarantee that all languages or even the dominant language will be ranked first). | |
| ## Provenance | |
| Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded `TFLITE_METADATA` bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`. | |
| | Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version | | |
| |---|---|---|---|---| | |
| | `language_detector.tflite` | [`language_detector.tflite`](https://storage.googleapis.com/mediapipe-models/language_detector/language_detector/float32/1/language_detector.tflite) | `1682624733160468` | Thu, 27 Apr 2023 19:45:33 GMT | — | | |
| ## License and Citation | |
| - License: `apache-2.0` — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF. | |