Text Classification
LiteRT
LiteRT
mediapipe
on-device
android
ios
web
python
flutter
nlp
language-identification
multilingual
fp32
Instructions to use litert-community/MediaPipe-LanguageDetector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/MediaPipe-LanguageDetector with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
File size: 7,988 Bytes
5811297 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | ---
license: apache-2.0
library_name: litert
pipeline_tag: text-classification
tags:
- mediapipe
- litert
- tflite
- text-classification
- on-device
- android
- ios
- web
- python
- flutter
- nlp
- language-identification
- multilingual
- fp32
---
# MediaPipe Language Detector
Ultra-lightweight UTF-8 text language identification model (`language_detector.tflite`, 315 KB, FP32) recognizing **110 languages** via ISO 639-1 / BCP-47 script codes (`af` through `zu`, plus `unknown`) in the [MediaPipe Language Detector (`LanguageDetector`)](https://developers.google.com/edge/mediapipe/solutions/text/language_detector) task.
<div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;">
<p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/text/language_detector">MediaPipe Language Detector Docs</a></p>
<p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/language_detector/python/%5BMediaPipe_Python_Tasks%5D_Language_Detector.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/language-detector.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/text/language_detector">Flutter</a></p>
<p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/text/language_detector">Web Demo</a></p>
</div>
Embedding-based neural classification network optimized with feature hashing and compact embedding lookup tables. Takes a single UTF-8 string (`Placeholder` `[1]`) and outputs a 111-way probability distribution (`pod/Softmax` `[1, 111]`), bundled with [`labels.txt`](./labels.txt) (`unknown` + 110 ISO 639-1 / `-Latn` transliterated language codes).
## Files
| File | Precision | Size | SHA-256 |
|---|---|---:|---|
| [`language_detector.tflite`](https://huggingface.co/litert-community/MediaPipe-LanguageDetector/blob/main/language_detector.tflite) | `float32` | 315.3 KB (315,294 B) | `7db4f23dfe1ad8966b050b419a865da451143fd43eb6b606a256aadeeb1e5417` |
### Companion Files in Repo Root
| File | Role | Size | SHA-256 |
|---|---|---:|---|
| [`LanguageDetector Model Card.pdf`](./LanguageDetector%20Model%20Card.pdf) | Upstream Google model card (PDF) | 250.2 KB (250,203 B) | `b383b7d189279949e1cc27aa04d09d967989eacd784cf91188a19c8811b69152` |
| [`labels.txt`](./labels.txt) | Label map | 0.4 KB (376 B) | `938a527f1ed52b8a10cfb5491ac3229e60baae5befbbc7d3a60863917d0a0841` |
### Download
```bash
pip install -U huggingface_hub
hf download litert-community/MediaPipe-LanguageDetector language_detector.tflite --local-dir .
```
## Inputs, Outputs, and Model Specifications
### `language_detector.tflite`
- Embedded `TFLITE_METADATA`: name `LangID` · version `1` · author `MediaPipe` · packed associated files: `labels.txt`
| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | `Placeholder` | `[1]` | `string` | Placeholder |
| Output | `pod/Softmax` | `[1, 111]` | `float32` | score — Prediction scores for each language. — Labels: `labels.txt` |
## Use with MediaPipe Tasks
### Python Quickstart
```bash
pip install -U mediapipe huggingface_hub
```
```python
import mediapipe as mp
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(repo_id="litert-community/MediaPipe-LanguageDetector", filename="language_detector.tflite")
BaseOptions = mp.tasks.BaseOptions
LanguageDetector = mp.tasks.text.LanguageDetector
LanguageDetectorOptions = mp.tasks.text.LanguageDetectorOptions
options = LanguageDetectorOptions(
base_options=BaseOptions(model_asset_path=model_path),
score_threshold=0.5,
)
with LanguageDetector.create_from_options(options) as detector:
result = detector.detect("Bonjour tout le monde, comment allez-vous ?")
for det in result.detections:
print(f"{det.language_code}: {det.probability:.3f}")
```
### Cross-Platform Guides & Samples
- **Task overview & options:** [Language Detector documentation](https://developers.google.com/edge/mediapipe/solutions/text/language_detector)
- **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter)
- **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/text/language_detector) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/language-detector.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/language_detector/python/%5BMediaPipe_Python_Tasks%5D_Language_Detector.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/text/language_detector)
## Latency Benchmarks
End-to-end MediaPipe Task pipeline average latency reported by the [Language Detector documentation](https://developers.google.com/edge/mediapipe/solutions/text/language_detector#benchmarks):
| Model Variant | Device | CPU Latency | GPU Latency |
|---|---|---:|---:|
| Language Detector | Pixel 6 | 0.31 ms | — |
Additionally, the bundled PDF model card reports **90 µs** average latency on Pixel 2 (350-token text) with **F1 = 0.9781** (Precision 0.9893, Recall 0.9673) on Creative Commons v2.5 `en.wikinews.org` multilingual samples.
## Intended Use, Limitations, and Fairness
Summarized from the bundled [`LanguageDetector Model Card.pdf`](./LanguageDetector%20Model%20Card.pdf) (Google Research Learn2Compress & MediaPipe Teams, May 10, 2023):
- **Intended applications:** Predicting the language of input text (110 supported languages) as preprocessing for Smart Reply, OCR, and downstream NLP pipelines.
- **Training data & bias:** Trained on PII-stripped translated anonymized web data covering languages across all 17 UN geographic subregions; has more training data for popular languages such as English and French.
- **Out-of-scope uses:**
- Not appropriate for **human life-critical decisions**, including financial, contractual, legal, or medical decisions.
- Predicting languages outside the 110 preset codes in `labels.txt`.
- Mixed multilingual strings (no guarantee that all languages or even the dominant language will be ranked first).
## Provenance
Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded `TFLITE_METADATA` bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`.
| Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version |
|---|---|---|---|---|
| `language_detector.tflite` | [`language_detector.tflite`](https://storage.googleapis.com/mediapipe-models/language_detector/language_detector/float32/1/language_detector.tflite) | `1682624733160468` | Thu, 27 Apr 2023 19:45:33 GMT | — |
## License and Citation
- License: `apache-2.0` — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF.
|