File size: 7,988 Bytes
5811297
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
---
license: apache-2.0
library_name: litert
pipeline_tag: text-classification
tags:
- mediapipe
- litert
- tflite
- text-classification
- on-device
- android
- ios
- web
- python
- flutter
- nlp
- language-identification
- multilingual
- fp32
---

# MediaPipe Language Detector

Ultra-lightweight UTF-8 text language identification model (`language_detector.tflite`, 315 KB, FP32) recognizing **110 languages** via ISO 639-1 / BCP-47 script codes (`af` through `zu`, plus `unknown`) in the [MediaPipe Language Detector (`LanguageDetector`)](https://developers.google.com/edge/mediapipe/solutions/text/language_detector) task.

<div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;">
  <p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/text/language_detector">MediaPipe Language Detector Docs</a></p>
  <p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/language_detector/python/%5BMediaPipe_Python_Tasks%5D_Language_Detector.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/language-detector.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/text/language_detector">Flutter</a></p>
  <p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/text/language_detector">Web Demo</a></p>
</div>

Embedding-based neural classification network optimized with feature hashing and compact embedding lookup tables. Takes a single UTF-8 string (`Placeholder` `[1]`) and outputs a 111-way probability distribution (`pod/Softmax` `[1, 111]`), bundled with [`labels.txt`](./labels.txt) (`unknown` + 110 ISO 639-1 / `-Latn` transliterated language codes).

## Files

| File | Precision | Size | SHA-256 |
|---|---|---:|---|
| [`language_detector.tflite`](https://huggingface.co/litert-community/MediaPipe-LanguageDetector/blob/main/language_detector.tflite) | `float32` | 315.3 KB (315,294 B) | `7db4f23dfe1ad8966b050b419a865da451143fd43eb6b606a256aadeeb1e5417` |

### Companion Files in Repo Root

| File | Role | Size | SHA-256 |
|---|---|---:|---|
| [`LanguageDetector Model Card.pdf`](./LanguageDetector%20Model%20Card.pdf) | Upstream Google model card (PDF) | 250.2 KB (250,203 B) | `b383b7d189279949e1cc27aa04d09d967989eacd784cf91188a19c8811b69152` |
| [`labels.txt`](./labels.txt) | Label map | 0.4 KB (376 B) | `938a527f1ed52b8a10cfb5491ac3229e60baae5befbbc7d3a60863917d0a0841` |

### Download

```bash
pip install -U huggingface_hub
hf download litert-community/MediaPipe-LanguageDetector language_detector.tflite --local-dir .
```

## Inputs, Outputs, and Model Specifications

### `language_detector.tflite`

- Embedded `TFLITE_METADATA`: name `LangID` · version `1` · author `MediaPipe` · packed associated files: `labels.txt`

| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | `Placeholder` | `[1]` | `string` | Placeholder |
| Output | `pod/Softmax` | `[1, 111]` | `float32` | score — Prediction scores for each language. — Labels: `labels.txt` |

## Use with MediaPipe Tasks

### Python Quickstart

```bash
pip install -U mediapipe huggingface_hub
```

```python
import mediapipe as mp
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="litert-community/MediaPipe-LanguageDetector", filename="language_detector.tflite")

BaseOptions = mp.tasks.BaseOptions
LanguageDetector = mp.tasks.text.LanguageDetector
LanguageDetectorOptions = mp.tasks.text.LanguageDetectorOptions

options = LanguageDetectorOptions(
    base_options=BaseOptions(model_asset_path=model_path),
    score_threshold=0.5,
)
with LanguageDetector.create_from_options(options) as detector:
    result = detector.detect("Bonjour tout le monde, comment allez-vous ?")
    for det in result.detections:
        print(f"{det.language_code}: {det.probability:.3f}")
```

### Cross-Platform Guides & Samples

- **Task overview & options:** [Language Detector documentation](https://developers.google.com/edge/mediapipe/solutions/text/language_detector)
- **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter)
- **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/text/language_detector) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/language-detector.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/language_detector/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/language_detector/python/%5BMediaPipe_Python_Tasks%5D_Language_Detector.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/text/language_detector)

## Latency Benchmarks

End-to-end MediaPipe Task pipeline average latency reported by the [Language Detector documentation](https://developers.google.com/edge/mediapipe/solutions/text/language_detector#benchmarks):

| Model Variant | Device | CPU Latency | GPU Latency |
|---|---|---:|---:|
| Language Detector | Pixel 6 | 0.31 ms | — |

Additionally, the bundled PDF model card reports **90 µs** average latency on Pixel 2 (350-token text) with **F1 = 0.9781** (Precision 0.9893, Recall 0.9673) on Creative Commons v2.5 `en.wikinews.org` multilingual samples.

## Intended Use, Limitations, and Fairness

Summarized from the bundled [`LanguageDetector Model Card.pdf`](./LanguageDetector%20Model%20Card.pdf) (Google Research Learn2Compress & MediaPipe Teams, May 10, 2023):

- **Intended applications:** Predicting the language of input text (110 supported languages) as preprocessing for Smart Reply, OCR, and downstream NLP pipelines.
- **Training data & bias:** Trained on PII-stripped translated anonymized web data covering languages across all 17 UN geographic subregions; has more training data for popular languages such as English and French.
- **Out-of-scope uses:**
  - Not appropriate for **human life-critical decisions**, including financial, contractual, legal, or medical decisions.
  - Predicting languages outside the 110 preset codes in `labels.txt`.
  - Mixed multilingual strings (no guarantee that all languages or even the dominant language will be ranked first).

## Provenance

Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded `TFLITE_METADATA` bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`.

| Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version |
|---|---|---|---|---|
| `language_detector.tflite` | [`language_detector.tflite`](https://storage.googleapis.com/mediapipe-models/language_detector/language_detector/float32/1/language_detector.tflite) | `1682624733160468` | Thu, 27 Apr 2023 19:45:33 GMT | — |

## License and Citation

- License: `apache-2.0` — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF.