Tongue: On-device Language Identification
On-device language identification that reads a short piece of text and names its language, across 84 languages, in under a millisecond.
Tongue reads a word or a sentence and returns the language it is written in, from "kann ich das haben" in German to "안녕하세요" in Korean, across 84 languages. It is tuned for short text such as search boxes, chat and keyboards, which is the hardest case for language detection.
A Unicode script layer settles anything the script alone decides, and a small hashed character n-gram model handles the rest. There is no tokenizer or vocabulary file, so the same weights give identical results on Swift, Kotlin and JavaScript.
Try it
Runs in your browser · nothing you type is sent anywhere
Performance
Across three public benchmarks the 2 MB model holds its own against detectors up to 130 times its size, with the other results shown alongside for reference.
FLORES-200
A standard translation benchmark built from Wikipedia, not part of any of these detectors' training data. Each line is truncated to its first 2, 3 and 5 words. Accuracy over the 20 languages they share; higher is better.
| Detector | Size | 2 words | 3 words | 5 words |
|---|---|---|---|---|
| Tongue | 2 MB | 0.869 | 0.933 | 0.974 |
| lingua | 266 MB | 0.800 | 0.887 | 0.956 |
| eld | ~1 MB | 0.780 | 0.856 | 0.912 |
The lingua test set
The benchmark the lingua library publishes: 1,000 single words, word pairs and sentences per language. Accuracy over the languages we share; higher is better.
| Detector | Size | Single words | Word pairs | Sentences |
|---|---|---|---|---|
| Tongue | 2 MB | 0.746 | 0.909 | 0.988 |
| lingua | 266 MB | 0.752 | 0.915 | 0.985 |
An independent benchmark (eld)
A separate benchmark from the Efficient Language Detector project, not part of any of these detectors' training data. Accuracy over shared languages; higher is better. Apple is the built-in system detector; HeLI-OTS is a 51 MB model.
| Detector | Size | Tweets | Single words | Word pairs | Sentences |
|---|---|---|---|---|---|
| Tongue | 2 MB | 0.992 | 0.759 | 0.887 | 0.971 |
| lingua | 266 MB | 0.984 | 0.756 | 0.894 | 0.950 |
| Apple | system | 0.997 | 0.641 | 0.719 | 0.748 |
| HeLI-OTS | 51 MB | 0.986 | 0.683 | 0.843 | 0.967 |
Single-word and two-word scores are largely common-vocabulary recognition — the frequent words of a language appear in everyone's data, so those buckets overlap for every detector. Word pairs and sentences are the clearest measure. Malay and Indonesian, which share the same words, are hard for all of them.
Use cases
Keyboards and input
Detect the language as the user types, to pick the right autocorrect, dictionary or layout, entirely on device.
Chat and messaging
Label or route a message by language before it is sent, with no network round-trip.
Search and content
Tag short queries and snippets by language to pick the right stemmer, index or translation source.
Offline and privacy-sensitive apps
Language detection that needs no server, so the text never leaves the device.
What it does
- 84 languages across 31 scripts, from Latin and Cyrillic to Cherokee and Thaana.
- Built for short text: a word or two, not only full paragraphs.
- Reports low confidence instead of guessing on ambiguous input.
- Small model, no server inference; identical results on every platform.
Specs
- Languages
- 84, across 31 scripts
- On-device size
- ~2 MB (int8)
- Latency
- Under 1 ms per detection
- Approach
- Unicode script routing plus a hashed character n-gram model
FAQ
What is Tongue?
On-device language identification that reads a short piece of text and names its language, across 84 languages, in under a millisecond.
Does Tongue run on device?
Yes. Tongue runs entirely on device: inference happens locally with no server call, so data never leaves the device.
Is Tongue available yet?
Tongue is available now: Swift Package Manager, Maven Central and npm. Source at github.com/Desert-Ant-Labs/tongue.
How much does Tongue cost?
Every model is free up to 100k monthly active devices per SDK. Unlimited inference per user. Contact us for custom licenses.
How accurate or fast is Tongue?
Across three public benchmarks the 2 MB model holds its own against detectors up to 130 times its size, with the other results shown alongside for reference.