---
license: cc-by-4.0
language: [ru, en, tr, zh]
library_name: gliner
pipeline_tag: token-classification
tags: [gliner, ner, toponym, geonames, multilingual]
base_model: urchade/gliner_multi-v2.1
---

# geosearch-ner

Multilingual toponym NER, fine-tuned from [`urchade/gliner_multi-v2.1`](https://huggingface.co/urchade/gliner_multi-v2.1).
Labels: `CITY`, `REGION`, `STATE`, `COUNTRY`.

**Span F1 0.763** (P 0.759 / R 0.767) on 47 held-out queries at threshold 0.4.

This is the first stage of a multilingual toponym search pipeline:

```
text
  -> GLiNER (this model)
  -> char-n-gram BM25 retrieval
  -> CatBoost reranker
  -> ranked GeoNames places
```

## Serving it: the word-splitter contract

**This checkpoint requires a per-ideograph word splitter, and serving it without
one loses every Chinese span with no error of any kind.**

GLiNER classifies *token spans*, and its default whitespace splitter treats a
run of Han characters as one token — so `莫斯科` inside `莫斯科新闻` is not a
span the model can even express, let alone predict. This model was trained with
one token per Han ideograph (`src/ner/tokenizer.py::CjkAwareSplitter`), which
`gliner_config.json` cannot record: `words_splitter_type` names only GLiNER's
built-in kinds.

So the requirement is recorded in **`ner_meta.json`**, shipped beside the
weights in this repo:

```json
{"words_splitter": "src.ner.tokenizer.CjkAwareSplitter"}
```

The serving code reads that file and refuses to start when the configured
splitter disagrees with it. If you load this model yourself, re-attach an
equivalent splitter — measured effect of getting it wrong: 56/56 Chinese spans
found versus 0.

## Decision threshold

Served at **0.3**, below GLiNER's own 0.5, and
that is a property of *serving* rather than of the model: retrieval is
recall-hungry, because a city span never extracted can never be retrieved,
while a spurious span only adds a candidate the reranker can demote. The value
is derived from the sweep in `make ner-eval --sweep`, not asserted.

## Results

### Overall

| bucket | precision | recall | F1 | gold | predicted |
|---|---|---|---|---|---|
| overall | 0.759 | 0.767 | 0.763 | 86 | 87 |

### By label

| bucket | precision | recall | F1 | gold | predicted |
|---|---|---|---|---|---|
| CITY | 0.855 | 0.887 | 0.870 | 53 | 55 |
| REGION | 0.800 | 0.267 | 0.400 | 15 | 5 |
| STATE | 0.333 | 0.833 | 0.476 | 6 | 15 |
| COUNTRY | 0.833 | 0.833 | 0.833 | 12 | 12 |

### By language

| bucket | precision | recall | F1 | gold | predicted |
|---|---|---|---|---|---|
| ru | 0.818 | 0.818 | 0.818 | 22 | 22 |
| en | 0.809 | 0.809 | 0.809 | 21 | 21 |
| tr | 0.708 | 0.708 | 0.708 | 24 | 24 |
| zh | 0.700 | 0.737 | 0.718 | 19 | 20 |

### Versus the zero-shot baseline

Zero-shot `urchade/gliner_multi-v2.1`, measured with the **same** word splitter (otherwise the Chinese delta would be an artefact of segmentation rather than of fine-tuning):

| bucket | precision | recall | F1 | gold | predicted |
|---|---|---|---|---|---|
| overall | 0.726 | 0.616 | 0.667 | 86 | 73 |

Metrics are micro-averaged over **spans**, not macro over queries: a query
naming three cities feeds three names into retrieval, so each one is a unit of
work the pipeline either gets right or does not.

## Training

Selected by **validation span F1**, not `eval_loss` — the checkpoint saved is
whichever epoch scored best when called the way the engine calls it
(`predict_entities` on raw text).

| | |
|---|---|
| best epoch | 12 |
| best val span F1 | 0.763 |
| epochs | 15 |
| batch size | 8 |
| learning rate | 5e-06 |
| others LR | 1e-05 |
| weight decay | 0.01 |
| warmup ratio | 0.1 |
| seed | 42 |
| trained at | 2026-08-07T08:49:14+00:00 |

Trained on synthetic queries generated for this project — see
[`mki0809/geosearch-queries`](https://huggingface.co/datasets/mki0809/geosearch-queries).

## Data licence and attribution

Derived from [GeoNames](https://www.geonames.org/), licensed
[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).

**Modifications made to the source data:**

- filtered to populated places only (`feature_class = 'P'`)
- restricted to RU, US, TR, CN
- restricted to name variants in ru, en, tr, zh
- dropped feature codes PPLH, PPLQ, PPLW, PPLX
- name variants grouped per place and joined into single document strings


The fine-tune inherits any licence conditions of its base model; check
[`urchade/gliner_multi-v2.1`](https://huggingface.co/urchade/gliner_multi-v2.1)
before commercial use.