Fix Chinese OCR labels with bundled CJK font

Chinese OCR labels rendered as missing-glyph squares because the raster overlay used a Latin font. Bundle Noto Sans CJK and use the same selected font for label measurement and drawing. Preserve recognized text and coordinates. Validate mixed Chinese/English, simplified and traditional Chinese, punctuation, long labels and canvas edges.

Validation: 32 relevant unit tests passed; UI/case integrity check passed (9 routes, 22 cases); release bundle includes the font and license.

.gitattributes

@@ -74,3 +74,4 @@ cases/referring/09_referring_round5_candidates_gam_humanref_2178.jpg filter=lfs
 cases/visual_prompt/20_visual_prompt_demo_cases_demo_visual_prompt_15.jpg filter=lfs diff=lfs merge=lfs -text
 cases/visual_prompt/21_visual_prompt_round5_candidates_gam_visual_dense200_142.jpg filter=lfs diff=lfs merge=lfs -text
 cases/visual_prompt/22_visual_prompt_round5_candidates_gam_visual_dense200_187.jpg filter=lfs diff=lfs merge=lfs -text
+assets/NotoSansCJKsc-Regular.otf filter=lfs diff=lfs merge=lfs -text

assets/NotoSansCJK-OFL.txt

@@ -0,0 +1,92 @@
+This Font Software is licensed under the SIL Open Font License,
+Version 1.1.
+
+This license is copied below, and is also available with a FAQ at:
+http://scripts.sil.org/OFL
+
+-----------------------------------------------------------
+SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
+-----------------------------------------------------------
+
+PREAMBLE
+The goals of the Open Font License (OFL) are to stimulate worldwide
+development of collaborative font projects, to support the font
+creation efforts of academic and linguistic communities, and to
+provide a free and open framework in which fonts may be shared and
+improved in partnership with others.
+
+The OFL allows the licensed fonts to be used, studied, modified and
+redistributed freely as long as they are not sold by themselves. The
+fonts, including any derivative works, can be bundled, embedded,
+redistributed and/or sold with any software provided that any reserved
+names are not used by derivative works. The fonts and derivatives,
+however, cannot be released under any other type of license. The
+requirement for fonts to remain under this license does not apply to
+any document created using the fonts or their derivatives.
+
+DEFINITIONS
+"Font Software" refers to the set of files released by the Copyright
+Holder(s) under this license and clearly marked as such. This may
+include source files, build scripts and documentation.
+
+"Reserved Font Name" refers to any names specified as such after the
+copyright statement(s).
+
+"Original Version" refers to the collection of Font Software
+components as distributed by the Copyright Holder(s).
+
+"Modified Version" refers to any derivative made by adding to,
+deleting, or substituting -- in part or in whole -- any of the
+components of the Original Version, by changing formats or by porting
+the Font Software to a new environment.
+
+"Author" refers to any designer, engineer, programmer, technical
+writer or other person who contributed to the Font Software.
+
+PERMISSION & CONDITIONS
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of the Font Software, to use, study, copy, merge, embed,
+modify, redistribute, and sell modified and unmodified copies of the
+Font Software, subject to the following conditions:
+
+1) Neither the Font Software nor any of its individual components, in
+Original or Modified Versions, may be sold by itself.
+
+2) Original or Modified Versions of the Font Software may be bundled,
+redistributed and/or sold with any software, provided that each copy
+contains the above copyright notice and this license. These can be
+included either as stand-alone text files, human-readable headers or
+in the appropriate machine-readable metadata fields within text or
+binary files as long as those fields can be easily viewed by the user.
+
+3) No Modified Version of the Font Software may use the Reserved Font
+Name(s) unless explicit written permission is granted by the
+corresponding Copyright Holder. This restriction only applies to the
+primary font name as presented to the users.
+
+4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
+Software shall not be used to promote, endorse or advertise any
+Modified Version, except to acknowledge the contribution(s) of the
+Copyright Holder(s) and the Author(s) or with their explicit written
+permission.
+
+5) The Font Software, modified or unmodified, in part or in whole,
+must be distributed entirely under this license, and must not be
+distributed under any other license. The requirement for fonts to
+remain under this license does not apply to any document created using
+the Font Software.
+
+TERMINATION
+This license becomes null and void if any of the above conditions are
+not met.
+
+DISCLAIMER
+THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
+OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
+DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
+OTHER DEALINGS IN THE FONT SOFTWARE.

assets/NotoSansCJKsc-Regular.otf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c76254f6fc379fddfce0a7e84fb5385bb135d3e399294f6eeb6680d0365b74b
+size 16437364

assets/SOURCES.md

@@ -2,6 +2,7 @@
 
 - `xpeng-logo.png`: unchanged asset reused from the existing Oxygen demo at `X-CG_demo/public/xpeng-logo.png`; SHA256 `ff0db78ddf8f9a122eec74e8eb79581f8e5dbc46803dddeb780a06f5ae295750`. XPENG brand ownership is unchanged. The image is not a newly drawn logo.
 - `Manrope-variable.ttf`: [Google Fonts Manrope source](https://github.com/google/fonts/tree/main/ofl/manrope), SIL Open Font License bundled as `Manrope-OFL.txt`. Embedded locally to avoid font-service requests through the Fuyao proxy.
+- `NotoSansCJKsc-Regular.otf`: [Noto Sans CJK upstream](https://github.com/notofonts/noto-cjk/blob/f8d157532fbfaeda587e826d4cd5b21a49186f7c/Sans/OTF/SimplifiedChinese/NotoSansCJKsc-Regular.otf), pinned at `f8d157532fbfaeda587e826d4cd5b21a49186f7c`. Unmodified font for Chinese and mixed-language image labels; SIL Open Font License bundled as `NotoSansCJK-OFL.txt`. Raster label measurement and drawing share this font; no system font installation is needed.
 - Empty-state, close and focus marks: authored SVG geometry in `grounding_demo/console.py` and `console.js`; focus geometry is generated exclusively from validated GAM coordinates.
 
 Design guidance consulted: [Anthropic frontend-design](https://github.com/anthropics/skills/tree/main/skills/frontend-design) and [Impeccable](https://github.com/pbakaus/impeccable), animate/polish workflows. These are design references, not runtime dependencies.

docs/ui_case_pool_22_lock.json

@@ -3,7 +3,9 @@
     "app.py": "79282810a5f15138950ffb9d2ff108067f7f73cf52c797a1931920e1fbff40b8",
     "assets/Manrope-OFL.txt": "e01b637272e0cbdfb240184dd98ea5cc671556d9894dae2668d92ab2c906787c",
     "assets/Manrope-variable.ttf": "d0639be45d0af36e798172419d7bd173c4bd4f29e2b76cbb69db1d11bf8b0a40",
-    "assets/SOURCES.md": "3cceb52136b55f86840e9585e3c37faed1adc762452b0fea50aecba696dac7fa",
+    "assets/NotoSansCJK-OFL.txt": "6a73f9541c2de74158c0e7cf6b0a58ef774f5a780bf191f2d7ec9cc53efe2bf2",
+    "assets/NotoSansCJKsc-Regular.otf": "2c76254f6fc379fddfce0a7e84fb5385bb135d3e399294f6eeb6680d0365b74b",
+    "assets/SOURCES.md": "0299a8a04951a07c896b28168496209ff900e60f6ae0c98bfdc86a7cc47c1712",
     "assets/console.js": "3af36a3aeafb72e85018b1c663f624b92877bad0152ee9e264a1c440d689f60e",
     "assets/frosted.css": "393bf5f1c04a49a6e9b17e855e573646489b4add85bc7973eb6bb277686b5c80",
     "assets/original.css": "7a244c7f65e40797850dcde7d6b6505b1b5de60aa24b129f44b33ae52291fd2e",
@@ -35,7 +37,7 @@
     "grounding_demo/catalog.py": "75e543fed7f0ca018fb41c4b528efc2901aa6a0f05232959297d6d40798f676f",
     "grounding_demo/console.py": "5754564abe58099c9406de639663d9d1416b96bc65d7a89b24f402ec41978fb8",
     "grounding_demo/protocol.py": "cf043c9f026a1dcd63d55c5393f0d0921cd15eae4a0cc9a42e2834fab23486ec",
-    "grounding_demo/render.py": "ba6201d2eda2f3bf7fb0988e35f76303a4c43455d18cc60c4db547d87155d73d"
+    "grounding_demo/render.py": "3b3fe5901edc41586bf5f1a1c263265d47c0fbe4d2ee4c30697c1bd0f78d484b"
   },
   "case_contract_sha256": "006774de24ef12d7cd8a05f73e27ec2d27278ba63fbe000db368541f946dec93",
   "cases": 22,

grounding_demo/render.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 from collections.abc import Iterable
+from functools import lru_cache
 from pathlib import Path
 
 from PIL import Image, ImageDraw, ImageFont
@@ -14,9 +15,15 @@ CYAN = (55, 171, 183)
 PALETTE = (PURPLE, GREEN, PINK, CYAN)
 
 
-def _font(size: int) -> ImageFont.FreeTypeFont | ImageFont.ImageFont:
+@lru_cache(maxsize=32)
+def _load_font(size: int, multilingual: bool) -> ImageFont.FreeTypeFont | ImageFont.ImageFont:
+    assets = Path(__file__).resolve().parents[1] / "assets"
+    if multilingual:
+        # Pillow does not perform the browser's font fallback. Bundle CJK glyphs
+        # so Chinese OCR labels render identically on macOS and the Linux Space.
+        return ImageFont.truetype(str(assets / "NotoSansCJKsc-Regular.otf"), size=size)
     candidates = (
-        str(Path(__file__).resolve().parents[1] / "assets/Manrope-variable.ttf"),
+        str(assets / "Manrope-variable.ttf"),
         "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
         "/usr/share/fonts/truetype/liberation2/LiberationSans-Regular.ttf",
     )
@@ -29,6 +36,10 @@ def _font(size: int) -> ImageFont.FreeTypeFont | ImageFont.ImageFont:
     return ImageFont.load_default()
 
 
+def _font(size: int, text: str = "") -> ImageFont.FreeTypeFont | ImageFont.ImageFont:
+    return _load_font(size, not text.isascii())
+
+
 def _xy(values: tuple[int, ...], width: int, height: int) -> tuple[int, ...]:
     if len(values) == 2:
         return round(values[0] * (width - 1) / 999), round(values[1] * (height - 1) / 999)
@@ -57,9 +68,10 @@ def layout_box_labels(primitives, width, height, font, padding=4):
     # Give small targets first choice of nearby space. Large regions have more
     # candidate anchors. Ordering affects label placement only, never predictions.
     for label, boxes in sorted(groups.items(), key=lambda item: max(map(area,item[1]))):
+        label_font = _font(getattr(font, "size", 13), label) if not label.isascii() else font
         text = label if len(label)<=38 else label[:37]+"…"
         while text:
-            bounds = measure.textbbox((0,0),text,font=font)
+            bounds = measure.textbbox((0,0),text,font=label_font)
             tw, th = bounds[2]-bounds[0]+2*padding, bounds[3]-bounds[1]+2*padding
             if tw<=width-2 and th<=height-2: break
             text = text[:-2]+"…" if len(text)>2 else ""
@@ -89,7 +101,7 @@ def layout_box_labels(primitives, width, height, font, padding=4):
                 _,b,t=min(free,key=lambda item:item[0]);chosen=(b,t)
         if chosen:
             b,target=chosen;occupied.append(b)
-            placed.append(dict(label=label,text=text,box=b,target=target,text_bounds=bounds))
+            placed.append(dict(label=label,text=text,box=b,target=target,text_bounds=bounds,font=label_font))
     return placed
 
 
@@ -161,10 +173,11 @@ def draw_primitives(
             draw.line((sx,sy,ex,ey),fill=(*color,230),width=1)
         light=tuple(round(c*.10+255*.90) for c in color)
         draw.rounded_rectangle(box,radius=3,fill=(*light,248),outline=(*color,255),width=1)
-        draw.text((tx+padding-tb[0],ty+padding-tb[1]),text,font=font,fill=(38,30,49,255))
+        draw.text((tx+padding-tb[0],ty+padding-tb[1]),text,font=badge["font"],fill=(38,30,49,255))
 
     if title:
-        tb = draw.textbbox((0, 0), title, font=font)
+        title_font = _font(getattr(font, "size", 13), title)
+        tb = draw.textbbox((0, 0), title, font=title_font)
         draw.rounded_rectangle(
             (10, 10, min(width - 10, tb[2] + 28), tb[3] + 24),
             radius=10,
@@ -172,5 +185,5 @@ def draw_primitives(
             outline=(*PURPLE, 255),
             width=line,
         )
-        draw.text((18, 16), title, font=font, fill=(*PURPLE, 255))
+        draw.text((18, 16), title, font=title_font, fill=(*PURPLE, 255))
     return Image.alpha_composite(canvas, overlay).convert("RGB")

scripts/build_bundle.py

@@ -40,7 +40,7 @@ def main():
     paths.extend([lock, ROOT / "docs/UI_PAPER_R7.md"])
     if (ROOT / "docs/UI_STUDIO_R8.md").exists():
         paths.append(ROOT / "docs/UI_STUDIO_R8.md")
-    for folder, suffixes in {"assets": {".css", ".js", ".png", ".ttf", ".txt", ".md"},
+    for folder, suffixes in {"assets": {".css", ".js", ".png", ".ttf", ".otf", ".txt", ".md"},
                              "grounding_demo": {".py"}, "tests": {".py"}}.items():
         paths.extend(p for p in (ROOT / folder).rglob("*") if p.is_file() and p.suffix in suffixes)
     for row, public_row in zip(rows, public_rows, strict=True):

tests/test_bundle.py

@@ -20,6 +20,8 @@ class BundleTests(unittest.TestCase):
             with ZipFile(target) as archive:
                 names = archive.namelist()
                 self.assertIn("space_app.py", names)
+                self.assertIn("assets/NotoSansCJKsc-Regular.otf", names)
+                self.assertIn("assets/NotoSansCJK-OFL.txt", names)
                 self.assertIn("docs/HF_SPACE_DEPLOYMENT.md", names)
                 self.assertFalse(any(n.startswith(("verification/", "logs/", ".runtime/", "deploy/")) for n in names))
                 self.assertNotIn("docs/FUYAO_REPRODUCTION.md", names)

tests/test_console.py

@@ -88,6 +88,17 @@ class ConsoleTest(unittest.TestCase):
             "&lt;script&gt;", targets_html([{"label": "<script>", "coordinates": [1, 2]}])
         )
 
+    def test_chinese_ocr_result_preserves_unicode_and_coordinates(self):
+        label = "测试文字：中文 OCR（2026年）"
+        raw = f"<|object_ref_start|>{label}<|object_ref_end|><|box_start|><100><200><800><400><|box_end|>"
+        result = app.result_bundle(str(app.ROOT / app.CASES[0]["image"]), "OCR", "ocr", raw)
+        self.assertEqual(json.loads(result["normalized"]), [
+            {"label": label, "coordinates": [100, 200, 800, 400]}
+        ])
+        self.assertIn(label, result["normalized"])
+        self.assertIn(label, result["targets"])
+        self.assertIn("Complete", result["status"])
+
     def test_focus_buttons_carry_exact_safe_geometry(self):
         class Buttons(HTMLParser):
             def __init__(self):

tests/test_render.py

@@ -1,10 +1,48 @@
 import unittest
-from PIL import Image
+from PIL import Image, ImageDraw
 from grounding_demo.protocol import Primitive
 from grounding_demo.render import draw_primitives, layout_box_labels, _font, PURPLE
 
 
 class RenderRegression(unittest.TestCase):
+    def test_chinese_and_mixed_text_have_real_glyphs(self):
+        font = _font(24, "中文 OCR：姓名，日期（2026年）")
+        missing = bytes(font.getmask("\u0378"))
+        masks = []
+        for char in "姓名出生日期地址簡體繁體，：（）":
+            with self.subTest(char=char):
+                mask = bytes(font.getmask(char))
+                self.assertNotEqual(mask, missing)
+                self.assertTrue(any(mask))
+                masks.append(mask)
+        self.assertEqual(len(set(masks[:6])), 6)
+
+    def test_chinese_badges_measure_and_draw_with_the_same_font(self):
+        text = "中文 OCR：测试标签 2026年（示例）"
+        primitives = [Primitive(text, (100, 300, 800, 500))]
+        badge = layout_box_labels(primitives, 1000, 700, _font(20))[0]
+        self.assertEqual(badge["text"], text)
+        expected = ImageDraw.Draw(Image.new("RGB", (1, 1))).textbbox(
+            (0, 0), text, font=_font(20, text)
+        )
+        self.assertEqual(badge["text_bounds"], expected)
+        x1, y1, x2, y2 = badge["box"]
+        self.assertEqual(x2-x1, expected[2]-expected[0]+8)
+        self.assertEqual(y2-y1, expected[3]-expected[1]+8)
+        rendered = draw_primitives(Image.new("RGB", (1000, 700), "white"), primitives)
+        self.assertEqual(rendered.size, (1000, 700))
+
+    def test_long_chinese_badges_stay_inside_canvas_without_changing_text_data(self):
+        text = "这是一段很长的中文识别结果，包含标点和 English 2026。"*4
+        primitive = Primitive(text, (900, 0, 999, 999))
+        badges = layout_box_labels([primitive], 140, 120, _font(18))
+        self.assertEqual(len(badges), 1)
+        badge = badges[0]
+        self.assertEqual(badge["label"], text)
+        self.assertTrue(badge["text"].endswith("…"))
+        x1,y1,x2,y2 = badge["box"]
+        self.assertTrue(0<=x1<x2<=140 and 0<=y1<y2<=120)
+
     def test_dense_labels_are_unique_visible_and_do_not_overlap(self):
         boxes = [Primitive(f"category {i}", (50+(i%5)*180, 50+(i//5)*210,
                                            150+(i%5)*180, 190+(i//5)*210)) for i in range(20)]