From Global Alignment to Local Grounding: Zero-Shot Chinese Character Recognition with Radical Verification

Yu-Heng Shih1, Bing-Chen Wu1, Tsz-To Wong1, Ting-En Yen1, Hong-Han Shuai1, Bin-Hua Hsieh2, Chien-An Chen2, Yi-Ren Yeh3, Ching-Chun Huang1,*
1National Yang Ming Chiao Tung University, 2E.SUN Financial Holding Co., Ltd., 3National Kaohsiung Normal University
* Corresponding author
ACCV 2026
Framework overview: STG-CLIP (left), RVM (top right), two-stage inference (bottom right).

STG-CLIP retrieves candidates with spatial-aware IDS prototypes; the Radical Verification Module checks each candidate's radicals against the handwriting to settle look-alike characters such as 阿 · 啊 · 呵.

83.06%
Top-1 on ICDAR2013, character zero-shot
with 2,755 seen classes (state of the art)
+4.10
points over the previous best
(FT-CLIP, 78.96%)
94.02%
Top-4 recall of Stage-1 STG-CLIP
vs. 82.26% Top-1

Abstract

Zero-shot Chinese character recognition (ZS-CCR) aims to recognize characters whose categories are never observed during training, and typically relies on the compositional structure shared between seen and unseen characters. Recent CLIP-style methods represent this structure with the Ideographic Description Sequence (IDS) and align it with glyph images in a shared embedding space. However, they rely on a single global image–IDS similarity that discards the spatial layout of radicals and, being learned only implicitly from seen classes, generalizes poorly to unseen ones; moreover, global matching often retrieves the correct character within the top candidates yet fails to rank it first when characters differ only in subtle local radicals.

To address these issues, we propose a global-to-local two-stage framework. In the first stage, STG-CLIP augments the IDS with explicit tree-position and radical-level geometric priors, yielding a spatial-aware prototype that provides a consistent spatial description across seen and unseen categories for high-recall global retrieval. In the second stage, the Radical Verification Module (RVM) uses the radical instances of each retrieved candidate as queries to verify whether the corresponding radicals can be matched to spatially compatible regions in the input glyph. A margin-based gating rule activates the RVM only when the leading global candidates receive similar similarity scores. Experiments on the ICDAR2013 benchmark demonstrate that our method achieves state-of-the-art performance under the character-level zero-shot setting, obtaining 83.06% top-1 accuracy with 2,755 seen classes. Ablation studies further show that the explicit geometric priors and radical-level verification provide complementary improvements.

Method

Spatial-aware IDS

Each IDS token gets depth and sibling embeddings plus the radical's canonical geometry (centroid, bounding box, area ratio). These priors come from canonical glyphs, so they are identical for seen and unseen classes.

STG-CLIP retrieval

A ViT image encoder is contrastively aligned with the spatial-aware IDS prototypes, with a style term pulling handwritten and printed glyphs together. At test time it returns a high-recall Top-K shortlist.

Radical verification

Each candidate's radicals act as decoder queries over local CNN features, predicting radical presence and a spatial mask. A margin gate calls the RVM only when the top-2 global scores are close.

Local verification score

For a candidate yk, the RVM combines radical presence (mean existence probability, penalising extraneous radicals) with glyph coverage (how much of the observed foreground the union of predicted radical masks explains):

The mask head is trained with pixel-level radical masks on printed glyphs and then frozen; on handwriting the RVM is fine-tuned with the presence loss only, so no handwritten mask labels are needed.

Margin-based gating

With Δ the gap between the top-1 and top-2 global scores, a clear global winner is kept and a near-tie is handed to the RVM:

K = 2 and δ = 0.03 are chosen on a held-out validation set.

Results

Trained on CASIA-HWDB1.0–1.1 and tested on ICDAR2013. Character zero-shot: train on the first m of the 3,755 GB2312 level-1 characters, test on the last 1,000. Radical zero-shot: a character is held out if it contains a radical whose frequency is below n.

Cumulative ablation

Character zero-shot, 2,755 seen classes.

SettingTop-1Δ
Baseline76.27–
+ IDS prototype encoding77.78+1.51
+ Radical geometry augmentation82.26+4.48
+ RVM83.06+0.80

Top-k accuracy of Stage-1 STG-CLIP

The correct class is usually in the shortlist even when it is not ranked first.

# SeenTop-1Top-2Top-3Top-4
150059.9171.6877.1080.27
200069.9681.3985.6988.13
275582.2690.1792.7194.02

Radical Mask Visualization

Printed ground-truth radical masks (top) and RVM predictions on the corresponding handwritten instance (bottom); each column overlays one radical on the glyph. Although the mask head never sees handwritten masks, its predictions stay on the correct radical for both seen and unseen characters.

Radical mask predictions on seen characters.

BibTeX

@inproceedings{
shih2026from,
title={From Global Alignment to Local Grounding: Zero-Shot Chinese Character Recognition with Radical Verification},
author={Yu-Heng Shih and Bing-Chen Wu and Tsz-To Wong and Ting-En Yen and Hong-Han Shuai and Bin-Hua Hsieh and ChienAn Chen and Yi-Ren Yeh and Ching-Chun Huang},
booktitle={Eighteenth Asian Conference on Computer Vision},
year={2026},
url={https://openreview.net/forum?id=LH7kTObY2d}
}