Multi-Expert Routing for Multi-Domain Low-Resource OCR: A Manchu Case Study Zhan Chen1,2 * Jiqiao Ma3† Chih-wen Kuo4‡ 1
2
Institute of Advanced Studies, Beijing Normal University, Zhuhai, China Faculty of Humanities and Social Sciences, Beijing Normal University–Hong Kong Baptist University United International College (UIC), Zhuhai, China 3 Independent Scholar 4 Department of Applied History, National Chiayi University, Chiayi, Taiwan
July 2026
arXiv:2607.14041v1 [cs.CV] 15 Jul 2026
Abstract
torical documents and manuscripts. Existing work in this setting commonly trains one recognizer per language, collection, or hand—from Tesseract [36] for print to Transkribus [16] and kraken [19] for historical manuscripts—using the limited annotated data available. This approach is reasonable when the visual material is homogeneous. It becomes less suitable when a single language is represented by several scripts, hands, periods, or media, because the recognizer must then accommodate substantially different image distributions.
Historical Manchu OCR must accommodate various visually distinct writing styles—like regular script, running script, and the semi-cursive chancery hand used in palace memorials—despite limited labeled data. We study a multi-expert system that reuses checkpoints from an iterative finetuning process as domain specialists and uses a lightweight page-level image classifier to dispatch pages by visual style. When the checkpoint pool lacks a suitable specialist, we train an additional expert for that domain. On three frozen test sets, the routed system matches the selected specialist for each style at two-decimal precision: 0.30% CER on regular script, 1.57% on memorials, and 4.83% on running script. The router achieves 99.3% page-level domain accuracy and matches the domain-label oracle at the same precision. Two of the three selected specialists were not trained specifically for their final domain; only the running-script expert was trained with that domain as its target. We report the evaluation protocol, router design, and per-page predictions to make the comparison reproducible.
1
Manchu provides a clear example of this problem. It was the administrative language of the Qing empire (1636–1912), and its records are important primary sources for the history of China, Inner Asia, and the early modern world [6, 9]. The surviving material is extensive: the First Historical Archives of China holds on the order of two million documents written wholly or partly in Manchu, while the Qing archive retrieval system of the National Palace Museum in Taipei catalogues 17,056 Manchu-language records, including 2,801 palace memorials. Only a small fraction of this material has been transcribed into machine-readable text. Page-level OCR therefore offers a practical route from digitized images to searchable text, provided that it can accommodate the visual variation within the corpus.
Introduction
Historical Manchu is also a demanding OCR tarOptical character recognition is well developed for get (Section 2 details the script and its material): a many modern, high-resource scripts, but remains vertical, connected script with position-dependent difficult for languages represented mainly in his- letterforms, degraded archival scans, and material that spans several visually distinct domains, from * [email protected] † the upright letterforms of woodblock print to the [email protected] ‡ semi-cursive chancery hand of palace memorials Corresponding author: [email protected] 1
are retained as candidate domain specialists; 2. a lightweight image-level domain router that dispatches pages to experts, with an explicit leakage-safe training protocol, a sourcebalancing fix for visually adjacent domains, and a domain-label oracle; 3. empirical evidence on historical Manchu that routing matches the best evaluated domain scores at two-decimal precision (regular 0.30%, memorial 1.57%, running 4.83% CER), which no single checkpoint attains, using two checkpoints not trained specifically for their selected domains and one purposetrained running-script checkpoint; and 4. a reproducible evaluation package consisting of the model assignments, router protocol, source-separated test sets, and per-page predictions.
to fluent running-script manuscript. Performance on regular-script pages therefore need not transfer to running-hand cursive. This within-language variation is the central problem considered here and is common to other historical scripts. Pooling all domains into one training set can produce a useful generalist, but it also creates a compromise: in our experiments, a checkpoint that performs well on regular script performs poorly on running script, and the converse also occurs. The limited size of each domain makes it undesirable to train every specialist independently from scratch. Our starting point is the checkpoint history produced by iterative fine-tuning. The project contains a sequence of versions (v3.0, v4.0, v4.1, and later versions), trained on different data snapshots and augmentation recipes. Their domain scores differ: one checkpoint performs best on regular script, another on memorials, and another on running script. Changes that improve a handwriting domain can therefore reduce performance on a domain that was already well represented. The resulting checkpoints form an observed trade-off in the three-domain error space (Section 5). We retain these checkpoints and select complementary specialists from them. A lightweight image-level domain router predicts the visual style of each page and dispatches the page to the corresponding specialist. In this finite checkpoint pool, the domain-label oracle combines the lowest observed domain-specific errors, and the learned router matches it at the reported precision. The additional computation is one classifier pass before page transcription; the page recognizers themselves are existing checkpoints. On the Manchu test sets used here, the routed system reaches sub-percent CER on regular-script pages while also evaluating memorial and runningscript material. The broader implication is methodological: in a low-resource, multi-domain project, earlier checkpoints can remain useful when their domain-specific strengths are preserved and selected at inference time. We report the model assignments, router protocol, evaluation procedure, and per-page predictions so that the result can be checked and extended. Our contributions are:
2
Background
The Manchu script and its romanization. Manchu is written in an alphabetic script adapted, by way of Mongolian, from Old Uyghur; it runs vertically from top to bottom, with columns ordered left to right. It is a connected script: letters join into a continuous word-stem, and most letters take distinct initial, medial, and final forms according to position, with further contextual variants. The standard scholarly transcription is the Möllendorff romanization, which maps the script to a Latin alphabet with a few diacritics (e.g., š, ū, ž). Following common practice and the prior benchmark [5], our system outputs Möllendorff romanization rather than Manchu Unicode: romanization is what most Manchu scholarship reads and searches, and it sidesteps the still-unsettled digital rendering of the script. The recognition target is therefore a Latin character string, and our metric is a character error rate over it.
The written record and its visual styles. Surviving Manchu materials were produced using different technologies and for different purposes. Manchu palaeography therefore distinguishes several writing styles rather than one uniform hand; the First Historical Archives of China groups much of its material into categories such as memorial1. a benchmark-scoped trade-off formulation of petition, book, proclamation, running-hand, and multi-domain low-resource OCR, in which woodblock styles. For recognition we group our checkpoints from a fine-tuning version stream corpus into three visual styles: 2
are organized around one model per language, collection, or hand [27]. Our setting differs in that one language contains several visual domains, and the training history itself supplies a pool of checkpoints with different domain strengths.
• regular script—the upright, well-separated letterforms of woodblock and movable-type imprints (the archives’ woodblock style) and of formal book hands (its book style): reign chronicles, primers, printed collections, and neatly written archival pages; • memorial script—the controlled semi-cursive chancery hand of palace memorials submitted to the throne (the archives’ memorial-petition style), frequently bearing vermilion imperial rescripts; • running script—the fluent, ligature-heavy semi-cursive of manuscripts and correspondence (the archives’ running-hand style).
Manchu OCR. Machine-learning work on Manchu OCR is recent, and existing systems are mostly word-level. One line of work segments pages into word images—using projection profiles or stroke growth—and recognizes each word with a CRNN [35], a sliding-window CNN [42], or related models; ManchuOCR is an open-source example [38]. More recent work fine-tunes vision–language models on synthetic word images and evaluates them at the word level [5]. Institutional recognizers also exist but are not publicly available; as far as we understand, they likewise use a segment-then-recognize design.
Although these materials use the same language and script, their letterforms and page layouts differ substantially. Our corpus—the Old Manchu Archives, the manc.hu manuscript database, printed collections, and palace memorials— therefore spans all three styles rather than a single Document image-to-sequence models. Pageclean source. level recognizers avoid an explicit segmentation stage by encoding a page image and emitting the transcription directly. Donut [20] and 3 Related Work Nougat [3] are examples of document encoder– OCR for ancient and historical scripts. Recent decoder models; Pix2Struct [21], Kosmos-2.5 [25], work on OCR and HTR for historical and low- and GOT-OCR2.0 [40] use related document or resource scripts addresses similar data constraints. screenshot-reading objectives. General-purpose Examples include cuneiform sign detection with vision–language models [1, 24] can also transcribe annotated corpora and processing pipelines [4]; pages, although specialized document models renew datasets for East Syriac [26] and Sahidic Cop- main useful for high-precision settings. Linetic [31]; recognition of Ge’ez/Ethiopic [2]; Dead and word-level recognizers, including CRNN/CTC Sea Scroll palaeography [29]; and Arabic-script re- [35], TrOCR [22], and kraken [19], require an upsources using open corpora [28] and multimodal stream segmentation stage that page-level models avoid. models [39].
Mixtures of experts, ensembles, and reusing checkpoints. Routing inputs to specialized models is a classical idea [14], and modern mixtureof-experts systems learn routers and experts jointly [34, 10, 33, 15, 7]. Our system operates at the model level: the experts are independently trained checkpoints and the router is an external classifier. Snapshot ensembles retain several checkpoints from one training run [12], whereas model soups and stochastic weight averaging merge model parameters [41, 13]. These methods generally target one data distribution. We instead use checkpoints with different domain scores and select one model per page, so the comparison concerns domain specialization rather than output averaging or weight
OCR for Asian and minority scripts. Related work covers palm-leaf datasets [18], Indic layout parsing [30], Old Uyghur transcription with vision–language fine-tuning [8], and routed adapters for Tibetan, Yi, Shui, and Dongba [23]. Broader benchmarks continue to show substantial variation in performance across scripts [17]. Methods for the low-resource regime. Common strategies include transfer learning and parameter-efficient fine-tuning of pretrained backbones, such as TrOCR [22]; synthetic data generation; few-shot learning from glyph exemplars [37]; and data augmentation [32]. Most systems 3
merging.
Table 1: Page-level CER (%) of representative Nougat checkpoints on the three frozen test sets (regular P125, memorial Z10, running C13). Bold 4 Task and Evaluation Protocol marks each test set’s best result (column minimum). No single version wins every column. The The task is image-to-text transcription of a com- upper group contains checkpoints selected after plete Manchu page or folio into Möllendorff-style training; the lower group contains purpose-trained Romanization. Input images follow a fixed con- v5.0 variants. vention recorded with the data, including image Version Regular Memorial Running enhancement and optional cropping or denoising. checkpoints selected after training The original orientation and reading order are pre- v3.0 0.30 3.40 47.09 served for evaluation and human review. v3.0 (seed 2) 0.30 1.57 51.59 0.45 3.65 26.68 For a page with reference string g and prediction v4.1 (seed 2) p, character error rate is deliberate (purpose-trained v5.0) Lev(p, g) CER(p, g) = , |g|
v5.0 (mem.-tuned) v5.0 (curs.-tuned)
(1)
0.41 0.79
1.60 3.29
6.86 4.83
with whitespace normalized consistently and the page evaluated in its original reading order. We 5 Domains and the Version Stream report the mean over pages. This is a page-level CER, not a word accuracy, and must not be compared directly with word-image benchmarks (Sec- The project maintains a source-aware data registry (about 9,054 page/folio records across nine sources tion 9). as of version v4.1, and still growing) with a permanent train/validation/test split; the three test sets Frozen, source-separated test sets. Test report- above are frozen subsets and are not used for model ing uses three domain-specific sets held fixed training or router checkpoint selection. They are, across the project so that successive versions re- however, used retrospectively to designate the expert pool after the training runs are complete, as main comparable: disclosed in Section 6. Successive training ver• regular script: 125 pages of regular, well- sions differ in their data snapshot and augmentaseparated letterforms from several sources tion recipe. In particular, regular-script sources (woodblock and movable-type collections use mild augmentation, whereas the memorial and the regular hand of the Old Manchu and running-script sources receive stronger, styleArchives), the principal regular-script test set; specific augmentation and source-aware oversam• memorial: 10 handwritten palace-memorial pling, so that later versions increasingly favor the harder handwriting styles. pages (a memorial-domain probe); • running script: 13 semi-cursive manuscript The key empirical observation is that no single pages (a running-script probe). version is best on all three test sets. Table 1 reports page-level CER for representative Nougat checkEach set is derived directly from the test split points. The strongest regular-script and memoof the master data registry by a single construc- rial checkpoints are v3.0 and v3.0 s2, respectively; tion script, so it remains synchronized with the neither was trained specifically for the final asfrozen split and is never drawn from a stale snap- signment. The best earlier checkpoint on running shot. The handwriting sets are small and are script, v4.1 s2, reaches only 26.68%. A later v5.0 treated as domain-shift probes rather than large- run trained with running-script data reduces this sample estimates; enlarging them with additional error to 4.83%, but its regular-script and memoexpert-transcribed held-out pages is ongoing par- rial scores remain above those of the earlier checkallel work, kept separate from these frozen sets to points. These results provide the basis for selecting different checkpoints for different domains. preserve comparability. 4
Checkpoint trade-off projection
oracle reported in our experiments. A stronger perpage oracle would choose arg minm c(em , x) and is only a theoretical upper bound, not an experimental baseline here. With one designated specialist per domain, the learned router equals the domain-label oracle when its domain prediction is correct, but it need not equal the per-page oracle. The system therefore reduces to domain prediction followed by expert selection. Since the experts are existing checkpoints, the added model is only the page-level classifier.
v3.0-s2
Cursive CER (%) → worse
50 v3.0
40 2D trade-off envelope
30
v4.1-s2
coordinate-wise minimum v5.0-qam
0
purpose-trained (v5.0) coordinate-wise minimum
20 10
selected after training
0.3
v5.0-curs
0.4 0.5 0.6 0.7 0.8 Regular-script CER (%) → worse
0.9
Expert pool. All experts share one architecture—a Nougat document encoder– decoder [3] adapted to Manchu Romanization— and differ in their training version. The appendix reports a common-recipe comparison of three candidate backbones; Nougat performed most consistently across the three styles. From this pool we designate three domain experts, each the best available checkpoint on its domain in Table 1: v3.0 for regular script, v3.0 s2 for memorials, and v5.0 for running script. The first two checkpoints were selected after training rather than trained for those final assignments; the v5.0 running-script checkpoint was trained specifically for that domain. Because the expert pool was designated retrospectively from the completed test-set results, the routed number is a frozen-benchmark comparison rather than an independently held-out estimate of expert-pool selection.
Figure 1: A two-dimensional projection of the checkpoint trade-off: regular-script CER versus running-script CER (from Table 1). Grey points are checkpoints selected after training; blue points are purpose-trained v5.0 variants. The red star marks the coordinate-wise minimum on the two displayed test sets. The plot does not characterize the full three-set error space. A benchmark-scoped trade-off view. Figure 1 projects errors on two of the three test sets, with lower CER indicating better performance. It visualizes the observed checkpoint trade-off rather than establishing non-dominance in the full threedimensional space. The coordinate-wise minima in this pool are (0.30, 1.57, 4.83) for regular, memorial, and running script, respectively. No single checkpoint attains all three values. The domainlabel oracle combines the corresponding specialists, and the learned router matches it at twodecimal precision. The result is specific to the checkpoints and test sets used here and should be reassessed as the data and model pool expand.
6
Domain router. The router is a small residual image classifier [11] (ResNet-18, ImageNetinitialized, 224×224 input) that maps a page image to a domain label (regular / running / memorial). At inference the page is transcribed by the expert corresponding to the predicted label. Because reading a low-resolution page is enough to judge visual style, the router is small and fast; it adds a single classifier forward pass per page.
Method: Version Experts and a Domain Router
Problem statement. Figure 2 sketches the routed system. Let a page image x belong to one of K visual domains, and let {e1 , . . . , eM } be a pool of transcription experts, each a complete page-totext model. A router r maps x to an expert index, and the system transcribes x as er(x) (x). Writing c(e, x) for the CER of expert e on page x, we distinguish two reference policies. The domain-label oracle uses the true domain label y(x) and dispatches to its designated expert ey(x) ; this is the
Source-balanced training. The router’s supervision is imbalanced: the training split contains roughly 8000 regular-script, 153 running-script, and 50 memorial pages. Inverse-frequency class weighting and weighted sampling address the class imbalance. A second imbalance occurs within the regular-script class, which is dominated by one archival source. A naive router underfits the visually distinct manc.hu woodblock print and con5
PAGE-LEVEL INFERENCE
CHECKPOINT-DERIVED EXPERTS the router dispatches one page to one expert
REGULAR expert checkpoint · v3.0
DOMAIN ROUTER
ResNet-18 style label
MÖLLENDORFF MEMORIAL
transcription
expert checkpoint · v3.0 s2
page image regular · memorial · running
RUNNING expert checkpoint · v5.0
selected path
idle alternatives
Figure 2: The routed system. A lightweight domain router (a small image classifier) reads each page and predicts its visual style; the page is then transcribed by the corresponding domain expert—a Nougat checkpoint drawn from the project’s own version stream. The solid orange path shows a memorial page routed to the memorial expert (v3.0 s2); the dashed gray branches are alternative experts left idle for that page. fuses it with running-hand pages from the same stronger per-page oracle defined above. collection. We therefore assign equal sampling mass to regular-script sources. In the ablation, this change reduces the resulting confusion between vi- 7 Experiments sually adjacent sources. We evaluate on the three frozen test sets. For every page we run each expert, record its CER, and Leakage-safe training. The router is trained and then report, for each set, every single expert apselected only on training-split pages; the frozen plied to the full set, the domain-label oracle, and test sets (regular, memorial, running) are not used the learned router. for router fitting or router checkpoint selection. Domain labels are derived from source meta- Routing matches the domain-label oracle at redata (running-script source → running; memorial ported precision. With three experts—regular source → memorial; all others → regular). Be- script (v3.0), running script (v5.0), and memocause project policy removes the memorial source rial (v3.0 s2)—the router dispatches pages to the from the global validation pool, we reserve a small corresponding specialist. It matches the domainheld-out slice of training memorial pages purely label oracle to two decimal places on all three test to monitor memorial recall during router check- sets: 0.30% on regular script, 1.57% on memoripoint selection; these pages are never part of the als, and 4.83% on running script (Table 2). Its domemorial test set. Separately, the expert pool itself main accuracy is 99.3% (147/148; Figure 3). The was designated retrospectively from the completed single-expert results show why selection matters: frozen-test-set results, so this experiment does not the regular-script expert reaches 47.1% CER on claim a fully blind estimate of model-selection per- running script, whereas the running-script expert formance. reaches 0.79% on regular script. Effect of domain-specific training. The comparison separates the contribution of the checkpoint history from that of the purpose-trained expert. The best earlier checkpoint scores 26.68% on running script; the running-script v5.0 checkpoint reduces this to 4.83%. The same v5.0 campaign
Domain-label oracle. To separate router error from expert quality, we report a domain-label oracle that uses each page’s true domain label and the fixed domain-to-expert mapping. The gap between the learned router and this oracle measures the cost of domain-routing mistakes; it is distinct from the 6
oracle (best expert)
Test set
reg only
run only
mem only
oracle
router
mis
regular memorial running
0.30 3.40 47.09
0.79 3.29 4.83
0.30 1.57 51.59
0.30 1.57 4.83
0.30 1.57 4.83
1/125 0/10 0/13
routed page CER (%)
Table 2: Three-expert routing (regular v3.0 + running v5.0 + memorial v3.0 s2), page-level CER (%) on the frozen test sets (regular P125, memorial Z10, running C13). The router matches the domain-label oracle to two decimal places on all three sets. Domain accuracy is 99.3% (147/148). Off-diagonal columns show the cost of sending a page to the wrong expert.
true domain
1
0/13
0/13
2
0/10 1/113 0/113
other regular
0/10
1/12
manc.hu regular
memorial
running
annotations = misrouted pages / total; orange meeting the dashed line = router matches oracle
Figure 4: Source-balanced router training (ablation) on the frozen test sets. Within the regularscript class, a naive router underfits the minority manc.hu woodblock print and misroutes most of it to the running-script expert (blue, 7/12), inflating its routed CER to 4.1% against a domain-label oracle of 0.5%; balancing the regular-script sources (orange) recovers domain-label-oracle-level routing (1/12, 0.6%). The dominant archival regular hand and the other styles are unaffected.
routed to (predicted domain) regular running memorial
124
source-balanced
7/12
4
0
router accuracy 147/148 = 99.3%
regular
naive sampling
6
·
running
·
13
·
memorial
·
·
10
are not balanced: the router confuses manc.hu woodblock pages with running-hand pages from the same collection (Figure 4). Equalizing the sampling mass of regular-script sources removes most of this confusion. Why a shared Nougat backbone. All experts share the Nougat encoder–decoder. Under a common recipe, a from-scratch ResNet-50 recognizer is comparable to Nougat on regular script (0.48% versus 0.56%) but performs poorly on handwriting (5.04% on memorials and 18.67% on running script). Donut (Swin + mBART) is weaker on regular script (1.23%) but reaches 5.89% on running script (Appendix A). These results support using Nougat for the pooled experts.
Figure 3: Domain-router confusion matrix on the frozen test sets (rows: true domain; columns: expert routed to). The learned router places 147 of 148 pages correctly (99.3%); the single error is one atypical manc.hu regular-script page sent to the running-script expert. Its regular-script CER is 0.299%, versus 0.296% for the domain-label oracle, which rounds to 0.30% for both.
does not replace the selected checkpoints on regular script or memorials: its best scores on those test 8 Discussion sets are 0.41% and 1.60%, compared with 0.30% and 1.57%. The selected pool therefore combines The benefit of routing is largest when domaincheckpoints from different stages of training rather specific experts differ substantially. On the regularscript test set, routing to v3.0 gives the same than using a single final version. rounded CER as applying that expert to every page. On running script, the purpose-trained expert reRouter errors and source balancing. One page duces CER without changing the selected regularis misrouted: an atypical manc.hu regular-script script and memorial checkpoints. This separation page is sent to the running-script expert. The ef- allows a new domain to be added as an expert and fect on the regular-script test set is small (0.299% a router class without replacing the existing recogrouted CER versus 0.296% for the domain-label nizers. Within this benchmark, a new specialist can oracle; both round to 0.30%). The ablation shows a larger failure mode when regular-script sources lower the routed error when it improves a domain 7
natural-language processing for low-resource languages, not only Manchu, with the aim of helping preserve cultural heritage. Previous outputs have addressed Syriac, Latin, Greek, and Manchu. We also welcome contact from researchers and cultural-heritage projects working with other lowresource languages and related NLP needs. The release plan has four stages: (1) freeze the split and preprocessing manifest; (2) publish the router, evaluation scripts, and per-page prediction files; (3) publish expert weights subject to the underlying data and model licenses; and (4) add a public, independently adjudicated handwriting benchmark. This arXiv version reports the current state and does not imply that the later stages are complete. A public trial interface for the current page-level OCR is available at http://124. 223.33.6/manchu-ocr/. Translation is being developed as a separate extension and is not evaluated in this paper. As a next-version extension, we are exploring a three-checkpoint ensemble within each visual domain: v3.0, res50-print, and Donut for regular script; qam-s42, res50-bal, and Donut for memorials; and curs-s43, res50-print, and Donut for running script. The current design uses anchored voting for regular script and memorials and ROVER decoding for running script, with provisional target CERs of 0.28%, 1.39%, and 3.81%, respectively. These targets are not part of the present benchmark results; they will be formally evaluated under the frozen P125/Z10/C13 protocol in a future version. A companion Manchu-language parser is available at http://124.223.33.6/ manchu-parser/. It currently supports parsing, translation, and word-form analysis. The first four integrated dictionaries come from the Tohoku University collection at http://hkuri.cneas.tohoku.ac.jp/ project1/manchu/list?groupId=11, with additional data from https://manc.hu/. The parser is a separate language resource and is not part of the OCR benchmark reported here.
not already covered by the selected pool. Future work includes stronger running-script models, specialists for seals or marginalia, and larger handwriting test sets so that these comparisons can be estimated with greater precision.
9
Relation to Prior Manchu OCR Results
The most relevant prior system fine-tunes a vision– language model on synthetic word images and evaluates at the word level on regular script [5]. An open-source predecessor, ManchuOCR [38]— a community code release without an accompanying paper—likewise operates at the word level, segmenting a page into word images and recognizing each. Because our system instead transcribes complete pages across three visual styles, neither is directly comparable to it. (At the word level on regular script, our page-to-page system reaches 98.86% word accuracy, for reference.) These lines of work are complementary; a shared public wordand-page benchmark would allow comparison on common data.
10
Limitations and Release Status
The handwriting test sets are small (the memorial and running-script sets have 10 and 13 pages), so they are domain-shift probes rather than largesample estimates; enlarging them with additional expert-transcribed held-out pages is ongoing. The experts come from successive data snapshots rather than a single controlled recipe sweep, so version differences confound data and augmentation. The router uses a coarse three-way domain label; finer style distinctions may require more classes and more monitoring data, especially for domains removed from the validation pool. Finally, this is a proof of concept with a small number of experts; scaling the pool and the router to more domains is future work. Further progress depends on collaboration with fluent Manchu readers. In particular, expert review of difficult pages, correction of transcription errors, and expansion of running-script annotations would help improve recognition accuracy where the current system remains weakest. We also welcome collaborators who would like to join our team. Our broader work focuses on
11
Conclusion
We presented a multi-expert routing approach for multi-domain low-resource OCR. The system reuses checkpoints from an iterative fine-tuning process as domain experts and selects among them with a page-level visual classifier. On the frozen 8
[7] Damai Dai, Chengqi Deng, Chenggang Zhao, R. X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, et al. DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models. arXiv preprint arXiv:2401.06066, 2024.
Manchu test sets, it matches the best evaluated domain scores at two-decimal precision: 0.30% on regular script, 1.57% on memorials, and 4.83% on running script. The result supports retaining complementary checkpoints and adding a specialist when a domain remains underrepresented. The paper specifies the evaluation protocol and routing analysis; the release of models and data remains subject to the applicable licenses.
[8] Mehmet Oguz Derin and Erdem Uçar. Old Uyghur OCR: The first work-in-progress via reproducing fine-tuning of VLMs. Journal of Old Turkic Studies, 9(2):313–318, 2025.
References
[1] Jinze Bai, Shuai Bai, Shusheng Yang, Shi- [9] Mark C. Elliott. The Manchu-language jie Wang, Sinan Tan, Peng Wang, Junarchives of the Qing dynasty and the origins yang Lin, Chang Zhou, and Jingren Zhou. of the palace memorial system. Late Imperial Qwen-vl: A frontier large vision-language China, 22(1):1–70, 2001. model with versatile abilities. arXiv preprint [10] William Fedus, Barret Zoph, and Noam arXiv:2308.12966, 2023. Shazeer. Switch transformers: Scaling to tril[2] Birhanu Hailu Belay, Isabelle Guyon, Tadele lion parameter models with simple and effiMengiste, Bezawork Tilahun, Marcus Licient sparsity. Journal of Machine Learning wicki, Tesfa Tegegne, Romain Egele, and Research, 23(120):1–39, 2022. Tsiyon Worku. HHD-Ethiopic: A historical handwritten dataset for Ethiopic OCR [11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for imwith baseline models and human-level perforage recognition. In Proceedings of the IEEE mance. In Document Analysis and RecogniConference on Computer Vision and Pattern tion – ICDAR 2024, pages 23–38. Springer, Recognition (CVPR), pages 770–778, 2016. 2024. [3] Lukas Blecher, Guillem Cucurull, Thomas [12] Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E. Hopcroft, and Kilian Q. WeinScialom, and Robert Stojnic. Nougat: Neuberger. Snapshot ensembles: Train 1, get ral optical understanding for academic docM for free. In International Conference on uments. arXiv preprint arXiv:2308.13418, Learning Representations (ICLR), 2017. 2023. [4] Wentao Che, Esteban Garcés Arias, Asim [13] Pavel Izmailov, Dmitrii Podoprikhin, Timur Niaz, Andreas Bender, and Enrique Jiménez. Garipov, Dmitry Vetrov, and Andrew Gordon Automated sign detection across the ElecWilson. Averaging weights leads to wider tronic Babylonian Library: A large-scale optima and better generalization. In Conferdataset and end-to-end cuneiform OCR ence on Uncertainty in Artificial Intelligence pipeline. arXiv preprint arXiv:2606.22608, (UAI), 2018. 2026. [14] Robert A Jacobs, Michael I Jordan, Steven J [5] Yan Hon Michael Chung and Donghyeok Nowlan, and Geoffrey E Hinton. Adaptive Choi. Finetuning vision-language models mixtures of local experts. Neural Computaas ocr systems for low-resource languages: tion, 3(1):79–87, 1991. A case study of manchu. arXiv preprint [15] Albert Q. Jiang, Alexandre Sablayrolles, AnarXiv:2507.06761, 2025. toine Roux, et al. Mixtral of experts. arXiv [6] Pamela Kyle Crossley and Evelyn S. Rawski. preprint arXiv:2401.04088, 2024. A profile of the Manchu language in ch’ing Kahle, Sebastian Colutto, history. Harvard Journal of Asiatic Studies, [16] Philip Günter Hackl, and Günter Mühlberger. 53(1):63–102, 1993. 9
Wang, Daji Ergu, and Ying Cai. OmTranskribus—a service platform for tranniOCR: Generalist OCR for ethnic minority scription, recognition and retrieval of languages. arXiv preprint arXiv:2602.21042, historical documents. In 14th IAPR Inter2026. national Conference on Document Analysis and Recognition (ICDAR), volume 4, pages [24] Haotian Liu, Chunyuan Li, Qingyang Wu, 19–24. IEEE, 2017. and Yong Jae Lee. Visual instruction tun[17] Amir Hossein Kargaran, Nafiseh Nikeghbal, ing. Advances in neural information processJana Diesner, François Yvon, and Hinrich ing systems, 36, 2024. Schütze. GlotOCR Bench: OCR models still struggle beyond a handful of Unicode scripts. [25] Tengchao Lv, Yupan Huang, Jingye Chen, Lei Cui, et al. Kosmos-2.5: A mularXiv preprint arXiv:2604.12978, 2026. timodal literate model. arXiv preprint [18] Made Windu Antara Kesiman, Sophea Prum, arXiv:2309.11419, 2023. Jean-Christophe Burie, and Jean-Marc Ogier. AMADI LontarSet: The first handwritten [26] Ameer Majeed and Hossein Hassani. Ancient but digitized: Developing handwritten optical Balinese palm leaf manuscripts dataset. In character recognition for East Syriac script 15th International Conference on Frontiers through creating KHAMIS dataset. arXiv in Handwriting Recognition (ICFHR), pages preprint arXiv:2408.13631, 2024. 168–173. IEEE, 2016. [19] Benjamin Kiessling. Kraken—an universal [27] Konstantina Nikolaidou, Mathias Seuret, Hamam Mokayed, and Marcus Liwicki. A text recognizer for the humanities. In Digital survey of historical document image datasets. Humanities Conference (DH2019), 2019. International Journal on Document Analy[20] Geewook Kim, Teakgyu Hong, Moonsis and Recognition (IJDAR), 25(4):305–338, bin Yim, JeongYeon Nam, Jinyoung Park, 2022. Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun [28] Open Islamicate Texts Initiative. OpenITI MAKHZAN: An open annotated dataset of Park. Ocr-free document understanding Arabic, Persian, Ottoman Turkish, and Urdu transformer. In European Conference on print and manuscript data. Zenodo, 2025. Computer Vision, pages 498–517. Springer, dataset. 2022. [21] Kenton Lee, Mandar Joshi, Iulia Raluca [29] Mladen Popović, Maruf A. Dhali, and Lambert Schomaker. Artificial intelligence based Turc, Hexiang Hu, Fangyu Liu, Julian Marwriter identification generates new evidence tin Eisenschlos, Urvashi Khandelwal, Pefor the unknown scribes of the Dead Sea ter Shaw, Ming-Wei Chang, and Kristina Scrolls exemplified by the great Isaiah scroll Toutanova. Pix2Struct: Screenshot parsing (1qisaa). PLOS ONE, 16(4):e0249769, 2021. as pretraining for visual language understanding. In International Conference on Machine Learning (ICML), pages 18893–18912, 2023. [30] Abhishek Prusty, Sowmya Aitha, Abhishek Trivedi, and Ravi Kiran Sarvadevabhatla. Indiscapes: Instance segmentation net[22] Minghao Li, Tengchao Lv, Jingye Chen, works for layout parsing of historical Indic Lei Cui, Yijuan Lu, Dinei Florencio, Cha manuscripts. In International Conference Zhang, Zhoujun Li, and Furu Wei. TrOCR: on Document Analysis and Recognition (ICTransformer-based optical character recogniDAR), pages 999–1006. IEEE, 2019. tion with pre-trained models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 13094–13102, [31] Fabio Quattrini, Carmine Zaccagnino, Costanza Bianchi, Silvia Cascianelli, and 2023. Rita Cucchiara. A text recognition dataset from Sahidic Coptic ancient manuscripts. [23] Bonan Liu, Zeyu Zhang, Bingbing Meng, arXiv preprint arXiv:2606.15987, 2026. Han Wang, Hanshuo Zhang, Chengping 10
adaptation. arXiv preprint arXiv:2506.02295, [32] Yassin Hussein Rassul, Aram M. Ahmed, 2025. Polla Fattah, Bryar A. Hassan, Arwaa W. Abdulkareem, Tarik A. Rashid, and Joan Lu. Advancing offline handwritten text recog- [40] Haoran Wei, Chenglong Liu, Jinyue Chen, et al. General OCR theory: Towards OCRnition: A systematic review of data aug2.0 via a unified end-to-end model. arXiv mentation and generation techniques. arXiv preprint arXiv:2409.01704, 2024. preprint arXiv:2507.06275, 2025. [41] Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Learning (ICML), [34] Noam Shazeer, Azalia Mirhoseini, Krzysztof pages 23965–23998, 2022. Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large [42] Diandian Zhang, Yan Liu, Zhuowei Wang, neural networks: The sparsely-gated mixtureand Depei Wang. OCR with the deep of-experts layer. In International Conference CNN model for ligature script-based lanon Learning Representations (ICLR), 2017. guages like Manchu. Scientific Programming,
[33] Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[35] Baoguang Shi, Xiang Bai, and Cong Yao. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE transactions on pattern analysis and machine intelligence, 39(11):2298–2304, 2016. [36] Ray Smith. An overview of the Tesseract OCR engine. In Ninth International Conference on Document Analysis and Recognition (ICDAR), volume 2, pages 629–633. IEEE, 2007. [37] Mohamed Ali Souibgui, Alicia Fornés, Yousri Kessentini, and Beáta Megyesi. Few shots are all you need: A progressive learning approach for low resource handwritten text recognition. Pattern Recognition Letters, 160:43–49, 2022. [38] tyotakuki. ManchuOCR: An OCR system for the Manchu script. https://github. com/tyotakuki/ManchuOCR. GitHub repository, accessed 2026-07-14. [39] Ahmed Wasfy, Omer Nacar, Abdelakreem Elkhateb, Mahmoud Reda, Omar Elshehy, Adel Ammar, and Wadii Boulila. QARIOCR: High-fidelity Arabic text recognition through multimodal large language model 11
2021:5520338, 2021.
A
Architecture comparison
Table 3 compares three page-level backbones trained with the same source-balanced (v5.0) recipe on the frozen test sets. Nougat is the most balanced of the three architectures: the from-scratch ResNet-50 recognizer is strong on regular script but performs poorly on handwriting, while Donut is weaker on regular script. This comparison motivates the use of Nougat for all experts in the routed pool. Table 3: Page-level CER (%) of three backbones under a common source-balanced recipe (regular P125, memorial Z10, running C13). Nougat balances all three styles; ResNet-50 is a regular-script specialist; Donut trails on regular script. Backbone
Regular
Memorial
Running
Nougat (enc–dec) ResNet-50 (from scratch) Donut (Swin + mBART)
0.56 0.48 1.23
2.33 5.04 3.75
5.92 18.67 5.89
12
B
Qualitative examples
Figure 5 shows the routed system’s output on one held-out page from each of the three visual styles, in Möllendorff romanization. The regular-script and running-script pages are transcribed exactly; the memorial page carries a small residual error rate. These pages are drawn from the frozen test sets and were never seen in training. Model output on one held-out page per visual style (Möllendorff romanization)
Regular script manc.hu regular hand
page CER exact OBURE BE SARA TEILE AKŪ TONDO OJORONGGE GELI HIYOOŠUN CI BANJINARA BE SACI OMBIKAI UBABE GUCUSE ULHICI …
Memorial script NPM palace memorial no. 156603
page CER 0.61% JERGI AHASI CI ANAME GELI AKŪNGGE URSE OCI EICI INI AHŪN DEO NIYAMAN HŪNCIHIN DOLO CIHANGGAI AISILAME …
Running script manc.hu manuscript mh5110
page CER exact JUWAN CUWAN I BELE UNGGIHE MIMBE ALIYA SEHEBI JE MA TANG NI BADE JUWAN EMU INENGGI INDEHE JUWAN JUWEDE …
Figure 5: Model output on one held-out page per visual style (regular, memorial, running), with the page-level CER and the opening of the predicted transcription. Live showcase: http://124.223. 33.6/manchu-ocr/.
13
C
Parser showcase
In addition to the OCR interface, the project provides a public Manchu-language assistance parser at http://124.223.33.6/manchu-parser/. The tool supports parsing, translation, and full word-form analysis. Its first four integrated Manchu dictionaries come from the Tohoku University collection at http://hkuri.cneas.tohoku.ac.jp/project1/manchu/list? groupId=11, with additional data from https://manc.hu/. The example below illustrates the current interface: dictionary glosses are retained in their original Chinese/Japanese form, while the English translation and retrieval-augmented morphological analysis remain experimental.
Figure 6: Example output from the public Manchu-language assistance parser. The interface shows parsing, Chinese translation, an experimental English translation, dictionary hits, and inline morphological segmentation for one sentence. The displayed “10/10” match is an interface example, not an independent accuracy evaluation.
14