Benchmark

Accuracy

In the conversion and detection tables, the best value in each row is in bold. The tables of text flagged as Zawgyi have no bold: a detector can lower them just by calling Zawgyi less often, so read them next to the detection table. Every set is measured on its distinct lines.

Conversion: Zawgyi → Unicode

Higher is better. Both reference sets come from Google's i18n work, and CLDR's expected output follows ICU, the converter myanmar-tools ships, so myanmar-tools has a home advantage on them. CLDR pairs that repeat Google's file are counted once, in the Google row. "NFC" compares after Unicode NFC normalization, which treats canonically equivalent spellings as equal (ဦ typed as U+1025 U+102E or as U+1026). The round trip turns Wikipedia lines into Zawgyi with Rabbit and converts them back; Rabbit is left out of that row because it made the input. The Wikipedia text has typing errors of its own, mostly ဝ typed for zero in numbers (၁ဝ for ၁၀). Since 2.10 knayi corrects them, and this row counts each correction as a miss.

Detection: real text recognised

Higher is better. "On evidence" passes fallback unicode, so a word counts only when the detector finds Zawgyi evidence (for myanmar-tools, p above 0.95). 308 of 2390 distinct WaitZar words read the same in both encodings (neither Rabbit nor myanmar-tools changes them) and are left out.

Unicode flagged as Zawgyi

Lower is better, but nothing is bolded: a detector can flag less Unicode just by calling Zawgyi less often, so read these next to the detection table. "default" is a plain fontDetect(text), where a tie falls back to zawgyi. "evidence" uses fallback unicode, so only real Zawgyi evidence counts. For myanmar-tools both use the thresholds of knayi's adapter: Zawgyi above p = 0.95, Unicode below 0.05, and the fallback in between. A few lines in these sets are real Zawgyi, so 0% is not always reachable.

Other Myanmar-script languages flagged as Zawgyi

Lower is better, but nothing is bolded: a detector can flag less Unicode just by calling Zawgyi less often, so read these next to the detection table. "default" is a plain fontDetect(text), where a tie falls back to zawgyi. "evidence" uses fallback unicode, so only real Zawgyi evidence counts. For myanmar-tools both use the thresholds of knayi's adapter: Zawgyi above p = 0.95, Unicode below 0.05, and the fallback in between.

Web text without labels (mC4 Burmese validation)

Share of lines called Zawgyi on evidence, and agreement with myanmar-tools on the 14,225 lines where myanmar-tools is confident (p < 0.05 or p > 0.95).

Speed

Real text

6,821 distinct lines of FLORES-200 and Wikipedia, and their Zawgyi form made by Rabbit. Mean of 10 runs after 3 warm-ups. Timings depend on the machine, so compare the ratio: below 1 means knayi 2.10.0 is faster.

Long input

Inputs that took quadratic time before 2.9.1. One run each.

Sweep of 1,059 long inputs: every Myanmar code point repeated 30,000 times, every ordered pair of 29 marks repeated 10,000 times, and three base letters each followed by every mark, through 8 call forms. 8,472 runs, slowest 118 ms, 0 over 250 ms.

Limits

• The reference pairs are few, and both sets come from Google's i18n work. CLDR's expected output follows ICU, the converter myanmar-tools ships, and in at least one pair it expects ICU's own ordering of asat before tall aa.
• Exact match counts canonically equivalent spellings as different; the NFC column does not.
• The round trip uses Rabbit to make the Zawgyi input, so Rabbit is not scored on it.
• The Unicode sets contain a few lines of real Zawgyi text, so their labels are slightly noisy.
• The Wikipedia and GlotCC rows come from the current revision of those datasets; the Wikipedia rows are 25 random blocks picked with a fixed seed.
• Timings are from one machine. Compare ratios, not absolute times.

Sources and licenses

The data is downloaded when the benchmark runs and is not copied into this repository. Every download must match a pinned sha256; GitHub files are also pinned to a commit, mC4 to a revision and Okell to a Zenodo record. Only aggregate numbers are published here.

Reproduce

npm run bench:page

That runs npm run eval and npm run bench -- --sweep and rebuilds this page. Raw results: benchmark.json. Method: scripts/eval/README.md.