bijoy-unicode

Convert Bangla text typed in Bijoy (the SutonnyMJ family of fonts) to Unicode, and Unicode back to Bijoy.

This is the same conversion engine that runs the Bijoy to Unicode tool on KovaPDF. It is plain code with no dependencies: it works in Node.js, in the browser and in Python, offline, and nothing is sent anywhere.

Need it for whole PDF or Word files? Use KovaPDF: https://www.kovapdf.com/bijoy-to-unicode

Built by KovaPDF, https://www.kovapdf.com

npm	@kovapdf/bijoy-unicode (ESM, CommonJS and TypeScript types, zero runtime dependencies)
PyPI	bijoy-unicode (pure Python, no dependencies, Python 3.10+)
CLI	bijoy-unicode to-unicode < in.txt
Directions	Bijoy to Unicode, Unicode to Bijoy

Install

npm install @kovapdf/bijoy-unicode
pip install bijoy-unicode

JavaScript

import { toUnicode, toBijoy } from '@kovapdf/bijoy-unicode';
// or: const { toUnicode, toBijoy } = require('@kovapdf/bijoy-unicode');

toUnicode('Avgvi †mvbvi evsjv');        // 'আমার সোনার বাংলা'
toBijoy('আমার সোনার বাংলা');            // 'Avgvi †mvbvi evsjv'

// Leave English words alone when the Bijoy text mixes in English
toUnicode('GB file wU download Kiæb', { keepEnglish: true }); // 'এই file টি download করুন'

// Older Bijoy 2000 codes instead of the modern (Bijoy 52) ones
toBijoy('শিক্ষা', { flavour: 'classic' }); // 'wk¶v'  (modern: 'wkÿv')

More functions:

import { convertWithReport, detectEncoding, isBijoyFont } from '@kovapdf/bijoy-unicode';

convertWithReport('Avwg 😀 €', 'bijoy-to-unicode');
// { text: 'আমি 😀 €', keptEnglish: [], unknown: ['😀', '€'], stats: { chars: 8, bangla: 3 } }

detectEncoding('evsjv‡`k miKvi').kind; // 'bijoy'  ('unicode', 'english', 'mixed' or 'empty' otherwise)
isBijoyFont('ABCDEF+SutonnyMJ-Bold');  // true

Function	What it does
toUnicode(text, options?) (also bijoyToUnicode)	Bijoy to Unicode. Options: keepEnglish (default false), nukta ('nfc' default, or 'precomposed' for ড় ঢ় য় as one code point), visargaColon (a lone t becomes :).
toBijoy(text, options?) (also unicodeToBijoy)	Unicode to Bijoy. Option: flavour ('modern' default, or 'classic'). Text that is not Bangla is left as it is.
convertWithReport(text, direction, options?)	Either direction ('bijoy-to-unicode' or 'unicode-to-bijoy') plus a report: words kept as English and characters that could not be converted.
detectEncoding(text)	Guesses whether text is Bijoy, Unicode Bangla, English or mixed, with a confidence.
isBijoyFont(name)	True for SutonnyMJ and the other "MJ" Bijoy fonts (PDF subset prefixes and style suffixes are ignored).
BIJOY_FONT_NAMES	Known Bijoy font names.

Python

from bijoy_unicode import to_unicode, to_bijoy, convert_with_report, detect_encoding, is_bijoy_font

to_unicode("Avgvi †mvbvi evsjv")                  # 'আমার সোনার বাংলা'
to_bijoy("আমার সোনার বাংলা")                      # 'Avgvi †mvbvi evsjv'
to_unicode("GB file wU download Kiæb", keep_english=True)
to_bijoy("শিক্ষা", flavour="classic")
detect_encoding("evsjv‡`k miKvi")["kind"]          # 'bijoy'

The Python functions take the same options as keyword arguments in snake case (keep_english, nukta, visarga_colon, flavour), and convert_with_report / detect_encoding return dicts with the same keys as the JavaScript objects.

Command line

npx @kovapdf/bijoy-unicode to-unicode < bijoy.txt > unicode.txt
bijoy-unicode to-bijoy unicode.txt > bijoy.txt
bijoy-unicode to-unicode --keep-english notes.txt
python -m bijoy_unicode to-unicode < bijoy.txt     # the Python package has the same CLI

Input is UTF-8 from a file or stdin; the result goes to stdout. Options: --keep-english, --precomposed (to-unicode), --classic (to-bijoy).

SutonnyMJ, Nikosh and why text looks like "Avgvi"

• SutonnyMJ is a Bijoy font. Bijoy does not store Bangla letters. It stores ordinary Latin codes (A, v, g, †, ¨ ...) and the SutonnyMJ font draws Bangla shapes on top of them. That is why Bijoy text turns into "Avgvi †mvbvi evsjv" when the font is missing, when it is copied into a web form, or when a search engine reads it. SutonnyOMJ, AnandaMJ and the other fonts whose names end in "MJ" work the same way.
• Nikosh is a Unicode font. Nikosh (like Kalpurush, SolaimanLipi or Noto Sans Bengali) shows real Unicode Bangla, which every modern phone, browser and search engine understands. Text converted with this library should be set in a Unicode font such as Nikosh, not in SutonnyMJ.
• To go the other way (for an old system that only knows SutonnyMJ), convert Unicode to Bijoy and set the result in SutonnyMJ.

This package contains no fonts. Install the fonts you need from their own publishers.

How accurate is it?

The engine is the one KovaPDF uses on its website. It handles conjuncts (যুক্তাক্ষর), the reordering of ি ে ৈ, ref (র্), ya-phala and ra-phala, the two Bijoy code generations (Bijoy 2000 and Bijoy 52), text copied out of PDFs, and English words mixed into Bijoy text. test/vectors.json holds over 600 input and output pairs made by running the website engine on real words, sentences, conjuncts, numbers and mixed English; the JavaScript and the Python tests both check every one.

Keeping English words in Bijoy text is a guess, because plain Bijoy letters are themselves ASCII letters. It is off by default, and convertWithReport lists every word it kept.

Development

npm install && npm test             # builds dist/ and runs the tests
cd python && pip install -e . pytest && pytest

The engine in src/engine and the Python tables in python/src/bijoy_unicode/_tables.py are generated from the KovaPDF website engine by a sync script; please report conversion problems as issues rather than editing those files.

Licence

The code is MIT licensed (see LICENSE). The KovaPDF name, logo and website content are not covered by that licence, and no fonts are included.