Show HN: Mojibake – a low-level Unicode library written in C

# Mojibake — Unicode 17 for C Source: [https://mojibake.zaerl.com/](https://mojibake.zaerl.com/) Unicode 17 \- C11 \- zero dependencies ## Unicode text processing, *without the baggage\.* Mojibake is a small, fast, self\-contained Unicode library for C11 and C\+\+17\. Ship standards\-compliant text handling without a runtime or dependency tree\. StandardUnicode 17\.0RuntimeNoneLicenseMIT Start here ## A practical Unicode toolkit **Mojibake**is a low\-level Unicode 17 text\-processing library written in C11 and compatible with C\+\+17\. It is released under the MIT License\. ## Usage You don't need to install anything\. There are two files \(`mojibake\.c`,`mojibake\.h`\) to add to your C/C\+\+ project\. Download it here[mojibake\-amalgamation\-027\.zip](https://github.com/zaerl/mojibake/releases/download/v0.2.7/mojibake-amalgamation-027.zip) Examples of normalization, characters count and NFKC casefold\. ``` #include <stdio.h> #include <string.h> #include "mojibake.h" void print_string(const char *input, size_t length); int main(int argc, char *const argv[]) { const char *input = "Cafe\xCC\x81"; size_t length = strlen(input); mjb_result result; // Normalize example: in NFC e + ◌́ -> é (U+00E9) if(mjb_normalize(input, length, MJB_ENC_UTF_8, MJB_NORMALIZATION_NFC, MJB_ENC_UTF_8, &result) != MJB_STATUS_OK) { return 1; } // Cafe + ◌́ (U+0301, COMBINING ACUTE ACCENT) -> Café print_string(input, length); // Caf + é (U+00E9, LATIN SMALL LETTER E WITH ACUTE) -> Café print_string(result.output, result.output_size); const char *mojibake = "文字化け"; length = strlen(mojibake); // String length example: mjb_string_length counts the number of characters in a // string, not the number of bytes. printf("\"%s\" encoded in UTF-8 is %zu bytes long, and %zu characters long\n", mojibake, length, mjb_string_length(mojibake, length, MJB_ENC_UTF_8)); mjb_result_free(&result); const char *case_input = "Straße"; // NFKC casefold example: in NFKC casefold, ß -> ss if(mjb_nfkc_casefold(case_input, strlen(case_input), MJB_ENC_UTF_8, MJB_ENC_UTF_8, &result) != MJB_STATUS_OK) { return 1; } printf("%s -> %.*s\n", case_input, (int)result.output_size, result.output); mjb_result_free(&result); return 0; } void print_string(const char *input, size_t length) { for(size_t i = 0; i < length; ++i) { unsigned char byte = (unsigned char)input[i]; if(byte >= 0x21 && byte <= 0x7E) { printf("%c", byte); } else { printf("<%02X>", byte); } } printf("\n"); } ``` This output: ``` Cafe<CC><81> Caf<C3><A9> "文字化け" encoded in UTF-8 is 12 bytes long, and 4 characters long Straße -> strasse ``` Mojibake aims to be: 1. Small 2. Easy to use 3. Fast 4. Self\-contained Mojibake do: 1. Run in all modern OSes \(Linux, macOS, FreeBSD, OpenBSD, NetBSD, Windows 10/11\) 2. Pass the official Unicode test suites for supported algorithms 3. Implement all Unicode standard algorithms 4. Satisfy all[Unicode Conformance Requirements](https://github.com/zaerl/mojibake/blob/main/CONFORMANCE_REQUIREMENTS.md) ## Feature highlights All the C files, together with the Unicode data tables, are concatenated into a single large file and header:`mojibake\.c`and`mojibake\.h`\. Zero dependencies\. **Text transformation** - **Normalization**: NFC/NFD/NFKC/NFKD \(`mjb\_normalize`\), identifier\-oriented NFKC case folding \(`mjb\_nfkc\_casefold`\), plus a fast quick\-check \(`mjb\_string\_is\_normalized`\) \([UAX \#15, Unicode 17\.0\.0](https://www.unicode.org/reports/tr15/tr15-57.html)\) - **Case conversion**: uppercase, lowercase, titlecase, and case folding with full special\-casing and conditional mappings \(`mjb\_case`\) - **Filtering**: strip controls, spaces, or numeric characters while normalizing \(`mjb\_string\_filter`\) **Text analysis** - **Character database**: every Unicode Character Database property: category, script and Script\_Extensions, block, plane, numeric value, name \(`mjb\_codepoint\_character`,`mjb\_codepoint\_script\_extensions`\) - **Segmentation**: grapheme clusters, words, sentences, and line\-break opportunities \([UAX \#29, Unicode 17\.0\.0](https://www.unicode.org/reports/tr29/tr29-47.html),[UAX \#14, Unicode 17\.0\.0](https://www.unicode.org/reports/tr14/tr14-55.html)\) - **Bidirectional text**: full Unicode Bidirectional Algorithm: paragraph resolution, line reordering, runs \([UAX \#9, Unicode 17\.0\.0](https://www.unicode.org/reports/tr9/tr9-51.html)\) - **Emoji**: codepoint properties, sequence analysis, RGI emoji detection - **Display width**: East Asian width and terminal display width, with width\-aware truncation \(`mjb\_display\_width`,`mjb\_truncate\_width`\) **Sorting and comparison** - **Collation**: Unicode Collation Algorithm string comparison and sort keys, in shifted and non\-ignorable modes \(`mjb\_string\_compare`,`mjb\_collation\_key`,[UTS \#10, Unicode 17\.0\.0](https://www.unicode.org/reports/tr10/tr10-53.html)\) **Security** - **Confusable detection**: generate reusable skeletons and check if strings are visually confusable \(`mjb\_confusable\_skeleton`,`mjb\_string\_is\_confusable`,[UTS \#39, Unicode 17\.0\.0](https://www.unicode.org/reports/tr39/tr39-32.html)\) - **Identifier validation**: XID/ID checks for parser and compiler authors \(`mjb\_string\_is\_identifier`,[UAX \#31, Unicode 17\.0\.0](https://www.unicode.org/reports/tr31/tr31-43.html)\) **Integration** - **Encodings**: the API accepts and outputs UTF\-8, UTF\-16LE, UTF\-16BE, UTF\-32LE, UTF\-32BE strings, with encoding detection and conversion \(`mjb\_string\_encoding`,`mjb\_string\_convert\_encoding`\) - **Parsing and string functions**: character\-by\-character iteration \(`mjb\_next\_character`\) and standard C`string\.h`\-style helpers \(`mjb\_string\_length`, and others\) - **Locales**: strict BCP 47 language tag parsing \(`mjb\_locale\_parse`\) - **Embeddable**: custom allocators \(`mjb\_set\_memory\_functions`\), build\-time feature flags to trim table size, a C\+\+17 wrapper \(`src/cpp/mojibake\.hpp`\), a CLI tool \(`src/shell`\), and a WASM \+ TypeScript API \(`src/api`\) - **Tested**: Mojibake uses[Attractor](https://github.com/zaerl/attractor/)as test suite and run[1\.5M\+ assertions](https://github.com/zaerl/mojibake/blob/main/TESTS.md)including the official Unicode conformance suites for supported algorithms - **Fuzz**Mojibake is fuzzed with[libFuzzer](https://llvm.org/docs/LibFuzzer.html)over untrusted byte input - `AddressSanitizer`and`UBSan`clean ### Build\-time features Mojibake can compile out optional feature tables to reduce binary size\. Feature macros default to enabled\. - `\#define MJB\_FEATURE\_CHARACTER\_NAMES`controls the Unicode character\-name tables used by`mjb\_codepoint\_character\(\.\.\.\)`to fill`mjb\_character\.name`\. When disabled, the tables are not compiled and`mjb\_character\.name`is reported as`Codepoint U\+XXXX`\. This will redude the output of**~30%**\. With CMake: ``` cmake -S . -B build-no-name -DMJB_FEATURE_CHARACTER_NAMES=OFF cmake --build build-no-name ``` With the provided Makefile: ``` make build BUILD_DIR=build-no-name FEATURE_CHARACTER_NAMES=OFF make test-no-names ``` ### API documentation See[API\.md](https://github.com/zaerl/mojibake/blob/main/API.md)or the site for the detailed documentation\. ### CLI The`src/shell`directory builds the`mojibake`CLI used to test the library\. Example usage: ``` # This outputs "NFC: Café", e + ◌́ -> é mojibake nfc $'Cafe\u0301' # The output an emoji sequence [1] Basic, [2] Fully-qualified of two characters U+263A U+FE0F mojibake emoji "☺️" ``` ## Building from source and contributing See[CONTRIBUTING\.md](https://github.com/zaerl/mojibake/blob/main/CONTRIBUTING.md)for instructions\. ## Licenses Mojibake is released under the MIT License \(see[LICENSE](https://github.com/zaerl/mojibake/blob/main/LICENSE)\)\. ## Legalese Here you can find the very detailed and boring informations needed to have this library conformant to the Unicode standard, or at least what I got, at[CONFORMANCE\_REQUIREMENTS\.md](https://github.com/zaerl/mojibake/blob/main/CONFORMANCE_REQUIREMENTS.md) ## Thanks Mojibake is built using the work of extraordinary individuals and teams\. 1. Unicode Character Database \- Copyright © 1991\-2026 Unicode, Inc\. \(see[license\.txt](https://www.unicode.org/license.txt)\) 2. Unicode CLDR Project \- Copyright © 2004\-2026 Unicode, Inc\. \(see[LICENSE](https://raw.githubusercontent.com/unicode-org/cldr/refs/heads/main/LICENSE)\) No installation required ## Try every function in your browser Each API reference below includes a live form backed by the WASM build\. Expand a function, enter its arguments, and inspect the result immediately\. [Download mojibake\-wasm\-027\.zip](https://github.com/zaerl/mojibake/releases/download/v0.2.7/mojibake-wasm-027.zip) Reference ## C API Functions are organized by their metadata section\. Select the plus button to see detailed behavior, examples, specifications, and the live WASM form\. ### [mjb\_normalize](https://mojibake.zaerl.com/#mjb_normalize) ``` mjb_status mjb_normalize( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_normalization form, mjb_encoding output_encoding, mjb_result *result ); ``` Normalize a string to the requested Unicode normalization form\. If the input is already normalized and no encoding conversion is needed, the input buffer is returned as\-is in`result\-\>output`with`result\-\>transformed`set to false, without allocating\. ### Returns - `MJB\_STATUS\_OK`— The string was normalized \(or already normal\) - `MJB\_STATUS\_INVALID\_ARGUMENT`—`result`is NULL, or`buffer`is NULL with a non\-zero size - `MJB\_STATUS\_INVALID\_FORM`—`form`is not NFC, NFD, NFKC, or NFKD - `MJB\_STATUS\_OVERFLOW`— The output size would overflow - `MJB\_STATUS\_NO\_MEMORY`— Allocation failed ### Example ``` const char *input = "Cafe\xCC\x81"; // "Cafe" + U+0301 COMBINING ACUTE ACCENT mjb_result result; if(mjb_normalize(input, strlen(input), MJB_ENC_UTF_8, MJB_NORMALIZATION_NFC, MJB_ENC_UTF_8, &result) != MJB_STATUS_OK) { return 1; } // NFC: Café printf("NFC: %.*s", (int)result.output_size, result.output); if(result.transformed) { mjb_free(result.output); } ``` ### Related - [`mjb\_string\_is\_normalized`](https://mojibake.zaerl.com/#mjb_string_is_normalized) - [`mjb\_string\_filter`](https://mojibake.zaerl.com/#mjb_string_filter) ### Specifications - [UAX \#15: Unicode Normalization Forms, Unicode 17\.0\.0](https://www.unicode.org/reports/tr15/tr15-57.html) ### [mjb\_string\_filter](https://mojibake.zaerl.com/#mjb_string_filter) ``` mjb_status mjb_string_filter( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_filter filters, mjb_encoding output_encoding, mjb_result *result ); ``` `MJB\_FILTER\_LIMIT\_COMBINING`removes combining marks after the first`MJB\_FILTER\_MAX\_COMBINING\_MARKS`consecutive marks in an emitted run\. This is useful for reducing Zalgo\-style text while keeping ordinary accents and stacked marks\. ### Example ``` const char *mixed_whitespace = "Hello\t\t\n\nworld"; mjb_result result; if(mjb_string_filter(mixed_whitespace, strlen(mixed_whitespace), MJB_ENC_UTF_8, MJB_FILTER_COLLAPSE_SPACES, MJB_ENC_UTF_8, &result) != MJB_STATUS_OK) { return 1; } // Filtered: Hello world printf("Filtered: %.*s", (int)result.output_size, result.output); if(result.transformed) { mjb_free(result.output); } const char *controls = "\x1\x2\t\n\v\f\r\x1f"; if(mjb_string_filter(controls, strlen(controls), MJB_ENC_UTF_8, MJB_FILTER_CONTROLS, MJB_ENC_UTF_8, &result) != MJB_STATUS_OK) { return 1; } // Filtered: \t\n\v\f\r printf("Filtered: %.*s", (int)result.output_size, result.output); if(result.transformed) { mjb_free(result.output); } ``` ### Related - [`mjb\_normalize`](https://mojibake.zaerl.com/#mjb_normalize) ### [mjb\_nfkc\_casefold](https://mojibake.zaerl.com/#mjb_nfkc_casefold) ``` mjb_status mjb_nfkc_casefold( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_encoding output_encoding, mjb_result *result ); ``` Apply the normative`NFKC\_Casefold`mapping and normalize the result to NFC\. This transform performs compatibility folding, full default case folding, and removal of default\-ignorable codepoints\. It is intended for identifier comparison and is not locale\-sensitive\. ### Returns - `MJB\_STATUS\_OK`— The transformed string was returned - `MJB\_STATUS\_INVALID\_ARGUMENT`—`result`is NULL, or`buffer`is NULL with a non\-zero size - `MJB\_STATUS\_OVERFLOW`— The output size would overflow - `MJB\_STATUS\_NO\_MEMORY`— Allocation failed ### Example ``` const char *input = "Stra\xC3\x9F" "e\xC2\xAD"; mjb_result result; if(mjb_nfkc_casefold(input, strlen(input), MJB_ENC_UTF_8, MJB_ENC_UTF_8, &result) != MJB_STATUS_OK) { return 1; } // strasse printf("%.*s", (int)result.output_size, result.output); mjb_result_free(&result); ``` ### Related - [`mjb\_normalize`](https://mojibake.zaerl.com/#mjb_normalize) - [`mjb\_case`](https://mojibake.zaerl.com/#mjb_case) - [`mjb\_string\_is\_identifier`](https://mojibake.zaerl.com/#mjb_string_is_identifier) ### Specifications - [The Unicode Standard, Version 17\.0\.0, Section 3\.13: Default Case Algorithms](https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G33992) - [UAX \#31: Unicode Identifiers and Syntax, Unicode 17\.0\.0](https://www.unicode.org/reports/tr31/tr31-43.html) - [UAX \#44: Unicode Character Database, Unicode 17\.0\.0](https://www.unicode.org/reports/tr44/tr44-36.html) ### [mjb\_codepoint\_encode](https://mojibake.zaerl.com/#mjb_codepoint_encode) ``` unsigned int mjb_codepoint_encode( mjb_codepoint codepoint, char *buffer, size_t byte_length, mjb_encoding encoding ); ``` ### Example ``` char encoded[4]; unsigned int size = mjb_codepoint_encode(0x20AC, encoded, sizeof(encoded), MJB_ENC_UTF_8); // € sign uses 3 UTF-8 bytes printf("%.*s sign uses %u UTF-8 bytes", (int)size, encoded, size); ``` ### [mjb\_string\_convert\_encoding](https://mojibake.zaerl.com/#mjb_string_convert_encoding) ``` mjb_status mjb_string_convert_encoding( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_encoding output_encoding, mjb_result *result ); ``` Convert a string between the supported encodings \(UTF\-8, UTF\-16LE/BE, UTF\-32LE/BE\)\. Generic UTF\-16/UTF\-32 input consumes a leading BOM as the encoding scheme signature and uses it to resolve byte order\. Explicit\-endian input preserves an initial U\+FEFF as text\. Generic UTF\-16/UTF\-32 without a BOM, and generic UTF\-16/UTF\-32 output, are rejected because the byte order is not specified\. ### Returns - `MJB\_STATUS\_OK`— The string was converted - `MJB\_STATUS\_INVALID\_ARGUMENT`—`result`is NULL,`buffer`is NULL with a non\-zero size, or the input is not valid in the source encoding - `MJB\_STATUS\_INVALID\_ENCODING`— A generic UTF\-16/UTF\-32 encoding did not provide enough byte order information - `MJB\_STATUS\_UNSUPPORTED`— The requested encoding conversion is not supported - `MJB\_STATUS\_OVERFLOW`— The output size would overflow - `MJB\_STATUS\_NO\_MEMORY`— Allocation failed ### Example ``` const char *input = "caf\xC3\xA9"; mjb_result result; if(mjb_string_convert_encoding(input, strlen(input), MJB_ENC_UTF_8, MJB_ENC_UTF_16LE, &result) != MJB_STATUS_OK) { return 1; } // UTF-16LE bytes: 8 printf("UTF-16LE bytes: %zu", result.output_size); mjb_result_free(&result); ``` ### Related - [`mjb\_string\_encoding`](https://mojibake.zaerl.com/#mjb_string_encoding) - [`mjb\_codepoint\_encode`](https://mojibake.zaerl.com/#mjb_codepoint_encode) ### [mjb\_case](https://mojibake.zaerl.com/#mjb_case) ``` mjb_status mjb_case( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_case_type type, mjb_encoding output_encoding, mjb_result *result ); ``` Convert a string to uppercase, lowercase, titlecase, or its case\-folded form\. Full case mappings are applied, including special casing and conditional mappings, so the output may have a different length than the input\. Titlecase uses UAX \#29 word boundaries: the first cased character in each word segment is titlecased, and subsequent characters in that segment are lowercased\. Casing is tailored by the process\-global locale set with`mjb\_locale\_set`: the default`MJB\_LOCALE\_EN`uses default non\-Turkic mappings\.`MJB\_LOCALE\_TR`and`MJB\_LOCALE\_AZ`apply Turkish/Azerbaijani dotted\-I casing and Turkic`T`case\-folding mappings\.`MJB\_LOCALE\_LT`applies Lithuanian dot\-above casing rules, while case folding remains the default non\-Turkic mapping\. ### Returns - `MJB\_STATUS\_OK`— The case conversion succeeded - `MJB\_STATUS\_INVALID\_ARGUMENT`—`result`is NULL,`buffer`is NULL with a non\-zero size, or`type`is not a valid case type - `MJB\_STATUS\_NO\_MEMORY`— Allocation failed ### Example ``` const char *input = "Stra\xC3\x9F""e"; // "Straße" mjb_result result; if(mjb_case(input, strlen(input), MJB_ENC_UTF_8, MJB_CASE_UPPER, MJB_ENC_UTF_8, &result) != MJB_STATUS_OK) { return 1; } // Upper: STRASSE printf("Upper: %.*s", (int)result.output_size, result.output); if(result.transformed) { mjb_free(result.output); } ``` ### Related - [`mjb\_locale\_set`](https://mojibake.zaerl.com/#mjb_locale_set) - [`mjb\_codepoint\_to\_uppercase`](https://mojibake.zaerl.com/#mjb_codepoint_to_uppercase) - [`mjb\_codepoint\_to\_lowercase`](https://mojibake.zaerl.com/#mjb_codepoint_to_lowercase) - [`mjb\_codepoint\_to\_titlecase`](https://mojibake.zaerl.com/#mjb_codepoint_to_titlecase) ### Specifications - [The Unicode Standard, Version 17\.0\.0, Section 3\.13: Default Case Algorithms](https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G33992) ### [mjb\_codepoint\_to\_lowercase](https://mojibake.zaerl.com/#mjb_codepoint_to_lowercase) ``` mjb_codepoint mjb_codepoint_to_lowercase( mjb_codepoint codepoint ); ``` Return the lowercase codepoint of a codepoint\. If the codepoint has no lowercase equivalent, the original codepoint is returned\. ### Returns - `codepoint`— The lowercase codepoint, or the original codepoint ### Example ``` mjb_codepoint codepoint; codepoint = mjb_codepoint_to_lowercase(0x0041); // U+0041 = 'A' // A > a printf("%c > %c", 'A', codepoint); codepoint = mjb_codepoint_to_lowercase(0x03A3); // U+03A3 = 'Σ' // U+03A3 > U+03C3, Σ > σ printf("U+%04X > U+%04X, %s > %s", 0x03A3, codepoint, "Σ", "σ"); ``` ### Related - [`mjb\_codepoint\_to\_uppercase`](https://mojibake.zaerl.com/#mjb_codepoint_to_uppercase) - [`mjb\_codepoint\_to\_titlecase`](https://mojibake.zaerl.com/#mjb_codepoint_to_titlecase) ### [mjb\_codepoint\_to\_uppercase](https://mojibake.zaerl.com/#mjb_codepoint_to_uppercase) ``` mjb_codepoint mjb_codepoint_to_uppercase( mjb_codepoint codepoint ); ``` Return the uppercase codepoint of a codepoint\. If the codepoint has no uppercase equivalent, the original codepoint is returned\. ### Returns - `codepoint`— The uppercase codepoint, or the original codepoint ### Example ``` mjb_codepoint uppercase = mjb_codepoint_to_uppercase(0x00E9); // é // Uppercase: U+00C9 printf("Uppercase: U+%04X", uppercase); ``` ### Related - [`mjb\_codepoint\_to\_lowercase`](https://mojibake.zaerl.com/#mjb_codepoint_to_lowercase) - [`mjb\_codepoint\_to\_titlecase`](https://mojibake.zaerl.com/#mjb_codepoint_to_titlecase) ### [mjb\_codepoint\_to\_titlecase](https://mojibake.zaerl.com/#mjb_codepoint_to_titlecase) ``` mjb_codepoint mjb_codepoint_to_titlecase( mjb_codepoint codepoint ); ``` Return the titlecase codepoint of a codepoint\. If the codepoint has no titlecase equivalent, the original codepoint is returned\. ### Returns - `codepoint`— The titlecase codepoint, or the original codepoint ### Example ``` mjb_codepoint titlecase = mjb_codepoint_to_titlecase(0x01F3); // dz // Titlecase: U+01F2 printf("Titlecase: U+%04X", titlecase); ``` ### Related - [`mjb\_codepoint\_to\_lowercase`](https://mojibake.zaerl.com/#mjb_codepoint_to_lowercase) - [`mjb\_codepoint\_to\_uppercase`](https://mojibake.zaerl.com/#mjb_codepoint_to_uppercase) ### [mjb\_codepoint\_character](https://mojibake.zaerl.com/#mjb_codepoint_character) ``` mjb_status mjb_codepoint_character( mjb_codepoint codepoint, mjb_character *character ); ``` Fill`character`with the Unicode Character Database record of a codepoint: name, category, combining class, bidirectional category, decomposition, numeric values, mirrored flag, and simple case mappings\. When the library is compiled with`MJB\_FEATURE\_CHARACTER\_NAMES=OFF`the name field is reported as`Codepoint U\+XXXX`\. ### Returns - `MJB\_STATUS\_OK`— The character was found and filled - `MJB\_STATUS\_INVALID\_ARGUMENT`—`character`is NULL or the codepoint is not valid - `MJB\_STATUS\_NOT\_FOUND`— The codepoint is not assigned ### Example ``` mjb_character character; if(mjb_codepoint_character(0x022A, &character) != MJB_STATUS_OK) { return 1; } // U+022A lowercase: U+022B printf("U+%04X lowercase: U+%04X", character.codepoint, character.lowercase); ``` ### Related - [`mjb\_codepoint\_block`](https://mojibake.zaerl.com/#mjb_codepoint_block) - [`mjb\_codepoint\_script`](https://mojibake.zaerl.com/#mjb_codepoint_script) - [`mjb\_codepoint\_property\_binary`](https://mojibake.zaerl.com/#mjb_codepoint_property_binary) - [`mjb\_codepoint\_property\_int`](https://mojibake.zaerl.com/#mjb_codepoint_property_int) ### Specifications - [UAX \#44: Unicode Character Database, Unicode 17\.0\.0](https://www.unicode.org/reports/tr44/tr44-36.html) ### [mjb\_string\_is\_normalized](https://mojibake.zaerl.com/#mjb_string_is_normalized) ``` mjb_quick_check_result mjb_string_is_normalized( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_normalization form ); ``` Run the normalization quick\-check on a string without allocating\.`MJB\_QC\_MAYBE`means the string may still be normalized, and only a full normalization pass with`mjb\_normalize`can decide\. ### Returns - `MJB\_QC\_YES`— The string is normalized to the requested form - `MJB\_QC\_NO`— The string is not normalized - `MJB\_QC\_MAYBE`— Inconclusive: a full normalization is needed to decide ### Example ``` const char *input = "caf\xC3\xA9"; mjb_quick_check_result check = mjb_string_is_normalized(input, strlen(input), MJB_ENC_UTF_8, MJB_NORMALIZATION_NFC); // NFC normalized: yes printf("NFC normalized: %s", check == MJB_QC_YES ? "yes" : "no"); ``` ### Related - [`mjb\_normalize`](https://mojibake.zaerl.com/#mjb_normalize) ### Specifications - [UAX \#15: Unicode Normalization Forms, Unicode 17\.0\.0](https://www.unicode.org/reports/tr15/tr15-57.html) ### [mjb\_string\_encoding](https://mojibake.zaerl.com/#mjb_string_encoding) ``` mjb_encoding mjb_string_encoding( const char *buffer, size_t byte_length ); ``` `mjb\_string\_encoding`reports BOM\-derived UTF\-16/UTF\-32 schemes with the generic family bit plus the resolved endian bit\. Passing that detected value consumes the leading BOM as a signature\. Passing an explicit\-endian encoding such as`MJB\_ENC\_UTF\_16BE`preserves an initial U\+FEFF as text\. When flags overlap, as with a UTF\-32LE BOM that also has the UTF\-16LE BOM prefix, decoding gives UTF\-32 precedence\. ### Example ``` const char utf16le[] = "\xFF\xFEH\0i\0"; mjb_encoding detected = mjb_string_encoding(utf16le, sizeof(utf16le) - 1); bool is_utf16le = detected == (MJB_ENC_UTF_16 | MJB_ENC_UTF_16LE); // UTF-16LE detected: yes printf("UTF-16LE detected: %s", is_utf16le ? "yes" : "no"); ``` ### [mjb\_string\_is\_ascii](https://mojibake.zaerl.com/#mjb_string_is_ascii) ``` bool mjb_string_is_ascii( const char *buffer, size_t byte_length ); ``` ### Example ``` const char *input = "Plain ASCII"; // ASCII: yes printf("ASCII: %s", mjb_string_is_ascii(input, strlen(input)) ? "yes" : "no"); ``` ### [mjb\_string\_is\_utf8](https://mojibake.zaerl.com/#mjb_string_is_utf8) ``` bool mjb_string_is_utf8( const char *buffer, size_t byte_length ); ``` ### Example ``` const char *input = "caf\xC3\xA9"; // Valid UTF-8: yes printf("Valid UTF-8: %s", mjb_string_is_utf8(input, strlen(input)) ? "yes" : "no"); ``` ### [mjb\_string\_is\_utf16](https://mojibake.zaerl.com/#mjb_string_is_utf16) ``` bool mjb_string_is_utf16( const char *buffer, size_t byte_length ); ``` ### Example ``` const char utf16be[] = "\xFE\xFF\0H\0i"; // BOM + "Hi" in UTF-16BE // UTF-16: yes printf("UTF-16: %s", mjb_string_is_utf16(utf16be, sizeof(utf16be) - 1) ? "yes" : "no"); ``` ### [mjb\_string\_length](https://mojibake.zaerl.com/#mjb_string_length) ``` size_t mjb_string_length( const char *buffer, size_t max_length, mjb_encoding encoding ); ``` Return the number of Unicode codepoints in a string, up to`max\_length`bytes\. ### Example ``` // The "Héllö" string is five Unicode characters, but has different byte lengths in different encodings. const char *utf8 = "H\xC3\xA9ll\xC3\xB6"; // 7 bytes const char utf16le[] = "H\0\xE9\0l\0l\0\xF6\0"; // 10 bytes const char utf16be[] = "\0H\0\xE9\0l\0l\0\xF6"; // 10 bytes const char utf32le[] = "H\0\0\0\xE9\0\0\0l\0\0\0l\0\0\0\xF6\0\0\0"; // 20 bytes const char utf32be[] = "\0\0\0H\0\0\0\xE9\0\0\0l\0\0\0l\0\0\0\xF6"; // 20 bytes // 5 UTF-8 characters printf("%zu UTF-8 characters", mjb_string_length(utf8, 7, MJB_ENC_UTF_8)); // 5 UTF-16LE characters printf("%zu UTF-16LE characters", mjb_string_length(utf16le, 10, MJB_ENC_UTF_16LE)); // 5 UTF-16BE characters printf("%zu UTF-16BE characters", mjb_string_length(utf16be, 10, MJB_ENC_UTF_16BE)); // 5 UTF-32LE characters printf("%zu UTF-32LE characters", mjb_string_length(utf32le, 20, MJB_ENC_UTF_32LE)); // 5 UTF-32BE characters printf("%zu UTF-32BE characters", mjb_string_length(utf32be, 20, MJB_ENC_UTF_32BE)); ``` ### [mjb\_string\_each\_character](https://mojibake.zaerl.com/#mjb_string_each_character) ``` mjb_status mjb_string_each_character( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_string_each_character_fn callback ); ``` ### Example ``` mjb_status status = mjb_string_each_character("ABC", 3, MJB_ENC_UTF_8, NULL); // A callback is required: yes bool callback_required = status == MJB_STATUS_INVALID_ARGUMENT; // A callback is required: yes printf("A callback is required: %s", callback_required ? "yes" : "no"); ``` ### [mjb\_codepoint\_property\_binary](https://mojibake.zaerl.com/#mjb_codepoint_property_binary) ``` mjb_status mjb_codepoint_property_binary( mjb_codepoint codepoint, mjb_property property, bool *value ); ``` Return`true`when the codepoint has the binary property and`false`when it does not\. Passing an enumerated property is a type mismatch and returns`MJB\_STATUS\_INVALID\_ARGUMENT`\. ### Example ``` bool is_alphabetic; if(mjb_codepoint_property_binary('A', MJB_PR_ALPHABETIC, &is_alphabetic) != MJB_STATUS_OK) { return 1; } // U+0041 is alphabetic: yes printf("U+0041 is alphabetic: %s", is_alphabetic ? "yes" : "no"); ``` ### Related - [`mjb\_codepoint\_property\_int`](https://mojibake.zaerl.com/#mjb_codepoint_property_int) ### Specifications - [UAX \#44: Unicode Character Database, Unicode 17\.0\.0](https://www.unicode.org/reports/tr44/tr44-36.html) ### [mjb\_codepoint\_property\_int](https://mojibake.zaerl.com/#mjb_codepoint_property_int) ``` mjb_status mjb_codepoint_property_int( mjb_codepoint codepoint, mjb_property property, int32_t *value ); ``` Passing a binary property is a type mismatch and returns`MJB\_STATUS\_INVALID\_ARGUMENT`\.`MJB\_STATUS\_NOT\_FOUND`means that the codepoint has no stored value for the requested property\. ### Example ``` int32_t script; if(mjb_codepoint_property_int('A', MJB_PR_SCRIPT, &script) != MJB_STATUS_OK) { return 1; } // U+0041 uses the Latin script: yes printf("U+0041 uses the Latin script: %s", script == MJB_SC_LATN ? "yes" : "no"); ``` ### Related - [`mjb\_codepoint\_property\_binary`](https://mojibake.zaerl.com/#mjb_codepoint_property_binary) ### Specifications - [UAX \#44: Unicode Character Database, Unicode 17\.0\.0](https://www.unicode.org/reports/tr44/tr44-36.html) ### [mjb\_codepoint\_numeric\_value](https://mojibake.zaerl.com/#mjb_codepoint_numeric_value) ``` mjb_status mjb_codepoint_numeric_value( mjb_codepoint codepoint, mjb_numeric_value *value ); ``` Return the numeric value of a codepoint, if any\. If the codepoint has no numeric value,`value\-\>decimal`and`value\-\>digit`are set to`MJB\_NUMBER\_NOT\_VALID`\(\-1\)\. ### Returns - `MJB\_STATUS\_OK`— The character was found and filled - `MJB\_STATUS\_INVALID\_ARGUMENT`—`value`is NULL or the codepoint is not valid ### Example ``` mjb_numeric_value num; if(mjb_codepoint_numeric_value(0x0031, &num) != MJB_STATUS_OK) { // U+0031 = 1 return 1; } // decimal=1, digit=1, numeric=1 printf("decimal=%d, digit=%d, numeric=%s", num.decimal, num.digit, num.numeric); if(mjb_codepoint_numeric_value(0x00BD, &num) != MJB_STATUS_OK) { // U+00BD = '½' return 1; } // decimal=-1, digit=-1, numeric=1/2 printf("decimal=%d, digit=%d, numeric=%s", num.decimal, num.digit, num.numeric); ``` ### Specifications - [UAX \#44: Unicode Character Database, Unicode 17\.0\.0](https://www.unicode.org/reports/tr44/tr44-36.html) ### [mjb\_codepoint\_block](https://mojibake.zaerl.com/#mjb_codepoint_block) ``` mjb_status mjb_codepoint_block( mjb_codepoint codepoint, mjb_block_info *block ); ``` ### [mjb\_codepoint\_script](https://mojibake.zaerl.com/#mjb_codepoint_script) ``` mjb_script mjb_codepoint_script( mjb_codepoint codepoint ); ``` ### [mjb\_codepoint\_script\_extensions](https://mojibake.zaerl.com/#mjb_codepoint_script_extensions) ``` mjb_status mjb_codepoint_script_extensions( mjb_codepoint codepoint, mjb_script *scripts, size_t *count ); ``` Return the explicit Script\_Extensions set, or the ordinary Script value when the codepoint has no explicit Script\_Extensions entry\. Call first with`scripts`set to NULL to obtain the required count\. ### Example ``` size_t count = 0; if(mjb_codepoint_script_extensions(0x30FC, NULL, &count) != MJB_STATUS_OK) { return 1; } mjb_script scripts[3]; if(count > 3 || mjb_codepoint_script_extensions(0x30FC, scripts, &count) != MJB_STATUS_OK) { return 1; } // U+30FC has 2 Script_Extensions printf("U+30FC has %zu Script_Extensions", count); ``` ### Related - [`mjb\_codepoint\_script`](https://mojibake.zaerl.com/#mjb_codepoint_script) ### Specifications - [UAX \#24: Unicode Script Property, Unicode 17\.0\.0](https://www.unicode.org/reports/tr24/tr24-39.html) ### [mjb\_codepoint\_is\_valid](https://mojibake.zaerl.com/#mjb_codepoint_is_valid) ``` bool mjb_codepoint_is_valid( mjb_codepoint codepoint ); ``` ### Example ``` // U+10FFFD valid: yes printf("U+10FFFD valid: %s", mjb_codepoint_is_valid(0x10FFFD) ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_graphic](https://mojibake.zaerl.com/#mjb_codepoint_is_graphic) ``` bool mjb_codepoint_is_graphic( mjb_codepoint codepoint ); ``` ### Example ``` // Letter A is graphic: yes printf("Letter A is graphic: %s", mjb_codepoint_is_graphic('A') ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_combining](https://mojibake.zaerl.com/#mjb_codepoint_is_combining) ``` bool mjb_codepoint_is_combining( mjb_codepoint codepoint ); ``` ### Example ``` // U+0301 is combining: yes printf("U+0301 is combining: %s", mjb_codepoint_is_combining(0x0301) ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_cjk\_ideograph](https://mojibake.zaerl.com/#mjb_codepoint_is_cjk_ideograph) ``` bool mjb_codepoint_is_cjk_ideograph( mjb_codepoint codepoint ); ``` ### Example ``` // U+4E00 is a CJK ideograph: yes printf("U+4E00 is a CJK ideograph: %s", mjb_codepoint_is_cjk_ideograph(0x4E00) ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_cjk\_ext](https://mojibake.zaerl.com/#mjb_codepoint_is_cjk_ext) ``` bool mjb_codepoint_is_cjk_ext( mjb_codepoint codepoint ); ``` ### Example ``` // U+20000 is a CJK extension ideograph: yes printf("U+20000 is a CJK extension ideograph: %s", mjb_codepoint_is_cjk_ext(0x20000) ? "yes" : "no"); ``` ### [mjb\_category\_is\_graphic](https://mojibake.zaerl.com/#mjb_category_is_graphic) ``` bool mjb_category_is_graphic( mjb_category category ); ``` ### Example ``` // Uppercase letters are graphic: yes bool graphic = mjb_category_is_graphic(MJB_CATEGORY_LU); // Uppercase letters are graphic: yes printf("Uppercase letters are graphic: %s", graphic ? "yes" : "no"); ``` ### [mjb\_category\_is\_combining](https://mojibake.zaerl.com/#mjb_category_is_combining) ``` bool mjb_category_is_combining( mjb_category category ); ``` ### Example ``` // Nonspacing marks are combining: yes bool combining = mjb_category_is_combining(MJB_CATEGORY_MN); // Nonspacing marks are combining: yes printf("Nonspacing marks are combining: %s", combining ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_id\_start](https://mojibake.zaerl.com/#mjb_codepoint_is_id_start) ``` bool mjb_codepoint_is_id_start( mjb_codepoint codepoint ); ``` ### Example ``` // Greek alpha starts an identifier: yes bool starts = mjb_codepoint_is_id_start(0x03B1); // Greek alpha starts an identifier: yes printf("Greek alpha starts an identifier: %s", starts ? "yes" : "no"); ``` ### Specifications - [UAX \#31: Unicode Identifiers and Syntax, Unicode 17\.0\.0](https://www.unicode.org/reports/tr31/tr31-43.html) ### [mjb\_codepoint\_is\_id\_continue](https://mojibake.zaerl.com/#mjb_codepoint_is_id_continue) ``` bool mjb_codepoint_is_id_continue( mjb_codepoint codepoint ); ``` ### Example ``` // Digit 7 continues an identifier: yes bool continues = mjb_codepoint_is_id_continue('7'); // Digit 7 continues an identifier: yes printf("Digit 7 continues an identifier: %s", continues ? "yes" : "no"); ``` ### Specifications - [UAX \#31: Unicode Identifiers and Syntax, Unicode 17\.0\.0](https://www.unicode.org/reports/tr31/tr31-43.html) ### [mjb\_codepoint\_is\_xid\_start](https://mojibake.zaerl.com/#mjb_codepoint_is_xid_start) ``` bool mjb_codepoint_is_xid_start( mjb_codepoint codepoint ); ``` ### [mjb\_codepoint\_is\_xid\_continue](https://mojibake.zaerl.com/#mjb_codepoint_is_xid_continue) ``` bool mjb_codepoint_is_xid_continue( mjb_codepoint codepoint ); ``` ### [mjb\_codepoint\_is\_pattern\_syntax](https://mojibake.zaerl.com/#mjb_codepoint_is_pattern_syntax) ``` bool mjb_codepoint_is_pattern_syntax( mjb_codepoint codepoint ); ``` ### [mjb\_codepoint\_is\_pattern\_white\_space](https://mojibake.zaerl.com/#mjb_codepoint_is_pattern_white_space) ``` bool mjb_codepoint_is_pattern_white_space( mjb_codepoint codepoint ); ``` ### Example ``` // Space is Pattern_White_Space: yes bool whitespace = mjb_codepoint_is_pattern_white_space(' '); // Space is Pattern_White_Space: yes printf("Space is Pattern_White_Space: %s", whitespace ? "yes" : "no"); ``` ### Specifications - [UAX \#31: Unicode Identifiers and Syntax, Unicode 17\.0\.0](https://www.unicode.org/reports/tr31/tr31-43.html) ### [mjb\_codepoint\_is\_extended\_pictographic](https://mojibake.zaerl.com/#mjb_codepoint_is_extended_pictographic) ``` bool mjb_codepoint_is_extended_pictographic( mjb_codepoint codepoint ); ``` ### Example ``` // Red heart is Extended_Pictographic: yes bool pictographic = mjb_codepoint_is_extended_pictographic(0x2764); // Red heart is Extended_Pictographic: yes printf("Red heart is Extended_Pictographic: %s", pictographic ? "yes" : "no"); ``` ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_codepoint\_plane](https://mojibake.zaerl.com/#mjb_codepoint_plane) ``` mjb_plane mjb_codepoint_plane( mjb_codepoint codepoint ); ``` ### Example ``` mjb_plane plane = mjb_codepoint_plane(0x1F600); // U+1F600 is in the SMP: yes printf("U+1F600 is in the SMP: %s", plane == MJB_PLANE_SMP ? "yes" : "no"); ``` ### [mjb\_plane\_is\_valid](https://mojibake.zaerl.com/#mjb_plane_is_valid) ``` bool mjb_plane_is_valid( mjb_plane plane ); ``` ### Example ``` // Plane 16 is valid: yes printf("Plane 16 is valid: %s", mjb_plane_is_valid(MJB_PLANE_PUA_B) ? "yes" : "no"); ``` ### [mjb\_plane\_name](https://mojibake.zaerl.com/#mjb_plane_name) ``` const char *mjb_plane_name( mjb_plane plane, bool abbreviation ); ``` ### Example ``` // Plane: Basic Multilingual Plane printf("Plane: %s", mjb_plane_name(MJB_PLANE_BMP, false)); ``` ### [mjb\_string\_compare](https://mojibake.zaerl.com/#mjb_string_compare) ``` int mjb_string_compare( const char *s1, size_t s1_byte_length, mjb_encoding s1_encoding, const char *s2, size_t s2_byte_length, mjb_encoding s2_encoding, mjb_collation_mode mode ); ``` Compare two strings using the Unicode Collation Algorithm and the default collation element table \(DUCET\), with`strcmp`\-style semantics\. ### Returns - `< 0`— The first string collates before the second - `0`— The strings are equal under UCA - `\> 0`— The first string collates after the second ### Example ``` int order = mjb_string_compare("apple", 5, MJB_ENC_UTF_8, "banana", 6, MJB_ENC_UTF_8, MJB_COLLATION_NON_IGNORABLE); // apple sorts before banana: yes printf("apple sorts before banana: %s", order < 0 ? "yes" : "no"); ``` ### Related - [`mjb\_collation\_key`](https://mojibake.zaerl.com/#mjb_collation_key) ### Specifications - [UTS \#10: Unicode Collation Algorithm, Unicode 17\.0\.0](https://www.unicode.org/reports/tr10/tr10-53.html) ### [mjb\_collation\_key](https://mojibake.zaerl.com/#mjb_collation_key) ``` mjb_status mjb_collation_key( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_collation_mode mode, mjb_result *result ); ``` Generate a binary sort key for a string\. Sort keys of different strings can be compared with`memcmp`and yield the same order as`mjb\_string\_compare`\. Useful when the same strings are compared many times, such as sorting or database indexing\. ### Returns - `MJB\_STATUS\_OK`— The sort key was generated - `MJB\_STATUS\_INVALID\_ARGUMENT`—`result`is NULL, or`buffer`is NULL with a non\-zero size - `MJB\_STATUS\_OVERFLOW`— The sort key size would overflow - `MJB\_STATUS\_NO\_MEMORY`— Allocation failed ### Example ``` mjb_result key; if(mjb_collation_key("r\xC3\xA9sum\xC3\xA9", 8, MJB_ENC_UTF_8, MJB_COLLATION_NON_IGNORABLE, &key) != MJB_STATUS_OK) { return 1; } // Sort key is non-empty: yes printf("Sort key is non-empty: %s", key.output_size > 0 ? "yes" : "no"); mjb_result_free(&key); ``` ### Related - [`mjb\_string\_compare`](https://mojibake.zaerl.com/#mjb_string_compare) ### Specifications - [UTS \#10: Unicode Collation Algorithm, Unicode 17\.0\.0](https://www.unicode.org/reports/tr10/tr10-53.html) ### [mjb\_string\_is\_identifier](https://mojibake.zaerl.com/#mjb_string_is_identifier) ``` bool mjb_string_is_identifier( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_identifier_profile profile ); ``` Validate a string as a Unicode identifier: the first character must be a valid identifier start and the following ones valid identifier continuations, using ID\_Start/ID\_Continue for the DEFAULT profile or XID\_Start/XID\_Continue for the NFKC profile\. ### Example ``` const char *identifier = "delta_2"; bool valid = mjb_string_is_identifier(identifier, strlen(identifier), MJB_ENC_UTF_8, MJB_IDENTIFIER_NFKC); // Valid identifier: yes printf("Valid identifier: %s", valid ? "yes" : "no"); ``` ### Related - [`mjb\_codepoint\_is\_id\_start`](https://mojibake.zaerl.com/#mjb_codepoint_is_id_start) - [`mjb\_codepoint\_is\_id\_continue`](https://mojibake.zaerl.com/#mjb_codepoint_is_id_continue) - [`mjb\_codepoint\_is\_xid\_start`](https://mojibake.zaerl.com/#mjb_codepoint_is_xid_start) - [`mjb\_codepoint\_is\_xid\_continue`](https://mojibake.zaerl.com/#mjb_codepoint_is_xid_continue) ### Specifications - [UAX \#31: Unicode Identifiers and Syntax, Unicode 17\.0\.0](https://www.unicode.org/reports/tr31/tr31-43.html) ### [mjb\_confusable\_skeleton](https://mojibake.zaerl.com/#mjb_confusable_skeleton) ``` mjb_status mjb_confusable_skeleton( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_encoding output_encoding, mjb_result *result ); ``` Compute the UTS \#39`bidiSkeleton\(LTR, input\)`: apply the Unicode Bidirectional Algorithm through L4, then NFD, remove default\-ignorables, substitute prototypes from`confusables\.txt`, and reapply NFD\. Skeletons can be stored or indexed so future confusable checks can compare them directly\. ### Returns - `MJB\_STATUS\_OK`— The confusable skeleton was returned - `MJB\_STATUS\_INVALID\_ARGUMENT`—`result`is NULL, or`buffer`is NULL with a non\-zero size - `MJB\_STATUS\_OVERFLOW`— The output size would overflow - `MJB\_STATUS\_NO\_MEMORY`— Allocation failed ### Example ``` const char *input = "h\xD0\xB5llo"; // Cyrillic U+0435 in place of e mjb_result result; if(mjb_confusable_skeleton(input, strlen(input), MJB_ENC_UTF_8, MJB_ENC_UTF_8, &result) != MJB_STATUS_OK) { return 1; } // hello printf("%.*s", (int)result.output_size, result.output); mjb_result_free(&result); ``` ### Related - [`mjb\_string\_is\_confusable`](https://mojibake.zaerl.com/#mjb_string_is_confusable) - [`mjb\_string\_is\_identifier`](https://mojibake.zaerl.com/#mjb_string_is_identifier) ### Specifications - [UTS \#39: Unicode Security Mechanisms, Unicode 17\.0\.0](https://www.unicode.org/reports/tr39/tr39-32.html) ### [mjb\_string\_is\_confusable](https://mojibake.zaerl.com/#mjb_string_is_confusable) ``` bool mjb_string_is_confusable( const char *s1, size_t s1_byte_length, mjb_encoding s1_encoding, const char *s2, size_t s2_byte_length, mjb_encoding s2_encoding ); ``` Compute the confusable skeleton of both strings and return true when the skeletons are equal, meaning the two strings are visually confusable, such as "good" and "gооd" with Cyrillic о\. ### Example ``` const char *latin = "hello"; const char *mixed = "h\xD0\xB5llo"; // Cyrillic е bool confusable = mjb_string_is_confusable(latin, strlen(latin), MJB_ENC_UTF_8, mixed, strlen(mixed), MJB_ENC_UTF_8); // Visually confusable: yes printf("Visually confusable: %s", confusable ? "yes" : "no"); ``` ### Related - [`mjb\_confusable\_skeleton`](https://mojibake.zaerl.com/#mjb_confusable_skeleton) - [`mjb\_string\_is\_identifier`](https://mojibake.zaerl.com/#mjb_string_is_identifier) ### Specifications - [UTS \#39: Unicode Security Mechanisms, Unicode 17\.0\.0](https://www.unicode.org/reports/tr39/tr39-32.html) ### [mjb\_break\_line](https://mojibake.zaerl.com/#mjb_break_line) ``` mjb_break_type mjb_break_line( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_next_line_state *state ); ``` ### Example ``` mjb_next_line_state state; state.index = 0; mjb_break_type type = mjb_break_line("Hello world", 11, MJB_ENC_UTF_8, &state); // First line-break result is set: yes printf("First line-break result is set: %s", type != MJB_BT_NOT_SET ? "yes" : "no"); ``` ### Related - [`mjb\_break\_grapheme\_cluster`](https://mojibake.zaerl.com/#mjb_break_grapheme_cluster) - [`mjb\_break\_word`](https://mojibake.zaerl.com/#mjb_break_word) - [`mjb\_break\_sentence`](https://mojibake.zaerl.com/#mjb_break_sentence) ### Specifications - [UAX \#14: Unicode Line Breaking Algorithm, Unicode 17\.0\.0](https://www.unicode.org/reports/tr14/tr14-55.html) ### [mjb\_break\_word](https://mojibake.zaerl.com/#mjb_break_word) ``` mjb_break_type mjb_break_word( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_next_word_state *state ); ``` ### Example ``` mjb_next_word_state state; state.index = 0; size_t boundaries = 0; while(mjb_break_word("Hello world", 11, MJB_ENC_UTF_8, &state) != MJB_BT_NOT_SET) { ++boundaries; } // Word-break positions: 11 printf("Word-break positions: %zu", boundaries); ``` ### Related - [`mjb\_break\_grapheme\_cluster`](https://mojibake.zaerl.com/#mjb_break_grapheme_cluster) - [`mjb\_break\_sentence`](https://mojibake.zaerl.com/#mjb_break_sentence) - [`mjb\_truncate\_word`](https://mojibake.zaerl.com/#mjb_truncate_word) ### Specifications - [UAX \#29: Unicode Text Segmentation, Unicode 17\.0\.0](https://www.unicode.org/reports/tr29/tr29-47.html) ### [mjb\_break\_sentence](https://mojibake.zaerl.com/#mjb_break_sentence) ``` mjb_break_type mjb_break_sentence( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_next_sentence_state *state ); ``` ### Example ``` mjb_next_sentence_state state; state.index = 0; size_t boundaries = 0; const char *input = "Hello. Goodbye."; while(mjb_break_sentence(input, strlen(input), MJB_ENC_UTF_8, &state) != MJB_BT_NOT_SET) { ++boundaries; } // Sentence-break positions: 15 printf("Sentence-break positions: %zu", boundaries); ``` ### Related - [`mjb\_break\_grapheme\_cluster`](https://mojibake.zaerl.com/#mjb_break_grapheme_cluster) - [`mjb\_break\_word`](https://mojibake.zaerl.com/#mjb_break_word) ### Specifications - [UAX \#29: Unicode Text Segmentation, Unicode 17\.0\.0](https://www.unicode.org/reports/tr29/tr29-47.html) ### [mjb\_break\_grapheme\_cluster](https://mojibake.zaerl.com/#mjb_break_grapheme_cluster) ``` mjb_break_type mjb_break_grapheme_cluster( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_next_state *state ); ``` Iterate the grapheme cluster \(user\-perceived character\) boundaries of a string\. Call repeatedly with the same state until it reports the end of the string\. ### Example ``` const char *input = "e\xCC\x81"; // e + combining acute accent mjb_next_state state; state.index = 0; size_t codepoints = 0; while(mjb_break_grapheme_cluster(input, strlen(input), MJB_ENC_UTF_8, &state) != MJB_BT_NOT_SET) { ++codepoints; } // Codepoints examined: 2 printf("Codepoints examined: %zu", codepoints); ``` ### Related - [`mjb\_break\_word`](https://mojibake.zaerl.com/#mjb_break_word) - [`mjb\_break\_sentence`](https://mojibake.zaerl.com/#mjb_break_sentence) - [`mjb\_break\_line`](https://mojibake.zaerl.com/#mjb_break_line) - [`mjb\_truncate`](https://mojibake.zaerl.com/#mjb_truncate) ### Specifications - [UAX \#29: Unicode Text Segmentation, Unicode 17\.0\.0](https://www.unicode.org/reports/tr29/tr29-47.html) ### [mjb\_truncate](https://mojibake.zaerl.com/#mjb_truncate) ``` size_t mjb_truncate( const char *buffer, size_t byte_length, mjb_encoding encoding, size_t max_graphemes ); ``` ### Example ``` const char *input = "A\xF0\x9F\x87\xAE\xF0\x9F\x87\xB9Z"; // A🇮🇹Z size_t bytes = mjb_truncate(input, strlen(input), MJB_ENC_UTF_8, 2); // First two graphemes use 9 bytes printf("First two graphemes use %zu bytes", bytes); ``` ### [mjb\_truncate\_word](https://mojibake.zaerl.com/#mjb_truncate_word) ``` size_t mjb_truncate_word( const char *buffer, size_t byte_length, mjb_encoding encoding, size_t max_segments ); ``` ### Example ``` const char *input = "Hello world"; size_t bytes = mjb_truncate_word(input, strlen(input), MJB_ENC_UTF_8, 1); // First word segment uses 5 bytes printf("First word segment uses %zu bytes", bytes); ``` ### [mjb\_truncate\_word\_width](https://mojibake.zaerl.com/#mjb_truncate_word_width) ``` size_t mjb_truncate_word_width( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_width_context context, size_t max_columns ); ``` ### Example ``` const char *input = "Hello world"; size_t bytes = mjb_truncate_word_width(input, strlen(input), MJB_ENC_UTF_8, MJB_WIDTH_CONTEXT_WESTERN, 6); // Six columns include 6 bytes printf("Six columns include %zu bytes", bytes); ``` ### [mjb\_bidi\_resolve](https://mojibake.zaerl.com/#mjb_bidi_resolve) ``` mjb_status mjb_bidi_resolve( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_direction direction, mjb_bidi_paragraph *result ); ``` Resolve the embedding levels of a paragraph following the Unicode Bidirectional Algorithm\. The resolved paragraph can then be split into lines and reordered visually with`mjb\_bidi\_reorder\_line`and`mjb\_bidi\_line\_runs`\. ### Returns - `MJB\_STATUS\_OK`— The paragraph was resolved - `MJB\_STATUS\_INVALID\_ARGUMENT`—`result`is NULL, or`buffer`is NULL with a non\-zero size - `MJB\_STATUS\_OVERFLOW`— The paragraph size would overflow - `MJB\_STATUS\_NO\_MEMORY`— Allocation failed ### Example ``` const char *input = "abc \xD7\x90\xD7\x91\xD7\x92"; // abc אבג mjb_bidi_paragraph paragraph; if(mjb_bidi_resolve(input, strlen(input), MJB_ENC_UTF_8, MJB_DIRECTION_AUTO, &paragraph) != MJB_STATUS_OK) { return 1; } // Paragraph codepoints: 7 printf("Paragraph codepoints: %zu", paragraph.count); mjb_bidi_free(&paragraph); ``` ### Related - [`mjb\_bidi\_free`](https://mojibake.zaerl.com/#mjb_bidi_free) - [`mjb\_bidi\_reorder\_line`](https://mojibake.zaerl.com/#mjb_bidi_reorder_line) - [`mjb\_bidi\_line\_runs`](https://mojibake.zaerl.com/#mjb_bidi_line_runs) ### Specifications - [UAX \#9: Unicode Bidirectional Algorithm, Unicode 17\.0\.0](https://www.unicode.org/reports/tr9/tr9-51.html) ### [mjb\_bidi\_reorder\_line](https://mojibake.zaerl.com/#mjb_bidi_reorder_line) ``` mjb_status mjb_bidi_reorder_line( const mjb_bidi_paragraph *paragraph, size_t line_start, size_t line_end, size_t *visual_order ); ``` ### Example ``` const char *input = "\xD7\x90\xD7\x91\xD7\x92"; // אבג mjb_bidi_paragraph paragraph; size_t visual_order[3]; if(mjb_bidi_resolve(input, strlen(input), MJB_ENC_UTF_8, MJB_DIRECTION_AUTO, &paragraph) != MJB_STATUS_OK || mjb_bidi_reorder_line(&paragraph, 0, paragraph.count, visual_order) != MJB_STATUS_OK) { return 1; } // First visual index: 2 printf("First visual index: %zu", visual_order[0]); mjb_bidi_free(&paragraph); ``` ### Related - [`mjb\_bidi\_resolve`](https://mojibake.zaerl.com/#mjb_bidi_resolve) - [`mjb\_bidi\_line\_runs`](https://mojibake.zaerl.com/#mjb_bidi_line_runs) ### Specifications - [UAX \#9: Unicode Bidirectional Algorithm, Unicode 17\.0\.0](https://www.unicode.org/reports/tr9/tr9-51.html) ### [mjb\_bidi\_line\_runs](https://mojibake.zaerl.com/#mjb_bidi_line_runs) ``` mjb_status mjb_bidi_line_runs( const mjb_bidi_paragraph *paragraph, const size_t *visual_order, size_t count, mjb_bidi_run *runs, size_t *run_count ); ``` ### Example ``` mjb_bidi_paragraph paragraph; size_t visual_order[3]; size_t run_count = 0; if(mjb_bidi_resolve("abc", 3, MJB_ENC_UTF_8, MJB_DIRECTION_LTR, &paragraph) != MJB_STATUS_OK || mjb_bidi_reorder_line(&paragraph, 0, 3, visual_order) != MJB_STATUS_OK || mjb_bidi_line_runs(&paragraph, visual_order, 3, NULL, &run_count) != MJB_STATUS_OK) { return 1; } // Visual runs: 1 printf("Visual runs: %zu", run_count); mjb_bidi_free(&paragraph); ``` ### Related - [`mjb\_bidi\_resolve`](https://mojibake.zaerl.com/#mjb_bidi_resolve) - [`mjb\_bidi\_reorder\_line`](https://mojibake.zaerl.com/#mjb_bidi_reorder_line) ### Specifications - [UAX \#9: Unicode Bidirectional Algorithm, Unicode 17\.0\.0](https://www.unicode.org/reports/tr9/tr9-51.html) ### [mjb\_bidi\_free](https://mojibake.zaerl.com/#mjb_bidi_free) ``` void mjb_bidi_free( mjb_bidi_paragraph *paragraph ); ``` ### Example ``` mjb_bidi_paragraph paragraph; if(mjb_bidi_resolve("abc", 3, MJB_ENC_UTF_8, MJB_DIRECTION_LTR, &paragraph) != MJB_STATUS_OK) { return 1; } mjb_bidi_free(&paragraph); // Paragraph released: yes printf("Paragraph released: %s", paragraph.chars == NULL ? "yes" : "no"); ``` ### Related - [`mjb\_bidi\_resolve`](https://mojibake.zaerl.com/#mjb_bidi_resolve) ### [mjb\_codepoint\_emoji](https://mojibake.zaerl.com/#mjb_codepoint_emoji) ``` mjb_status mjb_codepoint_emoji( mjb_codepoint codepoint, mjb_emoji_properties *emoji ); ``` ### Example ``` mjb_emoji_properties emoji; if(mjb_codepoint_emoji(0x1F600, &emoji) != MJB_STATUS_OK) { return 1; } // U+1F600 has Emoji_Presentation: yes printf("U+1F600 has Emoji_Presentation: %s", emoji.presentation ? "yes" : "no"); ``` ### Related - [`mjb\_string\_emoji\_sequence`](https://mojibake.zaerl.com/#mjb_string_emoji_sequence) - [`mjb\_codepoint\_is\_emoji`](https://mojibake.zaerl.com/#mjb_codepoint_is_emoji) ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_codepoint\_is\_emoji](https://mojibake.zaerl.com/#mjb_codepoint_is_emoji) ``` bool mjb_codepoint_is_emoji( mjb_codepoint codepoint ); ``` ### Example ``` // Number sign has the Emoji property: yes bool emoji = mjb_codepoint_is_emoji('#'); // Number sign has the Emoji property: yes printf("Number sign has the Emoji property: %s", emoji ? "yes" : "no"); ``` ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_codepoint\_is\_emoji\_presentation](https://mojibake.zaerl.com/#mjb_codepoint_is_emoji_presentation) ``` bool mjb_codepoint_is_emoji_presentation( mjb_codepoint codepoint ); ``` ### Example ``` // Grinning face defaults to emoji presentation: yes bool presentation = mjb_codepoint_is_emoji_presentation(0x1F600); // Grinning face defaults to emoji presentation: yes printf("Grinning face defaults to emoji presentation: %s", presentation ? "yes" : "no"); ``` ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_codepoint\_is\_emoji\_modifier](https://mojibake.zaerl.com/#mjb_codepoint_is_emoji_modifier) ``` bool mjb_codepoint_is_emoji_modifier( mjb_codepoint codepoint ); ``` ### Example ``` // Medium skin tone is an emoji modifier: yes bool modifier = mjb_codepoint_is_emoji_modifier(0x1F3FD); // Medium skin tone is an emoji modifier: yes printf("Medium skin tone is an emoji modifier: %s", modifier ? "yes" : "no"); ``` ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_codepoint\_is\_emoji\_modifier\_base](https://mojibake.zaerl.com/#mjb_codepoint_is_emoji_modifier_base) ``` bool mjb_codepoint_is_emoji_modifier_base( mjb_codepoint codepoint ); ``` ### Example ``` // Waving hand accepts an emoji modifier: yes bool modifier_base = mjb_codepoint_is_emoji_modifier_base(0x1F44B); // Waving hand accepts an emoji modifier: yes printf("Waving hand accepts an emoji modifier: %s", modifier_base ? "yes" : "no"); ``` ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_codepoint\_is\_emoji\_component](https://mojibake.zaerl.com/#mjb_codepoint_is_emoji_component) ``` bool mjb_codepoint_is_emoji_component( mjb_codepoint codepoint ); ``` ### Example ``` // Zero-width joiner is an emoji component: yes bool component = mjb_codepoint_is_emoji_component(0x200D); // Zero-width joiner is an emoji component: yes printf("Zero-width joiner is an emoji component: %s", component ? "yes" : "no"); ``` ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_string\_emoji\_sequence](https://mojibake.zaerl.com/#mjb_string_emoji_sequence) ``` mjb_status mjb_string_emoji_sequence( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_emoji_sequence *emoji ); ``` ### Example ``` const char *flag = "\xF0\x9F\x87\xAE\xF0\x9F\x87\xB9"; // 🇮🇹 mjb_emoji_sequence emoji; if(mjb_string_emoji_sequence(flag, strlen(flag), MJB_ENC_UTF_8, &emoji) != MJB_STATUS_OK) { return 1; } // Sequence codepoints: 2 printf("Sequence codepoints: %zu", emoji.codepoint_count); ``` ### Related - [`mjb\_string\_is\_emoji\_sequence`](https://mojibake.zaerl.com/#mjb_string_is_emoji_sequence) - [`mjb\_string\_is\_rgi\_emoji`](https://mojibake.zaerl.com/#mjb_string_is_rgi_emoji) ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_string\_is\_emoji\_sequence](https://mojibake.zaerl.com/#mjb_string_is_emoji_sequence) ``` bool mjb_string_is_emoji_sequence( const char *buffer, size_t byte_length, mjb_encoding encoding ); ``` ### Example ``` const char *keycap = "1\xEF\xB8\x8F\xE2\x83\xA3"; // 1️⃣ bool listed = mjb_string_is_emoji_sequence(keycap, strlen(keycap), MJB_ENC_UTF_8); // Listed emoji sequence: yes printf("Listed emoji sequence: %s", listed ? "yes" : "no"); ``` ### Related - [`mjb\_string\_is\_rgi\_emoji`](https://mojibake.zaerl.com/#mjb_string_is_rgi_emoji) - [`mjb\_string\_emoji\_sequence`](https://mojibake.zaerl.com/#mjb_string_emoji_sequence) ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_string\_is\_rgi\_emoji](https://mojibake.zaerl.com/#mjb_string_is_rgi_emoji) ``` bool mjb_string_is_rgi_emoji( const char *buffer, size_t byte_length, mjb_encoding encoding ); ``` ### Example ``` const char *flag = "\xF0\x9F\x87\xAE\xF0\x9F\x87\xB9"; // 🇮🇹 bool rgi = mjb_string_is_rgi_emoji(flag, strlen(flag), MJB_ENC_UTF_8); // RGI emoji: yes printf("RGI emoji: %s", rgi ? "yes" : "no"); ``` ### Related - [`mjb\_string\_is\_emoji\_sequence`](https://mojibake.zaerl.com/#mjb_string_is_emoji_sequence) - [`mjb\_string\_emoji\_sequence`](https://mojibake.zaerl.com/#mjb_string_emoji_sequence) ### Specifications - [UTS \#51: Unicode Emoji, Unicode 17\.0\.0](https://www.unicode.org/reports/tr51/tr51-29.html) ### [mjb\_truncate\_width](https://mojibake.zaerl.com/#mjb_truncate_width) ``` size_t mjb_truncate_width( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_width_context context, size_t max_columns ); ``` ### Example ``` const char *input = "A\xE7\x95\x8C"; // A界 size_t bytes = mjb_truncate_width(input, strlen(input), MJB_ENC_UTF_8, MJB_WIDTH_CONTEXT_WESTERN, 2); // Two columns include 1 byte printf("Two columns include %zu byte", bytes); ``` ### [mjb\_codepoint\_east\_asian\_width](https://mojibake.zaerl.com/#mjb_codepoint_east_asian_width) ``` mjb_status mjb_codepoint_east_asian_width( mjb_codepoint codepoint, mjb_east_asian_width *width ); ``` ### Example ``` mjb_east_asian_width width; if(mjb_codepoint_east_asian_width(0x754C, &width) != MJB_STATUS_OK) { // 界 return 1; } // U+754C is wide: yes printf("U+754C is wide: %s", width == MJB_EAW_WIDE ? "yes" : "no"); ``` ### Related - [`mjb\_display\_width`](https://mojibake.zaerl.com/#mjb_display_width) ### Specifications - [UAX \#11: East Asian Width, Unicode 17\.0\.0](https://www.unicode.org/reports/tr11/tr11-44.html) ### [mjb\_display\_width](https://mojibake.zaerl.com/#mjb_display_width) ``` mjb_status mjb_display_width( const char *buffer, size_t byte_length, mjb_encoding encoding, mjb_width_context context, size_t *width ); ``` Compute the number of display columns a string occupies in a terminal, accounting for wide and ambiguous East Asian characters, combining marks, and emoji sequences\. ### Returns - `MJB\_STATUS\_OK`— The width was computed - `MJB\_STATUS\_INVALID\_ARGUMENT`—`width`is NULL, or`buffer`is NULL with a non\-zero size - `MJB\_STATUS\_OVERFLOW`— The width would overflow ### Example ``` const char *input = "A\xE7\x95\x8C"; // A界 size_t width; if(mjb_display_width(input, strlen(input), MJB_ENC_UTF_8, MJB_WIDTH_CONTEXT_WESTERN, &width) != MJB_STATUS_OK) { return 1; } // Display columns: 3 printf("Display columns: %zu", width); ``` ### Related - [`mjb\_codepoint\_east\_asian\_width`](https://mojibake.zaerl.com/#mjb_codepoint_east_asian_width) - [`mjb\_truncate\_width`](https://mojibake.zaerl.com/#mjb_truncate_width) ### Specifications - [UAX \#11: East Asian Width, Unicode 17\.0\.0](https://www.unicode.org/reports/tr11/tr11-44.html) ### [mjb\_codepoint\_is\_hangul\_l](https://mojibake.zaerl.com/#mjb_codepoint_is_hangul_l) ``` bool mjb_codepoint_is_hangul_l( mjb_codepoint codepoint ); ``` ### Example ``` // U+1100 is a leading Jamo: yes printf("U+1100 is a leading Jamo: %s", mjb_codepoint_is_hangul_l(0x1100) ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_hangul\_v](https://mojibake.zaerl.com/#mjb_codepoint_is_hangul_v) ``` bool mjb_codepoint_is_hangul_v( mjb_codepoint codepoint ); ``` ### Example ``` // U+1161 is a vowel Jamo: yes printf("U+1161 is a vowel Jamo: %s", mjb_codepoint_is_hangul_v(0x1161) ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_hangul\_t](https://mojibake.zaerl.com/#mjb_codepoint_is_hangul_t) ``` bool mjb_codepoint_is_hangul_t( mjb_codepoint codepoint ); ``` ### Example ``` // U+11A8 is a trailing Jamo: yes printf("U+11A8 is a trailing Jamo: %s", mjb_codepoint_is_hangul_t(0x11A8) ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_hangul\_jamo](https://mojibake.zaerl.com/#mjb_codepoint_is_hangul_jamo) ``` bool mjb_codepoint_is_hangul_jamo( mjb_codepoint codepoint ); ``` ### Example ``` // U+1100 is Hangul Jamo: yes printf("U+1100 is Hangul Jamo: %s", mjb_codepoint_is_hangul_jamo(0x1100) ? "yes" : "no"); ``` ### [mjb\_codepoint\_is\_hangul\_syllable](https://mojibake.zaerl.com/#mjb_codepoint_is_hangul_syllable) ``` bool mjb_codepoint_is_hangul_syllable( mjb_codepoint codepoint ); ``` ### Example ``` // U+AC00 is a Hangul syllable: yes printf("U+AC00 is a Hangul syllable: %s", mjb_codepoint_is_hangul_syllable(0xAC00) ? "yes" : "no"); ``` ### [mjb\_hangul\_syllable\_name](https://mojibake.zaerl.com/#mjb_hangul_syllable_name) ``` mjb_status mjb_hangul_syllable_name( mjb_codepoint codepoint, char *buffer, size_t byte_length ); ``` ### Example ``` char name[32]; if(mjb_hangul_syllable_name(0xAC01, name, sizeof(name)) != MJB_STATUS_OK) { return 1; } // Name: HANGUL SYLLABLE GAG printf("Name: %s", name); ``` ### [mjb\_hangul\_syllable\_decomposition](https://mojibake.zaerl.com/#mjb_hangul_syllable_decomposition) ``` mjb_status mjb_hangul_syllable_decomposition( mjb_codepoint codepoint, mjb_codepoint *codepoints ); ``` ### Example ``` mjb_codepoint decomposition[3]; if(mjb_hangul_syllable_decomposition(0xAC01, decomposition) != MJB_STATUS_OK) { return 1; } // Decomposition starts with: U+1100 printf("Decomposition starts with: U+%04X", decomposition[0]); ``` ### [mjb\_hangul\_syllable\_composition](https://mojibake.zaerl.com/#mjb_hangul_syllable_composition) ``` size_t mjb_hangul_syllable_composition( mjb_buffer_character *characters, size_t characters_len ); ``` ### Example ``` mjb_buffer_character characters[] = { { 0x1100, 0 }, // choseong kiyeok { 0x1161, 0 }, // jungseong a { 0x11A8, 0 } // jongseong kiyeok }; size_t length = mjb_hangul_syllable_composition(characters, 3); // Composition: U+AC01 printf("Composition: U+%04X", length == 1 ? characters[0].codepoint : 0); ``` ### [mjb\_property\_name](https://mojibake.zaerl.com/#mjb_property_name) ``` const char *mjb_property_name( mjb_property property ); ``` ### [mjb\_locale\_parse](https://mojibake.zaerl.com/#mjb_locale_parse) ``` mjb_status mjb_locale_parse( const char *id, size_t byte_length, mjb_encoding encoding, mjb_locale_id *locale, mjb_error *error ); ``` Parse a BCP 47 language tag, such as`sr\-Latn\-RS`, into its components: language, extended language, script, region, variant, extensions, private use, and grandfathered tags\. Parsing is strict: malformed tags are rejected and`error`is filled with the failure reason\. ### Returns - `MJB\_STATUS\_OK`— The tag was parsed and`locale`filled - `MJB\_STATUS\_INVALID\_ARGUMENT`— An argument is NULL or the tag is not a valid BCP 47 language tag - `MJB\_STATUS\_NO\_MEMORY`— Allocation failed ### Example ``` mjb_locale_id locale; mjb_error error; if(mjb_locale_parse("sr-Latn-RS", 10, MJB_ENC_UTF_8, &locale, &error) != MJB_STATUS_OK) { return 1; } // Locale: sr Latn RS printf("Locale: %s %s %s", locale.language, locale.script, locale.region); ``` ### Related - [`mjb\_locale\_set`](https://mojibake.zaerl.com/#mjb_locale_set) ### Specifications - [BCP 47: Tags for Identifying Languages](https://www.rfc-editor.org/rfc/rfc5646) ### [mjb\_locale\_set](https://mojibake.zaerl.com/#mjb_locale_set) ``` mjb_status mjb_locale_set( unsigned int locale ); ``` Set the process\-global locale used by`mjb\_case`\. The default locale is`MJB\_LOCALE\_EN`, and`mjb\_shutdown`resets it to`MJB\_LOCALE\_EN`\. Only`MJB\_LOCALE\_TR`,`MJB\_LOCALE\_AZ`, and`MJB\_LOCALE\_LT`currently tailor casing\. Other valid locale values are accepted but do not change Unicode algorithm behavior\. ### Returns - `MJB\_STATUS\_OK`— The locale was set - `MJB\_STATUS\_INVALID\_ARGUMENT`—`locale`is not a valid`mjb\_locale`value ### Example ``` if(mjb_locale_set(MJB_LOCALE_TR) != MJB_STATUS_OK) { return 1; } // Turkish locale selected: yes printf("Turkish locale selected: yes"); if(mjb_locale_set(MJB_LOCALE_EN) != MJB_STATUS_OK) { return 1; } ``` ### Related - [`mjb\_case`](https://mojibake.zaerl.com/#mjb_case) ### [mjb\_result\_free](https://mojibake.zaerl.com/#mjb_result_free) ``` mjb_status mjb_result_free( mjb_result *result ); ``` Free the memory allocated for a`mjb\_result`\. The`result`pointer is set to NULL\. ### Returns - `MJB\_STATUS\_OK`— The result was freed - `MJB\_STATUS\_INVALID\_ARGUMENT`—`result`is NULL ### Example ``` mjb_result result; if(mjb_string_convert_encoding("A", 1, MJB_ENC_UTF_8, MJB_ENC_UTF_16LE, &result) != MJB_STATUS_OK || mjb_result_free(&result) != MJB_STATUS_OK) { return 1; } // Result released: yes printf("Result released: %s", result.output == NULL ? "yes" : "no"); ``` ### [mjb\_version](https://mojibake.zaerl.com/#mjb_version) ``` const char *mjb_version(void); ``` Output the current library version as a string, such as "1\.0\.0"\. ### Example ``` const char *version = mjb_version(); // Version is available: yes printf("Version is available: %s", version[0] != '\0' ? "yes" : "no"); ``` ### Related - [`mjb\_version\_number`](https://mojibake.zaerl.com/#mjb_version_number) - [`mjb\_unicode\_version`](https://mojibake.zaerl.com/#mjb_unicode_version) ### [mjb\_version\_number](https://mojibake.zaerl.com/#mjb_version_number) ``` unsigned int mjb_version_number(void); ``` Output the current library version number as an unsigned integer\. ### Example ``` unsigned int version = mjb_version_number(); // Version number is positive: yes printf("Version number is positive: %s", version > 0 ? "yes" : "no"); ``` ### Related - [`mjb\_version`](https://mojibake.zaerl.com/#mjb_version) - [`mjb\_unicode\_version`](https://mojibake.zaerl.com/#mjb_unicode_version) ### [mjb\_unicode\_version](https://mojibake.zaerl.com/#mjb_unicode_version) ``` const char *mjb_unicode_version(void); ``` Output the current supported Unicode version as a string, such as "15\.0\.0"\. ### Example ``` const char *version = mjb_unicode_version(); // Unicode version: 17.0.0 printf("Unicode version: %s", version); ``` ### Related - [`mjb\_version`](https://mojibake.zaerl.com/#mjb_version) - [`mjb\_version\_number`](https://mojibake.zaerl.com/#mjb_version_number) ### [mjb\_set\_memory\_functions](https://mojibake.zaerl.com/#mjb_set_memory_functions) ``` mjb_status mjb_set_memory_functions( mjb_alloc_fn alloc_fn, mjb_realloc_fn realloc_fn, mjb_free_fn free_fn ); ``` Replace the allocator used by the library for all internal allocations and for the buffers returned in`mjb\_result`\. Must be called before any other library call\. ### Example ``` mjb_shutdown(); // Ensure no allocator is currently locked in. if(mjb_set_memory_functions(malloc, realloc, free) != MJB_STATUS_OK) { return 1; } // Standard allocator installed: yes printf("Standard allocator installed: yes"); mjb_shutdown(); ``` ### Related - [`mjb\_alloc`](https://mojibake.zaerl.com/#mjb_alloc) - [`mjb\_realloc`](https://mojibake.zaerl.com/#mjb_realloc) - [`mjb\_free`](https://mojibake.zaerl.com/#mjb_free) ### [mjb\_shutdown](https://mojibake.zaerl.com/#mjb_shutdown) ### Example ``` mjb_shutdown(); // Library state reset: yes printf("Library state reset: yes"); ``` ### [mjb\_alloc](https://mojibake.zaerl.com/#mjb_alloc) ``` void *mjb_alloc( size_t byte_length ); ``` Allocate memory using the allocator set by`mjb\_set\_memory\_functions`\. If no allocator is set, the default allocator is used\. ### Example ``` char *buffer = (char*)mjb_alloc(32); if(buffer == NULL) { return 1; } strcpy(buffer, "allocated"); // Buffer: allocated printf("Buffer: %s", buffer); mjb_free(buffer); ``` ### Related - [`mjb\_realloc`](https://mojibake.zaerl.com/#mjb_realloc) - [`mjb\_free`](https://mojibake.zaerl.com/#mjb_free) ### [mjb\_realloc](https://mojibake.zaerl.com/#mjb_realloc) ``` void *mjb_realloc( void *ptr, size_t new_size ); ``` Reallocate memory using the allocator set by`mjb\_set\_memory\_functions`\. If no allocator is set, the default allocator is used\. ### Example ``` char *buffer = (char*)mjb_alloc(8); if(buffer == NULL) { return 1; } char *larger = (char*)mjb_realloc(buffer, 32); if(larger == NULL) { mjb_free(buffer); return 1; } // Reallocation succeeded: yes printf("Reallocation succeeded: yes"); mjb_free(larger); ``` ### Related - [`mjb\_alloc`](https://mojibake.zaerl.com/#mjb_alloc) - [`mjb\_free`](https://mojibake.zaerl.com/#mjb_free) ### [mjb\_free](https://mojibake.zaerl.com/#mjb_free) ``` void mjb_free( void *ptr ); ``` Free memory using the allocator set by`mjb\_set\_memory\_functions`\. If no allocator is set, the default allocator is used\. ### Example ``` void *memory = mjb_alloc(16); if(memory == NULL) { return 1; } mjb_free(memory); // Memory freed: yes printf("Memory freed: yes"); ``` ### Related - [`mjb\_alloc`](https://mojibake.zaerl.com/#mjb_alloc) - [`mjb\_realloc`](https://mojibake.zaerl.com/#mjb_realloc)