Changelog: 0.1.x

Archived verbatim from CHANGELOG.md. Newer releases are in the changelog; the archive index lists every older series.

[0.1.5] — 2026-03-27

Added

  • Reverse transliteration: transliterate(text, target="ru") converts Latin → native script for Russian, Ukrainian, and Greek. PHF tables generated at build time from inverted language TSV data.
  • Toned pinyin: transliterate("北京", tones=True) returns "běi jīng" with tone marks. Toned readings sourced from Unihan kMandarin field for all 20,924 CJK Unified Ideographs.
  • ISO 9:1995 scholarly Cyrillic: transliterate(text, strict_iso9=True) for scholarly romanization. GOST R 7.0.34 variant via gost7034=True.
  • Japanese Kunrei-shiki (lang="ja-kunrei"): alternative romanization profile, bringing total language count to 65.
  • Ancient scripts: Coptic, Gothic, Old Italic, Runic, Ogham transliteration tables.
  • CLI short aliases: t (transliterate), s (slugify), n (normalize), p (pipeline), d (demojize) — e.g. translit t "café".
  • CLI --target flag: translit t --target ru "Moskva" for reverse transliteration.
  • CLI --tones, --strict-iso9, --gost7034 flags for transliterate subcommand.
  • CLI --lang flag for slugify subcommand.
  • console_scripts entry point: translit command available after pip install translit-rs.
  • docs/cli.md: comprehensive CLI documentation with piping, exit codes, examples.
  • Links section in README.md and docs/index.md for RTD ↔ GitHub cross-references.

Changed

  • transliterate() API unified: reverse_transliterate() merged into transliterate() via target parameter. Old function removed.
  • transliterate_impl Rust signature now takes 7 arguments (added tones: bool).
  • Updated benchmark numbers after tones parameter addition (15–46% regression in transliteration hot path due to additional branch; throughput now 450M chars/sec Latin, 130M chars/sec Cyrillic).
  • Performance documentation updated across 4 files to reflect current benchmark results.

Fixed

  • clippy format_push_string lint in build.rs — replaced push_str(&format!()) with write!().
  • clippy unreadable_literal in PHF-generated reverse_translit_phf.rs — suppressed via inner attribute in src/reverse.rs.
  • All 219 integration test call sites updated for 7-argument transliterate_impl.

[0.1.4] — 2026-03-25

Added

  • lang="auto" script-based language detection: When lang="auto" is passed to transliterate(), slugify(), TextPipeline, Slugifier, or any other call site, the library detects the dominant non-Latin script in the input and maps it to a default language code automatically. Maps 28 scripts to language codes (e.g. Cyrillic→ru, Han→zh, Hiragana/Katakana→ja, Thai→th). Zero overhead for lang=None or explicit lang codes.
  • LANG_AUTO constant ("auto") in translit._enums.
  • Georgian transliteration (lang="ka"): 114 TSV entries covering Mkhedruli, Mtavruli, and supplement ranges. BGN/PCGN national romanization.
  • Armenian transliteration (lang="hy"): 86 TSV entries covering uppercase, lowercase, and 5 ligatures (U+FB13–FB17). BGN/PCGN romanization.
  • Sinhala transliteration (lang="si"): 90 TSV entries. Extended Indic Brahmic engine range from 0x0900..=0x0D7F to 0x0900..=0x0DFF with dedicated sinhala_char_role() function for Sinhala-specific offsets.
  • Thai transliteration (lang="th"): 87 TSV entries using RTGS romanization. New ScriptClass::Tai with tone-mark stripping and cancellation handling.
  • Lao transliteration (lang="lo"): 67 TSV entries using BGN/PCGN romanization. Shares Tai engine with Thai via offset masking.
  • Ethiopic transliteration (lang="am"): 307 TSV entries for Ge'ez alphasyllabary (34 consonant bases × 7 vowel orders + labialized forms + digits). Pure data addition — no engine changes needed.
  • Myanmar transliteration (lang="my"): 89 TSV entries. New myanmar_char_role() for Brahmic engine with virama (U+1039) and asat (U+103A) support. Medials (U+103B–103E) classified as dependent vowels.
  • Khmer transliteration (lang="km"): 110 TSV entries. New khmer_char_role() for Brahmic engine with coeng (U+17D2) as virama. All consonants normalized to inherent 'a' regardless of series.
  • Tibetan transliteration (lang="bo"): 147 TSV entries. New tibetan_char_role() for Brahmic engine with halanta (U+0F84) and subjoined consonants (U+0F90–0FBC).
  • Unicode range constants: TIBETAN (0x0F00–0x0FFF), MYANMAR (0x1000–0x109F), KHMER (0x1780–0x17FF) in src/unicode_ranges.rs.
  • Comprehensive test coverage: example-based tests for all 9 new scripts, property-based tests (hypothesis + proptest), multi-script mixture tests.
  • Built-in language count: 51 → 60.

Changed

  • is_indic() extended to include Tibetan, Myanmar, and Khmer ranges for Brahmic abugida processing.
  • indic_char_role() dispatches to script-specific functions for Sinhala, Tibetan, Myanmar, and Khmer codepoint ranges.

[0.1.3] — 2026-03-25

Added

  • strip_control and strip_zero_width now work as independent pipeline steps without requiring collapse_whitespace=True. Previously they were silently ignored when collapse_whitespace was disabled.
  • strip_control_chars() and strip_zero_width_chars() standalone Rust functions for filtering without whitespace collapsing.
  • decimal and hexadecimal flags in SlugConfig are now functional. Setting decimal=False preserves &#NNN; entities; hexadecimal=False preserves &#xHHH; entities. Previously these flags were accepted but silently ignored.
  • Rust integration tests: tests/integration_emoji.rs (10 tests), tests/integration_slugify.rs (20 tests), tests/integration_transliterate.rs (21 tests), tests/integration_whitespace.rs (12 tests).

Changed

  • TextPipeline parameters strip_control and strip_zero_width changed from bool (default True) to bool | None (default None). When None, they inherit from collapse_whitespace — True if collapse_whitespace=True, False otherwise. Set explicitly to True for standalone use without collapse_whitespace. This is backward compatible: existing code that passes collapse_whitespace=True gets the same behavior as before.
  • steps() now reports strip_control and strip_zero_width as separate entries when active, giving full visibility into pipeline behavior.
  • Pipeline step order updated: normalize → confusables → demojize → strip_accents → transliterate → fold_case → strip_control → strip_zero_width → collapse_whitespace.
  • Migrated from once_cell to std::sync::LazyLock / OnceLock; MSRV bumped to 1.80. Removed once_cell dependency.
  • needs_cjk_space() match arm tightened from wildcard _ to explicit Ideograph | Hangul | Kana to match the call-site is_cjk guard.

Fixed

  • decode_entities() corrupting multi-byte UTF-8 characters (BUG-1). The function used bytes[i] as char which treated each continuation byte as a separate Latin-1 codepoint (e.g. café → café). Now advances by full UTF-8 characters.
  • decode_numeric_entity_skip() panicking on malformed &# followed by multi-byte UTF-8 (BUG-2). The skip function walked through continuation bytes looking for ;, landing inside a multi-byte character. Now stops at the first non-ASCII byte.

Performance

  • ASCII fast-path in demojize_impl and demojize_rust: pure-ASCII text returns immediately without Vec<char> allocation or emoji scanning.
  • filter_stopwords replaced intermediate Vec<_> + .join() with a pre-allocated String fold, removing one allocation per slugify call.

[0.1.2] — 2026-03-25

Added

  • Python 3.14 support (classifier and CI test matrix).
  • ruff check --fix pre-commit hook for automatic lint fixing.
  • CI publish workflow using pypa/gh-action-pypi-publish with OIDC trusted publishers.
  • Multi-platform wheel builds: Linux (x86_64, aarch64), macOS (Intel, ARM64), Windows.
  • steps() method on _TextPipeline type stub.

Changed

  • Resolved all clippy pedantic warnings instead of suppressing them — reduced lint suppressions from 48 to 22 (remaining are genuine PyO3 constraints). Fixes include: combined identical match arms, replaced manual counters with .enumerate(), moved item declarations before statements, used clone_into(), merged identical branches, fixed doc comment formatting.
  • Widened stopwords and replacements type stubs from strict tuple/list to Sequence for better mypy compatibility.
  • Applied ruff format to all Python source and test files.
  • Switched docs publish from deprecated maturin upload to pypa/gh-action-pypi-publish.
  • macOS Intel wheels now cross-compiled on ARM64 runner (macos-14) instead of deprecated macos-13.
  • CI doctests now run against installed package (not source tree) with explicit shell: bash for Windows compatibility.

Fixed

  • TextPipeline.explain() doctest: output format is normalize (NFC) not normalize (form=NFC).
  • from __future__ import annotations placement in test files (must follow module docstring, not precede it).
  • Malformed HTML entity test expectation: decode_entities("&#xyz;") correctly returns "", not "yz;".
  • Rust benchmark CI: target bench_core binary explicitly to avoid passing Criterion flags to the test harness.
  • Ruff lint fixes: unsorted imports in test_encoding.py, unused import is_mixed_script in test_security_invariants.py.
  • Read the Docs trigger workflow: simplified curl status handling, graceful warning when RTD_TOKEN is missing.
  • Removed incorrect PyPy classifier (abi3 is CPython-only).

[0.1.1] — 2026-03-25

Added

  • src/unicode_ranges.rs — named constants for all Unicode codepoint ranges used by the library, eliminating magic numbers scattered across modules.
  • tests/test_concurrency.py — concurrent access tests for LANG_TABLES and HANGUL_CACHE, plus malformed Unicode input tests.
  • Code coverage reporting in CI (pytest-cov, XML report uploaded as artifact).
  • CLOCK$, KEYBD$, SCREEN$, COM0, LPT0 added to Windows reserved filename list.
  • casefold() alias for fold_case() — matches str.casefold() naming.
  • remove_accents() alias for strip_accents() — matches sklearn/ML ecosystem naming.
  • Compatibility parameter aliases: replacement_text/max_len on sanitize_filename() (pathvalidate), greedy/preferred_aliases on is_confusable() (confusable_homoglyphs), delimiters on demojize() (emoji library).
  • Complete API documentation for 19 previously undocumented exported functions: precompiled pipelines, grapheme clusters, encoding detection, Text builder, is_safe_hostname, demojize, strip_bidi, EmojiProvider protocol.
  • Three new API reference pages: Precompiled Pipelines, Grapheme Clusters, Encoding.
  • "Guides by role" section in docs/index.md and README.md.
  • Performance section in README.md with benchmark numbers.
  • Script enum documentation expanded from 28 to all 41 members.

Changed

  • transliterate_impl refactored: capacity estimation extracted to estimate_capacity(), character classification to classify_char(), and CJK spacing logic to needs_cjk_space().
  • All RwLock accesses now recover from lock poisoning using .unwrap_or_else(|e| e.into_inner()) instead of silently falling through.
  • Lambda closures in _compat.py replaced with named inner functions for clarity.
  • emoji.rs write!() call no longer uses .unwrap() (infallible, documented with a // SAFETY comment).
  • MkDocs theme switched from material to readthedocs.
  • All documentation references updated from "unirust" to "translit".
  • Development status promoted from Alpha to Beta.
  • Package renamed from translit to translit-rs on PyPI (interim until PEP 541 grants the translit name). Python import remains import translit.

Fixed

  • Type stub _text.pyi imported from wrong module name (unirust → translit).
  • Type stub _translit.pyi missing min_confidence parameter on _decode_to_utf8.
  • Type stub _text.pyi missing grapheme_split, grapheme_truncate, catalog_key methods.
  • security_clean() pipeline step order corrected in 5+ locations: strip_bidi runs before collapse_whitespace (matching Rust implementation).
  • catalog_key() step order corrected: transliterate before strip_accents.
  • Stale PyO3 boundary overhead corrected from ~4µs to ~240ns in docs and code comments.

Deprecated

  • translit._compat awesome-slugify compatibility layer (Slugify, UniqueSlugify, slugify_* instances) — planned removal in v1.0.

[0.1.0] — 2026-01-01

Added

  • Initial release.
  • Unicode transliteration for 60 language profiles.
  • Slugification, normalization, confusable detection, filename sanitization.
  • Emoji demojization with ZWJ sequence support.
  • Backward-compatible layers for Unidecode and awesome-slugify.