I have following paragraph here presevered until a decision is made.
- Currently, this optional key is a discussion proposal.
Background: "Unicode Normalization can break Biblical Hebrew."
Most modules with Unicode source text are encoded as UTF-8 and normalized to NFC, these being the default settings for both osis2mod & tei2mod.
Now these two module creation tools have a -N command line switch to prevent conversion to UTF-8 and Normalization.
Biblical Hebrew source text with both vowel accents and cantillation may be supplied properly with custom normalization as required by the text provider. It should still be encoded UTF-8.
As there is a need to create modules from source text that has such a custom ordering of the diacritics, it may be useful to provide information in the .conf file for such modules that are intentionally not normalized to NFC during build. The following method is proposed:
It should be assumed that modules where this is specified are made using the -N switch in the module creation tool.
Normalization is useful to ensure (e.g.) that a search index stores all words the same way. That's why for the most part, modules are expected to be in NFC form. Custom normalization is still a normalization. What's different about it is that the combining classes for each character are different from the canonical combining classes defined by the Unicode Consortium.
- Added to SWORD SVN by DM Smith on 2018-01-07.
- The UI in front-ends for creating a search index should ideally also be enhanced to support this option.
Missing sections for RTF, Continuation, Localisation
Some edits during the last 3 years seem to have removed the 3 sections linked by words in the Allowed column. The links don't go anywhere now. This error needs fixing. David Haslam (talk) 13:40, 14 March 2020 (UTC)