CrossWire Bible Society - User contributions [en]

Converting SFM Bibles to OSIS

2016-10-25T03:23:50Z

Osk: /* Converting USFM files to OSIS */

==Module Development==
Please first visit this page [[Module Development Collaboration]] – it may save you a lot of time.

==Introduction==

'''Standard Format Markers''' (SFM) and its more standardized derivative USFM have been used for decades to store Bibles for printing and display in programs like [http://paratext.org/ UBS Paratext]. The format is popular among Bible translation agencies and Bible societies. The basic format of SFM is simply plaintext with backslash(\) codes. For example, a new Bible verse is signaled by '''\v''' followed by the number of the verse.

Older texts in various dialects of SFM can still be found. It was common for various agencies, Bible societies, and even regional offices of such groups to have their own SFM standards. Unified Standard Format Markers (USFM) was developed to standardize SFM and encourage interoperability so that Bibles from one agency could be reasonably expected to operate with the software and style sheets employed by another. [http://paratext.org/usfm USFM] version 2.4 is the latest one documented.

==Preparing (U)SFM files for conversion==

The simplicity of writing SFM also makes it easy to write poor SFM that fails to correspond to any kind of standard. The first task in preparing to convert SFM files to OSIS is to clean the text. The more regular your source files are, the more likely the conversion process will operate correctly.

One method of cleaning up your files is to import them into an SFM editor such as [http://bibledit.org/ Bibledit] (which now runs on Android, iOS, Linux, Mac OS X, Windows and Cloud) or [http://www.sil.org/computing/fieldworks/BTE-FW_downloads.htm SIL FieldWorks Translation Editor] (which runs on Windows). The editors will frequently perform some basic corrections to the SFM syntax, but Bibledit in particular can perform a number of checks to correct specific errors common to SFM.

As of March 2013, after a formal agreement was signed with UBS, anyone who is a member of CrossWire's publishing staff is allowed to register and download [http://paratext.org/ Paratext] software - subject to individual approval by CrossWire's director. This agreement covers the software only, without giving access to the copyrighted resources associated with Paratext that are of greater use for Bible translators.

===Checking USFM files for versification issues===
Rather than waiting until you have the end result, a SWORD module, and then running the general utility '''emptyvss''', it is possible to use another Windows based utility called [http://gbcpreprocessor.codeplex.com/ GoBibleCreator USFM Preprocessor]. Despite its name, its use is not limited to conversion of documents for use towards making Go Bible applications. Although this was originally developed for supporting the Go Bible project, it does have some useful more general features. This is one of them, and is found under the "Check for Consistency" tab. Another is the ability to export the USFM Bible into BQ/DigiStudy format. The Search for Versification Issues is particularly helpful for Bibles structured primarily as "verse per line", rather than as paragraphs containing a verse range. Another useful feature is the identification of non-standard markers.

=== USFM tag statistics ===
Before embarking on any USFM conversion to another format, it's often very useful to list all the different USFM markers (tags) that are used in the project SFM files. [[User:osk|Osk]] has developed a Python script called usfmtags.py to perform this task. It can be downloaded from his GitHub repository [https://github.com/chrislit/usfm2osis].

[[User:David Haslam|David Haslam]] has developed a bespoke [http://www.datamystic.com/textpipe TextPipe] filter that performs a similar task. The tabbed output text file has columns for Count, Tag & Description. TextPipe is for Windows only. Details available on request.

==Converting USFM files to OSIS==

This section gives details of software utilities capable of converting USFM files to OSIS.

All such conversions require the use of [[List of eXtensions to OSIS used in SWORD|eXtensions to OSIS]] of one form or another. Whether these are supported by the SWORD engine should be researched before users conclude that there are faults with the particular conversion method.

=== usfm2osis ===
Some time ago CrossWire volunteer [[User:Osk|Osk]] developed a Python script to convert USFM to OSIS. Since 2014 it has been developed as a more modular Python module rather than a single script. It is maintained at [https://github.com/chrislit/usfm2osis GitHub], but most users should install it directly from [https://pypi.python.org/pypi/usfm2osis PyPI] via the pip command:
pip install usfm2osis

==== Usage ====
Once the usfm2osis module in installed, the usfm2osis.py conversion script itself may be invoked by calling:
usfm2osis.py

=== usfm2osis.py ===
Another CrossWire volunteer [[User:refdoc|RefDoc]] has adapted the script so that CrossWire can continue to use it for module development and release. Download this latest version from [https://github.com/refdoc/Module-tools].

The '''xulsword''' developer '''John Austin''' also has commit privileges to this github repo, such that this script is normally synchronised with his own version of the same Python script. ''See below''.

One of the original stated aims was to achieve '''full coverage''' of USFM according to the [http://paratext.org/usfm ParaTExt spec].

'''Notes:'''
# usfm2osis.py version 0.6 is the current release. Refer to the roadmap within the script.
# usfm2osis.py will always only generate best practice OSIS, to the best of its ability, so it won't generate stuff that SWORD requires due to its own shortcomings.
# usfm2osis.py generates OSIS with the '''milestone''' form of both chapter and verse elements.
# usfm2osis.py doesn't yet use SWORD bindings, so it's very limited in its knowledge of versification systems. This may mean that for the time being, expanding cross-references would typically have to be done by other means. ''See below''.
# In general, anyone interested in converting USFM to OSIS for use with SWORD ought now to employ usfm2osis.py as it's now the only supported pathway for USFM->OSIS->SWORD, inasmuch as its author has a fairly close connection with SWORD and regularly commits stuff to SWORD filters and whatnot. He will definitely not make the least effort to accommodate any non-standard output from converters other than usfm2osis.py.
# It is advised to use Python 2.7.x rather than Python 3.x as there are still some issues in getting the script to work with Python 3.
# Windows users are advised that should they encounter problems trying to run the script with ordinary CPython from python.org, to use the Python interpreter that can be installed with Cygwin. This would also help for specifying the input files using wildcards.
# Useful tip: Cygwin's bash shell recognizes [http://en.wikipedia.org/wiki/NTFS_symbolic_link NTFS symbolic links]. This means that the USFM input files and the Python script don't need to be copied to the cygwin/home/''user'' directory. Just make suitable links by means of the Windows mklink command.
# Bug reports and feature requests are welcomed at either [https://github.com/chrislit/usfm2osis/issues], [http://www.crosswire.org/tracker/browse/MODTOOLS] or [mailto:sword-support@crosswire.org sword-support].

==== Usage ====
usfm2osis.py requires a Python interpreter in the system path.

Optional: If you wish the script to validate the OSIS output, then you'll need to install [http://lxml.de/ lxml].

The usage of usfm2osis.py is output when the script is run without parameters, i.e. with the shell command line
python usfm2osis.py
The syntax is very close to that for usfm2osis.pl – ''see below''.

==== XML Syntax Checking ====

Until the Python script is fully debugged, it's possible that it may generate an OSIS file that fails the XML syntax check. As and when this occurs, it is always useful to isolate the location[s] of the XML file where the syntax is incorrect. This can be done by making separate OSIS XML files for each USFM file, or at least to exclude suspect files from the build by temporarily renaming the file extensions such that it does not match the wildcard pattern used in the command line.

==== OSIS Validation ====
Until the Python script is fully debugged, it's quite possible that even though you have an OSIS file that passes the XML syntax check, it may still fail to validate to the OSIS schema. Similar techniques are useful here too. Make an individual OSIS file from each USFM file, so that the valid files can be distinguished from the invalid files. This facilitates further investigation to more easily locate the invalid elements.

===== OSIS schema =====
CrossWire now validates OSIS files using our own custom adaptation of the OSIS schema. See [[OSIS 211 CR]] for details.

==== Compatibility with osis2mod ====
Until the Python script is fully debugged, it's quite possible that even though you have an OSIS file that validates to the OSIS schema, it may still lead to problems when used as the input file for osis2mod. Much will depend on the complexities of the original USFM files, in terms of which markers were used and in what sequences. Some post-processing of the OSIS XML may be necessary to fix such potential problems.

==== Unsupported Markers ====
Please refer to the roadmap within the script itself if you encounter any unsupported USFM tags.<ref>e.g. We recently encountered the as then undocumented use of 'nested' tags with the syntax '''\+tag_...\+tag*''' <BR>These are now documented in USFM Reference 2.4</ref>

'''Note:'''
<references />

=== Other osis-converters ===
==== John Austin ====
[https://github.com/johnaustindev/osis-converters osis-converters] is the location for the open source (U)SFM-to-OSIS and OSIS-to-SWORD module converters originally developed independently by the main programmer of [https://github.com/JohnAustinDev/xulsword xulsword]. He now uses his own maintained version of usfm2osis.py and has commit privilege to the one maintained for CrossWire by [[User:refdoc|RefDoc]]. ''See above''.

==== Aydeth ====
Aydeth has developed a Python script that is ostensibly faster than CrossWire's usfm2osis.py<BR>
Visit https://github.com/adyeths/u2o

==== Haiola ====
Michael Johnson's [http://haiola.org/ Haiola] Windows software includes the conversion from USFM to OSIS as one of its features.

=== usfm2osis.pl ===
[http://crosswire.org/svn/sword-tools/trunk/modules/perlconverters/usfm2osis.pl usfm2osis.pl] is a simple Perl script intended only for converting USFM files to OSIS. It does not provide comprehensive cover for all the tags in the USFM Reference.

'''Notes:'''
# usfm2osis.pl is now '''deprecated'''. Please use the Python script instead. ''See above''.
# Its output was geared towards the use of OSIS documents in preparing to make SWORD modules.
# usfm2osis.pl generates OSIS with the '''milestone''' form of both chapter and verse elements.
# usfm2osis.pl is no longer being actively maintained by CrossWire.

==== Usage ====
usfm2osis.pl requires a Perl interpreter in the system path. Then you can run:
perl usfm2osis.pl <osisWork> [-o OSIS-file] [-e USFM encoding] <USFM filenames|wildcard>

If usfm2osis.pl is not in the current directory, use its full path.

osisWork should be a value such as Bible.en.WEB.2007.

If you include an OSIS-file value, the output will be written there. Otherwise, it will be written to a file name based on your osisWork.

The USFM encoding argument should indicate the character encoding found in the source files. If none is given, UTF-8 is the default. The list of available encodings depends on your system. Executing the script with no arguments will print the list (as will executing it with an invalid encoding value).

The final argument is a list of filenames or a wildcard value such as '''*.sfm''' containing the SFM data.

'''Notes:'''
# In Windows, a wildcard as one of the <tt>usfm2osis.pl</tt> parameters does not work. This is due to the way the Windows command shell (cmd.exe) does not expand the wildcard parameter. If you have [http://cygwin.com/ CygWin] installed, the bash shell solves this problem, because it expands the wildcard filename specifier.
# In Windows, the following command line syntax will create an OSIS file for each SFM file. This assumes the current directory is the folder where the SFM files are stored, and that there is a same-level directory called osis. It also illustrates the use of (subsititute) drive p: for the full path to usfm2osis.pl, and omits the USFM encoding parameter merely for clarity. The additional use of %f within the output filenames is to give each OSIS XML file a unique name.

for %f in (*.SFM) do perl p:\usfm2osis.pl <osisWork> -o ..\osis\<osisWork>.%f.osis.xml %f

Similarly in CygWin or another bash environment you can use

for f in $(ls *.SFM); do perl usfm2osis.pl <osisWork> -o ../osis/<osisWork>.$f.osis.xml $f; done

==== Unsupported Markers ====

Not all USFM tags are currently supported, just those so far seen "in the wild" by the maintainers of the script. The comments of the script document the current set of tags coverage. In view of the many omissions, it is advised to use the new Python script instead.

=== Fixing the OSIS XML files ===
This section attempts to address some of the current deficiencies in SWORD, and lists tools to workaround these issues. Eventually (it is hoped) these will not be necessary, as the fixes will be implemented within usfm2osis.py during its further development, or as fixes are made to the SWORD engine and/or front-ends, as appropriate.

==== OSIS hacks ====
OSIS 2.1.1 still lacks some desirable features or improvements. While these are not yet forthcoming, sometimes it may be necessary to implement some hacks to the OSIS generated by scripted OSIS converters to workaround these "deficiencies" in the OSIS schema. You can read about OSIS change requests (or judiciously add to the page) [[OSIS 211 CR|here]].

==== Cross-references ====
<tt>usfm2osis.py</tt> and <tt>usfm2osis.pl</tt> do not produce [[OSIS Bibles#Marking_cross-references_notes|cross-references]] (or [[OSIS Bibles#Marking_parallel_passage_headings|parallel passage headings]]) that work properly as links with SWORD.<ref>A solution for this is envisaged in the roadmap for <tt>usfm2osis.py</tt></ref> If your text contains such, you need <tt>xreffix.pl</tt> which sits in directory [https://crosswire.org/svn/sword-tools/trunk/modules/crossreferences/]. Using it requires the SWORD Perl bindings be installed and a [[DevTools:Locale Files|sword-locale]]<ref>If the locale file has any errors or omissions, some references may end up with an incorrect osisRef. Thorough checking of the converted file is paramount.</ref> for the language of the module to be created.<ref>Special care is required when the cross-reference (or heading) includes additional text which is not a scripture reference. Until all these issues are fixed, it becomes almost impossible to parse the reference elements correctly such that the OSIS references can be added automatically.</ref>

'''Note:'''
<references />

==== Miscellaneous ====
Many texts in non-Latin scripts (and even for some Latin scripts) require batch conversion of characters and numbers. There are several scripts to assist with this, without harming the USFM or OSIS markup.<ref>These supplementary Perl scripts are located in various sub-directories under [https://crosswire.org/svn/sword-tools/trunk/modules/ sword-tools].</ref><ref>This has little to do with converting USFM to OSIS</ref>

'''Notes:'''
<references/>

=== ParaTExt (export to OSIS) ===

:''This section may require updating''.

Since version 6, [http://paratext.org/ ParaTExt] has a menu option to export to OSIS.

Checking | Paratext 6 Checks | Publishing | Convert USFM to OSIS (Best Practice)

:See [[Paratext to OSIS]] for illustration of this menu option.

This option generates XML files with the following schema definition file referenced.

osisCore.2.0_UBS_SIL_BestPractice.xsd

UBS SIL Best Practice OSIS makes use of several XML attributes that are not used within CrossWire, and some of these are custom (i.e. x-prefix named attributes). An accessible online copy does not seem to be available. Furthermore, the only undated copy of this .xsd file (40KB) that has been obtained includes the following lines:
<pre>

</pre>
Contrast the above with the standard definition file (92KB) used by CrossWire.

http://www.bibletechnologies.net/osisCore.2.1.1.xsd

Notwithstanding the above concerns, based on one sample exported from a Paratext project, it has been verified that replacing line 2 by the similar line output from usfm2osis.pl, such a converted XML file can be validated against the standard definition file.

Nevertheless, further testing (using <tt>osis2mod</tt>) of some OSIS files exported from Paratext revealed that they generated WARNING(NESTING) errors. Some of these were due to the eID milestone for a verse occurring BEFORE the sID milestone for the same verse! Others were due to <tt></note></tt> being placed AFTER the eID milestone for the verse where the note occurs.

Hence I conclude that what Paratext outputs as OSIS is not really "best practice", and would require some improvements to fix issues such as these. However, we know that work on OSIS export from Paratext has virtually stopped - so this may now prove to be of mere historical interest.

The bottom line is that the OSIS export from Paratext is not ready for publishing a SWORD module.

==Creation of publication ready modules from USFM and submission to CrossWire==

=== Checking and validating OSIS files ===
When checking OSIS XML files there are 3 steps:
# Is the OSIS well formed XML ?
# Is the file valid to the defined OSIS schema?<ref>e.g. Using [http://lxml.de/ lxml] or any other suitable means (e.g. the XML Tools plugin for Notepad++ on Windows).</ref>
# Is the file "fit for purpose" (i.e. suitable for immediate use by the SWORD conversion tool called <tt>osis2mod</tt>)
Step #1 does not guarantee step #2 and step #2 does not guarantee step #3. See above and below and [[OSIS Bibles#Limitations_of_XML_validators|limitations of XML validators]].

Before you convert your OSIS files to SWORD format, you should always check that it is valid OSIS.<BR>Before you submit any files to modules@crosswire.org, you '''must''' ensure that your files are valid OSIS. Invalid OSIS files will not be accepted.

'''Note:'''
<references />

=== Module creation process ===
:''This section requires updating to reflect the move to using the Python script <tt>usfm2osis.py</tt> in preference to its Perl predecessor''.

Our process to create publication ready modules from USFM looks like this:

==== Stage 1 ====
''The command line examples in this section are for Unix. In Windows, it's not as simple to make equivalent command lines for Perl, especially in regard to [http://en.wikipedia.org/wiki/Glob_%28programming%29 globbing] wildcards for filenames''.
*Iterate <tt>usfm2osis.pl</tt> over each single USFM file to produce a OSIS file per USFM file
for biblebook in 'ls *usfm'; do usfm2osis.pl $biblebook $biblebook; done
*Check each OSIS file for XML validity - this will throw up a lot of not obvious USFM encoding problems
for biblebook in 'ls *osis.xml'; do checkbible $biblebook; done
*Correct USFM files as per mistakes found

Rerun this stage until you have clean and validating OSIS files.

==== Stage 2 ====

# Check the OSIS files for not transformed USFM markers.<ref>Not many common ones are left unsupported, but you may encounter usually one or two new ones if you get files from a different new source.</ref><ref>Some supported USFM markers may remain unconverted due to anomalies in how the USFM files were edited.<br>Paratext does not check all potential mismatches against the USFM reference manual, and moreover, the manual itself leaves room for some ambiguities.</ref>
# Fix <tt>usfm2osis.pl</tt> and add the missing tags for correct OSIS transformation.
# Rerun above until you have a clean validating collection of OSIS files with no left over USFM tags.<ref>A lot of the repetitive aspects can be done with the help of some shell scripts, so do not worry that you have to run each time multiples of <tt>usfm2osis.pl</tt> by hand.</ref>
# Send the updates to usfm2osis.pl to CrossWire's SVN repository.
# Run <tt>usfm2osis.pl</tt> over all the USFM files and create a single large OSIS file.
# Check again for XML validity.<ref>See [[OSIS_Bibles#Valid_OSIS_test|OSIS Bible validation]] for further instructions on OSIS validation.</ref>
# Run the OSIS file through <tt>osis2mod</tt><ref>Instructions for running <tt>osis2mod</tt> are available at [[osis2mod#Usage|osis2mod usage]].</ref>, create a [[DevTools:conf Files|conf file]], and check the resultant module for problems in a variety of front-ends.<ref>Fixes should be made in the USFM files at the very start and then cascaded down into the module following the overall process again.</ref>
# Create a [[DevTools:SWORD#Locale_file_layout|sword locale file]] and store in your <tt>/usr/share/sword/locales.d/</tt> directory<ref>or the equivalent directory on your system</ref>
# If cross references are part of the USFM files fix these with the help of <tt>xreffix.pl</tt> (also in sword-tools).<ref>This requires a sword locale in the local installation at least.</ref>
# Run again through <tt>osis2mod</tt> and check again.
# Submit the final OSIS file and the conf file to CrossWire together with the locale file.<ref>Include advice regarding which [[Module Repositories|repository]] the module is intended for.</ref>
# Send any corrections on the USFM files back to the translation team. Chances are these are all valid and necessary corrections which help to improve the USFM.<ref>Both Paratext and Bibledit are a bit more forgiving than our tools, but some problems you encounter could have affected the final paper print.</ref>

'''Notes:'''
<references />

[[Category:Guides|Converting SFM Bibles to OSIS]]
[[Category:OSIS]]
[[Category:USFM]]
[[Category:Bibledit]]
[[Category:Paratext]]

User:Osk

2014-04-23T10:43:05Z

Osk:

[[Category:Users with a user page|O]]

User:Osk/Todo

2014-04-23T10:39:56Z

Osk:

== Module Tasks ==

=== Todo List ===
# MorphLXX
# Clear the rest of this list of completed items :D
# Clear module [http://www.crosswire.org/bugs/browse/MOD bugs]
# Clear beta [[Modules_in_the_beta_repository| list]]
# Remake JFB [[http://www.crosswire.org/bugs/browse/MOD-54 MOD-54]]
# Look into ZhEnglish [[http://www.crosswire.org/bugs/browse/MOD-61 MOD-61]] -- maybe unfixable
# Remake ASV, Codex Marianus, OrthJBC,
# KLV... needs work... inline notes
# Reimport: Alb, Aleppo, ArmZorhab, DRA, Diaglott, Geneva1599, Geo, GerElb1871, HebModern, ISV, ItaDio, ItaRive, LXX, Murdock, Peshitta, RomCor, Rotherham, RusMakarij, UMGreek
# Fix headings: DutSVV
# Morphology & associated modules
# Turkish glossary (others?) < Ergane,FD
# Lewis?
# Schaff content -pp
# transition TEI P4 modules to TEI P5 (Perseus, GermLex content)
# zLD linking support(?)
# Other English-language TEI dictionaries (Wi)
# TEI glossaries
# Gothic lexicon
# Wiki Bibles (x3?)
# Luther 22/30/46
# Look for new source of BurJudson with linked verses (or forward-link through empty vv); BurJudson Unicode 5.1 update?
# Remake Bible21 from http://online.bible21.cz/ (include paragraphs, poetic markup, titles, (etc.?))
# Think about further development of usfm2osis.py

=== Holding, pending resubmission or such ===
# pending new osis2mod (have linked verses): Tamajaq, GerLut1545lh, K&D, Clarke, Amharic, MHC ([[http://www.crosswire.org/bugs/browse/MOD-65 MOD-65]] [[http://www.crosswire.org/bugs/browse/MOD-52 MOD-52]] [[http://www.crosswire.org/bugs/browse/MOD-48 MOD-48]] [[http://www.crosswire.org/bugs/browse/MOD-34 MOD-34]])
# pending new osis2mod (have word of Christ): ABU, KJC, Worsley (rename)
# pending new osis2mod (pre-verse content problems): GerLut1912

=== Modules needing occasional updates ===
* [http://sourceforge.net/projects/structio/ SpaTDP]
* [http://www.ebible.org/web/ WEB]
* [http://www.ebible.org/hnv/ HNV]
* [http://www.freie-bibel.de/official/bibel/elberfelder/ GerElb1871r]
* [http://kimkat.org/amryw/1_testunau/sion_prys_003_beibl_mynegai_1281k.htm WelMorgan]
* [http://www.biblebc.com/Irish/IrishBible/status.htm IriODomhnuill] : [http://www.biblebc.com/Projects/irish_new_testament_project.htm main page]
* [http://vulsearch.sourceforge.net/plain.html Vulgate_Clem]
* [http://www.offene-bibel.de/ Offene Bibel]
* [http://cherokeenewtestament.com/ Cherokee New Testament] / Last checked 2013-10-10 (site down)
* [http://www.sacredbible.org/errata.htm CPDV]
* [https://github.com/openscriptures/morphhb/tree/master/wlc OSHB] / Last updated 2013-10-11
* [http://tanach.us/Books/ WLC] / Last updated 2013-10-11

=== Modules needing v11n rebuilds (post JSword av11n-compatibility) ===
* FinPR92, FinPR(?), FinBiblia(?) (>German?)
* HinERV (>NRSV)
* UrduGeo (>NRSV)
* UrduGeoDeva (>NRSV)

==Programming Tasks==

===Sword===
# UTF8HebrewVowels --> UTF8AbjadVocalization (merge with UTF8ArabicVowels? or is this task no longer necessary?)
# Make *LD linked entries work

===Utils===
# Make imp2gbs not crash
# Make tei2mod better (debug/commit memory leak fix & linked entry handling)

===ICU tasks===
# switch regex to ICU regex
# submit transliterators to CLDR

File Formats

2014-03-20T07:00:06Z

Osk: old linkspam

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, TEI, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===TEI P5===
''Text Encoding Initiative''

TEI is an text encoding standard used broadly in the humanities. It is developed by the [http://www.tei-c.org/ TEI Consortium].

A manual for the latest edition of TEI P5 can be found at [http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ P5: Guidelines for Electronic Text Encoding and Interchange].

CrossWire supports use of TEI for the encoding of dictionaries, encyclopedias, and similar works, as well as for free form books (GenBooks). We have developed our own version of the TEI P5 schema, available at http://www.crosswire.org/osis/teiP5osis.latest.xsd (http://www.crosswire.org/osis/teiP5osis.2.5.0.xsd is the latest at the time of writing). CrossWire's additions to the standard schema include the ability to create osisRefs and osisIDs to facilitate interoperability with OSIS documents and Bible reference tagging.

Further information can be found at the [[TEI Dictionaries]] page.

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

===ThML===
''Theological Markup Language'' '''(deprecated)'''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format'' '''(deprecated)'''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], and [http://ebible.org/bible/web/websf.zip WEB] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by QuickVerse and WORDsearch, and is currently used for some e-Sword books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. The SWORD Project will not support this dead format.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===USX===
'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T10:22:38Z

Osk: /* Compiling utilities */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Ensure that the Win32 Release configuration is selected and build the solution

=== Compiling zlib ===
# Download the latest release of [http://www.zlib.net/ zlib] source code. At writing, the latest version is 1.2.8: [http://prdownloads.sourceforge.net/libpng/zlib128.zip?download]
# Extract the archive to $dir$ and rename the directory "zlib" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/zlib
# Execute: nmake /f win32/Makefile.msc

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory "bzip2"
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Ensure that the Release configuration is selected and build the solution

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling CLucene ===
# Install [http://www.cmake.org/ cmake]
# Download the latest version of [http://www.boost.org/users/download/ boost]. At writing, the latest version is 1.5.5: [http://sourceforge.net/projects/boost/files/boost/1.55.0/]
# Extract the archive to $dir$
# Rename the boost directory to "boost"
# Download a snapshot of the [http://sourceforge.net/p/clucene/code/ci/master/tree/ CLucene 2_3_2 branch]
# Extract the archive to $dir$
# Rename the CLucene directory to "clucene"
# Start a regular Command Prompt (cmd.exe)
# cd to $dir$/clucene
# Set Boost environment variables by executing the following (with replacements for $dir$):
:: set BOOST_BUILD_PATH=$dir$\boost\tools\build\v2
:: set BOOST_PATH=$dir$
:: set BOOST_ROOT=$dir$\boost
# Execute: mkdir cmakebuild
# Execute: cd cmakebuild
# Execute: cmake -G "Visual Studio 12" ..
# Open $dir$\clucene\cmakebuild\clucene.sln
# Ensure that the Release configuration is selected and build the BUILD_ALL project

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project] or download a copy from SVN: https://www.crosswire.org/svn/sword/trunk/
# Extract the tarball to $dir$
::If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\libsword.sln
# Build the solution (Release, Debug, or both)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build the solution (Release, Debug, or both)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T10:00:21Z

Osk: /* Compiling libsword */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Ensure that the Win32 Release configuration is selected and build the solution

=== Compiling zlib ===
# Download the latest release of [http://www.zlib.net/ zlib] source code. At writing, the latest version is 1.2.8: [http://prdownloads.sourceforge.net/libpng/zlib128.zip?download]
# Extract the archive to $dir$ and rename the directory "zlib" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/zlib
# Execute: nmake /f win32/Makefile.msc

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory "bzip2"
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Ensure that the Release configuration is selected and build the solution

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling CLucene ===
# Install [http://www.cmake.org/ cmake]
# Download the latest version of [http://www.boost.org/users/download/ boost]. At writing, the latest version is 1.5.5: [http://sourceforge.net/projects/boost/files/boost/1.55.0/]
# Extract the archive to $dir$
# Rename the boost directory to "boost"
# Download a snapshot of the [http://sourceforge.net/p/clucene/code/ci/master/tree/ CLucene 2_3_2 branch]
# Extract the archive to $dir$
# Rename the CLucene directory to "clucene"
# Start a regular Command Prompt (cmd.exe)
# cd to $dir$/clucene
# Set Boost environment variables by executing the following (with replacements for $dir$):
:: set BOOST_BUILD_PATH=$dir$\boost\tools\build\v2
:: set BOOST_PATH=$dir$
:: set BOOST_ROOT=$dir$\boost
# Execute: mkdir cmakebuild
# Execute: cd cmakebuild
# Execute: cmake -G "Visual Studio 12" ..
# Open $dir$\clucene\cmakebuild\clucene.sln
# Ensure that the Release configuration is selected and build the BUILD_ALL project

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project] or download a copy from SVN: https://www.crosswire.org/svn/sword/trunk/
# Extract the tarball to $dir$
::If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\libsword.sln
# Build the solution (Release, Debug, or both)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T09:55:34Z

Osk: added zlib build instructions

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Ensure that the Win32 Release configuration is selected and build the solution

=== Compiling zlib ===
# Download the latest release of [http://www.zlib.net/ zlib] source code. At writing, the latest version is 1.2.8: [http://prdownloads.sourceforge.net/libpng/zlib128.zip?download]
# Extract the archive to $dir$ and rename the directory "zlib" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/zlib
# Execute: nmake /f win32/Makefile.msc

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory "bzip2"
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Ensure that the Release configuration is selected and build the solution

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling CLucene ===
# Install [http://www.cmake.org/ cmake]
# Download the latest version of [http://www.boost.org/users/download/ boost]. At writing, the latest version is 1.5.5: [http://sourceforge.net/projects/boost/files/boost/1.55.0/]
# Extract the archive to $dir$
# Rename the boost directory to "boost"
# Download a snapshot of the [http://sourceforge.net/p/clucene/code/ci/master/tree/ CLucene 2_3_2 branch]
# Extract the archive to $dir$
# Rename the CLucene directory to "clucene"
# Start a regular Command Prompt (cmd.exe)
# cd to $dir$/clucene
# Set Boost environment variables by executing the following (with replacements for $dir$):
:: set BOOST_BUILD_PATH=$dir$\boost\tools\build\v2
:: set BOOST_PATH=$dir$
:: set BOOST_ROOT=$dir$\boost
# Execute: mkdir cmakebuild
# Execute: cd cmakebuild
# Execute: cmake -G "Visual Studio 12" ..
# Open $dir$\clucene\cmakebuild\clucene.sln
# Ensure that the Release configuration is selected and build the BUILD_ALL project

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
::If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T09:45:15Z

Osk: /* Compiling ICU */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Ensure that the Win32 Release configuration is selected and build the solution

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Ensure that the Release configuration is selected and build the solution

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling CLucene ===
# Install [http://www.cmake.org/ cmake]
# Download the latest version of [http://www.boost.org/users/download/ boost]. At writing, the latest version is 1.5.5: [http://sourceforge.net/projects/boost/files/boost/1.55.0/]
# Extract the archive to $dir$
# Rename the boost directory to "boost"
# Download a snapshot of the [http://sourceforge.net/p/clucene/code/ci/master/tree/ CLucene 2_3_2 branch]
# Extract the archive to $dir$
# Rename the CLucene directory to "clucene"
# Start a regular Command Prompt (cmd.exe)
# cd to $dir$/clucene
# Set Boost environment variables by executing the following (with replacements for $dir$):
:: set BOOST_BUILD_PATH=$dir$\boost\tools\build\v2
:: set BOOST_PATH=$dir$
:: set BOOST_ROOT=$dir$\boost
# Execute: mkdir cmakebuild
# Execute: cd cmakebuild
# Execute: cmake -G "Visual Studio 12" ..
# Open $dir$\clucene\cmakebuild\clucene.sln
# Ensure that the Release configuration is selected and build the BUILD_ALL project

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
::If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T09:44:44Z

Osk: /* Compiling ICU */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Ensure that the Release configuration is selected and build the solution

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Ensure that the Release configuration is selected and build the solution

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling CLucene ===
# Install [http://www.cmake.org/ cmake]
# Download the latest version of [http://www.boost.org/users/download/ boost]. At writing, the latest version is 1.5.5: [http://sourceforge.net/projects/boost/files/boost/1.55.0/]
# Extract the archive to $dir$
# Rename the boost directory to "boost"
# Download a snapshot of the [http://sourceforge.net/p/clucene/code/ci/master/tree/ CLucene 2_3_2 branch]
# Extract the archive to $dir$
# Rename the CLucene directory to "clucene"
# Start a regular Command Prompt (cmd.exe)
# cd to $dir$/clucene
# Set Boost environment variables by executing the following (with replacements for $dir$):
:: set BOOST_BUILD_PATH=$dir$\boost\tools\build\v2
:: set BOOST_PATH=$dir$
:: set BOOST_ROOT=$dir$\boost
# Execute: mkdir cmakebuild
# Execute: cd cmakebuild
# Execute: cmake -G "Visual Studio 12" ..
# Open $dir$\clucene\cmakebuild\clucene.sln
# Ensure that the Release configuration is selected and build the BUILD_ALL project

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
::If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T09:43:30Z

Osk: /* Compiling CLucene */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Ensure that the Release configuration is selected and build the solution

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling CLucene ===
# Install [http://www.cmake.org/ cmake]
# Download the latest version of [http://www.boost.org/users/download/ boost]. At writing, the latest version is 1.5.5: [http://sourceforge.net/projects/boost/files/boost/1.55.0/]
# Extract the archive to $dir$
# Rename the boost directory to "boost"
# Download a snapshot of the [http://sourceforge.net/p/clucene/code/ci/master/tree/ CLucene 2_3_2 branch]
# Extract the archive to $dir$
# Rename the CLucene directory to "clucene"
# Start a regular Command Prompt (cmd.exe)
# cd to $dir$/clucene
# Set Boost environment variables by executing the following (with replacements for $dir$):
:: set BOOST_BUILD_PATH=$dir$\boost\tools\build\v2
:: set BOOST_PATH=$dir$
:: set BOOST_ROOT=$dir$\boost
# Execute: mkdir cmakebuild
# Execute: cd cmakebuild
# Execute: cmake -G "Visual Studio 12" ..
# Open $dir$\clucene\cmakebuild\clucene.sln
# Ensure that the Release configuration is selected and build the BUILD_ALL project

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
::If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T09:42:28Z

Osk: /* Compiling bzip2 */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Ensure that the Release configuration is selected and build the solution

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling CLucene ===
# Install [http://www.cmake.org/ cmake]
# Download the latest version of [http://www.boost.org/users/download/ boost]. At writing, the latest version is 1.5.5: [http://sourceforge.net/projects/boost/files/boost/1.55.0/]
# Extract the archive to $dir$
# Rename the boost directory to "boost"
# Download a snapshot of the [http://sourceforge.net/p/clucene/code/ci/master/tree/ CLucene 2_3_2 branch]
# Extract the archive to $dir$
# Rename the CLucene directory to "clucene"
# Start a regular Command Prompt (cmd.exe)
# cd to $dir$/clucene
# Set Boost environment variables by executing the following (with replacements for $dir$):
:: set BOOST_BUILD_PATH=$dir$\boost\tools\build\v2
:: set BOOST_PATH=$dir$
:: set BOOST_ROOT=$dir$\boost
# Execute: mkdir cmakebuild
# Execute: cd cmakebuild
# Execute: cmake -G "Visual Studio 12" ..
# Open $dir$\clucene\cmakebuild\clucene.sln
# Ensure that the Release configuration is selected and build the BUILD_ALL solution

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
::If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T09:41:13Z

Osk: added CLucene build instructions

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Visual Studio 2013 | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling CLucene ===
# Install [http://www.cmake.org/ cmake]
# Download the latest version of [http://www.boost.org/users/download/ boost]. At writing, the latest version is 1.5.5: [http://sourceforge.net/projects/boost/files/boost/1.55.0/]
# Extract the archive to $dir$
# Rename the boost directory to "boost"
# Download a snapshot of the [http://sourceforge.net/p/clucene/code/ci/master/tree/ CLucene 2_3_2 branch]
# Extract the archive to $dir$
# Rename the CLucene directory to "clucene"
# Start a regular Command Prompt (cmd.exe)
# cd to $dir$/clucene
# Set Boost environment variables by executing the following (with replacements for $dir$):
:: set BOOST_BUILD_PATH=$dir$\boost\tools\build\v2
:: set BOOST_PATH=$dir$
:: set BOOST_ROOT=$dir$\boost
# Execute: mkdir cmakebuild
# Execute: cd cmakebuild
# Execute: cmake -G "Visual Studio 12" ..
# Open $dir$\clucene\cmakebuild\clucene.sln
# Ensure that the Release configuration is selected and build the BUILD_ALL solution

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
::If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T08:27:14Z

Osk: /* Compiling cURL */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/curl/winbuild
# Execute: nmake /f Makefile.vc mode=static DEBUG=no

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
#:If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T08:24:05Z

Osk: /* Compiling ICU */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Follow the directions in winbuild/BUILD.WINDOWS.txt
## You need to build a static version of cURL
## Whether you build a debug or release version of libsword, you will need to make a release version of cURL (DEBUG=no)
## Sample command "nmake /f Makefile.vc mode=static DEBUG=no"

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]. At writing, the latest version is 52.1: [http://download.icu-project.org/files/icu4c/52.1/icu4c-52_1-src.zip]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
#:If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T08:22:16Z

Osk: /* Compiling bzip2 */ I just built bzip2 in VS2013 and didn't need to use the console (VS updated the .dsp project)

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Follow the directions in winbuild/BUILD.WINDOWS.txt
## You need to build a static version of cURL
## Whether you build a debug or release version of libsword, you will need to make a release version of cURL (DEBUG=no)
## Sample command "nmake /f Makefile.vc mode=static DEBUG=no"

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studio to update the project
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
#:If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T08:21:11Z

Osk: /* Compiling bzip2 */ wrong version

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Follow the directions in winbuild/BUILD.WINDOWS.txt
## You need to build a static version of cURL
## Whether you build a debug or release version of libsword, you will need to make a release version of cURL (DEBUG=no)
## Sample command "nmake /f Makefile.vc mode=static DEBUG=no"

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.0.6: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studo to update the project
## If you are using Visual Studio 2013 you need to open a VS command line
## cd $dir$\bzip2
## nmake /f makefile.msc
## mkdir Release
## move libbz2.lib Release
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
#:If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T08:16:54Z

Osk: /* Compiling icu-sword */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Follow the directions in winbuild/BUILD.WINDOWS.txt
## You need to build a static version of cURL
## Whether you build a debug or release version of libsword, you will need to make a release version of cURL (DEBUG=no)
## Sample command "nmake /f Makefile.vc mode=static DEBUG=no"

=== Compiling ICU ===
# Download the latest release of [http://site.icu-project.org/download ICU]
# Extract the tarball to $dir$
# Open $dir$\icu\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.6.0: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studo to update the project
## If you are using Visual Studio 2013 you need to open a VS command line
## cd $dir$\bzip2
## nmake /f makefile.msc
## mkdir Release
## move libbz2.lib Release
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
#:If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-19T06:27:46Z

Osk: Let's just cease support for pre-VS2013; our own project files are for VS2013

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2013 ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Follow the directions in winbuild/BUILD.WINDOWS.txt
## You need to build a static version of cURL
## Whether you build a debug or release version of libsword, you will need to make a release version of cURL (DEBUG=no)
## Sample command "nmake /f Makefile.vc mode=static DEBUG=no"

=== Compiling icu-sword ===
# Download the latest release of [http://crosswire.org/ftpmirror/pub/sword/source/icu-sword/ icu-sword]
# Extract the tarball to $dir$
# Open $dir$\icu-sword\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.6.0: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studo to update the project
## If you are using Visual Studio 2013 you need to open a VS command line
## cd $dir$\bzip2
## nmake /f makefile.msc
## mkdir Release
## move libbz2.lib Release
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
#:If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-14T07:36:32Z

Osk: /* Compiling The SWORD Project using MS Visual Studio 2008+ */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2008+ ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Follow the directions in winbuild/BUILD.WINDOWS.txt

=== Compiling icu-sword ===
# Download the latest release of [http://crosswire.org/ftpmirror/pub/sword/source/icu-sword/ icu-sword]
# Extract the tarball to $dir$
# Open $dir$\icu-sword\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===
# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.6.0: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studo to update the project
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===
# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
#:If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===
# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-04T15:32:54Z

Osk: /* Downloading liblzma */

:''For compiling on Linux, see [[Tutorial:Compiling & Installing SWORD]]''.

== Compiling The SWORD Project using [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio MS Visual Studio] 2008+ ==

# Create a build directory, hereafter referred to as $dir$. Much of the Sword build process expects that various downloads be unpacked to the same directory.

=== Compiling cURL ===
# Download the latest release of [http://curl.haxx.se/latest.cgi?curl=zip cURL]
# Extract the ZIP to $dir$
# Rename the cURL directory to "curl" (with no version number)
# Follow the directions in winbuild/BUILD.WINDOWS.txt

=== Compiling icu-sword ===

# Download the latest release of [http://crosswire.org/ftpmirror/pub/sword/source/icu-sword/ icu-sword]
# Extract the tarball to $dir$
# Open $dir$\icu-sword\source\allinone\allinone.sln
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling bzip2 ===

# Download the latest release of [http://www.bzip.org/downloads.html bzip2]. At writing, the latest version is 1.6.0: [http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz]
# Extract the archive to $dir$ and rename the directory bzip2
# Open $dir$\bzip2\libbz2.dsp and allow Visual Studo to update the project
# Build all (Click Build | Batch Build... | Select All | Build)

=== Downloading liblzma ===

# Download the latest release of xz prebuilt for Windows from [http://tukaani.org/xz/ tukaani.org]. At writing, the latest version is 5.0.5: [http://tukaani.org/xz/xz-5.0.5-windows.zip]
# Extract the archive to $dir$/xz
# Start a Developer Command Prompt (find this on your Start Menu in the Microsoft Visual Studios | Visual Studio Tools menu)
# cd to $dir$/xz/bin_i486
# Execute: lib /def:../doc/liblzma.def /out:liblzma.lib /machine:x86

=== Compiling libsword ===
# Download the latest release of [http://www.crosswire.org/ftpmirror/pub/sword/source/sword.tar.gz The SWORD Project]
# Extract the tarball to $dir$
#:If you are building utilities or diatheke, the following steps are not necessary:
# Open $dir$\sword-$version$\lib\vcppmake\vc8\libsword.vcproj
# Build all (Click Build | Batch Build... | Select All | Build)

=== Compiling utilities ===

# Open $dir$\sword-$version$\utilities\vcppmake\utilities.sln
# Build all (Click Build | Batch Build... | Select All | Build)

== Cross Compiling Sword for MS Windows on Suse Linux with MingW ==

===Suse with MingW===

Installation of all necessary dependencies of Sword, but for CLucene - libcurl, ICU, libgnurx etc -

===Compilation of Clucene===

clucene 0.9.21b source

The scripts for compiling sword and clucene in xiphos/win32/suse are meant to be used from ~/source/ with the clucene and sword directories directly under that. I would suggest reading the scripts. They aren't complicated, and I'm sure that I made assumptions about my environment (which is basically the *sole* reason for the warning that they aren't ready, I never meant they couldn't be used). You can go ahead and try to use them to cross-compile clucene. It will probably fail. If at that point, you send me config.log, that will help me remember what to do to fix it.

--------------------------------

edit m4/dps_static_const_type.m4, the last AC_DEFINE line. Change from:

AC_MSG_ERROR([Cannot figure out how to write static consts in classes. Check the m4 script or upgrade your compiler])

to:

AC_DEFINE([LUCENE_STATIC_CONSTANT_SYNTAX], 1, [How to define a static const in a class])

eg, the same as the first AC_DEFINE. I have no idea if it's correct, actually, but it seems to work ;)

Next, edit src/Makefile.am, line 6. Add <tt>-no-undefined</tt> at the end, like so:

libclucene_la_LDFLAGS = --version-info $(CLLIB_VERSION) --release
$(PACKAGE_VERSION) -no-undefined

------------------------
src/CLucene/store/FSDirectory.cpp

At the top of that file, after the #include directives, I just added this:

#define _mkdir mkdir

After changing these, you'll have to run autogen.sh and then re-configure using my script. It *should* build after this, at which point run sudo make install. Check to be sure it created a dll rather than just a static library (which will end in .la). It should be located in /usr/i686-w64-mingw32/sys-root/mingw/bin

After this Clucene compiles without errors and creates a working dll.

===Compilation of Sword===

Application of 2 patches:

patch1 patch2

[[Category:Tutorials]]

Tutorial:Compiling & Installing SWORD on Windows

2014-03-04T14:34:01Z

Osk: /* Downloading liblzma */

Tutorial:Compiling & Installing SWORD on Windows

2014-03-04T14:33:37Z

Osk: /* Compiling The SWORD Project using MS Visual Studio 2008+ */

Tutorial:Compiling & Installing SWORD on Windows

2014-03-04T13:34:55Z

Osk: /* Compiling libsword */

Tutorial:Compiling & Installing SWORD on Windows

2014-03-04T13:34:10Z

Osk: /* Compiling libsword */

Tutorial:Compiling & Installing SWORD on Windows

2014-03-04T13:33:55Z

Osk: /* Compiling The SWORD Project using MS Visual Studio 2008+ */

Tutorial:Compiling & Installing SWORD on Windows

2014-03-04T13:26:19Z

Osk: /* Compiling The SWORD Project using MS Visual Studio 2008 */

OSIS 211 CR

2014-02-23T08:04:24Z

Osk: /* Feature requests */

This page is for recording potential change requests to the OSIS XML schema.

== OSIS 2.1.1 Change Requests ==
Anyone with an outstanding OSIS bug report or feature proposal for consideration for inclusion into an updated OSIS schema, please write a very concise change request here in this page, including motivating use case.

== Bugs ==

=== osisGenRegex bug ===
Currently that regex looks like [1], but it should looks like [2]:

<pre>
[1] ((((\p{L}|\p{N}|_)+)(\.(\p{L}|\p{N}|_))*:)?([^:\s])+)
[2] ((((\p{L}|\p{N}|_)+)(\.(\p{L}|\p{N}|_)+)*:)?([^:\s])+)
(missing + right here ^)
</pre>

So our document with the following element isn't valid because the
string "Strong" cannot be more than 1 character long in the current schema:
<w morph="robinson:N-NSF" lemma="lemma.Strong:βίβλος">βίβλος</w>

--[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== milestoned <lg> ===

Since the <l> element can ''only'' occur within an <lg> element, use of milestoned <lg> prevents use of <l> elements (within that <lg>). Since <lg> is milestonable, one would presume that the following snippet would be valid, but it is not, for the above reason:
<pre>
<lg sID="eg1"/>
<l>Poetry line</l>
<l>Poetry line</l>
<lg eID="eg1"/>
</pre>
--[[User:Osk|Osk]] 18:18, 31 December 2011 (MST)

The <lg> element does not allow for mixed content. However the use of the milestoned <lg> wrongly allows for it.
<pre>
<lg sID="eg2"/>
text
<lg eID="eg2"/>
</pre>
--[[User:Dmsmith|Dmsmith]] 16:29, 14 October 2012 (MDT)

=== <closer> in <verse> container? ===

According to the OSIS manual (cf. 11.1.3 on p. 58), it should be possible to embed a <closer> element within a <verse> container, but the schema does not allow this. One or the other should be corrected.
--[[User:Osk|Osk]] 05:56, 6 July 2012 (MDT)

=== <seg> in <cell> ===

This was already reported to osis-users, but for the sake of completeness:
There's a typo that allows "seq" in <cell> instead of "seg".
--[[User:Osk|Osk]] 04:29, 22 February 2014 (MST)

== Feature requests ==

=== Allow <transChange> within <w> ===
An encoder ought to be ''allowed'' to put <transChange> on elements smaller than an orthographic word. If I'm translating an instance of "λόγος", but for some reason I believe that I should translate it as "words", I ought to be able to encode <w>word<transChange>s</transChange></w>. --[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== Add an element for morphology within <w> ===
Necessary for encoding documents like MORPH (WLC + morphology), we need an element to embed within <w> to carry lexical information. I suggest calling it <m> and giving it all of the attributes found on <w>. --[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== Allow <transChange> within <hi> ===
A highlighted sentence or part of a sentence is a unit, including any transChange parts of it.
At the moment a highlighted sentence with a transChange will look like this:
<hi type="bold"> Texttexttext </hi><transChange><hi type="bold"> moreText</hi></transChange><hi type="bold"> TextText</hi>

<hi type="bold"> Texttexttext <transChange>moreText</transChange> TextText</hi>

This would look cleaner and would be also closer to what is meant. [[User:Refdoc|refdoc]]:[[User_Talk:Refdoc|talk]] 16:02, 3 August 2011 (MDT)

=== Allow <catchWord> within <hi> ===
A highlighted sentence or part of a sentence is a unit, including any catchWord parts of it.
At the moment a highlighted sentence with a catchWord will look like this:
<hi type="bold"> Texttexttext </hi><catchWord><hi type="bold"> moreText</hi></catchWord><hi type="bold"> TextText</hi>

<hi type="bold"> Texttexttext <catchWord>moreText</catchWord> TextText</hi>
This is identical in form to the <transChange> issue. The problem with both of these is that <transChange> and <catchWord> may reasonably be styled in the same fashion as what is indicated by <hi>.
--[[User:Dmsmith|Dmsmith]] 16:58, 14 October 2012 (MDT)

=== Allow multiple types for <hi> ===
It'd really be convenient for
<hi type="bold italic small-caps">text</hi>
rather than
<hi type="bold"><hi type="italic"><hi type="small-caps">text</hi></hi></hi>
--[[User:Dmsmith|Dmsmith]] 16:57, 14 October 2012 (MDT)

=== Allow <transChange> within <note> ===
When translating an alternate Greek version of a passage, added words need to be indicated.

=== Allow remote header reference ===
When serving short passages via web services, as valid OSIS documents, a full header is obtrusive. Also, in a collection of related documents, for example separate book files for a Bible, one centralized header would be more maintainable. The simplest approach would probably be to allow @href on the header element, to abstract some or all of the header content. See Troy's [https://groups.google.com/d/topic/openscriptures/kygOM0yPHL8/discussion related post].

===Allow shadow/virtual elements===
A second requirement for distributing valid OSIS fragments through web services is a form of virtual, or shadow, element to supply the context of the given fragment. A new global attribute for indicating this virtual status is essential to distinguish them from the actual markup of the document. In the [http://www.esvapi.org/api ESV API], they have this construct via `virtual` attribute (see description for `include-virtual-attributes``). See Troy's [https://groups.google.com/d/topic/openscriptures/kygOM0yPHL8/discussion related post] (same as previous).

=== Improve Ketiv/Qere markup ===
A ketiv or qere can consist of one or more words, and so need to be grouped and related to one another. I propose adding <ketiv> with @id, and <qere> with @idref, to contain the content (<w> elements) and allow validation of the connection. A qere with no ketiv could be marked up without the @idref.

: This sounds like a good application for <seg>. I would recommend named types for <seg> instead: ketiv & qere. --[[User:Osk|Osk]] 00:37, 23 February 2014 (MST)

=== Improve Selah markup ===
Selah can be represented at the end of a line. The markup of <l type="selah">...</l> does not allow for the text identified as selah to be at the end of the current line. Maybe allow for a separate markup, rather than a type of line.

=== Milestonable <p> ===
For documents where the primary structure is book, chapter, verse, like the Authorized Version or the Hebrew Bible, we should be able to mark up paragraphs as milestones. This would allow for equality, rather than making book, section, paragraph a privileged system.

=== Quotation types ===
From the manual (p. 43): "The rendering for quotations marks after an interruption, for example, can be distinguished using the type attribute on this element, with values such as initial, medial, and final." Please make these @type values official: initial, medial, and final.

=== Add peripheral types from USFM to osisDivs ===
Add the additional USFM peripheral types to osisDivs to maintain feature parity. I believe OSIS 2.1.1 had this feature parity at the time of its release, but USFM has standardized additional peripheral types since then, which should be added as:
halfTitlePage, promotionalPage, foreword, alphabeticalContents, tableofAbbreviations, chronology, weightsandMeasures, mapIndex, ntQuotesfromLXX, spine
--[[User:Osk|Osk]] 01:04, 23 February 2014 (MST)

== Manual bugs & feature requests ==

=== Head Elements ===
The OSIS manual give the "head" element as a means of providing for titles. It is not in the schema as a child of div, but it is in the manual.

=== DivineName Element ===
Manual give type as x-yhwh in 11.5.1.2, perhaps elsewhere. It isn't necessary. It also has the content as LORD, but it should be Lord.

[[Category:OSIS]]

OSIS 211 CR

2014-02-23T07:37:33Z

Osk: /* Improve Ketiv/Qere markup */

This page is for recording potential change requests to the OSIS XML schema.

== OSIS 2.1.1 Change Requests ==
Anyone with an outstanding OSIS bug report or feature proposal for consideration for inclusion into an updated OSIS schema, please write a very concise change request here in this page, including motivating use case.

== Bugs ==

=== osisGenRegex bug ===
Currently that regex looks like [1], but it should looks like [2]:

<pre>
[1] ((((\p{L}|\p{N}|_)+)(\.(\p{L}|\p{N}|_))*:)?([^:\s])+)
[2] ((((\p{L}|\p{N}|_)+)(\.(\p{L}|\p{N}|_)+)*:)?([^:\s])+)
(missing + right here ^)
</pre>

So our document with the following element isn't valid because the
string "Strong" cannot be more than 1 character long in the current schema:
<w morph="robinson:N-NSF" lemma="lemma.Strong:βίβλος">βίβλος</w>

--[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== milestoned <lg> ===

Since the <l> element can ''only'' occur within an <lg> element, use of milestoned <lg> prevents use of <l> elements (within that <lg>). Since <lg> is milestonable, one would presume that the following snippet would be valid, but it is not, for the above reason:
<pre>
<lg sID="eg1"/>
<l>Poetry line</l>
<l>Poetry line</l>
<lg eID="eg1"/>
</pre>
--[[User:Osk|Osk]] 18:18, 31 December 2011 (MST)

The <lg> element does not allow for mixed content. However the use of the milestoned <lg> wrongly allows for it.
<pre>
<lg sID="eg2"/>
text
<lg eID="eg2"/>
</pre>
--[[User:Dmsmith|Dmsmith]] 16:29, 14 October 2012 (MDT)

=== <closer> in <verse> container? ===

According to the OSIS manual (cf. 11.1.3 on p. 58), it should be possible to embed a <closer> element within a <verse> container, but the schema does not allow this. One or the other should be corrected.
--[[User:Osk|Osk]] 05:56, 6 July 2012 (MDT)

=== <seg> in <cell> ===

This was already reported to osis-users, but for the sake of completeness:
There's a typo that allows "seq" in <cell> instead of "seg".
--[[User:Osk|Osk]] 04:29, 22 February 2014 (MST)

== Feature requests ==

=== Allow <transChange> within <w> ===
An encoder ought to be ''allowed'' to put <transChange> on elements smaller than an orthographic word. If I'm translating an instance of "λόγος", but for some reason I believe that I should translate it as "words", I ought to be able to encode <w>word<transChange>s</transChange></w>. --[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== Add an element for morphology within <w> ===
Necessary for encoding documents like MORPH (WLC + morphology), we need an element to embed within <w> to carry lexical information. I suggest calling it <m> and giving it all of the attributes found on <w>. --[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== Allow <transChange> within <hi> ===
A highlighted sentence or part of a sentence is a unit, including any transChange parts of it.
At the moment a highlighted sentence with a transChange will look like this:
<hi type="bold"> Texttexttext </hi><transChange><hi type="bold"> moreText</hi></transChange><hi type="bold"> TextText</hi>

<hi type="bold"> Texttexttext <transChange>moreText</transChange> TextText</hi>

This would look cleaner and would be also closer to what is meant. [[User:Refdoc|refdoc]]:[[User_Talk:Refdoc|talk]] 16:02, 3 August 2011 (MDT)

=== Allow <catchWord> within <hi> ===
A highlighted sentence or part of a sentence is a unit, including any catchWord parts of it.
At the moment a highlighted sentence with a catchWord will look like this:
<hi type="bold"> Texttexttext </hi><catchWord><hi type="bold"> moreText</hi></catchWord><hi type="bold"> TextText</hi>

<hi type="bold"> Texttexttext <catchWord>moreText</catchWord> TextText</hi>
This is identical in form to the <transChange> issue. The problem with both of these is that <transChange> and <catchWord> may reasonably be styled in the same fashion as what is indicated by <hi>.
--[[User:Dmsmith|Dmsmith]] 16:58, 14 October 2012 (MDT)

=== Allow multiple types for <hi> ===
It'd really be convenient for
<hi type="bold italic small-caps">text</hi>
rather than
<hi type="bold"><hi type="italic"><hi type="small-caps">text</hi></hi></hi>
--[[User:Dmsmith|Dmsmith]] 16:57, 14 October 2012 (MDT)

=== Allow <transChange> within <note> ===
When translating an alternate Greek version of a passage, added words need to be indicated.

=== Allow remote header reference ===
When serving short passages via web services, as valid OSIS documents, a full header is obtrusive. Also, in a collection of related documents, for example separate book files for a Bible, one centralized header would be more maintainable. The simplest approach would probably be to allow @href on the header element, to abstract some or all of the header content. See Troy's [https://groups.google.com/d/topic/openscriptures/kygOM0yPHL8/discussion related post].

===Allow shadow/virtual elements===
A second requirement for distributing valid OSIS fragments through web services is a form of virtual, or shadow, element to supply the context of the given fragment. A new global attribute for indicating this virtual status is essential to distinguish them from the actual markup of the document. In the [http://www.esvapi.org/api ESV API], they have this construct via `virtual` attribute (see description for `include-virtual-attributes``). See Troy's [https://groups.google.com/d/topic/openscriptures/kygOM0yPHL8/discussion related post] (same as previous).

=== Improve Ketiv/Qere markup ===
A ketiv or qere can consist of one or more words, and so need to be grouped and related to one another. I propose adding <ketiv> with @id, and <qere> with @idref, to contain the content (<w> elements) and allow validation of the connection. A qere with no ketiv could be marked up without the @idref.

: This sounds like a good application for <seg>. I would recommend named types for <seg> instead: ketiv & qere. --[[User:Osk|Osk]] 00:37, 23 February 2014 (MST)

=== Improve Selah markup ===
Selah can be represented at the end of a line. The markup of <l type="selah">...</l> does not allow for the text identified as selah to be at the end of the current line. Maybe allow for a separate markup, rather than a type of line.

=== Milestonable <p> ===
For documents where the primary structure is book, chapter, verse, like the Authorized Version or the Hebrew Bible, we should be able to mark up paragraphs as milestones. This would allow for equality, rather than making book, section, paragraph a privileged system.

=== Quotation types ===
From the manual (p. 43): "The rendering for quotations marks after an interruption, for example, can be distinguished using the type attribute on this element, with values such as initial, medial, and final." Please make these @type values official: initial, medial, and final.

== Manual bugs & feature requests ==

=== Head Elements ===
The OSIS manual give the "head" element as a means of providing for titles. It is not in the schema as a child of div, but it is in the manual.

=== DivineName Element ===
Manual give type as x-yhwh in 11.5.1.2, perhaps elsewhere. It isn't necessary. It also has the content as LORD, but it should be Lord.

[[Category:OSIS]]

OSIS 211 CR

2014-02-23T07:35:18Z

Osk: added manual issues section to separate from more important schema issues

This page is for recording potential change requests to the OSIS XML schema.

== OSIS 2.1.1 Change Requests ==
Anyone with an outstanding OSIS bug report or feature proposal for consideration for inclusion into an updated OSIS schema, please write a very concise change request here in this page, including motivating use case.

== Bugs ==

=== osisGenRegex bug ===
Currently that regex looks like [1], but it should looks like [2]:

<pre>
[1] ((((\p{L}|\p{N}|_)+)(\.(\p{L}|\p{N}|_))*:)?([^:\s])+)
[2] ((((\p{L}|\p{N}|_)+)(\.(\p{L}|\p{N}|_)+)*:)?([^:\s])+)
(missing + right here ^)
</pre>

So our document with the following element isn't valid because the
string "Strong" cannot be more than 1 character long in the current schema:
<w morph="robinson:N-NSF" lemma="lemma.Strong:βίβλος">βίβλος</w>

--[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== milestoned <lg> ===

Since the <l> element can ''only'' occur within an <lg> element, use of milestoned <lg> prevents use of <l> elements (within that <lg>). Since <lg> is milestonable, one would presume that the following snippet would be valid, but it is not, for the above reason:
<pre>
<lg sID="eg1"/>
<l>Poetry line</l>
<l>Poetry line</l>
<lg eID="eg1"/>
</pre>
--[[User:Osk|Osk]] 18:18, 31 December 2011 (MST)

The <lg> element does not allow for mixed content. However the use of the milestoned <lg> wrongly allows for it.
<pre>
<lg sID="eg2"/>
text
<lg eID="eg2"/>
</pre>
--[[User:Dmsmith|Dmsmith]] 16:29, 14 October 2012 (MDT)

=== <closer> in <verse> container? ===

According to the OSIS manual (cf. 11.1.3 on p. 58), it should be possible to embed a <closer> element within a <verse> container, but the schema does not allow this. One or the other should be corrected.
--[[User:Osk|Osk]] 05:56, 6 July 2012 (MDT)

=== <seg> in <cell> ===

This was already reported to osis-users, but for the sake of completeness:
There's a typo that allows "seq" in <cell> instead of "seg".
--[[User:Osk|Osk]] 04:29, 22 February 2014 (MST)

== Feature requests ==

=== Allow <transChange> within <w> ===
An encoder ought to be ''allowed'' to put <transChange> on elements smaller than an orthographic word. If I'm translating an instance of "λόγος", but for some reason I believe that I should translate it as "words", I ought to be able to encode <w>word<transChange>s</transChange></w>. --[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== Add an element for morphology within <w> ===
Necessary for encoding documents like MORPH (WLC + morphology), we need an element to embed within <w> to carry lexical information. I suggest calling it <m> and giving it all of the attributes found on <w>. --[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== Allow <transChange> within <hi> ===
A highlighted sentence or part of a sentence is a unit, including any transChange parts of it.
At the moment a highlighted sentence with a transChange will look like this:
<hi type="bold"> Texttexttext </hi><transChange><hi type="bold"> moreText</hi></transChange><hi type="bold"> TextText</hi>

<hi type="bold"> Texttexttext <transChange>moreText</transChange> TextText</hi>

This would look cleaner and would be also closer to what is meant. [[User:Refdoc|refdoc]]:[[User_Talk:Refdoc|talk]] 16:02, 3 August 2011 (MDT)

=== Allow <catchWord> within <hi> ===
A highlighted sentence or part of a sentence is a unit, including any catchWord parts of it.
At the moment a highlighted sentence with a catchWord will look like this:
<hi type="bold"> Texttexttext </hi><catchWord><hi type="bold"> moreText</hi></catchWord><hi type="bold"> TextText</hi>

<hi type="bold"> Texttexttext <catchWord>moreText</catchWord> TextText</hi>
This is identical in form to the <transChange> issue. The problem with both of these is that <transChange> and <catchWord> may reasonably be styled in the same fashion as what is indicated by <hi>.
--[[User:Dmsmith|Dmsmith]] 16:58, 14 October 2012 (MDT)

=== Allow multiple types for <hi> ===
It'd really be convenient for
<hi type="bold italic small-caps">text</hi>
rather than
<hi type="bold"><hi type="italic"><hi type="small-caps">text</hi></hi></hi>
--[[User:Dmsmith|Dmsmith]] 16:57, 14 October 2012 (MDT)

=== Allow <transChange> within <note> ===
When translating an alternate Greek version of a passage, added words need to be indicated.

=== Allow remote header reference ===
When serving short passages via web services, as valid OSIS documents, a full header is obtrusive. Also, in a collection of related documents, for example separate book files for a Bible, one centralized header would be more maintainable. The simplest approach would probably be to allow @href on the header element, to abstract some or all of the header content. See Troy's [https://groups.google.com/d/topic/openscriptures/kygOM0yPHL8/discussion related post].

===Allow shadow/virtual elements===
A second requirement for distributing valid OSIS fragments through web services is a form of virtual, or shadow, element to supply the context of the given fragment. A new global attribute for indicating this virtual status is essential to distinguish them from the actual markup of the document. In the [http://www.esvapi.org/api ESV API], they have this construct via `virtual` attribute (see description for `include-virtual-attributes``). See Troy's [https://groups.google.com/d/topic/openscriptures/kygOM0yPHL8/discussion related post] (same as previous).

=== Improve Ketiv/Qere markup ===
A ketiv or qere can consist of one or more words, and so need to be grouped and related to one another. I propose adding <ketiv> with @id, and <qere> with @idref, to contain the content (<w> elements) and allow validation of the connection. A qere with no ketiv could be marked up without the @idref.

=== Improve Selah markup ===
Selah can be represented at the end of a line. The markup of <l type="selah">...</l> does not allow for the text identified as selah to be at the end of the current line. Maybe allow for a separate markup, rather than a type of line.

=== Milestonable <p> ===
For documents where the primary structure is book, chapter, verse, like the Authorized Version or the Hebrew Bible, we should be able to mark up paragraphs as milestones. This would allow for equality, rather than making book, section, paragraph a privileged system.

=== Quotation types ===
From the manual (p. 43): "The rendering for quotations marks after an interruption, for example, can be distinguished using the type attribute on this element, with values such as initial, medial, and final." Please make these @type values official: initial, medial, and final.

== Manual bugs & feature requests ==

=== Head Elements ===
The OSIS manual give the "head" element as a means of providing for titles. It is not in the schema as a child of div, but it is in the manual.

=== DivineName Element ===
Manual give type as x-yhwh in 11.5.1.2, perhaps elsewhere. It isn't necessary. It also has the content as LORD, but it should be Lord.

[[Category:OSIS]]

OSIS 211 CR

2014-02-22T11:29:19Z

Osk: /* Bugs */

This page is for recording potential change requests to the OSIS XML schema.

== OSIS 2.1.1 Change Requests ==
Anyone with an outstanding OSIS bug report or feature proposal for consideration for inclusion into an updated OSIS schema, please write a very concise change request here in this page, including motivating use case.

== Bugs ==

=== osisGenRegex bug ===
Currently that regex looks like [1], but it should looks like [2]:

<pre>
[1] ((((\p{L}|\p{N}|_)+)(\.(\p{L}|\p{N}|_))*:)?([^:\s])+)
[2] ((((\p{L}|\p{N}|_)+)(\.(\p{L}|\p{N}|_)+)*:)?([^:\s])+)
(missing + right here ^)
</pre>

So our document with the following element isn't valid because the
string "Strong" cannot be more than 1 character long in the current schema:
<w morph="robinson:N-NSF" lemma="lemma.Strong:βίβλος">βίβλος</w>

--[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== milestoned <lg> ===

Since the <l> element can ''only'' occur within an <lg> element, use of milestoned <lg> prevents use of <l> elements (within that <lg>). Since <lg> is milestonable, one would presume that the following snippet would be valid, but it is not, for the above reason:
<pre>
<lg sID="eg1"/>
<l>Poetry line</l>
<l>Poetry line</l>
<lg eID="eg1"/>
</pre>
--[[User:Osk|Osk]] 18:18, 31 December 2011 (MST)

The <lg> element does not allow for mixed content. However the use of the milestoned <lg> wrongly allows for it.
<pre>
<lg sID="eg2"/>
text
<lg eID="eg2"/>
</pre>
--[[User:Dmsmith|Dmsmith]] 16:29, 14 October 2012 (MDT)

=== <closer> in <verse> container? ===

According to the OSIS manual (cf. 11.1.3 on p. 58), it should be possible to embed a <closer> element within a <verse> container, but the schema does not allow this. One or the other should be corrected.
--[[User:Osk|Osk]] 05:56, 6 July 2012 (MDT)

=== <seg> in <cell> ===

This was already reported to osis-users, but for the sake of completeness:
There's a typo that allows "seq" in <cell> instead of "seg".
--[[User:Osk|Osk]] 04:29, 22 February 2014 (MST)

== Feature requests ==

=== Allow <transChange> within <w> ===
An encoder ought to be ''allowed'' to put <transChange> on elements smaller than an orthographic word. If I'm translating an instance of "λόγος", but for some reason I believe that I should translate it as "words", I ought to be able to encode <w>word<transChange>s</transChange></w>. --[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== Add an element for morphology within <w> ===
Necessary for encoding documents like MORPH (WLC + morphology), we need an element to embed within <w> to carry lexical information. I suggest calling it <m> and giving it all of the attributes found on <w>. --[[User:Osk|Osk]] 19:48, 5 November 2010 (UTC)

=== Allow <transChange> within <hi> ===
A highlighted sentence or part of a sentence is a unit, including any transChange parts of it.
At the moment a highlighted sentence with a transChange will look like this:
<hi type="bold"> Texttexttext </hi><transChange><hi type="bold"> moreText</hi></transChange><hi type="bold"> TextText</hi>

<hi type="bold"> Texttexttext <transChange>moreText</transChange> TextText</hi>

This would look cleaner and would be also closer to what is meant. [[User:Refdoc|refdoc]]:[[User_Talk:Refdoc|talk]] 16:02, 3 August 2011 (MDT)

=== Allow <catchWord> within <hi> ===
A highlighted sentence or part of a sentence is a unit, including any catchWord parts of it.
At the moment a highlighted sentence with a catchWord will look like this:
<hi type="bold"> Texttexttext </hi><catchWord><hi type="bold"> moreText</hi></catchWord><hi type="bold"> TextText</hi>

<hi type="bold"> Texttexttext <catchWord>moreText</catchWord> TextText</hi>
This is identical in form to the <transChange> issue. The problem with both of these is that <transChange> and <catchWord> may reasonably be styled in the same fashion as what is indicated by <hi>.
--[[User:Dmsmith|Dmsmith]] 16:58, 14 October 2012 (MDT)

=== Allow multiple types for <hi> ===
It'd really be convenient for
<hi type="bold italic small-caps">text</hi>
rather than
<hi type="bold"><hi type="italic"><hi type="small-caps">text</hi></hi></hi>
--[[User:Dmsmith|Dmsmith]] 16:57, 14 October 2012 (MDT)

=== Allow <transChange> within <note> ===
When translating an alternate Greek version of a passage, added words need to be indicated.

=== Allow remote header reference ===
When serving short passages via web services, as valid OSIS documents, a full header is obtrusive. Also, in a collection of related documents, for example separate book files for a Bible, one centralized header would be more maintainable. The simplest approach would probably be to allow @href on the header element, to abstract some or all of the header content. See Troy's [https://groups.google.com/d/topic/openscriptures/kygOM0yPHL8/discussion related post].

===Allow shadow/virtual elements===
A second requirement for distributing valid OSIS fragments through web services is a form of virtual, or shadow, element to supply the context of the given fragment. A new global attribute for indicating this virtual status is essential to distinguish them from the actual markup of the document. In the [http://www.esvapi.org/api ESV API], they have this construct via `virtual` attribute (see description for `include-virtual-attributes``). See Troy's [https://groups.google.com/d/topic/openscriptures/kygOM0yPHL8/discussion related post] (same as previous).

=== Improve Ketiv/Qere markup ===
A ketiv or qere can consist of one or more words, and so need to be grouped and related to one another. I propose adding <ketiv> with @id, and <qere> with @idref, to contain the content (<w> elements) and allow validation of the connection. A qere with no ketiv could be marked up without the @idref.
=== Improve Selah markup ===
Selah can be represented at the end of a line. The markup of <l type="selah">...</l> does not allow for the text identified as selah to be at the end of the current line. Maybe allow for a separate markup, rather than a type of line.

=== Milestonable <p> ===
For documents where the primary structure is book, chapter, verse, like the Authorized Version or the Hebrew Bible, we should be able to mark up paragraphs as milestones. This would allow for equality, rather than making book, section, paragraph a privileged system.

=== Quotation types ===
From the manual (p. 43): "The rendering for quotations marks after an interruption, for example, can be distinguished using the type attribute on this element, with values such as initial, medial, and final." Please make these @type values official: initial, medial, and final.
=== Head Elements ===
The OSIS manual give the "head" element as a means of providing for titles. It is not in the schema as a child of div, but it is in the manual.
=== DivineName Element ===
Manual give type as x-yhwh in 11.5.1.2, perhaps elsewhere. It isn't necessary. It also has the content as LORD, but it should be Lord.

[[Category:OSIS]]

OSIS Tutorial

2014-02-07T21:43:19Z

Osk: /* The Root Element */ I suppose whenever subsumes while

=Creating a Basic OSIS Document=
[http://crosswire.org/osis/tutor.jsp Creating a basic OSIS document] can be an easy task with just a little direction, which this brief tutorial hopes to provide.

Throughout this tutorial, we will be marking up a Bible excerpt: Gen 1:1-3, from the King James Version of the Bible, 1611 edition. The excerpt reads as follows:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
1. In the beginning
God created the
Heauen and the
Earth.
2. And the
earth was with=
out forme , and
voyd;and darken=
eſſe ''was'' vpon
the face of the deepe : And the Spirit
of God mooued vpon the face of the
waters.
3. And God ſaid, *Let there be light :
and there was light.
* Pſal.33.6. and 136.5. acts.14.15. and 17.24. hebr.11.3.

While this looks like a bunch of typos, it is a faithful representation of the original.<br/>
For an image of this see: http://dewey.library.upenn.edu/sceti/printedbooksNew/index.cfm?TextID=kjbible&PagePosition=77<br/>
For the entire first chapter of Genesis see: [[KJV 1611]]

== Scope ==
In what follows, the following part of the above excerpt page is not yet example coded in this OSIS tutorial:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
Nonetheless, OSIS does provide a means for such headings and introductory content to be implemented.

'''Note:'''
# This tutorial does not cover the [[OSIS Bibles#OSIS_Milestones|milestones]] method of using OSIS to mark Bible text.

==XML==
At the core, OSIS is an [http://en.wikipedia.org/wiki/XML XML] markup standard and must comply with rules<ref>One such rule is that XML element (and attribute) names are case-sensitive!</ref> for basic XML documents. This means that we will need a basic XML header to begin our document. This line should do just fine:
<?xml version="1.0" encoding="UTF-8" ?>
<references />

==The Root Element==
The root element for an OSIS document has the name <code>osis</code>. Since OSIS uses XML Schema to define itself, we can place a link in the root element declaring our document's structure definition (we're an OSIS XML document, not just ''any'' XML document). Our complete root element will look like this:
<pre>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
</pre>
'''Notes:'''
# Whenever bibletechnologies.net is unavailable, a snapshot of the site is available via the [http://web.archive.org/web/20140118222742/http://www.bibletechnologies.net/ Wayback Machine].
# Whenever the Bible Technologies Group website is down, the significant portions of the website are mirrored at http://www.crosswire.org/osis/, including copies of the latest schema and manual.

==The Work Container==
OSIS documents can either be a corpus of multiple works, or merely a single text, like the KJV Bible. The latter will be our OSIS document type. We declare this by placing our entire work in an <code>osisText</code> element. This element will include attributes that declare our work's id and default reference scheme. The values of these elements are not as important as their function. They are used to link to a <code>work</code> section that we will discuss a little later, that completely define two functions for our document. For now, we'll just set them to: "KJV" and "defaultReferenceScheme". Here is our <code>osisText</code> line, which also includes the default language of our document:
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">

==The Header==
Each OSIS work must have a <code>header</code> section that defines information about the text. This will include copyright and cataloguing data, among other bibliographic information. The <code>header</code> not only contains information about our work, it also contains basic information about any works which we reference in our text. Below is our entire <code>header</code> for our text. We'll explain it in more detail, below.
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>

Our <code>header</code> includes two <code>work</code> elements. Each <code>work</code> element is uniquely distinguished by its <code>osisWork</code> attribute.

The first <code>work</code> element defines ''our'' work. This is designated by matching the <code>osisWork</code> attribute value to the <code>osisIDWork</code> attribute value of our <code>osisText</code> element in the section above. This sounds confusing but simply notice they both have the value "KJV"-- that's all there is to it.

Inside this work, we have an element that defines our <code>title</code>. We also have a special <code>identifier</code> element with a <code>type</code> value of "OSIS". This <code>identifier</code> element must be present and is used for assigning a canonical name to our OSIS document. We're claiming the "KJV.TutorEncoding" identifier for
our document, in case anyone wants to refer to our text if, say, we're included in a large library of OSIS documents.

When marking up Biblical materials, we have a need to reference certain <u>portions</u> of our materials, and not just the entire document. To allow this, we must provide two sides of the same coin:
*I am referencing this portion;
*I ''am'' this portion

The two functions are facilitated in OSIS with the <code>osisRef</code> and <code>osisID</code> attributes, respectively, and are reviewed in more detail in the next section.

The <code>refSystem</code> element in our work states that whenever we markup a portion of our text as something like, "Genesis 1:1" ''(side 2 of the coin)'' we are using the Bible.KJV reference system.

Remember that we had a second attribute on <code>osisText</code>, in the section above. This attribute is used to state which reference scheme our document will use by default, when citing references ''(side 1 of the coin)'' in our text. The second work element above matches our reference scheme, and declares that we are using the Bible.KJV reference scheme as our default. Other reference schemes are described in [[Alternate Versification]].

==Marking and Referencing==
Before going any further, we should talk about an OSIS concept that enables us to reference pieces of other works, and also label pieces of our own work as targets of references. These two concepts are represented by the OSIS attributes <code>osisRef</code> and <code>osisID</code>, respectively. The concepts are fairly straight forward, and in their simplest incarnation, easy to comprehend. As an example, let's say we have a commentary that wishes to reference James 1:19. OSIS includes a <code>reference</code> element for this purpose, and for our example, an instance may look something like:
Please see <reference osisRef="Jas.1.19">James 1:19</reference>

The counterpart, <code>osisID</code>, is very similar. Let's say a Bible wishes to mark a section as ''being'' James 1:19. OSIS provides a <code>verse</code> element for this, and our Bible may include something like:
<verse osisID="Jas.1.19">Wherefore, my beloved brethren,
let every man be swift to hear, slow to speak, slow to wrath:</verse>

==Text Divisions==
OSIS works can be subdivided into arbitrary smaller sections, if desired. For Bibles, it usually makes sense to keep the traditional divisions that have been used for decades: Testament, Book, Chapter, Verse. We will use these for our text. OSIS provides a <code>div</code> element to facilitate some of these divisions; other divisions are more specifically supported. Here are our 3 divisions, at the beginning of our work to get us down to Genesis, chapter 1.
<div type="bookGroup">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">

==Actual Text==
Now ''finally'' we're ready to start including our actual Bible text in the document. The first verse is Genesis 1:1. We'd like to let the world know the identification of this part of our document, so we'll include the text in a <code>verse</code> element with an appropriate <code>osisID</code> attribute. Here it is:
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>

==Literal Translations==
The text that we've chosen to encode claims to be a special type of Bible translation, sometimes referred to as a ''literal'' translation. Translations of these type attempt to preserve --as best they can-- the wording of the text from their original language. These works all tend to use similar mechanisms to indicate where they have needed to deviate from what was presented in the original source, for the purpose of clear target language understanding. Our second verse includes one such anomaly-- the second instance of the word ''was''. Most printed editions of the KJV will ''italicize'' this word, indicating that there was no such Hebrew counterpart that was translated into the English ''"was"'', but without this word, the sentence would not represent correct English. Since OSIS is ''presentation-agnostic'', instead of delegating a display type like ''italic'' for this purpose, we will mark the anomaly and let the publisher choose how they would like it displayed. OSIS provides a <code>transChange</code> element to allow these translations to mark deviations such as these, and a <code>type</code> attribute on this element to indicate the type of change made. Here is our Genesis 1:2, which includes this markup.
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>

==Quotes==
Our last verse that we will markup includes a quote by God, Himself! Let's be sure to get this one correct. In OSIS, quotes can be marked with a <code>q</code> element. A <code>who</code> attribute is also optionally allowed, which we will use to designate who is speaking in this portion of Scripture. Since the orthography of the KJV had no quotation marks, this is indicated with an empty <code>marker</code> attribute.
<verse osisID="Gen.1.3">
And God ſaid, *<q who="God" marker="">Let there be light</q> : and there was light.
</verse>

==Notes==
Genesis 1:3 has a note that still needs to be included. In OSIS, notes are inlined at the point of inclusion. Notes also have various types. In this instance, the note is a cross-reference. To retain the original orthography we use the <code>n</code> attribute to provide the original marker.
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>

The entire verse would be:
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>

==Finishing up==
Remember, in XML, we must have closing marks for every opening mark of an element. Let's be sure to close all of our elements before finishing up our OSIS text. First we'll close our Chapter, then Book, then Testament.
</chapter>
</div>
</div>

Then we'll close our <code>osisText</code>.
</osisText>

and finally, our root <code>osis</code> element
</osis>

==Complete example==
Neglecting the cross reference illustration for James, here is the complete example of the OSIS Tutorial:
<pre>
<?xml version="1.0" encoding="UTF-8" ?>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>
<div type="x-testament">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>
</chapter>
</div>
</div>
</osisText>
</osis>
</pre>

==Conclusion==
And that's it!!! Congratulations, you've just walked through your first entire OSIS document. With power comes complexity, so there is much more to learn if you wish to unlock the features of OSIS that will allow you to more richly markup your texts. But some people may prefer to keep the more intricate aspects of OSIS under lock and key, depending on their needs. With OSIS, the choice is yours, and you now know everything necessary to start encoding your own texts, making them usable by organizations all around the world in a variety of presentation venues. Blessings in your endeavours.

[[Category:OSIS]]

OSIS Tutorial

2014-02-07T21:33:08Z

Osk: /* Complete example */ noise

=Creating a Basic OSIS Document=
[http://crosswire.org/osis/tutor.jsp Creating a basic OSIS document] can be an easy task with just a little direction, which this brief tutorial hopes to provide.

Throughout this tutorial, we will be marking up a Bible excerpt: Gen 1:1-3, from the King James Version of the Bible, 1611 edition. The excerpt reads as follows:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
1. In the beginning
God created the
Heauen and the
Earth.
2. And the
earth was with=
out forme , and
voyd;and darken=
eſſe ''was'' vpon
the face of the deepe : And the Spirit
of God mooued vpon the face of the
waters.
3. And God ſaid, *Let there be light :
and there was light.
* Pſal.33.6. and 136.5. acts.14.15. and 17.24. hebr.11.3.

While this looks like a bunch of typos, it is a faithful representation of the original.<br/>
For an image of this see: http://dewey.library.upenn.edu/sceti/printedbooksNew/index.cfm?TextID=kjbible&PagePosition=77<br/>
For the entire first chapter of Genesis see: [[KJV 1611]]

== Scope ==
In what follows, the following part of the above excerpt page is not yet example coded in this OSIS tutorial:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
Nonetheless, OSIS does provide a means for such headings and introductory content to be implemented.

'''Note:'''
# This tutorial does not cover the [[OSIS Bibles#OSIS_Milestones|milestones]] method of using OSIS to mark Bible text.

==XML==
At the core, OSIS is an [http://en.wikipedia.org/wiki/XML XML] markup standard and must comply with rules<ref>One such rule is that XML element (and attribute) names are case-sensitive!</ref> for basic XML documents. This means that we will need a basic XML header to begin our document. This line should do just fine:
<?xml version="1.0" encoding="UTF-8" ?>
<references />

==The Root Element==
The root element for an OSIS document has the name <code>osis</code>. Since OSIS uses XML Schema to define itself, we can place a link in the root element declaring our document's structure definition (we're an OSIS XML document, not just ''any'' XML document). Our complete root element will look like this:
<pre>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
</pre>
'''Notes:'''
# While/whenever bibletechnologies.net is unavailable, a snapshot of the site is available via the [http://web.archive.org/web/20140118222742/http://www.bibletechnologies.net/ Wayback Machine].
# While/whenever the Bible Technologies Group website is down, the significant portions of the website are mirrored at http://www.crosswire.org/osis/, including copies of the latest schema and manual.

==The Work Container==
OSIS documents can either be a corpus of multiple works, or merely a single text, like the KJV Bible. The latter will be our OSIS document type. We declare this by placing our entire work in an <code>osisText</code> element. This element will include attributes that declare our work's id and default reference scheme. The values of these elements are not as important as their function. They are used to link to a <code>work</code> section that we will discuss a little later, that completely define two functions for our document. For now, we'll just set them to: "KJV" and "defaultReferenceScheme". Here is our <code>osisText</code> line, which also includes the default language of our document:
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">

==The Header==
Each OSIS work must have a <code>header</code> section that defines information about the text. This will include copyright and cataloguing data, among other bibliographic information. The <code>header</code> not only contains information about our work, it also contains basic information about any works which we reference in our text. Below is our entire <code>header</code> for our text. We'll explain it in more detail, below.
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>

Our <code>header</code> includes two <code>work</code> elements. Each <code>work</code> element is uniquely distinguished by its <code>osisWork</code> attribute.

The first <code>work</code> element defines ''our'' work. This is designated by matching the <code>osisWork</code> attribute value to the <code>osisIDWork</code> attribute value of our <code>osisText</code> element in the section above. This sounds confusing but simply notice they both have the value "KJV"-- that's all there is to it.

Inside this work, we have an element that defines our <code>title</code>. We also have a special <code>identifier</code> element with a <code>type</code> value of "OSIS". This <code>identifier</code> element must be present and is used for assigning a canonical name to our OSIS document. We're claiming the "KJV.TutorEncoding" identifier for
our document, in case anyone wants to refer to our text if, say, we're included in a large library of OSIS documents.

When marking up Biblical materials, we have a need to reference certain <u>portions</u> of our materials, and not just the entire document. To allow this, we must provide two sides of the same coin:
*I am referencing this portion;
*I ''am'' this portion

The two functions are facilitated in OSIS with the <code>osisRef</code> and <code>osisID</code> attributes, respectively, and are reviewed in more detail in the next section.

The <code>refSystem</code> element in our work states that whenever we markup a portion of our text as something like, "Genesis 1:1" ''(side 2 of the coin)'' we are using the Bible.KJV reference system.

Remember that we had a second attribute on <code>osisText</code>, in the section above. This attribute is used to state which reference scheme our document will use by default, when citing references ''(side 1 of the coin)'' in our text. The second work element above matches our reference scheme, and declares that we are using the Bible.KJV reference scheme as our default. Other reference schemes are described in [[Alternate Versification]].

==Marking and Referencing==
Before going any further, we should talk about an OSIS concept that enables us to reference pieces of other works, and also label pieces of our own work as targets of references. These two concepts are represented by the OSIS attributes <code>osisRef</code> and <code>osisID</code>, respectively. The concepts are fairly straight forward, and in their simplest incarnation, easy to comprehend. As an example, let's say we have a commentary that wishes to reference James 1:19. OSIS includes a <code>reference</code> element for this purpose, and for our example, an instance may look something like:
Please see <reference osisRef="Jas.1.19">James 1:19</reference>

The counterpart, <code>osisID</code>, is very similar. Let's say a Bible wishes to mark a section as ''being'' James 1:19. OSIS provides a <code>verse</code> element for this, and our Bible may include something like:
<verse osisID="Jas.1.19">Wherefore, my beloved brethren,
let every man be swift to hear, slow to speak, slow to wrath:</verse>

==Text Divisions==
OSIS works can be subdivided into arbitrary smaller sections, if desired. For Bibles, it usually makes sense to keep the traditional divisions that have been used for decades: Testament, Book, Chapter, Verse. We will use these for our text. OSIS provides a <code>div</code> element to facilitate some of these divisions; other divisions are more specifically supported. Here are our 3 divisions, at the beginning of our work to get us down to Genesis, chapter 1.
<div type="bookGroup">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">

==Actual Text==
Now ''finally'' we're ready to start including our actual Bible text in the document. The first verse is Genesis 1:1. We'd like to let the world know the identification of this part of our document, so we'll include the text in a <code>verse</code> element with an appropriate <code>osisID</code> attribute. Here it is:
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>

==Literal Translations==
The text that we've chosen to encode claims to be a special type of Bible translation, sometimes referred to as a ''literal'' translation. Translations of these type attempt to preserve --as best they can-- the wording of the text from their original language. These works all tend to use similar mechanisms to indicate where they have needed to deviate from what was presented in the original source, for the purpose of clear target language understanding. Our second verse includes one such anomaly-- the second instance of the word ''was''. Most printed editions of the KJV will ''italicize'' this word, indicating that there was no such Hebrew counterpart that was translated into the English ''"was"'', but without this word, the sentence would not represent correct English. Since OSIS is ''presentation-agnostic'', instead of delegating a display type like ''italic'' for this purpose, we will mark the anomaly and let the publisher choose how they would like it displayed. OSIS provides a <code>transChange</code> element to allow these translations to mark deviations such as these, and a <code>type</code> attribute on this element to indicate the type of change made. Here is our Genesis 1:2, which includes this markup.
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>

==Quotes==
Our last verse that we will markup includes a quote by God, Himself! Let's be sure to get this one correct. In OSIS, quotes can be marked with a <code>q</code> element. A <code>who</code> attribute is also optionally allowed, which we will use to designate who is speaking in this portion of Scripture. Since the orthography of the KJV had no quotation marks, this is indicated with an empty <code>marker</code> attribute.
<verse osisID="Gen.1.3">
And God ſaid, *<q who="God" marker="">Let there be light</q> : and there was light.
</verse>

==Notes==
Genesis 1:3 has a note that still needs to be included. In OSIS, notes are inlined at the point of inclusion. Notes also have various types. In this instance, the note is a cross-reference. To retain the original orthography we use the <code>n</code> attribute to provide the original marker.
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>

The entire verse would be:
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>

==Finishing up==
Remember, in XML, we must have closing marks for every opening mark of an element. Let's be sure to close all of our elements before finishing up our OSIS text. First we'll close our Chapter, then Book, then Testament.
</chapter>
</div>
</div>

Then we'll close our <code>osisText</code>.
</osisText>

and finally, our root <code>osis</code> element
</osis>

==Complete example==
Neglecting the cross reference illustration for James, here is the complete example of the OSIS Tutorial:
<pre>
<?xml version="1.0" encoding="UTF-8" ?>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>
<div type="x-testament">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>
</chapter>
</div>
</div>
</osisText>
</osis>
</pre>

==Conclusion==
And that's it!!! Congratulations, you've just walked through your first entire OSIS document. With power comes complexity, so there is much more to learn if you wish to unlock the features of OSIS that will allow you to more richly markup your texts. But some people may prefer to keep the more intricate aspects of OSIS under lock and key, depending on their needs. With OSIS, the choice is yours, and you now know everything necessary to start encoding your own texts, making them usable by organizations all around the world in a variety of presentation venues. Blessings in your endeavours.

[[Category:OSIS]]

OSIS Tutorial

2014-02-07T21:32:43Z

Osk: /* Complete example */

=Creating a Basic OSIS Document=
[http://crosswire.org/osis/tutor.jsp Creating a basic OSIS document] can be an easy task with just a little direction, which this brief tutorial hopes to provide.

Throughout this tutorial, we will be marking up a Bible excerpt: Gen 1:1-3, from the King James Version of the Bible, 1611 edition. The excerpt reads as follows:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
1. In the beginning
God created the
Heauen and the
Earth.
2. And the
earth was with=
out forme , and
voyd;and darken=
eſſe ''was'' vpon
the face of the deepe : And the Spirit
of God mooued vpon the face of the
waters.
3. And God ſaid, *Let there be light :
and there was light.
* Pſal.33.6. and 136.5. acts.14.15. and 17.24. hebr.11.3.

While this looks like a bunch of typos, it is a faithful representation of the original.<br/>
For an image of this see: http://dewey.library.upenn.edu/sceti/printedbooksNew/index.cfm?TextID=kjbible&PagePosition=77<br/>
For the entire first chapter of Genesis see: [[KJV 1611]]

== Scope ==
In what follows, the following part of the above excerpt page is not yet example coded in this OSIS tutorial:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
Nonetheless, OSIS does provide a means for such headings and introductory content to be implemented.

'''Note:'''
# This tutorial does not cover the [[OSIS Bibles#OSIS_Milestones|milestones]] method of using OSIS to mark Bible text.

==XML==
At the core, OSIS is an [http://en.wikipedia.org/wiki/XML XML] markup standard and must comply with rules<ref>One such rule is that XML element (and attribute) names are case-sensitive!</ref> for basic XML documents. This means that we will need a basic XML header to begin our document. This line should do just fine:
<?xml version="1.0" encoding="UTF-8" ?>
<references />

==The Root Element==
The root element for an OSIS document has the name <code>osis</code>. Since OSIS uses XML Schema to define itself, we can place a link in the root element declaring our document's structure definition (we're an OSIS XML document, not just ''any'' XML document). Our complete root element will look like this:
<pre>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
</pre>
'''Notes:'''
# While/whenever bibletechnologies.net is unavailable, a snapshot of the site is available via the [http://web.archive.org/web/20140118222742/http://www.bibletechnologies.net/ Wayback Machine].
# While/whenever the Bible Technologies Group website is down, the significant portions of the website are mirrored at http://www.crosswire.org/osis/, including copies of the latest schema and manual.

==The Work Container==
OSIS documents can either be a corpus of multiple works, or merely a single text, like the KJV Bible. The latter will be our OSIS document type. We declare this by placing our entire work in an <code>osisText</code> element. This element will include attributes that declare our work's id and default reference scheme. The values of these elements are not as important as their function. They are used to link to a <code>work</code> section that we will discuss a little later, that completely define two functions for our document. For now, we'll just set them to: "KJV" and "defaultReferenceScheme". Here is our <code>osisText</code> line, which also includes the default language of our document:
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">

==The Header==
Each OSIS work must have a <code>header</code> section that defines information about the text. This will include copyright and cataloguing data, among other bibliographic information. The <code>header</code> not only contains information about our work, it also contains basic information about any works which we reference in our text. Below is our entire <code>header</code> for our text. We'll explain it in more detail, below.
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>

Our <code>header</code> includes two <code>work</code> elements. Each <code>work</code> element is uniquely distinguished by its <code>osisWork</code> attribute.

The first <code>work</code> element defines ''our'' work. This is designated by matching the <code>osisWork</code> attribute value to the <code>osisIDWork</code> attribute value of our <code>osisText</code> element in the section above. This sounds confusing but simply notice they both have the value "KJV"-- that's all there is to it.

Inside this work, we have an element that defines our <code>title</code>. We also have a special <code>identifier</code> element with a <code>type</code> value of "OSIS". This <code>identifier</code> element must be present and is used for assigning a canonical name to our OSIS document. We're claiming the "KJV.TutorEncoding" identifier for
our document, in case anyone wants to refer to our text if, say, we're included in a large library of OSIS documents.

When marking up Biblical materials, we have a need to reference certain <u>portions</u> of our materials, and not just the entire document. To allow this, we must provide two sides of the same coin:
*I am referencing this portion;
*I ''am'' this portion

The two functions are facilitated in OSIS with the <code>osisRef</code> and <code>osisID</code> attributes, respectively, and are reviewed in more detail in the next section.

The <code>refSystem</code> element in our work states that whenever we markup a portion of our text as something like, "Genesis 1:1" ''(side 2 of the coin)'' we are using the Bible.KJV reference system.

Remember that we had a second attribute on <code>osisText</code>, in the section above. This attribute is used to state which reference scheme our document will use by default, when citing references ''(side 1 of the coin)'' in our text. The second work element above matches our reference scheme, and declares that we are using the Bible.KJV reference scheme as our default. Other reference schemes are described in [[Alternate Versification]].

==Marking and Referencing==
Before going any further, we should talk about an OSIS concept that enables us to reference pieces of other works, and also label pieces of our own work as targets of references. These two concepts are represented by the OSIS attributes <code>osisRef</code> and <code>osisID</code>, respectively. The concepts are fairly straight forward, and in their simplest incarnation, easy to comprehend. As an example, let's say we have a commentary that wishes to reference James 1:19. OSIS includes a <code>reference</code> element for this purpose, and for our example, an instance may look something like:
Please see <reference osisRef="Jas.1.19">James 1:19</reference>

The counterpart, <code>osisID</code>, is very similar. Let's say a Bible wishes to mark a section as ''being'' James 1:19. OSIS provides a <code>verse</code> element for this, and our Bible may include something like:
<verse osisID="Jas.1.19">Wherefore, my beloved brethren,
let every man be swift to hear, slow to speak, slow to wrath:</verse>

==Text Divisions==
OSIS works can be subdivided into arbitrary smaller sections, if desired. For Bibles, it usually makes sense to keep the traditional divisions that have been used for decades: Testament, Book, Chapter, Verse. We will use these for our text. OSIS provides a <code>div</code> element to facilitate some of these divisions; other divisions are more specifically supported. Here are our 3 divisions, at the beginning of our work to get us down to Genesis, chapter 1.
<div type="bookGroup">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">

==Actual Text==
Now ''finally'' we're ready to start including our actual Bible text in the document. The first verse is Genesis 1:1. We'd like to let the world know the identification of this part of our document, so we'll include the text in a <code>verse</code> element with an appropriate <code>osisID</code> attribute. Here it is:
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>

==Literal Translations==
The text that we've chosen to encode claims to be a special type of Bible translation, sometimes referred to as a ''literal'' translation. Translations of these type attempt to preserve --as best they can-- the wording of the text from their original language. These works all tend to use similar mechanisms to indicate where they have needed to deviate from what was presented in the original source, for the purpose of clear target language understanding. Our second verse includes one such anomaly-- the second instance of the word ''was''. Most printed editions of the KJV will ''italicize'' this word, indicating that there was no such Hebrew counterpart that was translated into the English ''"was"'', but without this word, the sentence would not represent correct English. Since OSIS is ''presentation-agnostic'', instead of delegating a display type like ''italic'' for this purpose, we will mark the anomaly and let the publisher choose how they would like it displayed. OSIS provides a <code>transChange</code> element to allow these translations to mark deviations such as these, and a <code>type</code> attribute on this element to indicate the type of change made. Here is our Genesis 1:2, which includes this markup.
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>

==Quotes==
Our last verse that we will markup includes a quote by God, Himself! Let's be sure to get this one correct. In OSIS, quotes can be marked with a <code>q</code> element. A <code>who</code> attribute is also optionally allowed, which we will use to designate who is speaking in this portion of Scripture. Since the orthography of the KJV had no quotation marks, this is indicated with an empty <code>marker</code> attribute.
<verse osisID="Gen.1.3">
And God ſaid, *<q who="God" marker="">Let there be light</q> : and there was light.
</verse>

==Notes==
Genesis 1:3 has a note that still needs to be included. In OSIS, notes are inlined at the point of inclusion. Notes also have various types. In this instance, the note is a cross-reference. To retain the original orthography we use the <code>n</code> attribute to provide the original marker.
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>

The entire verse would be:
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>

==Finishing up==
Remember, in XML, we must have closing marks for every opening mark of an element. Let's be sure to close all of our elements before finishing up our OSIS text. First we'll close our Chapter, then Book, then Testament.
</chapter>
</div>
</div>

Then we'll close our <code>osisText</code>.
</osisText>

and finally, our root <code>osis</code> element
</osis>

==Complete example==
Neglecting the cross reference illustration for James, here is the complete example of the OSIS Tutorial:
<pre>
<?xml version="1.0" encoding="UTF-8" ?>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>
<div type="x-testament">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>
</chapter>
</div>
</div>
</osisText>
</osis>
</pre>
This has been validated using [http://xml-copy-editor.sourceforge.net/ XML Copy Editor].

==Conclusion==
And that's it!!! Congratulations, you've just walked through your first entire OSIS document. With power comes complexity, so there is much more to learn if you wish to unlock the features of OSIS that will allow you to more richly markup your texts. But some people may prefer to keep the more intricate aspects of OSIS under lock and key, depending on their needs. With OSIS, the choice is yours, and you now know everything necessary to start encoding your own texts, making them usable by organizations all around the world in a variety of presentation venues. Blessings in your endeavours.

[[Category:OSIS]]

OSIS Tutorial

2014-02-07T21:32:11Z

Osk: /* Text Divisions */ was bad OSIS

=Creating a Basic OSIS Document=
[http://crosswire.org/osis/tutor.jsp Creating a basic OSIS document] can be an easy task with just a little direction, which this brief tutorial hopes to provide.

Throughout this tutorial, we will be marking up a Bible excerpt: Gen 1:1-3, from the King James Version of the Bible, 1611 edition. The excerpt reads as follows:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
1. In the beginning
God created the
Heauen and the
Earth.
2. And the
earth was with=
out forme , and
voyd;and darken=
eſſe ''was'' vpon
the face of the deepe : And the Spirit
of God mooued vpon the face of the
waters.
3. And God ſaid, *Let there be light :
and there was light.
* Pſal.33.6. and 136.5. acts.14.15. and 17.24. hebr.11.3.

While this looks like a bunch of typos, it is a faithful representation of the original.<br/>
For an image of this see: http://dewey.library.upenn.edu/sceti/printedbooksNew/index.cfm?TextID=kjbible&PagePosition=77<br/>
For the entire first chapter of Genesis see: [[KJV 1611]]

== Scope ==
In what follows, the following part of the above excerpt page is not yet example coded in this OSIS tutorial:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
Nonetheless, OSIS does provide a means for such headings and introductory content to be implemented.

'''Note:'''
# This tutorial does not cover the [[OSIS Bibles#OSIS_Milestones|milestones]] method of using OSIS to mark Bible text.

==XML==
At the core, OSIS is an [http://en.wikipedia.org/wiki/XML XML] markup standard and must comply with rules<ref>One such rule is that XML element (and attribute) names are case-sensitive!</ref> for basic XML documents. This means that we will need a basic XML header to begin our document. This line should do just fine:
<?xml version="1.0" encoding="UTF-8" ?>
<references />

==The Root Element==
The root element for an OSIS document has the name <code>osis</code>. Since OSIS uses XML Schema to define itself, we can place a link in the root element declaring our document's structure definition (we're an OSIS XML document, not just ''any'' XML document). Our complete root element will look like this:
<pre>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
</pre>
'''Notes:'''
# While/whenever bibletechnologies.net is unavailable, a snapshot of the site is available via the [http://web.archive.org/web/20140118222742/http://www.bibletechnologies.net/ Wayback Machine].
# While/whenever the Bible Technologies Group website is down, the significant portions of the website are mirrored at http://www.crosswire.org/osis/, including copies of the latest schema and manual.

==The Work Container==
OSIS documents can either be a corpus of multiple works, or merely a single text, like the KJV Bible. The latter will be our OSIS document type. We declare this by placing our entire work in an <code>osisText</code> element. This element will include attributes that declare our work's id and default reference scheme. The values of these elements are not as important as their function. They are used to link to a <code>work</code> section that we will discuss a little later, that completely define two functions for our document. For now, we'll just set them to: "KJV" and "defaultReferenceScheme". Here is our <code>osisText</code> line, which also includes the default language of our document:
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">

==The Header==
Each OSIS work must have a <code>header</code> section that defines information about the text. This will include copyright and cataloguing data, among other bibliographic information. The <code>header</code> not only contains information about our work, it also contains basic information about any works which we reference in our text. Below is our entire <code>header</code> for our text. We'll explain it in more detail, below.
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>

Our <code>header</code> includes two <code>work</code> elements. Each <code>work</code> element is uniquely distinguished by its <code>osisWork</code> attribute.

The first <code>work</code> element defines ''our'' work. This is designated by matching the <code>osisWork</code> attribute value to the <code>osisIDWork</code> attribute value of our <code>osisText</code> element in the section above. This sounds confusing but simply notice they both have the value "KJV"-- that's all there is to it.

Inside this work, we have an element that defines our <code>title</code>. We also have a special <code>identifier</code> element with a <code>type</code> value of "OSIS". This <code>identifier</code> element must be present and is used for assigning a canonical name to our OSIS document. We're claiming the "KJV.TutorEncoding" identifier for
our document, in case anyone wants to refer to our text if, say, we're included in a large library of OSIS documents.

When marking up Biblical materials, we have a need to reference certain <u>portions</u> of our materials, and not just the entire document. To allow this, we must provide two sides of the same coin:
*I am referencing this portion;
*I ''am'' this portion

The two functions are facilitated in OSIS with the <code>osisRef</code> and <code>osisID</code> attributes, respectively, and are reviewed in more detail in the next section.

The <code>refSystem</code> element in our work states that whenever we markup a portion of our text as something like, "Genesis 1:1" ''(side 2 of the coin)'' we are using the Bible.KJV reference system.

Remember that we had a second attribute on <code>osisText</code>, in the section above. This attribute is used to state which reference scheme our document will use by default, when citing references ''(side 1 of the coin)'' in our text. The second work element above matches our reference scheme, and declares that we are using the Bible.KJV reference scheme as our default. Other reference schemes are described in [[Alternate Versification]].

==Marking and Referencing==
Before going any further, we should talk about an OSIS concept that enables us to reference pieces of other works, and also label pieces of our own work as targets of references. These two concepts are represented by the OSIS attributes <code>osisRef</code> and <code>osisID</code>, respectively. The concepts are fairly straight forward, and in their simplest incarnation, easy to comprehend. As an example, let's say we have a commentary that wishes to reference James 1:19. OSIS includes a <code>reference</code> element for this purpose, and for our example, an instance may look something like:
Please see <reference osisRef="Jas.1.19">James 1:19</reference>

The counterpart, <code>osisID</code>, is very similar. Let's say a Bible wishes to mark a section as ''being'' James 1:19. OSIS provides a <code>verse</code> element for this, and our Bible may include something like:
<verse osisID="Jas.1.19">Wherefore, my beloved brethren,
let every man be swift to hear, slow to speak, slow to wrath:</verse>

==Text Divisions==
OSIS works can be subdivided into arbitrary smaller sections, if desired. For Bibles, it usually makes sense to keep the traditional divisions that have been used for decades: Testament, Book, Chapter, Verse. We will use these for our text. OSIS provides a <code>div</code> element to facilitate some of these divisions; other divisions are more specifically supported. Here are our 3 divisions, at the beginning of our work to get us down to Genesis, chapter 1.
<div type="bookGroup">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">

==Actual Text==
Now ''finally'' we're ready to start including our actual Bible text in the document. The first verse is Genesis 1:1. We'd like to let the world know the identification of this part of our document, so we'll include the text in a <code>verse</code> element with an appropriate <code>osisID</code> attribute. Here it is:
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>

==Literal Translations==
The text that we've chosen to encode claims to be a special type of Bible translation, sometimes referred to as a ''literal'' translation. Translations of these type attempt to preserve --as best they can-- the wording of the text from their original language. These works all tend to use similar mechanisms to indicate where they have needed to deviate from what was presented in the original source, for the purpose of clear target language understanding. Our second verse includes one such anomaly-- the second instance of the word ''was''. Most printed editions of the KJV will ''italicize'' this word, indicating that there was no such Hebrew counterpart that was translated into the English ''"was"'', but without this word, the sentence would not represent correct English. Since OSIS is ''presentation-agnostic'', instead of delegating a display type like ''italic'' for this purpose, we will mark the anomaly and let the publisher choose how they would like it displayed. OSIS provides a <code>transChange</code> element to allow these translations to mark deviations such as these, and a <code>type</code> attribute on this element to indicate the type of change made. Here is our Genesis 1:2, which includes this markup.
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>

==Quotes==
Our last verse that we will markup includes a quote by God, Himself! Let's be sure to get this one correct. In OSIS, quotes can be marked with a <code>q</code> element. A <code>who</code> attribute is also optionally allowed, which we will use to designate who is speaking in this portion of Scripture. Since the orthography of the KJV had no quotation marks, this is indicated with an empty <code>marker</code> attribute.
<verse osisID="Gen.1.3">
And God ſaid, *<q who="God" marker="">Let there be light</q> : and there was light.
</verse>

==Notes==
Genesis 1:3 has a note that still needs to be included. In OSIS, notes are inlined at the point of inclusion. Notes also have various types. In this instance, the note is a cross-reference. To retain the original orthography we use the <code>n</code> attribute to provide the original marker.
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>

The entire verse would be:
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>

==Finishing up==
Remember, in XML, we must have closing marks for every opening mark of an element. Let's be sure to close all of our elements before finishing up our OSIS text. First we'll close our Chapter, then Book, then Testament.
</chapter>
</div>
</div>

Then we'll close our <code>osisText</code>.
</osisText>

and finally, our root <code>osis</code> element
</osis>

==Complete example==
Neglecting the cross reference illustration for James, here is the complete example of the OSIS Tutorial:
<pre>
<?xml version="1.0" encoding="UTF-8" ?>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>
<div type="x-testament">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>
</chapter>
</div>
</div>
</osisText>
</osis>
</pre>
This has been validated using [http://xml-copy-editor.sourceforge.net/ XML Copy Editor].

==Conclusion==
And that's it!!! Congratulations, you've just walked through your first entire OSIS document. With power comes complexity, so there is much more to learn if you wish to unlock the features of OSIS that will allow you to more richly markup your texts. But some people may prefer to keep the more intricate aspects of OSIS under lock and key, depending on their needs. With OSIS, the choice is yours, and you now know everything necessary to start encoding your own texts, making them usable by organizations all around the world in a variety of presentation venues. Blessings in your endeavours.

[[Category:OSIS]]

OSIS Tutorial

2014-02-07T21:31:45Z

Osk: /* Marking and Referencing */

=Creating a Basic OSIS Document=
[http://crosswire.org/osis/tutor.jsp Creating a basic OSIS document] can be an easy task with just a little direction, which this brief tutorial hopes to provide.

Throughout this tutorial, we will be marking up a Bible excerpt: Gen 1:1-3, from the King James Version of the Bible, 1611 edition. The excerpt reads as follows:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
1. In the beginning
God created the
Heauen and the
Earth.
2. And the
earth was with=
out forme , and
voyd;and darken=
eſſe ''was'' vpon
the face of the deepe : And the Spirit
of God mooued vpon the face of the
waters.
3. And God ſaid, *Let there be light :
and there was light.
* Pſal.33.6. and 136.5. acts.14.15. and 17.24. hebr.11.3.

While this looks like a bunch of typos, it is a faithful representation of the original.<br/>
For an image of this see: http://dewey.library.upenn.edu/sceti/printedbooksNew/index.cfm?TextID=kjbible&PagePosition=77<br/>
For the entire first chapter of Genesis see: [[KJV 1611]]

== Scope ==
In what follows, the following part of the above excerpt page is not yet example coded in this OSIS tutorial:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
Nonetheless, OSIS does provide a means for such headings and introductory content to be implemented.

'''Note:'''
# This tutorial does not cover the [[OSIS Bibles#OSIS_Milestones|milestones]] method of using OSIS to mark Bible text.

==XML==
At the core, OSIS is an [http://en.wikipedia.org/wiki/XML XML] markup standard and must comply with rules<ref>One such rule is that XML element (and attribute) names are case-sensitive!</ref> for basic XML documents. This means that we will need a basic XML header to begin our document. This line should do just fine:
<?xml version="1.0" encoding="UTF-8" ?>
<references />

==The Root Element==
The root element for an OSIS document has the name <code>osis</code>. Since OSIS uses XML Schema to define itself, we can place a link in the root element declaring our document's structure definition (we're an OSIS XML document, not just ''any'' XML document). Our complete root element will look like this:
<pre>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
</pre>
'''Notes:'''
# While/whenever bibletechnologies.net is unavailable, a snapshot of the site is available via the [http://web.archive.org/web/20140118222742/http://www.bibletechnologies.net/ Wayback Machine].
# While/whenever the Bible Technologies Group website is down, the significant portions of the website are mirrored at http://www.crosswire.org/osis/, including copies of the latest schema and manual.

==The Work Container==
OSIS documents can either be a corpus of multiple works, or merely a single text, like the KJV Bible. The latter will be our OSIS document type. We declare this by placing our entire work in an <code>osisText</code> element. This element will include attributes that declare our work's id and default reference scheme. The values of these elements are not as important as their function. They are used to link to a <code>work</code> section that we will discuss a little later, that completely define two functions for our document. For now, we'll just set them to: "KJV" and "defaultReferenceScheme". Here is our <code>osisText</code> line, which also includes the default language of our document:
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">

==The Header==
Each OSIS work must have a <code>header</code> section that defines information about the text. This will include copyright and cataloguing data, among other bibliographic information. The <code>header</code> not only contains information about our work, it also contains basic information about any works which we reference in our text. Below is our entire <code>header</code> for our text. We'll explain it in more detail, below.
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>

Our <code>header</code> includes two <code>work</code> elements. Each <code>work</code> element is uniquely distinguished by its <code>osisWork</code> attribute.

The first <code>work</code> element defines ''our'' work. This is designated by matching the <code>osisWork</code> attribute value to the <code>osisIDWork</code> attribute value of our <code>osisText</code> element in the section above. This sounds confusing but simply notice they both have the value "KJV"-- that's all there is to it.

Inside this work, we have an element that defines our <code>title</code>. We also have a special <code>identifier</code> element with a <code>type</code> value of "OSIS". This <code>identifier</code> element must be present and is used for assigning a canonical name to our OSIS document. We're claiming the "KJV.TutorEncoding" identifier for
our document, in case anyone wants to refer to our text if, say, we're included in a large library of OSIS documents.

When marking up Biblical materials, we have a need to reference certain <u>portions</u> of our materials, and not just the entire document. To allow this, we must provide two sides of the same coin:
*I am referencing this portion;
*I ''am'' this portion

The two functions are facilitated in OSIS with the <code>osisRef</code> and <code>osisID</code> attributes, respectively, and are reviewed in more detail in the next section.

The <code>refSystem</code> element in our work states that whenever we markup a portion of our text as something like, "Genesis 1:1" ''(side 2 of the coin)'' we are using the Bible.KJV reference system.

Remember that we had a second attribute on <code>osisText</code>, in the section above. This attribute is used to state which reference scheme our document will use by default, when citing references ''(side 1 of the coin)'' in our text. The second work element above matches our reference scheme, and declares that we are using the Bible.KJV reference scheme as our default. Other reference schemes are described in [[Alternate Versification]].

==Marking and Referencing==
Before going any further, we should talk about an OSIS concept that enables us to reference pieces of other works, and also label pieces of our own work as targets of references. These two concepts are represented by the OSIS attributes <code>osisRef</code> and <code>osisID</code>, respectively. The concepts are fairly straight forward, and in their simplest incarnation, easy to comprehend. As an example, let's say we have a commentary that wishes to reference James 1:19. OSIS includes a <code>reference</code> element for this purpose, and for our example, an instance may look something like:
Please see <reference osisRef="Jas.1.19">James 1:19</reference>

The counterpart, <code>osisID</code>, is very similar. Let's say a Bible wishes to mark a section as ''being'' James 1:19. OSIS provides a <code>verse</code> element for this, and our Bible may include something like:
<verse osisID="Jas.1.19">Wherefore, my beloved brethren,
let every man be swift to hear, slow to speak, slow to wrath:</verse>

==Text Divisions==
OSIS works can be subdivided into arbitrary smaller sections, if desired. For Bibles, it usually makes sense to keep the traditional divisions that have been used for decades: Testament, Book, Chapter, Verse. We will use these for our text. OSIS provides a <code>div</code> element to facilitate some of these divisions; other divisions are more specifically supported. Here are our 3 divisions, at the beginning of our work to get us down to Genesis, chapter 1.
<div type="x-testament">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">

==Actual Text==
Now ''finally'' we're ready to start including our actual Bible text in the document. The first verse is Genesis 1:1. We'd like to let the world know the identification of this part of our document, so we'll include the text in a <code>verse</code> element with an appropriate <code>osisID</code> attribute. Here it is:
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>

==Literal Translations==
The text that we've chosen to encode claims to be a special type of Bible translation, sometimes referred to as a ''literal'' translation. Translations of these type attempt to preserve --as best they can-- the wording of the text from their original language. These works all tend to use similar mechanisms to indicate where they have needed to deviate from what was presented in the original source, for the purpose of clear target language understanding. Our second verse includes one such anomaly-- the second instance of the word ''was''. Most printed editions of the KJV will ''italicize'' this word, indicating that there was no such Hebrew counterpart that was translated into the English ''"was"'', but without this word, the sentence would not represent correct English. Since OSIS is ''presentation-agnostic'', instead of delegating a display type like ''italic'' for this purpose, we will mark the anomaly and let the publisher choose how they would like it displayed. OSIS provides a <code>transChange</code> element to allow these translations to mark deviations such as these, and a <code>type</code> attribute on this element to indicate the type of change made. Here is our Genesis 1:2, which includes this markup.
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>

==Quotes==
Our last verse that we will markup includes a quote by God, Himself! Let's be sure to get this one correct. In OSIS, quotes can be marked with a <code>q</code> element. A <code>who</code> attribute is also optionally allowed, which we will use to designate who is speaking in this portion of Scripture. Since the orthography of the KJV had no quotation marks, this is indicated with an empty <code>marker</code> attribute.
<verse osisID="Gen.1.3">
And God ſaid, *<q who="God" marker="">Let there be light</q> : and there was light.
</verse>

==Notes==
Genesis 1:3 has a note that still needs to be included. In OSIS, notes are inlined at the point of inclusion. Notes also have various types. In this instance, the note is a cross-reference. To retain the original orthography we use the <code>n</code> attribute to provide the original marker.
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>

The entire verse would be:
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>

==Finishing up==
Remember, in XML, we must have closing marks for every opening mark of an element. Let's be sure to close all of our elements before finishing up our OSIS text. First we'll close our Chapter, then Book, then Testament.
</chapter>
</div>
</div>

Then we'll close our <code>osisText</code>.
</osisText>

and finally, our root <code>osis</code> element
</osis>

==Complete example==
Neglecting the cross reference illustration for James, here is the complete example of the OSIS Tutorial:
<pre>
<?xml version="1.0" encoding="UTF-8" ?>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>
<div type="x-testament">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>
</chapter>
</div>
</div>
</osisText>
</osis>
</pre>
This has been validated using [http://xml-copy-editor.sourceforge.net/ XML Copy Editor].

==Conclusion==
And that's it!!! Congratulations, you've just walked through your first entire OSIS document. With power comes complexity, so there is much more to learn if you wish to unlock the features of OSIS that will allow you to more richly markup your texts. But some people may prefer to keep the more intricate aspects of OSIS under lock and key, depending on their needs. With OSIS, the choice is yours, and you now know everything necessary to start encoding your own texts, making them usable by organizations all around the world in a variety of presentation venues. Blessings in your endeavours.

[[Category:OSIS]]

OSIS Tutorial

2014-02-07T21:30:56Z

Osk: /* The Root Element */ pointed to our own stable copy of the manual & schema; removed lots of premature statements, errors, & noise

=Creating a Basic OSIS Document=
[http://crosswire.org/osis/tutor.jsp Creating a basic OSIS document] can be an easy task with just a little direction, which this brief tutorial hopes to provide.

Throughout this tutorial, we will be marking up a Bible excerpt: Gen 1:1-3, from the King James Version of the Bible, 1611 edition. The excerpt reads as follows:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
1. In the beginning
God created the
Heauen and the
Earth.
2. And the
earth was with=
out forme , and
voyd;and darken=
eſſe ''was'' vpon
the face of the deepe : And the Spirit
of God mooued vpon the face of the
waters.
3. And God ſaid, *Let there be light :
and there was light.
* Pſal.33.6. and 136.5. acts.14.15. and 17.24. hebr.11.3.

While this looks like a bunch of typos, it is a faithful representation of the original.<br/>
For an image of this see: http://dewey.library.upenn.edu/sceti/printedbooksNew/index.cfm?TextID=kjbible&PagePosition=77<br/>
For the entire first chapter of Genesis see: [[KJV 1611]]

== Scope ==
In what follows, the following part of the above excerpt page is not yet example coded in this OSIS tutorial:
THE FIRST BOOKE OF MOSES, called GENESIS.
Chap.j.
The creation of the world.
1 The creation of Heauen and Earth, 3 of the
light, 6 of the firmament, 9 of the earth ſe-
parated from the waters, 11 and made fruit-
full, 14 of the Sunne, Moone and Starres,
20 of fiſh and fowle, 24 of beaſts and cat-
tell, 26 of Man in the Image of God. 29 Al-
ſo the appointment of food.
Nonetheless, OSIS does provide a means for such headings and introductory content to be implemented.

'''Note:'''
# This tutorial does not cover the [[OSIS Bibles#OSIS_Milestones|milestones]] method of using OSIS to mark Bible text.

==XML==
At the core, OSIS is an [http://en.wikipedia.org/wiki/XML XML] markup standard and must comply with rules<ref>One such rule is that XML element (and attribute) names are case-sensitive!</ref> for basic XML documents. This means that we will need a basic XML header to begin our document. This line should do just fine:
<?xml version="1.0" encoding="UTF-8" ?>
<references />

==The Root Element==
The root element for an OSIS document has the name <code>osis</code>. Since OSIS uses XML Schema to define itself, we can place a link in the root element declaring our document's structure definition (we're an OSIS XML document, not just ''any'' XML document). Our complete root element will look like this:
<pre>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace
http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
</pre>
'''Notes:'''
# While/whenever bibletechnologies.net is unavailable, a snapshot of the site is available via the [http://web.archive.org/web/20140118222742/http://www.bibletechnologies.net/ Wayback Machine].
# While/whenever the Bible Technologies Group website is down, the significant portions of the website are mirrored at http://www.crosswire.org/osis/, including copies of the latest schema and manual.

==The Work Container==
OSIS documents can either be a corpus of multiple works, or merely a single text, like the KJV Bible. The latter will be our OSIS document type. We declare this by placing our entire work in an <code>osisText</code> element. This element will include attributes that declare our work's id and default reference scheme. The values of these elements are not as important as their function. They are used to link to a <code>work</code> section that we will discuss a little later, that completely define two functions for our document. For now, we'll just set them to: "KJV" and "defaultReferenceScheme". Here is our <code>osisText</code> line, which also includes the default language of our document:
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">

==The Header==
Each OSIS work must have a <code>header</code> section that defines information about the text. This will include copyright and cataloguing data, among other bibliographic information. The <code>header</code> not only contains information about our work, it also contains basic information about any works which we reference in our text. Below is our entire <code>header</code> for our text. We'll explain it in more detail, below.
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>

Our <code>header</code> includes two <code>work</code> elements. Each <code>work</code> element is uniquely distinguished by its <code>osisWork</code> attribute.

The first <code>work</code> element defines ''our'' work. This is designated by matching the <code>osisWork</code> attribute value to the <code>osisIDWork</code> attribute value of our <code>osisText</code> element in the section above. This sounds confusing but simply notice they both have the value "KJV"-- that's all there is to it.

Inside this work, we have an element that defines our <code>title</code>. We also have a special <code>identifier</code> element with a <code>type</code> value of "OSIS". This <code>identifier</code> element must be present and is used for assigning a canonical name to our OSIS document. We're claiming the "KJV.TutorEncoding" identifier for
our document, in case anyone wants to refer to our text if, say, we're included in a large library of OSIS documents.

When marking up Biblical materials, we have a need to reference certain <u>portions</u> of our materials, and not just the entire document. To allow this, we must provide two sides of the same coin:
*I am referencing this portion;
*I ''am'' this portion

The two functions are facilitated in OSIS with the <code>osisRef</code> and <code>osisID</code> attributes, respectively, and are reviewed in more detail in the next section.

The <code>refSystem</code> element in our work states that whenever we markup a portion of our text as something like, "Genesis 1:1" ''(side 2 of the coin)'' we are using the Bible.KJV reference system.

Remember that we had a second attribute on <code>osisText</code>, in the section above. This attribute is used to state which reference scheme our document will use by default, when citing references ''(side 1 of the coin)'' in our text. The second work element above matches our reference scheme, and declares that we are using the Bible.KJV reference scheme as our default. Other reference schemes are described in [[Alternate Versification]].

==Marking and Referencing==
Before going any further, we should talk about an OSIS concept that enables us to reference pieces of other works, and also label pieces of our own work as targets of references. These two concepts are represented by the OSIS attributes <code>osisRef</code> and <code>osisID</code>, respectively. The concepts are fairly straight forward, and in their simplest incarnation, easy to comprehend. As an example, let's say we have a commentary that wishes to reference James 1:19. OSIS includes a <code>reference</code> element for this purpose, and for our example, an instance may look something like:
Please see <reference osisRef="Jas.1.19">James 1:19</reference>

The counterpart, <code>osisID</code>, is very similar. Let's say a Bible wishes to mark a section as ''being'' James 1:19. OSIS provides a <code>verse</code> element for this, and our Bible may include something like:
<verse osisID="Jas.1.19">Wherefore, my beloved brethren, let every man be swift to hear, slow to speak, slow to wrath:</verse>

==Text Divisions==
OSIS works can be subdivided into arbitrary smaller sections, if desired. For Bibles, it usually makes sense to keep the traditional divisions that have been used for decades: Testament, Book, Chapter, Verse. We will use these for our text. OSIS provides a <code>div</code> element to facilitate some of these divisions; other divisions are more specifically supported. Here are our 3 divisions, at the beginning of our work to get us down to Genesis, chapter 1.
<div type="x-testament">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">

==Actual Text==
Now ''finally'' we're ready to start including our actual Bible text in the document. The first verse is Genesis 1:1. We'd like to let the world know the identification of this part of our document, so we'll include the text in a <code>verse</code> element with an appropriate <code>osisID</code> attribute. Here it is:
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>

==Literal Translations==
The text that we've chosen to encode claims to be a special type of Bible translation, sometimes referred to as a ''literal'' translation. Translations of these type attempt to preserve --as best they can-- the wording of the text from their original language. These works all tend to use similar mechanisms to indicate where they have needed to deviate from what was presented in the original source, for the purpose of clear target language understanding. Our second verse includes one such anomaly-- the second instance of the word ''was''. Most printed editions of the KJV will ''italicize'' this word, indicating that there was no such Hebrew counterpart that was translated into the English ''"was"'', but without this word, the sentence would not represent correct English. Since OSIS is ''presentation-agnostic'', instead of delegating a display type like ''italic'' for this purpose, we will mark the anomaly and let the publisher choose how they would like it displayed. OSIS provides a <code>transChange</code> element to allow these translations to mark deviations such as these, and a <code>type</code> attribute on this element to indicate the type of change made. Here is our Genesis 1:2, which includes this markup.
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>

==Quotes==
Our last verse that we will markup includes a quote by God, Himself! Let's be sure to get this one correct. In OSIS, quotes can be marked with a <code>q</code> element. A <code>who</code> attribute is also optionally allowed, which we will use to designate who is speaking in this portion of Scripture. Since the orthography of the KJV had no quotation marks, this is indicated with an empty <code>marker</code> attribute.
<verse osisID="Gen.1.3">
And God ſaid, *<q who="God" marker="">Let there be light</q> : and there was light.
</verse>

==Notes==
Genesis 1:3 has a note that still needs to be included. In OSIS, notes are inlined at the point of inclusion. Notes also have various types. In this instance, the note is a cross-reference. To retain the original orthography we use the <code>n</code> attribute to provide the original marker.
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>

The entire verse would be:
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>

==Finishing up==
Remember, in XML, we must have closing marks for every opening mark of an element. Let's be sure to close all of our elements before finishing up our OSIS text. First we'll close our Chapter, then Book, then Testament.
</chapter>
</div>
</div>

Then we'll close our <code>osisText</code>.
</osisText>

and finally, our root <code>osis</code> element
</osis>

==Complete example==
Neglecting the cross reference illustration for James, here is the complete example of the OSIS Tutorial:
<pre>
<?xml version="1.0" encoding="UTF-8" ?>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="KJV" osisRefWork="defaultReferenceScheme" xml:lang="en">
<header>
<work osisWork="KJV">
<title>King James Version of 1611</title>
<identifier type="OSIS">KJV.TutorEncoding</identifier>
<refSystem>Bible.KJV</refSystem>
</work>
<work osisWork="defaultReferenceScheme">
<refSystem>Bible.KJV</refSystem>
</work>
</header>
<div type="x-testament">
<div type="book" osisID="Gen">
<chapter osisID="Gen.1">
<verse osisID="Gen.1.1">In the beginning God created the Heauen and the Earth.</verse>
<verse osisID="Gen.1.2">
And the earth was without forme , and voyd;and darkeneſſe
<transChange type="added">was</transChange>
vpon the face of the deepe : And the Spirit of God mooued
vpon the face of the waters.
</verse>
<verse osisID="Gen.1.3">
And God ſaid,
<note type="crossReference" n="*">
<reference osisRef="Ps.33.6">Pſal.33.6.</reference>
and
<reference osisRef="Ps.136.5">136.5.</reference>
<reference osisRef="Acts.14.15">acts.14.15.</reference>
and
<reference osisRef="Acts.17.24">17.24.</reference>
<reference osisRef="Heb.11.3">hebr.11.3.</reference>
</note>
<q who="God" marker="">Let there be light</q>
: and there was light.
</verse>
</chapter>
</div>
</div>
</osisText>
</osis>
</pre>
This has been validated using [http://xml-copy-editor.sourceforge.net/ XML Copy Editor].

==Conclusion==
And that's it!!! Congratulations, you've just walked through your first entire OSIS document. With power comes complexity, so there is much more to learn if you wish to unlock the features of OSIS that will allow you to more richly markup your texts. But some people may prefer to keep the more intricate aspects of OSIS under lock and key, depending on their needs. With OSIS, the choice is yours, and you now know everything necessary to start encoding your own texts, making them usable by organizations all around the world in a variety of presentation venues. Blessings in your endeavours.

[[Category:OSIS]]

File Formats

2014-02-07T21:14:39Z

Osk: /* SWORD Input formats */ TEI info

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, TEI, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===TEI P5===
''Text Encoding Initiative''

TEI is an text encoding standard used broadly in the humanities. It is developed by the [http://www.tei-c.org/ TEI Consortium].

A manual for the latest edition of TEI P5 can be found at [http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ P5: Guidelines for Electronic Text Encoding and Interchange].

CrossWire supports use of TEI for the encoding of dictionaries, encyclopedias, and similar works, as well as for free form books (GenBooks). We have developed our own version of the TEI P5 schema, available at http://www.crosswire.org/osis/teiP5osis.latest.xsd (http://www.crosswire.org/osis/teiP5osis.2.5.0.xsd is the latest at the time of writing). CrossWire's additions to the standard schema include the ability to create osisRefs and osisIDs to facilitate interoperability with OSIS documents and Bible reference tagging.

Further information can be found at the [[TEI Dictionaries]] page.

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

===ThML===
''Theological Markup Language'' '''(deprecated)'''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format'' '''(deprecated)'''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], and [http://ebible.org/bible/web/websf.zip WEB] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by QuickVerse and WORDsearch, and is currently used for some e-Sword books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. The SWORD Project will not support this dead format.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===USX===
'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

File Formats

2014-02-07T21:02:11Z

Osk: /* SWORD Input formats */

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

===ThML===
''Theological Markup Language'' '''(deprecated)'''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format'' '''(deprecated)'''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], and [http://ebible.org/bible/web/websf.zip WEB] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by QuickVerse and WORDsearch, and is currently used for some e-Sword books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. The SWORD Project will not support this dead format.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===USX===
'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

File Formats

2014-02-07T20:59:46Z

Osk: /* USFM */ dead links

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===ThML===
''Theological Markup Language''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], and [http://ebible.org/bible/web/websf.zip WEB] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by QuickVerse and WORDsearch, and is currently used for some e-Sword books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. The SWORD Project will not support this dead format.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===USX===
'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

File Formats

2014-02-07T20:58:50Z

Osk: /* STEP */

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===ThML===
''Theological Markup Language''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], [http://ebible.org/bible/web/websf.zip WEB], [http://ebible.org/bible/hnv/hnvsf.zip HNV] and [http://ebible.org/pdg/tokpisinsf.zip PNG] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by QuickVerse and WORDsearch, and is currently used for some e-Sword books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. The SWORD Project will not support this dead format.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===USX===
'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

File Formats

2014-02-07T20:57:41Z

Osk: We don't actually have a STEP converter, just an exporter, and we

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===ThML===
''Theological Markup Language''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], [http://ebible.org/bible/web/websf.zip WEB], [http://ebible.org/bible/hnv/hnvsf.zip HNV] and [http://ebible.org/pdg/tokpisinsf.zip PNG] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by QuickVerse and WORDsearch, and is currently used for some e-Sword books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. It is unlikely that the SWORD Project will support this format in the future as it is largely dead.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===USX===
'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

File Formats

2014-02-07T20:54:01Z

Osk: /* Zefania Utilities */ dead link

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===ThML===
''Theological Markup Language''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by [http://www.quickverse.com/ QuickVerse] and [http://www.wordsearchbible.com/ WORDsearch], and is currently used for some [http://www.e-sword.net/ e-Sword] books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. It is unlikely that the SWORD Project will support this format in the future as it is largely dead.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], [http://ebible.org/bible/web/websf.zip WEB], [http://ebible.org/bible/hnv/hnvsf.zip HNV] and [http://ebible.org/pdg/tokpisinsf.zip PNG] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===USX===

'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

File Formats

2014-02-07T20:53:42Z

Osk: /* GBF Tools */ GBF is long deprecated (even by MPJ)

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===ThML===
''Theological Markup Language''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by [http://www.quickverse.com/ QuickVerse] and [http://www.wordsearchbible.com/ WORDsearch], and is currently used for some [http://www.e-sword.net/ e-Sword] books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. It is unlikely that the SWORD Project will support this format in the future as it is largely dead.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], [http://ebible.org/bible/web/websf.zip WEB], [http://ebible.org/bible/hnv/hnvsf.zip HNV] and [http://ebible.org/pdg/tokpisinsf.zip PNG] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===USX===

'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

===Zefania Utilities===
* Zefania_2_sword_win32 - ''sed based scripts maintained by JensG'' [http://www.grabner-online.de/download/zefania_2_sword_win32.zip]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

File Formats

2014-02-07T20:52:37Z

Osk: 'yet' implies it is planned and will likely happen, but there is no value in USX or supporting USX

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===ThML===
''Theological Markup Language''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by [http://www.quickverse.com/ QuickVerse] and [http://www.wordsearchbible.com/ WORDsearch], and is currently used for some [http://www.e-sword.net/ e-Sword] books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. It is unlikely that the SWORD Project will support this format in the future as it is largely dead.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], [http://ebible.org/bible/web/websf.zip WEB], [http://ebible.org/bible/hnv/hnvsf.zip HNV] and [http://ebible.org/pdg/tokpisinsf.zip PNG] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire has no intention of creating a converter==
===USX===

'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===GBF Tools===
* gbfconvertor, including gbf2osis, gbf2xsem, & gbf2sf - utilities for converting GBF to OSIS, XSEM, and SFM [http://ebible.org/translation/gbf.html]
* gbfsrc - utilities for converting GBF to "HTML, RTF, TeX, plain ASCII text, a format readable by BibleWorks 5 or later, and a couple of less useful formats" [http://ebible.org/translation/gbf.html]

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

===Zefania Utilities===
* Zefania_2_sword_win32 - ''sed based scripts maintained by JensG'' [http://www.grabner-online.de/download/zefania_2_sword_win32.zip]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

File Formats

2014-02-07T20:50:45Z

Osk: /* OSIS */

Bible study programs use a plethora of markup formats. Even more have been suggested for use in creating Bibles and other religious material.

CrossWire Bible Society respects [[copyright]]. As such, conversion of material that is under copyright without permission from the copyright holders is not supported by The SWORD Project.

This page lists some of the more common file formats ''relevant'' to The SWORD Project, associated utilities, and other CrossWire projects.

== SWORD modules ==
Other than the source code for the SWORD API, there is no documentation for the file format of a '''SWORD module'''. There is no other documentation. The intention is that the [[DevTools:SWORD|SWORD API]] (or the [[DevTools:JSword|JSword]] implementation) is used directly or via other [http://en.wikipedia.org/wiki/Language_binding language bindings].

Whilst our software is [http://en.wikipedia.org/wiki/Open_source ''open source''], our module file format is [http://en.wikipedia.org/wiki/Proprietary_format ''proprietary''], in the sense that we see no need to document it and certainly no need to stick to it. We change it when we need to. We therefore do not encourage direct interaction with it, but firmly recommend use of the API (either C++ or Java). This is the place where we seek stability and consistency.

==SWORD Input formats==
The SWORD Project supports the following markup: OSIS, ThML, GBF and plain text.

===OSIS===
''Open Scripture Information Standard''

The Open Scripture Information Standard (OSIS) is "a common format for many visions." It is an XML format for marking up scripture and related text, part of an initiative composed of translators, publishers, scholars, software manufacturers, and technical experts, coordinated by the [http://www.bibletechnologies.net/ Bible Technologies Group]. It is co-sponsored by the [http://www.americanbible.org/ American Bible Society] and the [http://www.sbl-site.org/ Society of Biblical Literature].

The most recent XML schema is [http://www.bibletechnologies.net/osisCore.2.1.1.xsd OSIS 2.1.1], and a [http://www.bibletechnologies.net/20Manual.dsp manual] is also available.

This markup format is recommended by the CrossWire Bible Society and can be used for creating all types of resources for The SWORD Project. Support for OSIS is actively maintained and support for any unsupported elements or features needed for a module you may be working on may be requested.

[http://www.princexml.com/ Prince XML] is a proprietary software program that converts XML and HTML documents into PDF files by applying Cascading Style Sheets (CSS). It is developed by YesLogic, a small company based in Melbourne, Australia. It can be used to create high quality PDF Bibles from OSIS files[http://www.princexml.com/samples/]. A paper by [http://www.bibletechconference.com/speakers/ Jim Albright] of Wycliffe Bible Translators was presented at [http://www.bibletechconference.com/ BibleTech 2010] on using the open-source GUI companion for Prince XML, called [http://code.google.com/p/princess-2010/ Princess].

===ThML===
''Theological Markup Language''

This format is a variant of XML based on TEI and ThML, developed by and for the [http://www.ccel.org/ Christian Classics Ethereal Library]. The specifications for this markup format are available at http://www.ccel.org/ThML/.

This markup format is used in some SWORD resources, but only the creation of free-form "General book" modules based on existing CCEL resources is currently supported. Other works and new works should be created using the OSIS or TEI format.

===GBF===
''General Bible Format''

This markup format is intended as an aid to preparing Bible texts (specifically the WEB and WEB:ME) for use with various Bible search programs. The complete specification is at http://www.ebible.org/bible/gbf.htm.

This markup format was previously used for some SWORD modules but is now [http://en.wikipedia.org/wiki/Deprecation deprecated] in favor of OSIS. The rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl gbf2osis.pl] Perl utility may be used to convert GBF to OSIS for import to SWORD's native format. Adyeth hosts a [http://sites.google.com/site/adyeths/theswordproject/gbf2osis.py?attredirects=0 gbf2osis] Python utility that he wrote to convert the GBF texts from [http://ebible.org/ ebible.org] to OSIS. See [http://sites.google.com/site/adyeths/theswordproject].

===VPL===
''Verse-Per-Line''

This plain-text format is used for by SWORD for import of Bibles. It consists of one verse per line, with an optional verse reference at the beginning. The [[#VPL_Tools|vpl2mod]] utility may be used for import. VPL is deprecated in favor of the IMP format, which is more widely useful. The [[#VPL_Tools|mod2vpl]] utility may be used for export to VPL. There is a command line switch to prepend the verse reference to each line.

===IMP===
''Import Format''

This proprietary file format is used by SWORD for import of all types of modules. The three utilities '''imp2vs''' (for Bibles and verse-indexed commentaries), '''imp2ld''' (for lexicons, dictionaries, and daily-devotionals), and '''imp2gbs''' (for all other types of books) can be used to import IMP files to SWORD's native formats.

An IMP file consists of any number of entries. Each entry consists of a key line and any number of content lines. The key line consists of a line beginning with "$$$". For example, "$$$Gen 1:1" would be the key line for the Genesis 1:1 entry of a Bible or commentary module.

The content lines of an entry may consist of any text (provided that the first three characters of the line are not "$$$"). The internal markup of the content may be in any format supported by SWORD, namely OSIS for any module type or ThML for freeform books from CCEL.

See also [[DevTools:Modules#IMP_Format]].

==== imp2osis ====
CrossWire has a Perl script called [http://crosswire.org/ftpmirror/pub/sword/utils/perl/imp2osis.pl imp2osis.pl], which will convert IMP to OSIS fairly well (except a few 'corner cases'). Whenever CrossWire receives an IMP submission, this is the first thing that is run, allowing CrossWire to do validation and other OSIS sanity checks. Some editing is usually necessary after converting an IMP file to an OSIS XML file. For example, the attribute '''canonical''' is omitted from the osisText and all <div> elements, and the language attribute '''xml:lang''' defaults to "en".

==The SWORD Project Utilities==
Precompiled versions of many of these programs are available in most Linux distributions, using the distribution's package installer. For Windows, they can be found [http://crosswire.org/ftpmirror/pub/sword/utils/win32 here].<ref>If you have Xiphos installed in Windows, the Sword utilities are available in the Xiphos\bin folder.</ref><ref>The latest binaries may be found [http://dl.thehellings.com/sword-utils/ here], though currently without emptyvss.exe</ref>

===Module Creation Tools===
It is recommended that Unicode text files used for module creation be [[Encoding|encoded]] as UTF-8.<ref>[http://en.wikipedia.org/wiki/Newline EOLs] should be either Unix style (LF) or Windows style (CRLF). Text files with Mac style EOLs (CR) may give rise to errors or other unexpected behaviour.</ref>
* imp2gbs - imports free-form General books in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2ld - imports lexicons, dictionaries, and daily devotionals in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* imp2vs - imports Bibles and commentaries in IMP format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vpl2mod - imports Bibles and commentaries in Verse-Per-Line format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[osis2mod]] - imports Bibles and commentaries in OSIS format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Diagnostic Tools===
* mod2imp - creates an IMP file<ref>The IMP file may contain a residue of XML markup</ref> from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* stepdump - dumps the contents of a STEP book [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* emptyvss - exports a list of verses missing from the module (useful for testing modules during development) [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Conversion Tools===
* gbf2osis.pl - a PERL utility for converting GBF to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/gbf2osis.pl &dagger;]
* step2vpl - export a STEP book in Verse-Per-Line (VPL) format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[DevTools:Misc#thml2osis|thml2osis]] - converts ThML to OSIS format.
* zef2osis.pl - a PERL utility for converting Zefania XML to OSIS [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl &dagger;]

===OSIS Utilities===
* [[mod2osis]] - creates an OSIS file from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* vs2osisref - returns the osisRef of a given (text form) verse reference [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* xml2gbs - imports free-form General books in OSIS or ThML format to SWORD format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

===Miscellaneous===
* cipherraw - used to encipher SWORD modules [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* diatheke - a basic CLI SWORD front-end [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mkfastmod - creates a search index for a module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* [[mod2zmod]] - creates a compressed module from an installed module [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* mod2vpl - exports the module to VPL format<ref>The VPL file may contain a residue of XML markup</ref> [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* modwrite - outputs the module contents in VPL format [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* treeidxutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]
* genbookutil - ''needs a description'' [http://crosswire.org/ftpmirror/pub/sword/utils/win32 &dagger;]

==== Notes on SWORD Tools ====

<references />

===Recommended Non-SWORD Utilities===
* uconv - a utility from [http://icu-project.org/ ICU] for converting between various character encodings, perform normalization, transliterate texts, etc. (It's similar to iconv, but much, much more powerful.) uconv.exe is part of the [http://crosswire.org/ftpmirror/pub/sword/utils/win32 sword utilities]
* xmllint - a utility (part of the [http://xmlsoft.org/ libxml2] distribution) for validating XML documents [http://crosswire.org/ftpmirror/pub/sword/utils/win32 *]

==Formats for which CrossWire maintains converters==
The SWORD Project uses primary source e-texts. These texts come in numerous formats. CrossWire maintains converters for a number of formats, described below. The converters may target other markup formats, e.g. TEI or OSIS, or may simply export binary data to text, as is the case with our STEP exporter. Specific discussion of each of the available converters is found elsewhere on this page.

===STEP===
''[http://en.wikipedia.org/wiki/STEP_Library Standard Template for Electronic Publishing]''

This file format was formerly used by [http://www.quickverse.com/ QuickVerse] and [http://www.wordsearchbible.com/ WORDsearch], and is currently used for some [http://www.e-sword.net/ e-Sword] books.

While not an open standard, the publicly released documentation and specifications for this format can be found partially mirrored at
http://www.crosswire.org/bsisg/. Some utilities for working with this format are listed below. It is unlikely that the SWORD Project will support this format in the future as it is largely dead.

''Not to be confused with STEP (Scripture Tools for Every Pastor) – and the new front-end application ([[Frontends:TyndaleStep|Tyndale STEP]]) being developed by Tyndale House, Cambridge in collaboration with CrossWire''.

===Unbound Bible Format===
''Unbound Bible Format''

The [http://unbound.biola.edu/ BIOLA's Unbound Bible] offers many of their resources for download in a proprietary, but relatively simple [http://en.wikipedia.org/wiki/Tab_delimited tab-delimited] plain-text format (TDT). There are usually two variants, one with versification mapping to the [http://en.wikipedia.org/wiki/American_Standard_Version ASV], and the other without verse mapping. All available downloads may be found on Unbound Bible's [http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain download page].

There is no widespread use of this format, but the rudimentary [http://crosswire.org/ftpmirror/pub/sword/utils/perl/unb2osis.pl unb2osis.pl] utility may be used to convert Unbound Bible format to OSIS for import to SWORD's native format.

===USFM===
[http://paratext.org/usfm ''Unified Standard Format Markers'']

This plain-text format is a common internal-use format within Bible translation agencies and Bible societies. It is the native format of [http://paratext.org/ ParaTExt]. Paratext is used by more than 60% of all Bible translators world-wide. The current release is Paratext 7.4.

CrossWire now has a Python script called usf2m2osis.py<ref>This replaces our earlier Perl script [http://crosswire.org/ftpmirror/pub/sword/utils/perl/usfm2osis.pl usfm2osis.pl].</ref> which converts USFM to OSIS for subsequent import to SWORD's native format. See [[Converting SFM Bibles to OSIS]].

USFM uses a separate file for each Bible book. USFM is also supported by the open-source program called [http://sites.google.com/site/bibledit/ Bibledit]. There are examples of Bibles in USFM format available for download at [http://ebible.org/]. These include the [http://ebible.org/bible/kjv/kjvsf.zip KJV], [http://ebible.org/bible/asv/asvsf.zip ASV], [http://ebible.org/bible/web/websf.zip WEB], [http://ebible.org/bible/hnv/hnvsf.zip HNV] and [http://ebible.org/pdg/tokpisinsf.zip PNG] Bibles.

USFM is one of the formats that can be used by [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]].

'''Note:'''
<references />

===Zefania XML===
Zefania is an XML format for Bible markup with only the most simple structural tags for book/chapter/verse, notes, etc.
The project is now hosted on [http://sourceforge.net/projects/zefania-sharp/ SourceForge].
The [http://sourceforge.net/projects/zefbiblereader/ Zefania Bible Reader] may be used to display Zefania XML Bibles through XSL transformation in browsers.
See also the related [http://zefania.blogspot.com/ Bible Resources Archive].

The CrossWire utility [http://crosswire.org/ftpmirror/pub/sword/utils/perl/zef2osis.pl zef2osis.pl] may be used to convert Zefania XML to OSIS for import to SWORD's native format.

===Go Bible===
Following an agreement made in July 2008 with the program's author Jolon Faichney, [[Projects:Go Bible|Go Bible]] was adopted by CrossWire as its Java ME software project.

To achieve the navigation speed and general ease of use on even the simplest of Java mobile phones, Go Bible data is fully indexed, as well as being compressed (as are all JAR files). The format is described in [http://code.google.com/p/gobible/wiki/GoBibleDataFormat Go Bible data format]. Go Bible data is structured as Book | Chapter | Verse text and does not support notes, headings and cross-references, etc. The developer kit [http://gobible.jolon.org/developer/welcome.html Go Bible Creator] can take either USFM, ThML or OSIS as the source text format, but they usually have to be made specially suitable. For example, OSIS files produced by Snowfall Software's SFMToOSIS script are not structured the same. Work has begun to make an [http://en.wikipedia.org/wiki/XSL_Transformations XSLT] script to convert such OSIS XML files to the format suitable for Go Bible. [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] version 2.3.2 and onwards can take a folder of USFM files as the source text format.

Go Bible source code is now available [https://crosswire.org/svn/gobible/ here] on the CrossWire Repository. ''To access this you will need to have an account''.

GoBibleDataFormat is being extended in the [[Projects:Go Bible/SymScroll|SymScroll]] branch.

==Formats for which CrossWire does not yet have its own converter==
===USX===

'''USX''' is an XML schema that is the underlying data structure in the latest release of [http://paratext.org/ UBS Paratext]. SIL's Language Software Development team is working with UBS on this. This version of Paratext can take in USFM projects and export USX files.

USX was defined to support the [http://everytribeeverynation.org/ Every Tribe Every Nation] alliance '''Digital Bible Library''' (DBL). The alliance brings together the United Bible Society, SIL/WBT, American Bible Society, Biblica and other Bible Agencies. Under the ETEN framework, Bible translations made publication ready in the DBL for access by approved End User Ministry Partners (EUMP) will be stored in USX format.

The [https://bitbucket.org/paratext/dblvalidation/src/tip/source/text USX schema] is available in the compact [http://relaxng.org/ Relax NG] Schema language. [https://bitbucket.org/paratext/dblvalidation dblvalidation] for the Digital Bible Library is hosted on the same site.

==Other Utilities==
These are not part of The SWORD Project, but may be useful. A link is given for each.

===Go Bible utilities===
* [[Projects:Go Bible/Go Bible Creator|Go Bible Creator]] - a Java SE program for converting either ThML or OSIS or USFM to [[Projects:Go Bible|Go Bible]]. It is being enhanced by SIL to be capable of converting source text in [[File Formats#XHTML|XHTML-TE]] format.

* [http://gbcpreprocessor.codeplex.com/ Go Bible Creator USFM Preprocessor] – This is a tool to parse through and identify, correct and publish USFM file formats into a file format that can easily be put into the Go Bible mobile phone program.

===GBF Tools===
* gbfconvertor, including gbf2osis, gbf2xsem, & gbf2sf - utilities for converting GBF to OSIS, XSEM, and SFM [http://ebible.org/translation/gbf.html]
* gbfsrc - utilities for converting GBF to "HTML, RTF, TeX, plain ASCII text, a format readable by BibleWorks 5 or later, and a couple of less useful formats" [http://ebible.org/translation/gbf.html]

===STEP Utilities===
* step2rtf - extracts the internal RTF text from STEP books [http://www.customconsulting.us/step2rtf.zip]
* stepr - a rudimentary STEP reader [http://www.customconsulting.us/stepr-0.3.1.tgz]

===ThML Utilities===
* CCEL Desktop - a program for viewing and developing CCEL books [http://ccel-desktop.sourceforge.net/]

===Zefania Utilities===
* Zefania_2_sword_win32 - ''sed based scripts maintained by JensG'' [http://www.grabner-online.de/download/zefania_2_sword_win32.zip]

== Optical Character Recognition ==
[[Non-CrossWire Text-Development Projects|Text development]] activities may be greatly assisted by using [http://en.wikipedia.org/wiki/Optical_character_recognition OCR] software. This section will list OCR programs that CrossWire volunteers have found useful. Proprietary programs should not be listed here, the preference at CrossWire being to use free and/or open-source software.

=== Tessaract ===

* [http://en.wikipedia.org/wiki/Tesseract_(software) Tesseract] is a free optical character recognition engine. It was originally developed at Hewlett-Packard from 1985 until 1995. After ten years with no development, Hewlett Packard and UNLV released it in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0. Tesseract is considered one of the the most accurate free software OCR engines currently available.

* [http://vietocr.sourceforge.net/ VietOCR] – A Java/.NET GUI frontend for Tesseract OCR engine. Supports optical character recognition for Vietnamese language. Tip: Visit [http://www.moheb.de/ocr.html] to read how Moheb Mekhaiel adapted VietOCR to scan Coptic documents.

== See also ==
* [[Frontends:Bookmarks Standard]]
* [[File Formats Cruft]]

[[Category:Development tools]]
[[Category:File formats]]
[[Category:OSIS]]
[[Category:ThML]]
[[Category:Utilities]]
[[Category:USFM]]
[[Category:Unicode]]
[[Category:Bibledit]]
[[Category:Paratext]]

OSIS Bibles

2014-02-07T20:48:57Z

Osk: /* Introduction */

==OSIS==
OSIS is an XML Schema definition for Bibles and other Biblical research texts, which enables ministries and other organizations to collaborate more easily. Traditionally, these organizations have stored their documents in disparate, proprietary markups, making it difficult when they wish to share in service with each other. OSIS provides a common markup for multiple visions.

CrossWire is committed to supporting the OSIS initiative. We have developed OSIS import and export tools which work with our SWORD engine, making OSIS documents available to all of our SWORD software.

The latest OSIS Schema definition and supporting information is available at: http://www.bibletechnologies.net <ref>Link broken. See [[OSIS Tutorial#The_Root_Element|OSIS Tutorial]]</ref>

<references/>

==Introduction==
This page is for practical examples of how to encode a Bible in OSIS 2.1.1 for building a SWORD module with [[osis2mod]]. It represents CrossWire's experience and best practices in creating modules.

Every OSIS SWORD module must be created from a well-formed and valid OSIS 2.1.1 document. While it is a desirable goal for any such document to be acceptable, SWORD has some particular requirements which are discussed here.

The schema for OSIS 2.1.1 that was formally at http://www.bibletechnologies.net/osisCore.2.1.1.xsd is preserved at http://crosswire.org/osis/osisCore.2.1.1.xsd.

The March 2006 version of the OSIS Manual may be found [http://www.crosswire.org/OSIS/OSIS%202.1.1%20User%20Manual%2006March2006.pdf here] (PDF).

A good example of an OSIS document can be found at http://www.crosswire.org/~dmsmith/kjv2006.

See also [[OSIS Book Abbreviations|OSIS Book Name Abbreviations]].

==General structure==

An OSIS document is a ''well-formed XML'' document, valid according to the ''OSIS schema''.
You can find the full normative description on the OSIS Website [http://www.bibletechnologies.net/].<ref>Link broken. See [[OSIS Tutorial#The_Root_Element|root element]]</ref>

To produce a Bible, you can use this template:

<pre>
<?xml version="1.0" encoding="UTF-8"?>
<osis
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:osis="http://www.bibletechnologies.net/2003/OSIS/namespace"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="{NAME}" osisRefWork="bible" xml:lang="{LANG}" canonical="true">
<header>
{HEADER}
</header>
<div type="bookGroup">
{BODY}
</div>
</osisText>
</osis>
</pre>

With the following values:
; {NAME}: Normalized name of the Bible version (Usually 3 letters for language, 3 for translation)
; {LANG}: IETF language code-- ISO 639-1 codes are preferred, and ISO 639-3 codes are preferred when ISO 639-1 codes do not exist for the given language. See [http://www.sil.org/iso639-3/codes.asp] for a list of codes.
; {HEADER}: Description of the included text; see below
; {BODY}: Text; see below

For text without any character outside ASCII, you can use US-ASCII encoding (usually for english text). For every other language, please use UTF-8 and NFC. See the [[OSIS Bibles#Tools|tools]] section if you need to convert.

<references />
===Header===
This is the minimum contents of the header required for validation. It is also what <tt>usfm2osis.pl</tt> produces. A valid OSIS header may include more than this.
<header>
<work osisWork={Name}/>
</header>

===Body===

Here is the general structure of the body content:

<pre>
<div type="bookGroup">
<title>Old Testament</title>
<div type="book" osisID="Gen" canonical="true">
<title type="main" short="Genesis">Genesis</title>
<chapter osisID="Gen.1" n="1">
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>In the
beginning...<verse eID="Gen.1.1"/>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>The earth was
formless and void...<verse eID="Gen.1.2"/>
...
</chapter>
</div>
</div>
</pre>
'''Notes:'''
# The bookGroup division for the New Testament should have a similar structure.
# Any <tt><div></tt> defaults canonical to false. You need to set it to true on elements representing the structure of the original text.
# A quirk of the SWORD compilation process is that the only kind of content which reliably displays outside of <verse> elements are titles.

===OSIS Milestones===
OSIS allows for two potentially overlapping structures: Document structure (BSP) and verse structure (BCV).

Document structure is dominated by book, sections and paragraphs (BSP), additionally with titles, quotes and poetic material. While verse structure is indicated by book, chapter and verse numbers (BCV). While a SWORD module requires verse structure, the best way to encode a module with deep markup is with document structure. [[Osis2mod]] is responsible for transforming document structure into verse structure.

Because these two systems can overlap and because XML does not allow for overlapping elements, OSIS defines a milestone mechanism for both document and verse structure elements.

For:
<X ... attribute list ...>
...
</X>
the milestoned form is:
<X sID="g1" ... attribute list .../>
...
<X eID="g1"/>

According to the OSIS manual, for any given element X that defines a milestoneable form, all the instances of X in the document must use one form or the other and may not use both. The value of each sID attribute must be unique within the document.

Verse milestone sID/eID attributes can even have values that denote a verse range. This is purely for convenience to human readers.

It is allowable to use milestone elements for verses alone, or for both verses and chapters. The body example above is for the former.

==== Limitations of XML validators ====

An [[OSIS Bibles#Valid_OSIS_test|XML validator]] cannot validate whether OSIS milestones are used properly. It cannot validate:
* that an element is consistently either milestoned or not.
* that for each element with an sID that there is a paired element with an eID.<ref>osis2mod does not crash if the eID milestones are all missing, but the resulting module may appear to be void of text.</ref>
* that each paired sID/eID have the same attribute value.
* that different sID/eID pairs of the same element type do not overlap.<ref>If they weren't milestones, one would say they should be properly nested.</ref>

'''Notes:'''
<references />

==== Notes about OSIS elements ====
* For an OSIS document to be valid it must use the non-milestoned <div> and <lg> elements.
* There is no milestoned version of the <p> element. From a practical perspective, this means that the milestoned verse element should be used when paragraphs are used.
* The milestoned chapter element must be used when the paragraph is spanning over a chapter.
* The SWORD engine cannot handle sub-identifiers separated by ! in an osisID, so osis2mod strips these off from the osisIDs for verses. They are only of use for the osisIDs for notes.

===Recommended approach===
* For chapters, use <chapter>...</chapter> container elements (except in the rare case that other container elements cross chapter boundaries)
* For verses, use milestone elements (unless container elements will suffice) – see [[OSIS Bibles/BSPExample]].
* For paragraphs, use the <p>...</p> container element
* For poetry, use container elements <lg>...</lg> to indicate stanzas (or other types of line groups) and <l>...</l> to indicate lines
* For quoted text, use the <q>...</q> container element
* For translation changes, use the <transChange>...</transChange> container element<ref>Except where the text is within a <w> which is not allowed by OSIS. For these cases use the alternative <seg subType="x-added" type="x-transChange">...</seg> </ref>

<references />

==Examples==
===Marking Paragraphs===
There is no milestoned version of the <tt><p></tt> element. Typically paragraphs surround whole verses. That is, they start and end between verses. If a paragraph begins or ends in a verse and extends beyond that verse, then the whole document must use the milestoned version of <tt><verse></tt>.
<pre>
<div type="book" osisID="Gen" canonical="true">
<title type="main">LE PREMIER LIVRE DE MOÏSE dit LA GENÈSE</title>
<chapter osisID="Gen.1" chapterTitle="Chapitre 1"><title type="chapter">Chapitre 1</title>
<p>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement Dieu créa les
cieux et la terre.<verse eID="Gen.1.1"/>
</p>
<p>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu
planait sur la face des eaux.<verse eID="Gen.1.2"/>
</p>
<p>
<verse sID="Gen.1.3" osisID="Gen.1.3" n="3"/>Et Dieu dit : Que la lumière
soit. Et la lumière fut.<verse eID="Gen.1.3"/>
<verse sID="Gen.1.4" osisID="Gen.1.4" n="4"/>Et Dieu vit la lumière, qu'elle
était bonne ; et Dieu sépara la lumière d'avec les ténèbres.
<verse eID="Gen.1.4"/>
<verse sID="Gen.1.5" osisID="Gen.1.5" n="5"/>Et Dieu appela la lumière Jour ;
et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin :
&#8212; premier jour.<verse eID="Gen.1.5"/>
</p>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Au commencement Dieu créa les cieux et la terre.

<sup>'''(2)'''</sup> Et la terre était désolation et vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

<sup>'''(3)''' </sup>Et Dieu dit : Que la lumière soit. Et la lumière fut. <sup>'''(4)'''</sup> Et Dieu vit la lumière, qu'elle était bonne ; et Dieu sépara la lumière d'avec les ténèbres. <sup>'''(5)'''</sup> Et Dieu appela la lumière Jour ; et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin : — premier jour.
|}

Note: [[osis2mod]] converts a paragraph start into <tt><div type="paragraph" sID="genX"/></tt> and a paragraph end into <tt><div type="paragraph" eID="genX"/></tt>.

===Marking Quotations===
Most of the SWORD front-end applications will show a chapter at a time and some will show isolated verses. This means that all of the SWORD applications show partial quotations, such as the Sermon on the Mount which begins in Matt 5 and ends in Matt 7.

====Default quotation marks====
By default, SWORD will use <tt>"</tt> for quotations. The following describes various ways to influence this.

====Indicating the nesting of a quote====
When a quote is contained in a quote, it is customary to set the level attribute to indicate the depth of the nesting. For example, Jeremiah 23:38 is part of a larger quote and has a back and forth dialog of nested quotes:

<pre>
But if you say,
<q level="2" sID="1"/>
The burden of the Lord,
<q level="2" eID="1"/>
thus says the Lord,
<q level="2" sID="3"/>
Because you have said these words,
<q level="3" sID="4"/>
The burden of the Lord,
<q level="3" eID="4"/>
when I sent to you, saying,
<q level="3" sID="5"/>
You shall not say,
<q level="4" sID="6"/>
The burden of the Lord,
<q level="4" eID="6"/>
<q level="3" eID="5"/>
</pre>
A couple of things to note about this verse. First, the level attribute is on both the sID and the eID pair, matching in value. Second, this is an example of a verse that has a quote that starts in the middle and finishes in another verse.

In this case, SWORD will use the level to determine whether to use <tt>"</tt> or <tt>'</tt> for quotes. Odd levels will use <tt>"</tt> and even levels will use <tt>'</tt>. This is in accordance with American English usage, which is the opposite of British English usage. Nesting levels up to five can be found in the Bible.<ref>Jeremiah 27:1-11; 29:1-28, 30-32; 34:1-5; and Ezekiel 1-36</ref>

<references />

====Supplying alternative quotation marks====
The quote element has a marker attribute that can be used to control the quotation marks. SWORD applications will always use this value when rendering the quote. When the marker attribute is present but empty, it will render no quotation mark at all.

To specify "curly" quotes you can use the following values:
{|border=1
!Description!!Char!!HTML Entity!!Unicode
|-align=center
|Opening double quote||“||&#8220;||U+201C
|-align=center
|Closing double quote||”||&#8221;||U+201D
|-align=center
|Opening single quote||‘||&#8216;||U+2018
|-align=center
|Closing single quote||’||&#8217;||U+2019
|}

To use different marks to start and end a quote, use the milestoned version of the quote.
<q marker="“" sID="qN"/> ... <q marker="”" eID="qN"/>

There is further information about English quotation marks and their usage in [http://en.wikipedia.org/wiki/Quotation_marks].

Quotation marks have a variety of forms in different languages and in different media. See [http://en.wikipedia.org/wiki/Quotation_mark,_non-English_usage Quotation mark, non-English usage].<ref>When modules are being converted from digitized source text used in other Bible software, it may be the case that quotation marks in the text source differ from those in the original published edition, whether due to inherent constraints of the other software, or for other causes.</ref>

'''Note:'''
<references />

====Continuation quotation marks====
The <tt><milestone type="cQuote"/></tt> can be used to indicate the presence of a continued quote. If the marker attribute is present, it will use that otherwise it will use a straight double quote, ". Since there is no level attribute on the milestone element, it is best to specify the marker attribute.

====Marking the Words of Christ====
To indicate that a quote is something that Jesus said<ref>http://en.wikipedia.org/wiki/Red_letter_edition</ref>, use the attribute <tt>who="Jesus"</tt>.

<pre>
<verse osisID="Luke.22.35 sID="Luke.22.35"/>
Then Jesus asked them, <q who="Jesus" marker="">When I sent you without purse,
bag or sandals, did you lack anything?</q>
<verse eID="Luke.22.35"/>
</pre>

{| border="1"
|+Result
|
Then Jesus asked them, <span style="color:#A00">When I sent you without purse, bag or sandals, did you lack anything?</span>
|}

'''Note:'''
<references />

===Marking poetic material===
Poetry is marked up with <lg>, line group, and <l>, line, elements. The line element supports indentation with the level attribute. When the level attribute is not present or it is level="1", this should be interpreted as the first level of the line group. When level="2" it is indented relative to level="1". The same is true for each subsequent level.

The level attribute is used to indicate indentation. A value of 1 means no indentation, the same as not specifying a level attribute. A value of 2 means to indent one. And so forth.

<pre>
<chapter osisID="Exod.15" chapterTitle="Chapitre 15"><title type="chapter">Chapter 15</title>
<p>
<verse sID="Exod.15.1" osisID="Exod.15.1" n="1"/>
Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,
</p>
<lg>
<l level="1">I will sing unto the LORD, for he hath triumphed gloriously:</l>
<l level="2">the horse and his rider hath he thrown into the sea.</l>
<verse eID="Exod.15.1"/>

<verse sID="Exod.15.2" osisID="Exod.15.2" n="2"/>
<l level="1">The LORD is my strength and song, and he is become my salvation:</l>
<l level="2">he is my God, and I will prepare him an habitation;</l>
<l level="2">my father's God, and I will exalt him.</l>
<verse eID="Exod.15.2"/>

<verse sID="Exod.15.3" osisID="Exod.15.3" n="3"/>
<l level="1">The LORD is a man of war:</l>
<l level="2">the LORD is his name.</l>
<verse eID="Exod.15.3"/>

<verse sID="Exod.15.4" osisID="Exod.15.4" n="4"/>
<l level="1">Pharaoh's chariots and his host hath he cast into the sea:</l>
<l level="2">his chosen captains also are drowned in the Red sea.</l>
<verse eID="Exod.15.4"/>

<verse sID="Exod.15.5" osisID="Exod.15.5" n="5"/>
<l level="1">The depths have covered them:</l>
<l level="2">they sank into the bottom as a stone.</l>
<verse eID="Exod.15.5"/>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,

::''I will sing unto the LORD, for he hath triumphed gloriously:''
::::''the horse and his rider hath he thrown into the sea.''

::<sup>'''(2)'''</sup> ''The LORD is my strength and song, and he is become my salvation:''
::::''he is my God, and I will prepare him an habitation;''
::::''my father's God, and I will exalt him.''

::<sup>'''(3)'''</sup> ''The LORD is a man of war:''
::::''the LORD is his name.''

::<sup>'''(4)'''</sup> ''Pharaoh's chariots and his host hath he cast into the sea:''
::::''his chosen captains also are drowned in the Red sea.''

::<sup>'''(5)'''</sup> ''The depths have covered them:''
::::''they sank into the bottom as a stone.''
|}

'''Note:'''

# While OSIS defines a milestoned version of the <tt><lg></tt> element, its use (rather than the container version) will not produce a valid XML document. The <tt><l></tt> element can only occur within an <tt><lg></tt> container, so use of <tt><lg/></tt> milestones prevents use of <tt><l></tt> elements.

==== Marking acrostic poetry headings ====
Use <tt>type="acrostic"</tt> as the title attribute for the stanza headings in acrostic passages such as Psalm 119.

=== Marking lemmas & morphology ===
====Marking Strong's numbers====
To mark up [http://en.wikipedia.org/wiki/Strong%27s_Concordance Strong's numbers], you first need to declare a workID in the header of the OSIS document:
<pre>
<header>
...
<work osisWork="strong">
<refSystem>Dict.Strongs</refSystem>
</work>
...
</header>
</pre>

SWORD does not actually use this declaration, but it is required to have a proper OSIS document.

And while OSIS allows arbitrary workIDs, SWORD can only handle "strong" and a few variants.

<pre>
<w lemma="strong:H0853 strong:H03045">knew</w>
</pre>

The <w> element is used to surround the text that is represented by the Strong's number. It may be that the text is a phrase and it may be that more than one Strong's number defines the text.

When more than one Strong's number defines the text, each must be prefixed with a workID and must be separated from each other by a space. (While OSIS allows for the defining of default workIDs, SWORD requires that the workIDs be used.)

The actual Strong's Number should indicate whether it is Hebrew (H) or Greek (G) followed by the number. The number can be 0 padded to 5 digits as in H00001.

====Marking morphology====
In a similar manner to marking with Strong's numbers, morphology can also be noted. Since morphology regards the original language, Strong's numbers will be shown at the same time.

As with Strong's numbers, a workID needs to be defined. Here we are defining one for Robinson's Morphology Codes. And while SWORD will ignore this declaration, "robinson" is hard-coded into SWORD for Greek morphology codes.
<pre>
<header>
...
<work osisWork="robinson">
<refSystem>Dict.Robinson</refSystem>
</work>
...
</header>
</pre>

Example markup of Robinson's Morphology Codes in the KJV module:
<pre>
<w lemma="strong:G3588 strong:G80" morph="robinson:T-APM robinson:N-APM" src="7 8">his brethren</w>
</pre>
In this example, lemma, morph and src form parallel arrays. The first strong: mapping to the first robinson: and the first src value.

The workID should be name of a current, future, or potential lexicon module in which the morphology code could be looked up. For example, morph="packard:D" represents a reference to morphology code "D" in a module named Packard, whether or not a Packard module has been created or released. (Currently, SWORD offers lexicon modules named Robinson and Packard, both for Greek morphology.)

The src attribute is used here to indicate the word position in the original Greek.

====Marking other lemmas====
The lemma attribute of the <tt><w></tt> element can contain any number of other lemmas. Like Strong's numbers and morphology codes, these need to have a workID declared in the header. SWORD presumes that these lemma workIDs all start with "lemma." (note the final period). The portion of the workID following "lemma." should be name of a current, future, or potential lexicon module in which the lemma could be looked up. For example, lemma="lemma.TWOT:271" represents a reference to lemma #271 in a module named TWOT (i.e the Theological Workbook of the Old Testament), whether or not a TWOT module has been created or released. As far as SWORD is concerned, there can be any number of these space-delimited values in a lemma attribute and they can be in any order, even interspersed among the "strong:" lemmas.

SWORD has the ability to show or hide non-Strong's lemmas as a group. See [[DevTools:conf Files#Elements_required_for_proper_rendering|GlobalOptionFilter=OSISLemma]] – for OSIS texts having lemmas.

==== Marking glosses ====
Gloss markup uses the <tt>gloss</tt> attribute of the <tt><w></tt> element. The syntax is illustrated by this line exported from the module JapMeiji.
<pre>
<w gloss="はじめ">元始</w>に<w gloss="かみ">神</w><w gloss="てんち">天地</w>を<w gloss="つくり">創造</w>たまへり
</pre>
Display of glosses can be toggled (in [[Choosing a SWORD program#Module_Support|compatible front-ends]]) by having this line in the [[DevTools:conf Files#Elements_required_for_proper_rendering|.conf file]]:
GlobalOptionFilter=OSISGlosses

===Marking the divine name===
The <tt><divineName></tt> tag is reserved for representations of the tetragrammaton יהוה (YHWH). These occur in the Old Testament as <span style="font-variant: small-caps;">Lord</span>, <span style="font-variant: small-caps;">God</span> and <span style="font-variant: small-caps;">Yah</span>. Not every instance of Lord or God is a translation of this.

The content of the divineName element is the word Lord, God or Yah, not in all upper case (i.e. not LORD, GOD, or YAH). SWORD will either convert it to small-caps or uppercase.

Note, if it is the use is possessive it is permissible to have the following:
<pre>
<divineName>Lord's</divineName>
</pre>

However, it is inadvisable to include any other punctuation within the tag pair. Thus the following is '''not''' good practice (the quotation mark should ''precede'' the start tag):
<pre>
<divineName>“God </divineName> .....”
</pre>

When also marking with Strong's numbers you will need to do it one of two ways:
<pre>
<divineName><w lemma="strong:H3068">Lord's</w><divineName>
or
<w lemma="strong:H3068">of the <seg><divineName>Lord</divineName></seg></w>
</pre>

The latter form uses a workaround to allow the embedding of <tt><divineName></tt> in a <tt><w></tt>, since OSIS does not allow for this, but does allow for <tt><seg></tt> to be in a <tt><w></tt> and to contain <tt><divineName></tt>.

===Marking sections and titles===
A section is marked with:
<pre>
<div type="section">
...
</div>
</pre>

In OSIS the <tt><title></tt> element is used to provide general headings. Titles should be placed at the top of the container that they title, not before.
<pre>
<div type="book">
<title>A book title</title>
<chapter>
<title type="chapter">A title chapter</title>
<div type="section">
<title>A section title</title>
...
</div>
...
</chapter>
</div>
</pre>
Using <tt>type="chapter"</tt> or <tt>type="main"</tt> is needed by [[osis2mod]] to distinguish chapter titles from verse titles. When SWORD stores an OSIS document it does so as an index of verses. It has special indexes for book and chapter titles. SWORD does not store the <tt><verse></tt> tags. So when it comes to storing a title in the following verse, [[osis2mod]] generates special markup to indicate that the title stands before the verse. SWORD uses this to place the verse number.

Note: The <tt><head></tt> element is used to provide headings for tables, lists and cast groups. There are errors in the OSIS 2.1.1 manual that use the <tt><head></tt> incorrectly.

==== Marking pre-verse titles ====
There is no special markup for pre-verse titles. Osis2mod will determine what titles belong to the book, the chapter and otherwise if the above advice is followed.

'''Notes:'''
# See also [[OSIS pre-verse titles]].

===Marking notes===
NB. The examples in this section include the use of sub-identifiers in osisIDs for notes.
<pre>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement
Dieu<note osisRef="Gen.1.1" osisID="Gen.1.1!1" n="1"><hi type="italic">en hébreu</hi> : Élohim,
(<hi type="italic">pluriel d</hi>'Éloah, le Dieu suprême), la Déité, <hi type="italic">dans
le sens absolu</hi>.</note> créa les cieux et la terre.<verse eID="Gen.1.1"/>

<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide<note osisRef="Gen.1.2" osisID="Gen.1.2!1" n="2">le vide.</note>, et il y avait des ténèbres
sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.<verse eID="Gen.1.2"/>
</pre>

{| border="1"
|+Result
|
# Au commencement Dieu¹ créa les cieux et la terre.
# Et la terre était désolation et vide², et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

*¹ ''en hébreu'': Élohim, (''pluriel d'''Éloah, le Dieu suprême), la Déité, ''dans le sens absolu''.
*² le vide.
|}

The note should be attached to what it refers to, either after (as is the case here) or before. There should no additional space surrounding the note, but only what is in the text.

These notes can have any type other than crossReference.

====Marking cross-references notes====
SWORD provides the ability for a user to show or hide cross-references. To achieve this you embed one or more <tt><reference></tt> elements in a <tt><note type="crossReference">...</note></tt>. If this is not done, then the cross-references will always show inline in the text.

<pre>
<note type="crossReference" n="t" osisID="Jer.24.7!crossReference.t">
<reference osisRef="Jer.32.39">ch. 32:39</reference>;
<reference osisRef="Deut.30.6">Deut. 30:6</reference>;
<reference osisRef="Ezek.11.19">Ezek. 11:19</reference>;
<reference osisRef="Ezek.36.26-Ezek.36.27">36:26, 27</reference>
</note>
</pre>

Here is a breakdown.

Regarding the <tt><note></tt> element:

:<tt>type="crossReference"</tt> is one of the predefined OSIS note types. SWORD looks for this value to show/hide cross-references.

:<tt>n="t"</tt> provides the author's desired footnote marker for the note. A couple of SWORD applications use this, but most manufacture their own marker.

:The given osisID is based upon the location of the note. In order to not conflict with the verse's osisID and to construct a unique id, the ! (extension mark, ''aka'' sub-identifier) syntax is used. This is further qualified by the note's type and n value, separated by a dot.

:This note pertains to a single verse and it is given in osisRef.

Regarding the <tt><reference></tt> elements:

:The <tt><reference></tt> element is replaced by SWORD with a link to the reference with the text of the element being shown as link text.

:While the osisRef can point to multiple verses, most SWORD applications cannot handle a link that goes to more than one verse or a contiguous range of verses.

:Here we see that each reference is separated by punctuation (the semicolon at the end of each line).

===Marking variants===
SWORD recognizes the element <seg> with type="x-variant" as marking variants present in different versions of a text. The attribute subType should be added, with a value of "x-1" or "x-2" to indicate whether the reading is the primary or secondary variant. At present, SWORD supports only 2 different readings per text. The method is illustrated below:
<pre>
The text of the Bible
<seg type="x-variant" subType="x-1">may </seg>
<seg type="x-variant" subType="x-2">can </seg>
contain variant readings.
</pre>
This illustrates a primary reading "may " and a secondary reading "can ".

==== Filter ====
Variant readings in OSIS modules may be switched by the SWORD engine when the module conf file includes:
GlobalOptionFilter=OSISVariants
'''Note:'''
# This feature requires the front-end to have been compiled with SWORD version 1.7 or later.

==== Examples ====
* The TR module contains 246 locations where two such variants are marked.
* The WHNU module contains 1473 locations where two variants are marked.

==== SWORD implementation ====
The SWORD API provides for these three choices:
[Primary Reading|Secondary Reading|All Readings]
Example: In Xiphos version 3.1.6, the module context menu provides these three options for any module that has variants.

==Tools==
===Charset conversion===

====Padma====

[http://padma.mozdev.org/ Padma] is a system for transforming Indic text between various public and proprietary formats. This extension applies the technology to Mozilla based applications. Padma is available as an extension for Firefox, Thunderbird, Netscape, Mozilla suite and SeaMonkey platforms. Padma can automatically transform web pages that use dynamic font schemes to Unicode.

Padma can be customised to include a user supplied conversion. This implies that its use is not restricted to Indic texts. See [http://tidbits.co.in/content/how-write-padma-conversion-file].

===Valid OSIS test===
A valid XML document one that is well-formed and conforms to the formal definition provided in a schema (or DTD). A document cannot have elements, attributes, or entities not defined in the schema. A schema can also define ''how'' entities may be nested, the possible values of attributes, etc.

Many programs capable of schema validation exist. Most [http://en.wikipedia.org/wiki/XML_editor XML editors] ([http://xml-copy-editor.sourceforge.net/ XML Copy Editor], [http://en.wikipedia.org/wiki/Oxygen_XML_Editor Oxygen], [http://en.wikipedia.org/wiki/XMLSpy XMLSpy], [http://www.topologi.com/ Topologi], etc.) support some sort of XML schema validation. The Windows based text editor [http://en.wikipedia.org/wiki/Notepad%2B%2B Notepad++] supports Unicode and has an '''XML Tools''' plugin which can perform syntax checking and validation.<ref>Currently, the XML validation check in Notepad++ does not accept the extended schema location syntax that includes <tt><nowiki>xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"</nowiki></tt>, etc</ref>
<references />

====xmllint====
libxml2, available for Linux, Windows, & MacOS, includes a command-line validator called xmllint. To check that a document is valid against OSIS schema, use the following command. (You need Internet access to validate your document.)

<pre>
$ xmllint --noout --schema http://www.bibletechnologies.net/osisCore.2.1.1.xsd myfile.osis.xml
</pre>

To install xmllint, simply install libxml2 via your distribution's standard package management system in Linux or download the Windows binary from our [http://www.crosswire.org/ftpmirror/pub/sword/utils/win32/ mirror].

====Online XML Validators====
The external links section of http://en.wikipedia.org/wiki/XML_Validation lists at least three online validators. Some or all of these can validate against external XML schema.

==Creating a SWORD Module==
Use [[osis2mod]] to create the module.

== See also ==
*[[OSIS]] – a partial list of other OSIS related pages and external links.

*[[Converting SFM Bibles to OSIS]] – describes how to prepare a single OSIS XML file for a Biblical text supplied as several USFM files.

[[Category:Guides|OSIS Bibles]]

[[Category:OSIS|OSIS Bibles]]

[[Category:Morphology|OSIS Bibles]]

OSIS Bibles

2014-02-07T20:24:40Z

Osk: /* Marking variants */

==OSIS==
OSIS is an XML Schema definition for Bibles and other Biblical research texts, which enables ministries and other organizations to collaborate more easily. Traditionally, these organizations have stored their documents in disparate, proprietary markups, making it difficult when they wish to share in service with each other. OSIS provides a common markup for multiple visions.

CrossWire is committed to supporting the OSIS initiative. We have developed OSIS import and export tools which work with our SWORD engine, making OSIS documents available to all of our SWORD software.

The latest OSIS Schema definition and supporting information is available at: http://www.bibletechnologies.net <ref>Link broken. See [[OSIS Tutorial#The_Root_Element|OSIS Tutorial]]</ref>

<references/>

==Introduction==
This page is for practical examples of how to encode a Bible in OSIS 2.1.1 for building a SWORD module with [[osis2mod]]. It represents CrossWire's experience and best practices in creating modules.

Every OSIS SWORD module must be created from a well-formed and valid OSIS 2.1.1 document. While it is a desirable goal for any such document to be acceptable, SWORD has some particular requirements which are discussed here.

The schema for OSIS 2.1.1 that was formally at http://www.bibletechnologies.net/osisCore.2.1.1.xsd is preserved at http://ebible.org/osisCore.2.1.1.xsd

The March 2006 version of the OSIS Manual may be found [http://www.bibletechnologies.net/utilities/fmtdocview.cfm?id=28871A67-D5F5-4381-B22EC4947601628B here] (PDF).

A good example of an OSIS document can be found at http://www.crosswire.org/~dmsmith/kjv2006.

See also [[OSIS Book Abbreviations|OSIS Book Name Abbreviations]].

==General structure==

An OSIS document is a ''well-formed XML'' document, valid according to the ''OSIS schema''.
You can find the full normative description on the OSIS Website [http://www.bibletechnologies.net/].<ref>Link broken. See [[OSIS Tutorial#The_Root_Element|root element]]</ref>

To produce a Bible, you can use this template:

<pre>
<?xml version="1.0" encoding="UTF-8"?>
<osis
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:osis="http://www.bibletechnologies.net/2003/OSIS/namespace"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="{NAME}" osisRefWork="bible" xml:lang="{LANG}" canonical="true">
<header>
{HEADER}
</header>
<div type="bookGroup">
{BODY}
</div>
</osisText>
</osis>
</pre>

With the following values:
; {NAME}: Normalized name of the Bible version (Usually 3 letters for language, 3 for translation)
; {LANG}: IETF language code-- ISO 639-1 codes are preferred, and ISO 639-3 codes are preferred when ISO 639-1 codes do not exist for the given language. See [http://www.sil.org/iso639-3/codes.asp] for a list of codes.
; {HEADER}: Description of the included text; see below
; {BODY}: Text; see below

For text without any character outside ASCII, you can use US-ASCII encoding (usually for english text). For every other language, please use UTF-8 and NFC. See the [[OSIS Bibles#Tools|tools]] section if you need to convert.

<references />
===Header===
This is the minimum contents of the header required for validation. It is also what <tt>usfm2osis.pl</tt> produces. A valid OSIS header may include more than this.
<header>
<work osisWork={Name}/>
</header>

===Body===

Here is the general structure of the body content:

<pre>
<div type="bookGroup">
<title>Old Testament</title>
<div type="book" osisID="Gen" canonical="true">
<title type="main" short="Genesis">Genesis</title>
<chapter osisID="Gen.1" n="1">
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>In the
beginning...<verse eID="Gen.1.1"/>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>The earth was
formless and void...<verse eID="Gen.1.2"/>
...
</chapter>
</div>
</div>
</pre>
'''Notes:'''
# The bookGroup division for the New Testament should have a similar structure.
# Any <tt><div></tt> defaults canonical to false. You need to set it to true on elements representing the structure of the original text.
# A quirk of the SWORD compilation process is that the only kind of content which reliably displays outside of <verse> elements are titles.

===OSIS Milestones===
OSIS allows for two potentially overlapping structures: Document structure (BSP) and verse structure (BCV).

Document structure is dominated by book, sections and paragraphs (BSP), additionally with titles, quotes and poetic material. While verse structure is indicated by book, chapter and verse numbers (BCV). While a SWORD module requires verse structure, the best way to encode a module with deep markup is with document structure. [[Osis2mod]] is responsible for transforming document structure into verse structure.

Because these two systems can overlap and because XML does not allow for overlapping elements, OSIS defines a milestone mechanism for both document and verse structure elements.

For:
<X ... attribute list ...>
...
</X>
the milestoned form is:
<X sID="g1" ... attribute list .../>
...
<X eID="g1"/>

According to the OSIS manual, for any given element X that defines a milestoneable form, all the instances of X in the document must use one form or the other and may not use both. The value of each sID attribute must be unique within the document.

Verse milestone sID/eID attributes can even have values that denote a verse range. This is purely for convenience to human readers.

It is allowable to use milestone elements for verses alone, or for both verses and chapters. The body example above is for the former.

==== Limitations of XML validators ====

An [[OSIS Bibles#Valid_OSIS_test|XML validator]] cannot validate whether OSIS milestones are used properly. It cannot validate:
* that an element is consistently either milestoned or not.
* that for each element with an sID that there is a paired element with an eID.<ref>osis2mod does not crash if the eID milestones are all missing, but the resulting module may appear to be void of text.</ref>
* that each paired sID/eID have the same attribute value.
* that different sID/eID pairs of the same element type do not overlap.<ref>If they weren't milestones, one would say they should be properly nested.</ref>

'''Notes:'''
<references />

==== Notes about OSIS elements ====
* For an OSIS document to be valid it must use the non-milestoned <div> and <lg> elements.
* There is no milestoned version of the <p> element. From a practical perspective, this means that the milestoned verse element should be used when paragraphs are used.
* The milestoned chapter element must be used when the paragraph is spanning over a chapter.
* The SWORD engine cannot handle sub-identifiers separated by ! in an osisID, so osis2mod strips these off from the osisIDs for verses. They are only of use for the osisIDs for notes.

===Recommended approach===
* For chapters, use <chapter>...</chapter> container elements (except in the rare case that other container elements cross chapter boundaries)
* For verses, use milestone elements (unless container elements will suffice) – see [[OSIS Bibles/BSPExample]].
* For paragraphs, use the <p>...</p> container element
* For poetry, use container elements <lg>...</lg> to indicate stanzas (or other types of line groups) and <l>...</l> to indicate lines
* For quoted text, use the <q>...</q> container element
* For translation changes, use the <transChange>...</transChange> container element<ref>Except where the text is within a <w> which is not allowed by OSIS. For these cases use the alternative <seg subType="x-added" type="x-transChange">...</seg> </ref>

<references />

==Examples==
===Marking Paragraphs===
There is no milestoned version of the <tt><p></tt> element. Typically paragraphs surround whole verses. That is, they start and end between verses. If a paragraph begins or ends in a verse and extends beyond that verse, then the whole document must use the milestoned version of <tt><verse></tt>.
<pre>
<div type="book" osisID="Gen" canonical="true">
<title type="main">LE PREMIER LIVRE DE MOÏSE dit LA GENÈSE</title>
<chapter osisID="Gen.1" chapterTitle="Chapitre 1"><title type="chapter">Chapitre 1</title>
<p>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement Dieu créa les
cieux et la terre.<verse eID="Gen.1.1"/>
</p>
<p>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu
planait sur la face des eaux.<verse eID="Gen.1.2"/>
</p>
<p>
<verse sID="Gen.1.3" osisID="Gen.1.3" n="3"/>Et Dieu dit : Que la lumière
soit. Et la lumière fut.<verse eID="Gen.1.3"/>
<verse sID="Gen.1.4" osisID="Gen.1.4" n="4"/>Et Dieu vit la lumière, qu'elle
était bonne ; et Dieu sépara la lumière d'avec les ténèbres.
<verse eID="Gen.1.4"/>
<verse sID="Gen.1.5" osisID="Gen.1.5" n="5"/>Et Dieu appela la lumière Jour ;
et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin :
&#8212; premier jour.<verse eID="Gen.1.5"/>
</p>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Au commencement Dieu créa les cieux et la terre.

<sup>'''(2)'''</sup> Et la terre était désolation et vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

<sup>'''(3)''' </sup>Et Dieu dit : Que la lumière soit. Et la lumière fut. <sup>'''(4)'''</sup> Et Dieu vit la lumière, qu'elle était bonne ; et Dieu sépara la lumière d'avec les ténèbres. <sup>'''(5)'''</sup> Et Dieu appela la lumière Jour ; et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin : — premier jour.
|}

Note: [[osis2mod]] converts a paragraph start into <tt><div type="paragraph" sID="genX"/></tt> and a paragraph end into <tt><div type="paragraph" eID="genX"/></tt>.

===Marking Quotations===
Most of the SWORD front-end applications will show a chapter at a time and some will show isolated verses. This means that all of the SWORD applications show partial quotations, such as the Sermon on the Mount which begins in Matt 5 and ends in Matt 7.

====Default quotation marks====
By default, SWORD will use <tt>"</tt> for quotations. The following describes various ways to influence this.

====Indicating the nesting of a quote====
When a quote is contained in a quote, it is customary to set the level attribute to indicate the depth of the nesting. For example, Jeremiah 23:38 is part of a larger quote and has a back and forth dialog of nested quotes:

<pre>
But if you say,
<q level="2" sID="1"/>
The burden of the Lord,
<q level="2" eID="1"/>
thus says the Lord,
<q level="2" sID="3"/>
Because you have said these words,
<q level="3" sID="4"/>
The burden of the Lord,
<q level="3" eID="4"/>
when I sent to you, saying,
<q level="3" sID="5"/>
You shall not say,
<q level="4" sID="6"/>
The burden of the Lord,
<q level="4" eID="6"/>
<q level="3" eID="5"/>
</pre>
A couple of things to note about this verse. First, the level attribute is on both the sID and the eID pair, matching in value. Second, this is an example of a verse that has a quote that starts in the middle and finishes in another verse.

In this case, SWORD will use the level to determine whether to use <tt>"</tt> or <tt>'</tt> for quotes. Odd levels will use <tt>"</tt> and even levels will use <tt>'</tt>. This is in accordance with American English usage, which is the opposite of British English usage. Nesting levels up to five can be found in the Bible.<ref>Jeremiah 27:1-11; 29:1-28, 30-32; 34:1-5; and Ezekiel 1-36</ref>

<references />

====Supplying alternative quotation marks====
The quote element has a marker attribute that can be used to control the quotation marks. SWORD applications will always use this value when rendering the quote. When the marker attribute is present but empty, it will render no quotation mark at all.

To specify "curly" quotes you can use the following values:
{|border=1
!Description!!Char!!HTML Entity!!Unicode
|-align=center
|Opening double quote||“||&#8220;||U+201C
|-align=center
|Closing double quote||”||&#8221;||U+201D
|-align=center
|Opening single quote||‘||&#8216;||U+2018
|-align=center
|Closing single quote||’||&#8217;||U+2019
|}

To use different marks to start and end a quote, use the milestoned version of the quote.
<q marker="“" sID="qN"/> ... <q marker="”" eID="qN"/>

There is further information about English quotation marks and their usage in [http://en.wikipedia.org/wiki/Quotation_marks].

Quotation marks have a variety of forms in different languages and in different media. See [http://en.wikipedia.org/wiki/Quotation_mark,_non-English_usage Quotation mark, non-English usage].<ref>When modules are being converted from digitized source text used in other Bible software, it may be the case that quotation marks in the text source differ from those in the original published edition, whether due to inherent constraints of the other software, or for other causes.</ref>

'''Note:'''
<references />

====Continuation quotation marks====
The <tt><milestone type="cQuote"/></tt> can be used to indicate the presence of a continued quote. If the marker attribute is present, it will use that otherwise it will use a straight double quote, ". Since there is no level attribute on the milestone element, it is best to specify the marker attribute.

====Marking the Words of Christ====
To indicate that a quote is something that Jesus said<ref>http://en.wikipedia.org/wiki/Red_letter_edition</ref>, use the attribute <tt>who="Jesus"</tt>.

<pre>
<verse osisID="Luke.22.35 sID="Luke.22.35"/>
Then Jesus asked them, <q who="Jesus" marker="">When I sent you without purse,
bag or sandals, did you lack anything?</q>
<verse eID="Luke.22.35"/>
</pre>

{| border="1"
|+Result
|
Then Jesus asked them, <span style="color:#A00">When I sent you without purse, bag or sandals, did you lack anything?</span>
|}

'''Note:'''
<references />

===Marking poetic material===
Poetry is marked up with <lg>, line group, and <l>, line, elements. The line element supports indentation with the level attribute. When the level attribute is not present or it is level="1", this should be interpreted as the first level of the line group. When level="2" it is indented relative to level="1". The same is true for each subsequent level.

The level attribute is used to indicate indentation. A value of 1 means no indentation, the same as not specifying a level attribute. A value of 2 means to indent one. And so forth.

<pre>
<chapter osisID="Exod.15" chapterTitle="Chapitre 15"><title type="chapter">Chapter 15</title>
<p>
<verse sID="Exod.15.1" osisID="Exod.15.1" n="1"/>
Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,
</p>
<lg>
<l level="1">I will sing unto the LORD, for he hath triumphed gloriously:</l>
<l level="2">the horse and his rider hath he thrown into the sea.</l>
<verse eID="Exod.15.1"/>

<verse sID="Exod.15.2" osisID="Exod.15.2" n="2"/>
<l level="1">The LORD is my strength and song, and he is become my salvation:</l>
<l level="2">he is my God, and I will prepare him an habitation;</l>
<l level="2">my father's God, and I will exalt him.</l>
<verse eID="Exod.15.2"/>

<verse sID="Exod.15.3" osisID="Exod.15.3" n="3"/>
<l level="1">The LORD is a man of war:</l>
<l level="2">the LORD is his name.</l>
<verse eID="Exod.15.3"/>

<verse sID="Exod.15.4" osisID="Exod.15.4" n="4"/>
<l level="1">Pharaoh's chariots and his host hath he cast into the sea:</l>
<l level="2">his chosen captains also are drowned in the Red sea.</l>
<verse eID="Exod.15.4"/>

<verse sID="Exod.15.5" osisID="Exod.15.5" n="5"/>
<l level="1">The depths have covered them:</l>
<l level="2">they sank into the bottom as a stone.</l>
<verse eID="Exod.15.5"/>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,

::''I will sing unto the LORD, for he hath triumphed gloriously:''
::::''the horse and his rider hath he thrown into the sea.''

::<sup>'''(2)'''</sup> ''The LORD is my strength and song, and he is become my salvation:''
::::''he is my God, and I will prepare him an habitation;''
::::''my father's God, and I will exalt him.''

::<sup>'''(3)'''</sup> ''The LORD is a man of war:''
::::''the LORD is his name.''

::<sup>'''(4)'''</sup> ''Pharaoh's chariots and his host hath he cast into the sea:''
::::''his chosen captains also are drowned in the Red sea.''

::<sup>'''(5)'''</sup> ''The depths have covered them:''
::::''they sank into the bottom as a stone.''
|}

'''Note:'''

# While OSIS defines a milestoned version of the <tt><lg></tt> element, its use (rather than the container version) will not produce a valid XML document. The <tt><l></tt> element can only occur within an <tt><lg></tt> container, so use of <tt><lg/></tt> milestones prevents use of <tt><l></tt> elements.

==== Marking acrostic poetry headings ====
Use <tt>type="acrostic"</tt> as the title attribute for the stanza headings in acrostic passages such as Psalm 119.

=== Marking lemmas & morphology ===
====Marking Strong's numbers====
To mark up [http://en.wikipedia.org/wiki/Strong%27s_Concordance Strong's numbers], you first need to declare a workID in the header of the OSIS document:
<pre>
<header>
...
<work osisWork="strong">
<refSystem>Dict.Strongs</refSystem>
</work>
...
</header>
</pre>

SWORD does not actually use this declaration, but it is required to have a proper OSIS document.

And while OSIS allows arbitrary workIDs, SWORD can only handle "strong" and a few variants.

<pre>
<w lemma="strong:H0853 strong:H03045">knew</w>
</pre>

The <w> element is used to surround the text that is represented by the Strong's number. It may be that the text is a phrase and it may be that more than one Strong's number defines the text.

When more than one Strong's number defines the text, each must be prefixed with a workID and must be separated from each other by a space. (While OSIS allows for the defining of default workIDs, SWORD requires that the workIDs be used.)

The actual Strong's Number should indicate whether it is Hebrew (H) or Greek (G) followed by the number. The number can be 0 padded to 5 digits as in H00001.

====Marking morphology====
In a similar manner to marking with Strong's numbers, morphology can also be noted. Since morphology regards the original language, Strong's numbers will be shown at the same time.

As with Strong's numbers, a workID needs to be defined. Here we are defining one for Robinson's Morphology Codes. And while SWORD will ignore this declaration, "robinson" is hard-coded into SWORD for Greek morphology codes.
<pre>
<header>
...
<work osisWork="robinson">
<refSystem>Dict.Robinson</refSystem>
</work>
...
</header>
</pre>

Example markup of Robinson's Morphology Codes in the KJV module:
<pre>
<w lemma="strong:G3588 strong:G80" morph="robinson:T-APM robinson:N-APM" src="7 8">his brethren</w>
</pre>
In this example, lemma, morph and src form parallel arrays. The first strong: mapping to the first robinson: and the first src value.

The workID should be name of a current, future, or potential lexicon module in which the morphology code could be looked up. For example, morph="packard:D" represents a reference to morphology code "D" in a module named Packard, whether or not a Packard module has been created or released. (Currently, SWORD offers lexicon modules named Robinson and Packard, both for Greek morphology.)

The src attribute is used here to indicate the word position in the original Greek.

====Marking other lemmas====
The lemma attribute of the <tt><w></tt> element can contain any number of other lemmas. Like Strong's numbers and morphology codes, these need to have a workID declared in the header. SWORD presumes that these lemma workIDs all start with "lemma." (note the final period). The portion of the workID following "lemma." should be name of a current, future, or potential lexicon module in which the lemma could be looked up. For example, lemma="lemma.TWOT:271" represents a reference to lemma #271 in a module named TWOT (i.e the Theological Workbook of the Old Testament), whether or not a TWOT module has been created or released. As far as SWORD is concerned, there can be any number of these space-delimited values in a lemma attribute and they can be in any order, even interspersed among the "strong:" lemmas.

SWORD has the ability to show or hide non-Strong's lemmas as a group. See [[DevTools:conf Files#Elements_required_for_proper_rendering|GlobalOptionFilter=OSISLemma]] – for OSIS texts having lemmas.

==== Marking glosses ====
Gloss markup uses the <tt>gloss</tt> attribute of the <tt><w></tt> element. The syntax is illustrated by this line exported from the module JapMeiji.
<pre>
<w gloss="はじめ">元始</w>に<w gloss="かみ">神</w><w gloss="てんち">天地</w>を<w gloss="つくり">創造</w>たまへり
</pre>
Display of glosses can be toggled (in [[Choosing a SWORD program#Module_Support|compatible front-ends]]) by having this line in the [[DevTools:conf Files#Elements_required_for_proper_rendering|.conf file]]:
GlobalOptionFilter=OSISGlosses

===Marking the divine name===
The <tt><divineName></tt> tag is reserved for representations of the tetragrammaton יהוה (YHWH). These occur in the Old Testament as <span style="font-variant: small-caps;">Lord</span>, <span style="font-variant: small-caps;">God</span> and <span style="font-variant: small-caps;">Yah</span>. Not every instance of Lord or God is a translation of this.

The content of the divineName element is the word Lord, God or Yah, not in all upper case (i.e. not LORD, GOD, or YAH). SWORD will either convert it to small-caps or uppercase.

Note, if it is the use is possessive it is permissible to have the following:
<pre>
<divineName>Lord's</divineName>
</pre>

However, it is inadvisable to include any other punctuation within the tag pair. Thus the following is '''not''' good practice (the quotation mark should ''precede'' the start tag):
<pre>
<divineName>“God </divineName> .....”
</pre>

When also marking with Strong's numbers you will need to do it one of two ways:
<pre>
<divineName><w lemma="strong:H3068">Lord's</w><divineName>
or
<w lemma="strong:H3068">of the <seg><divineName>Lord</divineName></seg></w>
</pre>

The latter form uses a workaround to allow the embedding of <tt><divineName></tt> in a <tt><w></tt>, since OSIS does not allow for this, but does allow for <tt><seg></tt> to be in a <tt><w></tt> and to contain <tt><divineName></tt>.

===Marking sections and titles===
A section is marked with:
<pre>
<div type="section">
...
</div>
</pre>

In OSIS the <tt><title></tt> element is used to provide general headings. Titles should be placed at the top of the container that they title, not before.
<pre>
<div type="book">
<title>A book title</title>
<chapter>
<title type="chapter">A title chapter</title>
<div type="section">
<title>A section title</title>
...
</div>
...
</chapter>
</div>
</pre>
Using <tt>type="chapter"</tt> or <tt>type="main"</tt> is needed by [[osis2mod]] to distinguish chapter titles from verse titles. When SWORD stores an OSIS document it does so as an index of verses. It has special indexes for book and chapter titles. SWORD does not store the <tt><verse></tt> tags. So when it comes to storing a title in the following verse, [[osis2mod]] generates special markup to indicate that the title stands before the verse. SWORD uses this to place the verse number.

Note: The <tt><head></tt> element is used to provide headings for tables, lists and cast groups. There are errors in the OSIS 2.1.1 manual that use the <tt><head></tt> incorrectly.

==== Marking pre-verse titles ====
There is no special markup for pre-verse titles. Osis2mod will determine what titles belong to the book, the chapter and otherwise if the above advice is followed.

'''Notes:'''
# See also [[OSIS pre-verse titles]].

===Marking notes===
NB. The examples in this section include the use of sub-identifiers in osisIDs for notes.
<pre>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement
Dieu<note osisRef="Gen.1.1" osisID="Gen.1.1!1" n="1"><hi type="italic">en hébreu</hi> : Élohim,
(<hi type="italic">pluriel d</hi>'Éloah, le Dieu suprême), la Déité, <hi type="italic">dans
le sens absolu</hi>.</note> créa les cieux et la terre.<verse eID="Gen.1.1"/>

<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide<note osisRef="Gen.1.2" osisID="Gen.1.2!1" n="2">le vide.</note>, et il y avait des ténèbres
sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.<verse eID="Gen.1.2"/>
</pre>

{| border="1"
|+Result
|
# Au commencement Dieu¹ créa les cieux et la terre.
# Et la terre était désolation et vide², et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

*¹ ''en hébreu'': Élohim, (''pluriel d'''Éloah, le Dieu suprême), la Déité, ''dans le sens absolu''.
*² le vide.
|}

The note should be attached to what it refers to, either after (as is the case here) or before. There should no additional space surrounding the note, but only what is in the text.

These notes can have any type other than crossReference.

====Marking cross-references notes====
SWORD provides the ability for a user to show or hide cross-references. To achieve this you embed one or more <tt><reference></tt> elements in a <tt><note type="crossReference">...</note></tt>. If this is not done, then the cross-references will always show inline in the text.

<pre>
<note type="crossReference" n="t" osisID="Jer.24.7!crossReference.t">
<reference osisRef="Jer.32.39">ch. 32:39</reference>;
<reference osisRef="Deut.30.6">Deut. 30:6</reference>;
<reference osisRef="Ezek.11.19">Ezek. 11:19</reference>;
<reference osisRef="Ezek.36.26-Ezek.36.27">36:26, 27</reference>
</note>
</pre>

Here is a breakdown.

Regarding the <tt><note></tt> element:

:<tt>type="crossReference"</tt> is one of the predefined OSIS note types. SWORD looks for this value to show/hide cross-references.

:<tt>n="t"</tt> provides the author's desired footnote marker for the note. A couple of SWORD applications use this, but most manufacture their own marker.

:The given osisID is based upon the location of the note. In order to not conflict with the verse's osisID and to construct a unique id, the ! (extension mark, ''aka'' sub-identifier) syntax is used. This is further qualified by the note's type and n value, separated by a dot.

:This note pertains to a single verse and it is given in osisRef.

Regarding the <tt><reference></tt> elements:

:The <tt><reference></tt> element is replaced by SWORD with a link to the reference with the text of the element being shown as link text.

:While the osisRef can point to multiple verses, most SWORD applications cannot handle a link that goes to more than one verse or a contiguous range of verses.

:Here we see that each reference is separated by punctuation (the semicolon at the end of each line).

===Marking variants===
SWORD recognizes the element <seg> with type="x-variant" as marking variants present in different versions of a text. The attribute subType should be added, with a value of "x-1" or "x-2" to indicate whether the reading is the primary or secondary variant. At present, SWORD supports only 2 different readings per text. The method is illustrated below:
<pre>
The text of the Bible
<seg type="x-variant" subType="x-1">may </seg>
<seg type="x-variant" subType="x-2">can </seg>
contain variant readings.
</pre>
This illustrates a primary reading "may " and a secondary reading "can ".

==== Filter ====
Variant readings in OSIS modules may be switched by the SWORD engine when the module conf file includes:
GlobalOptionFilter=OSISVariants
'''Note:'''
# This feature requires the front-end to have been compiled with SWORD version 1.7 or later.

==== Examples ====
* The TR module contains 246 locations where two such variants are marked.
* The WHNU module contains 1473 locations where two variants are marked.

==== SWORD implementation ====
The SWORD API provides for these three choices:
[Primary Reading|Secondary Reading|All Readings]
Example: In Xiphos version 3.1.6, the module context menu provides these three options for any module that has variants.

==Tools==
===Charset conversion===

====Padma====

[http://padma.mozdev.org/ Padma] is a system for transforming Indic text between various public and proprietary formats. This extension applies the technology to Mozilla based applications. Padma is available as an extension for Firefox, Thunderbird, Netscape, Mozilla suite and SeaMonkey platforms. Padma can automatically transform web pages that use dynamic font schemes to Unicode.

Padma can be customised to include a user supplied conversion. This implies that its use is not restricted to Indic texts. See [http://tidbits.co.in/content/how-write-padma-conversion-file].

===Valid OSIS test===
A valid XML document one that is well-formed and conforms to the formal definition provided in a schema (or DTD). A document cannot have elements, attributes, or entities not defined in the schema. A schema can also define ''how'' entities may be nested, the possible values of attributes, etc.

Many programs capable of schema validation exist. Most [http://en.wikipedia.org/wiki/XML_editor XML editors] ([http://xml-copy-editor.sourceforge.net/ XML Copy Editor], [http://en.wikipedia.org/wiki/Oxygen_XML_Editor Oxygen], [http://en.wikipedia.org/wiki/XMLSpy XMLSpy], [http://www.topologi.com/ Topologi], etc.) support some sort of XML schema validation. The Windows based text editor [http://en.wikipedia.org/wiki/Notepad%2B%2B Notepad++] supports Unicode and has an '''XML Tools''' plugin which can perform syntax checking and validation.<ref>Currently, the XML validation check in Notepad++ does not accept the extended schema location syntax that includes <tt><nowiki>xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"</nowiki></tt>, etc</ref>
<references />

====xmllint====
libxml2, available for Linux, Windows, & MacOS, includes a command-line validator called xmllint. To check that a document is valid against OSIS schema, use the following command. (You need Internet access to validate your document.)

<pre>
$ xmllint --noout --schema http://www.bibletechnologies.net/osisCore.2.1.1.xsd myfile.osis.xml
</pre>

To install xmllint, simply install libxml2 via your distribution's standard package management system in Linux or download the Windows binary from our [http://www.crosswire.org/ftpmirror/pub/sword/utils/win32/ mirror].

====Online XML Validators====
The external links section of http://en.wikipedia.org/wiki/XML_Validation lists at least three online validators. Some or all of these can validate against external XML schema.

==Creating a SWORD Module==
Use [[osis2mod]] to create the module.

== See also ==
*[[OSIS]] – a partial list of other OSIS related pages and external links.

*[[Converting SFM Bibles to OSIS]] – describes how to prepare a single OSIS XML file for a Biblical text supplied as several USFM files.

[[Category:Guides|OSIS Bibles]]

[[Category:OSIS|OSIS Bibles]]

[[Category:Morphology|OSIS Bibles]]

OSIS Bibles

2014-02-07T20:22:50Z

Osk: /* Marking the Divine Name */

==OSIS==
OSIS is an XML Schema definition for Bibles and other Biblical research texts, which enables ministries and other organizations to collaborate more easily. Traditionally, these organizations have stored their documents in disparate, proprietary markups, making it difficult when they wish to share in service with each other. OSIS provides a common markup for multiple visions.

CrossWire is committed to supporting the OSIS initiative. We have developed OSIS import and export tools which work with our SWORD engine, making OSIS documents available to all of our SWORD software.

The latest OSIS Schema definition and supporting information is available at: http://www.bibletechnologies.net <ref>Link broken. See [[OSIS Tutorial#The_Root_Element|OSIS Tutorial]]</ref>

<references/>

==Introduction==
This page is for practical examples of how to encode a Bible in OSIS 2.1.1 for building a SWORD module with [[osis2mod]]. It represents CrossWire's experience and best practices in creating modules.

Every OSIS SWORD module must be created from a well-formed and valid OSIS 2.1.1 document. While it is a desirable goal for any such document to be acceptable, SWORD has some particular requirements which are discussed here.

The schema for OSIS 2.1.1 that was formally at http://www.bibletechnologies.net/osisCore.2.1.1.xsd is preserved at http://ebible.org/osisCore.2.1.1.xsd

The March 2006 version of the OSIS Manual may be found [http://www.bibletechnologies.net/utilities/fmtdocview.cfm?id=28871A67-D5F5-4381-B22EC4947601628B here] (PDF).

A good example of an OSIS document can be found at http://www.crosswire.org/~dmsmith/kjv2006.

See also [[OSIS Book Abbreviations|OSIS Book Name Abbreviations]].

==General structure==

An OSIS document is a ''well-formed XML'' document, valid according to the ''OSIS schema''.
You can find the full normative description on the OSIS Website [http://www.bibletechnologies.net/].<ref>Link broken. See [[OSIS Tutorial#The_Root_Element|root element]]</ref>

To produce a Bible, you can use this template:

<pre>
<?xml version="1.0" encoding="UTF-8"?>
<osis
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:osis="http://www.bibletechnologies.net/2003/OSIS/namespace"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="{NAME}" osisRefWork="bible" xml:lang="{LANG}" canonical="true">
<header>
{HEADER}
</header>
<div type="bookGroup">
{BODY}
</div>
</osisText>
</osis>
</pre>

With the following values:
; {NAME}: Normalized name of the Bible version (Usually 3 letters for language, 3 for translation)
; {LANG}: IETF language code-- ISO 639-1 codes are preferred, and ISO 639-3 codes are preferred when ISO 639-1 codes do not exist for the given language. See [http://www.sil.org/iso639-3/codes.asp] for a list of codes.
; {HEADER}: Description of the included text; see below
; {BODY}: Text; see below

For text without any character outside ASCII, you can use US-ASCII encoding (usually for english text). For every other language, please use UTF-8 and NFC. See the [[OSIS Bibles#Tools|tools]] section if you need to convert.

<references />
===Header===
This is the minimum contents of the header required for validation. It is also what <tt>usfm2osis.pl</tt> produces. A valid OSIS header may include more than this.
<header>
<work osisWork={Name}/>
</header>

===Body===

Here is the general structure of the body content:

<pre>
<div type="bookGroup">
<title>Old Testament</title>
<div type="book" osisID="Gen" canonical="true">
<title type="main" short="Genesis">Genesis</title>
<chapter osisID="Gen.1" n="1">
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>In the
beginning...<verse eID="Gen.1.1"/>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>The earth was
formless and void...<verse eID="Gen.1.2"/>
...
</chapter>
</div>
</div>
</pre>
'''Notes:'''
# The bookGroup division for the New Testament should have a similar structure.
# Any <tt><div></tt> defaults canonical to false. You need to set it to true on elements representing the structure of the original text.
# A quirk of the SWORD compilation process is that the only kind of content which reliably displays outside of <verse> elements are titles.

===OSIS Milestones===
OSIS allows for two potentially overlapping structures: Document structure (BSP) and verse structure (BCV).

Document structure is dominated by book, sections and paragraphs (BSP), additionally with titles, quotes and poetic material. While verse structure is indicated by book, chapter and verse numbers (BCV). While a SWORD module requires verse structure, the best way to encode a module with deep markup is with document structure. [[Osis2mod]] is responsible for transforming document structure into verse structure.

Because these two systems can overlap and because XML does not allow for overlapping elements, OSIS defines a milestone mechanism for both document and verse structure elements.

For:
<X ... attribute list ...>
...
</X>
the milestoned form is:
<X sID="g1" ... attribute list .../>
...
<X eID="g1"/>

According to the OSIS manual, for any given element X that defines a milestoneable form, all the instances of X in the document must use one form or the other and may not use both. The value of each sID attribute must be unique within the document.

Verse milestone sID/eID attributes can even have values that denote a verse range. This is purely for convenience to human readers.

It is allowable to use milestone elements for verses alone, or for both verses and chapters. The body example above is for the former.

==== Limitations of XML validators ====

An [[OSIS Bibles#Valid_OSIS_test|XML validator]] cannot validate whether OSIS milestones are used properly. It cannot validate:
* that an element is consistently either milestoned or not.
* that for each element with an sID that there is a paired element with an eID.<ref>osis2mod does not crash if the eID milestones are all missing, but the resulting module may appear to be void of text.</ref>
* that each paired sID/eID have the same attribute value.
* that different sID/eID pairs of the same element type do not overlap.<ref>If they weren't milestones, one would say they should be properly nested.</ref>

'''Notes:'''
<references />

==== Notes about OSIS elements ====
* For an OSIS document to be valid it must use the non-milestoned <div> and <lg> elements.
* There is no milestoned version of the <p> element. From a practical perspective, this means that the milestoned verse element should be used when paragraphs are used.
* The milestoned chapter element must be used when the paragraph is spanning over a chapter.
* The SWORD engine cannot handle sub-identifiers separated by ! in an osisID, so osis2mod strips these off from the osisIDs for verses. They are only of use for the osisIDs for notes.

===Recommended approach===
* For chapters, use <chapter>...</chapter> container elements (except in the rare case that other container elements cross chapter boundaries)
* For verses, use milestone elements (unless container elements will suffice) – see [[OSIS Bibles/BSPExample]].
* For paragraphs, use the <p>...</p> container element
* For poetry, use container elements <lg>...</lg> to indicate stanzas (or other types of line groups) and <l>...</l> to indicate lines
* For quoted text, use the <q>...</q> container element
* For translation changes, use the <transChange>...</transChange> container element<ref>Except where the text is within a <w> which is not allowed by OSIS. For these cases use the alternative <seg subType="x-added" type="x-transChange">...</seg> </ref>

<references />

==Examples==
===Marking Paragraphs===
There is no milestoned version of the <tt><p></tt> element. Typically paragraphs surround whole verses. That is, they start and end between verses. If a paragraph begins or ends in a verse and extends beyond that verse, then the whole document must use the milestoned version of <tt><verse></tt>.
<pre>
<div type="book" osisID="Gen" canonical="true">
<title type="main">LE PREMIER LIVRE DE MOÏSE dit LA GENÈSE</title>
<chapter osisID="Gen.1" chapterTitle="Chapitre 1"><title type="chapter">Chapitre 1</title>
<p>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement Dieu créa les
cieux et la terre.<verse eID="Gen.1.1"/>
</p>
<p>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu
planait sur la face des eaux.<verse eID="Gen.1.2"/>
</p>
<p>
<verse sID="Gen.1.3" osisID="Gen.1.3" n="3"/>Et Dieu dit : Que la lumière
soit. Et la lumière fut.<verse eID="Gen.1.3"/>
<verse sID="Gen.1.4" osisID="Gen.1.4" n="4"/>Et Dieu vit la lumière, qu'elle
était bonne ; et Dieu sépara la lumière d'avec les ténèbres.
<verse eID="Gen.1.4"/>
<verse sID="Gen.1.5" osisID="Gen.1.5" n="5"/>Et Dieu appela la lumière Jour ;
et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin :
&#8212; premier jour.<verse eID="Gen.1.5"/>
</p>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Au commencement Dieu créa les cieux et la terre.

<sup>'''(2)'''</sup> Et la terre était désolation et vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

<sup>'''(3)''' </sup>Et Dieu dit : Que la lumière soit. Et la lumière fut. <sup>'''(4)'''</sup> Et Dieu vit la lumière, qu'elle était bonne ; et Dieu sépara la lumière d'avec les ténèbres. <sup>'''(5)'''</sup> Et Dieu appela la lumière Jour ; et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin : — premier jour.
|}

Note: [[osis2mod]] converts a paragraph start into <tt><div type="paragraph" sID="genX"/></tt> and a paragraph end into <tt><div type="paragraph" eID="genX"/></tt>.

===Marking Quotations===
Most of the SWORD front-end applications will show a chapter at a time and some will show isolated verses. This means that all of the SWORD applications show partial quotations, such as the Sermon on the Mount which begins in Matt 5 and ends in Matt 7.

====Default quotation marks====
By default, SWORD will use <tt>"</tt> for quotations. The following describes various ways to influence this.

====Indicating the nesting of a quote====
When a quote is contained in a quote, it is customary to set the level attribute to indicate the depth of the nesting. For example, Jeremiah 23:38 is part of a larger quote and has a back and forth dialog of nested quotes:

<pre>
But if you say,
<q level="2" sID="1"/>
The burden of the Lord,
<q level="2" eID="1"/>
thus says the Lord,
<q level="2" sID="3"/>
Because you have said these words,
<q level="3" sID="4"/>
The burden of the Lord,
<q level="3" eID="4"/>
when I sent to you, saying,
<q level="3" sID="5"/>
You shall not say,
<q level="4" sID="6"/>
The burden of the Lord,
<q level="4" eID="6"/>
<q level="3" eID="5"/>
</pre>
A couple of things to note about this verse. First, the level attribute is on both the sID and the eID pair, matching in value. Second, this is an example of a verse that has a quote that starts in the middle and finishes in another verse.

In this case, SWORD will use the level to determine whether to use <tt>"</tt> or <tt>'</tt> for quotes. Odd levels will use <tt>"</tt> and even levels will use <tt>'</tt>. This is in accordance with American English usage, which is the opposite of British English usage. Nesting levels up to five can be found in the Bible.<ref>Jeremiah 27:1-11; 29:1-28, 30-32; 34:1-5; and Ezekiel 1-36</ref>

<references />

====Supplying alternative quotation marks====
The quote element has a marker attribute that can be used to control the quotation marks. SWORD applications will always use this value when rendering the quote. When the marker attribute is present but empty, it will render no quotation mark at all.

To specify "curly" quotes you can use the following values:
{|border=1
!Description!!Char!!HTML Entity!!Unicode
|-align=center
|Opening double quote||“||&#8220;||U+201C
|-align=center
|Closing double quote||”||&#8221;||U+201D
|-align=center
|Opening single quote||‘||&#8216;||U+2018
|-align=center
|Closing single quote||’||&#8217;||U+2019
|}

To use different marks to start and end a quote, use the milestoned version of the quote.
<q marker="“" sID="qN"/> ... <q marker="”" eID="qN"/>

There is further information about English quotation marks and their usage in [http://en.wikipedia.org/wiki/Quotation_marks].

Quotation marks have a variety of forms in different languages and in different media. See [http://en.wikipedia.org/wiki/Quotation_mark,_non-English_usage Quotation mark, non-English usage].<ref>When modules are being converted from digitized source text used in other Bible software, it may be the case that quotation marks in the text source differ from those in the original published edition, whether due to inherent constraints of the other software, or for other causes.</ref>

'''Note:'''
<references />

====Continuation quotation marks====
The <tt><milestone type="cQuote"/></tt> can be used to indicate the presence of a continued quote. If the marker attribute is present, it will use that otherwise it will use a straight double quote, ". Since there is no level attribute on the milestone element, it is best to specify the marker attribute.

====Marking the Words of Christ====
To indicate that a quote is something that Jesus said<ref>http://en.wikipedia.org/wiki/Red_letter_edition</ref>, use the attribute <tt>who="Jesus"</tt>.

<pre>
<verse osisID="Luke.22.35 sID="Luke.22.35"/>
Then Jesus asked them, <q who="Jesus" marker="">When I sent you without purse,
bag or sandals, did you lack anything?</q>
<verse eID="Luke.22.35"/>
</pre>

{| border="1"
|+Result
|
Then Jesus asked them, <span style="color:#A00">When I sent you without purse, bag or sandals, did you lack anything?</span>
|}

'''Note:'''
<references />

===Marking poetic material===
Poetry is marked up with <lg>, line group, and <l>, line, elements. The line element supports indentation with the level attribute. When the level attribute is not present or it is level="1", this should be interpreted as the first level of the line group. When level="2" it is indented relative to level="1". The same is true for each subsequent level.

The level attribute is used to indicate indentation. A value of 1 means no indentation, the same as not specifying a level attribute. A value of 2 means to indent one. And so forth.

<pre>
<chapter osisID="Exod.15" chapterTitle="Chapitre 15"><title type="chapter">Chapter 15</title>
<p>
<verse sID="Exod.15.1" osisID="Exod.15.1" n="1"/>
Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,
</p>
<lg>
<l level="1">I will sing unto the LORD, for he hath triumphed gloriously:</l>
<l level="2">the horse and his rider hath he thrown into the sea.</l>
<verse eID="Exod.15.1"/>

<verse sID="Exod.15.2" osisID="Exod.15.2" n="2"/>
<l level="1">The LORD is my strength and song, and he is become my salvation:</l>
<l level="2">he is my God, and I will prepare him an habitation;</l>
<l level="2">my father's God, and I will exalt him.</l>
<verse eID="Exod.15.2"/>

<verse sID="Exod.15.3" osisID="Exod.15.3" n="3"/>
<l level="1">The LORD is a man of war:</l>
<l level="2">the LORD is his name.</l>
<verse eID="Exod.15.3"/>

<verse sID="Exod.15.4" osisID="Exod.15.4" n="4"/>
<l level="1">Pharaoh's chariots and his host hath he cast into the sea:</l>
<l level="2">his chosen captains also are drowned in the Red sea.</l>
<verse eID="Exod.15.4"/>

<verse sID="Exod.15.5" osisID="Exod.15.5" n="5"/>
<l level="1">The depths have covered them:</l>
<l level="2">they sank into the bottom as a stone.</l>
<verse eID="Exod.15.5"/>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,

::''I will sing unto the LORD, for he hath triumphed gloriously:''
::::''the horse and his rider hath he thrown into the sea.''

::<sup>'''(2)'''</sup> ''The LORD is my strength and song, and he is become my salvation:''
::::''he is my God, and I will prepare him an habitation;''
::::''my father's God, and I will exalt him.''

::<sup>'''(3)'''</sup> ''The LORD is a man of war:''
::::''the LORD is his name.''

::<sup>'''(4)'''</sup> ''Pharaoh's chariots and his host hath he cast into the sea:''
::::''his chosen captains also are drowned in the Red sea.''

::<sup>'''(5)'''</sup> ''The depths have covered them:''
::::''they sank into the bottom as a stone.''
|}

'''Note:'''

# While OSIS defines a milestoned version of the <tt><lg></tt> element, its use (rather than the container version) will not produce a valid XML document. The <tt><l></tt> element can only occur within an <tt><lg></tt> container, so use of <tt><lg/></tt> milestones prevents use of <tt><l></tt> elements.

==== Marking acrostic poetry headings ====
Use <tt>type="acrostic"</tt> as the title attribute for the stanza headings in acrostic passages such as Psalm 119.

=== Marking lemmas & morphology ===
====Marking Strong's numbers====
To mark up [http://en.wikipedia.org/wiki/Strong%27s_Concordance Strong's numbers], you first need to declare a workID in the header of the OSIS document:
<pre>
<header>
...
<work osisWork="strong">
<refSystem>Dict.Strongs</refSystem>
</work>
...
</header>
</pre>

SWORD does not actually use this declaration, but it is required to have a proper OSIS document.

And while OSIS allows arbitrary workIDs, SWORD can only handle "strong" and a few variants.

<pre>
<w lemma="strong:H0853 strong:H03045">knew</w>
</pre>

The <w> element is used to surround the text that is represented by the Strong's number. It may be that the text is a phrase and it may be that more than one Strong's number defines the text.

When more than one Strong's number defines the text, each must be prefixed with a workID and must be separated from each other by a space. (While OSIS allows for the defining of default workIDs, SWORD requires that the workIDs be used.)

The actual Strong's Number should indicate whether it is Hebrew (H) or Greek (G) followed by the number. The number can be 0 padded to 5 digits as in H00001.

====Marking morphology====
In a similar manner to marking with Strong's numbers, morphology can also be noted. Since morphology regards the original language, Strong's numbers will be shown at the same time.

As with Strong's numbers, a workID needs to be defined. Here we are defining one for Robinson's Morphology Codes. And while SWORD will ignore this declaration, "robinson" is hard-coded into SWORD for Greek morphology codes.
<pre>
<header>
...
<work osisWork="robinson">
<refSystem>Dict.Robinson</refSystem>
</work>
...
</header>
</pre>

Example markup of Robinson's Morphology Codes in the KJV module:
<pre>
<w lemma="strong:G3588 strong:G80" morph="robinson:T-APM robinson:N-APM" src="7 8">his brethren</w>
</pre>
In this example, lemma, morph and src form parallel arrays. The first strong: mapping to the first robinson: and the first src value.

The workID should be name of a current, future, or potential lexicon module in which the morphology code could be looked up. For example, morph="packard:D" represents a reference to morphology code "D" in a module named Packard, whether or not a Packard module has been created or released. (Currently, SWORD offers lexicon modules named Robinson and Packard, both for Greek morphology.)

The src attribute is used here to indicate the word position in the original Greek.

====Marking other lemmas====
The lemma attribute of the <tt><w></tt> element can contain any number of other lemmas. Like Strong's numbers and morphology codes, these need to have a workID declared in the header. SWORD presumes that these lemma workIDs all start with "lemma." (note the final period). The portion of the workID following "lemma." should be name of a current, future, or potential lexicon module in which the lemma could be looked up. For example, lemma="lemma.TWOT:271" represents a reference to lemma #271 in a module named TWOT (i.e the Theological Workbook of the Old Testament), whether or not a TWOT module has been created or released. As far as SWORD is concerned, there can be any number of these space-delimited values in a lemma attribute and they can be in any order, even interspersed among the "strong:" lemmas.

SWORD has the ability to show or hide non-Strong's lemmas as a group. See [[DevTools:conf Files#Elements_required_for_proper_rendering|GlobalOptionFilter=OSISLemma]] – for OSIS texts having lemmas.

==== Marking glosses ====
Gloss markup uses the <tt>gloss</tt> attribute of the <tt><w></tt> element. The syntax is illustrated by this line exported from the module JapMeiji.
<pre>
<w gloss="はじめ">元始</w>に<w gloss="かみ">神</w><w gloss="てんち">天地</w>を<w gloss="つくり">創造</w>たまへり
</pre>
Display of glosses can be toggled (in [[Choosing a SWORD program#Module_Support|compatible front-ends]]) by having this line in the [[DevTools:conf Files#Elements_required_for_proper_rendering|.conf file]]:
GlobalOptionFilter=OSISGlosses

===Marking the divine name===
The <tt><divineName></tt> tag is reserved for representations of the tetragrammaton יהוה (YHWH). These occur in the Old Testament as <span style="font-variant: small-caps;">Lord</span>, <span style="font-variant: small-caps;">God</span> and <span style="font-variant: small-caps;">Yah</span>. Not every instance of Lord or God is a translation of this.

The content of the divineName element is the word Lord, God or Yah, not in all upper case (i.e. not LORD, GOD, or YAH). SWORD will either convert it to small-caps or uppercase.

Note, if it is the use is possessive it is permissible to have the following:
<pre>
<divineName>Lord's</divineName>
</pre>

However, it is inadvisable to include any other punctuation within the tag pair. Thus the following is '''not''' good practice (the quotation mark should ''precede'' the start tag):
<pre>
<divineName>“God </divineName> .....”
</pre>

When also marking with Strong's numbers you will need to do it one of two ways:
<pre>
<divineName><w lemma="strong:H3068">Lord's</w><divineName>
or
<w lemma="strong:H3068">of the <seg><divineName>Lord</divineName></seg></w>
</pre>

The latter form uses a workaround to allow the embedding of <tt><divineName></tt> in a <tt><w></tt>, since OSIS does not allow for this, but does allow for <tt><seg></tt> to be in a <tt><w></tt> and to contain <tt><divineName></tt>.

===Marking sections and titles===
A section is marked with:
<pre>
<div type="section">
...
</div>
</pre>

In OSIS the <tt><title></tt> element is used to provide general headings. Titles should be placed at the top of the container that they title, not before.
<pre>
<div type="book">
<title>A book title</title>
<chapter>
<title type="chapter">A title chapter</title>
<div type="section">
<title>A section title</title>
...
</div>
...
</chapter>
</div>
</pre>
Using <tt>type="chapter"</tt> or <tt>type="main"</tt> is needed by [[osis2mod]] to distinguish chapter titles from verse titles. When SWORD stores an OSIS document it does so as an index of verses. It has special indexes for book and chapter titles. SWORD does not store the <tt><verse></tt> tags. So when it comes to storing a title in the following verse, [[osis2mod]] generates special markup to indicate that the title stands before the verse. SWORD uses this to place the verse number.

Note: The <tt><head></tt> element is used to provide headings for tables, lists and cast groups. There are errors in the OSIS 2.1.1 manual that use the <tt><head></tt> incorrectly.

==== Marking pre-verse titles ====
There is no special markup for pre-verse titles. Osis2mod will determine what titles belong to the book, the chapter and otherwise if the above advice is followed.

'''Notes:'''
# See also [[OSIS pre-verse titles]].

===Marking notes===
NB. The examples in this section include the use of sub-identifiers in osisIDs for notes.
<pre>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement
Dieu<note osisRef="Gen.1.1" osisID="Gen.1.1!1" n="1"><hi type="italic">en hébreu</hi> : Élohim,
(<hi type="italic">pluriel d</hi>'Éloah, le Dieu suprême), la Déité, <hi type="italic">dans
le sens absolu</hi>.</note> créa les cieux et la terre.<verse eID="Gen.1.1"/>

<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide<note osisRef="Gen.1.2" osisID="Gen.1.2!1" n="2">le vide.</note>, et il y avait des ténèbres
sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.<verse eID="Gen.1.2"/>
</pre>

{| border="1"
|+Result
|
# Au commencement Dieu¹ créa les cieux et la terre.
# Et la terre était désolation et vide², et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

*¹ ''en hébreu'': Élohim, (''pluriel d'''Éloah, le Dieu suprême), la Déité, ''dans le sens absolu''.
*² le vide.
|}

The note should be attached to what it refers to, either after (as is the case here) or before. There should no additional space surrounding the note, but only what is in the text.

These notes can have any type other than crossReference.

====Marking cross-references notes====
SWORD provides the ability for a user to show or hide cross-references. To achieve this you embed one or more <tt><reference></tt> elements in a <tt><note type="crossReference">...</note></tt>. If this is not done, then the cross-references will always show inline in the text.

<pre>
<note type="crossReference" n="t" osisID="Jer.24.7!crossReference.t">
<reference osisRef="Jer.32.39">ch. 32:39</reference>;
<reference osisRef="Deut.30.6">Deut. 30:6</reference>;
<reference osisRef="Ezek.11.19">Ezek. 11:19</reference>;
<reference osisRef="Ezek.36.26-Ezek.36.27">36:26, 27</reference>
</note>
</pre>

Here is a breakdown.

Regarding the <tt><note></tt> element:

:<tt>type="crossReference"</tt> is one of the predefined OSIS note types. SWORD looks for this value to show/hide cross-references.

:<tt>n="t"</tt> provides the author's desired footnote marker for the note. A couple of SWORD applications use this, but most manufacture their own marker.

:The given osisID is based upon the location of the note. In order to not conflict with the verse's osisID and to construct a unique id, the ! (extension mark, ''aka'' sub-identifier) syntax is used. This is further qualified by the note's type and n value, separated by a dot.

:This note pertains to a single verse and it is given in osisRef.

Regarding the <tt><reference></tt> elements:

:The <tt><reference></tt> element is replaced by SWORD with a link to the reference with the text of the element being shown as link text.

:While the osisRef can point to multiple verses, most SWORD applications cannot handle a link that goes to more than one verse or a contiguous range of verses.

:Here we see that each reference is separated by punctuation (the semicolon at the end of each line).

===Marking variants===
SWORD recognizes the element <seg> with type="x-variant" as marking variants present in different versions of a text. The attribute subType should be added, with a value of "x-1" or "x-2" to indicate whether the reading is the primary or secondary variant. At present, SWORD supports only 2 different readings per text. The method is illustrated below:
The text of the Bible <seg type="x-variant" subType="x-1">may </seg><seg type="x-variant" subType="x-2">can </seg>contain variant readings.
This illustrates a primary reading "may " and a secondary reading "can ".

==== Filter ====
Variant readings in OSIS modules may be switched by the SWORD engine when the module conf file includes:
GlobalOptionFilter=OSISVariants
'''Note:'''
# This feature requires the front-end to have been compiled with SWORD version 1.7 or later.

==== Examples ====
* The TR module contains 246 locations where two such variants are marked.
* The WHNU module contains 1473 locations where two variants are marked.

==== SWORD implementation ====
The SWORD API provides for these three choices:
[Primary Reading|Secondary Reading|All Readings]
Example: In Xiphos version 3.1.6, the module context menu provides these three options for any module that has variants.

==Tools==
===Charset conversion===

====Padma====

[http://padma.mozdev.org/ Padma] is a system for transforming Indic text between various public and proprietary formats. This extension applies the technology to Mozilla based applications. Padma is available as an extension for Firefox, Thunderbird, Netscape, Mozilla suite and SeaMonkey platforms. Padma can automatically transform web pages that use dynamic font schemes to Unicode.

Padma can be customised to include a user supplied conversion. This implies that its use is not restricted to Indic texts. See [http://tidbits.co.in/content/how-write-padma-conversion-file].

===Valid OSIS test===
A valid XML document one that is well-formed and conforms to the formal definition provided in a schema (or DTD). A document cannot have elements, attributes, or entities not defined in the schema. A schema can also define ''how'' entities may be nested, the possible values of attributes, etc.

Many programs capable of schema validation exist. Most [http://en.wikipedia.org/wiki/XML_editor XML editors] ([http://xml-copy-editor.sourceforge.net/ XML Copy Editor], [http://en.wikipedia.org/wiki/Oxygen_XML_Editor Oxygen], [http://en.wikipedia.org/wiki/XMLSpy XMLSpy], [http://www.topologi.com/ Topologi], etc.) support some sort of XML schema validation. The Windows based text editor [http://en.wikipedia.org/wiki/Notepad%2B%2B Notepad++] supports Unicode and has an '''XML Tools''' plugin which can perform syntax checking and validation.<ref>Currently, the XML validation check in Notepad++ does not accept the extended schema location syntax that includes <tt><nowiki>xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"</nowiki></tt>, etc</ref>
<references />

====xmllint====
libxml2, available for Linux, Windows, & MacOS, includes a command-line validator called xmllint. To check that a document is valid against OSIS schema, use the following command. (You need Internet access to validate your document.)

<pre>
$ xmllint --noout --schema http://www.bibletechnologies.net/osisCore.2.1.1.xsd myfile.osis.xml
</pre>

To install xmllint, simply install libxml2 via your distribution's standard package management system in Linux or download the Windows binary from our [http://www.crosswire.org/ftpmirror/pub/sword/utils/win32/ mirror].

====Online XML Validators====
The external links section of http://en.wikipedia.org/wiki/XML_Validation lists at least three online validators. Some or all of these can validate against external XML schema.

==Creating a SWORD Module==
Use [[osis2mod]] to create the module.

== See also ==
*[[OSIS]] – a partial list of other OSIS related pages and external links.

*[[Converting SFM Bibles to OSIS]] – describes how to prepare a single OSIS XML file for a Biblical text supplied as several USFM files.

[[Category:Guides|OSIS Bibles]]

[[Category:OSIS|OSIS Bibles]]

[[Category:Morphology|OSIS Bibles]]

OSIS Bibles

2014-02-07T20:22:16Z

Osk: /* Marking the Divine Name */

==OSIS==
OSIS is an XML Schema definition for Bibles and other Biblical research texts, which enables ministries and other organizations to collaborate more easily. Traditionally, these organizations have stored their documents in disparate, proprietary markups, making it difficult when they wish to share in service with each other. OSIS provides a common markup for multiple visions.

CrossWire is committed to supporting the OSIS initiative. We have developed OSIS import and export tools which work with our SWORD engine, making OSIS documents available to all of our SWORD software.

The latest OSIS Schema definition and supporting information is available at: http://www.bibletechnologies.net <ref>Link broken. See [[OSIS Tutorial#The_Root_Element|OSIS Tutorial]]</ref>

<references/>

==Introduction==
This page is for practical examples of how to encode a Bible in OSIS 2.1.1 for building a SWORD module with [[osis2mod]]. It represents CrossWire's experience and best practices in creating modules.

Every OSIS SWORD module must be created from a well-formed and valid OSIS 2.1.1 document. While it is a desirable goal for any such document to be acceptable, SWORD has some particular requirements which are discussed here.

The schema for OSIS 2.1.1 that was formally at http://www.bibletechnologies.net/osisCore.2.1.1.xsd is preserved at http://ebible.org/osisCore.2.1.1.xsd

The March 2006 version of the OSIS Manual may be found [http://www.bibletechnologies.net/utilities/fmtdocview.cfm?id=28871A67-D5F5-4381-B22EC4947601628B here] (PDF).

A good example of an OSIS document can be found at http://www.crosswire.org/~dmsmith/kjv2006.

See also [[OSIS Book Abbreviations|OSIS Book Name Abbreviations]].

==General structure==

An OSIS document is a ''well-formed XML'' document, valid according to the ''OSIS schema''.
You can find the full normative description on the OSIS Website [http://www.bibletechnologies.net/].<ref>Link broken. See [[OSIS Tutorial#The_Root_Element|root element]]</ref>

To produce a Bible, you can use this template:

<pre>
<?xml version="1.0" encoding="UTF-8"?>
<osis
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:osis="http://www.bibletechnologies.net/2003/OSIS/namespace"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="{NAME}" osisRefWork="bible" xml:lang="{LANG}" canonical="true">
<header>
{HEADER}
</header>
<div type="bookGroup">
{BODY}
</div>
</osisText>
</osis>
</pre>

With the following values:
; {NAME}: Normalized name of the Bible version (Usually 3 letters for language, 3 for translation)
; {LANG}: IETF language code-- ISO 639-1 codes are preferred, and ISO 639-3 codes are preferred when ISO 639-1 codes do not exist for the given language. See [http://www.sil.org/iso639-3/codes.asp] for a list of codes.
; {HEADER}: Description of the included text; see below
; {BODY}: Text; see below

For text without any character outside ASCII, you can use US-ASCII encoding (usually for english text). For every other language, please use UTF-8 and NFC. See the [[OSIS Bibles#Tools|tools]] section if you need to convert.

<references />
===Header===
This is the minimum contents of the header required for validation. It is also what <tt>usfm2osis.pl</tt> produces. A valid OSIS header may include more than this.
<header>
<work osisWork={Name}/>
</header>

===Body===

Here is the general structure of the body content:

<pre>
<div type="bookGroup">
<title>Old Testament</title>
<div type="book" osisID="Gen" canonical="true">
<title type="main" short="Genesis">Genesis</title>
<chapter osisID="Gen.1" n="1">
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>In the
beginning...<verse eID="Gen.1.1"/>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>The earth was
formless and void...<verse eID="Gen.1.2"/>
...
</chapter>
</div>
</div>
</pre>
'''Notes:'''
# The bookGroup division for the New Testament should have a similar structure.
# Any <tt><div></tt> defaults canonical to false. You need to set it to true on elements representing the structure of the original text.
# A quirk of the SWORD compilation process is that the only kind of content which reliably displays outside of <verse> elements are titles.

===OSIS Milestones===
OSIS allows for two potentially overlapping structures: Document structure (BSP) and verse structure (BCV).

Document structure is dominated by book, sections and paragraphs (BSP), additionally with titles, quotes and poetic material. While verse structure is indicated by book, chapter and verse numbers (BCV). While a SWORD module requires verse structure, the best way to encode a module with deep markup is with document structure. [[Osis2mod]] is responsible for transforming document structure into verse structure.

Because these two systems can overlap and because XML does not allow for overlapping elements, OSIS defines a milestone mechanism for both document and verse structure elements.

For:
<X ... attribute list ...>
...
</X>
the milestoned form is:
<X sID="g1" ... attribute list .../>
...
<X eID="g1"/>

According to the OSIS manual, for any given element X that defines a milestoneable form, all the instances of X in the document must use one form or the other and may not use both. The value of each sID attribute must be unique within the document.

Verse milestone sID/eID attributes can even have values that denote a verse range. This is purely for convenience to human readers.

It is allowable to use milestone elements for verses alone, or for both verses and chapters. The body example above is for the former.

==== Limitations of XML validators ====

An [[OSIS Bibles#Valid_OSIS_test|XML validator]] cannot validate whether OSIS milestones are used properly. It cannot validate:
* that an element is consistently either milestoned or not.
* that for each element with an sID that there is a paired element with an eID.<ref>osis2mod does not crash if the eID milestones are all missing, but the resulting module may appear to be void of text.</ref>
* that each paired sID/eID have the same attribute value.
* that different sID/eID pairs of the same element type do not overlap.<ref>If they weren't milestones, one would say they should be properly nested.</ref>

'''Notes:'''
<references />

==== Notes about OSIS elements ====
* For an OSIS document to be valid it must use the non-milestoned <div> and <lg> elements.
* There is no milestoned version of the <p> element. From a practical perspective, this means that the milestoned verse element should be used when paragraphs are used.
* The milestoned chapter element must be used when the paragraph is spanning over a chapter.
* The SWORD engine cannot handle sub-identifiers separated by ! in an osisID, so osis2mod strips these off from the osisIDs for verses. They are only of use for the osisIDs for notes.

===Recommended approach===
* For chapters, use <chapter>...</chapter> container elements (except in the rare case that other container elements cross chapter boundaries)
* For verses, use milestone elements (unless container elements will suffice) – see [[OSIS Bibles/BSPExample]].
* For paragraphs, use the <p>...</p> container element
* For poetry, use container elements <lg>...</lg> to indicate stanzas (or other types of line groups) and <l>...</l> to indicate lines
* For quoted text, use the <q>...</q> container element
* For translation changes, use the <transChange>...</transChange> container element<ref>Except where the text is within a <w> which is not allowed by OSIS. For these cases use the alternative <seg subType="x-added" type="x-transChange">...</seg> </ref>

<references />

==Examples==
===Marking Paragraphs===
There is no milestoned version of the <tt><p></tt> element. Typically paragraphs surround whole verses. That is, they start and end between verses. If a paragraph begins or ends in a verse and extends beyond that verse, then the whole document must use the milestoned version of <tt><verse></tt>.
<pre>
<div type="book" osisID="Gen" canonical="true">
<title type="main">LE PREMIER LIVRE DE MOÏSE dit LA GENÈSE</title>
<chapter osisID="Gen.1" chapterTitle="Chapitre 1"><title type="chapter">Chapitre 1</title>
<p>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement Dieu créa les
cieux et la terre.<verse eID="Gen.1.1"/>
</p>
<p>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu
planait sur la face des eaux.<verse eID="Gen.1.2"/>
</p>
<p>
<verse sID="Gen.1.3" osisID="Gen.1.3" n="3"/>Et Dieu dit : Que la lumière
soit. Et la lumière fut.<verse eID="Gen.1.3"/>
<verse sID="Gen.1.4" osisID="Gen.1.4" n="4"/>Et Dieu vit la lumière, qu'elle
était bonne ; et Dieu sépara la lumière d'avec les ténèbres.
<verse eID="Gen.1.4"/>
<verse sID="Gen.1.5" osisID="Gen.1.5" n="5"/>Et Dieu appela la lumière Jour ;
et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin :
&#8212; premier jour.<verse eID="Gen.1.5"/>
</p>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Au commencement Dieu créa les cieux et la terre.

<sup>'''(2)'''</sup> Et la terre était désolation et vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

<sup>'''(3)''' </sup>Et Dieu dit : Que la lumière soit. Et la lumière fut. <sup>'''(4)'''</sup> Et Dieu vit la lumière, qu'elle était bonne ; et Dieu sépara la lumière d'avec les ténèbres. <sup>'''(5)'''</sup> Et Dieu appela la lumière Jour ; et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin : — premier jour.
|}

Note: [[osis2mod]] converts a paragraph start into <tt><div type="paragraph" sID="genX"/></tt> and a paragraph end into <tt><div type="paragraph" eID="genX"/></tt>.

===Marking Quotations===
Most of the SWORD front-end applications will show a chapter at a time and some will show isolated verses. This means that all of the SWORD applications show partial quotations, such as the Sermon on the Mount which begins in Matt 5 and ends in Matt 7.

====Default quotation marks====
By default, SWORD will use <tt>"</tt> for quotations. The following describes various ways to influence this.

====Indicating the nesting of a quote====
When a quote is contained in a quote, it is customary to set the level attribute to indicate the depth of the nesting. For example, Jeremiah 23:38 is part of a larger quote and has a back and forth dialog of nested quotes:

<pre>
But if you say,
<q level="2" sID="1"/>
The burden of the Lord,
<q level="2" eID="1"/>
thus says the Lord,
<q level="2" sID="3"/>
Because you have said these words,
<q level="3" sID="4"/>
The burden of the Lord,
<q level="3" eID="4"/>
when I sent to you, saying,
<q level="3" sID="5"/>
You shall not say,
<q level="4" sID="6"/>
The burden of the Lord,
<q level="4" eID="6"/>
<q level="3" eID="5"/>
</pre>
A couple of things to note about this verse. First, the level attribute is on both the sID and the eID pair, matching in value. Second, this is an example of a verse that has a quote that starts in the middle and finishes in another verse.

In this case, SWORD will use the level to determine whether to use <tt>"</tt> or <tt>'</tt> for quotes. Odd levels will use <tt>"</tt> and even levels will use <tt>'</tt>. This is in accordance with American English usage, which is the opposite of British English usage. Nesting levels up to five can be found in the Bible.<ref>Jeremiah 27:1-11; 29:1-28, 30-32; 34:1-5; and Ezekiel 1-36</ref>

<references />

====Supplying alternative quotation marks====
The quote element has a marker attribute that can be used to control the quotation marks. SWORD applications will always use this value when rendering the quote. When the marker attribute is present but empty, it will render no quotation mark at all.

To specify "curly" quotes you can use the following values:
{|border=1
!Description!!Char!!HTML Entity!!Unicode
|-align=center
|Opening double quote||“||&#8220;||U+201C
|-align=center
|Closing double quote||”||&#8221;||U+201D
|-align=center
|Opening single quote||‘||&#8216;||U+2018
|-align=center
|Closing single quote||’||&#8217;||U+2019
|}

To use different marks to start and end a quote, use the milestoned version of the quote.
<q marker="“" sID="qN"/> ... <q marker="”" eID="qN"/>

There is further information about English quotation marks and their usage in [http://en.wikipedia.org/wiki/Quotation_marks].

Quotation marks have a variety of forms in different languages and in different media. See [http://en.wikipedia.org/wiki/Quotation_mark,_non-English_usage Quotation mark, non-English usage].<ref>When modules are being converted from digitized source text used in other Bible software, it may be the case that quotation marks in the text source differ from those in the original published edition, whether due to inherent constraints of the other software, or for other causes.</ref>

'''Note:'''
<references />

====Continuation quotation marks====
The <tt><milestone type="cQuote"/></tt> can be used to indicate the presence of a continued quote. If the marker attribute is present, it will use that otherwise it will use a straight double quote, ". Since there is no level attribute on the milestone element, it is best to specify the marker attribute.

====Marking the Words of Christ====
To indicate that a quote is something that Jesus said<ref>http://en.wikipedia.org/wiki/Red_letter_edition</ref>, use the attribute <tt>who="Jesus"</tt>.

<pre>
<verse osisID="Luke.22.35 sID="Luke.22.35"/>
Then Jesus asked them, <q who="Jesus" marker="">When I sent you without purse,
bag or sandals, did you lack anything?</q>
<verse eID="Luke.22.35"/>
</pre>

{| border="1"
|+Result
|
Then Jesus asked them, <span style="color:#A00">When I sent you without purse, bag or sandals, did you lack anything?</span>
|}

'''Note:'''
<references />

===Marking poetic material===
Poetry is marked up with <lg>, line group, and <l>, line, elements. The line element supports indentation with the level attribute. When the level attribute is not present or it is level="1", this should be interpreted as the first level of the line group. When level="2" it is indented relative to level="1". The same is true for each subsequent level.

The level attribute is used to indicate indentation. A value of 1 means no indentation, the same as not specifying a level attribute. A value of 2 means to indent one. And so forth.

<pre>
<chapter osisID="Exod.15" chapterTitle="Chapitre 15"><title type="chapter">Chapter 15</title>
<p>
<verse sID="Exod.15.1" osisID="Exod.15.1" n="1"/>
Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,
</p>
<lg>
<l level="1">I will sing unto the LORD, for he hath triumphed gloriously:</l>
<l level="2">the horse and his rider hath he thrown into the sea.</l>
<verse eID="Exod.15.1"/>

<verse sID="Exod.15.2" osisID="Exod.15.2" n="2"/>
<l level="1">The LORD is my strength and song, and he is become my salvation:</l>
<l level="2">he is my God, and I will prepare him an habitation;</l>
<l level="2">my father's God, and I will exalt him.</l>
<verse eID="Exod.15.2"/>

<verse sID="Exod.15.3" osisID="Exod.15.3" n="3"/>
<l level="1">The LORD is a man of war:</l>
<l level="2">the LORD is his name.</l>
<verse eID="Exod.15.3"/>

<verse sID="Exod.15.4" osisID="Exod.15.4" n="4"/>
<l level="1">Pharaoh's chariots and his host hath he cast into the sea:</l>
<l level="2">his chosen captains also are drowned in the Red sea.</l>
<verse eID="Exod.15.4"/>

<verse sID="Exod.15.5" osisID="Exod.15.5" n="5"/>
<l level="1">The depths have covered them:</l>
<l level="2">they sank into the bottom as a stone.</l>
<verse eID="Exod.15.5"/>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,

::''I will sing unto the LORD, for he hath triumphed gloriously:''
::::''the horse and his rider hath he thrown into the sea.''

::<sup>'''(2)'''</sup> ''The LORD is my strength and song, and he is become my salvation:''
::::''he is my God, and I will prepare him an habitation;''
::::''my father's God, and I will exalt him.''

::<sup>'''(3)'''</sup> ''The LORD is a man of war:''
::::''the LORD is his name.''

::<sup>'''(4)'''</sup> ''Pharaoh's chariots and his host hath he cast into the sea:''
::::''his chosen captains also are drowned in the Red sea.''

::<sup>'''(5)'''</sup> ''The depths have covered them:''
::::''they sank into the bottom as a stone.''
|}

'''Note:'''

# While OSIS defines a milestoned version of the <tt><lg></tt> element, its use (rather than the container version) will not produce a valid XML document. The <tt><l></tt> element can only occur within an <tt><lg></tt> container, so use of <tt><lg/></tt> milestones prevents use of <tt><l></tt> elements.

==== Marking acrostic poetry headings ====
Use <tt>type="acrostic"</tt> as the title attribute for the stanza headings in acrostic passages such as Psalm 119.

=== Marking lemmas & morphology ===
====Marking Strong's numbers====
To mark up [http://en.wikipedia.org/wiki/Strong%27s_Concordance Strong's numbers], you first need to declare a workID in the header of the OSIS document:
<pre>
<header>
...
<work osisWork="strong">
<refSystem>Dict.Strongs</refSystem>
</work>
...
</header>
</pre>

SWORD does not actually use this declaration, but it is required to have a proper OSIS document.

And while OSIS allows arbitrary workIDs, SWORD can only handle "strong" and a few variants.

<pre>
<w lemma="strong:H0853 strong:H03045">knew</w>
</pre>

The <w> element is used to surround the text that is represented by the Strong's number. It may be that the text is a phrase and it may be that more than one Strong's number defines the text.

When more than one Strong's number defines the text, each must be prefixed with a workID and must be separated from each other by a space. (While OSIS allows for the defining of default workIDs, SWORD requires that the workIDs be used.)

The actual Strong's Number should indicate whether it is Hebrew (H) or Greek (G) followed by the number. The number can be 0 padded to 5 digits as in H00001.

====Marking morphology====
In a similar manner to marking with Strong's numbers, morphology can also be noted. Since morphology regards the original language, Strong's numbers will be shown at the same time.

As with Strong's numbers, a workID needs to be defined. Here we are defining one for Robinson's Morphology Codes. And while SWORD will ignore this declaration, "robinson" is hard-coded into SWORD for Greek morphology codes.
<pre>
<header>
...
<work osisWork="robinson">
<refSystem>Dict.Robinson</refSystem>
</work>
...
</header>
</pre>

Example markup of Robinson's Morphology Codes in the KJV module:
<pre>
<w lemma="strong:G3588 strong:G80" morph="robinson:T-APM robinson:N-APM" src="7 8">his brethren</w>
</pre>
In this example, lemma, morph and src form parallel arrays. The first strong: mapping to the first robinson: and the first src value.

The workID should be name of a current, future, or potential lexicon module in which the morphology code could be looked up. For example, morph="packard:D" represents a reference to morphology code "D" in a module named Packard, whether or not a Packard module has been created or released. (Currently, SWORD offers lexicon modules named Robinson and Packard, both for Greek morphology.)

The src attribute is used here to indicate the word position in the original Greek.

====Marking other lemmas====
The lemma attribute of the <tt><w></tt> element can contain any number of other lemmas. Like Strong's numbers and morphology codes, these need to have a workID declared in the header. SWORD presumes that these lemma workIDs all start with "lemma." (note the final period). The portion of the workID following "lemma." should be name of a current, future, or potential lexicon module in which the lemma could be looked up. For example, lemma="lemma.TWOT:271" represents a reference to lemma #271 in a module named TWOT (i.e the Theological Workbook of the Old Testament), whether or not a TWOT module has been created or released. As far as SWORD is concerned, there can be any number of these space-delimited values in a lemma attribute and they can be in any order, even interspersed among the "strong:" lemmas.

SWORD has the ability to show or hide non-Strong's lemmas as a group. See [[DevTools:conf Files#Elements_required_for_proper_rendering|GlobalOptionFilter=OSISLemma]] – for OSIS texts having lemmas.

==== Marking glosses ====
Gloss markup uses the <tt>gloss</tt> attribute of the <tt><w></tt> element. The syntax is illustrated by this line exported from the module JapMeiji.
<pre>
<w gloss="はじめ">元始</w>に<w gloss="かみ">神</w><w gloss="てんち">天地</w>を<w gloss="つくり">創造</w>たまへり
</pre>
Display of glosses can be toggled (in [[Choosing a SWORD program#Module_Support|compatible front-ends]]) by having this line in the [[DevTools:conf Files#Elements_required_for_proper_rendering|.conf file]]:
GlobalOptionFilter=OSISGlosses

===Marking the Divine Name===
The <tt><divineName></tt> tag is reserved for representations of the tetragrammaton יהוה (YHWH). These occur in the Old Testament as <span style="font-variant: small-caps;">Lord</span>, <span style="font-variant: small-caps;">God</span> and <span style="font-variant: small-caps;">Yah</span>. Not every instance of Lord or God is a translation of this.

The content of the divineName element is the word Lord, God or Yah, not in all upper case (i.e. not LORD, GOD, or YAH). SWORD will either convert it to small-caps or uppercase.

Note, if it is the use is possessive it is permissible to have the following:
<pre>
<divineName>Lord's</divineName>
</pre>

However, it is inadvisable to include any other punctuation within the tag pair. Thus the following is '''not''' good practice (the quotation mark should ''precede'' the start tag):
<pre>
<divineName>“God </divineName> .....”
</pre>

When also marking with Strong's numbers you will need to do it one of two ways:
<pre>
<divineName><w lemma="strong:H3068">Lord's</w><divineName>
or
<w lemma="strong:H3068">of the <seg><divineName>Lord</divineName></seg></w>
</pre>

The latter form uses a workaround to allow the embedding of <tt><divineName></tt> in a <tt><w></tt>, since OSIS does not allow for this, but does allow for <tt><seg></tt> to be in a <tt><w></tt> and to contain <tt><divineName></tt>.

===Marking sections and titles===
A section is marked with:
<pre>
<div type="section">
...
</div>
</pre>

In OSIS the <tt><title></tt> element is used to provide general headings. Titles should be placed at the top of the container that they title, not before.
<pre>
<div type="book">
<title>A book title</title>
<chapter>
<title type="chapter">A title chapter</title>
<div type="section">
<title>A section title</title>
...
</div>
...
</chapter>
</div>
</pre>
Using <tt>type="chapter"</tt> or <tt>type="main"</tt> is needed by [[osis2mod]] to distinguish chapter titles from verse titles. When SWORD stores an OSIS document it does so as an index of verses. It has special indexes for book and chapter titles. SWORD does not store the <tt><verse></tt> tags. So when it comes to storing a title in the following verse, [[osis2mod]] generates special markup to indicate that the title stands before the verse. SWORD uses this to place the verse number.

Note: The <tt><head></tt> element is used to provide headings for tables, lists and cast groups. There are errors in the OSIS 2.1.1 manual that use the <tt><head></tt> incorrectly.

==== Marking pre-verse titles ====
There is no special markup for pre-verse titles. Osis2mod will determine what titles belong to the book, the chapter and otherwise if the above advice is followed.

'''Notes:'''
# See also [[OSIS pre-verse titles]].

===Marking notes===
NB. The examples in this section include the use of sub-identifiers in osisIDs for notes.
<pre>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement
Dieu<note osisRef="Gen.1.1" osisID="Gen.1.1!1" n="1"><hi type="italic">en hébreu</hi> : Élohim,
(<hi type="italic">pluriel d</hi>'Éloah, le Dieu suprême), la Déité, <hi type="italic">dans
le sens absolu</hi>.</note> créa les cieux et la terre.<verse eID="Gen.1.1"/>

<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide<note osisRef="Gen.1.2" osisID="Gen.1.2!1" n="2">le vide.</note>, et il y avait des ténèbres
sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.<verse eID="Gen.1.2"/>
</pre>

{| border="1"
|+Result
|
# Au commencement Dieu¹ créa les cieux et la terre.
# Et la terre était désolation et vide², et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

*¹ ''en hébreu'': Élohim, (''pluriel d'''Éloah, le Dieu suprême), la Déité, ''dans le sens absolu''.
*² le vide.
|}

The note should be attached to what it refers to, either after (as is the case here) or before. There should no additional space surrounding the note, but only what is in the text.

These notes can have any type other than crossReference.

====Marking cross-references notes====
SWORD provides the ability for a user to show or hide cross-references. To achieve this you embed one or more <tt><reference></tt> elements in a <tt><note type="crossReference">...</note></tt>. If this is not done, then the cross-references will always show inline in the text.

<pre>
<note type="crossReference" n="t" osisID="Jer.24.7!crossReference.t">
<reference osisRef="Jer.32.39">ch. 32:39</reference>;
<reference osisRef="Deut.30.6">Deut. 30:6</reference>;
<reference osisRef="Ezek.11.19">Ezek. 11:19</reference>;
<reference osisRef="Ezek.36.26-Ezek.36.27">36:26, 27</reference>
</note>
</pre>

Here is a breakdown.

Regarding the <tt><note></tt> element:

:<tt>type="crossReference"</tt> is one of the predefined OSIS note types. SWORD looks for this value to show/hide cross-references.

:<tt>n="t"</tt> provides the author's desired footnote marker for the note. A couple of SWORD applications use this, but most manufacture their own marker.

:The given osisID is based upon the location of the note. In order to not conflict with the verse's osisID and to construct a unique id, the ! (extension mark, ''aka'' sub-identifier) syntax is used. This is further qualified by the note's type and n value, separated by a dot.

:This note pertains to a single verse and it is given in osisRef.

Regarding the <tt><reference></tt> elements:

:The <tt><reference></tt> element is replaced by SWORD with a link to the reference with the text of the element being shown as link text.

:While the osisRef can point to multiple verses, most SWORD applications cannot handle a link that goes to more than one verse or a contiguous range of verses.

:Here we see that each reference is separated by punctuation (the semicolon at the end of each line).

===Marking variants===
SWORD recognizes the element <seg> with type="x-variant" as marking variants present in different versions of a text. The attribute subType should be added, with a value of "x-1" or "x-2" to indicate whether the reading is the primary or secondary variant. At present, SWORD supports only 2 different readings per text. The method is illustrated below:
The text of the Bible <seg type="x-variant" subType="x-1">may </seg><seg type="x-variant" subType="x-2">can </seg>contain variant readings.
This illustrates a primary reading "may " and a secondary reading "can ".

==== Filter ====
Variant readings in OSIS modules may be switched by the SWORD engine when the module conf file includes:
GlobalOptionFilter=OSISVariants
'''Note:'''
# This feature requires the front-end to have been compiled with SWORD version 1.7 or later.

==== Examples ====
* The TR module contains 246 locations where two such variants are marked.
* The WHNU module contains 1473 locations where two variants are marked.

==== SWORD implementation ====
The SWORD API provides for these three choices:
[Primary Reading|Secondary Reading|All Readings]
Example: In Xiphos version 3.1.6, the module context menu provides these three options for any module that has variants.

==Tools==
===Charset conversion===

====Padma====

[http://padma.mozdev.org/ Padma] is a system for transforming Indic text between various public and proprietary formats. This extension applies the technology to Mozilla based applications. Padma is available as an extension for Firefox, Thunderbird, Netscape, Mozilla suite and SeaMonkey platforms. Padma can automatically transform web pages that use dynamic font schemes to Unicode.

Padma can be customised to include a user supplied conversion. This implies that its use is not restricted to Indic texts. See [http://tidbits.co.in/content/how-write-padma-conversion-file].

===Valid OSIS test===
A valid XML document one that is well-formed and conforms to the formal definition provided in a schema (or DTD). A document cannot have elements, attributes, or entities not defined in the schema. A schema can also define ''how'' entities may be nested, the possible values of attributes, etc.

Many programs capable of schema validation exist. Most [http://en.wikipedia.org/wiki/XML_editor XML editors] ([http://xml-copy-editor.sourceforge.net/ XML Copy Editor], [http://en.wikipedia.org/wiki/Oxygen_XML_Editor Oxygen], [http://en.wikipedia.org/wiki/XMLSpy XMLSpy], [http://www.topologi.com/ Topologi], etc.) support some sort of XML schema validation. The Windows based text editor [http://en.wikipedia.org/wiki/Notepad%2B%2B Notepad++] supports Unicode and has an '''XML Tools''' plugin which can perform syntax checking and validation.<ref>Currently, the XML validation check in Notepad++ does not accept the extended schema location syntax that includes <tt><nowiki>xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"</nowiki></tt>, etc</ref>
<references />

====xmllint====
libxml2, available for Linux, Windows, & MacOS, includes a command-line validator called xmllint. To check that a document is valid against OSIS schema, use the following command. (You need Internet access to validate your document.)

<pre>
$ xmllint --noout --schema http://www.bibletechnologies.net/osisCore.2.1.1.xsd myfile.osis.xml
</pre>

To install xmllint, simply install libxml2 via your distribution's standard package management system in Linux or download the Windows binary from our [http://www.crosswire.org/ftpmirror/pub/sword/utils/win32/ mirror].

====Online XML Validators====
The external links section of http://en.wikipedia.org/wiki/XML_Validation lists at least three online validators. Some or all of these can validate against external XML schema.

==Creating a SWORD Module==
Use [[osis2mod]] to create the module.

== See also ==
*[[OSIS]] – a partial list of other OSIS related pages and external links.

*[[Converting SFM Bibles to OSIS]] – describes how to prepare a single OSIS XML file for a Biblical text supplied as several USFM files.

[[Category:Guides|OSIS Bibles]]

[[Category:OSIS|OSIS Bibles]]

[[Category:Morphology|OSIS Bibles]]

OSIS Bibles

2014-02-07T20:16:13Z

Osk: /* Marking ruby glosses */

==OSIS==
OSIS is an XML Schema definition for Bibles and other Biblical research texts, which enables ministries and other organizations to collaborate more easily. Traditionally, these organizations have stored their documents in disparate, proprietary markups, making it difficult when they wish to share in service with each other. OSIS provides a common markup for multiple visions.

CrossWire is committed to supporting the OSIS initiative. We have developed OSIS import and export tools which work with our SWORD engine, making OSIS documents available to all of our SWORD software.

The latest OSIS Schema definition and supporting information is available at: http://www.bibletechnologies.net <ref>Link broken. See [[OSIS Tutorial#The_Root_Element|OSIS Tutorial]]</ref>

<references/>

==Introduction==
This page is for practical examples of how to encode a Bible in OSIS 2.1.1 for building a SWORD module with [[osis2mod]]. It represents CrossWire's experience and best practices in creating modules.

Every OSIS SWORD module must be created from a well-formed and valid OSIS 2.1.1 document. While it is a desirable goal for any such document to be acceptable, SWORD has some particular requirements which are discussed here.

The schema for OSIS 2.1.1 that was formally at http://www.bibletechnologies.net/osisCore.2.1.1.xsd is preserved at http://ebible.org/osisCore.2.1.1.xsd

The March 2006 version of the OSIS Manual may be found [http://www.bibletechnologies.net/utilities/fmtdocview.cfm?id=28871A67-D5F5-4381-B22EC4947601628B here] (PDF).

A good example of an OSIS document can be found at http://www.crosswire.org/~dmsmith/kjv2006.

See also [[OSIS Book Abbreviations|OSIS Book Name Abbreviations]].

==General structure==

An OSIS document is a ''well-formed XML'' document, valid according to the ''OSIS schema''.
You can find the full normative description on the OSIS Website [http://www.bibletechnologies.net/].<ref>Link broken. See [[OSIS Tutorial#The_Root_Element|root element]]</ref>

To produce a Bible, you can use this template:

<pre>
<?xml version="1.0" encoding="UTF-8"?>
<osis
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:osis="http://www.bibletechnologies.net/2003/OSIS/namespace"
xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
<osisText osisIDWork="{NAME}" osisRefWork="bible" xml:lang="{LANG}" canonical="true">
<header>
{HEADER}
</header>
<div type="bookGroup">
{BODY}
</div>
</osisText>
</osis>
</pre>

With the following values:
; {NAME}: Normalized name of the Bible version (Usually 3 letters for language, 3 for translation)
; {LANG}: IETF language code-- ISO 639-1 codes are preferred, and ISO 639-3 codes are preferred when ISO 639-1 codes do not exist for the given language. See [http://www.sil.org/iso639-3/codes.asp] for a list of codes.
; {HEADER}: Description of the included text; see below
; {BODY}: Text; see below

For text without any character outside ASCII, you can use US-ASCII encoding (usually for english text). For every other language, please use UTF-8 and NFC. See the [[OSIS Bibles#Tools|tools]] section if you need to convert.

<references />
===Header===
This is the minimum contents of the header required for validation. It is also what <tt>usfm2osis.pl</tt> produces. A valid OSIS header may include more than this.
<header>
<work osisWork={Name}/>
</header>

===Body===

Here is the general structure of the body content:

<pre>
<div type="bookGroup">
<title>Old Testament</title>
<div type="book" osisID="Gen" canonical="true">
<title type="main" short="Genesis">Genesis</title>
<chapter osisID="Gen.1" n="1">
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>In the
beginning...<verse eID="Gen.1.1"/>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>The earth was
formless and void...<verse eID="Gen.1.2"/>
...
</chapter>
</div>
</div>
</pre>
'''Notes:'''
# The bookGroup division for the New Testament should have a similar structure.
# Any <tt><div></tt> defaults canonical to false. You need to set it to true on elements representing the structure of the original text.
# A quirk of the SWORD compilation process is that the only kind of content which reliably displays outside of <verse> elements are titles.

===OSIS Milestones===
OSIS allows for two potentially overlapping structures: Document structure (BSP) and verse structure (BCV).

Document structure is dominated by book, sections and paragraphs (BSP), additionally with titles, quotes and poetic material. While verse structure is indicated by book, chapter and verse numbers (BCV). While a SWORD module requires verse structure, the best way to encode a module with deep markup is with document structure. [[Osis2mod]] is responsible for transforming document structure into verse structure.

Because these two systems can overlap and because XML does not allow for overlapping elements, OSIS defines a milestone mechanism for both document and verse structure elements.

For:
<X ... attribute list ...>
...
</X>
the milestoned form is:
<X sID="g1" ... attribute list .../>
...
<X eID="g1"/>

According to the OSIS manual, for any given element X that defines a milestoneable form, all the instances of X in the document must use one form or the other and may not use both. The value of each sID attribute must be unique within the document.

Verse milestone sID/eID attributes can even have values that denote a verse range. This is purely for convenience to human readers.

It is allowable to use milestone elements for verses alone, or for both verses and chapters. The body example above is for the former.

==== Limitations of XML validators ====

An [[OSIS Bibles#Valid_OSIS_test|XML validator]] cannot validate whether OSIS milestones are used properly. It cannot validate:
* that an element is consistently either milestoned or not.
* that for each element with an sID that there is a paired element with an eID.<ref>osis2mod does not crash if the eID milestones are all missing, but the resulting module may appear to be void of text.</ref>
* that each paired sID/eID have the same attribute value.
* that different sID/eID pairs of the same element type do not overlap.<ref>If they weren't milestones, one would say they should be properly nested.</ref>

'''Notes:'''
<references />

==== Notes about OSIS elements ====
* For an OSIS document to be valid it must use the non-milestoned <div> and <lg> elements.
* There is no milestoned version of the <p> element. From a practical perspective, this means that the milestoned verse element should be used when paragraphs are used.
* The milestoned chapter element must be used when the paragraph is spanning over a chapter.
* The SWORD engine cannot handle sub-identifiers separated by ! in an osisID, so osis2mod strips these off from the osisIDs for verses. They are only of use for the osisIDs for notes.

===Recommended approach===
* For chapters, use <chapter>...</chapter> container elements (except in the rare case that other container elements cross chapter boundaries)
* For verses, use milestone elements (unless container elements will suffice) – see [[OSIS Bibles/BSPExample]].
* For paragraphs, use the <p>...</p> container element
* For poetry, use container elements <lg>...</lg> to indicate stanzas (or other types of line groups) and <l>...</l> to indicate lines
* For quoted text, use the <q>...</q> container element
* For translation changes, use the <transChange>...</transChange> container element<ref>Except where the text is within a <w> which is not allowed by OSIS. For these cases use the alternative <seg subType="x-added" type="x-transChange">...</seg> </ref>

<references />

==Examples==
===Marking Paragraphs===
There is no milestoned version of the <tt><p></tt> element. Typically paragraphs surround whole verses. That is, they start and end between verses. If a paragraph begins or ends in a verse and extends beyond that verse, then the whole document must use the milestoned version of <tt><verse></tt>.
<pre>
<div type="book" osisID="Gen" canonical="true">
<title type="main">LE PREMIER LIVRE DE MOÏSE dit LA GENÈSE</title>
<chapter osisID="Gen.1" chapterTitle="Chapitre 1"><title type="chapter">Chapitre 1</title>
<p>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement Dieu créa les
cieux et la terre.<verse eID="Gen.1.1"/>
</p>
<p>
<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu
planait sur la face des eaux.<verse eID="Gen.1.2"/>
</p>
<p>
<verse sID="Gen.1.3" osisID="Gen.1.3" n="3"/>Et Dieu dit : Que la lumière
soit. Et la lumière fut.<verse eID="Gen.1.3"/>
<verse sID="Gen.1.4" osisID="Gen.1.4" n="4"/>Et Dieu vit la lumière, qu'elle
était bonne ; et Dieu sépara la lumière d'avec les ténèbres.
<verse eID="Gen.1.4"/>
<verse sID="Gen.1.5" osisID="Gen.1.5" n="5"/>Et Dieu appela la lumière Jour ;
et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin :
&#8212; premier jour.<verse eID="Gen.1.5"/>
</p>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Au commencement Dieu créa les cieux et la terre.

<sup>'''(2)'''</sup> Et la terre était désolation et vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

<sup>'''(3)''' </sup>Et Dieu dit : Que la lumière soit. Et la lumière fut. <sup>'''(4)'''</sup> Et Dieu vit la lumière, qu'elle était bonne ; et Dieu sépara la lumière d'avec les ténèbres. <sup>'''(5)'''</sup> Et Dieu appela la lumière Jour ; et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin : — premier jour.
|}

Note: [[osis2mod]] converts a paragraph start into <tt><div type="paragraph" sID="genX"/></tt> and a paragraph end into <tt><div type="paragraph" eID="genX"/></tt>.

===Marking Quotations===
Most of the SWORD front-end applications will show a chapter at a time and some will show isolated verses. This means that all of the SWORD applications show partial quotations, such as the Sermon on the Mount which begins in Matt 5 and ends in Matt 7.

====Default quotation marks====
By default, SWORD will use <tt>"</tt> for quotations. The following describes various ways to influence this.

====Indicating the nesting of a quote====
When a quote is contained in a quote, it is customary to set the level attribute to indicate the depth of the nesting. For example, Jeremiah 23:38 is part of a larger quote and has a back and forth dialog of nested quotes:

<pre>
But if you say,
<q level="2" sID="1"/>
The burden of the Lord,
<q level="2" eID="1"/>
thus says the Lord,
<q level="2" sID="3"/>
Because you have said these words,
<q level="3" sID="4"/>
The burden of the Lord,
<q level="3" eID="4"/>
when I sent to you, saying,
<q level="3" sID="5"/>
You shall not say,
<q level="4" sID="6"/>
The burden of the Lord,
<q level="4" eID="6"/>
<q level="3" eID="5"/>
</pre>
A couple of things to note about this verse. First, the level attribute is on both the sID and the eID pair, matching in value. Second, this is an example of a verse that has a quote that starts in the middle and finishes in another verse.

In this case, SWORD will use the level to determine whether to use <tt>"</tt> or <tt>'</tt> for quotes. Odd levels will use <tt>"</tt> and even levels will use <tt>'</tt>. This is in accordance with American English usage, which is the opposite of British English usage. Nesting levels up to five can be found in the Bible.<ref>Jeremiah 27:1-11; 29:1-28, 30-32; 34:1-5; and Ezekiel 1-36</ref>

<references />

====Supplying alternative quotation marks====
The quote element has a marker attribute that can be used to control the quotation marks. SWORD applications will always use this value when rendering the quote. When the marker attribute is present but empty, it will render no quotation mark at all.

To specify "curly" quotes you can use the following values:
{|border=1
!Description!!Char!!HTML Entity!!Unicode
|-align=center
|Opening double quote||“||&#8220;||U+201C
|-align=center
|Closing double quote||”||&#8221;||U+201D
|-align=center
|Opening single quote||‘||&#8216;||U+2018
|-align=center
|Closing single quote||’||&#8217;||U+2019
|}

To use different marks to start and end a quote, use the milestoned version of the quote.
<q marker="“" sID="qN"/> ... <q marker="”" eID="qN"/>

There is further information about English quotation marks and their usage in [http://en.wikipedia.org/wiki/Quotation_marks].

Quotation marks have a variety of forms in different languages and in different media. See [http://en.wikipedia.org/wiki/Quotation_mark,_non-English_usage Quotation mark, non-English usage].<ref>When modules are being converted from digitized source text used in other Bible software, it may be the case that quotation marks in the text source differ from those in the original published edition, whether due to inherent constraints of the other software, or for other causes.</ref>

'''Note:'''
<references />

====Continuation quotation marks====
The <tt><milestone type="cQuote"/></tt> can be used to indicate the presence of a continued quote. If the marker attribute is present, it will use that otherwise it will use a straight double quote, ". Since there is no level attribute on the milestone element, it is best to specify the marker attribute.

====Marking the Words of Christ====
To indicate that a quote is something that Jesus said<ref>http://en.wikipedia.org/wiki/Red_letter_edition</ref>, use the attribute <tt>who="Jesus"</tt>.

<pre>
<verse osisID="Luke.22.35 sID="Luke.22.35"/>
Then Jesus asked them, <q who="Jesus" marker="">When I sent you without purse,
bag or sandals, did you lack anything?</q>
<verse eID="Luke.22.35"/>
</pre>

{| border="1"
|+Result
|
Then Jesus asked them, <span style="color:#A00">When I sent you without purse, bag or sandals, did you lack anything?</span>
|}

'''Note:'''
<references />

===Marking poetic material===
Poetry is marked up with <lg>, line group, and <l>, line, elements. The line element supports indentation with the level attribute. When the level attribute is not present or it is level="1", this should be interpreted as the first level of the line group. When level="2" it is indented relative to level="1". The same is true for each subsequent level.

The level attribute is used to indicate indentation. A value of 1 means no indentation, the same as not specifying a level attribute. A value of 2 means to indent one. And so forth.

<pre>
<chapter osisID="Exod.15" chapterTitle="Chapitre 15"><title type="chapter">Chapter 15</title>
<p>
<verse sID="Exod.15.1" osisID="Exod.15.1" n="1"/>
Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,
</p>
<lg>
<l level="1">I will sing unto the LORD, for he hath triumphed gloriously:</l>
<l level="2">the horse and his rider hath he thrown into the sea.</l>
<verse eID="Exod.15.1"/>

<verse sID="Exod.15.2" osisID="Exod.15.2" n="2"/>
<l level="1">The LORD is my strength and song, and he is become my salvation:</l>
<l level="2">he is my God, and I will prepare him an habitation;</l>
<l level="2">my father's God, and I will exalt him.</l>
<verse eID="Exod.15.2"/>

<verse sID="Exod.15.3" osisID="Exod.15.3" n="3"/>
<l level="1">The LORD is a man of war:</l>
<l level="2">the LORD is his name.</l>
<verse eID="Exod.15.3"/>

<verse sID="Exod.15.4" osisID="Exod.15.4" n="4"/>
<l level="1">Pharaoh's chariots and his host hath he cast into the sea:</l>
<l level="2">his chosen captains also are drowned in the Red sea.</l>
<verse eID="Exod.15.4"/>

<verse sID="Exod.15.5" osisID="Exod.15.5" n="5"/>
<l level="1">The depths have covered them:</l>
<l level="2">they sank into the bottom as a stone.</l>
<verse eID="Exod.15.5"/>
...
</pre>

{| border="1"
|+Result
|
<sup>'''(1)'''</sup> Then sang Moses and the children of Israel this song unto the LORD, and spake, saying,

::''I will sing unto the LORD, for he hath triumphed gloriously:''
::::''the horse and his rider hath he thrown into the sea.''

::<sup>'''(2)'''</sup> ''The LORD is my strength and song, and he is become my salvation:''
::::''he is my God, and I will prepare him an habitation;''
::::''my father's God, and I will exalt him.''

::<sup>'''(3)'''</sup> ''The LORD is a man of war:''
::::''the LORD is his name.''

::<sup>'''(4)'''</sup> ''Pharaoh's chariots and his host hath he cast into the sea:''
::::''his chosen captains also are drowned in the Red sea.''

::<sup>'''(5)'''</sup> ''The depths have covered them:''
::::''they sank into the bottom as a stone.''
|}

'''Note:'''

# While OSIS defines a milestoned version of the <tt><lg></tt> element, its use (rather than the container version) will not produce a valid XML document. The <tt><l></tt> element can only occur within an <tt><lg></tt> container, so use of <tt><lg/></tt> milestones prevents use of <tt><l></tt> elements.

==== Marking acrostic poetry headings ====
Use <tt>type="acrostic"</tt> as the title attribute for the stanza headings in acrostic passages such as Psalm 119.

=== Marking lemmas & morphology ===
====Marking Strong's numbers====
To mark up [http://en.wikipedia.org/wiki/Strong%27s_Concordance Strong's numbers], you first need to declare a workID in the header of the OSIS document:
<pre>
<header>
...
<work osisWork="strong">
<refSystem>Dict.Strongs</refSystem>
</work>
...
</header>
</pre>

SWORD does not actually use this declaration, but it is required to have a proper OSIS document.

And while OSIS allows arbitrary workIDs, SWORD can only handle "strong" and a few variants.

<pre>
<w lemma="strong:H0853 strong:H03045">knew</w>
</pre>

The <w> element is used to surround the text that is represented by the Strong's number. It may be that the text is a phrase and it may be that more than one Strong's number defines the text.

When more than one Strong's number defines the text, each must be prefixed with a workID and must be separated from each other by a space. (While OSIS allows for the defining of default workIDs, SWORD requires that the workIDs be used.)

The actual Strong's Number should indicate whether it is Hebrew (H) or Greek (G) followed by the number. The number can be 0 padded to 5 digits as in H00001.

====Marking morphology====
In a similar manner to marking with Strong's numbers, morphology can also be noted. Since morphology regards the original language, Strong's numbers will be shown at the same time.

As with Strong's numbers, a workID needs to be defined. Here we are defining one for Robinson's Morphology Codes. And while SWORD will ignore this declaration, "robinson" is hard-coded into SWORD for Greek morphology codes.
<pre>
<header>
...
<work osisWork="robinson">
<refSystem>Dict.Robinson</refSystem>
</work>
...
</header>
</pre>

Example markup of Robinson's Morphology Codes in the KJV module:
<pre>
<w lemma="strong:G3588 strong:G80" morph="robinson:T-APM robinson:N-APM" src="7 8">his brethren</w>
</pre>
In this example, lemma, morph and src form parallel arrays. The first strong: mapping to the first robinson: and the first src value.

The workID should be name of a current, future, or potential lexicon module in which the morphology code could be looked up. For example, morph="packard:D" represents a reference to morphology code "D" in a module named Packard, whether or not a Packard module has been created or released. (Currently, SWORD offers lexicon modules named Robinson and Packard, both for Greek morphology.)

The src attribute is used here to indicate the word position in the original Greek.

====Marking other lemmas====
The lemma attribute of the <tt><w></tt> element can contain any number of other lemmas. Like Strong's numbers and morphology codes, these need to have a workID declared in the header. SWORD presumes that these lemma workIDs all start with "lemma." (note the final period). The portion of the workID following "lemma." should be name of a current, future, or potential lexicon module in which the lemma could be looked up. For example, lemma="lemma.TWOT:271" represents a reference to lemma #271 in a module named TWOT (i.e the Theological Workbook of the Old Testament), whether or not a TWOT module has been created or released. As far as SWORD is concerned, there can be any number of these space-delimited values in a lemma attribute and they can be in any order, even interspersed among the "strong:" lemmas.

SWORD has the ability to show or hide non-Strong's lemmas as a group. See [[DevTools:conf Files#Elements_required_for_proper_rendering|GlobalOptionFilter=OSISLemma]] – for OSIS texts having lemmas.

==== Marking glosses ====
Gloss markup uses the <tt>gloss</tt> attribute of the <tt><w></tt> element. The syntax is illustrated by this line exported from the module JapMeiji.
<pre>
<w gloss="はじめ">元始</w>に<w gloss="かみ">神</w><w gloss="てんち">天地</w>を<w gloss="つくり">創造</w>たまへり
</pre>
Display of glosses can be toggled (in [[Choosing a SWORD program#Module_Support|compatible front-ends]]) by having this line in the [[DevTools:conf Files#Elements_required_for_proper_rendering|.conf file]]:
GlobalOptionFilter=OSISGlosses

===Marking the Divine Name===
The <tt><divineName></tt> tag is reserved for translations of YHWH. These occur in the Old Testament as Lord, God and Yah. Not every instance of Lord or God is a translation of this.

The content of the divineName element is the word Lord, God or Yah, not in all upper case (i.e. not LORD, GOD, or YAH). SWORD will either convert it to small-caps or uppercase.

Note, if it is the use is possessive it is permissible to have the following:
<pre>
<divineName>Lord's</divineName>
</pre>

However, it is inadvisable to include any other punctuation within the tag pair. Thus the following is '''not''' good practice (the quotation mark should ''precede'' the start tag):
<pre>
<divineName>“God </divineName> .....”
</pre>

When also marking with Strong's numbers you will need to do it one of two ways:
<pre>
<divineName><w lemma="strong:H3068">Lord's</w><divineName>
or
<w lemma="strong:H3068">of the <seg><divineName>Lord</divineName></seg></w>
</pre>

The latter form uses a [http://en.wikipedia.org/wiki/Hack_%28computer_science%29#In_computer_science hack] to allow the embedding of <tt><divineName></tt> in a <tt><w></tt>, since OSIS does not allow for this, but does allow for <tt><seg></tt> to be in a <tt><w></tt> and to contain <tt><divineName></tt>.

===Marking sections and titles===
A section is marked with:
<pre>
<div type="section">
...
</div>
</pre>

In OSIS the <tt><title></tt> element is used to provide general headings. Titles should be placed at the top of the container that they title, not before.
<pre>
<div type="book">
<title>A book title</title>
<chapter>
<title type="chapter">A title chapter</title>
<div type="section">
<title>A section title</title>
...
</div>
...
</chapter>
</div>
</pre>
Using <tt>type="chapter"</tt> or <tt>type="main"</tt> is needed by [[osis2mod]] to distinguish chapter titles from verse titles. When SWORD stores an OSIS document it does so as an index of verses. It has special indexes for book and chapter titles. SWORD does not store the <tt><verse></tt> tags. So when it comes to storing a title in the following verse, [[osis2mod]] generates special markup to indicate that the title stands before the verse. SWORD uses this to place the verse number.

Note: The <tt><head></tt> element is used to provide headings for tables, lists and cast groups. There are errors in the OSIS 2.1.1 manual that use the <tt><head></tt> incorrectly.

==== Marking pre-verse titles ====
There is no special markup for pre-verse titles. Osis2mod will determine what titles belong to the book, the chapter and otherwise if the above advice is followed.

'''Notes:'''
# See also [[OSIS pre-verse titles]].

===Marking notes===
NB. The examples in this section include the use of sub-identifiers in osisIDs for notes.
<pre>
<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement
Dieu<note osisRef="Gen.1.1" osisID="Gen.1.1!1" n="1"><hi type="italic">en hébreu</hi> : Élohim,
(<hi type="italic">pluriel d</hi>'Éloah, le Dieu suprême), la Déité, <hi type="italic">dans
le sens absolu</hi>.</note> créa les cieux et la terre.<verse eID="Gen.1.1"/>

<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
vide<note osisRef="Gen.1.2" osisID="Gen.1.2!1" n="2">le vide.</note>, et il y avait des ténèbres
sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.<verse eID="Gen.1.2"/>
</pre>

{| border="1"
|+Result
|
# Au commencement Dieu¹ créa les cieux et la terre.
# Et la terre était désolation et vide², et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.

*¹ ''en hébreu'': Élohim, (''pluriel d'''Éloah, le Dieu suprême), la Déité, ''dans le sens absolu''.
*² le vide.
|}

The note should be attached to what it refers to, either after (as is the case here) or before. There should no additional space surrounding the note, but only what is in the text.

These notes can have any type other than crossReference.

====Marking cross-references notes====
SWORD provides the ability for a user to show or hide cross-references. To achieve this you embed one or more <tt><reference></tt> elements in a <tt><note type="crossReference">...</note></tt>. If this is not done, then the cross-references will always show inline in the text.

<pre>
<note type="crossReference" n="t" osisID="Jer.24.7!crossReference.t">
<reference osisRef="Jer.32.39">ch. 32:39</reference>;
<reference osisRef="Deut.30.6">Deut. 30:6</reference>;
<reference osisRef="Ezek.11.19">Ezek. 11:19</reference>;
<reference osisRef="Ezek.36.26-Ezek.36.27">36:26, 27</reference>
</note>
</pre>

Here is a breakdown.

Regarding the <tt><note></tt> element:

:<tt>type="crossReference"</tt> is one of the predefined OSIS note types. SWORD looks for this value to show/hide cross-references.

:<tt>n="t"</tt> provides the author's desired footnote marker for the note. A couple of SWORD applications use this, but most manufacture their own marker.

:The given osisID is based upon the location of the note. In order to not conflict with the verse's osisID and to construct a unique id, the ! (extension mark, ''aka'' sub-identifier) syntax is used. This is further qualified by the note's type and n value, separated by a dot.

:This note pertains to a single verse and it is given in osisRef.

Regarding the <tt><reference></tt> elements:

:The <tt><reference></tt> element is replaced by SWORD with a link to the reference with the text of the element being shown as link text.

:While the osisRef can point to multiple verses, most SWORD applications cannot handle a link that goes to more than one verse or a contiguous range of verses.

:Here we see that each reference is separated by punctuation (the semicolon at the end of each line).

===Marking variants===
SWORD recognizes the element <seg> with type="x-variant" as marking variants present in different versions of a text. The attribute subType should be added, with a value of "x-1" or "x-2" to indicate whether the reading is the primary or secondary variant. At present, SWORD supports only 2 different readings per text. The method is illustrated below:
The text of the Bible <seg type="x-variant" subType="x-1">may </seg><seg type="x-variant" subType="x-2">can </seg>contain variant readings.
This illustrates a primary reading "may " and a secondary reading "can ".

==== Filter ====
Variant readings in OSIS modules may be switched by the SWORD engine when the module conf file includes:
GlobalOptionFilter=OSISVariants
'''Note:'''
# This feature requires the front-end to have been compiled with SWORD version 1.7 or later.

==== Examples ====
* The TR module contains 246 locations where two such variants are marked.
* The WHNU module contains 1473 locations where two variants are marked.

==== SWORD implementation ====
The SWORD API provides for these three choices:
[Primary Reading|Secondary Reading|All Readings]
Example: In Xiphos version 3.1.6, the module context menu provides these three options for any module that has variants.

==Tools==
===Charset conversion===

====Padma====

[http://padma.mozdev.org/ Padma] is a system for transforming Indic text between various public and proprietary formats. This extension applies the technology to Mozilla based applications. Padma is available as an extension for Firefox, Thunderbird, Netscape, Mozilla suite and SeaMonkey platforms. Padma can automatically transform web pages that use dynamic font schemes to Unicode.

Padma can be customised to include a user supplied conversion. This implies that its use is not restricted to Indic texts. See [http://tidbits.co.in/content/how-write-padma-conversion-file].

===Valid OSIS test===
A valid XML document one that is well-formed and conforms to the formal definition provided in a schema (or DTD). A document cannot have elements, attributes, or entities not defined in the schema. A schema can also define ''how'' entities may be nested, the possible values of attributes, etc.

Many programs capable of schema validation exist. Most [http://en.wikipedia.org/wiki/XML_editor XML editors] ([http://xml-copy-editor.sourceforge.net/ XML Copy Editor], [http://en.wikipedia.org/wiki/Oxygen_XML_Editor Oxygen], [http://en.wikipedia.org/wiki/XMLSpy XMLSpy], [http://www.topologi.com/ Topologi], etc.) support some sort of XML schema validation. The Windows based text editor [http://en.wikipedia.org/wiki/Notepad%2B%2B Notepad++] supports Unicode and has an '''XML Tools''' plugin which can perform syntax checking and validation.<ref>Currently, the XML validation check in Notepad++ does not accept the extended schema location syntax that includes <tt><nowiki>xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"</nowiki></tt>, etc</ref>
<references />

====xmllint====
libxml2, available for Linux, Windows, & MacOS, includes a command-line validator called xmllint. To check that a document is valid against OSIS schema, use the following command. (You need Internet access to validate your document.)

<pre>
$ xmllint --noout --schema http://www.bibletechnologies.net/osisCore.2.1.1.xsd myfile.osis.xml
</pre>

To install xmllint, simply install libxml2 via your distribution's standard package management system in Linux or download the Windows binary from our [http://www.crosswire.org/ftpmirror/pub/sword/utils/win32/ mirror].

====Online XML Validators====
The external links section of http://en.wikipedia.org/wiki/XML_Validation lists at least three online validators. Some or all of these can validate against external XML schema.

==Creating a SWORD Module==
Use [[osis2mod]] to create the module.

== See also ==
*[[OSIS]] – a partial list of other OSIS related pages and external links.

*[[Converting SFM Bibles to OSIS]] – describes how to prepare a single OSIS XML file for a Biblical text supplied as several USFM files.

[[Category:Guides|OSIS Bibles]]

[[Category:OSIS|OSIS Bibles]]

[[Category:Morphology|OSIS Bibles]]