Revision as of 19:05, 4 October 2007

Introduction

This page is for practical examples of how to encode a Bible in OSIS 2.1.1 for building a Sword module with osis2mod. It represents CrossWire's experience and best practices in creating modules.

Every OSIS Sword module must be created from a well-formed and valid OSIS 2.1.1 document. While it is a desirable goal for any such document to be acceptable, Sword has some particular requirements which are discussed here.

The schema for OSIS 2.1.1 can be found at http://www.bibletechnologies.net/osisCore.2.1.1.xsd.

A good example of an OSIS document can be found at http://www.crosswire.org/~dmsmith/kjv2006.

General structure

An OSIS document is a well-formed XML document, valid according to the OSIS schema. You can find the full normative description on the OSIS Website [1].

To produce a Bible, you can use this template:

<?xml version="1.0" encoding="UTF-8"?>
<osis
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
	xmlns:osis="http://www.bibletechnologies.net/2003/OSIS/namespace"
	xsi:schemaLocation="http://www.bibletechnologies.net/2003/OSIS/namespace http://www.bibletechnologies.net/osisCore.2.1.1.xsd">
	<osisText osisIDWork="{NAME}" osisRefWork="bible" xml:lang="{LANG}" canonical="true">
		<header>
			{HEADER}
		</header>
		<div type="bookGroup">
			{BODY}
		</div>
	</osisText>
</osis>

With the following values:

{NAME}

Normalized name of the Bible version (Usually 3 letters for language, 3 for translation)

{LANG}

ISO-639 [2] language code

{HEADER}

Description of the included text; see below

{BODY}

Text; see below

For text without any character outside ASCII, you can use US-ASCII encoding (usually for english text). For every other language, please use UTF-8. See the tools section if you need to convert.

Header

Body

Here is the general structure of the body content:

<div type="bookGroup">
	<title>Old Testament</title>
	<div type="book" osisID="Gen" canonical="true">
		<title type="main" short="Genesis">Genesis</title>
		<chapter osisID="Gen.1" n="1">
			<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>In the
			beginning...<verse eID="Gen.1.1"/>
			<verse sID="Gen.1.1" osisID="Gen.1.2" n="2"/>The earth was
			formless and void...<verse eID="Gen.1.2"/>
			...
		</chapter>
	</div>
</div>

Note any <div> defaults canonical to false. You need to set it to true on elements representing the structure of the original text.

Examples

Marking Paragraphs

<div type="book" osisID="Gen" canonical="true">
	<title type="main">LE PREMIER LIVRE DE MOÏSE dit LA GENÈSE</title>
	<chapter osisID="Gen.1" chapterTitle="Chapitre 1"><title type="chapter">Chapitre 1</title>
		<p>
			<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement Dieu créa les 
			cieux et la terre.<verse eID="Gen.1.1"/>
		</p>
		<p>
			<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et
			vide, et il y avait des ténèbres sur la face de l'abîme. Et l'Esprit de Dieu
			planait sur la face des eaux.<verse eID="Gen.1.2"/>
		</p>
		<p>
			<verse sID="Gen.1.3" osisID="Gen.1.3" n="3"/>Et Dieu dit : Que la lumière
			soit. Et la lumière fut.<verse eID="Gen.1.3"/>
			<verse sID="Gen.1.4" osisID="Gen.1.4" n="4"/>Et Dieu vit la lumière, qu'elle
			était bonne ; et Dieu sépara la lumière d'avec les ténèbres.
			<verse eID="Gen.1.4"/>
			<verse sID="Gen.1.5" osisID="Gen.1.5" n="5"/>Et Dieu appela la lumière Jour ;
			et les ténèbres, il les appela Nuit. Et il y eut soir, et il y eut matin : 
			&#8212; premier jour.<verse eID="Gen.1.5"/>
		</p>
...

Note: Osis2mod converts a paragraph start into <lb type="x-begin-paragraph"/> and a paragraph end into <lb type="x-end-paragraph"/>. If these are between verses they are appended to the prior verse.

Marking Quotations

Most of the Sword applications will show a chapter at a time and some will show isolated verses. This means that all of the Sword applications show partial quotations, such as the Sermon on the Mount which begins in Matt 5 and ends in Matt 7. For this reason, the milestoned version of the <q> should be used.

Default Quotation Marks

By default, Sword will use " for quotations. The following discusses various ways to influence this.

Indicating the nesting of a quote

When a quote is contained in a quote, it is customary to set the level attribute to indicate the depth of the nesting. For example, Jeremiah 23:38 is part of a larger quote and has a back and forth dialog of nested quotes:

But if you say,
<q level="2" sID="1"/>
	The burden of the Lord,
<q level="2" eID="1"/>
thus says the Lord,
<q level="2" sID="3"/>
	Because you have said these words,
	<q level="3" sID="4"/>
		The burden of the Lord,
	<q level="3" ="4"/>
	when I sent to you, saying,
	<q level="3" sID="5"/>
		You shall not say,
		<q level="4" sID="6"/>
			The burden of the Lord,
		<q level="4" eID="6"/>
	<q level="3" eID="5"/>

A couple of things to note about this verse. First, the level attribute is on both the sID and the eID pair, matching in value. Second, this is an example of a verse that has a quote that starts in the middle and finishes in another verse.

In this case, Sword will use the level to determine whether to use " or ' for quotes. Odd levels will use " and even levels will use '.

Supplying alternative quotation marks

The quote element has a marker attribute that can be used to control the quotation marks. Sword applications will always use this value when rendering the quote. When the marker attribute is present but empty, it will render no quotation mark at all.

To specify "curly" quotes you can use the following values:

Continuation Quotation Marks

The <milestone type="cQuote"/> can be used to indicate the presence of a continued quote. If the marker attribute is present, it will use that otherwise it will use a straight double quote, ". Since there is no level attribute on the milestone element, it is best to specify the marker attribute.

Marking the Words of Christ

	<verse osisID="Luke.22.35 sID="Luke.22.35"/>
	Then Jesus asked them, <q who="Jesus" marker="">When I sent you without purse,
	bag or sandals, did you lack anything?</q>
	<verse eID="Luke.22.35"/>

Marking poetic material

	<chapter osisID="Exod.15" chapterTitle="Chapitre 15"><title type="chapter">Chapter 15</title>
	<p>
		<verse sID="Exod.15.1" osisID="Exod.15.1" n="1"/>Then sang Moses and the children of
		Israel this song unto the LORD, and spake, saying,
	</p>
	<lg>
		<l>I will sing unto the LORD, for he hath triumphed gloriously: the horse and his rider
		hath he thrown into the sea.</l><verse eID="Exod.15.1"/>

		<verse sID="Exod.15.2" osisID="Exod.15.2" n="2"/><l>The LORD is my strength and song,
		and he is become my salvation: he is my God, and I will prepare him an habitation; my
		father's God, and I will exalt him.</l><verse eID="Exod.15.2"/>

		<verse sID="Exod.15.3" osisID="Exod.15.3" n="3"/><l>The LORD is a man of war: the LORD
		is his name.</l><verse eID="Exod.15.3"/>

		<verse sID="Exod.15.4" osisID="Exod.15.4" n="4"/><l>Pharaoh's chariots and his host
		hath he cast into the sea: his chosen captains also are drowned in the Red sea.</l>
		<verse eID="Exod.15.4"/>

		<verse sID="Exod.15.5" osisID="Exod.15.5" n="5"/><l>The depths have covered them: they 
		sank into the bottom as a stone.</l><verse eID="Exod.15.5"/>
...

Marking with Strong's Numbers

Marking with Other Lemma

Marking Section Headings

In OSIS the <title> element is used to provide general headings. The <head> element is used to provide headings for tables, lists and cast groups. There are errors in the OSIS 2.1.1 manual that use the <head> incorrectly.

If the <title type="main">...</title> or <title type="chapter">...</title> it is treated by Sword specially. These are treated by Sword as book and chapter titles. Otherwise, the title is attached to the immediately following verse.

When Sword stores an OSIS document it does so as an index of verses. Sword does not store the <verse> tags. So when it comes to storing a title in the following verse, osis2mod generates special markup to indicate that the title stands before the verse. Sword uses this to place the verse number.

To further complicate the matter, newline producing elements such as <div> and <p> that are between verses are generally placed with the preceding verse, even if they follow the title in the OSIS document. This is to prevent newline between the verse number and the verse text.

Marking Cross-References

Sword provides the ability for a user to show or hide cross-references. To achieve this you embed one or more <reference> elements in a <note type="crossReference">...</note>. If this is not done, then the cross-references will always show inline in the text.

<note type="crossReference" n="t" osisID="Jer.24.7!crossReference.t" osisRef="Jer.24.7">
  <reference osisRef="Jer.32.39">ch. 32:39</reference>; 
  <reference osisRef="Deut.30.6">Deut. 30:6</reference>; 
  <reference osisRef="Ezek.11.19">Ezek. 11:19</reference>; 
  <reference osisRef="Ezek.36.26-Ezek.36.27">36:26, 27</reference>
</note>

Here is a breakdown.

Regarding the <note> element:

type="crossReference" is one of the predefined OSIS note types. Sword looks for this value to show/hide cross-references.

n="t" provides the author's desired footnote marker for the note. A couple of Sword applications use this, but most manufacture their own marker.

The osisID is given based upon the location of the note. In order to not conflict with the verse's osisID and to construct a unique id, the ! (extension mark) is used to further qualify. This is followed by the note's type and n value, separated by a dot.

This note pertains to a single verse and it is given in osisRef.

Regarding the <reference> elements:

The <reference> element is replaced by Sword with a link to the reference with the text of the element being shown as link text.

While the osisRef can point to multiple verses, most Sword applications cannot handle a link that goes to more than one verse or a contiguous range of verses. Here we see that each reference is separated by punctuation.

Marking Notes

	<verse sID="Gen.1.1" osisID="Gen.1.1" n="1"/>Au commencement 
	Dieu<note osisRef="Gen.1.1" osisID="Gen.1.1!1" n="1"><hi type="italic">en hébreu</hi> : Élohim,
	(<hi type="italic">pluriel d</hi>'Éloah, le Dieu suprême), la Déité, <hi type="italic">dans
	le sens absolu</hi>.</note> créa les cieux et la terre.<verse eID="Gen.1.1"/>

	<verse sID="Gen.1.2" osisID="Gen.1.2" n="2"/>Et la terre était désolation et 
	vide<note osisRef="Gen.1.2" osisID="Gen.1.2!1">le vide.</note>, et il y avait des ténèbres
	sur la face de l'abîme. Et l'Esprit de Dieu planait sur la face des eaux.<verse eID="Gen.1.2"/>

osis2mod usage

Osis2mod is used to create Bible and Commentary modules. At this point in time, the commentary needs to be (incorrectly) encoded as if it were a Bible, but with the verse contents being the commentary on the verse.

usage: osis2mod <output/path> <osisDoc> [createMod] [compressType [blockType [cipherKey]]]
  createMod   : (default 0): 0 - create  1 - augment
  compressType: (default 0): 0 - no compression  1 - LZSS    2 - Zip
  blockType   : (default 4): 2 - verses  3 - chapters  4 - books
  cipherkey   : ascii string for module encryption

<output/path>
This a path to any existing directory. It is best for it to be empty.

<osisDoc>
This is a single, well-formed, valid OSIS document.

createMod
Osis2mod can create a Bible all at once or incrementally, depending on the setting of createMod. This provides for two abilities,

1. Assembling a Bible from book files:
   

   mkdir /tmp/mymodule
   osis2mod /tmp/mymodule 0 matt.xml
   osis2mod /tmp/mymodule 1 mark.xml
   ...
   osis2mod /tmp/mymodule 1 rev.xml
   

   Note: The book files can be in any order. Sword will order them correctly in the index.

2. Adding corrections to a Bible:
   

   osis2mod /tmp/mymodule 1 fixes.xml
   

   Note: When fixes are put into the module they are appended to the data file and do not actually replace the verses. The index file is adjusted to point to the new place in the data file.

compressType
All of Sword's Bible modules are compressed with Zip. This saves significant space over an uncompressed module. Uncompressed modules are useful for debugging.

blockType This setting is only useful for a compressed module. The choice as to whether to use Verse, Chapter or Book level compression depends upon the amount of data in the block. A typical Bible is best compressed book by book. A commentary, chapter by chapter. If the commentary is very robust and the amount of text per verse is really huge, then verse compression might make sense.

All of Sword's compressed Bible modules are compressed by book. Basically, all of the verses in a block are compressed and appended to the data file. For this reason, the datafile cannot be uncompressed by anything other than the Sword and JSword libraries.

When creating the module by appending it is important to do so by whole compression block. That is, if blockType is Chapter, then the osisDoc needs to contain one or more whole chapters.

cipherKey This is typically 16 characters in length, having no leading or trailing spaces, consisting of alternating sets of 4 alpha and 4 numeric characters, such as Aduf0274PjNq0328.

Tools

Charset conversion

XML well-formed test

Validate OSIS test

GNU/Linux

To check if a document is well formed, use xmlwf usually included in expat package; if your document is wrong, it will output error, and position (filename:row:col):

$ xmlwf goodfile.osis.xml
$ xmlwf badfile.xml
badfile.xml:11:0: no element found
$ 

To check if a document is valid against OSIS schema; use xmllint usually included in libxml2 package; you need Internet access to validate your document.

$ xmllint --noout --schema http://www.bibletechnologies.net/osisCore.2.1.1.xsd myfile.osis.xml

Windows

To check if an OSIS document is valid against OSIS schema, use this easy tool if your document is wrong, it will output error, and position.

OSIS - Easy Modul Validator (osisCore.2.1.1)

Perl and automatic text transformation

XSLT and automatic XML transformation

make and Makefile