Freely-Given.org Bible Organisational System (BOS)

Picture of the FG Logo

Introduction

The Bible Organisational System (BOS) is a combination of XML data files and Python3 software modules that handle the basic organisational foundations of internationalised Bible data.

It was first started in 2010 and has been in continual development since then (although only as a hobby project thus far due to other commitments). Some of the initial (and still current) design philosophy can be seen at the OpenScriptures blog. (You should really read that first before proceeding here if want to fully understand the system.) The source code is hosted on the OpenScriptures repository at GitHub.

Philosophy

The Bible Organisational System (BOS) uses hand-crafted XML files to represent the internationalised Biblical data tables (e.g., Books codes, lists of verses per chapter, etc.). However, each data file has a corresponding converter module that does several things:

  1. Reads and tightly checks the XML data files,
  2. Imports the data into Python data structures such as lists and dictionaries for use in the BOS and other Python programs, and
  3. Exports the data into various formats, including pickle files (Python), and JSON (Java) for use in programs written in other programming languages.

The code is all intended to be run from a terminal or command line (no windowed user interface— that’s in Biblelator editor and in the BOS Manager and Sword Manager below).

A tension

The BOS was written by a Bible translator. A Bible translator finishing a book wants/needs to know about any errors in the files, even minor ones. However, Bible processing software needs to be resilient and adverse to crashing— capable of loading finished or unfinished works from a range of sources containing a range of errors. The BOS handles this by defaulting to the generous "accept anything reasonable" mode, but by also allowing the use of command line flags or switches to change the behaviour— specifically, in this case, the --strict flag which makes it very fussy and pedantic.

Errors, warnings, and verbosity

Sometimes you just want a Bible to load. Other times you might be more interested in small compromises. The BOS separates errors and warnings about the data into four categories:

Normally only critical errors are displayed to the console. Use the --errors flag to display errors as well, or the --warnings flag to display the top three from the above list.

By default, the BOS explains what it’s doing, especially for long-running tasks. But you can get less or no chat at the console, for example if you are running a batch file known to work, or alternatively, you can get even more info, e.g., loading each book, not just each Bible. The BOS is set-up for five verbosity levels:

You can control these with the --silent --quiet --informative, or --verbose flags. In addition, the --debug flag will display additional information intended for the programmer. (All of these flags/switches also have shorter forms—use --help to see them or see below.)

Main modules

BOS Internal Bible format

Internally, the BOS uses Python objects and data structures to store representations of Bibles. The most important of these are:

  1. InternalBible:
  2. InternalBibleBook (stored in InternalBible.books): contains data for one Bible book (or an introduction or glossary, etc.)
  3. InternalBibleEntry (stored in InternalBibleEntryList): meaningful lines of Biblical text (stored in a pseudo-USFM format)
  4. InternalBibleExtra (stored in InternalBibleExtraList): stores extra fields which are not usually displayed as part of the inline text (such as footnotes, verse cross-references, Strong’s numbers, etc.)
  5. InternalBibleIndexEntry (stored in InternalBibleIndex): enables quick referencing of individual verses

BOS Bible reference synchronisation

It’s not uncommon for two or more Bible programs to want to synchronise references so that a change in one program causes the other program to also switch to the same chapter/verse reference. The BOS also defines a Bible synchronisation system.

Current status

Thus far the BOS is a bit over 90,000 lines of code (plus quite a bit of work invested in the XML data files). It’s not yet optimized for either memory or speed, but merely finished to the minimum standard of getting it to work reliably. (Actually, it’s optimised for readability and for pedantically catching errors and false assumptions.) Some things are not completed, especially mapping from one versification system to another (via a "reference" versification system). However, it is used in a production environment as the foundation of the Bible Drop Box and thus it has proved rugged enough to load and process several hundred different Bible versions in a large range of different formats.

BOS is developed on Ubuntu Linux and has had limited testing on Windows. As well as the Bible Drop Box mentioned above, it is also the foundation of the Biblelator editor Bible editor.

Installation

You need to have a fairly recent version of Python3 installed on your system. You can run any module in demo mode by starting it with no parameters. You can start any module with the --help parameter to get a list of available parameters. (It’s also recommended to start it in debug mode with the --debug command line flag in order to get extra debugging information if it fails to run immediately on your system.) The standard command line parameters for most modules are:

Sample

Here’s a sample program out of the Apps folder. Given a folder on your computer, it loads an autodetected Bible (one of several formats) and does one of several possible exports.

BOS Manager

BOSManager.py is a small graphical program which is packaged with the Biblelator editor Bible editor which allows the user to graphically view the data files from the Bible Organisational System. It’s easiest to work through the Biblelator installation instructions here and then to run BOSManager.py rather than Biblelator.py. (It can also be run from within Biblelator.)

Sword Manager

SwordManager.py is a small graphical program which is packaged with the Biblelator editor Bible editor which allows the user to download and install Sword Bible or commentary modules and view the versified data. It’s easiest to work through the Biblelator installation instructions here and then to run SwordManager.py rather than Biblelator.py. (It can also be run from within Biblelator.)

Future plans

Licence

GPLv3 —see also this quick guide.

This means that you get the program source code, and are even encouraged to change or extend how the programs work.

Help

If you’re a Python programmer who could help with writing code or testing functions, please see the Opportunities Page.

Notes

Any of our code which is still in proof-of-concept stage and not yet publicly released can be requested using the Contact Page.

If you want a JavaScript package with similar functionality (read and index and serve multiple Bible resources in various different ways but via GraphQL), try Proskomma.