ODF: Our Document Future

Donna Benjamin, Open Source Industry Australia and Creative Contingencies.

Paper presented at XTech Conference, Amsterdam, May 2006.

The XML based OpenDocument format represents our document future because of the part it will play in the practice of digital preservation. The National Archives of Australia took part in the development and standardisation of the open office xml file format that has gone on to become the OASIS OpenDocument Format for Office Applications. Members of their digitial preservation software team sat on the technical committee that drafted the standard.

As we've increased our capacity to store data externally from ourselves we've decreased our own ability to remember it. Whilst undertaking the background research for this paper, I was surprised to discover that one of the reasons behind Socrates' dislike of reading and writing, was his belief those skills diminished his students capacity to remember and recall spoken information. It was later acknowledged that by not having to remember the text itself, students engaged in higher level analysis and understanding.

Whatever the means, knowledge has endured. The irony is that if it weren't for the writings of Plato and Phaedrus, we wouldn't know what Socrates had thought or said. Nevertheless, oral traditions are important, and every society that didn't develop writing relied completely on transmitting ancestral wisdom by speaking it. This survives today in the form of family stories. Most things we learn about our families are because of this oral tradition. Think about the stories you may in turn pass on to your children, nieces or cousins.

In Shakespeare's time, most of the audience at his plays would not have been able to read and write, but many would have remembered, verbatim, an entire play they had just seen. We can't remember things like we used to. Preserving what we've recorded is even more important.

Over time we've recorded information on stone, clay, papyrus, vellum, paper, celluloid, magnetic tape, floppy disks, cds, dvds and hard drives. Stone and paper endures, but things recorded on floppies and magnetic tape are in danger. It seems we've adopted a much more short-term approach to information storage.

There are four elements of digital technology that threaten our future access to stored data.

1. The media on which it is stored.

2. The hardware used to design and create it.

3. The software used to create, read or edit that data and

4. The standards we rely on to record and format data.

Digital archivists understand the dangers of those elements and are working to address these issues. A number of strategies have been identified. Naturally there is disagreement about what will prove to be the best approach. However, many agree that a range of approaches will be required to prevent our time being declared the digital dark age.

Some key strategies have been identified to address the preservation of digital information. These are technology preservation, emulation, migration and encapsulation.

Technology Preservation means maintaining museums with collections of obsolete computer technology kept in working order ready to access archived documents.

Emulation means developing virtual machines to run archived software and data. These systems can be developed when the data is archived, or at a later point, when it needs to be accessed by using the specification and standards that were archived along with the data in question.

Migration means bringing the data forward onto modern systems, storing it on modern hardware and converting it to more recent or standard file formats. It must be constantly refreshed onto new media. This is a particularly useful strategy for documents that are likely to be reused and accessed frequently.

Encapsulation involves storing the data inside a digital envelope that records important metadata.

These four preservation strategies can be organised into two main approaches

1. preserve the technical environment with hardware museums and emulation.

2. overcome obsolescence of file formats with encapsulation and migration.

Which brings us to XML. This really seems to be the key. XML can be created and interpreted independantly of any particular computer platform or program. It is an open standard that is widely accepted. It was developed with the aim of separating form, structure and content, and it is flexible and extensible. Most importantly perhaps, it is free and it is readable by both machines and human beings. Files can easily be converted into XML formats and XML is a useful way of recording and preserving metadata. It would seem that XML is part of the answer for addressing our digital preservation needs.

OpenDocument is a file format for office application documents. It's been developed and ratified by an OASIS technical committee and has just been approved as a standard by the International Organisation for Standardization. OpenDocument uses XML to store and describe data. This means the document content can be read independently of the application that created it. The specification on how to read, and develop software that can read and write ODF files is completely open.

Michael Brauer, from Sun Microsystems, spoke at this conference last year. The title was "From Open Source to Open Standard: The OASIS OpenDocument Format". I'd urge you to read his paper for more detail on the OpenDocument and how it came to be considered and adopted as a standard. http://idealliance.org/proceedings/xtech05/papers/03-02-02/

When talking to a friend about this paper, he asked "Why does ODF and XML matter for office file formats?" Does anyone here remember using word processors in the 1980s? Remember the difficulties of sharing files? Dealing with different file formats? I reminded him of a typewriter based word processor he used to write essays on in High School. He could save his work to a floppy disc, but no other computer he had access to at the time could read that floppy disc, let alone access the files stored on it.

I'm sure many of you would have your own story of frustration regarding file incompatibilities. This is why ODF matters, because it holds promise, as an independent standard, to help us 'solve' this problem and I'm of the view that 'standards' are a sign of mature technology.

Open standards and open formats are critical if we are to make serious efforts to preserve the 'born digital' documents of our age. I spoke earlier of four approaches that have been identified to attack this problem; preservation of the technology itself, emulation of the hardware that runs the software, migration of the documents and digital artifacts to modern formats, and encapsulation. ODF will be most useful for migration and encapsulation.

Applications like OpenOffice, Koffice and StarOffice can already open a number of existing file formats, and convert them to ODF, this is enormously helpful for migration strategies. Encapsulation strategies will also benefit from using ODF because it zips the different elements of the document together into one file. An ODF file is effectively a collection of files and folders compressed together to form one file. Style is separated from content, embedded images are saved in the 'pictures' folder, and metadata is included there too.

It is true that Microsoft Word has become a de facto standard due to its ubiquitous presence on 85% of the world's desktop computers. But not everyone can afford the ticket price of Word, and governments especially are increasingly sensitive to this. Many are moving towards delivering services and information on-line and are becoming aware they should not be mandating that their people buy a particular product in order to access government information and services.

Word files, however, are probably the least of our problems. A lot of precious information is stored in other formats with extensions like .xls and .ppt or even .psd and .mdb. Let's look at spreadsheets - essentially these are rows and columns of data, usually numbers, and they often contain formulas for calculations associated with that data. Printing and archiving a text document is a valid preservation method. We know paper works if it's looked after, just look at Britain's Domesday book from 1086. It's fragile, but it's legible. But paper can't help us preserve the functionality of a spreadsheet with complex macros and calculations. This is further complicated if that spreadsheet accesses data stored in a network database. Or the calculations in a budget vary due to different implementations of floating point mathematics.

For more on the real technical nitty gritty of the OpenDocument Format take a look at Daniel Carrera's "Introduction to the Format Internals" article on the OpenDocument fellowship website.http://opendocumentfellowship.org/Articles/IntroductionToTheFormatInternals

There are a number of digital preservation initiatives being implemented and tested in Australia. This is not to say Australia is alone in this. There is brilliant work being done here in the Netherlands. A White Paper called "XML and Digital Preservation" gives a more in depth analysis of XML and digital preservation issues.http://www.digitaleduurzaamheid.nl/bibliotheek/docs/white-paper_xml-en.pdf

Three organisations in Australia are leading the charge. The National Library of Australia with the PADI and PANDORA projects, The Public Record Office of Victoria has the VERS initiative, and the National Archives of Australia has a warrior called Xena.