Jena I/O Mini HowTo

This is a guide to the I/O subsystem of Jena. All Jena users need to know to use InputStreams and OutputStreams rather than Readers and Writers. The first section gives a quick introduction to the I/O subsystem. The other sections are aimed at users wishing to use advanced features within the I/O subsystem.

Contents

• 1. Quick Introduction
  • 1.1 RDF/XML, RDF/XML-ABBREV
  • 1.2 N3
  • 1.3 N-TRIPLE
  • 1.4 TURTLE
• 2. Character Encoding Issues
  • 2.1 Encodings Supported in Jena 2.2 and later
• 3. When to Use Reader and Writer?
• 4. Introduction to Advanced Jena I/O
• 5. Advanced RDF/XML Input
  • 5.1 ARP properties
  • 5.2 Interrupting ARP
• 6. Advanced RDF/XML Output
• 7. Advanced N3 Output
• 8. Conformance
• 9. Faster RDF/XML I/O

1. Quick Introduction

The main I/O methods in Jena use InputStream's and OutputStream's. These methods are found on the Model interface. These are:

The built-in languages are "RDF/XML", "RDF/XML-ABBREV", "N-TRIPLE", "N3" and "TURTLE". In addition, for N3 output the language can be specified as: "N3-PP", "N3-PLAIN" or "N3-TRIPLE", which controls the style of N3 produced.

There are also methods which use Reader's and Writer's. Do not use them, unless you are sure it is correct to. In advanced applications, they are useful, see below; and there is every intention to continue to support them. The RDF/XML parser now checks to see if the Model.read(Reader …) calls are being abused, and issues ERR_ENCODING_MISMATCH and WARN_ENCODING_MISMATCH errors. Most incorrect usage of Reader's for RDF/XML input will result in such errors. Most incorrect usage of Writer's for RDF/XML output will produce correct XML by using an appropriate XML declaration giving the encoding - e.g. <?xml version='1.0' encoding='ISO-8859-15'?> on my system. However, such XML is less portable than XML in UTF-8. Using the Model.write(OutputStream …) methods allows the Jena system code to choose UTF-8 encoding, which is the best choice.

1.1 RDF/XML, RDF/XML-ABBREV

For input, both of these are the same, and fully implement the RDF Syntax Recommendation, see conformance.

For output, "RDF/XML", produces regular output reasonably efficiently, but it is not readable. In contrast, "RDF/XML-ABBREV", produces readable output without much regard to efficiency.

All the readers and writers for RDF/XML are configurable, see below, input and output.

1.2 N3

The N3 readers and writers implement Tim Berners-Lee's N3 language.

There are actually 4 writers:

• N3: The standard writer that choose one of the other 3.
• N3-PP: The full N3 pretty writer
• N3-PLAIN: An N3 writer that does not nest bNode strutures but does write record-like groups of all properties for a subject
• N3-TRIPLE: Writer one statement per line, like N-TRIPLES, but also does qname conversion of URIrefs.

The standard Jena writer "N3" chooses which writer to use based on the system property com.hp.hpl.jena.n3.N3JenaWriter.writer.

If none of these produce N3 to your liking, it is possible to further customize the output.

1.3 N-TRIPLE

The N-TRIPLE readers and writers implement RDF Core's N-Triples language. They are not configurable.

1.4 TURTLE

The N3 reader accepts any valid Turtle. Note that the N3 and Turtle writers produce internationalized qnames, with the character set from XML Namespaces (except for ':'), not restricted to ASCII as is the definition of N3 and Turtle.

The Turtle writer is the N3 writer configured with: usePropertySymbols=false, useTripleQuotedStrings=false, useDoubles=false.

2. Character Encoding Issues

The easiest way to not read or understand this section is always to use InputStreams and OutputStreams with Jena, and to never use Readers and Writers. If you do this, Jena will do the right thing, for the vast majority of users. If you have legacy code that uses Readers and Writers, or you have special needs with respect to encodings, then this section may be helpful. The last part of this section summarizes the character encodings supported by Jena.

Character encoding is the way that characters are mapped to bytes, shorts or ints. There are many different character encodings. Within Jena, character encodings are important in their relationship to Web content, particularly RDF/XML files, which cannot be understood without knowing the character encoding, and in relationship to Java, which provides support for many character encodings.

The Java approach to encodings is designed for ease of use on a single machine, which uses a single encoding; often being a one-byte encoding, e.g. for European languages which do not need thousands of different characters.

The XML approach is designed for the Web which uses multiple encodings, and some of them requiring thousands of characters.

On the Web, XML files, including RDF/XML files, are by default encoded in "UTF-8" (Unicode). This is always a good choice for creating content, and is the one used by Jena by default. Other encodings can be used, but may be less interoperable. Other encodings should be named using the canonical name registered at IANA, but other systems have no obligations to support any of these, other than UTF-8 and UTF-16.

Within Java, encodings appear primarily with the InputStreamReader and OutputStreamWriter classes, which convert between bytes and characters using a named encoding, and with their subclasses, FileReader and FileWriter, which convert between bytes in the file and characters using the default encoding of the platform. It is not possible to change the encoding used by a Reader or Writer while it is being used. The default encoding of the platform depends on a large range of factors. This default encoding may be useful for communicating with other programs on the same platform. Sometimes the default encoding is not registered at IANA, and so Jena application developers should not use the default encoding for Web content, but use UTF-8.

2.1 Encodings Supported in Jena 2.2 and later

On RDF/XML input any encoding supported by Java can be used. If this is not a canonical name registered at IANA a warning message is produced. Some encodings have better support in Java 1.5 than Java 1.4; for such encodings a warning message is produced on Java 1.4, suggesting upgrading.

On RDF/XML output any encoding supported by Java can be used, by constructing an OutputStreamWriter using that encoding, and using that for output. If the encoding is not registered at IANA then a warning message is produced. Some encodings have better support in Java 1.5 than Java 1.4; for such encodings a warning message is produced on Java 1.4, suggesting upgrading.

Java can be configured either with or without a jar of extra encodings on the classpath. This jar is charsets.jar and sits in the lib directory of the Java Runtime. If this jar is not on your classpath then the range of encodings supported is fairly small.

The encodings supported by Java are listed by Sun, for 1.4.2, and 1.5.0. For an encoding that is not in these lists it is possible to write your own transcoder as documented in the java.nio.charset package documentation.

Earlier versions of Jena supported fewer encodings.

3. When to Use Reader and Writer?

Infrequently.

Despite the character encoding issues, it is still sometimes appropriate to use Readers and Writers with Jena I/O. A good example is using Readers and Writers into StringBuffers in memory. These do not need to be encoded and decoded so a character encoding does not need to be specified. Other examples are when an advanced user explicitly wishes to correctly control the encoding.

Incorrect use of these read(Reader, …) methods results in warnings and errors with RDF/XML and RDF/XML-ABBREV (except in a few cases where the incorrect use cannot be automatically detected). Incorrect use of the write(Writer, …) methods results in peculiar XML declarations such as <?xml version="1.0" encoding="WINDOWS-1252"?>. This would reflect that the character encoding you used (probably without realizing) in your Writer is registered with IANA under the name "WINDOWS-1252". The resulting XML is of reduced portability as a result. Glenn Marcy notes:

With N-TRIPLE incorrect use is usually benign, since N-TRIPLE is ascii based.

Character encoding issues of N3 are not well-defined; hence use of these methods may require changes in the future. Use of the InputStream and OutputStream methods will allow your code to work with future versions of Jena which do the right thing - whatever that is. Currently the OutputStream methods use UTF-8 encoding.

4. Introduction to Advanced Jena I/O

The RDF/XML input and output is configurable.

However, to configure it, it is necessary to access an RDFReader or RDFWriter object that remains hidden in the simpler interface above.

The four vital calls in the Model interface are:

Each of these calls returns an RDFReader or RDFWriter that can be used to read or write any Model (not just the one which created it). As well as the necessary read and write methods, these interfaces provide:

RDFErrorHandler	setErrorHandler(RDFErrorHandler errHandler)           Set an error handler for the reader
java.lang.Object	setProperty(java.lang.String propName, java.lang.Object propValue)           Set the value of a reader property.

Setting properties, or the error handler, on an RDFReader or an RDFWriter allows the programmer to access non-default behaviour. Moreover, since the RDFReader and RDFWriter is not bound to a specific Model, a typical idiom is to create the RDFReader or RDFWriter on system initialization, to set the appropriate properties so that it behaves exactly as required in your system, and then to do all subsequent I/O through it.

    Model m = Modelfactory.createDefaultModel();
    RDFWriter writer = m.getRDFWriter();
    m = null; // m is no longer needed.
    writer.setErrorHandler(myErrorHandler);
    writer.setProperty("showXmlDeclaration","true");
    writer.setProperty("tab","8");
    writer.setProperty("relativeURIs","same-document,relative");
    …
    Model marray[];
    …
    for (int i=0; i<marray.length; i++) {
    …
        OutputStream out = new FileOutputStream("foo" + i + ".rdf");
        writer.write(marray[i],
                           out,
          "http://example.org/");
        out.close();
    }

Note that all of the current implementations are synchronized, so that a specific RDFReader cannot be reading two different documents at the same time. In a multihtreaded application this may suggest a need for a pool of RDFReaders and/or RDFWriters, or alternatively to create, initialize, use and discard them as needed.

For N-TRIPLE there are currently no properties supported for neither the RDFReader nor the RDFWriter. Hence this idiom above is not very helpful, and just using the Model.write() methods may prove easier.

For RDF/XML and RDF/XML-ABBREV there are many options in both the RDFReader and the RDFWriter. N3 has options on the RDFWriter. These options are detailed below. For RDF/XML they are also found in the javadoc for JenaReader.setProperty(String, Object) and RDFXMLWriterI.setProperty(String, Object).

5. Advanced RDF/XML Input

For access to these advanced features, first get an RDFReader object that is an instance of an ARP parser, by using the getReader() method on any Model. It is then configured using the setProperty(String, Object) method. This changes the properties for parsing RDF/XML. Many of the properties change the RDF parser, some change the XML parser. (The Jena RDF/XML parser, ARP, implements the RDF grammar over a Xerces2-J XML parser). However, changing the features and properties of the XML parser is not likely to be useful, but was easy to implement.

ARP properties

These allow fine grain control over the extensive error reporting capabailities of ARP. And are detailed directly below.

SAX2 features

See Xerces features. Value should be given as a String "true" or "false" or a Boolean.

SAX2 properties

See Xerces properties.

Xerces features

See Xerces features. Value should be given as a String "true" or "false" or a Boolean.

Xerces properties

See Xerces properties.

5.1 ARP properties

An ARP property is referred to either by its property name, (see below) or by an absolute URL of the form http://jena.hpl.hp.com/arp/properties/<PropertyName>. The value should be a String, an Integer or a Boolean depending on the property.

ARP property names and string values are case insensitive.

As an example, if you are working in an environment with legacy RDF data that uses unqualified RDF attributes such as "about" instead of "rdf:about", then the following code is appropriate:

    Model m = Modelfactory.createDefaultModel();
    RDFReader arp = m.getReader();
    m = null; // m is no longer needed.
    // initialize arp
    // Do not warn on use of unqualified RDF attributes.
    arp.setProperty("WARN_UNQUALIFIED_RDF_ATTRIBUTE","EM_IGNORE");

    …
        InputStream in = new FileInputStream(fname);
        arp.read(m,in,url);
        in.close();

As a second example, suppose you wish to work in strict mode, but allow "daml:collection", the following works:

     …
     arp.setProperty("error-mode", "strict" );
     arp.setProperty("IGN_DAML_COLLECTION","EM_IGNORE");
     …

The other way round does not work.

     …
     arp.setProperty("IGN_DAML_COLLECTION","EM_IGNORE");
     arp.setProperty("error-mode", "strict" );
     …

This is because in strict mode IGN_DAML_COLLECTION is treated as an error, and so the second call to setProperty overwrites the effect of the first.

5.2 Interrupting ARP

ARP can be interrupted using the Thread.interrupt() method. This causes an ERR_INTERRUPTED error during the parse, which is usually treated as a fatal error.

Here is an illustrative code sample:

  ARP a = new ARP();
  final Thread arpt = Thread.currentThread();
  Thread killt = new Thread(new Runnable() {
       public void run() { 
  	     try {
	        Thread.sleep(tim);
	     } catch (InterruptedException e) {
	     } 
	     arpt.interrupt();
       }
    });
  killt.start();
  try {
    in = new FileInputStream(fileName);
    a.load(in);
    in.close();
    fail("Thread was not interrupted.");
  } catch (SAXParseException e) {
  }

6. Advanced RDF/XML Output

The first RDF/XML output question is whether to use the "RDF/XML" or RDF/XML-ABBREV writer. While some of the code is shared, this two writers are really very different, resulting in different but equivalent output. RDF/XML-ABBREV is slower, but should produce more readable XML.

For access to advanced features, first get an RDFWriter object, of the appropriate language, by using getWriter("RDF/XML") or getWriter("RDF/XML-ABBREV") on any Model. It is then configured using the setProperty(String, Object) method. This changes the properties for writing RDF/XML.

  w.setProperty("prettyTypes",
           new Resource[]{
                DAML_OIL.Ontology,
                DAML_OIL.Class,
                DAML_OIL.Datatype,
                DAML_OIL.Property,
                DAML_OIL.ObjectProperty,
                DAML_OIL.DatatypeProperty,
                DAML_OIL.TransitiveProperty,
                DAML_OIL.UnambigousProperty,
                DAML_OIL.UniqueProperty,
                });
   

As an example,

     RDFWriter w = m.getWriter("RDF/XML-ABBREV");
     w.setProperty("attribtueQuoteChar","'");
     w.setProperty("showXMLDeclaration","true");
     w.setProperty("tab","1");
     w.setProperty("blockRules",
       "daml:collection,parseTypeLiteralPropertyElt,"
       +"parseTypeResourcePropertyElt,parseTypeCollectionPropertyElt");

creates a writer that does not use rdf:parseType (preferring rdf:datatype for rdf:XMLLiteral), indents only a little, and produces the XMLDeclaration. Attributes are used, and are quoted with "'".

Note that property attributes are not used at all, by default. However, the RDF/XML-ABBREV writer includes a rule to produce property attributes when the value does not contain any spaces. This rule is normally switched off. This rule can be turned on selectively by using the blockRules property as detailed above.

7. Advanced N3 Output

The N3 pretty printer (which is used by default)  and the N3 plain writer provide a number of properties to control their output.  The properties all start: http://jena.hpl.hp.com/n3/properties/. The name used can be the full name, starting with this string, or the short form of just the name below.  All values are strings; they may be interpreted as integer, boolean or string as defined below.

Notes:

1. Only the N3 pretty printer print object lists as comma separated lists.

8. Conformance

The RDF/XML I/O endeavours to conform with the RDF Syntax Recommendation.

The parser must be set to strict mode. (Note that, the conformant behaviour for rdf:parseType="daml:collection" is to silently turn "daml:collection" into "Literal").

The RDF/XML writer is conformant. But does not exercise much of the grammar.

The RDF/XML-ABBREV writer exercises all of the grammar and is conformant except that it uses the "daml:collection" construct for DAML ontologies. This non-conformant behaviour can be switched off using the "blockRules" property.

To create such conformant readers and writers use the following:

   Model m;
   …
   RDFReader conformantParser = m.getReader();
   conformantParser.setProperty("error-mode","strict");
   …
   RDFWriter conformantAbbrevWriter = m.getWriter("RDF/XML-ABBREV");
   conformantAbbrevWriter.setProperty("blockRules","daml:collection");

9. Faster RDF/XML I/O

To optimise the speed of writing RDF/XML it is suggested that all URI processing is turned off. Also do not use RDF/XML-ABBREV. It is unclear whether the longId attribute is faster or slower; the short IDs have to be generated on the fly and a table maintained during writing. The longer IDs are long, and hence take longer to write. The following creates a faster writer:

   Model m;
   …
   …
   RDFWriter fasterWriter = m.getWriter("RDF/XML");
   fasterWriter.setProperty("allowBadURIs","true");
   fasterWriter.setProperty("relativeURIs","");
   fasterWriter.setProperty("tab","0");

When reading RDF/XML the check for reuse of rdf:ID has a memory overhead, which can be significant for very large files. In this case, this check can be suppressed by telling ARP to ignore this error.

   Model m;
   …
   …
   RDFReader bigFileReader = m.getReader("RDF/XML");
   bigFileReader.setProperty("WARN_REDEFINITION_OF_ID","EM_IGNORE");
   …