D2R Server: Accessing databases with SPARQL and as Linked Data

D2R Server is a tool for publishing relational databases on the Semantic Web. It enables RDF and HTML browsers to navigate the content of the database, and allows querying the database using the SPARQL query language. It is part of the D2RQ Platform.

• 1. About D2R Server
• 2. Features
• 3. Running D2R Server
  • 3.1 Running D2R Server from the command line
  • 3.2 Running D2R Server in a servlet container
• 4. D2R Server configuration
  • 4.1 Server-level configuration options
  • 4.2 Dataset and Resource metadata
• 5. Optimizing performance

1. About D2R Server

D2R Server is a tool for publishing the content of relational databases on the Semantic Web, a global information space consisting of Linked Data.

Data on the Semantic Web is modelled and represented in RDF. D2R Server uses a customizable D2RQ mapping to map database content into this format, and allows the RDF data to be browsed and searched – the two main access paradigms to the Semantic Web.

Requests from the Web are rewritten into SQL queries via the mapping. This on-the-fly translation allows publishing of RDF from large live databases and eliminates the need for replicating the data into a dedicated RDF triple store.

2. Features

Browsing database contents

A simple web interface allows navigation through the database's contents and gives users of the RDF data a “human-readable” preview.

Resolvable URIs

Following the Linked Data principles, D2R Server assigns a URI to each entity that is described in the database, and makes those URIs resolvable – that is, an RDF description can be retrieved simply by accessing the entity's URI over the Web. Semantic Web browsers like Marbles or LinkSailor can follow links from one entity to the next, surfing the Web of Data.

Content negotiation

Following best practices, the regular web interface and the browsable RDF graph share the same URIs. Read more about the interfaces offered by D2R Server, including example HTTP requests and responses, in the Technical Note Publishing Databases on the Semantic Web.

SPARQL endpoint and explorer

The SPARQL interface enables applications to query the database using the SPARQL 1.1 query language over the SPARQL protocol. A simple SPARQL explorer is included.

Downloading contents of BLOBs/CLOBs

D2R Server can be configured to serve files stored in the database in BLOBs or CLOBs. more…

Serving the vocabulary

If new classes and properties are introduced for a D2R deployment, the server can make their URIs resolvable in the spirit of Linked Data, and allows configuration of their labels, comments, and additional properties. more…

Publishing metadata

Metadata such as licensing and provenance information can be attached to every RDF document and web page published by D2R Server. more…

3. Running D2R Server

First, download the D2RQ distribution.

For development and testing, D2R Server can be started from the command line. The recommended setup for production is to deploy the D2R Server web application into a servlet container such as Apache Tomcat or Jetty.

See also: Getting started with D2RQ

3.1 Running D2R Server from the command line

D2R Server can be run as a stand-alone server application that includes its own web server. This is recommended for testing and development.

d2r-server [--port port] [-b serverBaseURI]
      [--fast] [--verbose] [--debug]
      mapping-file.ttl

mapping-file.ttl

The name of the D2RQ mapping file to use.

If no mapping file is provided, then the database connection must be specified on the command line using the same connection parameters as for the generate-mapping tool, and a default mapping will be used.

--port port

D2R Server will be started on this port. Defaults to 2020.

-b serverBaseURI

The base URI where D2R Server is running. Defaults to http://localhost:2020/. Must be specified if the D2R Server is to be accessible from other machines or if it is supposed to be run on a port other than 2020.

--fast

Enables bleeding-edge optimizations that result in better performance but may not be as well-tested. Generally we recommend the use of this option unless problems are observed.

--verbose

Output more logging information.

--debug

Output much more logging information.

3.2 Running D2R Server in a servlet container

D2R Server can be run as a J2EE web application inside an existing servlet container, such as Apache Tomcat or Jetty. This is recommended for production use.

1. Make sure that your mapping file includes a configuration block, as described in the server configuration section. Set the base URI to something like http://servername/webappname/.
2. Change the configFile param in /webapp/WEB-INF/web.xml to the name of your configuration file. For deployment, we recommend placing the mapping file into the /webapp/WEB-INF/ directory.
3. In D2R Server's main directory, Run ant war. This creates the d2rq.war file. You need Apache Ant for this step.
4. Optionally, if you want a different name for your web application, rename the file to webappname.war
5. Deploy the war file into your servlet container, e.g. by copying it into Tomcat's webapps directory.

4. D2R Server configuration

The server can be configured by adding a configuration block to the mapping file. This consists of a d2r:Server instance with configuration properties. An example follows:

@prefix d2r: <http://sites.wiwiss.fu-berlin.de/suhl/bizer/d2r-server/config.rdf#> .
@prefix meta: <http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/metadata#> .


<> a d2r:Server;
  rdfs:label "My D2R Server";
  d2r:baseURI <http://localhost:2020/>;
  d2r:port 2020;
  d2r:vocabularyIncludeInstances true;
  
  d2r:sparqlTimeout 300;
  d2r:pageTimeout 5;

  meta:datasetTitle "My dataset" ;
  meta:datasetDescription "My dataset contains many nice resources." ;
  meta:datasetSource "This other dataset" ;
  
  meta:operatorName "John Doe" ;
  meta:operatorHomepage  ;
  .

4.1 Server-level configuration options

The following configuration properties can be set for the d2r:Server instance:

Note that further configuration options can be set elsewhere in the mapping file. Database-level configuration is specified on the d2rq:Database instances, and configuration of the D2RQ query engine is specified on a d2rq:Configuration instance.

4.2 Dataset and Resource metadata

Often, providing additional information for the served resources is desirable. Main areas for this additional information include licensing, proveance, and general dataset descriptions. D2R Server has comprehensive support for this so-called metadata. First, metadata can be provided on two levels: Every served resource can have metadata assigned, and the entire dataset served by the D2R server installation can also have metadata assigned. Metadata templates are RDF documents, that can contain placeholders, which are replaced with user-specified information, configuration values, or run-time information. The resource metadata are added to the RDF and HTML responses for each requested resource, while the dataset metdata is served at a single URL which is created by appending /dataset to the configuration value d2r:baseURI. Most of the dataset metadata is auto-generated from the mapping file, with the user-specified dataset metadata then being mixed into the RDF and HTML representation of the dataset metadata.

4.2.1 Customizable Default Templates

To simplify the process of serving dataset and resource metadata, D2R server is shipped with two default templates, which are easily customized through the server configuration. These default templates use a combination of the Provenance Vocabulary to describe the data creation process in a machine-readable way, the VoID Vocabulary for interlinked datasets, and the Dublin Core Vocabulary for general authorship terms. The default templates are located in the webapp/WEB-INF/ directory in the D2R source distribution.

The default templates are customized in the server configuration block, for example:

@prefix d2r: <http://sites.wiwiss.fu-berlin.de/suhl/bizer/d2r-server/config.rdf#> .
@prefix meta: <http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/metadata#> .


<> a d2r:Server;
  #[...]
  
  meta:datasetTitle "My dataset" ;
  meta:datasetDescription "My dataset contains many nice resources." ;
  meta:datasetSource "This other dataset" ;
  
  meta:operatorName "John Doe" ;
  meta:operatorHomepage  ;
  .

The following user-specified values can be set for the default metadata templates:

4.2.2 Custom Metadata Templates

Even though we recommend using the default templates, there will still be users who wish to set their own metadata templates for the dataset and each resource. For these cases, D2R server supports overriding the default templates for resource and dataset metadata. In the server configuration, the flags d2r:metadataTemplate and d2r:datasetMetadataTemplate can be used for this purpose, respectively. We recomment using the default templates as a starting point. It should be noted that in the case of the dataset description, most information there is auto-generated from the server configuration and D2RQ mapping. The default templates can then be overridden as follows:

@prefix d2r: <http://sites.wiwiss.fu-berlin.de/suhl/bizer/d2r-server/config.rdf#> .

<> a d2r:Server;
  #[...]
  d2r:metadataTemplate "/some/absolute/path/to/resource-metadata.ttl" ;
  d2r:datasetMetadataTemplate "../some/relative/path/to/dataset-metadata.ttl" ;
  .

Inside these templates, placeholders can be used to represent configuration values, or run-time information. Template placeholders are organized into three groups that represent the available data sources: runtime, config, database, and metadata. All placeholder URIs follow the scheme about:metadata:groupname:identifier.

The following placeholder groups are currently available:

Placeholder group: runtime

Placeholders in this group are replaced with D2R Server runtime data. Currently, D2R Server supports the following runtime placeholders:

Placeholder group: config

This group of placeholders corresponds to the configuration options for the d2r:Server instance. For example, the placeholder about:metadata:config:baseURI is replaced with the value of the d2r:baseURI property defined on the d2r:Server instance.

Placeholder group: server

This group of placeholders corresponds to the configuration options for the d2rq:Database instance. For example, the placeholder about:metadata:database:jdbcDSN is replaced with the value of the d2rq:jdbcDSN property defined on the d2r:Database instance.

Placeholder group: metadata

Placeholders in this group allow the inclusion of additional metadata attributes, whose replacement attributes are defined as properties of the d2r:Server instance from the http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/metadata# namespace (abbreviated with the prefix meta: in the following). Each placeholder about:metadata:metadata:identifier is replaced by the value of the property specified using a corresponding meta:identifier predicate.

In the following example, the property meta:operatorHomepage is provided, whose value will replace mentions of about:metadata:metadata:operatorHomepage in the metadata template:

@prefix meta: <http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/metadata#> .

<> a d2r:Server;
    d2r:metadataTemplate "metadata.ttl";
    meta:operatorHomepage <http://example.org/URI_of_publisher> ;

4.2.3. Sample Template

@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix xhv:  <http://www.w3.org/1999/xhtml/vocab#> .

<about:metadata:runtime:graph> foaf:primaryTopic <about:metadata:runtime:resource> .
<about:metadata:runtime:graph> xhv:license <about:metadata:metadata:license> .

This template adds two triples to the RDF graphs served by the corresponding D2R Server instance. The first triple asserts that the primary topic of the served RDF graph is the resource that was requested. The second triple identifies the license of the served RDF graph and requires the metadata attribute http://www4.wiwiss.fu-berlin.de/bizer/d2r-server/metadata#license to be specified in the mapping file.

5. Optimizing performance

Here are some simple hints to improve D2R's performance:

• Define primary keys whenever you can and create indexes where applicable (e.g. on foreign keys) – besides optimizing database performance, these will be picked up and used by various optimizations within D2RQ.
• Use the latest optimizations by launching D2R Server with --fast (or activate d2rq:useAllOptimizations).
• Provide hint properties.
• Indicate directions in d2rq:joins to enable join optimizations.
• Give D2R Server more heap space by means of Java's -Xmx parameter in d2r-server or d2r-server.bat (default: -Xmx1G).
• To prevent excessively large pages in the web interface, consider changing the values of d2r:limitPerClassMap and. d2r:limitPerPropertyBridge.
• Consider changing the default timeout values for SPARQL queries and for page generation.
• To speed up the generation of large pages, consider setting d2rq:resultSizeLimit, d2rq:limit, and d2r:vocabularyIncludeInstances.
• Set d2rq:autoReloadMapping to false where it is not required.
• Databases often ship with development configurations that are designed for a small footprint rather than performance. For instance, some good pointers for optimizing MySQL can be found here.