SPARQL 1.1 Incremental Protocol

1. Introduction

For applications that maintain long-lived queries over changing data — such as incremental view maintenance (IVM) engines, live dashboards, and stream processors — repeatedly reissuing queries and re-fetching full results is inefficient and adds latency.

This specification defines an extension to the SPARQL Protocol that allows a SPARQL Protocol service to deliver results as a server-sent events event stream. Instead of returning a single result document, the service emits:

• an initial event containing a complete result set.

• zero or more processing event indicating that the service is working on new changes.

• zero or more update events conveying incremental additions and deletions.

• one or more up-to-date events indicating that, at the moment of emission, the service has processed all known changes for the query.

• zero or one error event indicating that an error has occurred, after which no further events will be sent.

This version of the specification normatively defines JSON-based and XML-based serializations over SSE result streams.

1.1. Terminology

• Incremental SPARQL Query: A SPARQL query that is intended to be maintained incrementally over time as the underlying data changes.

2. SPARQL Incremental Protocol

1. Content negotiation to select the SSE-based incremental result format.

2. The body of the success response format itself, which is now a stream of SSE events.

2.1. Content Negotiation

2.2. Events Semantics

initial

This event conveys the initial state of the query results. It MUST be the first event emitted by the server upon successfully establishing the stream. The data payload contains the complete result set, serialized according to the negotiated format.

processing

This event indicates that the server is processing updates to the underlying data. It serves as a heartbeat or progress indicator. The payload MAY contain a timestamp representing the latest data change being processed. This event does not guarantee that an update event will follow (e.g., if the processed changes do not affect the query result).

update

This event conveys incremental changes to the query results since the last initial or update event. The payload describes a set of additions and/or removals to the result set. The head with variable names and the optional link array SHOULD NOT be included in these events. The server MUST prune deletions and additions that cancel each other out before emitting this event. The server MAY emit multiple update events before an up-to-date event to batch changes.

NOTE: When processing updates, the deletions should be applied before additions to avoid transient inconsistencies in the result set.

up-to-date

This event signals that the client’s view is consistent with the server’s state at a specific point in time. It indicates that all changes known up to the included timestamp have been processed and any resulting update events have been emitted. The payload MUST include a timestamp indicating this consistency point.

error

This event indicates a terminal failure in the stream generation. After emitting this event, the server MUST close the connection. The payload MUST include an error code and a description. This event is reserved for errors occurring after the stream has started; initial request errors MUST be handled via standard HTTP status codes.

2.3. Events JSON

This section defines the JSON serialization for the incremental events. The event field of the SSE message corresponds to the event type defined in § 2.2 Events Semantics. The data field contains the JSON payload.

2.3.1. initial Event

The initial event payload follows the SPARQL Results JSON format.

event: initial
data: {
data:   "head": { "vars": [ "book", "title" ] },
data:   "results": {
data:     "bindings": [
data:       {
data:         "book": { "type": "uri", "value": "http://example.org/book/book1" },
data:         "title": { "type": "literal", "value": "SPARQL Tutorial" }
data:       }
data:     ]
data:   }
data: }

2.3.2. processing Event

The processing event payload is a JSON object that MAY contain a timestamp property (xsd:dateTime) indicating the time of the latest processed change.

event: processing
data: { "timestamp": "2024-11-24T10:00:00Z" }

2.3.3. update Event

The update event payload is a JSON object containing additions and deletions arrays. Each array contains bindings formatted exactly as the bindings array in SPARQL Results JSON.

event: update
data: {
data:   "additions": [
data:     {
data:       "book": { "type": "uri", "value": "http://example.org/book/book2" },
data:       "title": { "type": "literal", "value": "The Semantic Web" }
data:     }
data:   ],
data:   "deletions": [
data:     {
data:       "book": { "type": "uri", "value": "http://example.org/book/book1" },
data:       "title": { "type": "literal", "value": "SPARQL Tutorial" }
data:     }
data:   ]
data: }

2.3.4. up-to-date Event

The up-to-date event payload is a JSON object that MUST contain a timestamp property (xsd:dateTime) indicating the consistency point.

event: up-to-date
data: { "timestamp": "2024-11-24T10:05:00Z" }

2.3.5. error Event

The error event payload is a JSON object containing status (integer) and statusText (string) properties.

event: error
data: {
data:   "status": 500,
data:   "statusText": "Internal Server Error: Query execution timeout"
data: }

2.4. Events XML

This section defines the XML serialization for the incremental events. The event field of the SSE message corresponds to the event type defined in § 2.2 Events Semantics. The data field contains the XML payload. The incremental protocol elements are defined in the namespace http://www.w3.org/ns/sparql-incremental#. Standard SPARQL results elements are used within this wrapper in the http://www.w3.org/2005/sparql-results# namespace.

The http://www.w3.org/ns/sparql-incremental# namespace needs to be registered.

2.4.1. initial Event

The initial event payload follows the SPARQL Results XML format. It uses the standard SPARQL results namespace.

event: initial
data: <?xml version="1.0"?>
data: <sparql xmlns="http://www.w3.org/2005/sparql-results#">
data:   <head>
data:     <variable name="book"/>
data:     <variable name="title"/>
data:   </head>
data:   <results>
data:     <result>
data:       <binding name="book"><uri>http://example.org/book/book1</uri></binding>
data:       <binding name="title"><literal>SPARQL Tutorial</literal></binding>
data:     </result>
data:   </results>
data: </sparql>

2.4.2. processing Event

The processing event payload is an XML element processing in the incremental namespace that MAY contain a timestamp attribute (xsd:dateTime).

event: processing
data: <processing xmlns="http://www.w3.org/ns/sparql-incremental#" timestamp="2024-11-24T10:00:00Z"/>

2.4.3. update Event

The update event payload is an XML element update in the incremental namespace containing additions and deletions child elements. These elements contain result elements formatted exactly as in SPARQL Results XML, using the standard SPARQL results namespace.

event: update
data: <update xmlns="http://www.w3.org/ns/sparql-incremental#">
data:   <additions>
data:     <result xmlns="http://www.w3.org/2005/sparql-results#">
data:       <binding name="book"><uri>http://example.org/book/book2</uri></binding>
data:       <binding name="title"><literal>The Semantic Web</literal></binding>
data:     </result>
data:   </additions>
data:   <deletions>
data:     <result xmlns="http://www.w3.org/2005/sparql-results#">
data:       <binding name="book"><uri>http://example.org/book/book1</uri></binding>
data:       <binding name="title"><literal>SPARQL Tutorial</literal></binding>
data:     </result>
data:   </deletions>
data: </update>

2.4.4. up-to-date Event

The up-to-date event payload is an XML element up-to-date in the incremental namespace that MUST contain a timestamp attribute (xsd:dateTime).

event: up-to-date
data: <up-to-date xmlns="http://www.w3.org/ns/sparql-incremental#" timestamp="2024-11-24T10:05:00Z"/>

2.4.5. error Event

The error event payload is an XML element error in the incremental namespace containing status and statusText attributes.

event: error
data: <error xmlns="http://www.w3.org/ns/sparql-incremental#" status="500" statusText="Internal Server Error: Query execution timeout"/>

2.5. Events CSV and TSV

This section defines the CSV and TSV serializations for the incremental events. The event field of the SSE message corresponds to the event type defined in § 2.2 Events Semantics. The data field contains the payload formatted according to SPARQL Results CSV or SPARQL Results TSV. For TSV, the values MUST be encoded as defined in SPARQL Results TSV (e.g., IRIs enclosed in <>).

2.5.1. initial Event

The initial event payload follows the standard SPARQL Results CSV or SPARQL Results TSV format.

event: initial
data: book,title
data: http://example.org/book/book1,"SPARQL Tutorial"

2.5.2. processing Event

The processing event payload is a CSV/TSV document with a single column timestamp.

event: processing
data: timestamp
data: 2024-11-24T10:00:00Z

2.5.3. update Event

The update event payload is a CSV/TSV document. To distinguish between additions and deletions, the payload MUST include a special column named _op as the first column. The values in this column MUST be add for additions and del for deletions. The remaining columns correspond to the variables in the query results.

event: update
data: _op,book,title
data: add,http://example.org/book/book2,"The Semantic Web"
data: del,http://example.org/book/book1,"SPARQL Tutorial"

NOTE: This serialization does not allow to update the head information (variable names).

2.5.4. up-to-date Event

The up-to-date event payload is a CSV/TSV document with a single column timestamp.

event: up-to-date
data: timestamp
data: 2024-11-24T10:05:00Z

2.5.5. error Event

The error event payload is a CSV/TSV document with columns status and statusText.

event: error
data: status,statusText
data: 500,"Internal Server Error: Query execution timeout"

2.6. Events RDF

This section defines the RDF serialization for the incremental events. This serialization is used when the client negotiates an RDF media type (e.g., text/turtle, application/trig, application/ld+json). It is applicable only to CONSTRUCT and DESCRIBE queries, which return an RDF Graph. The event field of the SSE message corresponds to the event type defined in § 2.2 Events Semantics. The data field contains the RDF payload serialized in the negotiated format.

The incremental protocol terms are defined in the namespace http://www.w3.org/ns/sparql-incremental# (prefix sip).

2.6.1. initial Event

The initial event payload contains the complete RDF Graph result.

event: initial
data: @prefix : <http://example.org/> .
data: :book1 :title "SPARQL Tutorial" .

2.6.2. processing Event

The processing event payload contains an instance of sip:Processing.

event: processing
data: @prefix sip: <http://www.w3.org/ns/sparql-incremental#> .
data: @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
data:
data: [] a sip:Processing ;
data:    sip:timestamp "2024-11-24T10:00:00Z"^^xsd:dateTime .

2.6.3. update Event

The update event payload conveys additions and deletions. The additions and deletions MUST be described using standard RDF Reification. The sip:Update instance links to the reified statements via sip:additions and sip:deletions.

event: update
data: @prefix sip: <http://www.w3.org/ns/sparql-incremental#> .
data: @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
data: @prefix : <http://example.org/> .
data:
data: [] a sip:Update ;
data:    sip:additions [
data:      a rdf:Statement ;
data:      rdf:subject :book2 ; rdf:predicate :title ; rdf:object "The Semantic Web"
data:    ] ;
data:    sip:deletions [
data:      a rdf:Statement ;
data:      rdf:subject :book1 ; rdf:predicate :title ; rdf:object "SPARQL Tutorial"
data:    ] .

NOTE: A future version of this specification will adopt the RDF 1.2 terminology of RDF terms and will use RDF 1.2 quoted triples for describing additions and deletions.

2.6.4. up-to-date Event

The up-to-date event payload contains an instance of sip:UpToDate.

event: up-to-date
data: @prefix sip: <http://www.w3.org/ns/sparql-incremental#> .
data: @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
data:
data: [] a sip:UpToDate ;
data:    sip:timestamp "2024-11-24T10:05:00Z"^^xsd:dateTime .

2.6.5. error Event

The error event payload contains an instance of sip:Error.

event: error
data: @prefix sip: <http://www.w3.org/ns/sparql-incremental#> .
data:
data: [] a sip:Error ;
data:    sip:status 500 ;
data:    sip:statusText "Internal Server Error: Query execution timeout" .