Where ShExC, ShExJ and ShExR disagree

Deltas between the three renderings of a ShEx schema, with the evidence for each. Written up because the abstract syntax they are all supposed to share is not quite the same in each.

ShEx schemas exist in three syntaxes, and each has its own grammar:

ShExJ is the pivot: ShExC parses to it, and ShExJ plus doc/ShExJ-context.jsonld is ShExR — which is how bin/mkShExR.js derives ShExR.ttl and ShExR.ntriples. So a delta between ShExC and ShExR is a place where that round trip is not total.

1. ShExR lets start be a ShapeDecl; nothing else does

The three grammars say:

start ::= "start" '=' inlineShapeExpression          ShExC

start:shapeExprOrRef ?                             ShExJ
shapeExprOrRef = shapeExpr | shapeDeclRef ;
shapeExpr      = ShapeOr | ShapeAnd | ShapeNot
               | NodeConstraint | Shape | ShapeExternal ;

sx:start @<#shapeDeclOrExpr> ? ;                    ShExR
<#shapeDeclOrExpr> @<#ShapeDecl> OR @<#shapeExpr>

Since ShEx 2.1 moved id onto ShapeDecl, a ShExJ shapeExpr cannot itself be a ShapeDecl. ShExR's shapeDeclOrExpr says it can. Running the same four start forms through all three grammars:

Why ShExR has to be this way

RDF cannot distinguish reference from containment. In ShExJ a reference is a bare string:

{ "type": "Schema", "start": "http://ex/S",
  "shapes": [ { "type": "ShapeDecl", "id": "http://ex/S", "shapeExpr": {…} } ] }

structurally unlike an inline object. In RDF there is no such distinction — the arc is the thing:

<http://ex/sch> a sx:Schema ; sx:start <http://ex/S> ; sx:shapes ( <http://ex/S> ) .
<http://ex/S>   a sx:ShapeDecl ; sx:shapeExpr [ a sx:Shape ] .

So ShExR must let sx:start reach a ShapeDecl, or it could not encode start = @<S> at all. The cost is that it also admits an anonymous inline ShapeDecl, which no ShExC document can write and ShExJ's own grammar rejects.

The obvious tightening does not work either. You might add AND IRI to the ShapeDecl branch, forcing references to be IRIs — but shapeDeclLabel = IRIREF | BNODE, so blank-node-labelled ShapeDecls are legal, and a reference to one is a blank-node arc, indistinguishable from inlining it. The ambiguity is irreducible unless ShExJ drops BNODE from shapeDeclLabel.

Not confined to start

The same shapeDeclOrExpr sits under every slot where ShExJ writes shapeExprOrRef:

start is just where it is easiest to see.

2. An abstract start slips past the abstract-shape requirement

ShExR accepts sx:start [ a sx:ShapeDecl ; sx:abstract true ; … ]. The specification's abstract-shape requirement is worded over shapeExprRefs:

Every shapeExprRef referer MUST identify at least one non-abstract shape.

An inline abstract ShapeDecl at start is not a ref, so the requirement's wording does not reach it — leaving a start shape that nothing can satisfy. This is the one delta with semantic teeth rather than just surplus expressivity.

3. inlineShapeExpression restricts start only in ShExC

ShExC's start takes an inline shape expression. Comparing the inline and general forms, the only difference is the tail:

shapeDefinition       ::= (includeSet | extraPropertySet | "CLOSED")* '{' tripleExpression? '}' annotation* semanticActions
inlineShapeDefinition ::= (includeSet | extraPropertySet | "CLOSED")* '{' tripleExpression? '}'

So an inline shape definition may not carry annotations or semantic actions, and per the grammar this is ill-formed:

start = { <p> . } %<a>%          # annotation/semAct on an inlineShapeDefinition

while parenthesising reaches the general production through inlineShapeAtom's '(' shapeExpression ')' branch:

start = ( { <p> . } %<a>% )      # well-formed

Neither ShExJ nor ShExR has any such restriction — sx:start reaches <#Shape>, which permits sx:semActs and sx:annotation unconditionally. The restriction is purely syntactic and has no counterpart in the abstract syntax, so it is an obligation on writers: a ShExJ→ShExC serialiser must parenthesise a start carrying annotations or semantic actions, or it emits invalid ShExC.

Implementation note

shex.js accepts the bare form above and attaches semActs to the start Shape, so its parser is more permissive than the published grammar here. That is an implementation/grammar divergence rather than a syntax/syntax one, but it means the bare form round-trips in practice and so is likely to appear in the wild.

Reproducing

ShExC and ShExJ, from a shex.js checkout:

node -e "const p=require('./packages/shex-parser');
  console.log(JSON.stringify(p.construct('http://ex/',{}).parse('<S> {} start = @<S>').start))"
# "http://ex/S"  -- a shapeDeclRef, not an object

npx json-grammar doc/ShExJ.jsg start-shapedecl.json
# rejects { "type":"Schema", "start":{ "type":"ShapeDecl", … } }

ShExR, validating a candidate graph against doc/ShExR.shex:

const shexr = ShExParser.construct('http://www.w3.org/ns/shex',{},{index:true})
                        .parse(fs.readFileSync('doc/ShExR.shex','utf8'));
new ShExValidator(shexr, rdfjsDB(graph), {})
  .validateShapeMap([{node:'http://ex/sch', shape:'http://www.w3.org/ns/shex#Schema'}]);

PREFIX sx: <http://www.w3.org/ns/shex#>
<http://ex/sch> a sx:Schema ;
  sx:start [ a sx:ShapeDecl ; sx:abstract true ; sx:shapeExpr [ a sx:Shape ] ] .
# conformant -- though no ShExC or ShExJ document can denote this

Open question

What ShExUtil.ShExRtoShExJ does with an anonymous ShapeDecl at start is not established here: the loader failed earlier (Cannot read properties of undefined (reading 'validator')) before reaching the conversion. If it emits start: {type: "ShapeDecl", …}, that is a ShExJ document its own grammar rejects, which would turn this from a specification gap into a reachable bug.