Skip to content

Latest commit

 

History

History
245 lines (192 loc) · 10.2 KB

custom-tests.md

File metadata and controls

245 lines (192 loc) · 10.2 KB

Custom tests

Any ontology development project can have specific requirements that olivaw default tests cannot cover.

With olivaw it is possible to add custom tests that can then be added to the model tests or the data tests.

Once a test is launched, olivaw will check the custom tests and add them to their related test suite. These tests will then be treated as any other test from the same suite.

These tests will be launched also during a GitHub Actions test trigger or during a pre-commit test trigger.

These tests can also be skipped using all the available test skipping features or be considered as blocking error using the .acimov/parameters.json, check the olivaw parameters file documentation.

How to add a custom test

Any custom test is represented as a .shacl file and put in the .acimov/custom-tests/model or .acimov/custom-tests/data depending if it should be considered as a data test or a custom test.

This turtle file will at least use the following prefixes:

@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix earl: <http://www.w3.org/ns/earl#> .
@prefix dcterms: <http://purl.org/dc/terms/> .

This file should also match with the following criterias:

  • the turtle file must be syntaxically valid
  • this file should provide one and only one entity of type earl:TestCriterion (see the test documentation) with the following characteristics:
    • It must be declared using a relative path URIs and the @base keyword must stay unused
    • It must have one a only one dcterms:title property pointing to a string
    • It must have one a only one dcterms:description property pointing to a string
    • It must have one a only one dcterms:identifier property pointing to a string
    • The literal pointed by the dcterms:identifier property must be composed of lowercase letter groups, separated with dashes
      • Said another way it should conform to the following python regex: ^([a-z]+(-[a-z]+)*){1}$
    • The literal pointed by the dcterms:identifier property must not be identical to any other test criterion identifier

Some examples of custom tests will be provided in the olivaw custom tests examples section so that all of these criterias can be understood.

Once a model test or a data test is launched, olivaw will scan the .acimov/custom-tests/model and .acimov/custom-tests/data folder for any files that match all of these criterias.

For each file that does, a message will be outputed to tell that the given file has been integrated to the suite.

If a custom test fails to match all of the criterias, a message will be outputed to explain why a given test was not integrated.

Once a test is integrated, the assertions related to this test will mention the provided earl:TestCriterion. The criterion will also use the earl:TestCriterion identifier and a generic message will be generated for the report.

The pointers of these errors will always be:

  • The ShaCL shape that caused the error
  • The ShaCL violation node that was generated by the ShaCL engine
  • The definition of the focus node that was caught by the violation node

Non-standard ShaCL features

In order to develop a proper custom test, it is important to know the limitations of ShaCL and keep in mind some workarounds that are told in this section.

There are 2 limitations related to the use of ShaCL:

  • ShaCL expressivity limitations
  • ShaCL targetting limitations

About the expressivity problem:

It is not possible to express some constraints that can be expressed in SPARQL (example: express the detection of cycle of rdfs:subClassOf properties)

About the targetting problem:

ShaCL has a limited targetting due to its basic use case, which was the validation of a dataset given a vocabulary that is already known. in our case, the terms that are going to be implemented during the development process to come are not known in advance.

In order to tackle these problems, it is possible to take advantage of the engine runnning these tests, CORESE use all the non standard features that are documented in the CORESE SHACL documentation.

Here there are 2 features that solve these issues.

  • The implementation of Corese version of the non standard ShaCL-SPARQL which can add all the expressivity that SPARQL can have
  • The features concerning the triple targetting and the path extensions, allowing to select any triple and explore the graph to search for the exact sought shapes without knowing precisely the URIs of nodes or classes required to target.

A last feature is also available in order to help the developper.

If the ontology namespace is needed for a ShaCL-SPARQL request, it can be used without any hard code using the variable "$ontology_namespace". When loading a custom test containing a SPARQL request, the clause "VALUES ($ontology_namespace) { ("...") }" will dynamically be added to the SPARQL requests.

Custom tests examples

Here are some examples that can be directly copy/pasted to an acimov development project, or used as inspiration in order to make another custom test.

Custom model tests

Here is a test that will detect and forbid any cycle of rdfs:subClassOf properties.

@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix earl: <http://www.w3.org/ns/earl#> .
@prefix dcterms: <http://purl.org/dc/terms/> .

<#criterion> a earl:TestCriterion ;
    dcterms:identifier "subclass-cycle" ;
    dcterms:title "Subclass cycle test" ;
    dcterms:description "A test meant to detect rdfs:subClassOf cycles" .

<#shape> a sh:NodeShape ;
  sh:message "Some classes make cycles of rdfs:subClassOf" ;
	sh:targetSubjectsOf rdfs:isDefinedBy ;
	sh:sparql [
		sh:select """
select ?this where {
  ?this rdfs:subClassOf/rdfs:subClassOf+ ?this
}
        """ ;
        sh:severity sh:Violation ;
        sh:maxCount 0
	] .

Here is another example that will detect if any term declared in the ontology have one and only one rdfs:comment property, and test if this comment is set in English, contains to line break and finishes with a dot.

@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix earl: <http://www.w3.org/ns/earl#> .
@prefix dcterms: <http://purl.org/dc/terms/> .

<#criterion> a earl:TestCriterion ;
    dcterms:identifier "comment-format" ;
    dcterms:title "Comment format test" ;
    dcterms:description "A test meant to test a comment format" .

<#shape> a sh:NodeShape ;
	sh:targetSubjectsOf rdfs:isDefinedBy ;
	sh:property [
		sh:message "Ontology term should have one and only one rdfs:comment" ;
		sh:severity sh:Violation ;
		sh:path rdfs:comment ;
		sh:maxCount 1 ;
		sh:minCount 1
	] , [
		sh:message "Comment not in @en/without line break/ending with full stop" ;
		sh:severity sh:Warning ;
		sh:path rdfs:comment ;
		sh:languageIn ( "en" ) ;
		sh:pattern "^[^\\n]+\\.$"
	] .

Finally, here a test detecting if any term defined in the ontology has at least a property pointing to another term from the ontology.

@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix earl: <http://www.w3.org/ns/earl#> .
@prefix dcterms: <http://purl.org/dc/terms/> .

<#criterion> a earl:TestCriterion ;
    dcterms:identifier "linked-schema" ;
    dcterms:title "Linked schema test" ;
    dcterms:description "A test meant to detect isolated ontology terms" .

<#shape> a sh:NodeShape ;
  sh:message "Isolated ontology term from the others" ;
	sh:targetSubjectsOf rdfs:isDefinedBy ;
	sh:sparql [
		sh:select """
select ?this where {
  {
    ?this rdfs:isDefinedBy ?m ;
    ?p ?o .
  } union {
    ?this rdfs:isDefinedBy ?m .
    ?o ?p ?this .
  }
  filter(?o != ?m)
  filter(strstarts(?o, $ontology_namespace))
}
        """ ;
        sh:severity sh:Warning ;
        sh:minCount 1 ;
	] .

Custom data tests

Here is a test detecting if any named node in a data fragment has at least a rdf:type property.

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix earl: <http://www.w3.org/ns/earl#> .
@prefix dcterms: <http://purl.org/dc/terms/> .

<#criterion> a earl:TestCriterion ;
    dcterms:identifier "node-with-class" ;
    dcterms:title "Classed node test" ;
    dcterms:description "A test meant to test if each node has a rdf:type property" .

<#shape> a sh:NodeShape ;
    sh:message "Any ABox node should have a rdf:type property or be a blank node" ;
    sh:severity sh:Warning ;
	sh:targetTriplesOf [] ;
	sh:or (
        [
            sh:path (
                [sh:nodePath sh:subject]
                [sh:triplePath (sh:subject rdf:type)]
            ) ;
            sh:minCount 1
        ]
        [
            sh:path (
                [sh:nodePath sh:subject]
                [sh:filter ([sh:nodeKind sh:BlankNode])]
            ) ;
            sh:minCount 1
        ]
    ) .

Here is a last custom test checking if any subject in the graph has at least a property that is not rdf:type.

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix earl: <http://www.w3.org/ns/earl#> .
@prefix dcterms: <http://purl.org/dc/terms/> .

<#criterion> a earl:TestCriterion ;
    dcterms:identifier "data-richness" ;
    dcterms:title "The vocabulary is linked to by other vocabularies" ;
    dcterms:description "Each node should have predicates other than rdf:type" .

<#shape> a sh:NodeShape ;
    sh:message "Some subjects only have a rdf:type property" ;
    sh:severity sh:Warning ;
	sh:targetSubjectsOf rdf:type ;
	sh:sparql [
        sh:select """
select ?this where {
  ?this ?p ?o .
   filter (?p != rdf:type)
}
        """ ;
        sh:minCount 1
    ] .