Architecture

Definition

A controlled vocabulary (or simply vocabulary) is an organized collection of terms (i.e. names, codes, etc.) under the control of a governing authority. They can be simple — name authority files, code lists, synonym rings — or fairly complex — taxonomies and thesauri --, but at the core all of them aim to organize knowledge described in natural language.

Terms from controlled vocabularies are commonly used by reference as enumeration values in information systems which need to restrict what values are permissible. This has several benefits, such as a standardized way to reuse terms and their semantic descriptions, and allowing non-technical domain experts to contribute to vocabularies and not have to be familiar with programming.

Identification

Rationale
  • Hash IRIs are used because they reflect that terms cannot exist without an authoritative vocabulary in which they are described.[1] Furthermore they allow for a trivial — and frankly, desirable — implementation of dereferenceable IRIs, namely by resolving to the web documenation representation of the vocabulary.

  • Regarding the vocabulary and term names, although we do recognize the maintenance benefits of using opaque names, for the benefit of developers we have opted to go for meaningful names.

  • No version numbers are encoded into the IRIs, since we choose to evolve vocabularies.

Each controlled vocabulary is identified by its IRI, and so are each of the terms contained in them:

Table 1. IRI templates

Vocabulary

https://modellen.netbeheernederland.nl/controlled-vocabularies/{vocab}

Representation

Web documentation (HTML)

https://modellen.netbeheernederland.nl/controlled-vocabularies/{vocab}/\/index.html

SKOS (Turtle)

https://modellen.netbeheernederland.nl/controlled-vocabularies/{vocab}.skos.ttl

SHACL (Turtle)

https://modellen.netbeheernederland.nl/controlled-vocabularies/{vocab}.shacl.ttl

Term

https://modellen.netbeheernederland.nl/controlled-vocabularies/{vocab}#{term}

where:

{vocab}

vocabulary name (non-opaque in "kebab-case")

{term}

term name (non-opaque)

Life cycle management

Rationale

Maintenance is greatly simplified if version numbers can be avoided:

  • no redirection difficulties (and hacks in Antora)

  • trivial IRI dereferencing

  • simpler identities to reduce cognitive load

  • simpler to understand and maintain build logic

Not using version numbers for vocabularies could cause confusing situations when data models which do have versions use them. Let me explain by means of an hypothetical situation where Team Semantiek models the EQ profile, based on which Liander produces data and EDSN ingests it.

Suppose both EDSN and Liander have implemented their applications based on EQ 2.0. In the SHACL, the node shape which enumerates all values in the NBNL controlled vocabulary Base voltages is imported, and it is used to enforce the use of only those base voltages across the graph. All is well so far.

Then, Team Semantiek releases EQ 3.0, which makes use of a new base voltage value which is added to the Base voltages vocabulary. Liander is ahead of the curve and has quickly implemented this version. EDSN then starts receiving datasets based on this new 3.0 version, and processes them with their EQ 2.0 based stack, starting with a SHACL validation. Interestingly, because the SHACL imports the Base voltages SHACL, it will have knowledge of the newly added base voltage, and therefore deem the data valid.

It is then, however, that things might break. Once the data is parsed into tables or objects, these are all still based on EQ 2.0, and therefore the new enumeration value for base voltage will not be expected. Depending on the technology and implementation, this is likely to cause an error.

This is the only situation I’ve been able to come up with that could potentially cause issues, but frankly, I don’t think it warrants introducing version numbers for our vocabularies at all:

  1. The main problem is that EDSN should never process datasets based on a different version than the one being agreed upon. This is a communication problem first and foremost, and second, the application should check the specified version number to decide whether it can process it or not.

  2. For that matter, the error is actually useful in signalling that something went wrong in communication. For comparison, had we worked with version numbers for the vocabularies in this hypothetical, then the EQ 2.0 SHACL would have pointed to a fixed version of the Base voltages vocabulary, causing it to deem the data invalid because of the unexpected new base voltage. So ultimately, your pipeline would fail earlier, which is preferable, but given how unlikely and troubled this scenario already is, and the fact that you will certainly get an error and nothing can be altered or broken in any meaningful sense, all of this does not warrant a strong case for introducing version numbers.

Therefore, we feel comfortable in keeping things simple for our goals of maintainbility and (semantic) stability and choose to evolve our vocabularies and refrain from adopting a version numbering scheme.

No release cycle or version number scheme will be used. Instead, in order to keep things simple, predictable and maintainable, controlled vocabularies will simply evolve [2]. New knowledge can always be added, including the introduction of new terms, but changes which alter the meaning or semantic scope of a term or the vocabulary are prohibited. In such cases a new term must be created with its own definition which may be stated to supersede the old term, and (if applicable) the old term can be deprecated. Terms and their definitions may also never be deleted. This way, any reference to a term or vocabulary will remain valid and accessible indefinitely (FAIR principles F1, A1 and A2).

Each modification to the vocabulary does result in a different version of it, but none of these representations are exposed through its identifying IRI and are merely accessible through the VCS. Effectively, this encodes having a single evolving identity whilst preserving historical lineage of the evolution.

Furthermore, it deserves mention that terms themselves do not have versions. Different versions of a vocabulary might provide different descriptions of terms contained in it, but there is no such thing as a "version of a term".

What changes can and can’t I make?

For guidance, the following table may be referenced to see what actions are allowed or prohibited when evolving the vocabulary. It is not normative nor does it pretend to be exhaustive.

Table 2. Rules for evolving
Allowed Prohibited

Adding knowledge, either about existing terms (e.g. new notes or lifecycle status updates) or including the introduction of new ones

Removing terms, their definition and preferred labels

Removing knowledge that is not definitions and preferred labels (e.g. notes and alignments)

Changing the preferred label of a term

Improving a definition whilst retaining backward compatibility

Changing a definition such that the meaning of term changes incompatibly (e.g. when contradictions arise or the semantic scope has changed completely)

Build process

Rationale

SKOS tooling cannot be used to work on these source files, but the gains of the much easier maintenance easily outweighs that minor deficit given how simple and small these ontologies are.

build process
Figure 1. Build process

Each controlled vocabulary is maintained through a single source file in YAML. The build process converts this to JSON and adds a JSON-LD context, creating a RDF graph. Furthermore, term IRIs are derived from the notations, explicit type statements are added to make the graph a proper SKOS ontology, and the concepts key is inverted into more standard skos:inScheme statements. Finally some additional information is added, such as the dcterms:source of the file.

This way of working keeps maintenance simple for authors, leaving the tedious and error-prone transforming and adding of information to the deterministic build process.


1. Containment in an authoritative vocabulary is a rigid property of terms.
2. In particular this avoids complicated scenarios with term deprecation. Assuming you favor stable term IRIs which mention no version number, what should these dereference to if in the latest version the term is deprecated?