EKGF 1.0 beta

The Data Product (DPROD) Ontology

EKGF 1.0 beta

Specification PDF
Preamble

Abstract§

The concept of Data Products has emerged as organizations increasingly recognize the value of data as an asset to be managed and distributed like any other product. As more companies adopt decentralized data architectures, such as Data Mesh, the need for standardized methods to describe and manage data products consistently across platforms has become critical. This is where the dprod specification, built on W3C Linked Data standards, becomes essential. Without such a standard, organizations face significant challenges: inconsistent metadata across diverse data products, limited discoverability, and interoperability issues that hinder data integration from various sources. As data ecosystems grow, the lack of a common framework also impedes scalability, increases vendor lock-in, and makes it difficult to manage these products effectively.

DPROD offers a solution by providing a clear schema for describing data products, ensuring they are discoverable, interoperable, and treated with the same level of accountability as traditional products.

The W3C tech stack is perfectly suited to address these challenges because it was designed to foster interconnected, decentralized systems, providing a robust framework for creating metadata that is both machine-readable and human-understandable. DPROD enables consistent terminology across different platforms, domains, and organizations, allowing advanced users to semantically enrich their data products and connect them into distributed knowledge graphs.

As more organizations strive to build and scale data products, DPROD provides the standardization needed to ensure interoperability and unlock the full potential of decentralized data ecosystems in a controlled and mature way.

Status§

This is a GraphArch rendering of DPROD 1.0 beta from the main branch. The date shown is the ontology’s recorded modification date, 31 August 2024; it is not an asserted approval date.

1. Scope§

The Data Product (DPROD) specification is a profile of the Data Catalog (DCAT) Vocabulary, designed to describe Data Products. This document defines the schema and provides examples of its use.

DPROD extends DCAT to enable publishers to describe Data Products and data services in a decentralized way. By using a standard model and ontology, DPROD facilitates the consumption and aggregation of metadata from multiple Data Marketplaces. This approach increases the discoverability of products and services, supports decentralized data publishing, and enables federated search across multiple sites using a uniform query mechanism and structure.

The namespace for DPROD terms is https://www.omg.org/spec/DPROD/dprod/

The suggested prefix for the DPROD namespace is dprod

DPROD follows two basic principles:

  • Decentralize Data Ownership: To make data integration more efficient, tasks should be shared among multiple teams. DCAT helps by offering a standard way to publish datasets in a decentralized manner.
  • Harmonize Data Schemas: Using shared schemas helps unify different data formats. For instance, the DPROD specification provides a common set of rules for defining a Data Product. Users of DPROD can extend this schema as needed.

The dprod specification builds on DCAT vocab-dcat-3 by connecting DCAT Data Services to DPROD Data Products using input and output ports. These ports are used to publish and consume data from a Data Product. DPROD treats ports as DCAT data services, so the data exchanged can be described using DCAT's highly expressive metadata around distributions and datasets. This approach also allows publishers to create their own descriptions for the data they are sharing. They can use a special property called conformsTo from DCAT to link to their own set of rules or guidelines for their data.

The DPROD specification has four main aims:

  • To provide unambiguous and sharable semantics to answer the question: 'What is a data product?'
  • Be simple for anyone to use, but expressive enough to power large data marketplaces
  • Allow organisations to reuse their existing data catalogues and dataset infrastructure
  • To share common semantics across different Data Products and promote harmonization

2. Conformance§

A data product is conformant with this specification if it satisfies the SHACL constraints provided in the file dprod-shapes.ttl.

3. Terms and Definitions§

All terms introduced in this specification are given definitions in the Data Product Model defined later.

4. Symbols§

The following acronyms are used in this specification.

5. Normative namespaces§

Namespaces and prefixes used in normative parts of this Profile are shown in the following table.

PrefixNamespace IRISource
dprodhttps://www.omg.org/spec/DPROD/dprod/dprod
dcathttp://www.w3.org/ns/dcat#vocab-dcat-3
dcthttp://purl.org/dc/terms/dcterms
odrlhttp://www.w3.org/ns/odrl/2/odrl-model
owlhttp://www.w3.org/2002/07/owl#owl2-quick-reference
provhttp://www.w3.org/ns/prov#prov-overview
rdfhttp://www.w3.org/1999/02/22-rdf-syntax-ns#rdf11-primer
rdfshttp://www.w3.org/2000/01/rdf-schema#rdf-schema
shhttp://www.w3.org/ns/shacl#shacl
xsdhttp://www.w3.org/2001/XMLSchema#xmlschema-2

6. Data Product (DPROD) Model§

Data Mesh Architectures Data Mesh use input and output ports to manage how data enters and leaves a Data Product. These ports can handle different formats, schemas, and protocols. Input ports bring in data, while output ports send data to other Data Products for aggregation, reuse, analysis or reporting, etc.

In the vocab-dcat-3 framework, a Data Service is a way to describe services that provide access to data. Data Services give standardized, machine-readable descriptions of how to access one or more datasets or data processing functions.

Data Services specify how to access and download the data. In DPROD, Data Services are connected to Distributions by a property called isAccessServiceOf; on the Distribution one can specify formats (like CSV or JSON etc.) and provide metadata about the "physical model" of the data. Distributions link to Datasets and DCAT has a very rich vocabulary for describing every aspect of a dataset. Finally, Datasets use the conformsTo property to link to the "logical model" where one can specify one's own rich semantic metadata.

By linking Data Product ports to DCAT DataServices (vocab-dcat-3), DPROD can describe Data Products in a way that machines can read across the organization. This makes it easier for data teams to build and manage their own data products independently, while still working well with the rest of the organization's data.

Using standards like DCAT helps create a strong and clear way to define Data Products. It ensures that as data becomes more complex, the methods for describing, sharing, and using data stay consistent and reliable. It also allows different organizations to share data securely and in a standardized way.

Information model for the Profile

Overview of the DPROD model and its relationship with DCAT classes

The Profile consists of the following classes:

  • Data Mesh (dcat:Catalog) - The collection of Data Products
  • Data Product (dprod:DataProduct) - A data product may have input and output ports, code and metadata
  • Port (dcat:DataService) - A digital interface that provides access to a Dataset. This can be an HTTP URL, a Database or a FileShare, etc.
  • Distribution (dcat:Distribution) - A specific representation of a dataset (CSV, JSON, ADLS etc.) which can conform to a physical model
  • Dataset (dcat:Dataset) - A collection of related data that can conform to a logical model

As DCAT Data Services, the DPROD input and output ports can specify connection details, they have distributions that define formats, and link to datasets that conform to shared schemas. In this example, the UK Bonds Data Product includes an output port, which is a RESTful API. This API delivers JSON data conforming to the shared FIBO specification for callable bonds.

{
    "@context": "https://ekgf.org/dprod/spec/archive/1.0/dprod.jsonld",
    "id": "https://y.com/products/uk-bonds",
    "type": "DataProduct",
    "title": "UK Bonds",
    "description": "UK Bonds is your one-stop-shop for all your bonds!",
    "dataProductOwner": "https://www.linkedin.com/in/tonyseale/",
    "lifecycleStatus" : "https://www.omg.org/spec/DPROD/data/lifecycle-status/Consume",
    "outputPort": {
      "type": "DataService",
      "endpointURL": "https://y.com/uk-10-year-bonds",
      "isAccessServiceOf": {
        "type": "Distribution",
        "format": "https://www.iana.org/assignments/media-types/application/json",
        "isDistributionOf": {
          "type": "Dataset",
          "id": "https://y.com/products/uk-bonds/datasets/10-year",
          "conformsTo": "https://spec.edmcouncil.org/fibo/ontology/SEC/Debt/Bonds/CallableBond"
        }
      }
     }
  }

Note

The examples map the type of the above classes to @type in the JSON-LD serializations. One can use JSON-LD to extend the familiar JSON syntax with the shared semantics defined by DCAT and DPROD.

The JSON above can be pasted into https://json-ld.org/playground. One can see that the schema resolves.

Note

The following sections are driven by the Shapes definitions for DPROD, which represent the properties expected to be used for instances of the above classes. As such, they include properties defined by the DCAT specification which DPROD extends. The prefix in the identifier for each entry indicates the ontology defining the property.

7. Reference§

The following sections document the vocabulary and SHACL shapes collected from this same source revision. The authoritative source files remain unchanged.

7.1. OWL classes§

7.1.1 dprod:DataProduct§

A rational, managed, and governed collection of data, with purpose, value and ownership, meeting consumer needs over a planned life-cycle.

A data product may have input and output ports, code and metadata

dprod:DataProduct
https://www.omg.org/spec/DPROD/dprod/DataProduct
RDF predicateValue
rdf:typerdfs:Class
rdf:typeowl:Class
rdfs:labeldata product
rdfs:subClassOfdcat:Resource
UML class diagram for data product. Diagram labels: «Resource»; dcat:; DataService; «Resource»; dcat:; Dataset; «Resource»; dcat:; Resource; «Resource»; odrl:; Policy; «Resource»; prov:; Agent; «owl:Class»; data product; dct:description : xsd:string; rdfs:label : xsd:string; domain; purpose : xsd:string; «owl:Class»; data product lifecycle; status; data product owner; input dataset; input port; odrl:; hasPolicy; lifecycle status; output dataset; output port; data product rdfs:subClassOf dcat:Resource. data product owner rdfs:domain data product. data product owner rdfs:range prov:Agent. domain rdfs:domain data product. input dataset rdfs:domain data product. input dataset rdfs:range dcat:Dataset. input port rdfs:domain data product. input port rdfs:range dcat:DataService.«Resource»dcat:DataService«Resource»dcat:Dataset«Resource»dcat:Resource«Resource»odrl:Policy«Resource»prov:Agent«owl:Class»data productdct:description : xsd:stringrdfs:label : xsd:stringdomainpurpose : xsd:string«owl:Class»data product lifecyclestatusdata product ownerinput datasetinput portodrl:hasPolicylifecycle statusoutput datasetoutput port
Class neighborhood: data product [1]

7.1.2 dprod:DataProductLifecycleStatus§

The development status of the data product taken from a controlled list (e.g. Ideation, Design, Build, Deploy, Consume).

The lifecycle of the data product as defined by EDM Council CDMC

dprod:DataProductLifecycleStatus
https://www.omg.org/spec/DPROD/dprod/DataProductLifecycleStatus
RDF predicateValue
rdf:typerdfs:Class
rdf:typeowl:Class
rdfs:labeldata product lifecycle status
rdfs:seeAlsohttps://edmcouncil.org/wp-content/uploads/2024/03/EDMWebinar_DPSIG_04-30-Presentation.pdf#page=7
rdfs:subClassOfdprod:Enumeration
UML class diagram for data product lifecycle status. Diagram labels: «Resource»; dcat:; DataService; «Resource»; dcat:; Dataset; «Resource»; odrl:; Policy; «Resource»; prov:; Agent; «owl:Class»; data product; dct:description : xsd:string; rdfs:label : xsd:string; domain; purpose : xsd:string; «owl:Class»; data product lifecycle; status; «owl:Class»; enumeration; lifecycle status; odrl:; hasPolicy; data product owner; input dataset; input port; output dataset; output port; data product lifecycle status rdfs:subClassOf enumeration. lifecycle status rdfs:domain data product. lifecycle status rdfs:range data product lifecycle status.«Resource»dcat:DataService«Resource»dcat:Dataset«Resource»odrl:Policy«Resource»prov:Agent«owl:Class»data productdct:description : xsd:stringrdfs:label : xsd:stringdomainpurpose : xsd:string«owl:Class»data product lifecyclestatus«owl:Class»enumerationlifecycle statusodrl:hasPolicydata product ownerinput datasetinput portoutput datasetoutput port
Class neighborhood: data product lifecycle status [1]

7.1.3 dprod:Enumeration§

The superclass of enumeration lists referenced from data product related artifacts.

dprod:Enumeration
https://www.omg.org/spec/DPROD/dprod/Enumeration
RDF predicateValue
rdf:typerdfs:Class
rdf:typeowl:Class
rdfs:labelenumeration
UML class diagram for enumeration. Diagram labels: «owl:Class»; data product lifecycle; status; «owl:Class»; enumeration; data product lifecycle status rdfs:subClassOf enumeration.«owl:Class»data product lifecyclestatus«owl:Class»enumeration
Class neighborhood: enumeration [1]

7.1.4 dprod:InformationSensitivityClassification§

A classification of the information within a dataset to indicate the level of control and protection that must be applied.

dprod:InformationSensitivityClassification
https://www.omg.org/spec/DPROD/dprod/InformationSensitivityClassification
RDF predicateValue
rdf:typerdfs:Class
rdf:typeowl:Class
rdfs:labelinformation sensitivity classification
rdfs:seeAlsohttps://spec.edmcouncil.org/cdmc/ontology/Classification/InformationSensitivity
UML class diagram for information sensitivity classification. Diagram labels: «Resource»; dcat:; Dataset; «owl:Class»; information sensitivity; classification; information sensitivity; classification; information sensitivity classification rdfs:domain dcat:Dataset. information sensitivity classification rdfs:range information sensitivity classification.«Resource»dcat:Dataset«owl:Class»information sensitivityclassificationinformation sensitivityclassification
Class neighborhood: information sensitivity classification [1]

7.1.5 dprod:Protocol§

A detailed specification, possibly including a specific version, for how to communicate with a service.

dprod:Protocol
https://www.omg.org/spec/DPROD/dprod/Protocol
RDF predicateValue
rdf:typerdfs:Class
rdf:typeowl:Class
rdfs:labelprotocol
UML class diagram for protocol. Diagram labels: «owl:Class»; protocol; «owl:Class»protocol
Class neighborhood: protocol [1]

7.1.6 dprod:SecuritySchemaType§

A classification encompassing a set of rules used for authentication and communication.

dprod:SecuritySchemaType
https://www.omg.org/spec/DPROD/dprod/SecuritySchemaType
RDF predicateValue
rdf:typerdfs:Class
rdf:typeowl:Class
rdfs:labelsecurity schema type
UML class diagram for security schema type. Diagram labels: «owl:Class»; security schema type; «owl:Class»security schema type
Class neighborhood: security schema type [1]

7.2. Object properties§

7.2.1 dprod:dataProductOwner§

The agent that is accountable overall for the data product, including managing it through its lifecycle.

dprod:dataProductOwner
https://www.omg.org/spec/DPROD/dprod/dataProductOwner
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindprod:DataProduct
rdfs:labeldata product owner
rdfs:rangeprov:Agent

7.2.2 dprod:domain§

The business or information area supported by the data product.

The domain is intended to be a resource in its own right. This specification does not constrain the class to be used.

dprod:domain
https://www.omg.org/spec/DPROD/dprod/domain
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindprod:DataProduct
rdfs:labeldomain

7.2.3 dprod:informationSensitivityClassification§

More granular classification that indicates the level of control and protection that must be applied to the asset due to the nature of the data and its sensitivity or importance to the organization.

dprod:informationSensitivityClassification
https://www.omg.org/spec/DPROD/dprod/informationSensitivityClassification
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindcat:Dataset
rdfs:labelinformation sensitivity classification
rdfs:rangedprod:InformationSensitivityClassification

7.2.4 dprod:inputDataset§

The source data made available to the data product through input data services. Depending on the lifecycle of the data product, this may be a stated or inferred relationship aligned with the input ports.

dprod:inputDataset
https://www.omg.org/spec/DPROD/dprod/inputDataset
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindprod:DataProduct
rdfs:labelinput dataset
rdfs:rangedcat:Dataset

7.2.5 dprod:inputPort§

A set of services exposed by a data product to collect its source data and makes it available for further internal transformation. An input port can receive data from one or more upstream sources in a push (i.e. asynchronous subscription) or pop mode (i.e. synchronous query). Each data product may have one or more input ports.

dprod:inputPort
https://www.omg.org/spec/DPROD/dprod/inputPort
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindprod:DataProduct
rdfs:labelinput port
rdfs:rangedcat:DataService

7.2.6 dprod:isAccessServiceOf§

The dataset distribution that is being offered through this data service.

dprod:isAccessServiceOf
https://www.omg.org/spec/DPROD/dprod/isAccessServiceOf
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindcat:DataService
rdfs:labelis access service of
rdfs:rangedcat:Distribution
owl:inverseOfdcat:accessService

7.2.7 dprod:isDistributionOf§

The dataset that this distribution makes available.

dprod:isDistributionOf
https://www.omg.org/spec/DPROD/dprod/isDistributionOf
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindcat:Distribution
rdfs:labelis distribution of
owl:inverseOfdcat:distribution

7.2.8 dprod:lifecycleStatus§

The development status of the data product.

dprod:lifecycleStatus
https://www.omg.org/spec/DPROD/dprod/lifecycleStatus
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindprod:DataProduct
rdfs:labellifecycle status
rdfs:rangedprod:DataProductLifecycleStatus

7.2.9 dprod:outputDataset§

The data that is exposed by the data product through output data services in a way that can be understood and trusted. Depending on the lifecycle of the data product, this may be a stated or inferred relationship aligned with the output ports.

dprod:outputDataset
https://www.omg.org/spec/DPROD/dprod/outputDataset
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindprod:DataProduct
rdfs:labeloutput dataset
rdfs:rangedcat:Dataset

7.2.10 dprod:outputPort§

A set of services exposed by a data product to share the generated data in a way that can be understood and trusted.

dprod:outputPort
https://www.omg.org/spec/DPROD/dprod/outputPort
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindprod:DataProduct
rdfs:labeloutput port
rdfs:rangedcat:DataService

7.2.11 dprod:protocol§

A protocol (possibly one of many options) used to communicate with this data service.

dprod:protocol
https://www.omg.org/spec/DPROD/dprod/protocol
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindcat:DataService
rdfs:labelprotocol

7.2.12 dprod:securitySchemaType§

The security schema type used for authentication and communication with this Data Service.

Warning

No English or language-neutral label found in the collected sources.

dprod:securitySchemaType
https://www.omg.org/spec/DPROD/dprod/securitySchemaType
RDF predicateValue
rdf:labelsecurity schema type
rdf:typerdf:Property
rdf:typeowl:ObjectProperty
rdfs:domaindcat:DataService

7.3. Datatype properties§

7.3.1 dprod:purpose§

A description of the objectives and intended usage of the data product.

dprod:purpose
https://www.omg.org/spec/DPROD/dprod/purpose
RDF predicateValue
rdf:typerdf:Property
rdf:typeowl:DatatypeProperty
rdfs:domaindprod:DataProduct
rdfs:labelpurpose
rdfs:rangexsd:string

7.4. SHACL node shapes§

7.4.1 dprod-shapes:DataProductLifecycleStatusShape§

The development status of the data product taken from a controlled list (e.g. Ideation, Design, Build, Deploy, Consume).

dprod-shapes:DataProductLifecycleStatusShape
https://www.omg.org/spec/DPROD/shapes/DataProductLifecycleStatusShape
RDF predicateValue
rdf:typesh:NodeShape
rdfs:labeldata product lifecycle status shape
sh:targetClassdprod:DataProductLifecycleStatus
rdfs:isDefinedBydprod-shapes:

7.4.2 dprod-shapes:DataProductShape§

A rational, managed, and governed collection of data, with purpose, value and ownership, meeting consumer needs over a planned life-cycle.

A data product may have input and output ports, code and metadata

dprod-shapes:DataProductShape
https://www.omg.org/spec/DPROD/shapes/DataProductShape
RDF predicateValue
rdf:typesh:NodeShape
rdfs:labeldata product shape
sh:propertydprod-shapes:DataProduct-dataProductOwner
sh:propertydprod-shapes:DataProduct-description
sh:propertydprod-shapes:DataProduct-domain
sh:propertydprod-shapes:DataProduct-hasPolicy
sh:propertydprod-shapes:DataProduct-inputDataset
sh:propertydprod-shapes:DataProduct-inputPort
sh:propertydprod-shapes:DataProduct-label
sh:propertydprod-shapes:DataProduct-lifecycleStatus
sh:propertydprod-shapes:DataProduct-outputDataset
sh:propertydprod-shapes:DataProduct-outputPort
sh:propertydprod-shapes:DataProduct-purpose
sh:targetClassdprod:DataProduct
rdfs:isDefinedBydprod-shapes:

7.4.3 dprod-shapes:DataServiceShape§

A collection of operations that provides access to one or more datasets or data processing functions.

dprod-shapes:DataServiceShape
https://www.omg.org/spec/DPROD/shapes/DataServiceShape
RDF predicateValue
rdf:typesh:NodeShape
rdfs:labeldata service shape
sh:propertydprod-shapes:DataService-endpointDescription
sh:propertydprod-shapes:DataService-endpointURL
sh:propertydprod-shapes:DataService-isAccessServiceOf
sh:propertydprod-shapes:DataService-protocol
sh:propertydprod-shapes:DataService-securitySchemaType
sh:targetClassdcat:DataService
rdfs:isDefinedBydprod-shapes:

7.4.4 dprod-shapes:DatasetShape§

A collection of data, published or curated by a single source, and available for access or download in one or more representations.

dprod-shapes:DatasetShape
https://www.omg.org/spec/DPROD/shapes/DatasetShape
RDF predicateValue
rdf:typesh:NodeShape
rdfs:labeldataset shape
sh:propertydcat:Dataset-informationSensitivityClassification
sh:propertydprod-shapes:Dataset-conformsTo
sh:propertydprod-shapes:Dataset-description
sh:propertydprod-shapes:Dataset-distribution
sh:propertydprod-shapes:Dataset-hasPolicy
sh:propertydprod-shapes:Dataset-label
sh:propertydprod-shapes:Dataset-type
sh:targetClassdcat:Dataset
rdfs:isDefinedBydprod-shapes:

7.4.5 dprod-shapes:DistributionShape§

A specific representation of a dataset. A dataset might be available in multiple serializations that may differ in various ways, including natural language, media-type or format, schematic organization, temporal and spatial resolution, level of detail or profiles (which might specify any or all of the above).

dprod-shapes:DistributionShape
https://www.omg.org/spec/DPROD/shapes/DistributionShape
RDF predicateValue
rdf:typesh:NodeShape
rdfs:labeldistribution shape
sh:propertydprod-shapes:Distribution-accessService
sh:propertydprod-shapes:Distribution-conformsTo
sh:propertydprod-shapes:Distribution-format
sh:propertydprod-shapes:Distribution-isDistributionOf
sh:targetClassdcat:Distribution
rdfs:isDefinedBydprod-shapes:

7.4.6 dprod-shapes:InformationSensitivityClassificationShape§

A classification of the information within a dataset to indicate the level of control and protection that must be applied.

dprod-shapes:InformationSensitivityClassificationShape
https://www.omg.org/spec/DPROD/shapes/InformationSensitivityClassificationShape
RDF predicateValue
rdf:typesh:NodeShape
rdfs:labelinformation sensitivity classification shape
sh:targetClassdprod:InformationSensitivityClassification
rdfs:isDefinedBydprod-shapes:

7.4.7 dprod-shapes:ProtocolShape§

A detailed specification, possibly including a specific version, for how to communicate with a service.

dprod-shapes:ProtocolShape
https://www.omg.org/spec/DPROD/shapes/ProtocolShape
RDF predicateValue
rdf:typesh:NodeShape
rdfs:labelprotocol shape
sh:targetClassdprod:Protocol
rdfs:isDefinedBydprod-shapes:

7.4.8 dprod-shapes:SecuritySchemaTypeShape§

A classification encompassing a set of rules used for authentication and communication.

dprod-shapes:SecuritySchemaTypeShape
https://www.omg.org/spec/DPROD/shapes/SecuritySchemaTypeShape
RDF predicateValue
rdf:typesh:NodeShape
rdfs:labelsecurity schema type shape
sh:targetClassdprod:SecuritySchemaType
rdfs:isDefinedBydprod-shapes:

7.5. SHACL property shapes§

7.5.1 dcat:Dataset-informationSensitivityClassification§

More granular classification that indicates the level of control and protection that must be applied to the asset due to the nature of the data and its sensitivity or importance to the organization.

dcat:Dataset-informationSensitivityClassification
http://www.w3.org/ns/dcat#Dataset-informationSensitivityClassification
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldataset information sensitivity classification shape
sh:classdprod:InformationSensitivityClassification
sh:pathdprod:informationSensitivityClassification
rdfs:isDefinedBydprod-shapes:

7.5.2 dprod-shapes:DataProduct-dataProductOwner§

The agent that is accountable overall for the data product, including managing it through its lifecycle.

dprod-shapes:DataProduct-dataProductOwner
https://www.omg.org/spec/DPROD/shapes/DataProduct-dataProductOwner
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product owner shape
sh:classprov:Agent
sh:pathdprod:dataProductOwner
rdfs:isDefinedBydprod-shapes:

7.5.3 dprod-shapes:DataProduct-description§

A free text description of the data product.

dprod-shapes:DataProduct-description
https://www.omg.org/spec/DPROD/shapes/DataProduct-description
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product description shape
sh:datatypexsd:string
sh:pathdct:description
rdfs:isDefinedBydprod-shapes:

7.5.4 dprod-shapes:DataProduct-domain§

The business or information area supported by the data product.

The target class is left open

dprod-shapes:DataProduct-domain
https://www.omg.org/spec/DPROD/shapes/DataProduct-domain
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product domain shape
sh:pathdprod:domain
rdfs:isDefinedBydprod-shapes:

7.5.5 dprod-shapes:DataProduct-hasPolicy§

An ODRL conformant policy expressing the rights associated with the data product. This is an inferred relationship based on the rights expressed on the individual datasets of the data product.

dprod-shapes:DataProduct-hasPolicy
https://www.omg.org/spec/DPROD/shapes/DataProduct-hasPolicy
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product has policy shape
sh:classodrl:Policy
sh:pathodrl:hasPolicy
rdfs:isDefinedBydprod-shapes:

7.5.6 dprod-shapes:DataProduct-inputDataset§

The source data made available to the data product through input data services. Depending on the lifecycle of the data product, this may be a stated or inferred relationship aligned with the input ports.

dprod-shapes:DataProduct-inputDataset
https://www.omg.org/spec/DPROD/shapes/DataProduct-inputDataset
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product input dataset shape
sh:classdcat:Dataset
sh:pathdprod:inputDataset
rdfs:isDefinedBydprod-shapes:

7.5.7 dprod-shapes:DataProduct-inputPort§

A set of services exposed by a data product to collect its source data and makes it available for further internal transformation. An input port can receive data from one or more upstream sources in a push (i.e. asynchronous subscription) or pop mode (i.e. synchronous query). Each data product may have one or more input ports.

dprod-shapes:DataProduct-inputPort
https://www.omg.org/spec/DPROD/shapes/DataProduct-inputPort
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product input port shape
sh:classdcat:DataService
sh:pathdprod:inputPort
rdfs:isDefinedBydprod-shapes:

7.5.8 dprod-shapes:DataProduct-label§

The name given to the data product.

dprod-shapes:DataProduct-label
https://www.omg.org/spec/DPROD/shapes/DataProduct-label
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product label shape
sh:datatypexsd:string
sh:pathrdfs:label
rdfs:isDefinedBydprod-shapes:

7.5.9 dprod-shapes:DataProduct-lifecycleStatus§

The development status of the data product.

dprod-shapes:DataProduct-lifecycleStatus
https://www.omg.org/spec/DPROD/shapes/DataProduct-lifecycleStatus
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product lifecycle status shape
sh:classdprod:DataProductLifecycleStatus
sh:pathdprod:lifecycleStatus
rdfs:isDefinedBydprod-shapes:

7.5.10 dprod-shapes:DataProduct-outputDataset§

The data that is exposed by the data product through output data services in a way that can be understood and trusted. Depending on the lifecycle of the data product, this may be a stated or inferred relationship aligned with the output ports.

dprod-shapes:DataProduct-outputDataset
https://www.omg.org/spec/DPROD/shapes/DataProduct-outputDataset
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product output dataset shape
sh:classdcat:Dataset
sh:pathdprod:outputDataset
rdfs:isDefinedBydprod-shapes:

7.5.11 dprod-shapes:DataProduct-outputPort§

A set of services exposed by a data product to share the generated data in a way that can be understood and trusted.

dprod-shapes:DataProduct-outputPort
https://www.omg.org/spec/DPROD/shapes/DataProduct-outputPort
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product output port shape
sh:classdcat:DataService
sh:pathdprod:outputPort
rdfs:isDefinedBydprod-shapes:

7.5.12 dprod-shapes:DataProduct-purpose§

A description of the objectives and intended usage of the data product.

dprod-shapes:DataProduct-purpose
https://www.omg.org/spec/DPROD/shapes/DataProduct-purpose
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product purpose shape
sh:datatypexsd:string
sh:pathdprod:purpose
rdfs:isDefinedBydprod-shapes:

7.5.13 dprod-shapes:DataProduct-type§

The nature or genre of the data product.

dprod-shapes:DataProduct-type
https://www.omg.org/spec/DPROD/shapes/DataProduct-type
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata product description shape
sh:pathdct:type
rdfs:isDefinedBydprod-shapes:

7.5.14 dprod-shapes:DataService-endpointDescription§

A description of the services available via the end-points, including their operations, parameters etc.

dprod-shapes:DataService-endpointDescription
https://www.omg.org/spec/DPROD/shapes/DataService-endpointDescription
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata service endpoint description shape
sh:pathdcat:endpointDescription
rdfs:isDefinedBydprod-shapes:

7.5.15 dprod-shapes:DataService-endpointURL§

The root location or primary endpoint of the service.

dprod-shapes:DataService-endpointURL
https://www.omg.org/spec/DPROD/shapes/DataService-endpointURL
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata service endpoint URL shape
sh:pathdcat:endpointURL
rdfs:isDefinedBydprod-shapes:

7.5.16 dprod-shapes:DataService-isAccessServiceOf§

The dataset distribution that is being offered through this data service.

dprod-shapes:DataService-isAccessServiceOf
https://www.omg.org/spec/DPROD/shapes/DataService-isAccessServiceOf
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata service is access service of shape
sh:classdcat:Distribution
sh:pathdprod:isAccessServiceOf
rdfs:isDefinedBydprod-shapes:

7.5.17 dprod-shapes:DataService-protocol§

A protocol (possibly one of many options) used to communicate with this data service.

dprod-shapes:DataService-protocol
https://www.omg.org/spec/DPROD/shapes/DataService-protocol
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata service protocol shape
sh:classdcat:Protocol
sh:pathdprod:protocol
rdfs:isDefinedBydprod-shapes:

7.5.18 dprod-shapes:DataService-securitySchemaType§

The security schema type used for authentication and communication with this Data Service.

dprod-shapes:DataService-securitySchemaType
https://www.omg.org/spec/DPROD/shapes/DataService-securitySchemaType
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldata service security schema type shape
sh:classdcat:SecuritySchemaType
sh:pathdprod:securitySchemaType
rdfs:isDefinedBydprod-shapes:

7.5.19 dprod-shapes:Dataset-conformsTo§

A model, schema, ontology, view or profile that the dataset conforms to.

dprod-shapes:Dataset-conformsTo
https://www.omg.org/spec/DPROD/shapes/Dataset-conformsTo
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldataset conforms to shape
sh:pathdct:conformsTo
rdfs:isDefinedBydprod-shapes:

7.5.20 dprod-shapes:Dataset-description§

Free text description of the dataset.

dprod-shapes:Dataset-description
https://www.omg.org/spec/DPROD/shapes/Dataset-description
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldataset distribution shape
sh:datatypexsd:string
sh:pathdct:description
rdfs:isDefinedBydprod-shapes:

7.5.21 dprod-shapes:Dataset-distribution§

An available distribution of the dataset.

dprod-shapes:Dataset-distribution
https://www.omg.org/spec/DPROD/shapes/Dataset-distribution
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldataset distribution shape
sh:classdcat:Distribution
sh:pathdcat:distribution
rdfs:isDefinedBydprod-shapes:

7.5.22 dprod-shapes:Dataset-hasPolicy§

An ODRL conformant policy expressing the rights associated with the resource.

dprod-shapes:Dataset-hasPolicy
https://www.omg.org/spec/DPROD/shapes/Dataset-hasPolicy
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldataset has policy shape
sh:pathodrl:hasPolicy
rdfs:isDefinedBydprod-shapes:

7.5.23 dprod-shapes:Dataset-label§

The name given to the dataset

dprod-shapes:Dataset-label
https://www.omg.org/spec/DPROD/shapes/Dataset-label
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldataset label shape
sh:datatypexsd:string
sh:pathrdfs:label
rdfs:isDefinedBydprod-shapes:

7.5.24 dprod-shapes:Dataset-type§

The type or genre of the dataset.

dprod-shapes:Dataset-type
https://www.omg.org/spec/DPROD/shapes/Dataset-type
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldataset type shape
sh:pathdct:type
rdfs:isDefinedBydprod-shapes:

7.5.25 dprod-shapes:Distribution-accessService§

A data service that gives access to the distribution of the dataset.

dprod-shapes:Distribution-accessService
https://www.omg.org/spec/DPROD/shapes/Distribution-accessService
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldistribution access service shape
sh:classdcat:DataService
sh:pathdcat:accessService
rdfs:isDefinedBydprod-shapes:

7.5.26 dprod-shapes:Distribution-conformsTo§

The schema that the distribution conforms to that is format and technology dependent.

dprod-shapes:Distribution-conformsTo
https://www.omg.org/spec/DPROD/shapes/Distribution-conformsTo
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldistribution conforms to shape
sh:pathdct:conformsTo
rdfs:isDefinedBydprod-shapes:

7.5.27 dprod-shapes:Distribution-format§

The file format of the distribution.

dprod-shapes:Distribution-format
https://www.omg.org/spec/DPROD/shapes/Distribution-format
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldistribution format shape
sh:pathdct:format
rdfs:isDefinedBydprod-shapes:

7.5.28 dprod-shapes:Distribution-isDistributionOf§

The dataset that this distribution makes available.

dprod-shapes:Distribution-isDistributionOf
https://www.omg.org/spec/DPROD/shapes/Distribution-isDistributionOf
RDF predicateValue
rdf:typesh:PropertyShape
rdfs:labeldistribution is distribution of shape
sh:classdcat:Dataset
sh:pathdprod:isDistributionOf
rdfs:isDefinedBydprod-shapes:

8. Core Data Product Extensions§

For real world data products, the core data product details will be part of a wider set of metadata that allows the data and data product to be used effectively.

Below is an example of extending the DPROD data product, specifically by adding an agreement to a data product.

In this example, a Data Product Agreement is defined as a subclass of FIBO Agreement.

Definition of a simple Agreement based on FIBO:

[
  {
    "@context": [
      "https://ekgf.org/dprod/spec/archive/1.0/dprod.jsonld",
      {
        "fibo": "http://spec.edmcouncil.org/fibo/ontology/FND/Agreements/MetadataFNDAgreements/#",
        "ex": "http://example.org/dp#"
      }
    ],
    "@id": "ex:isSubjectToAgreement",
    "@type": "rdf:Property",
    "rdfs:label": "Data Product is Subject To FIBO Agreement",
    "rdfs:domain": {
      "@id": "DataProduct"
    },
    "rdfs:range": {
      "@id": "DataProductAgreement"
    }
  },
  {
    "@id": "ex:DataProductAgreement",
    "@type": "rdfs:class",
    "rdfs:label": "DataProductAgreement",
    "rdfs:subClassOf": {
      "@id": "fibo:Agreement"
    }
  }
]

A full definition of agreements for data products is likely to be more complex than a single class and may use other information models or their profiles (such as ODRL Policy) or create dedicated definitions.

Below is an example of a Data Product with an associated Data Product Agreement with an effective date.

Using the agreement:

{
  "@context": [
    "https://ekgf.org/dprod/spec/archive/1.0/dprod.jsonld",
    {
      "fibo": "http://spec.edmcouncil.org/fibo/ontology/FND/Agreements/MetadataFNDAgreements/#",
      "ex": "http://example.org/dp#"
    }
  ],
  "dataProducts": [
    {
      "id": "https://y.com/data-product/company-sales",
      "type": "DataProduct",
      "outputPort": {
        "id": "https://y.com/data-product/company-sales/port/2025-sales",
        "type": "DataService",
        "label": "Sales",
        "endpointURL": "https://y.com/data-product/company-sales/port/2025-sales",
        "isAccessServiceOf": {
          "type": "Distribution",
          "format": "https://www.iana.org/assignments/media-types/application/json",
          "isDistributionOf": {
            "type": "Dataset",
            "label": "Sales",
            "id": "https://y.com/data-product/company-sales/dataset/2025-sales",
            "conformsTo": "https://y.com/schema/Sale"
          }
        }
      },
      "ex:iSubjectToAgreement": {
        "@id": "ex:VVSimpleAgreement",
        "@type": "ex:DataProductagreement"
      }
    }
  ],
  "agreements": [
    {
      "@id": "ex:VVSimpleAgreement",
      "@type": "ex:DataProductAgreement",
      "rdfs:label": "Very Simple Data Product Agreement",
      "fibo:hasEffectiveDate": {
        "@type": "xsd:date",
        "@value": "2024-08-31"
      }
    }
  ]
}

9. High Level Lineage: Between Data Products§

It is important to be able to trace the lineage of data. Within DPROD, this can be done in two ways: at a high level from one data product to another and, if desired, at the more detailed level of the underlying datasets.

9.1 High Level Lineage: Between Data Products§

Data products have input and output ports, and one data product’s input port will point to another data product’s output port.

This allows a user to query the lineage. The data products all have URLs as identifiers, and properties all connect to each other, so a query can walk from one data product to the downstream data products that feed it.

One can follow the path that leads from one data product to another like this:

Data Product >> inputPort >> isAccessServiceOf >> isDistributionOf >> Input Data Product 

The following example data has three data products that connect to each other through their input and output ports:

{
  "@context": "https://ekgf.org/dprod/spec/archive/1.0/dprod.jsonld",
  "dataProducts": [
    {
      "id": "https://y.com/data-product/company-finance",
      "type": "DataProduct",
      "inputPort": [
        {
          "id": "https://y.com/data-product/company-sales/port/2025-sales",
          "type": "DataService"
        },
        {
          "id": "https://y.com/data-product/company-hr/port/2025-payroll",
          "type": "DataService"
        }
      ],
      "outputPort": {
        "id": "https://y.com/data-product/company-sales/port/2025-balance-sheet",
        "type": "DataService",
        "label": "Balance Sheet",
        "endpointURL": "https://y.com/data-product/company-sales/port/2025-c",
        "isAccessServiceOf": {
          "type": "Distribution",
          "format": "https://www.iana.org/assignments/media-types/application/json",
          "isDistributionOf": {
            "type": "Dataset",
            "id": "https://y.com/data-product/company-sales/dataset/2025-balance-sheet",
            "conformsTo": "https://y.com/schema/BalanceSheet"
          }
        }
      }
    },
    {
      "id": "https://y.com/data-product/company-sales",
      "type": "DataProduct",
      "outputPort": {
        "id": "https://y.com/data-product/company-sales/port/2025-sales",
        "type": "DataService",
        "label": "Sales",
        "endpointURL": "https://y.com/data-product/company-sales/port/2025-sales",
        "isAccessServiceOf": {
          "type": "Distribution",
          "format": "https://www.iana.org/assignments/media-types/application/json",
          "isDistributionOf": {
            "type": "Dataset",
            "label": "Sales",
            "id": "https://y.com/data-product/company-sales/dataset/2025-sales",
            "conformsTo": "https://y.com/schema/Sale"
          }
        }
      }
    },
    {
      "id": "https://y.com/data-product/company-hr",
      "type": "DataProduct",
      "outputPort": {
        "id": "https://y.com/data-product/company-sales/port/2025-payroll",
        "type": "DataService",
        "label": "Payroll",
        "endpointURL": "https://y.com/data-product/company-hr/port/2025-payroll",
        "isAccessServiceOf": {
          "type": "Distribution",
          "format": "https://www.iana.org/assignments/media-types/text/csv",
          "isDistributionOf": {
            "type": "Dataset",
            "label": "Payroll",
            "id": "https://y.com/data-product/company-sales/dataset/2025-payroll",
            "conformsTo": "https://y.com/schema/Payroll"
          }
        }
      }
    }
  ]
}

Given this example data, starting at the data product https://y.com/data-product/company-finance, one could walk the relationships to find the input data products that feed it:

https://y.com/data-product/company-finance >> 
    :inputPort >> 
    :isAccessServiceOf >> 
    :isDistributionOf >> [
        https://y.com/data-product/company-sales, 
        https://y.com/data-product/company-hr
    ]

In Linked Data, this would use a query such as:

PREFIX :      <https://y.com/data-product/>
PREFIX rdfs:  <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dprod: <https://www.omg.org/spec/DPROD/dprod/>

SELECT DISTINCT ?input
WHERE
{ 
  :company-finance dprod:inputPort ?inputPort .
  ?inputPort dprod:isAccessServiceOf/dprod:isDistributionOf/rdfs:label ?input .
}

9.1.1 Detailed Level: Between Datasets§

To track lineage at a more granular level, one can also use PROV (https://www.w3.org/TR/prov-o/) at the dataset level.

dap:atnf-P366-2003SEPT
  rdf:type dcat:Dataset ;
  dcterms:bibliographicCitation "Burgay, M; McLaughlin, M; Kramer, M; Lyne, A; Joshi, B; Pearce, G; D'Amico, N; Possenti, A; Manchester, R; Camilo, F (2017): Parkes observations for project P366 semester 2003SEPT. v1. CSIRO. Data Collection. https://doi.org/10.4225/08/598dc08d07bb7" ;
  dcterms:title "Parkes observations for project P366 semester 2003SEPT"@en ;
  dcat:landingPage <https://data.csiro.au/dap/landingpage?pid=csiro:P366-2003SEPT> ;
  prov:wasGeneratedBy dap:P366 ;
  .

dap:P366
  rdf:type prov:Activity ;
  dcterms:type <http://dbpedia.org/resource/Observation> ;
  prov:startedAtTime "2000-11-01"^^xsd:date ;
  prov:used dap:Parkes-radio-telescope ;
  prov:wasInformedBy dap:ATNF ;
  rdfs:label "P366 - Parkes multibeam high-latitude pulsar survey"@en ;
  rdfs:seeAlso <https://doi.org/10.1111/j.1365-2966.2006.10100.x> ;
  .

See: https://www.w3.org/TR/vocab-dcat-3/#examples-dataset-provenance.

10. Data Quality§

  • DQV is a standard vocabulary to describe data quality and is proposed along with the DCAT vocabulary to measure the quality of datasets.
  • It is based on 3 basic entities: Dimensions, the Metrics that belong to its dimension, and the Measurement for each metric.
  • Data Quality can be computed at different levels, eg. data product, dataset, table, column level etc.
  • Usually, data quality metrics are measured at a dataset level and can inform more higher-level quality metrics at a data product level.

11. Data Rights§

ODRL is a W3C standard to describe rights and entitlements. Based on ODRL, data product and dataset publishers can describe policies in a consistent, standard and machine-readable manner. Policies contain permissions and prohibitions on specific actions that are required to be met by stakeholders.

In addition, policies may be limited by constraints (eg. temporal or geographical constraints) and duties (eg. payments) that may be imposed on the permissions.

Policies and their permitted or prohibited actions can be described at different levels, eg. a policy can target a data product, a dataset, a data service or even a column.

Sophisticated engines should interpret and enforce the ODRL policies at the appropriate level eg.:

examplePolicyA odrl:targets exampleProduct:ProductA .
examplePolicyB odrl:targets exampleDataset:DatasetA1 .

An example of a policy follows, that describes permission to distribute the data only within a specific geographic region:

examplePolicyA odrl:permission
   {
    "action": "odrl:distribute",
    "constraint": [
       {"leftOperand": "region",
         "operator": "eq",
        "rightOperator": "region:EMEA"
       }
     ]
    }

12. definition of the entity as a Node Shape (1)§

The Data Product provides to the consumers (dprod:outputDataset) datasets defined based on DCAT. Datasets should be described (dcat:conforms) with logical models. Logical models describe business entities and their properties (attributes and relationships) with consistent business terms and they are technology independent. Ideally, logical models are based on existing standards eg, FIBO, CDM etc. If a logical model does not exist to describe the dataset, then the dataset publisher can create one, preferably by using SHACL modelling language:

Example of a Dataset conforming to a SHACL Schema:

exampleDataset dcat:conforms exampleSchema:DatasetLogicalSchema.
exampleSchema:DatasetLogicalSchema a owl:Ontology, dct:Standard.

Based on SHACL all entities that exist in the dataset are Node Shapes (1). The attributes of the entities are described as Property Shapes with sh:datatype (2) The relationships are also defined as Property Shaped with sh:class the target class of the relationship (3)

# definition of the entity as a Node Shape (1)
example:Account a sh:NodeShape ;
    # human readable name of the entity
    rdfs:label "Account"@en ;
    # description of the entity
    dc:description "An Account is..." ;
    # an account has a property shape Account Age. 
    # Definition of the property shape follows (2)
    sh:property example:Account-AccountAge ; 
    # an account has a property shape Account Branch. 
    # Definition of the property shape follows (3)
    sh:property example:Account-AccountBranch ;     
    rdfs:isDefinedBy exampleSchema:DatasetLogicalSchema;
.

# (2) Definition of the Account-AccountAge property shape
#     describing that an account MUST have exactly one
#     AccountAge attribute and its datatype is integer
example:Account-AccountAge a sh:PropertyShape ;   
    sh:path example:AccountAge ;
    sh:datatype xsd:integer ;
    sh:minCount 1 ;
    sh:maxCount 1 ;
    rdfs:isDefinedBy exampleSchema:DatasetLogicalSchema ;
.

# (3) Definition of the Account-AccountBranch property shape
#     describing than an account must have at least one 
#     Account Branch which is another entity
example:Account-AccountBranch a sh:PropertyShape ;   
    sh:path example:AccountBranch ;
    sh:class  example:Branch ;
    sh:minCount 1 ;
    rdfs:isDefinedBy exampleSchema:DatasetLogicalSchema ;
.

# definition of the entity Branch as a Node Shape (1)
example:Branch a sh:NodeShape ;
    rdfs:label "Branch"@en ;
    dc:description "A Branch is.." ;
    rdfs:isDefinedBy exampleSchema:DatasetLogicalSchema ;
.

13. Equity Trade§

Example of a data product for Equity Trades.

The Equity Trades Data Product provides two datasets to the consumers: one for trades in London Stock Exchange (LSEG) and one in Euronext.

14. Defining Observability Ports in DPROD§

An Observability Port is a designated interface or endpoint in a system or application specifically used for monitoring and diagnostic purposes. It allows external tools or services to collect and analyze data related to the system's performance, health, and behaviour. By exposing metrics, logs, and traces through this port, administrators and developers can gain insights into the system's state, troubleshoot issues, and ensure it operates efficiently and reliably.

14.1 Defining Observability Ports in DPROD§

DPROD has a schema-first design. The first thing to do is define a schema for the logging information. It could be a schema based on OpenTelemetry, but this uses RLOG (which is a semantic ontology for logging).

To find the Observability Port, query the ports to identify the ones that return an RLOG:Entry:

outputPort >> isAccessServiceOf >> isDistributionOf >> conformsTo >> rlog:Entry

14.2 Example Data Product with Observability Port§

One can see that the example data product has two ports, one with the data and one with the logging. This query will return the URI of the port that returns logging data: https://y.com/uk-bonds/observability-port.

Here is an example of a data product with an observability port:

{
  "@context": "https://ekgf.org/dprod/spec/archive/1.0/dprod.jsonld",
  "dataProducts": [
    {
      "id": "https://y.com/data-product/uk-bonds",
      "type": "DataProduct",
      "inputPort": [
        {
          "id": "https://y.com/data-product/uk-bonds/port/2024-data",
          "type": "DataService"
        }
      ],
      "outputPort": [
        {
          "id": "https://y.com/data-product/uk-bonds/port/2024-observability",
          "type": "DataService",
          "label": "Observability Port",
          "endpointURL": "https://y.com/data-product/uk-bonds/port/2024-observability",
          "isAccessServiceOf": {
            "type": "Distribution",
            "format": "https://www.iana.org/assignments/media-types/application/json",
            "isDistributionOf": {
              "type": "Dataset",
              "id": "https://y.com/data-product/uk-bonds/dataset/2024-observability",
              "conformsTo": "https://y.com/schema/ObservabilityLog"
            }
          }
        },
        {
          "id": "https://y.com/data-product/uk-bonds/port/2024-data",
          "type": "DataService",
          "label": "Data Port",
          "endpointURL": "https://y.com/data-product/uk-bonds/port/2024-data",
          "isAccessServiceOf": {
            "type": "Distribution",
            "format": "https://www.iana.org/assignments/media-types/application/json",
            "isDistributionOf": {
              "type": "Dataset",
              "id": "https://y.com/data-product/uk-bonds/dataset/2024-data",
              "conformsTo": "https://y.com/schema/Data"
            }
          }
        }
      ]
    }
  ]
}

Given that the schema defines the class for an observation, it can be used to find all observability ports on data product like this:

[https://y.com/data-product/uk-bonds/port/2024-observability] >> isAccessServiceOf >> isDistributionOf >> conformsTo >> https://y.com/schema/ObservabilityLog

In Linked Data a SPARQL query would do that:

SELECT ?port
WHERE
{ 
  ?port a dcat:DataService .
  ?port (dprod:isAccessServiceOf/dprod:isDistributionOf)/dcat:conformsTo rlog:Entry
}

This query will return the URI of the port that provides logging data: https://y.com/data-product/uk-bonds/port/2024-observability.

15. Sba Pool Rates§

Rates for SBA Pool that are Mortgage Backed Securities.

The data product is provided through 3 ports:

  • 1st port: providing all sba pool rates through a query to a database
  • 2nd port: providing only EMEA rates through an api
  • 3rd port: providing only US rates through a Kafka topic

16. Acknowledgements§

The editors gratefully acknowledge the feedback and contributions made by individuals who have participated in EDM Council's CDMC team.

A. Quality issues§

Review findings preserve competing source statements; they are not automatically RDF errors.

No label found§

  • https://www.omg.org/spec/DPROD/dprod/securitySchemaType

    No English or language-neutral label found in the collected sources.

B. Document provenance§

This record identifies the generator and the source revisions collected for this publication. Committers below describe recent repository changes, rather than authorship of the whole document.

Generated by
Grapharch 0.2.26 (build 78)
Build operator
Build operator was not recorded.

Generator source commit: 615760949a54246d2cb29b1b164478fa3ed0ccdf

Generator build: 78

Source repository

Repository
https://github.com/EKGF/dprod.git
Source commit
e146b2be541a4aac3f3709eb878fbd7ec871cead
Source state
Committed source tree pinned to this exact revision.

Recent committers

Up to five commits, following the first-parent history from the source commit. This is not the full contributor list.

  1. GitHub
    e146b2be541a4aac3f3709eb878fbd7ec871cead
  2. Jacobus Geluk
    e964fd38f6cb7ff88e3e7e05dbaec2177cc92c2c
  3. Jacobus Geluk
    8e271f065af01410ab50456df20f929c9d87aaae
  4. Jacobus Geluk
    d386c310bedfc75fd69f2a975599abf332ef78da
  5. Jacobus Geluk
    7c8a4436d3f76ee90330b313ea7073f373e2a4ac

Diagram interpretation

Class diagrams use UML shorthand for asserted OWL facts and declared SHACL constraints, without inference or a claim that validation passed. RDF properties remain independent resources even when drawn as class members or labelled associations. Domain/range declarations alone do not impose requiredness or cardinality. A property with a declared domain and range is a labelled association from the domain to the range; a datatype property, or one without a specific range, is an attribute of its domain. Multiple domains or ranges apply conjunctively, not as alternatives: each declared domain-range pair is drawn and holds for every use of the property. An explicitly declared union domain is drawn as one source box whose class alternatives are joined by or. A dotted rdfs:Resource source box is a presentation placeholder when no specific domain is declared in the selected inputs, not an asserted domain or a claim that none can be entailed. Simple existential subclass restrictions use a class-scoped association with target-end 1..* and {qualified}, meaning at least one value in the named filler class; this does not constrain all property values or declare a global range. Datatype-qualified restrictions appear as class fields with the declared datatype and bracketed count of distinct data values; they do not declare a global property range. Qualified cardinality associations show n..* for minima, n for exact counts, or 0..n for maxima at the named target with {qualified}: only values in that target class count, and other property values are not excluded. A zero maximum prohibits values in that named target class; it is never omitted. A simple zero qualified minimum imposes no restriction and is omitted visually. An existential with a simple zero-minimum filler still requires a value; a positive qualified minimum or exact count on the same class and property already guarantees that existence. Unsupported anonymous OWL expressions remain explicit. Types and counts from simple property shapes enrich the UML notation. Identical qualifiers are shown once; where declaring shapes disagree, the disputed type or multiplicity is omitted from the drawing and remains in the dedicated shape documentation. No shape is chosen arbitrarily and conflicting counts are not silently intersected. An absent count imposes no lower or upper bound beyond zero or unbounded, respectively. Complex constraints require the linked full shape definition. Original OWL facts and shape provenance remain in the published source graph. Some diagrams show at most five complete OWL relationship groups within an eighteen-fact budget, selected in deterministic IRI order; omitted groups remain in the published source graph. These views are not exhaustive. Names from the same ontology as the documented class are shown by label; names from another ontology add their prefixed name in parentheses, and unlabelled names show their prefixed name.

Search this specification