Output model

The output follows the EKGF Raw Data Ontology and conforms to its SHACL shapes.

Named graphs

Every source file gets its own named graph, with its structure and its metadata. One more graph describes the run. N-Quads and TriG keep these graphs; N-Triples writes their union.

Source structure

Source structureRaw RDF
the fileraw:CsvFile, raw:TsvFile or raw:XlsxFile, and always raw:File, raw:Source, dcat:Distribution, prov:Entity
a worksheet or the CSV bodyraw:View (raw:viewNumber, sheet name as rdfs:label)
a columnraw:Column (raw:columnNumber, header text as rdfs:label)
a data rowraw:Row (raw:viewRowNumber, 0-based)
a non-empty cellraw:Value (raw:valueOf, raw:forColumn, raw:literalValue)
an error cell (#N/A)raw:NullValue with raw:sourceText
a formula cellthe cached result as above, plus raw:formula (A1) and raw:formulaR1C1
a column filled with one formularaw:formulaR1C1 on the raw:Column

XLS and ODS files are typed raw:File. The first physical row of each view is its header, and the rows after it are data.

Capture rules

  • No guessing. CSV fields are always xsd:string, so "00123" stays "00123". Empty fields produce no value. Spreadsheet cells keep the type the workbook stored them with: string, integer, double, boolean, date-time or duration.
  • Parquet keeps its schema. A Parquet file is one raw:ParquetFile with one view. Its columns are the schema's top-level fields, which also serve as the header labels. The types map as follows:
    • integers become xsd:integer, floats xsd:double, decimals xsd:decimal (with their scale, e.g. 12.50) and booleans xsd:boolean;
    • DATE becomes xsd:date, and TIME becomes xsd:time;
    • a timestamp with a timezone becomes a UTC xsd:dateTime (…Z), and one without stays a zone-less xsd:dateTime;
    • strings become xsd:string, and binary becomes xsd:hexBinary;
    • a Parquet null is an explicit source null, so it becomes a raw:NullValue;
    • nested lists, structs and maps are captured as their display text ([x, y]) until the raw model supports nesting.
  • Columns are physical. Header text is only a label, and ragged rows just add columns.
  • Duplicates once. Identical files found twice are triplified once.

A two-row CSV file, id,qty and 00123,4, produces these values (TriG, IRIs shortened):

<urn:uuid:a4bc…> a raw:Row ;
    raw:isRowInView <urn:uuid:d0af…> ;
    raw:viewRowNumber 0 .
<urn:uuid:1ae9…> a raw:Value ;
    raw:valueOf <urn:uuid:a4bc…> ;
    raw:forColumn <urn:uuid:906c…> ;
    raw:literalValue "00123" .
<urn:uuid:86ef…> a raw:Value ;
    raw:valueOf <urn:uuid:a4bc…> ;
    raw:forColumn <urn:uuid:da82…> ;
    raw:literalValue "4" .

Distribution metadata

Each source is a dcat:Distribution with:

  • raw:fileName and dcterms:title: the file name;
  • dcat:downloadURL and prov:atLocation: its file: URL;
  • dcat:mediaType: its IANA media type;
  • dcat:byteSize and dcterms:modified.

Checksums

The source bytes are read once, in a single streaming pass, for both the SHA-256 digest and the Git blob id. The digest is an spdx:Checksum with spdx:algorithm spdx:checksumAlgorithm_sha256 and an xsd:hexBinary spdx:checksumValue. The Git blob id is raw:gitBlob, also for files outside a Git repository.

Git origin

For a file inside a Git work tree, the source also records:

  • raw:gitRepository: the fetch URL of the default remote, and prov:wasDerivedFrom that URL when it is a valid IRI;
  • raw:gitCommit: the HEAD commit;
  • raw:gitPath: the path relative to the work tree;
  • raw:gitModified: whether the bytes read differ from that path in HEAD.

Run provenance

The run graph holds a prov:Activity with its start and end times, associated with a prov:SoftwareAgent for this triplify version. The activity prov:used every source. Each source graph is a dcat:Dataset that prov:wasDerivedFrom its source and prov:wasGeneratedBy the run.

Column profiles

With --profile, every column also carries exact measures computed over its values. Header cells are not values.

MeasureMeaning
raw:count, raw:nullCountnon-null values and raw:NullValues
raw:unique, raw:top, raw:freqdistinct values, the most frequent one and how often it occurs
raw:inferredDatatype, raw:conformingCounta datatype and how many values conform to it
raw:min, raw:maxextremes, typed in the inferred datatype
raw:mean, raw:stdmean and sample standard deviation, numeric datatypes only
raw:25pct, raw:50pct, raw:75pctpercentiles by linear interpolation (Hyndman and Fan type 7)

The inferred datatype describes the column. The values keep their source literals. A CSV column with 98 ISO dates and two n/a entries gets raw:inferredDatatype xsd:date ; raw:conformingCount 98 ; raw:count 100, and every value remains an xsd:string. A later step can cast with known confidence and look directly at the two exceptions.

Inference policy:

  • Candidates are tried from most to least specific: xsd:boolean, xsd:integer, xsd:decimal, xsd:double, xsd:date, xsd:dateTime. The first one that at least 95% of the non-null values conform to is chosen.
  • Text conforms only by strict XSD lexical form, with these exceptions:
    • integers with leading zeros (00123) are identifiers, not numbers;
    • only true and false are booleans, so 0/1 columns stay integers;
    • SQL-style 2026-11-17 18:46:00 timestamps are xsd:dateTime.
  • Spreadsheet cells conform by the type the workbook stored. A date-time at midnight is also a date.

Every measure is exact or omitted, never approximated. raw:unique, raw:top and raw:freq are omitted for a column with more than 100,000 distinct values. Percentiles are omitted for a column with more than 1,000,000 numbers. The measures and their SHACL shape (raw:ColumnProfileShape) are defined in the EKGF Raw Data Ontology.

Formulas

Formulas are data too. Each formula cell keeps its cached result as raw:literalValue, or becomes a raw:NullValue when the workbook stored no result. It also keeps the formula verbatim (raw:formula "A2*B2") and the same formula rewritten relative to its own cell (raw:formulaR1C1 "RC[-2]*RC[-1]"). When every formula in a column has the same relative form, which is the usual case for a filled-down formula, the column itself gets that raw:formulaR1C1. The column is then a derivation rule (total = qty × price) rather than a list of numbers.

Identifiers

Every resource has a canonical name. Its IRI is the name-based UUIDv8 with SHA-256 of RFC 9562 (section 6.5, appendix B.2):

digest = SHA-256( NAMESPACE.bytes ‖ UTF-8(name) )
uuid   = digest[0..16] with version := 8, variant := 0b10
iri    = urn:uuid:<lowercase hyphenated uuid>

NAMESPACE = uuidv8_sha256(NameSpace_URL, "https://ekgf.org/ontology/raw#")

In the names, <sha256> is the lowercase hex SHA-256 of the source bytes. Integers are decimal with no leading zeros.

ResourceName
sourcesource/<sha256>
its named graphsource/<sha256>/graph
its checksumsource/<sha256>/checksum
view (1-based)source/<sha256>/view/<v>
column (1-based)source/<sha256>/view/<v>/column/<c>
row (0-based)source/<sha256>/view/<v>/row/<r>
valuesource/<sha256>/view/<v>/row/<r>/column/<c>
software agentagent/triplify/<version>

A run is an event, not content. It gets a time-ordered UUIDv7, and its graph is uuidv8_sha256(<run uuid>, "graph").

Search documentation

Enter words to search this site.