dbp-schema

JSON-LD schema of metadata for DataBrewingPlatform

It also contains schema for RWD Profile (dbp:RealWorldDataFieldProfile), which describes structure of Real-World Data.

Build

for Rust

cargo build

for Go-lang

protoc -I . --go_out=paths=source_relative:. --experimental_allow_proto3_optional --go_opt=Mdbp_schema.proto=exdata.co.jp/project/protos/dbp_schema  dbp_schema.proto

Data Field Catalogue

dbp:DataFieldType describes a kind of data field independent of any dataset, and dbp:compressionHints hangs dbp:CompressionHint off it — one per use case — to say how that field should be compressed for that use case.

Value range on a `dbp:CompressionHint`

dbp:precisionBytes is meant to follow from the value range (dbp:rangeMin / dbp:rangeMax) and the granularity (dbp:decimalPlaces): how many bytes it takes to represent that range at that granularity. A dbp:DataFieldType has to allow the widest range any of its use cases can produce, so a use case that only spans a narrow part of it would otherwise inherit a width far larger than it needs. dbp:rangeMin / dbp:rangeMax are therefore allowed on a dbp:CompressionHint as well:

Omitted — the hint uses the range of the dbp:DataFieldType it belongs to. This is what every hint written before the range was allowed here means, so nothing changed meaning.
Declared — the hint narrows that range to the subset the use case actually covers, and dbp:precisionBytes follows from the subset.
One end at a time — the two ends are inherited independently. Declaring only dbp:rangeMin leaves dbp:rangeMax inherited from the dbp:DataFieldType; that is a valid hint, not an incomplete one. There is no pairing rule.

dbp:decimalPlaces is read the same way: omitted on a hint, the value of the dbp:DataFieldType applies; declared, it applies to that hint. It is the third input to dbp:precisionBytes alongside the two range ends, so leaving its inheritance undefined would leave the derivation undefined for any hint that omits it.

Two things this vocabulary states but does not enforce, because it carries no constraint language (no SHACL or equivalent):

A declared end narrows, never widens. A hint range is expected to stay inside the range of its dbp:DataFieldType.
A declared dbp:decimalPlaces coarsens, never refines. A hint is expected to keep no more places than its dbp:DataFieldType does. Compression discards digits; a hint that asked for finer granularity than the field type declares would be asking for digits the field type says are not there.

Nothing here rejects either, so the consumer that derives dbp:precisionBytes is the one that has to check them — both break the premise the derivation rests on.

RWD Profile Example

CSV

Data

2023-01-01T00:00:00,6909fcbb-ef24-4f9b-8981-4a81297cb133,100,35.987903,136.88374
2023-01-01T00:01:02,ba3364b5-662e-4909-b6d1-0e59d8b34beb,101,35.987008,136.88371
2023-01-01T00:01:58,72b81471-ed17-4f2d-8df7-1249f7daf08f,102,35.988905,136.83374
2023-01-01T00:03:00,2159cb54-b22d-4fa8-8a12-7eeeb41f4348,103,35.967909,136.88344
2023-01-01T00:04:01,98f1b873-31fc-4d6c-a3c6-3ca562830c14,104,35.917918,136.85375
...

RWD Profile

{
  "@id": "https://path.to/this.jsonld",
  "@type": "dbp:RealWorldDataFieldProfile",
  "@context": {
    "dbp": "https://exdata.co.jp/dbp/schema/",
    "rdf": "https://www.w3.org/1999/02/22-rdf-syntax-ns#",
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
    "schema": "https://schema.org/"
  },
  "schema:name": "Sample RWD Profile for CSV",
  "dbp:structure": {
    "@type": "dbp:RealWorldDataStructureGraph",
    "@graph": [
      {
        "@id": "root",
        "@type": "rdf:List",
        "rdfs:label": "root",
        "schema:rangeIncludes": {
          "@id": "class1"
        },
        "dbp:dbpaCompress": true,
        "dbp:dbpaCompressListID": 0
      },
      {
        "@id": "class1",
        "@type": "rdfs:Class",
        "rdfs:label": "class1",
        "schema:rangeIncludes": [
          {
            "@id": "datetime"
          },
          {
            "@id": "id"
          },
          {
            "@id": "seq"
          },
          {
            "@id": "latitude"
          },
          {
            "@id": "longitude"
          }
        ],
        "schema:domainIncludes": {
          "@id": "root"
        },
        "dbp:dbpaCompressRowID": 0
      },
      {
        "@id": "datetime",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "datetime",
        "dbp:itemType": "DateTime",
        "rdfs:comment": "Timestamp when an item was recorded.",
        "schema:rangeIncludes": {
          "@id": "schema:Date"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Interval",
        "dbp:dbpaDatetimeFormat": "%Y-%m-%dT%H:%M:%S",
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 0
      },
      {
        "@id": "id",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "id",
        "dbp:itemType": "String(32)",
        "rdfs:comment": "Unique identifier for each item.",
        "schema:rangeIncludes": {
          "@id": "schema:Text"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Nominal",
        "dbp:isEnumValue": false,
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 1
      },
      {
        "@id": "seq",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "seq",
        "dbp:itemType": "Integer",
        "rdfs:comment": "Sequential Number of data record.",
        "schema:rangeIncludes": {
          "@id": "schema:Integer"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Ordinal",
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 2,
        "dbp:rangeMin": "100",
        "dbp:rangeMax": "999999",
        "dbp:baseIncrement": 1,
        "dbp:useBaseIncrement": true
      },
      {
        "@id": "latitude",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "latitude",
        "dbp:itemType": "Float",
        "rdfs:comment": "Geographic coordinate representing north-south position",
        "schema:unitCode": "DD",
        "schema:rangeIncludes": {
          "@id": "schema:Float"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Proportional",
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 3,
        "dbp:rangeMin": "35.15300000",
        "dbp:rangeMax": "35.15999999",
        "dbp:decimalPlaces:": 6,
        "dbp:precisionBytes": 2
      },
      {
        "@id": "longitude",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "longitude",
        "dbp:itemType": "Float",
        "rdfs:comment": "Geographic coordinate representing east-west position",
        "schema:unitCode": "DD",
        "schema:rangeIncludes": {
          "@id": "schema:Float"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Interval",
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 4,
        "dbp:rangeMin": "136.9700000",
        "dbp:rangeMax": "136.9899999",
        "dbp:decimalPlaces:": 5,
        "dbp:precisionBytes": 2
      }
    ]
  },
  "schema:dateCreated": "2024-12-31T23:59:59.999999+09:00",
  "schema:encodingFormat": "text/csv",
  "dbp:csvHasHeader": false,
  "dbp:newLineCharacter": "LF"
}

JSON

Data

[
  {
    "dateStart": "2023-01-01T00:00:00",
    "dateEnd": "2023-01-01T01:59:59",
    "id": "6909fcbb-ef24-4f9b-8981-4a81297cb133",
    "deviceType": "ABC",
    "positions": [
      {
        "latitude": 35.9879087,
        "longitude": 136.883749
      },
      {
        "latitude": 35.9879086,
        "longitude": 136.883751
      },
      ...
    ]
  },
  {
    "dateStart": "2023-01-01T00:01:02",
    "dateEnd": "2023-01-01T02:01:01",
    "id": "ba3364b5-662e-4909-b6d1-0e59d8b34beb",
    "deviceType": "DEF",
    "positions": [
      {
        "latitude": 35.9879035,
        "longitude": 136.883732
      },
      {
        "latitude": 35.9879037,
        "longitude": 136.883730
      },
      ...
    ]
  },
  ...
]

RWD Profile

{
  "@id": "https://path.to/this.jsonld",
  "@type": "dbp:RealWorldDataFieldProfile",
  "@context": {
    "dbp": "https://exdata.co.jp/dbp/schema/",
    "rdf": "https://www.w3.org/1999/02/22-rdf-syntax-ns#",
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
    "schema": "https://schema.org/"
  },
  "schema:name": "Sample RWD Profile for JSON",
  "dbp:structure": {
    "@type": "dbp:RealWorldDataStructureGraph",
    "@graph": [
      {
        "@id": "root",
        "@type": "rdf:List",
        "rdfs:label": "root",
        "schema:rangeIncludes": {
          "@id": "class1"
        },
        "dbp:dbpaCompress": true,
        "dbp:dbpaCompressListID": 0
      },
      {
        "@id": "class1",
        "@type": "rdfs:Class",
        "rdfs:label": "class1",
        "schema:rangeIncludes": [
          {
            "@id": "dateStart"
          },
          {
            "@id": "dateEnd"
          },
          {
            "@id": "id"
          },
          {
            "@id": "deviceType"
          },
          {
            "@id": "points"
          }
        ],
        "schema:domainIncludes": {
          "@id": "root"
        },
        "dbp:dbpaCompressRowID": 0
      },
      {
        "@id": "dateStart",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "dateStart",
        "dbp:itemType": "DateTime",
        "rdfs:comment": "Timestamp when an item was started to record.",
        "schema:rangeIncludes": {
          "@id": "schema:Date"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Interval",
        "dbp:dbpaDatetimeFormat": "%Y-%m-%dT%H:%M:%S",
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 0
      },
      {
        "@id": "dateEnd",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "dateEnd",
        "dbp:itemType": "DateTime",
        "rdfs:comment": "Timestamp when an item was ended to record.",
        "schema:rangeIncludes": {
          "@id": "schema:Date"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Interval",
        "dbp:dbpaDatetimeFormat": "%Y-%m-%dT%H:%M:%S",
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 1
      },
      {
        "@id": "id",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "id",
        "dbp:itemType": "String(32)",
        "rdfs:comment": "Unique identifier for each item.",
        "schema:rangeIncludes": {
          "@id": "schema:Text"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Nominal",
        "dbp:isEnumValue": false,
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 2
      },
      {
        "@id": "deviceType",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "deviceType",
        "dbp:itemType": "String(3)",
        "rdfs:comment": "Type of the device.",
        "schema:rangeIncludes": {
          "@id": "schema:Text"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:variableScaleType": "Nominal",
        "dbp:isEnumValue": true,
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 3
      },
      {
        "@id": "points",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "points",
        "dbp:itemType": "list",
        "rdfs:comment": "Location history of a record.",
        "schema:rangeIncludes": {
          "@id": "list2"
        },
        "schema:domainIncludes": {
          "@id": "class1"
        },
        "dbp:dbpaCompressParentListID": 0,
        "dbp:dbpaCompressColumnID": 4,
        "dbp:dbpaChildrenLists": [
          1
        ]
      },
      {
        "@id": "list2",
        "@type": "rdf:List",
        "rdfs:label": "list2",
        "schema:domainIncludes": {
          "@id": "points"
        },
        "schema:rangeIncludes": {
          "@id": "class2"
        },
        "dbp:dbpaCompress": true,
        "dbp:dbpaCompressListID": 1
      },
      {
        "@id": "class2",
        "@type": "rdfs:Class",
        "rdfs:label": "class2",
        "schema:rangeIncludes": [
          {
            "@id": "latitude"
          },
          {
            "@id": "longitude"
          }
        ],
        "schema:domainIncludes": {
          "@id": "list2"
        },
        "dbp:dbpaCompressRowID": 1
      },
      {
        "@id": "latitude",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "latitude",
        "dbp:itemType": "Float",
        "rdfs:comment": "Geographic coordinate representing north-south position",
        "schema:unitCode": "DD",
        "schema:rangeIncludes": {
          "@id": "schema:Float"
        },
        "schema:domainIncludes": {
          "@id": "class2"
        },
        "dbp:variableScaleType": "Proportional",
        "dbp:dbpaCompressParentListID": 1,
        "dbp:dbpaCompressColumnID": 0,
        "dbp:rangeMin": "35.153000000",
        "dbp:rangeMax": "35.159999999",
        "dbp:decimalPlaces:": 6,
        "dbp:precisionBytes": 2
      },
      {
        "@id": "longitude",
        "@type": "dbp:RealWorldDataStructureProperty",
        "rdfs:label": "longitude",
        "dbp:itemType": "Float",
        "rdfs:comment": "Geographic coordinate representing east-west position",
        "schema:unitCode": "DD",
        "schema:rangeIncludes": {
          "@id": "schema:Float"
        },
        "schema:domainIncludes": {
          "@id": "class2"
        },
        "dbp:variableScaleType": "Interval",
        "dbp:dbpaCompressParentListID": 1,
        "dbp:dbpaCompressColumnID": 1,
        "dbp:rangeMin": "136.97000000",
        "dbp:rangeMax": "136.98999999",
        "dbp:decimalPlaces:": 7,
        "dbp:precisionBytes": 2
      }
    ]
  },
  "schema:dateCreated": "2024-12-31T23:59:59.999999+09:00",
  "schema:encodingFormat": "application/json"
}

Name		Name	Last commit message	Last commit date
Latest commit History 110 Commits
.github/workflows		.github/workflows
src		src
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
Cargo.toml		Cargo.toml
LICENSE		LICENSE
ProtoToJsonld.py		ProtoToJsonld.py
README.md		README.md
build.rs		build.rs
convert-jsonld-graph-to-ntriple.ipynb		convert-jsonld-graph-to-ntriple.ipynb
dbdiagram-oss_exported_data.json		dbdiagram-oss_exported_data.json
dbp-schema.ja.jsonld		dbp-schema.ja.jsonld
dbp-schema.jsonld		dbp-schema.jsonld
dbp_schema.ja.proto		dbp_schema.ja.proto
dbp_schema.pb.go		dbp_schema.pb.go
dbp_schema.proto		dbp_schema.proto
dbp_schema.sql		dbp_schema.sql

Navigation Menu

Uh oh!

Repository files navigation

dbp-schema

Build

for Rust

for Go-lang

Data Field Catalogue

Value range on a `dbp:CompressionHint`

RWD Profile Example

CSV

JSON

About

Releases

Packages

Used by

Contributors

Languages

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

dbp-schema

Build

for Rust

for Go-lang

Data Field Catalogue

Value range on a dbp:CompressionHint

RWD Profile Example

CSV

JSON

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

Value range on a `dbp:CompressionHint`