Skip to content

Commit

Permalink
Merge pull request #106 from bitol-io/dev
Browse files Browse the repository at this point in the history 
Dev
  • Loading branch information
@pflooky
pflooky authored Jan 8, 2025
2 parents 66b46a3 + e645ae9 commit f313988
Show file tree
Hide file tree
Showing 25 changed files with 2,529 additions and 76 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/validate-examples.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
name: validate-examples
on:
pull_request:
branches: ["*"]
push:
branches: ["*"]

Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ image: "https://raw.githubusercontent.com/bitol-io/artwork/main/horizontal/color

This document tracks the history and evolution of the **Open Data Contract Standard**.

# v3.0.1 - 2024-12-22 - APPROVED

* Added field `authoritativeDefinitions` into JSON schema
* Added field `description.customProperties` into JSON schema
* Added field `description.authoritativeDefinitions` into JSON schema
* Added field `role.customProperties` into JSON schema
* Updated `status` field to include examples
* Updated `authoritativeDefinitions` description to be vendor agnostic
* Updated `tags` description and included examples

# v3.0.0 - 2024-10-21 - APPROVED

* **New section**: Support & communication channels.
Expand Down
65 changes: 35 additions & 30 deletions docs/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,13 @@ This section contains general information about the contract.
### Example

```YAML
apiVersion: v3.0.0 # Standard version
apiVersion: v3.0.1 # Standard version
kind: DataContract

id: 53581432-6c55-4ba2-a65f-72344a91553a
name: seller_payments_v1
version: 1.1.0 # Data Contract Version
status: production
status: active
domain: seller
dataProduct: payments
tenant: ClimateQuantumInc
Expand All @@ -59,26 +59,30 @@ description:
limitations: null
usage: null

tags: null
tags: ['finance']
```
### Definitions
| Key | UX label | Required | Description |
|-------------------------|------------------|----------|------------------------------------------------------------------------------------------------|
| apiVersion | Standard version | Yes | Version of the standard used to build data contract. Default value is `v3.0.0`. |
| kind | Kind | Yes | The kind of file this is. Valid value is `DataContract`. |
| id | ID | Yes | A unique identifier used to reduce the risk of dataset name collisions, such as a UUID. |
| name | Name | No | Name of the data contract. |
| version | Version | Yes | Current version of the data contract. |
| status | Status | Yes | Current status of the data contract. Valid values are `production`, `test`, or `development`. |
| tenant | Tenant | No | Indicates the property the data is primarily associated with. Value is case insensitive. |
| domain | Domain | No | Name of the logical data domain. |
| dataProduct | Data Product | No | Name of the data product. |
| description | Description | No | Object containing the descriptions. |
| description.purpose | Purpose | No | Intended purpose for the provided data. |
| description.limitations | Limitations | No | Technical, compliance, and legal limitations for data use. |
| description.usage | Usage | No | Recommended usage of the data. |
| Key | UX label | Required | Description |
|--------------------------------------|---------------------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| apiVersion | Standard version | Yes | Version of the standard used to build data contract. Default value is `v3.0.1`. |
| kind | Kind | Yes | The kind of file this is. Valid value is `DataContract`. |
| id | ID | Yes | A unique identifier used to reduce the risk of dataset name collisions, such as a UUID. |
| name | Name | No | Name of the data contract. |
| version | Version | Yes | Current version of the data contract. |
| status | Status | Yes | Current status of the data contract. Examples are "proposed", "draft", "active", "deprecated", "retired". |
| tenant | Tenant | No | Indicates the property the data is primarily associated with. Value is case insensitive. |
| domain | Domain | No | Name of the logical data domain. |
| dataProduct | Data Product | No | Name of the data product. |
| authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the data contract. |
| description | Description | No | Object containing the descriptions. |
| description.purpose | Purpose | No | Intended purpose for the provided data. |
| description.limitations | Limitations | No | Technical, compliance, and legal limitations for data use. |
| description.usage | Usage | No | Recommended usage of the data. |
| description.authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the dataset; examples would be a link to privacy statement, terms and conditions, license agreements, data catalog, or another tool. |
| description.customProperties | Custom Properties | No | Custom properties that are not part of the standard. |



## Schema
Expand Down Expand Up @@ -112,7 +116,7 @@ schema:
type: businessDefinition
- url: https://youtu.be/jbY1BKFj9ec
type: videoTutorial
tags: null
tags: ['finance']
dataGranularityDescription: Aggregation on columns txn_ref_dt, pmt_txn_id
properties:
- name: txn_ref_dt
Expand Down Expand Up @@ -214,16 +218,16 @@ schema:

#### Applicable to Elements (either Objects or Properties)

| Key | UX label | Required | Description |
|--------------------------|------------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| name | Name | Yes | Name of the element. |
| physicalName | Physical Name | No | Physical name. |
| description | Description | No | Description of the element. |
| businessName | Business Name | No | The business name of the element. |
| authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the table; examples would be a link to an external definition, a training video, a GitHub repo, Collibra, or another tool. See `authoritativeDefinitions` below. |
| quality | Quality | No | List of data quality attributes. |
| tags | Tags | No | A list of tags that may be assigned to the elements (object or property); the tags keyword may appear at any level. |
| customProperties | Custom Properties | No | Custom properties that are not part of the standard. |
| Key | UX label | Required | Description |
|--------------------------|------------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| name | Name | Yes | Name of the element. |
| physicalName | Physical Name | No | Physical name. |
| description | Description | No | Description of the element. |
| businessName | Business Name | No | The business name of the element. |
| authoritativeDefinitions | Authoritative Definitions | No | List of links to sources that provide more details on the element; examples would be a link to privacy statement, terms and conditions, license agreements, data catalog, or another tool. |
| quality | Quality | No | List of data quality attributes. |
| tags | Tags | No | A list of tags that may be assigned to the elements (object or property); the tags keyword may appear at any level. Tags may be used to better categorize an element. For example, `finance`, `sensitive`, `employee_record`. |
| customProperties | Custom Properties | No | Custom properties that are not part of the standard. |

#### Applicable to Objects

Expand Down Expand Up @@ -643,6 +647,7 @@ roles:
| roles.access | Access | No | The type of access provided by the IAM role. |
| roles.firstLevelApprovers | 1st Level Approvers | No | The name(s) of the first-level approver(s) of the role. |
| roles.secondLevelApprovers | 2nd Level Approvers | No | The name(s) of the second-level approver(s) of the role. |
| roles.customProperties | Custom Properties | No | Any custom properties. |


## Service-Level Agreement (SLA)
Expand Down Expand Up @@ -733,7 +738,7 @@ servers:
- **description**: A description of the server.
- **environment**: The environment where the server operates (e.g., `prod`, `dev`, `uat`). There are no set values.
- **roles**: An optional array of roles that have access to the server.
- **customProperties**: Any additional custom properties specific to the server.
- **customProperties**: Any additional custom properties specific to the server that are not part of the standard.

### Specific Server Properties

Expand Down
11 changes: 7 additions & 4 deletions docs/examples/all/full-example.odcs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,21 @@
domain: seller # Domain
dataProduct: my quantum # Data product name
version: 1.1.0 # Version (follows semantic versioning)
status: current
status: active
id: 53581432-6c55-4ba2-a65f-72344a91553a

# Lots of information
description:
purpose: Views built on top of the seller tables.
limitations: Data based on seller perspective, no buyer information
usage: Predict sales over time
authoritativeDefinitions:
- type: privacy-statement
url: https://example.com/gdpr.pdf
tenant: ClimateQuantumInc

kind: DataContract
apiVersion: v3.0.0 # Standard version (follows semantic versioning)
apiVersion: v3.0.1 # Standard version (follows semantic versioning)

# Infrastructure & servers
servers:
Expand All @@ -35,7 +38,7 @@ schema:
type: businessDefinition
- url: https://youtu.be/jbY1BKFj9ec
type: videoTutorial
tags: [ ]
tags: [ 'finance', 'payments']
dataGranularityDescription: Aggregation on columns txn_ref_dt, pmt_txn_id
properties:
- name: txn_ref_dt
Expand Down Expand Up @@ -74,7 +77,7 @@ schema:
partitioned: false
partitionKeyPosition: -1
criticalDataElement: false
tags: [ ]
tags: [ 'uid' ]
classification: restricted
- name: rcvr_cntry_code
primaryKey: false
Expand Down
6 changes: 4 additions & 2 deletions docs/examples/all/postgresql-adventureworks-contract.odcs.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
version: "1.0.0"
apiVersion: "v3.0.0"
id: "6aeafdc1-ed62-4c8f-bf0a-da1061c98cdb"
status: "Development"
status: active
kind: "DataContract"
description: {}
schema:
schema:
- name: "department"
physicalName: "department"
physicalType: "table"
Expand Down Expand Up @@ -82,6 +82,7 @@ schema:
criticalDataElement: false
primaryKey: false
required: false
tags: ['sensitive']
- name: "loginid"
logicalType: "string"
physicalType: "varchar[256]"
Expand Down Expand Up @@ -115,6 +116,7 @@ schema:
criticalDataElement: false
primaryKey: false
required: false
tags: ['sensitive']
- name: "maritalstatus"
logicalType: "string"
physicalType: "bpchar"
Expand Down
4 changes: 2 additions & 2 deletions docs/examples/data-types/all-data-types.odcs.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
version: 1.0.0
kind: DataContract
id: 53581432-6c55-4ba2-a65f-72344a91553a
status: current
status: active
name: my_table
dataProduct: my_quantum
apiVersion: v3.0.0
apiVersion: v3.0.1
schema:
- name: transactions_tbl
description: Provides core payment metrics
Expand Down
4 changes: 2 additions & 2 deletions docs/examples/fundamentals/table-column-description.odcs.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
version: 1.0.0
kind: DataContract
id: 53581432-6c55-4ba2-a65f-72344a91553a
status: current
status: active
name: my_quantum
apiVersion: v3.0.0
apiVersion: v3.0.1
schema:
- name: tbl
description: Provides core payment metrics
Expand Down
4 changes: 2 additions & 2 deletions docs/examples/quality/column-accuracy.odcs.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
version: 1.0.0
kind: DataContract
id: 53581432-6c55-4ba2-a65f-72344a91553a
status: current
status: active
name: my_table
dataProduct: my_quantum
apiVersion: v3.0.0
apiVersion: v3.0.1
schema:
- name: Air_Quality
description: Air quality of the city of New York
Expand Down
7 changes: 4 additions & 3 deletions docs/examples/quality/column-completeness.odcs.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
version: 1.0.0
apiVersion: v3.0.0
apiVersion: v3.0.1
kind: DataContract
id: 53581432-6c55-4ba2-a65f-72344a91553a
status: current
status: active
name: my_table
dataProduct: my_quantum
schema:
Expand All @@ -13,6 +13,7 @@ schema:
type: Reference definition on Data.gov
dataGranularityDescription: Raw records
physicalType: table
tags: ['nyc', 'regional']
properties:
- name: UniqueID
primaryKey: true
Expand All @@ -24,4 +25,4 @@ schema:
dimension: completeness
severity: error
rule: nullCheck
businessImpact: operational
businessImpact: operational
4 changes: 2 additions & 2 deletions docs/examples/quality/column-custom.odcs.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
version: 1.0.0
apiVersion: v3.0.0
apiVersion: v3.0.1
kind: DataContract
id: 53581432-6c55-4ba2-a65f-72344a91553a
status: current
status: active
name: my_table
dataProduct: my_quantum
schema:
Expand Down
4 changes: 2 additions & 2 deletions docs/examples/quality/column-validity.odcs.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
version: 1.0.0
apiVersion: v3.0.0
apiVersion: v3.0.1
kind: DataContract
id: 53581432-6c55-4ba2-a65f-72344a91553a
status: current
status: active
name: my_table
dataProduct: my_quantum
schema:
Expand Down
Loading

0 comments on commit f313988

Please sign in to comment.