Skip to content

Using Data Products

Overview

The ability to define, use and maintain data products is a very useful feature within MetaKarta. It is one of the primary features of the larger data mesh architecture approach. Depending on the nature of the data within a domain (known as domain data) and its consumption models, data can be served as events, batch files, JDBC relational tables, graphs, etc., while maintaining the same semantics (e.g, meaning and utilization).

Requests and Contracts and the Data Products Model

You must first create a data products model as a custom model in order to store and manage the data products, data domains, ports, usage requests, etc.

In this way,

  • A data products model may contain one or more data domains, and these data domains have a hierarchical structure with a particular data products model.
  • Each data model then may contain
  • One or more data products. Each data product may contain
    • One or more ports, which are data sources. Each port may be associated with any object in the repository and will therefore be represented by that object's metadata (e.g., table and column specifications, file format, etc.)
    • One or more usage requests
  • One or more data contracts, each of which may be associated with up to one data product

Create and Approve a Data Product Usage Request

Create a Data Product Usage Request

Once you have created a data product, you may add a usage request for it.

Steps

  1. Navigate to the model and click the Overview tab.
  2. Navigate to the Data Domain and Data Product in which you wish to make the request.
  3. Click the + CREATE to the right of the Usage Request section.
  4. Specify:
    • NAME of the request
    • DESCRIPTION
    • REQUESTER (pick the user making the request)
    • USAGE REQUEST STATUS (leave empty when requesting)
    • FROM and TO DATE (date range of the request)

Example

Sign in as the Bob user and Navigate to the model and click the Overview tab. Navigate to the Cloud DW Finance Customer data product in the Finance domain in the My Company Data Products. Click the + CREATE to the right of the Usage Request section.

Specify the values as above and click OK.

Approve a Data Product Usage Request

Once the data usage request is approved then that user has access to the data product metadata, scoring, etc.

Approval users, must have the Usage Request Workflow - Request Approving capability.

  1. Sign in as a user with at least the Usage Request Workflow - Request Approving capability object role assignment on the data products model in which you wish to make the usage request.
  2. Navigate to the model and click the Overview tab.
  3. Navigate to the Data Domain and Data Product to which the request was made.
  4. Click the request in the Data Product Request section.
  5. Specify:
    • NAME of the contract
    • DESCRIPTION of the details around the contract.

Example

Sign in as the Administrator user and Navigate to the model and click the Overview tab. Navigate to the Cloud DW Finance Customer data product in the Finance domain in the My Company Data Products. Click Bob's usage request in the Usage Request section.

Click APPROVE REQUEST to approve the request.

Click OK.

Create and Edit a Data Product Contract

Once you have created a data domain, you may add a data contract to it.

A data contract is a formal agreement between a data producer and its consumers that defines the data's structure, quality, and terms of use, while a data product is the packaged, read-only dataset itself, made available for consumption. Data contracts ensure data products are reliable and consistent by defining specifications for things like schema, quality metrics, and performance, which allows data products to be used with confidence across different teams and use cases.

Steps

  1. Sign in as a user with at least the Metadata Management capability object role assignment on the data products model in which you wish to make the usage request.
  2. Navigate to the model and click the Overview tab.
  3. Navigate to the Data Domain in which you wish to place the contract.
  4. Click the + CREATE to the right of the Data Contracts section.
  5. Specify:
    • NAME of the contract
    • DESCRIPTION of the details around the contract.

Example

Sign in as the Administrator user and Navigate to the My Company Data Products data products model Overview tab. Navigate to the Finance domain. Click the + CREATE to the right of the Data Contract** section.

Specify the values as above and click OK.

Full DESCCRIPTION:

Finance Data

Data Contract

Data Contract concerning the Finance domain data products.

Service Level Expectations:

  • update_frequency: "Daily"

  • update_time: "6:00 AM PST"

  • latency: "1 day (for withdrawal-related data)"

  • data_quality: "The customer_id and transaction_id fields must not be null or empty."