Skip to content

Data Source Access History

Data access history is available selected supported tools (such as Snowflake or Google BigQuery), the same way data sampling, profiling, and classification is also available. This feature is called "data source" access history in order to avoid any confusion with the access history of the same object (such as a table) by users of MetaKarta. The data source access history is also referred to as database table freshness and/or operational metadata. When available, the supported objects (such as Tables, Views, Stored Procedures, Functions) of these imported models have the following system attributes prefixed by Source History (as a short for Data Source Access History):

  • Source History Popularity Count as the total count of any data read/write

  • Source History Last Access Users as the list of users who read the data

  • Source History Last Access Date as the last data read access date (such as a select statement)

  • Source History Last Updated Date as the last data write access date (such as a insert or update statement). Note that this date can be newer than the last access date showing a table that is updated while no one is reading it.

All the above data source access history attributes are calculated by parsing system logs over a period of 7 days by default (such period can be adjusted in MANAGE > Schedules).

Note that the data source access history of some supported tools can be limited both in terms of supported objects and supported data source access History attributes. In other words, the import data source access history operation is implemented differently for each data source technology.

Details are provided for specific tools at Data Source Access History Definitions.

Import Data Source Access History

Data source access history is imported/populated separately from the model import operation. After you import a model from one of the supported source formats, you may then schedule an operation to run on that same model

Steps

  1. Sign in as a user with at least the Application Administrator capability global role assignment.

  2. Go to MANAGE > Schedules in the banner.

  3. Click Add.

  4. Choose the imported model from the repository on which you will schedule the data source access history operation.

  5. Specify Data Source Access History as the Operation.

  6. Specify the number of days to go back in the history to collect the information.

  7. Click SAVE.

Example

Sign in as Administrator and go to MANAGE > Schedules. Pick the Tableau model.

Specify Data Source Access History as the Operation.

Click SAVE.

Right-click on the resulting schedule and select Run Operation Now.

Once completed, you will see the resulting data source access history properties.

View Data Source Access History

View Data Source Access History in Object Page

Sign in as Administrator and navigate to the Snowflake Dimensional DW model.

Expand to the MITI_SUPPORT database and FINANCWAREHOUSDIMENSIONAL schema.

The Source History Popularity Count is presented as a percentage of all the tables.

View Data Source Access History in Worksheet

Steps

  1. Go to WORKSHEETS > Snowflake > Tables.

  2. Add the Source History Popularity Count column to the results.

Example

Go to WORKSHEETS > Snowflake > Tables, add the four Source History columns to the results.

Data Source Access History Definitions

Snowflake Data Source Access History Definition

  • Source History Popularity Count as the count of any data read/write (over the past 7 days by default, see below)

Please refer to the ACCESS_HISTORY view for more details specific to Snowflake. We perform a select on the ACCESS HISTORY view where query_start_time = - Source History Popularity Count Period (number of days, 7 by default)

The history is determined based upon the last 7 days by default, but the period can be adjusted in the Source History Popularity Count Period MANAGE > Schedules. Snowflake itself keeps the usage records for the last 365 days.

  • Source History Last Access Users as the list of users who read the data (limited to the last 5 users)

  • Source History Last Access Date as the last data read access date (such as a select statement)

  • Source History Last Updated Date as the last data write access date (such as a insert or update statement).

This Last Updated Date can be newer than the Last Access Date showing for a table that is updated while no one is reading it.

Snowflake does not update its ACCESS HISTORY (data) in real time, so there may be a delay of up to 3 hours.

Tableau Data Source Access History Definition

  • Source History Popularity Count as the as the total number of accesses to a given worksheet since it has been created.

Please refer to the Get View REST API call for more details specific to Tableau.