Using the concept of web browser extensions, MetaKarta is capable of providing detailed documentation, cataloging, governance and lineage information directly within the web UI of third party business intelligence (BI) tool. This metadata may be made available for any third party BI software models you have harvested, stitched (into the data flow lineage) and documented.

You merely need to generate, download and then enable the extension in your web browser (Amazon Chrome is currently supported) and an icon with an optional side panel will be available within the third party software web UI.

Architecture

Teams bot consists of 3 major parts:

• Azure service “Azure bot” with Entra ID app. Receives all events from the Teams conversation and redirects them to the bot backend. Unique bot registration which is registered under MITI Azure organization.
• Bot backend. Located in the AWS(MITI-MM account, MMTM sandbox EKS cluster). Does the actual job. Publicly accessible from Azure.
• Bot installation. Bot could be installed for different organizations/tenants, Teams team/channels, users. Single backend distinguishes event context by tenant ID and user/conversation/channel ID.
• MM with configured “Teams integration”.

The Teams bot communicates with MM using HTTPS and WSS(WebSocket secure) protocols:

• MM server registration is HTTPS call from MM to Teams bot
• All ongoing messaging(bidirectional) uses WSS tunnel initiated by MM

General Tool Integration Options

The EXTERNAL URL used by outside tools to reference back to MetaKarta, e.g.,

• Browser extension call back URL to MetaKarta

• URL used to define links in notification emails back to MetaKarta

Please be sure to set the EXTERNAL URL before setting up the specific types of tool integration.

The EXTERNAL URL must be accessible from the external internet so the Teams agent can access the application server.

Steps

1. Go to MANAGE > Tool Integration.

2. Go to the General tab.

3. Enter the URL that is a good reference back to MetaKarta in MM EXTERNAL URL, see General Tool Integration Options

4. If you intend to set up tool integration with Microsoft Teams, you may click Yes under Microsoft Graph API and configure the graph API .

5. Click SAVE.

Example

Go to MANAGE > Tool Integration and go to the General tab.

Enter a URL that is visible from the external internet.

Set up the Browser Extension

To configure this capability as part of the broser, you must set the proper options to ensure the correct call back to MetaKarta works properly.

Steps

1. Sign in as Adminstrator.

2. Go to MANAGE > Tool Integration.

3. Go to the General tab.

4. Enter the URL that is a good reference back to MetaKarta in MM EXTERNAL URL, see General Tool Integration Options

5. Click SAVE.

6. Go to the Browser Extension tab.

7. Click Yes under ENABLE BROWSER EXTENSION.

8. Specify:

9. DEFAULT CONFIGURATION to pick the configuration containing the BI report metadata

10. DEFAULT OPEN MODE options - The choice of presentation in the browser UI when invoking the extension

    i. Tab - Present the repository product UI as an additional tab in the browser opened to the specific object being viewed in the custom tool

    ii. Side Panel- Present the repository product UI as a collapsable in the browser showing details for the specific object being viewed in the custom tool

    iii. Popup - Present the repository product UI as a popup dialog in the browser showing details for the specific object being viewed in the custom tool

11. USER DOWNLOADABE to allow users to download and install the extension in their own browsers.

12. CUSTOM TOOL DOMAINS - Itemized list of custom tools supported by the browser extension, the domain signatures for which the extension is enabled for activation, and the configuration under which the repository to be invoked..

13. Click SAVE

By default browser extension will use MetaKarta icons unless 2 customized icons are provided (BrowserExtensionIcon.png and BrowserExtensionInactiveIcon.png) in the $MM_HOME/conf/resources folder.

The customized icons will be copied at the extension download time from the conf folder if available.

Example

Sign in as Administrator Go to MANAGE > Tool Integration. Go to the Browser Extension tab.

Click Yes under ENABLE BROWSER EXTENSION.

Specify the fields as shown or specific to your example.

Now, for the tool specific settings.

Then click SAVE.

Setup Browser Extension for a Tool

Specify the fields as shown or specific to your example.

Set Up Email Notification

MetaKarta provides notification emails which may be sent to users based on his/her roles/capabilities, including:

• Workflow transitions on objects where the user has a Workflow Viewer capability object role assignment, as often as defined by the server or a user, as well as reminders after a specified number of days.

• Configuration changes (e.g. add/remove model, edit connections) as often as defined by the server or a user, or build errors (in real time) on configurations where the user has the Metadata Manager capability object role assignment to that configuration.

• Server errors (e.g. server down) in real time to the user with Application Administrator capability global role assignment.

In addition, one may subscribe to or be a watcher of models in the repository and thus receive notifications of changes to those particular models. One may also be notified after being mentioned in a HTML editor text field.

In order for any of the above to send email notification, you must first turn on SMTP notification and specify connectivity information.

In addition to the watchers of a model, the following are the recipient rules for different types of models:

• For configurations the model change notification is sent to the users who have the metadata management capability role on the configuration

• For harvested models and user models, the model change notification is sent to the users who are the watchers of the models

• The import failed and other operation failed notifications are sent to the users who have the metadata management capability on the model.

Checklist: In order to ensure that notification will be sent to watchers or based upon mentions of users, please ensure that the following conditions are true:

• The system has an email server configured (see below)

• The system notification settings have Enable Watcher Notifications set to Yes (see below)

• The user must have a valid email address specified

• The user's Notification Frequency is NOT set to Never

• The user has at least the Metadata Viewing capability object role assignment on the object.

Users who have a Near Real Time notification frequency get mentioned user notifications based on the BATCH TIME (IN MINUTES) (see below).

Notification may also be accomplished by using the REST API capability and operating system based scheduling software. E.g., in order to see security based audit event notification, there is a REST API call GET /admin/auditLog/listSecurityAuditEvents to allow you to download security audit events.

Specify the Connectivity Information for the Email Server

SMTP (Custom) Service Provider

For standard SMTP servers, the setup is as follows:

Steps

1. Sign in as a user with at least the Application Administrator capability global role assignment.

2. Go to MANAGE > Tool Integration.

   • Enter the URL that is a good reference back to MetaKarta in MM EXTERNAL URL, see General Tool Integration Options

3. Go to the Email Notification tab.

4. Pick Yes for ENABLE SMTP

User Information

• The SENDER ADDRESS to provide to the SMTP server what will appear as the sender in the e-mail notifications
• The REPLY TO ADDRESS to provide to the SMTP server what will appear as the reply-to address in the e-mail notifications. By default it is the same as the SENDER ADDRESS.

Server Information

• Specify Custom for the SERVICE PROVIDER.

Microsoft Graph is the gateway to data and intelligence in Microsoft 365. This product allows the user to send mail via the Microsoft Graph API. If the user wants to use Microsoft Graph to send mail they must specify the Microsoft 365 settings.

• The HOST NAME for this SMTP server

• The PORT to use when connecting to this SMTP server

• The USER NAME to use when connecting to the SMTP server

• The PASSWORD to use when connecting to the SMTP server

Notification Settings

• The ENABLE WATCHER NOTIFICATIONS to allow users to subscribe to or watch an object and thus receive email notifications of any changes to those objects notification due to mentions of themselves. In this case, you may specify how long a time a particular the object change email notification summary will cover.

• One then may also set the BATCH TIME which determines the meaning of Near Real Time notification

• The WORKFLOW OVERDUE PERIOD (IN DAYS) specifies the number of days after a workflow task is assigned to user before that user is sent an email reminder of the task.

A WORKFLOW OVERDUE PERIOD (IN DAYS) of "0" means no reminders.

1. Click SAVE.

Example

Explore Further

SSL/TSL

Email notification supports SSL/TSL for email servers. Note, the server detects SSL/TSL automatically and you do not need to explicitly request it.

Notification emails do not mention individual changes, like before and after attribute values. They inform the user about the total number of changed objects and sample object identifiers in a summary. Each email contains a hyperlink to the application page displaying recently changed objects.

You can get email notifications with the following frequency:

• Near real time

• Daily

• Never

For Daily notification, there is a scheduled job named Send scheduled notifications and the time of day when the notification occurs may be set there.

In order to send out notification of changes, operations or service interruptions, MetaKarta requires access to an e-mail server that provides SMTP access for sending mail.

Microsoft Graph as Service Provider

Microsoft Graph is the gateway to data and intelligence in Microsoft 365. MM allows the user to send mail via the Microsoft Graph API. If the user wants to use Microsoft Graph to send mail they must specify the Microsoft 365 settings.

Steps

1. Sign in as a user with at least the Application Administrator capability global role assignment.

2. Go to MANAGE > Email Notification in the banner.

3. Pick Yes for ENABLE SMTP

User Information

• The SENDER ADDRESS to provide to the SMTP server what will appear as the sender in the e-mail notifications

• The REPLY TO ADDRESS to provide to the SMTP server what will appear as the reply-to address in the e-mail notifications. By default it is the same as the SENDER ADDRESS.

The URL to link back to is set

Server Information

• Specify Custom for the SERVICE PROVIDER.

• The TENANT ID for this SMTP server

• The CLIENT ID to use when connecting to this SMTP server

• The CLIENT SECRET to use when connecting to the SMTP server

Notification Settings

• The ENABLE WATCHER NOTIFICATIONS to allow users to subscribe to or watch an object and thus receive email notifications of any changes to those objects. In this case, you may specify how long a time a particular the object change email notification summary will cover.

• One then may also set the BATCH TIME which determines the meaning of Near Real Time notification

• The WORKFLOW OVERDUE PERIOD (IN DAYS) specifies the number of days after a workflow task is assigned to user before that user is sent an email reminder of the task.

A WORKFLOW OVERDUE PERIOD (IN DAYS) of "0" means no reminders.

1. Click SAVE.

Example

Notification of Server Down

MetaKarta will send out an email notification when a harvesting server goes down only if you should specify the MAX MISSED HEARTBEATS for that server. Notification will be sent once the number of missed pings is reached, if email notification is configured properly.

All users who are assigned any roles that have the Application Administration global capability assignment will receive a notification email when the server goes down and MAX MISSED HEAERBEATS is reached.

Notification of Change

Data cataloging and governance are collaborative efforts. You often need to collaborate with your teammates on the meaning, certification, or disposition of objects in the catalog. It is important that you can share what is happening to these objects in order to keep your team up to date. In addition, you may also need to keep up with the changes to objects you may not be directly working on. With object change notification, you can receive notifications of changes to objects.

MetaKarta provides:

• Watcher capabilities at allowing a user to subscribe or watch models and thus be notified of changes to objects in those models

• Workflow transition notification feature on objects in custom models with workflow enabled so that when a user has a Workflow capability object role assignment they will be notified of changes in the workflow state of those objects.

Notification of Model Change (Watchers)

Data cataloging and governance are collaborative efforts. You often need to collaborate with your teammates on the meaning, certification, or disposition of objects in the catalog. It is important that you can share what is happening to these objects in order to keep your team up to date. In addition, you may also need to keep up with the changes to objects you may not be directly working on. With object change notification, you can receive notifications of changes to objects.

MetaKarta provides watcher capabilities for model level objects allowing a user to effectively subscribe to objects and be notified of changes and there is an adjustable frequency of notification from near real-time to daily.

One may set the watcher notification frequency (of a given user) from daily to near real time (as setup at the server level). Note that the same watcher frequency can also be setup by each individual user on their top right menu for user profile / preferences.

The watcher capabilities are supported on both imported models and custom models. The watcher capabilities are available at the model level only (e.g. not down to just a column or a term object). In case of imported models harvested as multi-model, one can watch the entire multi-model (e.g. entire database server), or individually watch any desired sub-model such as a given schema of database or report in a BI tool.

Watchers of imported models receive a separate email per model and per type of activity as follows:

• Any metadata harvesting driven changes at any level (e.g. add/delete/update of any schema/table/column/type) as soon as (in real time) an import (incremental harvesting) is successful with changes, or failed. In such case, the watcher notification email includes change summary statistics (e.g. number of added, deleted, updated objects), and an MM server URL link to its model version comparator report for full details.

• Any other changes such as data documentation (e.g. business name, description, or term classification), social curation, etc. at any lower level (e.g. table, column, data type) as often as defined by the server or a user. In such case, the watcher notification email includes a change summary statistics (e.g. number of changed objects), the top 5 changed objects (with a MM server URL link to the overview page of that object), and finally the detailed changes (with a MM server URL link to the search UI filtered by the content of that model and ordered by last modified),

Watchers of custom models, Data Mapping, Semantic Mapping, Physical Data Models receive a separate email per model on any change at any level.

Any changes at any level (e.g. add/delete objects, update attributes, add/delete relationships, etc.) as often as defined by the server or a user. In such case, the watcher notification email includes change summary statistics (e.g. number of changed objects), the top 5 changed objects (with an MM server URL link to the overview page of that object), and finally the detailed changes (with an MM server URL link to the search UI filtered by the content of that model and ordered by last modified).

These same users will be notified for any data profiling and sampling errors in the following circumstances:

• The data profiling and sampling was invoked by an Import new version of model(s) operation that was invoked according to a schedule

• The data profiling and sampling was invoked by an Import model operation that was manually started from a user session and the user session has expired (either the user was logged out or the user session timed out)

In order to receive change notifications, one must have:

• Configure SMTP settings for an email server

• Enable Watcher Notifications

• The user must have a valid email address specified

• The user must be added as a watcher of the model

Example

The email notification messages use the Product Title set in the MM.properties file by the property name miti.mimm.title. For example you can set

miti.mimm.title=MyCompany

The MM.properties file is located in /conf/resources. It you don't see the file there you can copy the file from /conf/Template/resources.

Notification of Workflow Transition

In order to receive notification of workflow changes, one must have:

• Configure SMTP settings for an email server

• The user must have a valid email address specified

• The user must have workflow responsibilities for the object

• Personal Notification Frequency must be set.

In addition, in order to receive reminder notifications, you must:

• All the above

• A non-zero WORKFLOW OVERDUE PERIOD (IN DAYS)

Example

Sample email notification message:

My Workflow Tasks dashboard:

Sample email reminder message:

Notification of User or Group Mentions

One may also schedule notifications be sent to user and groups after being mentioned in a HTML editor text field.

Example

Set up just as for Watcher notification.

Make a mention of Bob in the Customer PO Invoice Item term.

Go to MANAGE > Schedules and run the operation:

The notification is emailed to Bob:

****	****	****	****

Hi Bob,

You have been mentioned in 1 objects since 2024-04-29 13:02:17.

Here are some of the objects where you were mentioned:

• Customer PO Invoice Item (Term) from Finance > Terminology

Please review them to find out details.

Notification Types

Notification Type	Recipients	When a notification is sent
Model changes (imported model, custom model, Data mapping, Semantic mapping, PDM)	Users who are model watchers	User’s notification frequency
Configuration changes	Users who have Metadata Management capability on the configuration	User’s notification frequency
Custom model workflow transitions	Users who have a workflow capability on the term	User’s notification frequency
Custom model workflow transitions reminder	Users who have a workflow capability on the term that requires an action (workflow task) and the WORKFLOW OVERDUE PERIOD has passed since the last change to that object.	Every day after the WORKFLOW OVERDUE PERIOD has passed without an update.
Imported model structural changes	Users who are model watchers	As soon as import succeeds bringing changes
Import failed	Users/groups who have Metadata Management capability on the model	As soon as import fails (import is started by a schedule or user session has expired)
Data sampling and data profiling failed during an import	Users/groups who have Metadata Management capability on the model	As soon as the data sampling and data profiling operation fails (import is started by a schedule or user session has expired)
Configuration build failed	Users/groups who have Metadata Management capability on the configuration	As soon as the configuration build fails (build is started by a schedule or user session has expired)
Import server down	Users/groups who have the Application Management capability (global)	As soon as the server is down and the number of heartbeats defined in the server is reached
Mentioned user or group	Users who have the Metadata Viewing capability on the objects and are in an HTML text attribute for that object	User’s notification frequency

Set up Jira Integration

You may integrate objects in the repository with the Jira based Atlassian product in order to create, link, and track issues in Jira with two way connectivity between MetaKarta and Jira.

Steps

1. Sign in as Adminstrator.

2. Go to MANAGE > Tool Integration.

3. Go to the General tab.

   • Enter the URL that is a good reference back to MetaKarta in MM EXTERNAL URL, see General Tool Integration Options

4. Click SAVE.

5. Go to the Jira tab.

6. Click Yes under ENABLE JIRA INTEGRATION.

7. Specify:

   • SERVER URL - the URL of the Atlassian Jira.

   • USER NAME - Valid Jira user name used to authenticate with Jira. This user must have Automation permissions within the Jira environment.

   • PASSWORD -- valid password to match the user name.

8. Click TEST CONNECTION

9. Specify

   • DEFAULT PROJECT -- Default proposed Jira project for the tickets/issues in JIRA with which objects be integrated with. Only projects visible to the user name above will be possible.

   • DEFAULT ISSUE TYPE: Default type when creating new tickets/issues in Jira.

10. Click SAVE

Example

Sign in as Administrator Go to MANAGE > Tool Integration. Go to the Browser Extension tab.

Click YES under ENABLE JIRA INTEGRATION.

Specify the fields as shown or specific to your example.

Set Up Teams Integration

MetaKarta provides Microsoft Teams messenger integration using a bot. Once configured, the integration provides:

• From Teams:

• MM objects search

• MM object preview (link unfurling)

• From MetaKarta:

• Personal notifications

• Teams Post creation

• Linked objects (repository object <-> Teams object)

• First the bot app should be branded and installed into Teams; then, integration should be enabled in MetaKarta.

These steps are outlined in MANAGE > Tool Integration > Teams tab:

Teams Setup Requirements

By default the Teams user doesn't have permissions to install the bot app from the package. Bot installation and setup should be done either by the Teams Administrator or a user with the proper permissions. There are 2 Teams messenger versions ("old one" and a "new one" starting from 2023/10/05). Some functionality might not work in the "old Teams" version. So it is a recommended to use the new one.

Teams Bot Branding

You may perform Teams bot customization in terms of the "branding" of the bot. You can change the bot name, short description, long description and icon.

To customize the bot::

1. Go to MANAGE > Branding

2. Go to the Teams App tab

Teams Setup

The bot app should be downloaded to be installed later in Teams:

1. MANAGE > Tool integration

2. Go to the Teams tab

3. Click the DOWNLOAD TEAMS APP button

A Save As dialog box is presented to save the bot app as a file.

To manage user groups permissionsthe manager user should have Application Administrator capability global role assignment.

Install the Teams Bot App

Now, the previously downloaded bot should be installed into Teams as a Team/channel context".

1. In Microsoft Teams go to Apps

2. Click Manage your apps

3. Click Upload an app

1. Click "Upload a custom app"
2. the package (file you downloaded earlier)
3. Select Team/channel where install to

After installation the bot will send a "welcome message" with the "secret token" to be used by MM.

The Secret token has an expiration time of 1 minute, so have everything ready for the next step. If it expires, the bot must be deleted (Microsoft Teams Apps -> Manage your apps -> select bot app -> remove) and then installed again.

Command box search requires the bot to be installed for personal (1 to 1) chat.

If you see the “personal chat welcome message”:

Welcome in MIMM chat bot.

Type 'help' to see debug commands.

This message is shown by the bot to a user only if the bot was installed in personal (1 to 1) chat.

To get a secret token user should install the bot into a team/channel. Please see Step (6.) above.

If the bot installed with a team/channel selected different team/channel welcome message will be shown. For example:

Application Server-Setup

The correct MM EXTERNAL URL should be set (to match actual DNS/hostname of the application server.

To enable "Teams integration":

1. Go to MANAGE > Tool integration

2. Go to the Teams tab

3. Click Enable Teams

4. Enter the secret token copied from Teams (in the bot welcome message)

After that user will see the installed Team name, selected "default channel" and organization/tenant name (optionally if Graph API integration configured) showing that connection has been made.

Graph API setup

The Graph API may be enabled to assist with the full Teams integration. If enabled it will be used to:

• Query organization/tenant name

• Query team channels list

• Query Teams link title or summary

This setup requires an Azure "Entra ID" app registration with the following permissions:

• Get org name: "Organization.Read.All"

• Select team channels list: "Channel.ReadBasic.All"

• Search post subject/message(post): "ChannelMessage.Read.All"

To enable the Graph API the user should first:

1. Go to MANAGE > General

2. Enable the Microsoft Graph API with "yes"

3. Enter the TENANT ID, CLIENT ID, CLIENT SECRET from the Azure app (Entra ID).

4. Click TEST.

5. Click SAVE.

Then one should:

1. Go to MANAGE > Tool integration

2. Go to the Email tab

3. Click yes for Enable SMTP

4. Enter User Information -> SENDER ADDRESS -> any email (won't be used)

5. Enter Server Information -> SERVER PROVIDER -> Office 365

To enable "Enable Teams" in the MM, MM user should haves "Application administrator" capability(e.g. from the "Application administrator" global role).

[microsoft-graph-as-service-provider]: