Meta Integration® Metadata Management (MIMM)
"Metadata Management" Solution

README for Release Notes, Installation & Setup

Table of Contents

  1. Overview
  2. Copyright Notice
  3. Release Changes
  4. System requirements
  5. Database Server Setup
    1. Database on Oracle
    2. Database on Microsoft SQL Server
    3. Database on PostgreSQL
  6. Application Server Setup
    1. Application Server Installation and Configuration
    2. Application Server Upgrade
    3. Application Server Execution and Initialization
    4. Custom integration with Authentication Environments
    5. Custom integration for Secure Socket Layer (SSL) communication
    6. Security and Vulnerability Considerations
    7. Lucene Search Engine Troubleshooting
    8. High Availability Considerations
  7. Metadata Harvesting Model Bridge (MIMB) Setup
  8. User Interface Look & Feel Customization
    1. Login and Headers
    2. Metadata Explorer for Business Users
  9. REST API SDK
  10. Database Server Backup/Restore
    1. Database on Oracle
    2. Database on Microsoft SQL Server
    3. Database on PostgreSQL

1. Overview

The Meta Integration® Metadata Management (MIMM) Application Server is
based on the Meta Integration® Repository (MIR) for metadata storage (in a database server),
and the Meta Integration® Model Bridge (MIMB) middleware for metadata harvesting.

2. Copyright Notice

Copyright © Meta Integration Technology, Inc. 1997-2023.
All Rights Reserved.
Meta Integration® is a registered trademark of Meta Integration Technology, Inc.
Other product and company names (or logos) mentioned herein may be the trademarks of their respective owners.
http://www.metaintegration.com

3. Release Changes

v11.2.0 (v11 2023 Winter update scheduled for 10/31/2023)

 

v11.1.0 (v11 2023 Summer update scheduled for 05/31/2023)

 

v11.0.0 (01/31/2022 with above continuous deployment new feature updates)

 

v10.1.0 (GA 01/31/2020 - Deprecated 12/31/2021 - EOL 12/31/2022)

 

v10.0.1 (GA 04/24/2019 - Deprecated 12/31/2020 - EOL 12/31/2021)

 

v10.0.0 (GA 09/21/2018 - Deprecated 09/21/2020 - EOL 12/31/2020)

 

v9.1.0 (GA 04/24/2017 - Deprecated 12/31/2018 - EOL 12/31/2019)

 

v9.0.2 (GA 06/10/2016 - Deprecated 12/31/2017 - EOL 12/31/2018)

 

v9.0.1 GA (12/15/2015)

 

v8.0.3 GA (05/19/2015)
v8.0.2 LA (03/31/2015)

 

v8.0.1 (12/02/2014)

 

v8.0.0 (LA 10/01/2014)

 

v7.2 (11/01/2013)

 

v7.1 LA (04/05/2013)

 

v7.0.4 LA (10/12/2012)

 

v7.0.2 GA (1/31/2012)

General Availability for complete Metadata Management

 

v7.0.1 LA (10/6/2011)

Limited Availability and Limited Functionality for data modeling tool management

 

v6.2 (10/15/2010)

 

v6.0.6 (12/04/2009)

 

v6.0.5 (MIMB GA, MIMM GA) (09/28/2009)

 

v6.0.4 (MIMB GA, MIMM beta5) (06/02/2009)

 

v6.0.3 (MIMB GA, MIMM beta4) (01/28/2009)

 

v6.0.2 (MIMB GA, MIMM beta3) (10/31/2008)

 

v6.0.1 (MIMB GA, MIMM beta2) (07/22/2008)

 

v6.0.0 (MIMB GA, MIMM Beta1) (05/31/2008)

 

4. System requirements

4.1 Important preliminary disclaimer notice on all requirements

The following requirements only define the minimal requirements to run the application server with reasonable performance based on the provided tutorial, or small business use cases. The actual requirements for enterprise wide use cases based on larger models and configurations do require significantly greater resources to obtain acceptable performance.

The following requirements are based on:

Any other hardware/software configurations are acceptable as long as they provide the same (or better) results on the provided performance benchmark. In such case, if any problem is discovered (e.g. scalability or performance issues), then the customer must be able to reproduce the issue using an environment that conforms to the minimum performance requirements as defined herein.

Potential known issues include (but are not limited to) the following:

4.2 Web Client requirements

Users only need an internet browser:

4.3 Application Server Requirements

Hardware Minimum Requirements (based on physical hardware performance, not a virtual environment):

However, the Hardware Recommended Requirements for a full enterprise application server suggests at least 64 cores and 300 GB of memory, as can be calculated below:
                   
    Apache Tomcat Application Server Configuration  
    Active Concurrent Users see (1) Memory (GB)
see (4) 		CPU (cores) Database Conection
see (5) 		 
    Per User Total Per User Total Per User Total  
  Light Users see (2) 200 0.1 20 0.1 20 0.2 40  
  Heavy Users see (3) 100 2 200 0.25 25 1 100  
  Server Database caching   10          
  Server Search index caching   10          
  Extra safety buffer 25% 55 25% 11.25 25% 35  
        295   56.25   175  
                   
  (1) Active Concurrent Users during peak daytime (among thousands of potential users on SSO / LDAP), and assuming all metadata harvesting and indexing is scheduled over night  
  (2)  Light users are typical business / end users performing search, browse, report, review, comment  
  (3) Heavy users are typical advanced developers performing editing, mapping and complex lineage.  
  (4)  Memory (GB) as configured in $MM_HOME/tomcat/conf/tomcat.properties or with the $MM_HOME/Setup.sh utility.  
  (5) Database connections as configured in $MM_HOME/tomcat/conf/MetaIntegration/localhost/MM.xml  
                   

Operating System Requirements:

Application Server Engine Requirements:

Java Runtime Environment (JRE):

4.4 Database Server Requirements

Hardware Minimum Requirements (based on physical hardware performance, not a virtual environment):

For small deployments (or quick proof of concepts), the MIMM software package bundles PostgreSQL (for Windows only) as the MIMM Database Server (which can run on the same machine as the MIMM Application Server) See Application Server Installation and Configuration setup for details.

The MIMM Database Server can reuse your existing Oracle, SQL Server, or PostgreSQL server:

Database Administrator privileges are required to install/setup/uninstall the database.

In general, one must ALWAYS install the latest service packs for a given database version BEFORE creating the MIMM database. E.g., for Oracle 11.2 one is required to apply the patches to upgrade to 11.2.0.3, or whatever is the latest patch level at the time. In addition, Oracle 11.2.0.4 must have patch 17501296 applied.

Virtual Memory: For a Windows based database server, be sure to either:

Thus, you must have more than that much free disk space (at least 3 time the amount of memory or RAM) on the drive where the page file is defined to reside.

5. Database Server Setup

The MIMM Application Server requires the connection to an existing Database server for metadata storage (metadata repository)

However, a quick install for tests or QA purpose can be achieved by using the bundle PostgreSQL database.
See the section Metadata Management (MIMM) Application Server Setup for more details.

The following database setup scripts and instructions assume the following by default:
   Database Name: MM
   Database User: MM
   Database Password: = MM123! The database name and user name can be changed, and the password should of course be different.

After the product is fully installed and web connectivity has been made, one may connect to a different database by way of the web based user interface at Tools -> Administration -> Database.

5.1 Database on Oracle

Create a user MM and a database MM with the following privileges:
sqlplus.exe SYS@<DB-NAME> as SYSDBA
 
   -- Delete previous user and database if needed
   -- DROP USER MM CASCADE;
 
   CREATE USER MM IDENTIFIED BY MM123!;
 
   GRANT CONNECT TO MM;
 
   GRANT CREATE TABLE TO MM;
   GRANT CREATE VIEW TO MM;
   GRANT CREATE SEQUENCE TO MM;
   GRANT CREATE TRIGGER TO MM;
   GRANT CREATE PROCEDURE TO MM;
   GRANT CREATE TYPE TO MM;
 
   GRANT EXECUTE ON DBMS_LOB TO MM;
 
   -- If you get the error "Database exception occurred: ORA-01950: no privileges on tablespace 'USERS'"
   -- ALTER USER MM QUOTA UNLIMITED ON USERS;

5.2 Database on Microsoft SQL Server

5.2.1 Database Requirement 1 - Case Insensitive

The database must be configured to interpret SQL in a case insensitive manner.

The case insensitive collation must be Latin1_General_CI_AS.

5.2.2 Database Requirement 2 - Mixed-Authentication Mode

The Mixed-Authentication Mode is usually set during the SQL Server installation process.

The Mixed-Authentication Mode can be verified or changed by using the SQL Server Management Studio: first sign in, then right click on the root of the tree (instance of SQL Server Express), go to Security, and finally select "SQL Server and Windows Authentication mode"

5.2.3 Database Requirement 3 - TCP/IP Protocol Enabled

The TCP/IP Protocol must be enabled in the SQL Server Configuration Manager for both the named instance and the client protocols (Make sure you restart the service after changing).

5.2.4 Database Requirement 4 - Login must be Database Owner

The database login (e.g., MM) that will connect MIMM to the SQL Server database must be the owner of the database.

5.2.5 Database Requirement 5 - SQL Common Language Runtime (CLR) Strict Security for SQL Server 2017 or newer

All database intensive operations (not just database maintenance, but also trace lineage, delete models, etc.) are implemented in SQL Server by stored procedures written in C# compiled/delivered in a stored procedure assembly called MIRRepo which has been signed that will be created with permission set SAFE.

If you are using SQL server 2017 or newer, then CLR Strict Security is enabled by default. Therefore, the certificate used to sign the stored procedure assembly must be imported in the database and granted the UNSAFE assembly permission (See Microsoft SQL Docs on CLR strict security) using the following commands:
   CREATE CERTIFICATE MIRRepoCert FROM BINARY = 0x308203663082024ea00302010202045eece216300d06092a864886f70d01010b05003075310b30090603550406130255533113301106035504080c0a43616c69666f726e69613116301406035504070c0d4d6f756e7461696e2056696577312a3028060355040a0c214d65746120496e746567726174696f6e20546563686e6f6c6f67792c20496e632e310d300b06035504030c044d6d4462301e170d3230303630313037303030305a170d3330303630313037303030305a3075310b30090603550406130255533113301106035504080c0a43616c69666f726e69613116301406035504070c0d4d6f756e7461696e2056696577312a3028060355040a0c214d65746120496e746567726174696f6e20546563686e6f6c6f67792c20496e632e310d300b06035504030c044d6d446230820122300d06092a864886f70d01010105000382010f003082010a0282010100c2ccf729a28a90958f71a68f6acca9f20b5c256b7c76565b2ece0cd1789bec85e9ab538ac38dc268e48c10e17d3eca1aeb14034bc67bafc05475ed013495aada683c74885f12a8bdbf2025ec3c5a0172010e7055ab27a853e77611ee6ae846453702d18ae3080977ddaee50a282b9dab3f077fe1630804b24f05c58280621dc1426fff7115e8a791435687096c09f754608bb9a6ce00002f7131f09cffd417678bddb8f7a703e4e688f2f0af501c52ecef2cbea3d37c45da4239ddb53295adaddb11dc0118b3188adf812c983d5676c5b7356d68e2258ea32cd3216db21dae49df16d2aa1aef39c618e393ce7e1b131b241c557414424fb6c17c825022a5a4270203010001300d06092a864886f70d01010b05000382010100a1db34a6cda0729a796e5ed0fe5b2f4813ff74bf96300c9ca30fb84be44bd7d0bc46c96a0726eae5e829985429ff4ff09b50ece907c5b8c7f8a71f7a16781103d7eaf2e1c7afa39e4774293610e0d04e6b0c76dc9a85891e6f5fed09059960dc7e2a7c1dc14d64aab9718747752d394b22e339da2c7e6ced1626dde991818cbcaf049d8f112a98b2aa2e80d1168f797a6c992e304e4572b4edcf40d270a281f82d7bde64e8d8b5d83574ecf5470f3d1a9d710498e133e9309a043f63b1682972678fba2a33267999795b5d040524e2f875b667dcec08d310e27b6086b2667dde70d4401fe501944f70581e559d5f3f5b72e49ff722e58594b84a8d15d5dd1414;
   CREATE LOGIN MIRRepoCertLogin FROM CERTIFICATE MIRRepoCert;
   GRANT UNSAFE ASSEMBLY TO MIRRepoCertLogin;

Alternatively, you can disable the SQL Server CLR Strict Security as follows:
   EXEC sp_configure 'show advanced options', 1;
   RECONFIGURE;
   EXEC sp_configure 'clr strict security', 0;
   RECONFIGURE;
   EXEC sp_configure 'show advanced options', 0;
   RECONFIGURE;

5.2.6 Database Preparation

Login to SQL server as a user with server admin role and execute the following commands to create a database "MM" and a user "MM" with password "MM123!" (or another one):

Enable clr, and create the database and user:
   EXEC sp_configure 'clr enabled', 1;
   RECONFIGURE;
   Go
 
   CREATE LOGIN MM WITH PASSWORD = 'MM123!';
   CREATE DATABASE MM;
   ALTER DATABASE MM SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
   ALTER DATABASE MM SET READ_COMMITTED_SNAPSHOT ON;
   ALTER DATABASE MM SET MULTI_USER WITH ROLLBACK IMMEDIATE;
   ALTER AUTHORIZATION ON DATABASE::MM to MM;

Warning: The product relies on one assembly (named MIRRepo) which is loaded from binary and not from file. This binary is created with the SAFE permissions. So in addition to being the database owner, the MM user should be granted the CREATE ASSEMBLY permission.

5.2.7 Database Connection

Advanced SQL Server Administrators may define ("hard-code") a set of TCP/IP ports for SQL Server to run over the network. However, Microsoft now recommends to run the "SQL Server Browser" service which can be done either in the Services panel or the SQL Server Configuration Manager.

For more information, read: How to: Configure Express to accept remote connections

The connection string syntax is:
   jdbc:sqlserver://<dbServer>:<dbPortNumber>;databasename=<dbName>

To connect to a named SQL server instance other than the default:

To connect to SQL Server using domain account:

  1. Find the mssql JDBC driver under $MM_HOME/java/jdbc/mssql, e.g. mssql-jdbc-7.4.1.jre11.jar.
  2. Download a Microsoft JDBC driver for SQL Server with the same version, e.g. 7.4, and extract the content. There you will find a sqljdbc_auth.dll from Microsoft JDBC driver x.x for SQL Server\sqljdbc_x.x\enu\auth\x64, and a mssql-jdbc-x.x.x.jre11.jar from Microsoft JDBC driver x.x for SQL Server\sqljdbc_x.x\enu
  3. Copy the sqljdbc_auth.dll to $MM_HOME/bin and replace the mssql-jdbc-x.x.x.jre11.jar under $MM_HOME/java/jdbc/mssql with the one from Microsoft JDBC driver x.x for SQL Server\sqljdbc_x.x\enu
  4. At the Configure Database Connection window add the string ;integratedSecurity=true at the end of the Database url, such as jdbc:sqlserver://localhost:1433;databasename=MM;integratedSecurity=true. Specify other fields and click TEST CONNECTION.

Note 1: The default database instance name for SQL Server Express is "sqlexpress, and "sqlserver" for any other SQL Server edition.

Note 2: The default SQL Server TCP/IP port number is 1433.

5.3 Database on PostgreSQL

Login to an existing database as a database superuser or a user who has CREATEROLE and CREATEDB privileges
psql.exe -h <HOST-NAME> -W -U <USER_NAME> -p <PORT> -d <DATABASE_NAME>
 
   -- Delete previous user if needed
   -- DROP USER "MM";
 
   -- If the user cannot be dropped due to any ownership issues, you'll need to reassign those objects to another user
   -- REASSIGN OWNED BY "MM" TO <OTHER-USER-NAME>;
   -- Or drop those objects
   -- DROP OWNED BY "MM"
 
   -- Create a user MM with LOGIN privilege
   CREATE USER "MM" LOGIN PASSWORD 'MM123!';
 
   -- Create a database MM with UTF8 encoding.
   CREATE DATABASE "MM" WITH OWNER "MM" ENCODING 'UTF8';

Note: For maintenance reasons, PostgreSQL database indexes can be rebuilt as follows:

For more information on reindexdb, refer to the PostgreSQL documentation

6. Application Server Setup

6.1 Server Installation and Configuration

The MIMM Application Server is installed as follows:

If your are using an existing database and do not wish to customize the application server (e.g. memory allocation, Windows services), then you can skip this step and go directly to the section on Application Server Execution and Initialization

Otherwise, go to the software home directory and execute the $MM_HOME/Setup.sh utility. Note On Windows, you must "Run as administrator" the %MM_HOME%\Setup.bat utility. This setup utility will allow you to setup the configuration parameters defined below through a user friendly application. After any change on any panel (tab) below, remember to press the Configure button in order to perform the configuration changes. A dialog box will be issued to confirm success or failure (with error messages).

Alternatively, this setup utility also works at the Windows command line or Linux shell, use the -? the options:
   [{ -? | --? | /? | --help }] Asking for help
   [{ -tp | --tomcat-port }] Tomcat: set the port number
   [{ -tm | --tomcat-memory }] Tomcat: set the maximum memory in Mb
   [{ -ta | --tomcat-agent }] Tomcat: switch to metadata harvesting only
   [{ -ts | --tomcat-service }] Tomcat: install or remove tomcat as a service
   [{ -s | --ssl }] SSL: enable or disable ssl
   [{ -sc | --ssl-cert-file }] SSL: the X.509 file containing the SSL certificate
   [{ -sk | --ssl-key-file }] SSL: the X.509 file containing the SSL key
   [{ -sp | --ssl-key-password }] SSL: the password for the SSL key file
   [{ -sr | --ssl-root-chain-file }] SSL: the X.509 file containing the SSL certificate chain
   [{ -ds | --database-service }] Database: install or remove database as a service
   [{ -dp | --database-port }] Database: set the port number
   [{ -dc | --database-connection }] Database: set the connection string
   [{ -du | --database-user }] Database: set the connection user
   [{ -dw | --database-password }] Database: set the connection password
   [{ -da | --database-auto-upgrade }] Database: automatically run the upgrades on startup
   [{ -mh | --mail-host }] Email: set the mail server host name
   [{ -mp | --mail-port }] Email: set the mail server port number
   [{ -mu | --mail-user }] Email: set the mail user name
   [{ -mw | --mail-password }] Email: set the mail user password
   [{ -ms | --mail-sender }] Email: set the mail sender's address
   [{ -mx | --mail-external-url }] Email: set the external URL to use in email templates
   [{ -ma | --mail-admin }] Email: a comma separated list of emails that will be notified of startup failures
   [{ -ep | --erwin-product }] Erwin: the edition to switch to: EWP or EDG
   [{ -op | --oracle-product }] Oracle: the edition to switch to: OEMM or OMM4OBI
   [{ -ch | --certificate-host }] Certificate: the host name to retrieve the certificate for
   [{ -cp | --certificate-port }] Certificate: the port number to connect to
   [{ -we | --webapp-enable }]... WebApp: enable the specified webapp (MMDoc)
   [{ -wd | --webapp-disable }]... WebApp: disable the specified webapp (MMDoc)
Some of these MM setup values (and other ones) can also be defined in $MM_HOME/conf/mm.conf.properties.

6.2 Application Server Upgrade

6.2.1 Understanding the Data Locations
Most application data is obviously located inside your database server, you are responsible for regular backup of such database. Upgrading your application may also upgrade the associated database content (database schema, stored procedures, indexes and of course data). It is important to understand to understand that part of the MIMM software is implemented as database stored procedures like tracing the lineage. Therefore a given version of MIMM corresponds to a version of the MM software in the application server (tomcat) and a version of the MM software in the database (e.g. tomcat) Therefore make sure you always backup your database before any upgrade.

Furthermore, the upgrade process may take several hours (on large repositories) and also need extra space for temp data during the migration. Therefore make sure the database has at least 20% free space.

Finally, it is also important to understand that the software installation directory (known as $MM_HOME in this document) also contains some critical application data and application setup customizations that have to be taken into account in your backup or upgrade process, including:

6.2.2 Upgrade Process
We recommend the following upgrade process:
  1. Stop your MIMM application server in the same way it was started such as stopping the Windows services / Linux daemon, stopping the desktop command windows, or using the tomcat/bin/shutdown. Remove the application server service if one was created (Windows only).
  2. Stop your MIMM database server and ONLY if you are using the bundled PostgreSQL database on Windows then use the $MM_HOME/Setup.sh utility to remove the Database Service as follows: Go to the Database Server tab, enable disable or uncheck the database service check box, and click on the Configure button.
  3. Backup your MIMM data including your database and data file directories as explained above.
  4. Backup your MIMM software by copying the $MM_HOME directory as $MM_HOME_OLD.
  5. Install the complete MIMM new software (ONLY needed for clean install of a new version) by deleting the old $MM_HOME, and then creating a new one by unzipping the new MIMM full package.
  6. Apply the latest MIMM software cumulative patch by unzipping it from inside the new $MM_HOME directory.
    • WARNING 1: Make sure you unzip with overwrite on Windows, and use unzip -u on linux to update files while retaining permissions.
    • WARNING 2: You cannot use cumulative patches for major version upgrades, you must first start from a clean install of the new major GA version.
    • WARNING 3: You cannot reverse / unzip an older cumulative patch, you must restart from a clean install of the original GA version.
  7. Restore your MIMM data and software customization/setup (ONLY needed for a clean install of a new version) by copying the appropriate files and directories (as previously explained) from $MM_HOME_OLD to $MM_HOME, including at least $MM_HOME/data and $MM_HOME/conf/conf.properties but possibly more as used and customized such as $MM_HOME/conf/resources, or $MM_HOME/tomcat/conf.
  8. Integrate the MIMM new software features in your configuration by copying potential new versions of files from $MM_HOME/conf/Template into their matching directories in $MM_HOME/conf/. For example the new $MM_HOME/conf/ModelBridgeList.xml may contain some new or updated bridges. WARNING: if you had customized some files such as $MM_HOME/conf/resources/MM.properties, you must re-apply/merge such customization starting from the new version of that file copied Template.
  9. Reconfigure your MIMM Database Server (ONLY if you are using the bundled PostgreSQL database on Windows). As the PostgresSQL server software version may have changed and the database server needs to be upgraded. Therefore, you must:
    1. Execute the (old renamed installation) $MM_HOME_OLD/Setup.sh utility to restore the old Database Service as follows: Go to the Database Server tab, enable the database service check box, and click on the Configure button.
    2. Execute the (new installation) $MM_HOME/Setup.sh utility as follows: go to the Database Server tab, enable the database service check box, and click on the Configure button. At this point the $MM_HOME/Setup.sh utility will retrieve the existing PostgreSQL data from the old install directory (which was already running as a service), will migrate them to the new install directory, and will remove the old PostgreSQL service, before starting the new PostgreSQL service on the new directory.
  10. Restart your MIMM database server (ONLY if you are using the bundled PostgreSQL database on Windows).
  11. Restart your MIMM application server after which your first login as Administrator may prompt you for an upgrade of your MM database.
  12. Redo all above steps for any other MIMM Application server configured as MIMB harvesting agent.
  13. Update your MIMM repository content (ONLY as needed) if the upgrade contains new and improved import bridges that would require to fully re-import the model (remove incremental harvesting in such case) and this will therefore require to re-build the related Configurations.
6.2.3 Version Specific Upgrade Issues and Recommendations
Upgrading to a new version may have version specific issues or recommendations that are listed at the bottom of the release notes: see Release Changes for more details.
6.2.4 Upgrade and Migration Best Practice
The following critical steps represent the best practice in MIMM Server upgrade or migration to a new machine (on prem or cloud)
Backup
As with any migration / upgrade process, it is critical to backup the underlying data:
  1. the installation data and conf directories (see Understanding the Data Locations)
  2. the repository database. (see Database Server Backup/Restore)
Repository Cleanup
One of the most critical first step is to save disk space and speed up performance by performing a major cleanup of the repository:
  1. Apply the latest MIMM and MIMB Cumulative patches (in the MIMM Server and all MIMB Harvesting Agents).
  2. Make sure that all Configurations are NOT on auto update.
  3. Stop any scheduled operations, including all automatic metadata harvesting and database maintenance (MANAGE > Schedules).
  4. Delete all unused / test Directories, Configurations, Models, Mappings, etc. (MANAGE > Repository).
  5. Delete as many versions as possible (e.g. retaining the last few versions) of the remaining critical Configurations (MANAGE > Repository).
  6. Delete unused model versions (MANAGE > Repository: Repository object > Operations > Delete unused versions).
  7. Run the database maintenance to purge all deleted objects from the database (MANAGE > Schedules: Run Database Maintenance). Note that each database maintenance run purges deleted models for only 2 hours, therefore the database maintenance has to run as many times as needed until the log no long shows any models to delete.
Model Imports
In order to avoid surprises after the upgrade/migration, it is critical to make sure that you started with a stable environment. The most important aspect of that is to make sure all imports are working before any upgrade because the source may no longer be available, may not have been imported with the latest version of the bridge, or may simply not have been imported for a long time.
  1. Apply the latest MIMM and MIMB Cumulative patches (in the MIMM Server and all MIMB Harvesting Agents).
  2. Make sure that all Configurations are NOT on auto update.
  3. Stop any scheduled operations, including all automatic metadata harvesting and database maintenance (MANAGE > Schedules).
  4. Delete the model import cache in $MM_HOME/data/MIMB/cache).
  5. Manual full import (no incremental harvesting) of all Models (one by one) with clear import cache and uncheck "Create new versions only when new import has changes".
  6. Manual force build of all Configurations (one by one) with testing (connection stitching as needed for lineage).
Repository Database
If the MIMM repository database server has to be migrated to a new machine (e.g. from on prem to cloud), make sure you follow the proper the proper database vendor process (see Database Server Backup/Restore).

Note that the above process works well as long as you stay with the same database technology (e.g. PostgreSQL), but such database cannot be performed between database technologies, such as Oracle or SQL Server to PostgreSQL because the repository database implementations are different / optimized for each database technology. In such case, one can use the MIMM Application's Backup/Restore format (directories of XML files) but there are known limitations as some content is not back up in that format.

Harvesting Agents
If the MIMM Application Server has to be migrated to a new machine (e.g. from on prem to cloud), Make sure to reconnect each MIMB harvesting agents to the new MIMM Application Server.

Note that such MIMB harvesting agents can remain on prem, while connecting to the new MIMM Application Server on cloud.

6.3 Application Server Execution and Initialization

The easiest way to start the MIMM Application Server is to execute the $MM_HOME/RestartApplicationServer.sh utility. On Windows, you must "Run as administrator" the %MM_HOME%\RestartApplicationServer.bat utility.

The final initialization steps of the setup are performed over the web browser as follows:

  1. Connection
    Connecting to the server on Windows can be simply achieved by opening the Metadata Management link in the home directory. In all cases, you can connect to the server using your internet browser to open by default: http://localhost:19980/MM. Note that the default port of this URL number may have been changed by the Setup utility in the section Server Installation and Configuration..
  2. Database
    Define the connection to the previously created database (in the above steps), by providing the database type, user, password, and URL (JDBC connection). If you are using the PostgreSQL database bundled with the software package for Windows, then all these parameters should be already preset. Press Test Connection button to verify proper database connectivity. Finally, when the pressing the Save button, the MIMM Application Server will create all the necessary tables in the database.
  3. License
    Click on the Download License Information link to obtain the obtained your HostInfo.xml file that should be sent with your license request. Warning: Make sure your are NOT connected to any VPN during that step, then your license will work independently of your VPN connection. After you have received your MM.lic license file, browse for it and click on the Save License button.
    Alternatively, the host info file can be generated at the Windows command line using $MM_HOME/bin/HostFileGen.sh, or HostFileGen.bat on Windows. This will generate your HostInfo.xml file (in the bin directory) that should be sent with your license request. After you have received your MM.lic license file, copy it to $MM_HOME/conf/MM.lic and restart the application server.
  4. Login
    Login as "Administrator" with password "Administrator". Note that you should change that password later in the application by going to: Tools -> Administration -> Users)

6.4 Custom integration with authentication environments

MIMM is able to support three authentication methods:
  1. Native Authentication, where the password is managed by the software and stored within the database.
  2. LDAP Authentication, where the software does not manage or store the LDAP passwords at all. Instead, it is simply passed it through to LDAP in order to authenticate.
  3. External Authentication such as Single Sign On (SSO), where the software does not perform any authentication, and leaves that responsibility to a local single sign on service managed by the customer.

In Tools->Administration->Users one may specify either:

  1. Mixed Native and LDAP authentication where users may be authenticated either as native users or LDAP users
  2. External authentication where the system does not perform any authentication, leaving it up to a local Single Sign On environment.

6.4.1 Native Authentication Configuration Issues
There are no specific configuration steps for Native Authentication.
6.4.2 LDAP Authentication Configuration Issues
There are no special server configuration issues for LDAP Authentication. LDAP connectivity configuration is documented in the online help.
6.4.3 Windows Authentication Issues
It is also possible to enable the Application Server to obtains authentication for users from Windows authentication via the browser (client). This way, users will automatically be authenticated if they are running from a Windows session.

To do so, one must install a third party product named Waffle (Windows Authentication Functional Framework) as an addon (see here):

  1. Please ensure that all LDAP settings are correct and users are able to log into the product via LDAP authentication. LDAP connectivity configuration is documented in the online help.
  2. Unzip the Waffle zip.
  3. Copy all the jar files from it to $MM_HOME/tomcat/lib
  4. Open $MM_HOME/tomcat/conf/web.xml. Search for "Windows authentication support". Uncomment the block following that.
  5. Restart MIMM.
  6. You should have windows authentication enabled now. Any valid windows user will be logged in as guest by default as long as licensing allows it. If you need to get an administrator interface, you can access: http://host:port/MM/Auth?nativeLogin (optionally you can force a redirect to either &redirectTo=/MM/Explorer or /MM/Manager)
  7. Provide connection information for the database you created above.

Note: Waffle is designed around Windows libraries and thus it is recommended that you use a Windows OS based machine as the Application Server. While it is possible to use Waffle on a Linux based machine, it will require a great deal of manual setup and compilation. Please follow the Waffle documentation for such an implementation (see here).

Note: When using Waffle on a Windows based Application Server (as is recommended) you must run as run the MIMM software as a Windows service (not as an Application) in order for Waffle to work properly.

Note: Automatic Windows authentication will not allow one to use the browser refresh (f5) with IE 8.x when used as the client browser. Refresh will force a re-authentication on IE 8.x browsers and will not be automatically authenticated. If this occurs, the user must close all instances of the browser and start again. To avoid this issue, one must use IE 9.x or later or another approved browser (see System requirements) In addition, for Firefox, you must configure the browser at each client to support automatic Windows authentication. Please refer to the Waffle web site here.

6.5 Custom integration for Secure Socket Layer (SSL) communication

Important Disclaimer: SSL is primarily used for HTTPS secure communications from the web browser clients to the MIMM Server itself. Such common HTTPS setup can be fully achieved with the Setup utility as explained in Server Installation and Configuration. The following steps are provided for illustration purpose only (manual steps), describing what the Setup utility already performs automatically. THEREFORE, YOU DO NOT HAVE TO PERFORM THESE STEPS BELOW.

If you want to manually install a your own certificate, you must:

  1. Change the referenced (in server.xml) connector entry parameters (keystoreFile and keystorePass) to point to the correct keystore file and password.
  2. Import that certificate into the JRE that is being used by this tomcat. The default JRE is located under:
       $MM_HOME/jre.
  3. Use the following commands:
       cd $MM_HOME/jre/lib/security
       move jssecacerts jssecacerts.old
       $MM_HOME/jre/bin/keytool -importkeystore -srckeystore {your_keystore} -keystore jssecacerts
       $MM_HOME/RestartServerApplication.sh (or RestartServerService.bat on Windows)

After the configuration, use the default URL to Access MIMM: https://localhost:19980/MM

Or use the ports specified in the server.xml file. For example:
   <Connector port="19980" maxThreads="200"
              scheme="https" secure="true" SSLEnabled="true"
              keystoreFile="conf/keystore" keystorePass="changeit"
              clientAuth="false" sslProtocol="TLS" />

6.5.1 Configuring MIMM to securely connect via HTTPS to another MIMM server for Metadata Harvesting

The MIMM Server can import metadata from another MIMM Server configured as a metadata harvesting only MIMB Agent. The two typical use cases are:

Important Disclaimer: the following steps are needed ONLY IF you use a self signed certificate for SSL (WHICH IS NOT RECOMENDED), AND ONLY in the case of configuring MIMM to securely connect via HTTPS to another MIMM server for Metadata Harvesting. Only in such exceptional use case, then the following additional steps have to be performed

In order to support HTTPS from a MIMM Server acting as the "Metadata Manager" to a MIMM Server acting as "Metadata Harvesting" Agent, the Administrator needs to import the trusted certificate that the MIMM "Harvesting Agent" Server is using into the JRE that the MIMM "Metadata Manager" server is using. The following page describes the process: http://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html.

The command looks like the following:
cd $MM_HOME/jre/lib/security
../../bin/keytool.exe -importcert -alias john -file YourOwnCertificate.cer -keystore jssecacerts

6.5.2 Configuring MIMM to securely connect via LDAPS to the Enterprise Directory
In LDAP Authentication, the user password is not managed by the software and is simply passed through to the LDAP system.

Note: this password is not encrypted when communicated between the client and the server. Thus, in order to ensure encryption you may wish to specify HTTPS protocol communication, as above.

Note: this password is also not encrypted when communicated between the server and LDAP. Thus, in order to ensure encryption you may wish to also specify LDAPS protocol communication and thus use SSL to encrypt.

In order to support LDAPS, the MIMM Tomcat service does not itself need to be configured to work with LDAPS for encryption of passwords. However, to enable secure SSL communication between MIMM and LDAP servers the Administrator needs to import the trusted certificate that the LDAP server is using into the JRE that the MIMM Application server is using. The following page describes the process: http://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html.

The command looks like the following:
cd $MM_HOME/jre/lib/security
../../bin/keytool.exe -importcert -alias john -file YourOwnCertificate.cer -keystore jssecacerts

This is an entirely different certificate from the one used by the HTTPS protocol.

6.5.3 SSL Security Vulnerabilities

Poodle is a "Man In The Middle" (MITM) vulnerability which needs to be primarily fixed server side. An attacker can trick the server into downgrading the encryption protocol used to communicate. The servers should be configured to disallow TLS fallback, or to disable SSLv3 as a valid protocol.

If Tomcat has been configured with SSL support, the customer should add the following to the connector description in the $MM_HOME/tomcat/conf/server.xml as follows:
   sslEnabledProtocols="TLSv1.2,TLSv1.1,TLSv1"

6.6 Security and Vulnerability Considerations

For extra security and vulnerability reasons, the Apache Tomcat bundled within MIMM can be updated as follows:

6.6.1 Tomcat Upgrade To The Current Patches

Any new version MIMM includes the latest version of Tomcat for bug fixes, performance, security and vulnerability reasons.
According to the tomcat 9 changelog https://tomcat.apache.org/tomcat-9.0-doc/changelog.html, the last Common Vulnerabilities and Exposures (CVE) fix was made in 9.0.30. Therefore, there is no real benefit to upgrade to the latest version.
In addition, it is not recommended that you manually install / patch the version of Tomcat bundled in MIMM, as Tomcat new versions may not be compatible and require more files to be copies. The following process describe the patch process anyway although not officially supported:

  1. Check the version of Apache Tomcat that was bundled within your MIMM (e.g. Tomcat 9.0.45) which is expressed in:
      $MM_HOME/Documentation/License/MIMM-ThirdParty-LICENSES.html and
      $MM_HOME/tomcat/RELEASE-NOTES
  2. Download the latest patch version (e.g. 9.0.46) within the same major version:
     https://tomcat.apache.org/download-90.cgi
  3. Unzip in temporary directory
  4. Stop Tomcat
  5. Copy the lib directory from that temporary directory to the $MM_HOME/tomcat/lib (overwriting files if necessary)
  6. Restart Tomcat

6.6.2 Tomcat Check For Allowed Referrer

For additional protection, we recommend enabling Tomcat to check for allowed referrer:

  1. Edit $MM_HOME/tomcat/conf/web.xml
    1. Uncomment the two filter sections in the 'Checks referrer is allowed' section. The variable ${server.fqdn} will be substituted with the value of M_SERVER_FQDN in tomcat.properties
  2. Edit $MM_HOME/tomcat/conf/tomcat.properties by:
    1. Changing the M_SERVER_FQDN variable from localhost to <myMMServer.myDomain>

6.6.3 Tomcat Configuration to avoid TLS violation of Cryptographic Standard STD-IT-0005

MIMM uses Java 11 that supports Transport Layer Security (TLS) 1.3, but it is compiled for Java 8 that supports TLS 1.2. To ensure that Tomcat does not "drop down to using" a lower version (1.1. and 1.0), one must configure Tomcat appropriately.
Please look for all the information available online on their website at https://tomcat.apache.org/tomcat-9.0-doc/config/http.html#SSL_Support

6.6.4 Tomcat Configuration to include HTTP security headers

For additional protection, you can edit $MM_HOME/tomcat/conf/web.xml

By default, the application sets the following to the recommended values:
   * Content-Security-Policy
   * X-Content-Type-Options
   * X-XSS-Protection

The X-Frame-Options is not set by default, it can be done manually by adding the following fragment:
   <init-param>
      <param-name>X-Frame-Options</param-name>
      <param-value>sameorigin</param-value>
   </init-param>

The HSTS headers are not necessary as when the application is configured for HTTPS then HTTP is not allowed at all, and do not provide automatic redirection. However, you may want/need to add it, you can do so manually by adding the following fragment:
   <init-param>
      <param-name>Strict-Transport-Security</param-name>
      <param-value>max-age=31536000; includeSubDomains</param-value>
   </init-param>

6.6.5 Secret / Password Encryption

There are a few cases where any account secret / user password is stored in the MIMM repository database using an encryption method that is two-way in order to restore the original password just before calling a third part API later:

  1. When configuring metadata harvesting (Model > Import > Setup), some bridge parameters require authentication to the source technology / server (e.g. user / password of a database or a BI server)
  2. When configuring LDAP based authentication (MANAGE > Users > LDAP)
  3. When configuring Email notification (MANAGE > Email Notification)
  4. When configuring Cloud Identity (MANAGE > Cloud Identity)
Because of this requirement, MIMM cannot use key-based industry standard encryption.
MIMM instead stores such user/password in the repository database (i.e. at rest) using a confidential proprietary reversible encryption algorithm based upon industry standards.

NOTE 1: A second level of encryption can also be used during transport (i.e in motion) using 6.5 Custom integration for Secure Socket Layer (SSL) communication

  1. HTTPS for remote metadata harvesting from the main MIMM Server and a remote Harvesting Agent / Server.
    See 6.5.1 Configuring MIMM to securely connect via HTTPS to another MIMM server for Metadata Harvesting
  2. LDAPS for authentication to the Enterprise Directory.
  3. When using LDAP based authentication.
  4. See 6.5 2 Configuring MIMM to securely connect via LDAPS to the Enterprise Directory

NOTE 2: Alternative secret / password encryption and external storage solutions are available using Cloud Identity and Cloud Secret Vaults (such as Amazon Web Services, Microsoft Azure, or Google Cloud).
See, MANAGE > Cloud Identity

6.7 Lucene Search Engine Troubleshooting

The MM search capabilities are implemented by a Lucene search engine with indexes located in in $MM_HOME/data
If this Lucene search index directory has been lost, the MM server will automatically recreate it.

If the Lucene search index has been corrupted for any reason (e.g power outage during indexing, out of memory, concurrent write to the index, etc.), then the search index directory can be deleted and the MM server will automatically recreate it.
Although not officially supported, the Administrator can attempt to use use the Lucene CheckIndex to exorcise corrupted documents from the index.
Here are the steps you can following (replace lucene-xxxxxxxx with the actual directory name of your search index).

  1. cp -R $MM_HOME/data/search/lucene_xxxxxxxx /backup
  2. cd /tmp
  3. mkdir CheckIndex
  4. cd CheckIndex
  5. $MM_HOME/jre/bin/jar -xvf $MM_HOME/tomcat/webapps/MM.war
  6. cd WEB-INF
  7. java -classpath "lib/*" -ea:org.apache.lucene... org.apache.lucene.index
  8. CheckIndex $MM_HOME/data/search/lucene_xxxxxxxx
  9. Examine the output of the above command to see if there is any corrupted segment. If there is run the same command above with an extra option "-exorcise".
  10. You may delete the CheckIndex directory after you are done.
For more details, see more about about Lucene CheckIndex.

6.8 High Availability Considerations

MM can be deployed under High Availability (HA) clusters based on a configuration of Active/Passive MM servers. In this HA architecture, all user URL requests are connected to a centralized HA Management server (machine / node 1) which redirects all calls to the Active MM server (machine / node 2) , or the Passive MM server (machine / node 3) if the Active MM server (or the machine itself ) is down. In such switch-over case, the delay is limited to the time it takes to startup the Passive MM server which is usually a couple of minutes assuming the machine was already active.

In the above HA architecture, both the Active and Passive MM servers must be sharing the same data on a shared database server, and on a shared file system server. For more details, see Understanding the Data Locations. In order to achieve an overall High Availability (HA) of the total MM solution, these shared database and file system servers must also have their own HA deployments, therefore involving a lot more machines / nodes for each of these database and file system servers. Starting a MM server starts both the Tomcat web application server and the Lucene search engine, therefore file system sharing is very important for the user experience as the response time is heavily driven by the License search engine storing its indexes on that shared file system in $MM_HOME/data).

Note that the HA deployment cannot be achieved with the bundled PostgreSQL database $MM_HOME/postgresql available with the MM entry level edition on Windows. In fact, HA deployment is only available on the MM most advanced edition as the license stored in the shared database must be enabled to support both the Active and Passive servers (obviously on different host ids).

7. Metadata Harvesting Model Bridge (MIMB) Setup

The Metadata Integration or Metadata Harvesting from third party databases, data modeling, data integration or business intelligence tools is performed by the integrated Meta Integration® Model Bridge (MIMB) software. By default, the installer software deploys and configures both MIMM and MIMB on the same machine, where the MIMM Application Server accesses the MIMB Web Services locally. MIMB can also be installed and configured as a remote MIMB Agent on another machine. This is very useful in architecture deployments where the metadata management server is:

Essential customizations (e.g. directories, memory) of the MIMB Application Server can be performed in the following configuration file:
   $MM_HOME/conf/conf.properties

Recommended customizations include:

When the MIMB Application Server is used a local metadata harvesting agent connected to a MIMM Application Server on the cloud, the additional customizations are needed in the $MM_HOME/conf/agent.properties configuration file where:

8. User Interface Look & Feel Customization

8.1 Login and Headers

Customize the following files and directories using the embedded instructions (in comments):
   $MM_HOME/conf/resources/MM.properties
   $MM_HOME/conf/resources/FavIcon.ico (can be .png)
   $MM_HOME/conf/resources/LoginLogo.svg (can be .png)
   $MM_HOME/conf/resources/HeaderLogo.svg (can be .png) with the following requirements:
       - Height must be exactly 27px (required)
       - Ideally single color (recommended)
       - Color should match miti.mimm.header.fontcolor (recommended)
       - Color must be compatible (look nice) with selected miti.mimm.header.bgcolor (recommended)
   $MM_HOME/conf/resources/web (optional advanced only)

8.2 Metadata Explorer for Business Users

Customize the following files using the embedded instructions (in comments):
   $MM_HOME/conf/resources/MetadataExplorer.xml

9. REST API SDK

The REST API SDK documentation is available within the UI by going to the Help menu (top right corner) under Help on REST API, or go directly to: http://localhost:19980/MM/REST-API/.

10. Database Server Backup/Restore

The MIMM Application Server requires the connection to an existing Database server for metadata storage (metadata repository). Metadata Integration recommends to backup the MIMM metadata repository regularly and especially before any upgrade.

This document describes the commands and instructions to perform the MIMM repository database backup and restore tasks. These database commands and instructions assume the following by default:

The database name and user name can be changed, and the password should of course be different.

We assume that objects from backup are restored into the same database later.

Always use the same database software version to perform the backup and restore. Backups created by more recent version of a database server may not be restored in earlier versions of the database server.

Note that any data that is saved after a backup is done will be lost if you restore the backup.

Stop the MIMM Application Server before you perform the backup and restore tasks. Restart the MIMM Application Server afterwards.

To ensure the optimal MIMM Application Server performance after a restore operation, run the database maintenance script in the MIMM Management UI using Tools → Administration → Schedules → Run Database Maintenance to update the database statistics.

Backup and restore on a very large MIMM repository database may take a long time. Refer to the database backup and restore documentation to enable parallelism, incremental backup and restore for better performance. The instructions given below are for a full database backup and restore.

10.1 Database on Oracle

We can use the Oracle Data Pump technology for Oracle 10g, 11g, 12c, 18c and 19c database.

First create an Oracle directory BACKUP_DIRECTORY that points to an operating system directory on the database server machine for reading and writing files.

Assume ORCL is the database server name or SID.
Sqlplus.exe / as sysdba
   CREATE OR REPLACE DIRECTORY BACKUP_DIRECTORY as '/backups/Oracle';
   GRANT read, write ON DIRECTORY BACKUP_DIRECTORY TO MM;

Then use Oracle Data Pump (expdp, impdp) to backup and restore the MIMM metadata repository.

Backup

To backup (export) the MIMM metadata repository database to a file MM.dmp and write the export log to expdpMM.log in the operating system directory /backups/Oracle:
   expdp MM/MM123! schemas=MM directory=BACKUP_DIRECTORY dumpfile=MM.dmp logfile=expdpMM.log

Restore

Before you restore the backup to the MIMM repository database, you need to drop the schema MM to delete existing objects and data from the MIMM repository database. Restore will recreate the schema MM.
   Sqlplus.exe SYS@ORCL as SYSDBA
      DROP USER MM CASCADE;

To restore (import) the MIMM metadata repository database from a file MM.dmp and write the import log to impdpMM.log in the operating system directory /backups/Oracle:
   Impdp schemas=MM directory=BACKUP_DIRECTORY dumpfile=MM.dmp logfile=impdpMM.log

When prompted for Username, enter / as sysdba.

You may refer to the Oracle Data Pump documentation for more details on the expdp and impdp commands. Backup Restore using Recovery Manager (RMAN) You may also use the Oracle Recovery Manager (RMAN) to backup and restore your MIMM repository database. It is a good practise to create a separate table space for MIMM repository database and restore only from that table space. For more information refer to the Oracle Database Backup and Recovery User’s Guide.

10.2 Database on SQLServer

To perform backup or restore log in to the SQL Server Management Studio, open a new query window and execute the backup or restore commands given below. You can also use the SQL Server Management Studio Object Explorer UI to perform the backup and restore tasks.

Backup

To backup the MIMM repository database MM into a file /backups/SQLServer/MM.bak, use the following command:
   BACKUP DATABASE [MM] TO DISK = N'/backups/SQLServer/MM.bak' WITH NAME = N'MM-Full Database Backup'
   GO

Restore

To restore the file /backups/SQLServer/MM.bak into the MIMM repository database MM use the following commands.
   USE [master]
   GO
   ALTER DATABASE MM SET SINGLE_USER WITH ROLLBACK IMMEDIATE
   GO
   RESTORE DATABASE [MM] FROM DISK = N'/backups/SQLServer/MM.bak' WITH REPLACE
   GO

You may refer to the Microsoft SQL Server backup and restore documentation for more details on the above commands.

10.3 Database on PostgreSQL

Backup

To backup the MIMM repository database MM into a file /backups/PostgreSQL/MM.dmp use the following command:
   pg_dump -b -f /backups/PostgreSQL/MM.dmp -F t -d "MM" -h localhost -w -p 5432 -U MM

Restore

To restore the file /backups/PostgreSQL/MM.dmp into the MIMM repository database MM use the following command. All database objects and data are dropped before recreating them.
   pg_restore -c -F t -d "MM" -h localhost -w -p 5432 -U MM --if-exists /backups/PostgreSQL/MM.dmp

You may refer to the PostgreSQL pg_dump and pg_restore documentation for more details on the above commands.

Copyright © Meta Integration Technology, Inc. 1997-2023 All Rights Reserved.

Meta Integration® is a registered trademark of Meta Integration Technology, Inc.
All other trademarks, trade names, service marks, and logos referenced herein belong to their respective companies.