Understanding the Data Locations
Most application data is obviously located inside your database server, you are responsible for regular backup of such database. Upgrading your application may also upgrade the associated database content (database schema, stored procedures, indexes and of course data). It is important to understand to understand that part of the MetaKarta software is implemented as database stored procedures like tracing the lineage. Therefore, a given version of MetaKarta corresponds to a version of the MM software in the application server (tomcat) and a version of the MM software in the database (e.g. tomcat) Therefore make sure you always backup your database before any upgrade.
Furthermore, the upgrade process may take several hours (on large repositories) and also need extra space for temp data during the migration. Therefore, make sure the database has at least 20% free space.
Finally, it is also important to understand that the software
installation directory (known as $MM_HOME
in this document) also
contains some critical application data and application setup
customizations that have to be taken into account in your backup or
upgrade process, including:
-
postgresql/data which contains the actual PostgreSQL database data only on Windows and only when configured with the Setup utility (in the "Database Server" tab).
-
data/ which contains other application data, including:
-
solr/ contains the Solr search engine index files.
-
search/ contains the lucene search engine index files from previous versions prior to Solr upgrade (they will be automatically fully rebuilt from scratch if the folder is empty which will take a lot of time)
-
MIMB/
- cache/ contains bridge execution files organized as BridgeId/ImportId/mir_Version,nativeVersion
- parameters/ contains bridge execution parameter files (only for some bridges like DI/ETL with runtime variables) organized as BridgeId/ImportId/Parameters
-
download/MIMB/ contains third-party download software packages.
-
analytics/ contains the generated analytics data files.
-
files/mimb/ contains files for upgrade of MIMB Web Services
-
files/mm/ contains files for MM activities:
- backups/ contains model backups for model comparison purpose.
-
operations/ contains the operation execution generated files such as a troubleshooting package or a backup ready to download. As these generated files can be very large, it is critical to execute the operation "Delete operation logs and files" either manually (MANAGE > Repository, right click on at the repository root) or automatically (MANAGE > Schedules).
-
sessions/ contains any temporary files for user login session.
-
logs/ for the log files:
-
search/ contains the search query log files.
-
tomcat/ contains the tomcat log files.
-
-
sessions/ contains files for sessions of MIMB Web Services.
-
temp/ for any temporarily files (including from the MIMB bridges)
-
upgrade / contains files for upgrade of MM Web Services
-
webapps/ for application server (tomcat) cache
-
Remember that the actual location of this MetaKarta Application Server data directory can be configured with the Setup utility (in the "Application Server" tab).
-
conf which contains the MM configuration / customizations organized as follows:
-
conf.properties contains file containing most customizations defined with the Setup utility (in the "Application Server" tab)
-
ModelBridgeList.xml contains the list of enabled bridges and their names
-
resources/ directory containing any User Interface Customizations, in particular MM.properties and MetadataExplorer.xml.
-
Template/ contains the default template files of all the above files/directories, including a potentially new updated ModelBridgeList.xml or MetadataExplorer.xml after cumulative patches.
-
-
tomcat/conf with the tomcat.properties file containing the tomcat port and memory customizations defined the Setup utility in the "Application Server" tab, and the keystore file containing the tomcat SSL certificates defined with the Setup utility (in the "Application Server" tab).
-
jre/lib/security which also contains some SSL customizations defined with the Setup utility (in the "Application Server" tab). It is recommended to not reuse such directory, but rather reinstall the SSL keys with the Setup utility.
Upgrade Process
We recommend the following upgrade process:
-
Stop your application server in the same way it was started such as stopping the Windows services / Linux daemon, stopping the desktop command windows, or using the
tomcat/bin/shutdown
. Remove the application server service if one was created (Windows only). -
Stop your database server and ONLY if you are using the bundled PostgreSQL database on Windows then use the
$MM_HOME/Setup.sh
utility to remove the Database Service as follows: Go to the Database Server tab, enable disable or uncheck the database service check box, and click on the Configure button. -
Backup your MetaKarta data including your database and data file directories as explained above.
-
Backup your MetaKarta software by copying the
$MM_HOME
directory as$MM_HOME.yyyymmdd.
-
Install the complete MetaKarta new software (ONLY needed for clean install of a new version) by deleting the old
$MM_HOME
, and then creating a new one by unzipping the new MetaKarta full package. -
Apply the latest software cumulative patch by unzipping it from inside the new
$MM_HOME
directory.- WARNING 1: Make sure you unzip with overwrite on Windows, and use
unzip -u
on linux to update files while retaining permissions. - WARNING 2: You cannot use cumulative patches for major version upgrades, you must first start from a clean install of the new major GA version.
- WARNING 3: You cannot reverse / unzip an older cumulative patch, you must restart from a clean install of the original GA version.
- WARNING 1: Make sure you unzip with overwrite on Windows, and use
-
Restore your MetaKarta data and software customization/setup (ONLY needed for a clean install of a new version) by copying the appropriate files and directories (as previously explained) from
$MM_HOME.yyyymmdd
to$MM_HOME
, including at least$MM_HOME/data
and$MM_HOME/conf/conf.properties
but possibly more as used and customized such as$MM_HOME/conf/resources
, or$MM_HOME/tomcat/conf
. -
Integrate the new software features in your configuration by copying potential new versions of files from
$MM_HOME/conf/Template
into their matching directories in$MM_HOME/conf/.
For example, the new$MM_HOME/conf/ModelBridgeList.xml
may contain some new or updated bridges. WARNING: if you had customized some files such as$MM_HOME/conf/resources/MM.properties
, you must re-apply/merge such customization starting from the new version of that file copied Template. -
Reconfigure your Database Server (ONLY if you are using the bundled PostgreSQL database on Windows). As the PostgresSQL server software version may have changed and the database server needs to be upgraded. Therefore, you must:
- Execute the (old renamed installation)
$MM_HOME.yyyymmdd/Setup.sh
utility to restore the old Database Service as follows: Go to the Database Server tab, enable the database service check box, and click on the Configure button. - Execute the (new installation)
$MM_HOME/Setup.sh
utility as follows: go to the Database Server tab, enable the database service check box, and click on the Configure button. At this point the$MM_HOME/Setup.
sh utility will retrieve the existing PostgreSQL data from the old install directory (which was already running as a service), will migrate them to the new install directory, and will remove the old PostgreSQL service, before starting the new PostgreSQL service on the new directory.
- Execute the (old renamed installation)
-
Restart your database server (ONLY if you are using the bundled PostgreSQL database on Windows).
-
Restart your application server after which your first login as Administrator may prompt you for an upgrade of your MM database.
The upgrade log may be found in MANAGE > Operations. You should then select the upgrade operation and select Show Log.
-
Redo all above steps for any other MetaKarta Application server configured as MIMB harvesting agent.
-
Update your repository content* (ONLY as needed) if the upgrade contains new and improved import bridges that would require to fully re-import the model (remove incremental harvesting in such case) and this will therefore require to re-build the related Configurations.
Typical DevOps Linux bash scripts include the following commands:
#! /bin/bash
##
# MM INSTALL #
##
## START SETUP ##
sudo su
MM_INSTALL=/home
MM_INSTALLER=MetaKarta-1110-20241231
MM_SOFTWARE=MetaKarta
MM_HOME=${MM_INSTALL}/${MM_SOFTWARE}
## DOWNLOAD INSTALLER TO INSTALL DIRECTORY ##
# get $MM_INSTALL/$MM_INSTALLER
## STOP SERVER ##
$MM_HOME/ShutdownServerApplication.sh
# double check for any tomcat process still running
# ps -edf | grep tomcat
# kill -9 <Process Id>
## FULL CLEAN INSTALL NEW SERVER ##
# Backup old install and extract new one
cd $MM_INSTALL
mv $MM_SOFTWARE $MM_SOFTWARE.yyyymmdd
tar -xvf $MM_INSTALLER.tbz2
rm MM_INSTALLER.tbz2
# optional ThirdParty software extract for brand new install on
machines with no access to internet
mv *ThirdParty*.zip $MM_SOFTWARE
cd $MM_SOFTWARE
unzip *ThirdParty*.zip
rm *ThirdParty*.zip
## COPY SETUP AND DATA ##
cd $MM_INSTALL_DIR/$MM_SOFTWARE.yyyymmdd
# data directory with large MM search index and MIMB cache
# cp -r data ../$MM_SOFTWARE
mv data ../$MM_SOFTWARE
# conf properties (memory, path, etc.)
#M_BROWSE_PATH=
#M_JAVA_OPTIONS=-Xmx64G
cp conf/conf.properties ../$MM_SOFTWARE/conf
# tomcat conf properties (URL, ports, services, etc)
cp tomcat/conf/tomcat.properties ../$MM_SOFTWARE/tomcat/conf
## HARVESTING AGENT SERVERS ##
# conf agent identification to MM server
cp conf/agent.properties ../$MM_SOFTWARE/conf
# conf agent access to SSL based MM server
$MM_HOME/Setup.sh -ch abc.def.com -cp 443
## MM MAIN SERVER ##
# MM conf properties (database connection, email setup, etc.)
cp conf/mm.conf.properties ../$MM_SOFTWARE/conf
# MM resources such as the UI look & feel
alias cp=cp
cp -rf conf/resources ../$MM_SOFTWARE/conf
# Tomcat SSH Setup
cp tomcat/conf/keystore ../$MM_SOFTWARE/tomcat/conf
cp tomcat/conf/server.xml ../$MM_SOFTWARE/tomcat/conf
## RESTART SERVER ##
$MM_HOME/RestartServerApplication.sh
## END ##
Version Specific Upgrade Issues and Recommendations
Upgrading to a new version may have version specific issues or recommendations that are listed at the bottom of the release notes: see Release Changes for more details.
Upgrade and Migration Best Practice
The following critical steps represent the best practice in application server upgrade or migration to a new machine (on prem or cloud)
Backup
As with any migration / upgrade process, it is critical to backup the underlying data:
-
the installation data and conf directories (see Understanding the Data Locations)
-
the repository database. (see Database Server Backup/Restore)
Repository Cleanup
One of the most critical first step is to save disk space and speed up performance by performing a major cleanup of the repository:
-
Apply the latest MIMM and MIMB Cumulative patches (in the application server and all MIMB Harvesting Agents).
-
Make sure that all Configurations are NOT on auto update.
-
Stop any scheduled operations, including all automatic metadata harvesting and database maintenance (MANAGE > Schedules).
-
Delete all unused / test Directories, Configurations, Models, Mappings, etc. (MANAGE > Repository).
-
Delete as many versions as possible (e.g. retaining the last few versions) of the remaining critical Configurations (MANAGE > Repository).
-
Delete unused model versions (MANAGE > Repository: Repository object > Operations > Delete unused versions).
-
Run the database maintenance to purge all deleted objects from the database (MANAGE > Schedules: Run Database Maintenance). Note that each database maintenance run purges deleted models for only 2 hours, therefore the database maintenance has to run as many times as needed until the log no longer shows any models to delete.
Model Imports
To avoid surprises after the upgrade/migration, it is critical to make sure that you start with a stable environment. The most important aspect of that is to make sure all imports are working before any upgrade because the source may no longer be available, may not have been imported with the latest version of the bridge, or may simply not have been imported for a long time.
-
Apply the latest MIMM and MIMB Cumulative patches (in the application server and all MIMB Harvesting Agents).
-
Make sure that all Configurations are NOT on auto update.
-
Stop any scheduled operations, including all automatic metadata harvesting and database maintenance (MANAGE > Schedules).
-
Delete the model import cache in
$MM_HOME/data/MIMB/cache
). -
Manual full import (no incremental harvesting) of all Models (one by one) with clear import cache and uncheck "Create new versions only when new import has changes".
-
Manual force build of all Configurations (one by one) with testing (connection stitching as needed for lineage).
Repository Database
If the MetaKarta repository database server must be migrated to a new machine (e.g. from on prem to cloud), make sure you follow the proper the proper database vendor process (see Database Server Backup/Restore).
Note that the above process works well if you stay with the same database technology (e.g. PostgreSQL), but such database cannot be performed between database technologies, such as Oracle or SQL Server to PostgreSQL because the repository database implementations are different / optimized for each database technology. In such case, one can use the MetaKarta Application's Backup/Restore format (directories of XML files) but there are known limitations as some content is not back up in that format.
Harvesting Agents
If the Application Server has to be migrated to a new machine (e.g. from on prem to cloud), Make sure to reconnect each MIMB harvesting agent to the new Application Server.
Note that such MIMB harvesting agents can remain on prem, while connecting to the new Application Server on cloud.