The information in this document is provided on an as-is basis. You use it at your own risk. We accept no responsibility for errors or omissions, nor do we have any obligation to provide support for implementing or maintaining the configuration described here. Furthermore, we do not warrant that the design presented here is appropriate for your requirements.

Access Anywhere stores metadata in a stateful MariaDB relational database. In the case of HA/DR setups, we use database replication to have multiple nodes running an up to date version of that stateful data.

During regular operations, no administrative effort is required to maintain replication. However you may run into physical device or network events which may cause the replication between the databases to fail. This document will assist you with the work needed to reseed the secondary server and re-enable replication.

Here are some important points to consider when establishing replication between your database nodes:

• When database replication is used, changes are sent from the primary to the secondary database after they have been applied to the primary.
• The lag between when a change is applied to the primary and when it is delivered to the secondary is usually a few seconds or less.
• If the lag exceeds some very short interval, such as two minutes, replication may have failed.
• You can use monitoring to keep track of the lag, and alerting to let you know if it exceeds the interval you have selected.
• Possible failures should be investigated and resolved promptly.
• Failing over to a secondary database that is out of sync with the primary and then allowing writes to that secondary database will leave your data in an incoherent state from which there may be no easy way to recover.
• Replication is not a substitute for database backups. Data changes - intentional or unintentional - will quickly be copied from the primary database to the replicant. If that happens, you will need a backup so you can roll back to the pre-changes state.

This guide assumes the use of GTID based replication, which is the preferred method for database replication since Access Anywhere v2006. If you are currently not set up with GTID replication, do not use this guide until GTID is setup. If you are running v2006 or greater follow this guide first Converting to GTID for v2006+:

Before you start, please be sure to collect/prepare the necessary information.

• 2 NAADatabase Appliances deployed
• NAALinux root password
• NAALinux smeconfiguser password
• Existing user previously setup for database replication (In this guide we refer to the user as “repl_user”).

For Linux command line operations, you must run the commands shown in this document as the root user unless otherwise specified. However, for security reasons you cannot connect with ssh to a Access Anywhere virtual machine directly root. Instead, you should ssh to the VM using smeconfiguser and then become root (su for on-premises deployments, sudo for cloud based deployments)

For database operations, to login to the smestorage database as root you do not need a password. As root from the command line, type:

mysql smestorage

Note: This command only works if you are root; smeconfiguser does not have privilege to run this command.

This process will involve taking the current active database, and reseeding the secondary based on that active database. If you are using keepalived verify which db is primary using:

 
$ cat /var/run/keepalived_DB.state
MASTER

Make note of that active server and save its IP address for use later in this process.

 
ip addr show eth0

During initial setup of replication you created a user (repl_user per instructions) that was used for replication. You will need the password for that account to set up replication again. If you do not know the password for that account, you will need to enter a new password like this:

In the mysql console

 
set password for 'repl_user'@'%' = PASSWORD('<secure password>');  

Run this on both nodes to ensure the repl_user password is set to a known value, to be used below.

During the reseeding of the database, we need to ensure that changes are not occurring to the primary database. This requires downtime for Access Anywhere, so please ensure you do the following during a planned maintenance window.

We now reset the existing replication status and lock the database. In the mysql cli run the following:

 
reset master; stop slave; reset slave; set global gtid_slave_pos=''; flush tables with read lock;

Leave that session open while we run the rest of the commands below.

On the secondary server, we will need to backup/copy over the database and restore. Please review the available free space on your server and ensure sufficient free space to create this backup.

Assuming you have enough available space, please run the following commands in order to reseed the database:

mysqldump -u smestore -pbesp5fyx smestorage | gzip -c --fast > orig_dbbackup_`date -I`.sql.gz
mysql -e "create or replace database smestorage;"
mysql -e "reset master; stop slave; reset slave; set global gtid_slave_pos=''"
mysql -e "set GLOBAL log_bin_trust_function_creators = 1;"
ssh smeconfiguser@<active db ip> mysqldump  --single-transaction=TRUE -u smestore -pbesp5fyx smestorage | gzip -c --fast > dbbackup_`date -I`.sql.gz
gzip -cd  dbbackup_`date -I`.sql.gz | mysql smestorage
mysql -e "set GLOBAL log_bin_trust_function_creators = 0;"
mysql -e "RESET MASTER; STOP SLAVE; RESET SLAVE; set global gtid_slave_pos='';"

We will now set up and start replication based on the repl_user credential we made note of above.

 mysql -e "CHANGE MASTER TO MASTER_HOST='<active db above>',MASTER_USER='repl_user',MASTER_PASSWORD='<secure password from above>', 
 MASTER_USE_GTID=slave_pos;"
 mysql -e "START SLAVE;"
 mysql -e "SHOW SLAVE STATUS \G"

You will be looking for the following two lines to verify you have setup replication correctly.

            Slave_IO_Running: Yes
            Slave_SQL_Running: Yes       

We now have replication running from the Primary node to the Secondary Node. Best practice for Access Anywhere is to have replication also setup from the Secondary back to the Primary. This is done so that in the case of a disaster, and the Secondary becomes active, we are able to copy all the changes to that Secondary DB back to the Primary when its back online.

So first we will grab the IP address of the Secondary DB node:

 
ip addr show eth0

Back on the active server, where we still have the mysql console open we can unlock the database and setup replication.

First we exit out of the mysql cli session:

exit;

Now we run the following commands, using again the IP of the secondary and the password for repl_user set above:

 mysql -e "STOP SLAVE; RESET SLAVE;"
 mysql -e "CHANGE MASTER TO MASTER_HOST='<secondary db above>',MASTER_USER='repl_user',MASTER_PASSWORD='<secure password from above>', MASTER_USE_GTID=slave_pos;"
 mysql -e "START SLAVE;"
 mysql -e "SHOW SLAVE STATUS \G"

You will be looking for the following two lines to verify you have setup replication correctly.

            Slave_IO_Running: Yes
            Slave_SQL_Running: Yes       

Now your Access Anywhere DB nodes are back replicating data again.

Once you have verified that both nodes are set with “IO” and “SQL” Running having values of “Yes” you can end the maintenance window and allow users access into the system as normal again.