Disaster Recovery Procedure For Primary Server

The primary objective of the disaster recovery procedure is to ensure stability and orderly recovery after a failed primary server to enable minimal disruption in the use of PMC.

The following outlines the steps to backup and restore a PMC primary server.

Backup a primary server

1. To take a backup of all the runtime configurations, truststore, log4j config files and the license

Go to the folder /opt/hcp

cd /opt/hcp

Take a backup of the conf folder

tar -cvzf conf.tar.gz conf

where conf.tar.gz – name of the backup
conf – name of the folder that is backed up

2. To take a backup of the runtime information, server-ids and the local database
Go to the folder /var/lib

cd /var/lib

Take a backup of the eopng folder

tar -cvzf eopng.tar.gz eopng

3. If you are using an external postgres DB then take a backup of the database as well

pg_dump -f primarydbdump -b -c -C -h <DBHOST> -U postgres -p <DBPORT> eopng_config

Where DBHOST and DBPORT are the database cluster address and port number.

Restore a primary server

In case the server fails, the below steps can be taken on a clean system

4. Run the PMC installer

./hcp-server-*-release-setup-linux-x86_64.bin [-f]

The optional -f switch can be specified to force the full installation instead of upgrade.

5. Fill out all the details in the installer (All the details in the installer will only be used to get through the wizard and setup all the binaries we do not need to backup)
6. Opt out of starting PMC in the final installer step (uncheck the “Start PMC server” checkbox). Or stop it afterwards with systemctl stop hcp-server command.
7. Stop the database (only if bundled postgresql was used and not the external database)

systemctl stop pmc-database

8. Replace the new /opt/hcp/conf with the backup taken in step 1

cd /opt/hcp
tar -xvzfp /path/to/backedup/conf.tar.gz

9. Replace the new /var/lib/eopng with the backup taken in steps 2

cd var/lib
tar -xvzfp /path/to/backedup/eopng.tar.gz

10. Start the database (only if bundled postgresql was used and not the external database)

systemctl start pmc-database

10. Start the PMC services again

systemctl start hcp-server

While restoring the folders please make sure that the permissions on the folders backed up are retained