Running a Parallel Standby Environment

To run a parallel standby environment, your current environment must be backed-up and then restored to the parallel one.

Restoring a K2 environment

The following scenarios are addressed with restoration of a K2 environment:

Scenarios
Cold Standby A method of redundancy in which the secondary (i.e., backup) system is restored and brought online when the primary system fails. The system on cold standby receives scheduled data backups, but less frequently than a warm standby. Cold standby systems are used for non-critical applications or in cases where data is changed infrequently. The Disaster Recovery server names are the same as the server names currently in use.
Hot Standby A method of redundancy in which the primary and secondary (i.e., backup) systems run simultaneously. The data is mirrored to the secondary server in real time so that both systems contain identical information. The Disaster Recovery server names are different than the non-standby server names. See Database Disaster Recovery Options for issues when using SQL Log Shipping and Database Mirroring.
Hot Standby is also known as Active-Active replication.
See the appropriate vendors’ product documentation for information on implementing Hot Standby for supporting systems such as Operating Systems, Databases or other underlying systems required by your K2 environment.
Warm Standby A method of redundancy in which the secondary (i.e., backup) system runs in the background of the primary system. Data is mirrored to the secondary server at regular intervals, which means that there are times when both servers do not contain the exact same data. See Database Disaster Recovery Options for issues when using SQL Log Shipping and Database Mirroring.
Warm Standby is also known as Active-Passive replication.
See the appropriate vendors’ product documentation for information on implementing Hot Standby for supporting systems such as Operating Systems, Databases or other underlying systems required by your K2 environment.
If loss of data is a concern, another option would be to perform SAN (system area network) mirroring, i.e. everything that is written to a SAN in the main site is replicated via the SAN link to the disaster recovery site. The downside of this option is that it can be cost-prohibitive.

Restoring to a Cold Standby environment

Follow the steps below to restore to a cold standby:

  1. Ensure that the Windows machine is in working order.
  2. Install the K2 Server and K2 Site components.
  3. Apply service packs and / or updates.
  4. Restore the K2 installation directory from backup.
  5. Restore databases from backup.
  6. Restore K2 servers and IIS servers from backup.
  7. Restore any other files for your K2 processes.
  8. Run the K2 Setup Manager to update the new license key.
  9. Start your K2 server.
  10. Test your K2 processes and data.
  11. Check the K2 Host Server log and the server event log for any errors.

Preparing a Hot or Warm Standby environment for restoration

It is important to note that preparation of the Hot or Warm Standby environment is required prior to the disaster occurring, as K2 stores configuration and licensing information within the K2 configuration database.

Follow the steps below to prepare the environment for a Hot or Warm Standby restoration:

  1. Install K2 on the Disaster Recovery servers.
  2. Perform a full system backup of the K2 Disaster Recovery servers.
  3. Perform a backup of the data in the following two tables in the K2 database:
    • Server.Server
    • HostServer.LicenseKey
  4. Take the Hot or Warm Standby servers offline.

Restoration of a Hot or Warm Standby environment

Follow the steps below to restore a Hot or Warm Standby environment:

  1. Bring the Hot or Warm Standby environment online.
  2. Restore the K2 ServiceBroker directory ("%ProgramFiles%\K2\ServiceBroker") from backup.
  3. Restore any other custom assemblies that your processes may require to the machine.
  4. Restore the backed up database.
  5. Restore the K2 Disaster Recovery Servers.
  6. Restore the data from the two tables backed up earlier.
  7. Update all references to the old machine such as ASPX pages, links to the Workspace, Worklist Web parts in SharePoint etc to point to the new machine name.
  8. Start the K2 Service.
  9. Test your K2 processes and data.
  10. Check the K2 Host Server log and the server event log for any errors.

This topic covers the K2 recommended procedures to enable Disaster Recovery (DR) within the infrastructure that leverage physically different data centers for production and disaster scenarios.

The following parameters are assumed:

  • Accepted data loss of 4 hours.
  • K2 system data is consistent upon a cut over to the DR site.

Should the window of accepted data loss need to be adjusted, the scheduled jobs laid out below should be recalculated accordingly.

The procedure below leverage an initial set of database backups and then transaction log shipping with a point-in-time recovery to allow for consistent K2 data in the event of a disaster event.

Prerequisite

The following should be done:

  • Within the DR site:
    1. Record the K2 licensing information in the following SQL tables:
      • Server.Server
      • HostServer.LicenseKey
        • The data should be similar in both locations.
    2. Create a SQL Script that will reapply this information during an actual disaster recovery event.
      This is required because the K2 licensing is stored within the database and is bound to a specific environment. As such restoring databases across environments as in a DR scenario will replace the DR license with the PRODUCTION license. Creating a SQL script to restore the original DR license, while not always necessary, will make this a more repeatable procedure should it be desired.
    3. Keep the K2 host server nodes turned off until a DR cut over is required. This ensures that the DR site does not try to process transactions based upon its copy of the K2 database.

Initial Setup

The following setup is required:

  • Within the PRODUCTION site:
    1. Turn off the K2 service on all nodes within the K2 farm. This insures no K2 data manipulation.
    2. Create Full Database Backups of all K2 product databases as mentioned in K2 databases Backup and Restore. Alternatively, incremental backups may also be used . Either way, compression of databases is advised for efficiency.
    3. Script out all external database server artifacts, e.g. database logins, users, etc.
    4. Backup additional database(s) leveraged by the solution.
    5. Turn on the K2 service on all nodes within the K2 farm.
    6. Move the database backups to the DR site.
  • Within the DR site:
    1. Restore the PRODUCTION database backups.
    2. Reapply external artifacts (logins, etc).
    3. Re-apply K2 licensing, as identified in the Prerequisite step.

Ongoing

  • Within the PRODUCTION site:
    1. Create a scheduled SQL Agent job (recommended hourly) within the SQL Server that backs up the transaction logs for the following:
      • All K2 product databases.
      • Solution database(s).
    2. Schedule a Log Shipping job to transmit these logs to the DR site.
  • Within the DR site:
    1. Within the DR site create a scheduled job (recommended hourly) that will apply any transaction log backups for the databases that are greater than 3 hours hold.
  • Archive the just-applied Transaction Log files.

Disaster Recovery Cutover Procedure

  1. If possible, backup the ‘tail end’ of the log(s) from the primary site and transmit them to the secondary site.
  2. Determine the time of the disaster at the production site and subtract 2 hours. Choose an exact time of recovery that makes sense, for instance 13:35:00:000.
  3. Recover queued transaction logs for all databases to that same specific point in time (the chosen time may or may not include the ‘tail end’ backups obtained in 1). In this example, all databases should be recovered until 13:35:00:000.
  4. Reapply the appropriate K2 server licenses within the DR environment (preferably leveraging the script file recommended in the Prerequisite section, point number 2).
As a result of the time difference, there will always be 1 (or 2 or 3 as need be) log difference. This allows a restore to a point in time operation on the final log in the event of a DR event. After this restore to a point in time operation all databases will be consistent.

Testing

A manual failover process should be executed during development / QA to ensure that the procedure works as expected. All K2 operations should be tested in order to confirm that the primary data files and external artifacts (such as logins) are valid.