02 - Restore-Specific Database Columns

02 - Restore-Specific Database Columns

The following columns are commonly used when Universal Migrator is restoring data into a destination system.

Restore Ordering and Priority

Batch_Number

The Batch_Number allows you to mass-segment data.  Only the data with the lowest Batch_Number in each table will be processed.

If an item is processed successfully, its Batch_Number will increment by +10.
If an item encounters errors, its Batch_Number will increment by +1.

This make it easy to retry items with errors after the restore is complete.

Batch_Group

The Batch_Group allows you to manually control the order data is restored.

When a table is restoring, all the records in the lowest Batch_Number are grouped by the Batch_Group.
All of the items in the lowest Batch_Group are process first.  Once they are all processed, then all the items in the next Batch_Group are processed, etc.
Only one group is processed at a time and the group must be fully processed before the next group is started.

This gives you the ability to have course-grained control over the order that data is restored.
Using this field, you can instruct Universal Migrator to do things like restore documents for open matters first.

Finer Details on Restore Ordering and Priority

Many destination system have challenges when a "parent" item (like a Matter) has multiple "child" items (like documents) being restored to it at the same time.  To avoid these issues, within a Batch_Group, all the items are further grouped by their parent and each parent is sorted by the number of items in it (parents with the most children process first).  Each restore thread will pick a single parent and work on all the children within it.  This process is able to avoid issues that many systems have with concurrent restores and generally avoids delays with giant groups processing last.  However, if you have data that is extremely lop-sided (for example, a most matters have 20 documents but one matter has 200,000), you may find that all other matters finish first before the giant one just because there are so many items in it.  If this happens, the migration will essentially be processing as a single-threaded migration because only one parent is available for processing.  While this is not ideal, this is preferred because the alternative is having sporadic errors caused by concurrency.

Result Tracking

Result_Id

The Result_Id column contains the primary key value of the corresponding object that has been restored into the Destination system.  If this column is blank, the item has not yet been restored.

 

The Result_Id column serves three purposes.

  1. When restoring a record, it indicates whether or not the record has been restored.  If it contains a non-blank value, then it will not be restored.
  2. When tweaking an existing record, it indicates which record in the destination system should be modified.
  3. When restoring a dependent record, this value is used to determine where in the destination system the object lives.

For example, if Contact 001 has a Result_Id of 12345:

  1. Restoring contacts will have no effect on this record because it has already been restored.
  2. Tweaking contacts will cause the contact in the destination system with Id 12345 to be modified.
  3. Restoring matters linked to Contact 001 will cause it to be linked to the contact in the destination system with Id 12345.

Similarly, if Contact 002 has a Result_Id that is blank:

  1. Restoring contacts will cause this record to be restored into the destination system because it has not yet been restored.
  2. Tweaking contacts will have no effect on this record because it can't be tweaked because it hasn't been restored yet.
  3. Restoring matters linked to Contact 002 will fail because Contact 002 has not yet been restored so there is no contact in the destination system to link the matters to.

 

Result_ReferenceCode

Generally speaking, the Result_ReferenceCode will be the same as the Result_Id but, when restoring data into a system that has ReferenceCodes, it will be populated with a copy of the result's ReferenceCode instead.  This most commonly happens with Document Management Systems where a Document might have a primary key of:

  1. :Q9:z:z:e:9:~23184536973905389.nev|1

But has ReferenceCode of:

  1. 1234-4567-6789


Result_Raw_Content

This will contain a copy of the JSON returned by a destination system when a record has been successfully restored.


Error Logging

Status_ReferenceCode

If an error occurs while restoring a record into the destination system, this will contain the error code.

 

Status_Subject

If an error occurs while restoring a record into the destination system, this will contain the error message.

 

Status_Description

This contains the full log of everything that happened while processing this record.

 


    • Related Articles

    • 01 - Tweaking a Restore

      Sometimes after restoring a backup, a customer may want you to tweak the data that is in their new system. Tweaking data post-restore is easy! Let's say that the customer wants you to "mark as complete" all tasks that are related to archived (closed) ...

    • Training: Overview

      Welcome to the Migration Academy! This series will help you learn everything you need to know in order to deliver high-value, rock-solid data and document migrations. Completion of the academy generally takes 4-8 hours and is required in order to ...

    • 01 - Reviewing a Backup

      The Universal Database contains tables for each type of data you might backup or restore between systems. While most of the columns are unique to each table, certain columns exist on all tables. The purpose of this article is to describe what each ...

    • Migration Type: Delta Migrations

      A "Delta" migration is necessary when you migrate some data now and some data later. To do a Delta migration, please follow the steps below after you have complete the initial backup and restore. 1. Create the "Phase 2" Backup Using Universal ...

    • Merging Data into an Existing Application

      This is a highly technical process Experience with SQL queries is necessary to perform Join Migrations. Sometimes you may have customers who have data that needs to be restored into an existing system that is already set up and active. This type of a ...