SUMMARY
VCD stops trying to replicate when the queue scheme is set to anything other than manual.
ISSUE
When replicating, the following error condition is encountered:
Failed: Unable to process initial meta info on target, out of order backup sequence detected. This instance is "paused" and may require manual intervention to resume.
RESOLUTION
Query the database on the source to check for out of order backups. If the status is 128 then proceed with this KB.
psql bpdb -U postgres -c "SELECT * from bp.replication_queue WHERE status = 128"
backup_no | vault | position | status | rep_start
-----------+--------------+----------+--------+------------
35779 | den-12xxx0 | 17579 | 128 | 1531051778
(1 row)
Use Google Chrome or Mozilla Firefox and log into the Target appliance. Navigate to:
- CONFIGURE (page) > Appliances (tab) and select the name of the Target. Click the Edit button.
- Go to the Advanced (tab) and click on the General Configuration button at the bottom.
- Copy the word allowCleanupOutOfOrderBackups into the Name column.
- Click on the line item for the Section Replication. Set the Value to True, Click Save, Close, Save.
- Then, from the source system, execute the following command:
/usr/bp/bin/vcd util fix-paused-instance <target> <backup_no>
- Once the issue has been resolved, restore the allowCleanupOutOfOrderBackups (step 3 and 4 above value back to False and save the settings.
To do this from the source appliance run these commands:
Allow cleanup of out of order backup copies:
nTarget=`psql -c "select vault from bp.replication_queue WHERE status=128 LIMIT 1" | grep -v "vault\|row\|-\-"`; bputil -p -c $nTarget "Replication" allowCleanupOutOfOrderBackups True /usr/bp/bpinit/master.ini;
Revert back:
nTarget=`psql -c "select vault from bp.replication_queue LIMIT 1" | grep -v "vault\|row\|-\-"` bputil -p -c $nTarget "Replication" allowCleanupOutOfOrderBackups False /usr/bp/bpinit/master.ini;
CAUSE
This occurs when attempting to replicate a backup that is older than backups on the target system. Sometimes this condition may also be present after removing bad hashes and the associated backups.