SUMMARY
Troubleshooting XenServer Snapshot Issues (SR_BACKEND_FAILURE_44 and _109)
ISSUE
Customers Running Citrix XenServer environments may encounter errors during the backup process specifically related to the snapshot process.
RESOLUTION
SR_BACKEND_FAILURE_44
PHD Virtual Backup uses XenServer's snapshotting feature during the backup process. However, when checking the logs, you may encounter a DEBUG message that includes SR_BACKEND_FAILURE_44 insufficient space. This error indicates that XenServer was unable to create a snapshot for the VM because your Storage Repository didn't have enough free space. The process of snapshotting requires additional overhead on your SR, so you do need sufficient room to perform the operation.
Please check that you have enough space in the SR where the VM resides to take the snapshot and that you can manually snapshot the VM. If you cannot manually snap the VM, you will need to free up additional space by adjusting your XenServer configuration. For additional details, refer to the Citrix Knowledge Center Document ID CTX134190 found here: http://support.citrix.com/article/CTX134910.
If you can take a manual snapshot, this error may be the result of the sum of the size of all VMs on the SR that that are being backed up simultaneously since they will each require a snapshot during the backup process. You can try to reduce the number of data streams for the VBA or create separate jobs running at different intervals so that the snapshots are not taken at the same time.
This error can also be associated with Citrix's snapshot space reclamation issue. If you are affected by this problem, you would see utilization on your SR that adds up to more than the total of your visible vDisks. Due to this, you would expect to have more space available on your SR than you actually do, which could cause your snapshots to fail. Citrix has provided a work-around, called Leaf Coalesce, or Coalescing. Coalescing can provide an effective means for keeping this unreclaimed space issue at bay. For XenServer 5.5 and 5.6, please refer to Citrix Knowledge Center Document ID CTX123400 found here: http://support.citrix.com/article/CTX123400. For XenServer 6.0 and 6.1 p lease refer to Citrix Knowledge Center Document ID CTX130420 found here: http://support.citrix.com/article/CTX130420
SR_BACKEND_FAILURE_109
If you encounter a DEBUG message that includes SR_BACKEND_FAILURE_109, XenServer was unable to create a snapshot for the VM because the snapshot chain was too long. The process of snapshotting and removal of the snapshot is done through a call to the Citrix API.
You may be able to clear this error by manually deleting snapshots you have created or running an sr-scan on the affected storage repository.
This error is most often associated with Citrix's snapshot coalesce issue. If you are affected by this problem, you would see utilization on your SR that adds up to more than the total of your visible vDisks. Due to this, you would expect to have more space available than you actually do on your SR, which could cause your snapshots to fail. Citrix has provided a work-around, called Leaf Coalesce, or Coalescing. Coalescing can provide an effective means for keeping this unreclaimed space issue at bay. Please refer to Citrix Knowledge Center Document ID CTX133470 found here: http://support.citrix.com/article/CTX133470
CITRIX 6.1 Hot Fix Available
If you are running XenServer 6.1 with a storage repositories (SR) greater than 1 TB you may experience issues deleting snapshots. Citrix has released Hotfix XS61E015 - For XenServer 6.1.0 to correct the issue. Information about downloading and applying the Hotfix can be found here: http://support.citrix.com/article/CTX137996
This hotfix may address one of the root causes of the SR_BACKEND_FAILURE_44 or SR_BACKEND_FAILURE_109 errors that may have been encountered when running backups.
XENSERVER LOGS
/var/log/SMlog: Reports coalesce process fails and provides stacktrace..