SUMMARY
Details on how backups are stored, managed, and deleted by available retention policies, as well as how new backups are prioritized. This is using the legacy MIN\MAX model. This has been replaced with Unitrends Long term retention feature or LTDR.
ISSUE
Retention is one of the most fundamental benefits of any backup solution. However, without an understanding of how backups are stored, managed, and deleted by the available retention policies – as well as how your backup software prioritizes new backups landing – you could find yourself struggling to meet your retention policies and even lose data accidentally.
RESOLUTION
The Default Retention Policy
Unitrends Enterprise Backup and Recovery Series maintain a default retention strategy that is a First In First out model. When space is required, but default, the oldest data is first deleted.
Some data is protected from purging based on a few rules.
- Data in the current backup set cannot be purged. You must always have space to create a second master backup before the first master of a client can be removed.
- Data pending replication in the job queue cannot be purged until replaced by a newer job or successfully replicated
- Data flagged for legal Hold cannot be automatically deleted
Normally, deletions are estimated, and data is removed 24 hours in advance of storage need. The appliances reviews historical backup loads vs currently available free space within the logical backup device and determines how much data must be removed. This data is deleted ahead of this need to ensure free disk is already available at time of backup, allowing the fastest ingest rates for new data.
In some cases, you may not want to keep the same retention policy for all clients. Some server data may not have valuable historical data and may only need short retention policies. Other servers may be subject to extended online recovery RPOs or legal hold requirements and must have significantly longer retention policies vs other systems. This can be readily configured in the Unitrends appliance on a per-asset, per VM, or per database level.
Before we cover how to set varios retention settings, it is important to understand what they are. There are 3 retention variables:
- Legal Hold, also called "Keep backups for N days". A backup less than this many days old cannot be purged.
- Minimum, also called "Warn when less than" When retention of a client falls below this value a trap is generated as a warning.
- Maximum, also called "Delete backups after X days". A backup that is older than this will be purged even if free space to land new jobs exists.
When to Use Legal Hold Retention Settings
If you want to ensure that your older backups are NEVER deleted to make space for newer backups until a certain point, then set the Keep backups for N days option. This applies a standard retention policy that will ensure that no backups are deleted unless they are older than N days. If you run out of storage space before meeting your retention of at least N days, new backups will fail as old data will be unable to be removed.
WARNING: If the appliance is full of backups and all remaining backups are subject to legal hold, incoming backups will fail. It is critical to set legal hold to a low enough value such that some backups are always outside of this threshold. The best practice is to set Legal Hold to a value at least one full backup rotation less than the desired backup retention maximum.
Important note, Legal Hold protects every backup in a chain going back to the chain's full (called the head). Because Unitrends can only purge the entire chain at once (keeping an incremental without keeping it;s full would render that incremental unrecoverable, so we always keep the full too). This means that if legal hold is set to 15 days, we cannot purge any part of a chain untuil every part of the chain is over 14 days old. Typically when using incremental forever backup strategies as recommended a synthetic full happens every 14 days. It is possible that the head "full" of a backup chain would be as old as 29 days before the most recent incremental dependent on that full is more than 15 days old. To set a legal hold of 15, you would need to ensure you could always exceed a retention temporarily of 29. If you cannot meet that higher number, legal hold may cause backup failures due to out of space conditions.
Legal Hold is typically used in conjunction with a regimented cold copy archive strategy, or hot copy replication to ensure 2 or more copies of data always exist for recovery. In most organizations subject to disaster recovery compliance (HIPAA, SOx, PCI, HiTEK, etc) data must not only be kept for a specific period of time, it is usually the law that at least one copy be offsite within 7 days (some compliance is next business day). In these cases, Legal Hold is used to ensure the data is not locally purged until it has reasonably been archived to another offsite media. Typically a legal hold of 14 or 30 days is used which is more than sufficient to ensure other copies of compliance data have been made or issues with such have been remediated. In these cases legal hold is also typically only used for systems that are subject to that compliance, but not all systems typically are.
In other cases, you may wish to ensure selective files for certain systems are available for online recovery quickly without the need to import from other copies and a longer retention may wish to be enforced. In these cases, legal hold may be a larger value. For example, a file server may tend to have users periodically request individual files or folders to be recovered with great speed going back many weeks or even months. Generally these higher settings are applied selectively only to specific servers that require them, and are not used globally.
When to use Minimum Retention Settings
This setting has been more aptly renamed to Warn when less than N days of backups remain option. The idea is backups CAN be purged to below this minimum value, but you as an admin would like to be expressly notified when that is the case. Should the appliance need to purge data, any data that is not subject to the never-purge rules above can be purged, even if it would fall below this limit. Minimum in this case is an "expectation" of retention you would always like to remain above, but permit going below. Typically this value is set to a number that is greater than the legal hold value by one full backup rotation. Setting this value less in combination with legal hold would effectively leave legal hold as the minimum.
If you do not want trap warnings when clients fall below expected retention values, then this value may be left at the default "0". The value for this setting does not prevent or influence any purge activity, it is only a value under which we generate traps/alarms. When you are not reliably able to meet your setting for this value, you may need to address your data set size protected, increase the storage available for backups, or change rates of the data protected on client systems, or reassess your business needs for data retention.
When to Use Maximum Retention Settings
When a defined Max retention is set for a client, backups older than that age will be purged, including all backups in the chain for that client dependent on the full.
This setting may be used on servers of less importance where only current state recovery and not long term retention is required, to enable additional free space to be used for other clients, or in some cases where a business does not wish to have data beyond a certain age to limit discovery scope. Remember data is purged as a set, and the current set can never be purged, so a setting for this shoudl always be at least a full backup rotation larger than the setting of MIN.
A warning will be generated in the UI when a client is above MAX for more than 1 day. Is MAX is set too restrictively, or legal hold is set too close to MAX value, it may not be possible to purge below MAX due to other rules on purging data. To avoid false positive errors, always ensure MAX is set to at least one full backup rotation of days larger than either MIN or Legal Hold.
Note, Setting MAX for all clients may result in less than complete use of available storage as data will be purged based on age instead of the default rule only when space for new backups is needed. MAX should be used only when you intend retention to be expressly limited for some clients and do not intend to retain "all that is possible" which is the default behavior. In rare cases where incoming expected backup load may vary wildly from day to day, or where manually running unplanned or unscheduled backups is common, support may recommend setting MAX values on some clients to retain a larger available landing zone for new data.
Warning: When using incremental forever strategies, setting MAX to a value less than MIN + 15 can cause synthetic backups to happen more frequently than desired, which may actually negatively impact retention or appliance performance. If scheduled fulls are used, MAX should always be a value at least the number of days between fulls + 1 larger than MIN to avoid unnecessary retention warnings in the UI.
TASKS
Customized Retention Settings
Retention may be customized for each protected asset (a VM, server, database, etc.). To do so, open the user interface and:
- Select "Configure" from the left menu
- Select the "Protected Assets" tab at the top of the interface
- Check the box next to the asset for which retention settings will be changed
- Click the "Edit" button at the top of the asset list
- In the Edit Asset dialog, select "Manage Retention" at the bottom to view and/or change the retention settings.
The settings are also described as follows in the Administrator’s Guide:
To ensure that specific backups are never deleted by any retention setting, reference: Keep backups forever
NOTES
Unitrends recommends that implement Backup Copy Best Practices and utilize a minimum of a 3-2-1 archive model or 3-2-2 archive model for compliance and best practice disaster recovery. See our site blog.unitrends.com and look for our articles on these strategies for more information. The basic assumption is that all customers have 3 copies of data at all times, on 2 different types of media, one of which is always offsite. Unitrends offers a suite of options for performing Cold and Hot Backup Copies to an array of devices and services to make this cost effective. Please remember, a Unitrends appliance is a server like any other, (a very busy heavily utilized server with critical database architecture that manages deduplicated and encrypted data) and is subject to many types of failure which could result in data loss. Depending solely on local appliance retention without copies may eventually lead to data loss scenarios and could be a violation of government regulations on your business vertical. Please always leverage proper power management including use of automated power shutdown services to protect your critical systems, including Unitrends, from an unplanned power outage.