Learn from the errors of their ways

French | Spanish | Portuguese | Italian | German | Japanese | Chinese | Korean | Russian | Arabic

Backup is a part of any data protection strategy, but there are other technologies, such as replication, that are part of it as well. The key to a sound strategy is to incorporate all the different technologies. I have been involved in too many discussions with people who were trying to recover from an outage only to discover they were not as protected as they thought. One particular case involved a company that had lost their primary server that ran their most critical application. They spent several hours trying to recover from mirrored disks, when the actual failure was filesystem corruption. Mirrors did not help in this case. Their outage was extended, but they were able to recover, since they had backups. My worst call while working in support was from a system administrator who had done a mass delete from the wrong window and had removed enough of the operating system that he could not reboot. When he asked what he had to do to recover, I told him part of the process would be to restore from his latest backup. To this, he answered that configuring for backups was on his 'list of things to do.'

If a storage area network (SAN) is available, it can allow for more flexibility in the backup and recovery strategy. The backup hardware can be better shared amongst the large data-resident systems while still keeping the data off the production LAN. This can also allow large systems to be backed up directly to tape without making the application servers general-purpose media servers and having these systems back up other LAN-based clients.

As you identify all the systems in the enterprise, you should note the specific recovery requirements of each system. This is very helpful in setting up the backup strategy. If an order-processing application can tolerate an eight-hour outage without severe business consequences, for example, an incremental backup strategy that minimizes backup time at the expense of restore time may be appropriate. For a Web retail application, on the other hand, where every minute of downtime means permanently lost sales, a strategy that replicates data in real time might be more appropriate, even with its greater impact on application performance. The other item to note is the order in which systems need to be recovered as part of your overall disaster recovery (DR) plan.

As you assess the backup requirements of each system, you should also make sure you know which of the database applications must be kept up- remain 'hot'-during the backup and which can be shut down to be backed up 'cold.' There are performance trade-offs involved with backing up a database while it is online, but sometimes this is necessary. This is due to the increased I/O activity, since the database activity is continuing, as well as the additional backup I/O. There are other methods of handling database backups, either hot or cold, using frozen image technologies and possibly off-host backup methods. You have several options for moving the data from disk to tape. Each has its own advantages and disadvantages. The methods include the following:

Files. This involves using the operating system to read all the appropriate files within the backup set and move that data from disk to tape. This method has more operating system overhead but allows for single files to be backed up and restored. It also enables the application to check each file to determine access or modification time so incremental backups can be performed.

Volumes. An entire volume can be backed up without reading the filesystem structure but by doing a bit-by-bit copy of the data from disk to tape. This is called a raw backup. This method allows for much faster data transfers but in general does not allow for single-file backups and restores. It also does not allow incremental backups. This backup method results in an entire volume being backed up, even the portions that do not contain valid data.

Block level. If the filesystem has enough information about the files, it is possible to determine which blocks have been changed. If the backup application can interface with the filesystem, you can back up just the changed blocks. This type of backup is called block level incremental.

Mapped raw backup. Some backup applications, such as VERITAS Software's NetBackup, can map a raw volume and then perform a raw volume backup while retaining the filesystem map so single files can be restored. This also allows for incremental backups.

Off-host backups. This is a mechanism where data is moved from disk to tape without the application host being directly involved in the disk reads or tape writes.

As you learn more about the nature of the data and the reasons for backing it up, you will probably find some data or some systems that do not need to be backed up on a regular basis at all. Many people have decided that the static OS files generally do not change except when OS patches are applied or the system is reconfigured. They do not have regularly scheduled backups of this data and only perform them when applying patches or making configuration changes. Also, some systems can be easily rebuilt online from other systems, which is usually faster than a total recovery. In this case, you might only need a single backup to protect all of these systems in your enterprise. In addition, there is a cost incentive to fully understanding the data. The cost of tape media is one of the driving forces behind not backing up too much data, as well as the fact that backing up data that does not require it ties up tape drives that could otherwise be used for more important data.

The real key to this entire discussion on determining why the data is being backed up is to truly understand the nature of the data. In a large enterprise, it is unlikely the backup administrator will have this knowledge. You will have to get this information from the people who control the data throughout the enterprise. The best way to do this is to develop a questionnaire that you can distribute. This will help you gather the information you need to correctly architect the backup and recovery system. It is common to use this kind of tool to determine when to back up, what to back up, how long to keep the backups, and where the backups should be kept. However, as we have seen, you also need to know why certain backups must be performed.

Keep in mind that just because you sit down and do all of this work now, you are not 'done.' Things change, and you need to develop an ongoing dynamic process that lends itself to constant growth. Gathering the information that will be needed from others in the company may be time-consuming, and it may not be readily available. Formulating a backup and recovery strategy may take a while, but the first step starts with, well, a first step.