There is a bible Psalm that says “The way of the fool is right in his own eyes, But the wise one accepts advice.” It is wise on your part to have skilled people advising you during disaster recovery planning. Why is that so important? This is because a plan explains in detail, what needs to be done, when, how, and by whom, which often includes best case (all planets align), what is expected to happen, and worst case scenarios (if everything goes wrong). This requires people who have a strong knowledge of the subject matter. In this case, what happens to your business when disaster strikes. Now, that may sound scary but if you simply close your eyes and hope for the best it can all be gone in an instant.

At the 2008 at the Republican National Convention, the former Mayor Rudy Giuliani stated “change is not a destination, just as hope is not a strategy”. That seems like a no-brainer to some but you would be surprised how many business owners ‘hope’ that nothing is going to happen to their businesses when a natural or man-made disaster occurs. Yes keeping your head in the proverbial sand, doesn’t help you to plan for the inevitable. So how does this relate to a disaster recovery plan?

As mentioned in the previous article, a disaster recovery plan (DRP) is a procedural process of recovering IT systems, in the event of the loss of a datacenter that is driven by a natural disaster or a man-made catastrophic event. What makes a solid DRP successful? It starts with planning. Why do I say that planning is the start of the plan? Well, it provides the mechanisms to bring together a framework that leads to a plan. It all starts with understanding what could happen.

 


 

Know The Face of Your Enemy

 

It can be said that a disaster is like a predator. One that is lurking and waiting to pounce on its prey. You don’t know when it’s going to attack, but you know it will. If you view these events this way, you will want to know how the enemy manifests itself.

Under the section “Types of Disasters”, a Wikipedia article called disaster recovery plan describes that types of disasters whether man-made or natural that could happen to your business. Now, depending on where you live and the type of business you have, these issues have varying risk. For example, say you live in California, your risk is much higher for an earthquake or wild-fire than a hurricane. In effect, your response would be much different from those at lower risk. However, since California is on an earthquake zone, you may also have man-made hazards like nuclear radiation accidents. Case in point, look at the fukushima nuclear disaster after a magnitude 9 quake in Japan a back in 2011. Nobody would have thought that a quake would have caused so much non-related destruction in its aftermath. In fact, this particular disaster had three parts: (1) the great quake which caused massive infrastructure damage, (2) a historic tsunami which destroyed tens of thousands of homes and businesses not to mention the loss of thousands of lives and then (3) the nuclear disaster which still continues today which rendered untold acreage unusable for centuries.

So this is the foundation for what comes next, the Business Impact Analysis.

 


 

Making Management Aware The Enemy Is At The Gates

 

Once you know who the enemy is, it’s vital to understand the impact these events could have on the business. The typical way of knowing this is through a Business Impact Analysis (BIA). This is not rocket science! This document simply looks at the risk factors and provides analysis of what happens should there be a business interruption. This includes the impact to operations, finances and the organization’s reputation. It also provides recommendations for minimizing risk such as a DRP.

Most people agree that something should be done for DR. The real issue becomes cost. That’s usually where IT and executive management part ways. This document now becomes the basis for selling the cost to the executive team. As mentioned in the previous article, there is much apathy with DR. Therefore, you’d want to have the correct facts at hand. The BIA should not be gloom and doom but, should be well balanced with a sound analysis of the facts all while providing value added recommendations that shows the enemy can be defeated if the organization is attacked. I typically like to show what happens if nothing is done, what happens if there is a half hearted approach and what happens if the organization executes a comprehensive plan.

I always have great dismay using the term ‘insurance policy’. Its a terrible…I mean TERRIBLE cliche considering the impact of what a DRP really means to the business. It’s so much more than an insurance policy, its is very much about survival of your organization! This is what has to be conveyed in your BIA when presented to the executive team.

Interestingly, TechTarget has a very informative article on the BIA that also includes a template for starting your own BIA. Now, they recommend that the BIA and risk analysis be done separately. I think with larger organizations, that is wise advice. However for smaller SMBs, this could be included directly in the BIA. That’s really for you to decide.

 


 

Know What You Need To Protect From The Enemy

 

The next part of the planning is understanding what is at risk. What you will need first is an audit of the server infrastructure and the applications that are hosted on them. This will give you a sense of the scale of the recovery plan. This becomes essential to making sure that the systems that need to be online WILL be online when the time comes.

Why is this important? Well, during Superstorm Sandy, there was a company that thought that they had an excellent DR plan. They even tested it several times to a degree of satisfaction. However, Sandy was a historic storm which flooded the basements of many businesses in NYC for weeks. Of course, this took down a lot of datacenters! It became apparent to this business that there were secondary systems that were owned by the business units and were not deemed critical IT infrastructure but they were indeed mission critical! This particular application (the payroll system) was missed on the DRP. Needless to say, this was the subject of major executive team discussions after the disaster.

This particular issue exposed a serious question of which business unit actually owns the application or server. Given the experience just mentioned, would it not have been better if IT knew all the systems and who owns them in their datacenters? If that would have been the case there would have been conversations with the business units to know what those systems are and what the risk is to business.

 


 

What Can We Afford to Lose To The Enemy

 

Have you ever watched a nature show that has shows how an animal in impossible circumstances, that could have potentially taken their lives, will make sudden and even extraordinary decisions that will save their lives at the loss of maybe a limb or appendages like toes or fingers. There was even a movie (127 Hours) and a best selling book (Between a Rock and a Hard Place) about a hiker, Aron Ralston who cut off his own hand to survive after it was pinned in between to boulders in a Utah canyon. In this case, Aron determined his life was of more value than his hand.

Of course, these are extreme decisions of what a person is prepared to lose in the event of a disastrous situation. However, organizations have to make similar decisions for the IT infrastructure since not everything has the same value to the business and there are systems they could lose that are deemed non-essential. These systems could either be totally lost or they could live without for a very long time.

In the last article we talked high level about what the recovery point objective (RPO) and recovery time objective (RTO). These two subjects are the next planning decisions that will need to be made after the initial server and application assessments are completed. These two issues describe what the organization is prepared to lose in the event of a emergency (RPO) and how fast they need to bring them back (RTO) because of the value it had to the business.

This process takes the assessment and provides a value to each server/application that’s in the list. The value whether by intrinsic or extrinsic factors can be weighted. This value then will of course be prescribed to a certain RPO/RTO.

Defining this step could take a lot of effort and conversation within the business. Once that is done, the organization can then plan on what technologies will be used to effectively enforce the RPO/RTO. These inherently have their own cost to the business. So a technology that keeps IT system running regardless of the event will be much more expensive than a backup technology that only protects the data. So the process of deciding where an server or application will fall in the RPO/RTO landscape is called recovery tiering.

 

Example of Recovery Tiering

Recovery Tier
RPO/RTO
Application
DR Technologies
1
1 Minutes / 5 Minutes
ERP Databases
Payroll, e-Commerce, Email
High Availability with Real-Time Replication
2
1 Hour / 6 Hour
Collaboration Application
Scheduled Data Replication
3
24 Hour / 1 One Week
Archival Applications
Legacy Applications
Backup and Recovery

This is a very simple example of a tiered approach to planning. But that’s not all….there’s still more.

 


 

Executing The Plan when the Enemy Finally Arrives

 

Does your organization has the technical know how to provide people, processes and products to accomplish this vital task. For many organizations, especially SME and Enterprises, they have assets and people already on staff to put this plan into action. They may even own their own DR datacenter. But what about the smaller organizations that does not have these resources? There are two options:

1. Build your own disaster recovery site with some outside help like systems integrators or DR specialists.
2. Completely or in part outsource building a private datacenter for DR or invest in a cloud service provider who offers Disaster Recovery-as-a-Service (DRaaS). Some also refer to it as just Recovery as-a-Service (RaaS). *

 

PEOPLE: DR requires people to work. These people must understand what people need to be in place during a DR event. This is crucial to a successful plan. These people (whether in house or a cloud service provider) should understand what needs to be done and have a practice of making a DR plan happen. People drive the DR plan and should have strong relationships with others that do.

People must be assigned to various duties to ensure the plan comes together properly. These duties should be part of their day to day responsibilities and not treated as an afterthought. If they are treated as an afterthought, execution will be at best flawed or at worst a complete failure.

PROCESSES: If the planning is done properly, recovery from a disaster event really becomes a matter of executing the documented set of refined procedures that comes from the planning stage. These processes are what drive human beings to during execution. They must be documented and tested over and over and over again on a regular bases. Why? Because the status quo can change slowly in the organization without anyone even realizing it. What is relevant now may not be relevant over the next year. Therefore you may end up with processes for issues that do not exist any longer or new issues have been raised to which no documented procedure has been developed.

PRODUCTS: There are a varying degrees of products on the market to address many aspects of disaster recovery based on the type of RPO/RTO an organization wishes to achieve. There are software products if an organization would like to manage it themselves or cloud services that are either fully managed or not. Organizations need to evaluate what will work best for them for the risks factors and costs involved. #

 


 

Conclusions

 

The person who presents the BIA MUST convince management that their advice is sound and is a wise course of action to implement a DRP by those who are their subject matter experts. Organizations without a DRP are putting their very existence at serious risk.

In addition, organizations must know their enemy inside and out and the plan must support defending against the inevitable attack. They should know what will be affected by the outage and what the plan is for recovery. They should also have a means to refine and test it. All this is driven by the value an organization gives to the server or application via RPO and RTO.

Finally, organizations must remember there is a price to pay for doing nothing, for doing something half hearted and for do it the right way. This certainly will give management the right perspective when planning for a DR event.

Were living in tumultuous times and the risks are higher now than ever before so prudent planning for the inevitable is a course of action of responsible executive leadership. Planning a DRP prepares an organization for what it should never have to face…a catastrophic event. The next article will describe considerations when building a DRP.

 


 

* Incidentally, the difference between them is still debateable though some say that difference is technology used to perform recovery. With RaaS a service provider might employ varying technologies to under a single umbrella such as the technologies listed above. DRaaS for some service providers may ONLY be second site recovery in the event of the production datacenter is destroyed or offline in prolonged event.

# This site offers insight details into what products are available and where they fit into the business continuity or disaster recovery objectives. These articles could be used by organizations to at least qualify what products or services best meet their needs.

Leave a Reply

Your email address will not be published. Required fields are marked *

*