10 lessons learned in Sandy's aftermath on disaster recovery and business continuity
The impacts of Hurricane Sandy have crystallized many executives' minds on the importance business continuity planning. Lessons learned in the on-going aftermath of this event have to do with ensuring that plans work when they are needed.
- Utilize Cloud service providers who have their facilities located in a multiple geographical areas other than were you are: With the extensive outages that occurred with Sandy, companies that had facilities in Manhattan found that back facilities in New Jersey did little or no good.
- Assume your company will be the last one that will be back in operation: Once the fanfare of everyone wanting to help wears off, many companies found that they were the last ones that the ‘government recovery' agencies were apple to help. Companies with the greatest amount of ‘political capital' were the first ones that were helped.
- Consider the safety and personal needs of your employees: Once the event occurred, many of the employees had to deal with their own personal needs. This included medical, food and water issues. Have a plan in place to address this. For example within 3 days of Sandy FEMA ran out of water and had to wait several days for it to be shipped in. In addition a number of hospitals in the disaster area had to be evacuated.
- Have enough generating power to meet long term needs: Identifying your facility's critical loads is important. Understand the costs and risks associated with utility power interruptions, production losses and downtime. Plan for outages of a few seconds to extended outages of one to two weeks. While other back-up electrical supply alternatives may exist, they can often take longer to engage and have shorter supply capabilities, have higher costs, lower reliability or no reasonable refueling options during an event. With Sandy the outage was so extensive that facilities had to be evacuated.
- Install standby generator in protected areas: Work with qualified generator engineers to properly specify, locate and install generator units. Be sure that the units are properly sized to handle vital loads and installed by certified electricians and power experts and in accordance with all state and local fire and environmental operating codes. In Sandy with the storm surge, many generators were destroyed by the rising water.
- Have sufficient fuel storage: Diesel fuel's energy density and the engine's high efficiency allow for smaller fuel storage facilities compared to other fuels, which provides a cost savings to owners. Still, it is important to make sure that you have sufficient fuel storage capacity on-site for an extended outage of several days, and contingencies for refueling during a serious weather or other event in the event of extended outages. With no power in place fuel could not be delivered.
- Have source of fuel for your employees: With an outage the size of Sandy many people could not get to work without driving and they had limited fuel. Public transportation was not available and there were few if any gas stations that were open and proving fuel in a timely manner. Consider having a stockpile of gasoline available to refuel employee vehicles.
- Have an alternative source of food, water, and fuel distribution and plan to get it: If an outage goes over several days, supplies will be used up and generator fuel storage will need to be replenished. In the case of Sandy so much of the infrastructure was damaged or destroyed that once the outages lasted more than one week generators just stopped.
- Frequently test readiness and keep equipment in top operating condition: Business continuity plans, including standby generators, should be ‘exercised' periodically to ensure they will operate as designed in the event of an emergency. Proper maintenance and servicing are the key to reliability.
- Contract for reliable providers of resources: If installing your own standby generation is not feasible for your business, you might consider contracting with a reliable firm to reserve rental generator power for use in the event of an extended outage. In the case of Sandy so many companies needed power that an instant shortage occurred. Even those who contracted for this service were not able to get the equipment they needed.
- Disaster Recovery and Business Continuity Top 10 “Disaster Recovery and business continuity are all about being ready for everything. The question that every IT manager and CIO has to answer every day. . .
- Options for a data center disaster recovery strategy Data Center disaster recovery strategy - options A critical component of a disaster recovery business continuity is the data center disaster recovery strategy - Hot. . .
- Sandy shows that not being prepared can be fatal to an enterprise Business Continuity Plans Are Expensive A company's disaster recovery and business continuity programs would be incomplete without covering compliance risks and without using compliance tools. . .
- High Availability Versus Disaster Recovery High Availability High Availability is when A machine that can immediately take over in case of a problem with the main machine with little down. . .
- Top 10 Selection Criteria for a Disaster Recovery Cloud Provider Cloud disaster recovery business continuity When looking for cloud providers of Disaster Recovery and Business Continuity Services you need to establish that they will be. . .