Did you forget something? Business Continuity and Disaster recovery in Corona virus times.

In the times of Corona virus, Business Continuity and Disaster Recovery became the hot potatoes for all businesses around the globe.

I do not recall a healthcare incident of that spread and that effect to the global economy. Logistics paralyzing, commerce experiencing its worst days in the past decade and recession numbers coming out of economists’ worst nightmares.

There’s a question here. Were businesses prepared for that? And when someone starts analyzing this question understands that there are more questions to be answered.

• Was cash flow able to withstand such a turnover drop?
• Can I just press pause in my business?
• Will businesses be able to continue the day after this?

A lawyer could easily call this force majeure, which as a term stands for unforeseeable circumstances that prevent someone from fulfilling a contract. Yes, covid-19 is a force majeure. Governments, tax offices and contracts can be paused, and obligations postponed, but what happens next is something you cannot predict.

The past two decades, consultants started talking about Business Continuity and Disaster Recovery, using the acronyms BC and DR accordingly. That started changing the business attitude towards the “continuity” of what we do for living. Many organizations, world-wide, spent thousands of dollars investing to continuity. It worked for some of them, it didn’t for some others. The amount of BC investment is correlated with people’s opinion regarding risk and can be parallelized with our view regarding whether we should buy a health insurance product or not. The idea “come on, it won’t happen to me” is one of the things many of us appear to reconsider after the covid-19 outbreak. Yes, it did happen to you one way or another. Either you lost someone you loved or knew, either because part of your business does not work as you planned, or even if you did not get that fancy phone or smartwatch you ordered from an e-shop. It affects human resources, and this subsequently affect businesses.

BC means a lot of things, IT infrastructures, alternative supply chain paths, alternative goods and providers and the list goes on. I won’t speak for the financial or commercial part of this crisis, but as an IT Professional and business owner connected to various other businesses I do see many companies facing the problem of not having the required personnel to fulfill daily simple tasks that they would perform when personnel was in the office, or cannot take advantage of a market gap they could fill just because their personnel is away and home isolated.

Working remotely has become a new trend the past 5 years, and if it works in the corona-days, it will change a lot of things regarding labor in general, after this corona-thing comes to an end.

People ask me: “Can I do it? Is it safe? Can my infrastructure support it?” Many companies are opening unpatched servers to public, while hackers are in queue waiting to have their annual gathering on their systems. While under pressure, we humans, tend to make mistakes, and when someone takes advantage of that mistake this is when real problems arise. Just imagine that you are facing this healthcare crisis, having personnel working from home -with reduced productivity- and in addition you become a ransomware victim, just because someone took advantage of a security flaw in your company’s infrastructure. Can it be worse? Well, it’s rather common those days.

ISO 22301 describes what a BCP (Business Continuity Plan) should include. When carefully designed, it will provide management an emergency walkthrough, however it does not clearly tell you one little thing: “Make sure you didn’t forget anything”. There are many organizations that certify their BCP in line with ISO 22301, just because they were obliged by legislation or their fellow CEO did it as well. I have seen numerous examples of BCPs that are a copy-paste product combined with a 10 hour effort on editing documents and procedures based on someone else’s similar system. And yes, you may have your BCP, which you are proud of as you paid a lot, but you forgot 1.000 things just because you never went through the process of building it yourself. The human brain considers thinking just as cats consider water as a threat. Cats don’t have a problem in water, they won’t get drowned, they just don’t like it! Brain does not have a problem thinking, it was made for that in the first place, but most of the times it prefers not to, especially when under pressure. Unfortunately, BCP is meant to be used in the case of a real emergency and if you want it to be useful, make sure you prepare it properly by putting yourself in the delivery team.

As an IT professional in Creative People, I have the privilege to explore various business cultures, meet people taking decisions and continuously evaluate BCPs and DRPs. The management’s “true” view upon their BCP and DR defines them as a value-added customer or not. I have customers that could not afford a BCP in line with ISO 22301 and put in their plan a plafond of what they could do and what they could not. They knew their limitations and they were a-priori prepared that on an emergency they would be able to support just an X% of their day to day workflow. Believe me, this is the correct way to do it. You will never be able to cover a 100% of your company when bad times come, such as Covid-19. Knowing though that those parts of your business will work under Z circumstances is like knowing yourself and where you can go.

Disaster Recovery (DR) is a part of your BCP. To be exact, it’s the worst scenario stated in your BCP, in which everything is lost and describes how quickly and to which extent your business will recover the day after a disaster. A disaster such as an IT system imminent failure or loss or a force majeure.

We have a customer that on November 2018 experienced a break in and in no time, burglars, bypassing all security measures, removed a 42U rack with their IT infrastructure, weighting more than 400 kilos, put it on a truck and disappeared. Alarms, security fences and systems, all failed to stop them. This customer was following a DR of daily off-premises rotating backup. This means that every day, they took their last night’s backup outside their headquarters. Unfortunately, the previous night management left both backup disks inside their backup machine. This incident resulted the loss of the off-premises backup element from their DR strategy. You cannot treat this as a design fault, it’s clearly a human mistake and in addition, nobody could imagine that a group of 20 people would break in and remove -just- the most precious part of their business, their IT system. You cannot cover all risks. Even if you deploy a disaster recovery site under continuous replication, you cannot cover all potential risks. As long as emergency does not take place, and you are in the planning phase, you have the privilege of deciding what to take and what to leave, when this force majeure will take place. It’s rather impressive that this customer is still up and running!

BCP and DR are risk assessments and in risk assessments you define scenarios. Spend a weekend thinking of those scenarios, use your imagination, think as the attacker, think as the defender and put in as many scenarios as you can. It’s rather painful, as you will face your fears and find out more of them, especially if you are the owner of this business. Different scenarios may have solutions that could walk similar paths, therefore efficiency and feasibility come in.

Covid-19 put the world’s BCPs to a test. All our customers requested remote access for personnel during those difficult times. We have a customer that thought his BCP would work with just Office 365. Can you imagine what happened when a company of 250 employees stopped going to their offices and had to login to their Office 365 accounts? 69% of the employees called our service desk requesting their password, or a password reset as they did not remember what they had put, while 29% had lost their emergency user manual in which the simple sign-in to https://portal.office.com with this username and that password was described. Oh, and I should not forget the ones that did not have a pc at home. Some of them have latest iPhones though. I should also mention that 16%, while on their Office 365, asked us “how will I login to my Backoffice system”. 2% of them where frustrated and angry when we informed them that their Backoffice system is not accessible over the web, as at first, it’s a desktop application, and as their management decided that this need should not be covered in their BCP. Problem was solved when we incorporated Remote Desktop services over IPsec VPN. Management failed to cover major part of the business needs in the BCP and the BCP implementation failed due to lack of preparedness on the people part.

We make plans, we design and implement but when it comes to deployment, those plans are supposed to be implemented and supported by people, and people are either not well educated or just because they never thought it would happen to them, they were not prepared. Did this BCP work? I would say it did, as there was a small and committed team to support the situation. Every BCP needs its heroes but it’s not always the IT, it’s also the ones who fund those plans and moreover take the decision to put a line on what will be served and what will not, in the unlike scenario of an emergency, just like covid-19.