Stories about Hurricanes Katrina, Rita and Wilma hit the front pages of newspapers in September and October last year. Most of the stories focused on the human tragedies and the failure of our government.
What you didn't read about was how business continuity planners were taken by surprise. It was the floods, the mass exodus of people and the resulting problems that were unexpected and caused problems particularly in the print-to-mail recovery department. By taking a look at what actually happened to several companies forced into recovery by last fall's hurricanes, printing and mailing department managers, working in concert with their corporate business continuity planners, can use the lessons learned to help ensure their companies won't face similar problems.
Let's begin by looking at one actual recovery timeline.
You want to hear about backlog? Has anyone planned how to manage a two-week backlog in processing, while staying current with incoming processing? Sometimes it takes a disaster, like a Category 5 hurricane, to put the realities of print-to-mail recovery in perspective.
What was learned as a result of Katrina, Rita and Wilma? Clearly, recovery efforts demonstrated that best practices make the difference. In print-to-mail recovery, there are three best practices you need to recover efficiently. This is where print and mail managers need to work hand-in-hand with the business continuity planner. Without knowledge and guidance, complete plan cannot be written by a planner.
Best Practices for Efficient Recovery
1. All of the mission critical print-to-mail applications and equipment must be included in the plan and covered in the contract.
2. Test the plan all of it. Print-to-mail recovery should be included in the data disaster recovery testing.
3. Plan for a worst case scenario.
Best Practice #1 Cover everything in the plan and in the contract. One financial institution did not. Its contract covered only enough equipment for small-volume print services (i.e. internal reports). But because it is a financial institution, recovery of statements and bills was critical. They knew they didn't have all of their critical applications covered and then took 12 days to determine how and where to recover their statements. They declared on September 1, but their data didn't start arriving at the print-to-mail recovery site until September 13. This two-week hesitation created a major backlog of client statements and bills. They were faced with a major backlog and incoming processing but didn't have enough equipment to handle both.
Did they recover? Yes. The print-to-mail recovery provider had excess capacity enabling the company to get what it needed but at a price. Because they had failed to include what they needed to recover in their contract, they had to pay a premium for the additional equipment extra printers and inserters (they didn't have a single inserter included in their plan or in their contract). Lucky for them, the recovery provider not only had the capacity to handle normal production requirements but also enough inserters to clear the two-week backlog.
This situation could have turned out much differently. When a print-to-mail recovery provider is recovering multiple clients, capacity and equipment are at a premium. The recovery provider must abide by the contracts in place... and if you have only planned for and contracted for a fraction of what you need, that is all you are going to get. So in a worse case disaster, you may not be able to recover. After what happened in
Best Practice # 2 Test the plan. Another company knew that print-to-mail recovery was somewhere in its corporate disaster recovery plan but the employees in the printing and mailing departments had never seen the plan, let alone tested it. They did not even know who their print-to-mail recovery provider was until disaster struck.
This company declared a disaster on September 9. The fact that data started being transmitted from the hot site on September 14, five days later, was bad enough. But the fact that they had no specifics on the data being transmitted that was like a second disaster. What application was being transmitted, what paper was needed and how long was the transmission? No one knew. They couldn't provide start commands or page or form definitions. You can't get any · more unprepared. It was weeks before the process began running smoothly. What can go wrong if you haven't tested? Another company had problems right from the start. First, they encountered start-up issues due to a recent system change they hadn't made their hot site provider aware of.
Data was to be transmitted through a T1 connection from the hot site to the print-to-mail recovery vendor using a host direct to printer configuration. Direct connect at time of disaster is not a best business practice. Here's what happened. The undocumented hardware change required the hot site vendor to implement a work-around to enable data transmission. The work-around made it impossible to know what exactly was being transmitted. The result: there was no way to prioritize what jobs they were transmitting and no way to know if an entire file had been received and printed. In short, there was no verification process. This causes an entirely different problem than processing; it creates a violation of the Gramm-Leach-Bliley Data Protection Act. Compliance requires the ability to validate.
If they had conducted just one complete test (meaning data transmission from the hot site and all of their critical applications), they would have had all of the equipment they needed when they needed it; could have prevented wasted time and wasted paper; would have had the documentation they needed to enable them to prioritize the jobs, thereby significantly shortening the time it took to make up their backlog; and more importantly, would have had the tools to validate the printed output.
Has the business continuity planner at your company requested print-to-mail be included in a data recovery disaster exercise? Have you ever tested your entire process even in a standalone print-to-mail recovery test? Have you tested the timeline of shipping or ordering forms and envelopes for delivery to your print and mail recovery provider? Not one of the clients referenced here had plans in place for shipping specialized forms or envelopes. They found work-arounds or they had to wait for the materials to arrive. One client used a blank envelope with labels for return addresses and for large window envelopes they used labels for addresses. Then how about postage? How will you handle paying for postage? Is the money easily accessible? And what if no one who helped develop the plan is around to help manage the print-to-mail recovery?
Best Practice #3 If you have a plan, you have a recovery team, right? Not necessarily. You need bench strength backups to backups. Many companies learned this lesson after the
The special circumstances of Katrina and Rita exacerbated an a
Some knowledgeable employees did come to the recovery facility, but their families had evacuated and their homes were at risk... were they giving 100% of their attention to recovery? How could they? Still others came and oversaw what was happening for three days and then left the production work to the vendor's staff. Leaving your company's processing solely to the vendor staff is not a good business practice. There is no way they can know everything your production staff knows about your printing and mailing processes and requirements. Your staff lives it all year, not once a year at a test.
Bench strength. Make sure everyone in printing and mail knows there is a recovery plan, and everyone who has a potential role in the recovery, even if they never participate in a test, has the plan either on disc or in hard copy. Tell everyone to bring the plan with them at the time of a disaster. Even if nine people forget, maybe number 10 will remember. You have to have a staff backup plan.
The reality following Katrina and Rita? Most people who arrived at the recovery facility did not know the steps outlined in their plan and/or what was covered in the contract. Not only didn't they know what was in the contract, but they said "not my job to know." If you want to recover efficiently, EVERYONE in your print and mail departments needs to know and should have a sense of ownership for a successful recovery. Finally, there was a company that actually employed the best practices outlined and took all of the recommended steps to prepare for a disaster, declared as a result of Wilma. They wrapped up their disaster in five days.
Best practices really do make a difference. The proof is as easy as 1, 2, 3.
Gerald A. Montella serves as the Vice President of Mail-Gard. Additional information can be found at www.mailgard.com.