As the storage solution provider model falters, banks are developing in-house systems to support information backup
Although digital information often ends up as an intangible on the balance sheet, keeping corporate data both safe and accessible requires substantial investments in all-too-tangible property, plant and equipment. Accordingly, financial institutions of all sizes face the buy or build decision-whether to outsource applications and storage to dedicated technology partners, or to build in-house facilities with homegrown talent.
For Southwest Corporate Federal Credit Union, based in Dallas, the answer was to draw upon the expertise of its own 23-person IT staff to build redundant facilities. Southwest Corporate FCU provides its business customers investments, automated clearinghouse (ACH) processing and, most recently, check capture and same-day check imaging. As a service provider to numerous smaller institutions, the organization needs to maintain operations during adverse conditions.
"We've been establishing a hot site where we'll have redundant data at our headquarters and at our co-location facility," said Sam Palmgren, senior project leader at Southwest Corporate FCU. "We kept the hot site in the Dallas area, but we moved it about 20 miles away."
Following the common "2+1" strategy of two mirrored servers plus a periodic offsite backup, Southwest Corporate uses data management software from EMC to mirror its Dell servers across two locations, and Double-Take from NSI Software, Hoboken, N.J., to create an additional copy of the data.
Before implementing the mirroring solution, the daily backup required four to six hours per night. What's more, disaster recovery would have taken days. "We would have had to purchase the servers and then load from backup tapes," said Palmgren.
Smaller organizations have also incorporated offsite storage into their disaster planning. With only four IT employees, a consultant and a manager, Santa Ana, Calif.-based First American Trust has taken business continuity into its own hands-up to a point. Although the company decided not to go with an enterprise-wide storage outsourcing solution, it still has several relationships with major application service providers, who in turn are protected by comprehensive disaster recovery procedures developed under Federal Reserve oversight.
First American Trust is in the process of establishing a backup facility at a San Diego branch office of its parent company, First American Corporation. The San Diego site has an existing computer room, complete with backup generators and uninterruptible power supply facilities.
"We already have the air conditioning and the power we needed, so it was just a matter of clearing out some space and throwing in some racks," said Henry Jenkins, vice president of information services at First American Trust.
The solution for First American was SnapMirror, a centralized storage system from Network Appliance, based in Sunnyvale, Calif. First American's wealth management business has employed SnapMirror for almost three years.
"We realized we were running out of storage on each Windows-based server that we had, but we also realized that the marginal cost of additional storage was outrageous," said Jenkins. "Centralizing the storage made sense."
Once fully deployed, SnapMirror will perform block-level replication of First American's network-attached storage devices between the Santa Ana and San Diego sites, located 100 miles apart.
The San Diego site will also maintain backup data for First American's various business units. In addition to providing wealth management, First American Trust also acts as the commercial bank for its parent company's title and escrow services.
"We're hoping that if you do escrow through a First American company, that the actual deposit will be held at First American Trust," said Jenkins.
The commercial bank will use Mimix, from Lakeview Technology, based in Oakbrook Terrace, Ill., to backup and restore its core account processing application, Kirchman Bankway from Kirchman Corporation, Orlando, Fla.
"We're going to be doing data replication to another AS/400 that we're going to have in our San Diego site," said Jenkins.
The company's business continuity plan includes the re-establishment of contact with its outsourcing partners-SunGard for trust accounting, S1 for online banking and Bankserv for wire processing-in the event of a disaster.
In such relationships, the responsibility for physically protecting the data falls upon the service provider. "They're held to an even stricter level of disaster recovery than we are," said Jenkins."They communicate with the Fed, which actually puts them under more scrutiny than we are with just the Office of Thrift Supervision watching over us."
OUT OF THE ZONE
Bankserv, a San Francisco-based provider of wire transfer software offered via site-license or outsourcing agreement, protects its clients' data by moving it far from its corporate headquarters. "We have a primary site and a backup site," said David Kvederis, president and chief executive officer of Bankserv. "The primary site is not even located in California. We wanted to keep it out of the earthquake zone and in a stable electric power environment."
Even during an emergency, customers of the outsourced service would still be able to comply with an urgent information request from law enforcement. The "120-hour" rule in the Bank Secrecy Act, as amended by the USA PATRIOT Act, puts a strict time limit on banks' response to information requests from law enforcement.
For its part, Bankserv keeps six months of transaction history online and also provides access to older records on demand, thus helping banks meet the new record-keeping challenge. "What we do is provide them with a tool that makes it more likely that they won't have an error, and more likely that things will get done in a proper fashion," said Kvederis.
But outsourcing has its limits, especially where risk is concerned. "The banks never escape ultimate responsibility," said Kvederis.