September 20, 2010

Software analyst Curt Monash has offered an explanation for the JPMorgan outage last week, in which online and mobile banking were down for more than 24 hours. The bank itself said on Friday that a third-party database company’s software had caused a corruption of systems information, disabling its ability to process customer log-ins to

According to Monash's blog, which is based on insider tips from "a credible source close to the situation," the following is what occurred:

* The JPMorgan Chase database outage was caused by corruption in an Oracle database. * This Oracle database stored user profiles, which are more than just authentication data. * Applications that went down include but may not be limited to: o The main JPMorgan Chase portal. o JPMorgan Chase’s ability to use the ACH (Automated Clearing House). o Loan applications. o Private client trading portfolio access. * The Oracle database was back up by 1:12 Wednesday morning. But on Wednesday a second problem occurred, namely an overwhelming number of web requests. This turned out to be a cascade of retries in the face of – and of course exacerbating – poor response time.

The blog also offered that $132 million in ACH transfers were held up by the JPMorgan Chase database outage, that about 1,000 each auto and student loan applications were lost due to the outage, that the Oracle cluster has 8 biggish Solaris boxes (T5420 with 64 GB of RAM); and that EMC is the storage provider.