The National Security Agency’s massive new Utah Data Center has reportedly been suffering from technology meltdowns for months, according to a Wall Street Journal report.
Ten meltdowns have occurred in the last 13 months, apparently preventing the spy agency from using its computers, the Journal reported, citing documents it obtained.
An official working on the project described the electrical surges as “a flash of lightning inside a two-foot box,” according to the newspaper.
NSA spokeswoman Vanee Vines seemed to suggest the problem is not a current one.
“The failures that occurred during testing have been mitigated,” Vines said. “A project of this magnitude requires stringent management, oversight and testing before the government accepts any building.”
Late last month, an NSA spokeswoman said the data center has been coming online as equipment is installed.
Here’s more from WSJ regarding these infrastructure issues (emphasis added):
The data-center delays show that the NSA’s ability to use its powerful capabilities is undercut by logistical headaches. Documents and interviews paint a picture of a project that cut corners to speed building.
Backup generators have failed numerous tests, according to project documents, and officials disagree about whether the cause is understood. There are also disagreements among government officials and contractors over the adequacy of the electrical control systems, a project official said, and the cooling systems also remain untested.
The Army Corps of Engineers is overseeing the data center’s construction. Chief of Construction Operations, Norbert Suter said, “the cause of the electrical issues was identified by the team, and is currently being corrected by the contractor.” He said the Corps would ensure the center is “completely reliable” before handing it over to the NSA.
But another government assessment concluded the contractor’s proposed solutions fall short and the causes of eight of the failures haven’t been conclusively determined. “We did not find any indication that the proposed equipment modification measures will be effective in preventing future incidents,” said a report last week by special investigators from the Army Corps of Engineers known as a Tiger Team.
The Wall Street Journal reported that the first system failure occurred in August 2012 and the latest happened Sept. 25, 2013, each causing up to $100,000 of damage. Only the cause of two of the 10 failures were identifiable, but quality control issues to speed up the project were mentioned in the report.
While government contractors said they had found the “root cause” of the problem and proposed a fix, the Journal reported that Army Corps of Engineers’ Tiger Team said the failures are “not yet sufficiently understood to ensure” a fix that would prevent these issues in the future.
Read The Wall Street Journal’s full report about the documents regarding the electrical failures.
(H/T: PC World)