[RTC List] About 165, 000 Web sites knocked offline by NaviSite outage

[Lynn Corbaley]

About 165,000 Web sites knocked offline by NaviSite outage
Linda Rosencrance



November 06, 2007 (Computerworld) Approximately 165,000 Web sites have been
offline since Saturday, thanks to a failed data center migration involving
Andover, Mass.-based Web hosting company NaviSite Inc.

The problems started Saturday when NaviSite attempted to migrate and replace
hundreds of servers operated by Baltimore-based Alabanza Corp., a Web
hosting company acquired by NaviSite in August.

According to NaviSite spokesman Rathin Sinha, NaviSite decided to physically
move 200 of the 850 servers operated by Alabanza to NaviSite's data center
in Andover and then virtually migrate the data from the rest of the older
servers to new boxes, also in Andover.

NaviSite let its customers know that their sites would be down for a while
on Saturday, with the migration expected to be finished that day, Sinha
said. But when NaviSite attempted to transfer the data from the 650 servers
still in Baltimore it ran into a number of synchronization failures that
kept multiplying.

As Saturday progressed, NaviSite realized it would probably miss its
completion deadline; as a result, company officials decided to physically
transfer another 200 servers from Baltimore to Andover to help reduce the
scope of the virtual migration and speed up the data transfer.

But then NaviSite ran into more problems. As the hosts came up, their URLs
did not, so although customers could access their Web sites from their IP
address, they could not do so using their URLs, Sinha said.

"That was unanticipated," he said.

As NaviSite tried to solve that problem, the network became overloaded
because of all the customers trying to get online, Sinha said. "What
happened was first the URL could not match with the IP address and then IP
did not match with the machine, so it took some time, and all this time we
have a highly trafficked overloaded network," he said. "If there is one
little problem, they multiply because there is a lot of dependencies."

Although Sinha said a "big chunk" of sites are back online, he could not say
when everything might be back to normal. He also couldn't say how much this
failed migration would cost -- NaviSite is a publicly-traded company.

To put it mildly, one of NaviSite's customers, Cynthia Brumfield, president
of Emerging Media Dynamics Inc., an analyst firm in Washington, seems to be
furious.

In an interview, Brumfield said she's going into her fourth day without
access to her Web sites. And she said she doesn't believe the way NaviSite
is spinning the story. While NaviSite said it has brought a large number of
Web sites back online, she claims it hasn't.

"According to people who have talked to NaviSite's tech personnel, they were
ill equipped for the relocation and ignorant of how to accomplish even basic
tasks," she said in a blog post. "At this point, NaviSite's poorly planned
data center consolidation has slipped from mere incompetence to outrageous
indifference to its customers' needs and should be grounds for legal action,
if not government sanctions of some kind."

Brumfield said that, in effect, NaviSite yanked the servers for 200,000 Web
sites, put them on trucks and then didn't know what to do once the servers
arrived in Andover. "But what's worse, NaviSite had informed its clients of
a completely different timetable and process for the server relocation than
the one implemented," she said in the blog posting.

In the interview, Brumfield said that because all of her backup files are
also stored on Alabanza's servers, she has no choice but to hop on a plane
to Boston on Wednesday and drive to Andover to retrieve her data. And she
said she's bringing a video camera with her to document NaviSite's response
to her request.

-- 
Lynn
spazyone.blogspot.com
"I have found the paradox, that if you love until it hurts, there can be no
more hurt, only more love." Mother Theresa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://redwoodtech.org/pipermail/list_redwoodtech.org/attachments/20071107/d73deeb5/attachment.html