The Other McCain

"One should either write ruthlessly what one believes to be the truth, or else shut up." — Arthur Koestler

ORCA: Romney’s Killer Fail

Posted on | November 9, 2012 | 61 Comments

Back home this evening after a 12-day road trip to Ohio, I’m perusing accounts of how the Romney campaign recruited 37,00 volunteers to participate in its ORCA project to monitor Election Day turnout, a project that turned out to be a spectacular failure of epic proportions. The name that keeps coming up in these accounts is Dan Centinello, deputy political director for the Romney campaign:

Centinello was quoted in The Huffington Post on Nov. 1 touting ORCA to volunteers in these grandiose terms: “There’s nothing that the Obama data team, there’s nothing that the Obama campaign, there’s nothing that President Obama himself can do to even come close to what we are putting together here.”

That’s a boast suitable for framing, eh? Centinello is an experienced operative whose LinkedIn profile traces his career going back to his start with the Republican Party in San Diego in 2005, his subsequent work with the 2006 California gubernatorial campaign of Arnold Schwarzenegger, on to Romney’s unsuccessful 2008 presidential primary campaign, Chris Christie’s 2009 gubernatorial campaign in New Jersey, and Roy Blunt’s 2010 senatorial campaign in Missouri. He was hired as a deputy to Romney campaign political director Rich Beeson in April, not long after Romney essentially clinched the GOP nomination.

Nothing in Centinello’s resumé suggests that he is the kind of clueless bungler capable of devising a digital Rube Goldberg device like ORCA and yet his fingerprints are all over this laughable catastrophe. The scope of the failure is hard to exaggerate and John Ekdahl at Ace of Spades HQ explains that the failure was predictable:

From the very start there were warning signs. After signing up, you were invited to take part in nightly conference calls. The calls were more of the slick marketing speech type than helpful training sessions. There was a lot of “rah-rahs” and lofty talk about how this would change the ballgame. . . .
They assured us that the system had been relentlessly tested and would be a tremendous success.

When your “training sessions” seem more like an multi-tiered marketing sales pitch, this is not generally a good omen. In hindsight, the bad omens were there all along. In July, Centinello’s name cropped up in an account by Sascha Issenberg of Slate about “a new data-science team within Romney’s strategy department,” which was at that time evidently involved in trying to copy the Obama campaign’s strategy:

“We watch where the president goes,” says Dan Centinello, a Romney deputy political director who oversees the weekly meetings. “We’re trying to piece together what we think are his top ranks.” . . .
Romney aides scoff at the glowing pieces written about Obama’s data-driven methods, but their obsession with reverse-engineering his analytics is its own concession. It is the statistician’s version of trailing a motorcade in a honking bus to find out where the president is headed, in the hopes of later divining why. “It’s one thing to know ours,” says Centinello, “but it’s even better to know what his strategy is.”

This is the guy who, three months later, was personally answering e-mails from volunteers asking questions about the ORCA system, as Bethany Mandel of Commentary reports:

Shoshanna McCrimmon signed up to volunteer on Romney’s website several months ago. She was contacted by Dan Centinello of the Romney campaign and underwent online and phone training that lasted for several hours in order to volunteer locally on Election Day. . . .
Thousands of man-hours went into designing and implementing a program that was useful on one day and one day only, and on that day, it crashed. My source familiar with the campaign described it this way, “It was a giant [mess] because a political operative sold a broken product with no support or backup plan. Just another arrogant piece of the arrogant Romney campaign.”
The operative in question, Dan Centinello, Romney’s Deputy Political Director, was Shoshanna’s only point of contact with the campaign. After a two-and-a-half-hour conference call with volunteers across the country, Shoshanna still had questions about minor details about ORCA and volunteering at her polling place. Her emails were answered within 24 hours, always by Centinello. There appears to have been no delegation on Centinello’s part, and every question sent was answered by the ORCA project manager personally. It’s likely that if this was taking place with the thousands of volunteers in Project ORCA, Centinello was spending hundreds of hours answering basic questions from volunteers that could have been addressed by lower level staffers.

So the deputy campaign manager apparently didn’t have sufficient staff to handle the massive internal communications load necessary when dealing with an organization of 37,00 volunteers using customized software that had never been adequately field tested. Anyone could anticipate trouble in such a situation and yet, before Election Day, a Team Romney spokeswoman was bragging on TV about how 800 volunteers in Boston were going to run this awesome ORCA operation:

“The Obama campaign likes to brag about their ground operation, but it’s nothing compared to this.”
— Gail Gitcho, Romney campaign communications director

Yeah: Team Obama had “nothing compared to this” colossal disaster. Sean Gallagher at Ars Technica discusses the basic problems:

[A]ccording to volunteers who saw and used the system, it was hardly a model of stability, having been developed in just seven months on a lightning schedule following the Republican primary elections. . . .
To build Orca, the Romney campaign turned to Microsoft and an unnamed application consulting firm. . . .
Orca turned out to be toothless, thanks to a series of deployment blunders and network and system failures. While the system was stress-tested using automated testing tools, users received little or no advance training on the system. Crucially, there was no dry run to test how Orca would perform over the public Internet.
Part of the issue was Orca’s architecture. While 11 backend database servers had been provisioned for the system –probably running on virtual machines — the “mobile” piece of Orca was a Web application supported by a single Web server and a single application server. Rather than a set of servers in the cloud, “I believe all the servers were in Boston at the Garden or a data center nearby,” wrote Hans Dittuobo, a Romney volunteer at Boston Garden, to Ars by e-mail.
Throughout the day, the Orca Web page was repeatedly inaccessible. It remains unclear whether the issue was server load or a lack of available bandwidth, but the result was the same: Orca had not been tested under real-world conditions and repeatedly failed when it was needed the most.

From the perspective of volunteers in key swing states, ORCA was a Kafkaesque nightmare, as Joel Pollak writes at Breitbart.com:

At one point during Election Day, the system had malfunctioned so badly that desperate volunteers wondered if the program had been hacked.
Romney volunteers in Virginia confirmed that the campaign had relied entirely on Project Orca to turn out the vote in key areas such as Roanoke, where Romney and Ryan had made appearances. Volunteers who had driven to Virginia from safely-Republican Tennessee were shocked at the disorganization they encountered.
While the Romney campaign waited for Orca to function as planned, the Obama campaign had placed signs outside every one of the city’s thirty-three polling places, and was fully staffed with two volunteers outside each polling place, and a strike list volunteer inside, all day long from 6 a.m. to 7 p.m. The best that the short-handed Tennessee volunteers could manage was 40% coverage of polling places; the local GOP, they said, had relied entirely on the campaign’s centralized Orca system in Boston to turn out the local vote.

John Ekdahl notices that top staff on the Romney campaign are trying to claim that ORCA actually worked and otherwise covering their asses:

Please, if anyone at the RNC is listening, do not let these guys near a major political campaign ever again.

Accountability? What a concept!

UPDATE: An Ohio reader e-mails:

On Saturday and Sunday, I walked my precinct for the Romney Campaign and received a list of voters from which I was to survey and find out if they were planning to vote on Tuesday and if so when.
Knowing one of these voters was well into her 90s and I was pretty sure had voted absentee, I took the liberty of checking against the Hamilton County Board of Elections records of who voted early and had returned their ballot. I was able to check off one-third of the voters I would have otherwise contacted, 32 of 106, and avoid having any of them or their friends or family find out just how inept the Romney folk were for contacting someone whose ballot was mailed back in weeks earlier.
When I returned to the Victory Center and told the person in charge that I found one-third of them had already voted, I was not thanked for finding this out, praised for efficiency, or anything like that.
No, the 22-year-old bitched at me for not following the procedures set forth in the instructions by the “professionals.”

UPDATE II: Ben Howe’s sources with the Romney campaign lay the blame with the consultants running the campaign:

The result of all of these false numbers and inaccurate ground reports is simple: Mitt Romney had no idea what was coming on election day and his false sense of confidence directly translated into how the campaign operated in the closing weeks. In the words of one source, it was a con job.

UPDATE III: Lots of interesting input in the comments — our readership evidently includes plenty of IT geeks — and this from Adjoran:

It sounds like a violation of the First Rule of Management: “You cannot manage what you do not understand.”

There’s also the saying, “Personnel is policy.” If you get the right people in management roles, they’ll hire the right people to do the work and manage them well. On the other hand, if you get the wrong person in management, he’ll inevitably hire people who don’t have enough knowledge to criticize his mismanagement, so that there is no possibility that feedback within the organization to expose incompetence and failure. This is a general trend in organizations, not limited to campaigns and politics.

 

Category: Election 2012, Fear. Loathing., Mitt Romney, Republicans

Comments

61 Responses to “ORCA: Romney’s Killer Fail”

  1. Debby Berg
    November 17th, 2012 @ 6:48 pm

    Please read your post again and figure it out for yourself. Tea Party = crazy. Get it? Can’t get it any clearer for you.

  2. REBEKAH.MAJORSMANLEY
    November 17th, 2012 @ 11:50 pm

    YOU CANNOT MANAGE WHAT YOU DO NOT UNDERSTAND

  3. K-Bob
    November 19th, 2012 @ 9:45 pm

    Oh No0Oo0o, they wouldn’t stoop to using “Mike’s nature trick” or anything to try and “hide the decline.” It could never happen.

    Because numbers can’t be racist!

  4. K-Bob
    November 19th, 2012 @ 9:46 pm

    Aren’t you supposed to be over at some Paulbot site bashing Israel?

  5. K-Bob
    November 19th, 2012 @ 10:00 pm

    Translation: circular reasoning makes you smart!

  6. CatherineFitzpatrick
    December 9th, 2012 @ 1:09 pm

    Oh, nonsense. While it’s true that geeks can be nihilists and cynics, the reality is, Google, for business reasons, wanted Obama to win; so did a lot of Silicon Valley Big IT. California is a big state, whatever you want to babble about Meg Whitman and GOP CEOS of old computer firms. And they have a lot of geeks in their network who are happy to help them achieve their goals. Indeed they *do* develop political allegiance when they want to, although they might drop their love of Obama tomorrow if he doesn’t deliver start-up visas for entrepreneurial immigrants or whatever other issue they lobby on, and then they might switch to a libertarian. The reality is there is a mixture of technocommunist, old leftist “red diaper baby” type of Marxist politics in the web 2.0 world as much as there are libertarian Ron Paul or Gary Johnson kind of anarchy. It’s not just all or one.

  7. CatherineFitzpatrick
    December 9th, 2012 @ 1:13 pm

    Everything you’re saying is indeed true and indeed makes sense, and yes, Romney had those things against him — time and logistics. But he also thought that he could merely buy the best that the market offered, and that happened to be Obama’s old campaign digital directors and other Obama voters. And I maintain that it’s about lack of spirit and determination in the geeks that were put on this job. This is an inherent problem of geeks in general, but it’s fixable by methodically recruiting to get people who share the GOP values. They do exist.

  8. CatherineFitzpatrick
    December 9th, 2012 @ 1:19 pm

    Everything you say here is excellent and comes from grounded good sense and expertise in this field. And I hope you will be listened to by those who matter in the GOP leadership. One of the things that we saw failing from the Ace of Spades blog and others is over-centralization. I’ve heard volunteers in some states personally complain very bitterly that the Romney people seemed to want to centralize everything out of the Boston office instead of letting states’ HQs make decisions as they saw best for their own local conditions.

    One of the problems in general of having a national system even if it functions perfectly with local nodes is that it tends to work against state rights and override local conditions and homogenize them or weed them out. And so I’m glad you’ve conceived of an architecture that preserves states’ rights as an emulation of the Constitution and US law. Of course, no machine can do that perfectly, and I suspect that even if GOTV worked perfectly between center and states, the tendency would be for the center to homogenize the states and subordinate them.

    And one way this happens is not about the machine, but about the digital analytics director and marketing directors in the campaign who decide what the “stories” are. What “stories” will “sell”. For example, for Obama, there was the “war on women” story and the “voter suppression” story and these were marketed through the networks to ensure GOTV was avid.

    You also have to take into account now that Obama’s campaign is refusing to turn over the data drilled and the software developed to the DNC. And this is an inherent problem for our whole democratic election system. So I’m glad you’ve stated that from the outside, this has to be in the Party’s hands. The aggregate information should also be open to the public as well.

  9. CatherineFitzpatrick
    December 9th, 2012 @ 1:21 pm

    A chief feature of the Democratic Party as it stands now (where I remain registered but critical) is this scary belief in its own “scientism” and this arrogant notion — akin to eugenics — that the Republicans are “stupid”. Once you see people saying this sort of thing all the time about their fellow human beings, you start to realize where the problem lies.

  10. CatherineFitzpatrick
    December 9th, 2012 @ 1:22 pm

    Like I said: eugenics. Not a pretty ideology.

  11. CatherineFitzpatrick
    December 9th, 2012 @ 1:22 pm

    “I am for sale by the highest bidder, why didn’t they buy me?”