Using Social Networks for Testing

Facebook's engineering boss reveals how the social network uses you to test new features

By Ellis Hamburger on August 8, 2012 12:47 pm

Facebook's Director of Engineering Andrew "Boz" Bosworth wasn't trained to "ship early and ship often" — the social network's motto. Bosworth went to Microsoft for a year and a half after college, where he was trained to ship slowly and less frequently. "I thought the right way was to slow down and take our time pushing it out," Bosworth says. All that changed when Mark Zuckerberg recruited him. "Mark deserves a lot of credit for how quickly we still move this company," says Bosworth. After six years on the job, he just published a lengthy blog post outlining the methods Facebook uses to test and push out features at such breakneck speed.

The key trick Facebook uses to gauge responses from users is testing features and tweaks on groups of users and seeing how they react. Bosworth cites the ability to save News Feed stories to read later as one of the recent tests that only some users got to experience. The company seems to always be testing new features, like Highlight, which let regular users promote their stories in friends' News Feeds, or streamlined Timeline profile pages. Many features graduate to worldwide rollout, but just as many get trashed or put on a shelf for the time being. "If you're not willing to test things and prove that they do or don't work, some things that were wrong a few years ago may not be wrong anymore," Bosworth says.

"SOME THINGS THAT WERE WRONG A FEW YEARS AGO MAY NOT BE WRONG ANYMORE."

He mentions Pulse, a feature from 2006 that let users see what books and movies were trending among friends (long before Open Graph debuted), as one of these things. Facebook realized that users would click on Pulse, then never return again, so they killed it. "Maybe Pulse will come back in the future," he says. "We don't keep a greater catalog of what has and hasn't worked." The sentiment echoes something Bosworth once wroteabout CEO Mark Zuckerberg: "Every time Zuck looks at a product, it is as if he does so with fresh eyes."

Before rolling out new features to select groups of users, Facebook uses its own employees as guinea pigs. Thanks to a tool called Gatekeeper, none of the tests ever overlap, inside or outside the company's Menlo Park headquarters. Even so, constantly rolling out new features from different teams seems like it could accumulate pretty quickly. Bosworth corrects me. "One of the biggest misconceptions about Facebook is that we're always adding things," he says. "It's a mistake to think of these things as strictly additive, because most of these changes are modifications. There's probably more removed then added out of this process," Bosworth says. And it's happening all the time. "The odds are good that everyone on Facebook has been, at some time, part of a test," he wrote in his blog post. Facebook often rolls out two slightly different versions of the same feature to gauge user reaction, then pick the better of the two.

While Facebook, like Google, makes a great many choices based on data from users, the company does occasionally take a leap of faith — most of which are guided directly by CEO Mark Zuckerberg, who has "very good instincts," Bosworth says. "News Feed was a thing where at first users weren't happy. Now we know that people really care a lot about it... We have to have a rapid development pace where we're always trying to deliver the best version of what exists today, but at same time think of what doesn't exist — the things people don't know how to ask for, and that will be useful a year from now," Bosworth says.

"DO PEOPLE SAY IT FEELS GOOD? IS THIS WHAT THEY EXPECT?"

Yet, sometimes new initiatives flop. Bosworth cites a change to the social network's chat window that decreased the number of chats initiated by nearly 10 percent. While Facebook thought that downsizing the "Online Buddies" window would help (since most people chat with only a few people), it ends up that users just love browsing to see who's online. Facebook hadn't tested the new chat window with random people before it launched, and thus had no idea how the hundreds of millions of monthly Facebook users at large would respond.

Facebook quickly learned from its mistakes, but will likely make mistakes again. The company is in a constant balancing act between being adventurous and data-driven, going with Zuck's gut or going with hard data. Bosworth says, "We're dealing with human emotions. Everything doesn't boil down to a number you can measure. Do people say it feels good? Is this what they expect? Sometimes, that isn't boiled down by data."

The Verge

Posted in: Facebook,redes sociales