You Want Innovation? Offer a Prize
31-Jan-07
Via Marginal Revolution
http://www.nytimes.com/2007/01/31/business/31leonhardt.html?ex=1327899600&en=4aca8bec1f9a18d3&ei=5090&partner=rssuserland&emc=rss
January 31, 2007
Economix
You Want Innovation? Offer a Prize
By DAVID LEONHARDT
Besides the fact that both are considered great movies, “The Wizard of Oz” and “Silence of the Lambs” don’t have much in common. One is the story of a girl from Kansas who’s transported to a magical land where animals dance and sing, and the other is about a serial killer who eats his victims. You wouldn’t necessarily expect people to have similar reactions to the two movies.
But it turns out that, for whatever reason, they usually do. Those who love one tend to love the other, and those who think one is overrated generally think the other one is, too.
This odd little fact comes from an enormous database of movie ratings collected by Netflix, the online movie rental store. On its Web site, customers can give any movie 1 to 5 stars, and the company then uses these ratings — 1.6 billion of them — to find connections like the one between “Oz” and “Silence of the Lambs.”
The system, called Cinematch, allows Netflix’s Web site to bombard users with recommendations of movies they are likely to enjoy. Netflix executives hope Cinematch will give them a leg up as digital downloading allows dozens of other companies to sell movies over the Internet.
So on the Ides of March last year, Reed Hastings, the company’s chief executive, and three other executives were meeting at their Silicon Valley headquarters to talk about making the system better. They had just finished discussing one failed effort — a promising algorithm designed by a hotshot computer scientist from Stanford (since lured to Google) — when Mr. Hastings threw out an idea.
“We should run a prize,” he said, an open competition challenging people to come up with a better version of Cinematch.
One of the other executives asked how much the company should offer, recalled James Bennett, the vice president who oversees Cinematch.
“A million dollars,” Mr. Hastings said.
With that, Netflix unwittingly started down the path of proving that today’s economy doesn’t have nearly enough prizes.
BACK in the 1700s, prizes were a fairly common way to reward innovation. Most famously, the British Parliament offered the £20,000 longitude prize to anyone who figured out how to pinpoint location on the open sea. Dava Sobel’s best-selling 1995 book “Longitude” told the story of the competition that ensued, and Mr. Hastings mentioned the longitude prize as a model at that meeting back in March.
Eventually, though, prizes began to be replaced by grants that awarded money upfront. Some of this was for good reason. As science became more advanced, scientists often needed to buy expensive equipment and hire a staff before having any chance of making a discovery.
But grants also became popular for a less worthy reason: they made life easier for the government bureaucrats who oversaw them and for the scientists who received them. Robin Hanson, an economist at George Mason University who has studied the history of prizes, points out that they create a lot of uncertainty — about who will receive money and when a government will have to pay it. Grants, on the other hand, allow a patron (and the scientists advising that patron) to choose who gets the money. “Bureaucracies like a steady flow of money, not uncertainty,” said Mr. Hanson, who worked as a physicist at NASA before becoming an economist. “But prizes are often more effective if what you want is scientific progress.”
In fact, when Netflix announced its prize in October, Mr. Hastings said he didn’t necessarily expect contestants to make a lot of quick progress. Computer scientists say that Cinematch, along with Amazon’s recommendation system, was already one of the most sophisticated. “We thought we built the best darn thing ever,” Mr. Hastings said.
But Mr. Hastings underestimated the power of an open competition. Within days, many of the top people in a field known as machine learning were downloading the 100 million movie ratings Netflix had made public. The experts have since been locked in a Darwinian competition to build a better Cinematch, with the latest results posted on a leader board at Netflix’s Web site.
Last week, I called Geoffrey Hinton, a professor of computer science at the University of Toronto whose team had been in first place when I last checked. But by the time I reached him, his team had been bumped down to second by a Hungarian team.
(The contestants have also turned up some good trivia about movie preferences. Benji Smith of Salt Lake City deserves credit for the “Oz”-“Lambs” connection.)
To claim the million-dollar prize, a team has to build a system that is at least 10 percent better than Cinematch at predicting how many stars someone would give a movie. There are a small number of people, for instance, who love “The Wizard of Oz” but can’t stomach “Silence of the Lambs.” Perhaps it is possible to identify them based on their attitude toward an eclectic group of other movies — but only an advanced algorithm can find this pattern.
With four and a half years to go in the contest, the Hungarians’ model is already 6.75 percent better than Cinematch. And Netflix hasn’t had to pay for their time. In effect, the company “has recruited a large fraction of the machine learning community for almost no money,” as Mr. Hinton, the Toronto professor, put it.
These are the two essential advantages of prizes. They pay for nothing but performance, and they ensure that anyone with a good idea — not just the usual experts — can take a crack at a tough problem. Much to the horror of the leading astronomers of the day, a clockmaker ultimately claimed the longitude prize.
Grants are still crucial. (Someone has to be paying those computer scientists while they’re trying to win the Netflix prize.) But it seems pretty clear that our research system doesn’t pay for results often enough.
Just look at how both political parties have so far tried to deal with global warming. They have handed out grants and subsidies for various alternative energy sources like ethanol, even though nobody knows what the best sources will ultimately be. A much smarter approach would be to mandate that the economy use less carbon. This would effectively set up a multibillion-dollar prize — in the form of new customers — for whichever companies came up with efficient energy sources.
The good news is that the Netflix prize is one of a handful of recent high-profile prizes. One of the others is the X Prize, which was created in 1996 as a $10 million purse for the first private manned flight to the cusp of space. It was awarded just eight years later, showing once again that prize money has a way of focusing the mind.
