Measuring the Unmeasurable

Key Scientific Challenges, Entry #3: Microeconomics and Social Systems

On January 27 we announced the kick-off of our 2010 Key Scientific Challenges Program.  To highlight the scientific challenge areas included in the program, we launched a series of guest blog posts earlier this month on Yodel Anecdotal. Read our previous post on privacy and security, “Data, Data Everywhere, but How to Keep it Safe.”

Another big challenge are Yahoo!’s research scientists are continually examining is microeconomics and social systems. In this entry, Sharad Goel from Yahoo! Labs shares some thoughts on how Yahoo! is tackling the new opportunities for research into the social sciences that the Web is making possible and why it’s a fascinating field.

What do your friends really know about you? How much do they influence your decisions? How often do we stray from the cultural herd? How do groups organize to solve complex problems?

Answers to such fundamental questions about social behavior have often eluded us. With microscopes we peered into the intangibly small building blocks of life, and with telescopes we found our place in an unimaginably expansive universe. But without the tools to faithfully document human activity—a challenge that by comparison seems so palpable—we had no way to investigate the inner workings of our own communities. Now with an explosion of information on every aspect of our everyday existence—from what we buy, to where we travel, to whom we know—we can measure what until quite recently was thought unmeasurable. In the Microeconomics and Social Systems Group at Yahoo! Labs, we are using this proliferation of data to explore how societies function. It’s a fascinating area of study that is just beginning to shed light on new layers of human behavior, making it a perfect fit for the Key Scientific Challenges Program.

In a recent study that’s garnered some attention, for example, we asked, “How eccentric are people?” Looking at consumer preferences across movies, music, and web browsing, we came to the surprising conclusion that ordinary people have pretty extraordinary tastes. In particular, we found that typical Netflix and Yahoo! Music users regularly watch movies and listen to songs that are not even available in the largest brick-and-mortar retailers. This result not only challenges stereotypes about people blindly following the herd, but also highlights the importance of offering consumers broad selection. That is, specialty products may dramatically boost user satisfaction by providing buyers the convenience of “one-stop shopping” for both their mainstream and niche interests.

In other work, we used web search queries to forecast the commercial success of movies, songs, and video games. Weeks, sometime even months, before a movie opens or a video game is released, one can find traces of pent up consumer demand in the search query logs. We found that these telltale signs of early interest are remarkably good predictors of future success. The catch? Although the search logs do reflect user intent, more mundane indicators, such as production budgets and reviewer ratings, perform equally well at forecasting sales. Thus, the benefit of web search as a prediction tool may have less to do with its superiority over other methods than with its generality, low cost, and real-time nature.

At a time when we are drowning in data, at Yahoo! Labs we’re asking a simple question: what can you do with it? The answer is limited only by our imaginations.

So long, bunnies ears. Hello, digital clarity. If you have an analog television, today is D-Day (in the U.S. at least) — the long-fabled digital switchover is here. If you haven’t gotten your converter box yet, your TV is SOL — and static-y ghosts have gone the way of Pong, Betamax, and the 8-track. Here’s what we digitized this week:

• Rock on, iPhone: There’s just something about radio — that element of surprise, that anticipation of what’s coming next, that man-I-haven’t-heard-that-song-in-years sensation. That’s the promise of our new Yahoo! Music iPhone app. Powered by CBS Radio, it lets you browse through 300+ stations within more than 20 genres (from Bollywood to Goth to Naughty Comedy), skip up to six tracks an hour, find local stations via GPS, share stations with friends, and buy music you like via iTunes. Never get stuck listening to Muzak again. Read more here, then download the app.
• Cirrus, cumulus, Flickr: The fact that Flickr has democratized photography and turned many hacks into professionals is not lost on Getty Images, the legendary stock photo agency. So back in March, we debuted the Flickr Collection on gettyimages.com, allowing mere mortals to license their work for commercial use. Today, Getty has introduced “photo clouds,” a new way to navigate groups of related photos within the Collection. You can join the community and create your own cloud around topics like “winter,” “out of Africa,” “bizarre photos,” and “fishy.” (Note the spiffy massage music that soothes as you explore). But wait! There’s also a screensaver, desktop widget, and a new Facebook app that lets you create a photo cloud to reflect your status mood. Brings new meaning to cloud computing. More here.
• Go commando with Colbert: If you’re addicted to Comedy Central’s Colbert Report, you probably know Stephen Colbert has been broadcasting from Iraq this week in support of troops stationed there. But he’s also leading an effort to make sure the children of said troops are getting what they need from a public school education. He’s teamed up with DonorsChoose.org to raise funds for military-serving schools, pitting the Army, Navy, Air Force, Marines and Coast Guard against each other. You can help by making a donation or spread the word by sporting a Colbert Avatar, with critical accessories like an eagle, bear, Colbert Nation hat, Che Colbert T-shirt, fireplace, and a desk. More here.

Subscribe to the RSS feed (or add it to My Yahoo!) to get this Product Pulse every week.

Nicki Dugan
Blog Editor

Nine years ago today, the NASDAQ composite peaked at 5132.52, marking the beginning of the end of the dot.com boom. In the year that followed, we bade farewell to the Pets.com sock puppet, those wonderful Webvan groceries, buying a pack of gum via Kozmo.com, or getting fashion advice from the Boo.com avatar. Here’s what persevered this week:

• Musical revival: Blowing its doors off their hinges, Yahoo! Music has fully embraced the open web with launch of its new Artist Pages. You’ll find downloads from iTunes; albums from Amazon; streams from Pandora, Last.fm and Rhapsody; music videos from YouTube and Yahoo!; photos from Flickr; and tickets from Ticketmaster – for more than 500,000 artists. And you call the shots on whatever modules you want to see. Eventually artists will be able to publish directly into their pages and developers will beef up our modules gallery. Music to our ears. More here.
• Instant iGratification: No need to get all twitchy when you leave your desk — Yahoo! Messenger is now available for the iPhone and iPod Touch. Released into the wild this week, the free Yahoo! Messenger app can now be found in the iTunes App Store. It’s got a lot of the same bells and whistles — photo sharing, stealth mode, status messages, archiving, emoticons. More here. What?! You don’t have an iPhone? Lucky for you Yahoo! Messenger works with other phones, too — check the roster here.

Subscribe to the RSS feed (or add it to My Yahoo!) to get this Product Pulse every week.

You might notice some really familiar names on Yahoo! Music starting today. No, not classic musicians. Names like YouTube, Pandora, Last.fm, Rhapsody, Amazon, Ticketmaster, and iTunes. That’s because we’re turning Yahoo! inside out and opening it up to what we think are the best music services online.

We’ve been evolving our approach to music for the last year or so. We found that our subscription music offering required a huge amount of resources for a relatively small user base, leading to our partnership with Rhapsody. And new royalty rates made us to rethink our streaming radio service, which is now run by CBS. Through all of these tough decisions as the industry evolved, the biggest constant has been our core music offering, which connects music fans to their favorite musicians. We said, “Why compete with everyone, fighting on all fronts, if we can just move up a layer and be a resource to our users?”

So now we can be all things to all music people with a little help from our industry friends. The new Yahoo! Music Artist Pages now bring together downloads from iTunes, albums from Amazon; streams from Pandora, Last.fm and Rhapsody; music videos from YouTube; and tickets from Ticketmaster – for more than 500,000 artists. We also include photos from Flickr as well as our own Yahoo! music videos. You can customize your Artist Pages to include your favorite content modules. Previously a closed service with proprietary licensed content, this is the first major effort by any online music site to truly open itself up to third-party services.

And we’ll soon open up our site to developers so that any music service can build an application to host in our gallery. And eventually artists will be able to directly publish content to our site – so fans can keep tabs on their latest blog posts, photos, music videos, live recordings, etc.

The ultimate goal is help you discover, experience, and consume your favorite music online as easily as possible. So go out there and enjoy the music.

Michael Spiegelman
Head of Yahoo! Music

Seventy-four years ago today, the phrase “Do Not Pass Go” came into existence as Parker Brothers sold its first box of Monopoly. Since then, 750 million people have done jail time as they tried to buy up (the misspelled) Marvin Gardens, Park Place, and Boardwalk. Here’s what we added to our holdings this week:

• Calendar gets cooler: Back in October, we brought you a Yahoo! Calendar revival. This week, it got a little louder. You can now search your calendar events and notepad by keyword, as well as print your calendar in a special print format for day/week/month/event for better readability. If you’re a list-maker, you can now create to-do lists by topic (i.e., “kids,” “work,” “groceries”) and share them with others. Not a to-do fan? Just drag the column shut to banish it. And finally, new right-click functionality lets you make quicker work of adding, editing, printing, and viewing events. Head to the suggestion board if you want to help steer future enhancements. Haven’t switched from Classic yet? What’s the holdup?
• Finer FoxyTunes: The popular music toolbar we acquired a year ago this week just got a refresh. The FoxyTunes team unleashed no fewer than 20 enhancements in their latest Firefox plugin release. They include a new switch menu that lets you multitasking music listeners jump between media players (iTunes, Pandora, Rhapsody, etc.) along with a bevy of smarter, simpler improvements for menus, buttons, and an overall streamlined design. More here.

Subscribe to the RSS feed (or add it to My Yahoo!) to get this Product Pulse every week.

After two years of exclusive Nissan Live Sets on Yahoo! Music performances with A-list acts from the Foo Fighters to Kelly Clarkson to Buddy Guy, this October — for the first time — Yahoo! Music took this show of ours on the road to meet up with a little band called Coldplay.

Three thousand miles from the Los Angeles stage that was home for nearly 50 prior shows, we renewed our appreciation for roadies by traveling to New York City, taking over Hammerstein Ballroom, and playing host to one of the biggest bands in the world. Now, you can take a Nissan Live Sets show out of LA, but rest assured, we added all the usual ingredients our viewers love to this historic venue: 300 of the band’s biggest fans (including Jay-Z and Beyonce), a candid audience Q&A, and a rockin’ playlist of fan favorites and brand new tracks.

The result was not only one of the most electrifying, contagiously head-bobbing concert events we’ve ever produced, but also a major milestone for the program. For the first time in Nissan Live Sets history, Coldplay’s global super-stardom gave us the opportunity to launch this exclusive, once-in-a-lifetime performance on Yahoo! Music in 12 different countries –- simultaneously.

Even though only 300 lucky fans got to catch this in person, pretty much the whole world can view the entire performance today on Yahoo! Music. If you need a teaser, check out a couple of the highlights below.

Enjoy –- and viva la Coldplay!!!!