Thursday, May 25, 2006

He Who Graphs Last, Graphs Best

We're in the middle of a media explosion. More network shows. More cable channels airing new content. More items are professionally produced for limited release. And definitely more amateur content.

How will you ever find anything in the long tail of media if few others sample the content before you? Maybe you're interested in content that was found organically (via search) and hasn't been tagged according to some standard convention. Microsoft Music Maps may lead us down the pathway of automated content classification.

The USPTO just published Microsoft's latest endeavor (US Patent Application 20060112098) in this area: Client-based generation of music playlists via clustering of music similarity vectors.

Why music? Look at the pathway for content delivery. It all started with text, migrated to music, moved to short video clips, then TV eps and finally movies. What was the main driver? File size versus available bandwith.

This same content sequence (text -> music -> TV eps -> films) also represents an increase in complexity. We have no prayer of automatically classifying short video clips (with audio) if we can't even classify short audio clips (music singles).

And guess what. Algorithms still can't reliably detect music genres. If you're interested in the technical details, check out this summary or this one from Sony.

Fair enough. But what about classifying TV shows?

Imagine using the Music Mapper to analyze the soundtrack for a TV show — opening credits, closing credits, background music, laugh track. Next, use voice recognition to convert the dialogue to text and use text analysis (used to classify novellas and short stories) to specify the genre and subgenre while simultaneously using the Music Mapper results.

No comments: