Licensed listening based on the habits of pirates or lessons from sloppy item resolution

from flickr user nozoomii, CC by-nc-sa

So I spent Thursday and Friday a couple weeks ago at the Barcelona Music Hackday, part of the Sónar Music Festival. There were loads of excellent hacks (full list), including my own, Legalize It!. In this post I'm going to go into a bit more depth about the hack, lessons learned and teasers for things I might do.

Motivation

The core idea is a simple one – straightforward listening to things that are popular on Bittorrent (note that popular on Bittorrent is a slightly fuzzy concept, since Bittorrent is protocol for ad-hoc distribution, but we'll get back to that in a bit), without all the nastiness (and DNS blocking!) of looking at, say, The Pirate Bay's top music torrents (that's a proxy of tpb btw).  And of course this removes any legal trouble that would be associated with gather and listening to music via those torrent charts.

How it all works

tl;dr - It's a torrent chart metadata-based content resolver, written in python and JS, you can fork the code.

Legalize It! has two parts, client and server. The server is a fairly simple pyramid webserver with two main tasks (it's deployed on heroku). The first involves fetching the torrent charts and resolving torrent release groups to legal streaming albums (Spotify, currently).  This is simply a matter of fetching the daily torrent releasegroup chart from the Musicmetric API (full disclosure, they're my employer and I wrote most of the chart endpoints...) then walking through the top N items that look like albums and matching them to spotify albums. The matching is done through a fantastically naive string title + artist search on spotify's metadata API via the very useful Spotimeta python wrapper. This album resolution process has a simple web interface (if it returns an error try a refresh, heroku workers on the free tier sleep a bit too much), that I mostly built for testing, but can be quite useful without the commitment of installing the Spotify App. In addition to the human readable page, you can get the response back as JSON, which is handy on the client side.

The second job for the server is necessary to help select which songs from the top albums we'll be listening to. The final Spotify app will only select on song per album, so users can get a taste of every album in the top N without having to listen to N complete albums.  But this should be done with care and grace to enhance the listening experience.  Thankfully, with the assistance of The Echo Nest's searchable audio summary song-level features this is quite easy.  Using their API we can search for a song by title and artist name (see a pattern of terrible id matching forming? What can I say, it's a hack.) and get back a set of descriptors that looks like this:

Once we have the set of song-level descriptors for every song in the neighbouring albums, it's simply a matter of minimizing the step size.  I've done a bit of work on playlists before, and this seemed like a reasonable approach.  While there are a number of approaches to step-size minimization, for this particular application we're doing a greedy sort of optimization that goes something like:

1.  Select the song a in album A and the song b in album B, such that for a given audio descriptor (we'll use dancibility by default, but it could just as easily be a different measure, eg. tempo, loudness) the absolute value of the descriptor of a less the descriptor of b is minimized.  That is to say we're looking for the song-pair from these two albums that are closest in terms of whatever descriptor is being used.
2. Select song c from album C such that the absolute difference from it's audio descriptor to song b's  is similarly minimized.
3. Repeat (2) for remaining albums, using the last chosen song against the next album.

It's worth noting that this is not the globally optimal shortest path from the first album to the last album, but it comes with a tremendous advantage -- the first two songs are selected without any need to deal with the rest of the playlist, which can be worked on overtime.  This allows for pseudo real time playlist creation, since we just need to know the next song before the current one is done playing.

To facilitate this algorithm the server has a hook that performs step (1) or (2) on on either a pair of albums or a song and album (specified as spotify URIs) can be accessed via a url that looks like  

http://legalize-it.herokuapp.com/paired/[spotify URI of track or album]/[spotify URI of album]

For example, http://legalize-it.herokuapp.com/paired/spotify:album:6LBiuhK7PZKjVXyMfPxPoh/spotify:album:5nVUqrdkEMlWTm9sqjrYBt which returns a bit of JSON showing the closest (by dancability) song pair between My Beautiful Dark Twisted Fantasy by Kanye West and Up All Night by One Direction.  This method also supports using any other audio summary feature being used for distance by adding it to the end of the URI.  For example http://legalize-it.herokuapp.com/paired/spotify:album:6LBiuhK7PZKjVXyMfPxPoh/spotify:album:5nVUqrdkEMlWTm9sqjrYBt/tempo find the closest song pair from the same two records, but by tempo rather than by dancebility.

These two features are laced together in the client, which is a simple spotify app, that's basically just a small bit of jQuery that grabs the list of the top 25 albums then asynchronously gets the selected track for each album updating the playlist as each track is selected.

The Legalize It! app, as demo'd at the Music Hack Day

If you'd like to install the App, here are some instructions for installation in developer mode.  I'll be cleaning up the UI to conform to Spotify's app guidelines and submitting the app to their App finder thing, but that will take a while.

Going Forward

While this is mostly complete for what it does there are a number of feature adds that I'll be slowly dealing with as time goes on. Highest among these is taking the same idea and applying it to different resolvers (maybe use tomahawk rather than picking one...). There are also some smaller feature adds to the app, things like autoplay once the first track has loaded and switching between different echonest summary features or musicmetric charts (eg. P2P release groups by acceleration).

Also, thanks very much to Spotify, who awarded me a prize for my hack!

So what do you think?  Should we care about the taste of a bunch of peers on Bittorrent? Or am I doing it wrong?

Upcoming talks and travels

I'm going to be up to a number of things that may be of interest to the readers of this (rather sparse) blog.

Quick summary

• Co-chairing The Workshop on Music Recommendation and Discovery (WOMRAD) Oct 23rd, Chicago
• Presenting some new Musicmetric API features during ISMIR, Oct 24-28 (demo on the 28th), Miami
• Giving a talk about beer and critical tasting at Ignite London 5, Nov 8th, London

WOMRAD

Our workshop (other co-chairs are Amélie Anglade, Òscar Celma, Paul Lamere, and Brian McFee) will cover a diverse array of approaches and angles for music recommendation and discovery. The workshop is runs the full and is part of RecSys 2011, though I sadly can't stay for most of the conference aside from our workshop (see below). It should prove to be an interesting day of research. Are you planning on attending? Let us know.

ISMIR

I'll be dashing off from Chicago to Miami to attend ISMIR, and to present some new (not-yet-released) API features from Musicmetric. While things aren't quite live yet, I can say that in addition to our artist based endpoints, we'll be offering track-based endpoints soon as well, and aligning them with an I-bet-you-can-guess-which large public audio feature test set. More detail on this one to come.

Ignite London 5

Ignite is a series of lightning talks that have taken place in cities all over the world, unified not by a common theme, but a common format: all talks last five minutes, contain 20 slides, and the slides are automatically advanced every 15 seconds. The matching ethos of this structure is perhaps best seen in the Ignite slogan, "Enlighten us, but make it quick." I'll be speaking about beer, style and critical tasting in a talk titled "Ale or Lager and Other False Choices." Here's a brief description:

In a word, my talk is about beer. In a few more words, the driving narrative behind the talk is a crash course in beer styles and more generally, critical tasting. After an extraordinarily brief description of beer, broad ideas of style and the critical tasting process, the core of the talk will be made up of live lightning tastes of commercial examples of various styles of beer (one slide per style, 12 styles covered with one commercial example each). For coherence these tasting slides will grouped into broader styles, with aim toward width, rather than depth of coverage. The styles will be approximately based on those from the BJCP and the Brewer's Association.

I still haven't sorted out the exact spread of beers or how they will be grouped, though I'm leaning toward something simple and obviously ingredient tied (something like -- Lagers, ales:yeast driven, ales:malt forward, ales:hop heavy, with 3 beers each from a different recognized style in each). If anyone has any thoughts about style divisions or specific examples do let me know. If you'd like to go to ignite (and you know you would) the tickets will be available over this way later this week.

So, lots of things going on. Plus there's this other thing I've been working on.

Right, back to it.

Doing ridiculous things with natural language processing

Over this past weekend before last, while I jealously followed from afar the SF musichackday that I was unable to attend as I'm awaiting the result of a visa application, I started mucking about with the beta musicmetric api (full disclosure, they are my employer), in particular the sentiment analyzer.

So the first thing I put together was a bit of python to fetch the content of a tweet and use the mm_api to determine it's tone. This can be done quite simply (full source):

Which gives a number between 1 and 5, with 1 indicating the text is 'very negative' and 5 indicating the text is 'very positive' (gory details of the sentiment analyzer). While the sentiment analyzer is trained for larger chunks of text (500 word album/movie reviews and that sort of thing) it in fact does fairly well with tweets (though sarcasm is its downfall). So I thought I'd do something a bit silly and built a 'flamewar detector and troll finder' for conversations on twitter.

I've called the initial command line tool firealarm.

To gather the conversations, I'm just piggybacking on @jwheare's great tool exquisite tweets. Once a conversation is archived over on exquisite tweets, the cli can be pointed to it via the conversation's url. Each tweet in the conversation is pushed through the sentiment analyzer; the simple mean (µ) of all the sentiment scores is then dubiously used to determine if the conversation is a flamewar. If the sentiment is generally negative (µ < 3) it's a flamewar, if it's generally positive (µ>3) it's not a flamewar, and if it is exactly neutral (µ ==3) it's declared a tossup. The troll finder is equally straight forward (and equally dubious!). Across the sequence of tweets, the author of the tweet with the highest magnitude negative delta sentiment proceeding it is considered the troll. In the case of a tie the first occurance wins.

Here's an example (note that the linked-to example is a nasty, nasty flame war. If you offend easily, might want to skip it. Also, obviously, views expressed are not mine, etc.):

$ python firealarm.py http://www.exquisitetweets.com/collection/RodBegbie/402

This generates the following output:

A plot of the sentiments, with the maximum negative delta in red, looks like this:

A quick read of the tweets and you can see that the actual sentiment of the tweets is a bit more negative overall then the analyzer output, but this is good enough for binary classification and a fairly reasonable troll ID mechanism, albeit fairly naive.

The code is over at github if you want to have a look or run it yourself. If you want to run it, you'll need a musicmetric api key which takes about 30 seconds to get (apply here). Eventually I'm going to turn this into a web app, and when that happens I'll let everybody know. Also if you happen to find any really bad mislabels, let us know, as it helps us tune up our process.

Have fun being algorythmically judgemental!