introductory thoughts on a playlist generation task in MIREX 2009

I'm currently knee deep in ISMIR activities, of which I will do a more thorough write up when everything has finished. Now, however, I think it will be useful to briefly discuss a MIREX 2009 task I'll be pushing for, that of automatic playlist generation. I think it will be very useful to formalize as much of this task as possible as soon as possible, in effort to encourage participation and avoid the nearly standardized MIREX august rush.

With that in mind, here some the basic starting questions followed by my [brief and biased] thinking on them:

1. What is a playlist?
   
   

   From wikipedia:

   In its most general form, a playlist is simply a list of songs. The term has several specialized meanings in the realms of radio broadcasting and personal computers.
   The term originally came about in the early days of top 40 radio formats when stations would devise (and, eventually, publish) a limited list of songs to be played. ...
   As music storage and playback using personal computers became common, the term playlist was adopted by various media player software programs intended to organize and control music on a PC such as Nexus from NexTune. ...
   Some websites allow categorization, editing, and listening of playlists online, such as Project Playlist, Plurn, imeem and Webjay. ...
   

   
   This is a sensible starting place from a broad definitional stand, especially when we acknowledge that the exact MIREX will obviously be a subset within the idea of a playlist (more on that later...)
   




2. How do we quantitatively evaluate an automatically generated playlist?

   Upon a brief reflection this is clearly the sticky bit. As such this question is being posed prior to any sort attempt to specify an exact task. Functionally speaking, if the evaluation is properly specified the ask description will practically fall out of it.
   There seem to be a few approach to this, though many tend to revolve around an idea of mining some form of ground truth out of crowd wisdom (here and here for instance)




3. And of course, what exactly should the task be?

   Again, this is of course all very draftish, but my prime thought on this is to have a highly specific task of one (or all) of the following forms: query with song A B C algorithm generates song D (and perhaps song E, where order matters), the simple deviation of that form given A B E, generate C D. The other possibility is something alog the lines of give a query song and provide the next song that should occur in the list.

So anyway, those are my initial thoughts on this potential task for next years MIREX. Please comment if you have thoughts...