Annotating YouTube videos programmatically

I want to be able to display a normal YouTube video with overlaid annotations, consisting of coloured rectangles for each frame. The only requirement is that this should be done programmatically.

YouTube has annotations now, but require you to use their front end to create them by hand. I want to be able to generate them. What's the best way of doing this?

Some ideas:

> 1. Build your own Flash player (ew?) > 2. Somehow draw over the YouTube Flash player. Will this work? > 3. Reverse engineer & hijack YouTube's annotation system. Either messing with the local files or redirecting its attempt to download > the annotations. (using Greasemonkey? Firefox plugin?)

Idea that doesn't count:

> download the video

YouTube provides an ActionScript API.

Using this, you could load the videos into Flash using their API and then have your Flash app create the annotations on a layer above the video.

Or, alternatively, if you want to stay away from creating something in Flash, using YouTube's JavaScript API you could draw HTML DIVs over the YouTube player on your web page. Just remember when you embed the player to have WMODE="transparent" in the params list.

So using the example from YouTube:

  <script type="text/javascript">

    var params = { allowScriptAccess: "always" };
    var atts = { id: "myytplayer", wmode: "transparent" };
    swfobject.embedSWF("http://www.youtube.com/v/VIDEO_ID&enablejsapi=1&playerapiid=ytplayer", 
                       "ytapiplayer", "425", "356", "8", null, null, params, atts);

  </script>

And then you should be able to draw your annotations over the YouTube movie using CSS/DHTML.

Joe Berkovitz has written a sample application called ReviewTube which "Allows users to create time-based subtitles for any YouTube video, a la closed captioning. These captions become publicly accessible, and visitors to the site can browse the set of videos with captions. Think of it as a “subtitle graffiti wall” for YouTube!"

The app is the example used to demonstrate the MVCS framework/approach for building Flex applications.

http://www.joeberkovitz.com/blog/reviewtube/

Not sure if this will help with the colored rectangles and whatnot, but it's a decent place to start.

The player itself has a Javascript API that might be useful for syncing the video if you choose to make your own annotation-thingamajig.