Cross-origin video

Back in December when we were working on Popcorn Maker v0.1, we were trying to integrate a template by Brian Chirls that implemented chroma-keying. Chroma keying is a process where you film a video with a pure color (usually blue or green) backdrop, and then in post production, replace it with other content. This is used in movies to film dangerous or impossible scenes, and in the weather forecasts to display the weather behind the newscaster. Brian’s template allowed people to do the same live, within their web browser. Then we tried the template with videos other than the default, and found it wouldn’t actually work. This is due to the browser same-origin security restriction, and the lack of cross-origin resource sharing (CORS) support on the <video> element.

The browser same-origin security restriction is a fundamental part of protecting users from malicious users. What it does is make cross-origin data write-only; it can be written to the display, but scripts cannot read data from it. There are only two exceptions to this rule:

  • Loading JavaScript
  • CORS

CORS is a protocol to selectively enable the reading of cross-origin data on an HTML element. It is a two-part client-server handshake. First, the client adds a crossorigin=”…” attribute to the HTML element. There are 3 valid crossorigin states:

  • none, which is the default mode
  • anonymous, which does not send cookies data to the server
  • use-credentials, which does send cookies to the server

When the browser loads the resource, it sends an additional header (Origin) to the server, which contains the host portion of the HTML page. On the server side, it needs to send back an additional header called “Access-Control-Allow-Origin”, which contains a comma-separated list of hosts that can read cross-origin data, or a * which signifies that anyone can read cross-origin data.

Browser support for the cross-origin specification varies. They all support it on the XMLHttpRequest object. But support on fonts (for @font-face rules), images, videos, and audio is very limited. In fact, for video and audio, it doesn’t exist. But Firefox is open-source software. We can fix it!

Bug 682299

This bug was filed back in August 2011 by Benoit Jacob, Mozilla’s WebGL expert. Cross-origin video textures were actually supported in WebGL in Firefox 4. However, a security exploit (https://bugzilla.mozilla.org/show_bug.cgi?id=656277) that performed a timing attack based on the brightness value of each pixel meant that all cross-domain WebGL textures must be disabled.

My first patch for this bug cargo-culted heavily off the Image elements CORS support (thanks Joe!). The key parts were:

  • Adding CORS-related attributes and enumerations to the nsHTMLMediaElement
  • Adding the crossorigin attribute to the nsHTMLMediaElement IDL
  • Changing nsHTMLMediaElement to use nsCORSListenerProxy if crossorigin is set
  • Changing nsLayoutUtils::SurfaceFromElement() to look at the nsHTMLMediaElement CORS mode when grabbing a frame from the Video element.

The trickiest part here was figuring out that I needed to add code in SurfaceFromElement() to get this all working. I started by coming up with a failing test case, where I would drawImage(videoElement) into a canvas, then try to getImageData() from it. This causes a NS_ERROR_DOM_SECURITY_ERR to appear in the web console. I searched MXR for the error, and worked my way backwards through the Canvas2DContext code until I found that it was SurfaceFromElement() that set the origin and CORS mode of a frame.

It was a working patch, but it needed clean-up before landing in mozilla-central. In the end, I had to unify the CORS-related attributes and enumerations in nsGenericHTMLElement, remove old preferences, fix my lack of specification-compliance, fix some logical errors in my code, fix existing tests, and add new ones.

It was landed on January 25th so this feature will be shipping in Firefox 12. But we’re still not done; after all, we need the server to be CORS-compatible as well.

Bug 703566

This bug here was to enable CORS on Mozilla’s video CDN. Once the feature landed in mozilla-central, I asked the folks in #it where it should go, and they got their part done in 2 days.

So Firefox has CORS support for pretty much every resource. Now all we need to do is convice the rest of the web to enable CORS where it makes sense.

This entry was posted in Open Source. Bookmark the permalink.