Solving HTML5 Video Problems with Adaptive Streaming

March 1, 2012 in Tech

Solving HTML5 Video Problems with Adaptive StreamingIn my last article, I discussed the confusing landscape of options in online video delivery. Now, I want to outline an online video solution that has the greatest payoff for your investment of time and resources. Every client approaches me with a unique set of objectives for their video content; some clients want to offer video services to their customers within their own web portals, while others simply want to “own” their media assets and not utilize free video hosting services such as YouTube and Vimeo. There’s no one single right way to deploy video content. Before I commit my clients to a video solution, I make sure we’ve answered the following questions:

  • Purpose: Why are you offering video online? Do you have product demos or marketing content that everyone who visits your online presence can view? Or, do you have training videos, or exclusive content that only specific customers can view?
  • Target audience: What devices and platforms are watching the video content? Have you identified a particular audience that needs specific optimized video delivery (e.g. Apple’s HTTP Live Streaming requirements for iOS applications)?
  • Production: Is your video content produced with high visual and audio quality? How frequently do you need to deploy new video content to an online audience?
  • Resources: What is your budget for online video, from production to post-production to deployment? Do you have managed resources (e.g. internal staff or contracted services) that can handle ongoing encoding and hosting tasks?

The answers to all of these questions determine how your video content will be produced, encoded, and distributed. For my approach outlined here to be worthwhile and meaningful, here’s a sample response that could benefit from an optimized online video deployment:

  • Purpose: You have video content that you want to keep within your control and domain. You’d prefer not to have videos in their entirety cached on the viewer’s device or computer.
  • Target audience: You have content that you want to optimally display on iOS devices, and play well across desktop browsers new and old.
  • Production: You have high quality video content, and you create it on a semi-regular basis (once a month, a week, or daily)
  • Resources: You don’t mind paying for premium video services, but you also don’t want a high upfront cost for encoding and hosting.

Note: In a future article, I will position services such as YouTube and Vimeo within a larger comprehensive online video strategy.

Goal: Reduce encoding efforts

As I mentioned in my earlier post, there is no single HTML5 video codec. Over the last decade, most encoding tools have honed their H.264 encoding options and their overall output quality. WebM and Theora encoding tools and options though are still quite limited.  If you stick with H.264, you’ll encode your video content with H.264 Baseline profile for smartphone delivery, and H.264 Main profile for tablet and desktop delivery. Within the next three years, I suspect we won’t need to worry as much as about H.264 Baseline profile as smartphones will universally have the GPU and/or CPU power to handle more advanced video decoding. Apple iPhone 4 and 4S already support H.264 Main profile playback. If you’re really savvy with your deployment, you’ll encode multiple bitrates in both H.264 Baseline and Main profiles for adaptive streaming. On, our automated presets output anywhere from 4 to 16 targets for adaptive streaming.

Goal: Reduce player logic/overhead

The standard mantra is to use the HTML5 <video> tag and only use a Flash-based video player as a fallback where the <video> tag isn’t recognized. I recommend to my clients the exact opposite approach: position a Flash-based video experience first, and fallback to HTML5 <video> support if the viewer isn’t on a platform that supports the Flash Player. With this approach, you can always use H.264 as your single video codec, because Flash Player supports it, and non-Flash-compatible platforms such as Apple iOS do support H.264 natively.

You can still create HTML5 and SEO friendly web content–you’re just going to use JavaScript to replace a <video> tag with Flash <object>/<embed> tags to play the video wherever the Flash Player is installed. This approach is not the same as putting <object> and <embed> tags within a <video> tag. On the contrary, JavaScript is used to swap out the <video> tag even within HTML5 capable browsers, to ensure consistency of H.264 playback. If you only have H.264 content, a non-H.264 compliant HTML5 browser such as Mozilla Firefox will not fallback to a nested <object>/<embed> Flash video player without the aid of JavaScript. MediaElement.js is one such library that will automatically insert an alternate playback engine if you only have H.264 content available. However, MediaElement.js doesn’t detect or implement adaptive streaming capabilities for both iOS and Flash Player. Therefore, if you want to use adaptive streaming techniques, you have to build your own JavaScript library like I’ve done on

Note: If you’re wondering why I’m positioning the Flash Player as the first tier of playback over HTML5, just look at the lack of true fullscreen playback in HTML5 browsers. Firefox has only recently released its fullscreen API for their latest beta releases of Firefox 10. There are a slew of other features that we don’t have in current HTML5 specifications that the Flash Player has had for quite a while, but I’ll leave that discussion to another article.

Goal: Optimize delivery for mobile and desktop

Finally, encode and deploy using adaptive streaming technologies. While adaptive streaming was once a domain for broadcast and movie distribution companies, today any video content creator can deploy adaptive streaming video with even the most meager of budgets. I’m a huge fan of Wowza Media Server (WMS), which reduces much of the complexity and overhead with streaming management. You can use WMS on the cloud with Amazon EC2 and Amazon S3 storage for a relatively minor cost (or use a AWS-backed  service like to even further minimize hosting costs). While you can use a standard web (HTTP) server with Apple HTTP Live Streaming (HLS) deployment , you need to pre-process all of your video into manifests and timed MPEG TS chunks. Once you’ve committed to that pre-processing workflow, you have to also encode non-HLS H.264 video files available for playback environments that don’t support HLS, regardless of whether you use Flash as a first responder or a fallback technology.

A Comprehensive Solution with AVC/H.264 and Adaptive Streaming

If you follow my recommendations, you’ll end up with a final scenario that includes the following tech setup:

  1. H.264 Baseline and Main profile encoded video content in two or more bitrates. That’s a minimum of four H.264 video files.
  2. A <div> container with an optional <video> tag for SEO purposes that is replaced with the appropriate Flash-based video player for adaptive streaming, or updated with an adequate source for the <video> tag that uses Apple HLS or progressive download.
  3. A JavaScript library that detects if you’re using a browser that’s HLS compliant, and use an appropriate M3U8 manifest file for H.264 Baseline playback on smartphones or a M3U8 manifest file for H.264 Main playback on tablet or desktop computers. If Flash Player is installed (or available for the platform viewing the video), the JS library will write Flash Player embed code for a video player that supports adaptive streaming such Adobe’s free Strobe Media Playback SWF.
  4. A Wowza Media Server instance that handles the transmuxing of your H.264 encoded content for Apple HLS and Adobe HTTP Dynamic Streaming (HDS), as well as the creation of M3U8 manifests for Apple HLS and F4M manifests for Adobe HDS.

To see past examples of this approach, you can review the HTML and JavaScript I’ve employed on the samples page.

The world of online video can certainly be intimidating, especially as non-Flash components enter your workflow and distribution. And there are real costs and trade-offs that flow directly from the decisions you and your clients make. I hope this post, and the one before it, help you decipher your options, and devise solutions that work for your needs.