Solving HTML5 Video Problems with Adaptive Streaming
March 1, 2012 in Tech
In my last article, I discussed the confusing landscape of options in online video delivery. Now, I want to outline an online video solution that has the greatest payoff for your investment of time and resources. Every client approaches me with a unique set of objectives for their video content; some clients want to offer video services to their customers within their own web portals, while others simply want to “own” their media assets and not utilize free video hosting services such as YouTube and Vimeo. There’s no one single right way to deploy video content. Before I commit my clients to a video solution, I make sure we’ve answered the following questions:
- Purpose: Why are you offering video online? Do you have product demos or marketing content that everyone who visits your online presence can view? Or, do you have training videos, or exclusive content that only specific customers can view?
- Target audience: What devices and platforms are watching the video content? Have you identified a particular audience that needs specific optimized video delivery (e.g. Apple’s HTTP Live Streaming requirements for iOS applications)?
- Production: Is your video content produced with high visual and audio quality? How frequently do you need to deploy new video content to an online audience?
- Resources: What is your budget for online video, from production to post-production to deployment? Do you have managed resources (e.g. internal staff or contracted services) that can handle ongoing encoding and hosting tasks?
The answers to all of these questions determine how your video content will be produced, encoded, and distributed. For my approach outlined here to be worthwhile and meaningful, here’s a sample response that could benefit from an optimized online video deployment:
- Purpose: You have video content that you want to keep within your control and domain. You’d prefer not to have videos in their entirety cached on the viewer’s device or computer.
- Target audience: You have content that you want to optimally display on iOS devices, and play well across desktop browsers new and old.
- Production: You have high quality video content, and you create it on a semi-regular basis (once a month, a week, or daily)
- Resources: You don’t mind paying for premium video services, but you also don’t want a high upfront cost for encoding and hosting.
Note: In a future article, I will position services such as YouTube and Vimeo within a larger comprehensive online video strategy.
Goal: Reduce encoding efforts
As I mentioned in my earlier post, there is no single HTML5 video codec. Over the last decade, most encoding tools have honed their H.264 encoding options and their overall output quality. WebM and Theora encoding tools and options though are still quite limited. If you stick with H.264, you’ll encode your video content with H.264 Baseline profile for smartphone delivery, and H.264 Main profile for tablet and desktop delivery. Within the next three years, I suspect we won’t need to worry as much as about H.264 Baseline profile as smartphones will universally have the GPU and/or CPU power to handle more advanced video decoding. Apple iPhone 4 and 4S already support H.264 Main profile playback. If you’re really savvy with your deployment, you’ll encode multiple bitrates in both H.264 Baseline and Main profiles for adaptive streaming. On videoRx.com, our automated presets output anywhere from 4 to 16 targets for adaptive streaming.
Goal: Reduce player logic/overhead
The standard mantra is to use the HTML5 <video> tag and only use a Flash-based video player as a fallback where the <video> tag isn’t recognized. I recommend to my clients the exact opposite approach: position a Flash-based video experience first, and fallback to HTML5 <video> support if the viewer isn’t on a platform that supports the Flash Player. With this approach, you can always use H.264 as your single video codec, because Flash Player supports it, and non-Flash-compatible platforms such as Apple iOS do support H.264 natively.
You can still create HTML5 and SEO friendly web content–you’re just going to use JavaScript to replace a <video> tag with Flash <object>/<embed> tags to play the video wherever the Flash Player is installed. This approach is not the same as putting <object> and <embed> tags within a <video> tag. On the contrary, JavaScript is used to swap out the <video> tag even within HTML5 capable browsers, to ensure consistency of H.264 playback. If you only have H.264 content, a non-H.264 compliant HTML5 browser such as Mozilla Firefox will not fallback to a nested <object>/<embed> Flash video player without the aid of JavaScript. MediaElement.js is one such library that will automatically insert an alternate playback engine if you only have H.264 content available. However, MediaElement.js doesn’t detect or implement adaptive streaming capabilities for both iOS and Flash Player. Therefore, if you want to use adaptive streaming techniques, you have to build your own JavaScript library like I’ve done on videoRx.com.
Note: If you’re wondering why I’m positioning the Flash Player as the first tier of playback over HTML5, just look at the lack of true fullscreen playback in HTML5 browsers. Firefox has only recently released its fullscreen API for their latest beta releases of Firefox 10. There are a slew of other features that we don’t have in current HTML5 specifications that the Flash Player has had for quite a while, but I’ll leave that discussion to another article.
Goal: Optimize delivery for mobile and desktop
Finally, encode and deploy using adaptive streaming technologies. While adaptive streaming was once a domain for broadcast and movie distribution companies, today any video content creator can deploy adaptive streaming video with even the most meager of budgets. I’m a huge fan of Wowza Media Server (WMS), which reduces much of the complexity and overhead with streaming management. You can use WMS on the cloud with Amazon EC2 and Amazon S3 storage for a relatively minor cost (or use a AWS-backed service like videoRx.com to even further minimize hosting costs). While you can use a standard web (HTTP) server with Apple HTTP Live Streaming (HLS) deployment , you need to pre-process all of your video into manifests and timed MPEG TS chunks. Once you’ve committed to that pre-processing workflow, you have to also encode non-HLS H.264 video files available for playback environments that don’t support HLS, regardless of whether you use Flash as a first responder or a fallback technology.
A Comprehensive Solution with AVC/H.264 and Adaptive Streaming
If you follow my recommendations, you’ll end up with a final scenario that includes the following tech setup:
- H.264 Baseline and Main profile encoded video content in two or more bitrates. That’s a minimum of four H.264 video files.
- A <div> container with an optional <video> tag for SEO purposes that is replaced with the appropriate Flash-based video player for adaptive streaming, or updated with an adequate source for the <video> tag that uses Apple HLS or progressive download.
- A JavaScript library that detects if you’re using a browser that’s HLS compliant, and use an appropriate M3U8 manifest file for H.264 Baseline playback on smartphones or a M3U8 manifest file for H.264 Main playback on tablet or desktop computers. If Flash Player is installed (or available for the platform viewing the video), the JS library will write Flash Player embed code for a video player that supports adaptive streaming such Adobe’s free Strobe Media Playback SWF.
- A Wowza Media Server instance that handles the transmuxing of your H.264 encoded content for Apple HLS and Adobe HTTP Dynamic Streaming (HDS), as well as the creation of M3U8 manifests for Apple HLS and F4M manifests for Adobe HDS.
To see past examples of this approach, you can review the HTML and JavaScript I’ve employed on the videoRx.com samples page.
The world of online video can certainly be intimidating, especially as non-Flash components enter your workflow and distribution. And there are real costs and trade-offs that flow directly from the decisions you and your clients make. I hope this post, and the one before it, help you decipher your options, and devise solutions that work for your needs.
[...] unfortunate mess of confounding options that we solution providers must present to decision makers. In my next post, I’ll outline the simplest and most effective solution to reduce the pain of distr…. Tweet(function() { var po = document.createElement('script'); po.type = 'text/javascript'; [...]
Very helpful. Thanks Robert!
I used Robert’s videoRx adaptive bit rate coding service and now host my videos on his Amazon S3 streaming server. 1) This was easy to set up and encode my videos and 2) my videos streaming just works without ever having go think about it. I had to digitize and preserve some history of 16mm file to preserve some of the history of a company named Tektronix. You can view my encoding and streaming efforts here: http://www.vintagetek.org/video-gallery/
If you are requiring multiple platform streaming and rendering of your videos, then you should look at the service solution discussed in this article.
I must say I find it very strange that you advise all your clients to go for a solution that will
a) Provide many of their customer with a troublesome experience, due to he CPU intensive nature of flash video, especially on OSX and linux, where flash gpu offloading is sill somewhat of a joke, or proof of concept.
Especially considering a video tag with H.264 plays SMOOTH as warm butter in both these OS-es, your advice seems very strange On OSX support is buit-in and no-wory, and HTML5 ith H.264 should ALWAYS be the first choice on that platform (Unless using a non-compatile browser). Linux users will have to have the right codecs and plugins instaled, but that is unlikely to be a major problem. Linux users is so seldom used to anything working out of the box anyway, ad as you can fall back to flash, this is not really an issue from a video publishers PoW
b) Make all the Apple fanboys angry, because they see it as their duty to bash, or not use, Flash anywhere they can. Norwegian state broadcaster NRK experienced that 75 seconds after opening their new streaming service to the public. They also have a Flash first-policy, and so far it is the single most complained about “feature”, both from OSX, linux and Windows users ho prefer a working, smooth stremaing exeprience.
Anyone recommending “Flash first” I usually consider an imbecile regarding video streaming an anything Non-Windows. As you obviously aren’t regarding videos, I can only assume you are lost concerning anything non-Windows, or don’t really care about the happiness of your customers customers. While flash is not so completely void of right to exist as Silverlight, it is still the second worst solution for any customer not running Windows with a GPU that flash support HD-acceleration for, with drivers installed that supports this, and a decent enough CPU.
If you’re only pushing a non-adaptive stream with H.264, then I would agree to not use a “Flash first” solution. However, adaptive streaming with H.264 is not universally supported across HTML5 browsers, and as a proponent of adaptive streaming for high-end online experiences, you must rely on “plugin-assist”, whether it be Flash or Silverlight where HLS implementations are not an option. I have to ask: how familiar are you with adaptive streaming, and implementing it for your clients/solutions?
I am a “Mac guy”, and in my tests with HTML5 fullscreen APIs and layering assets on top of video, I can tell you CPU/GPU with native H.264 is not all that great–I’m sure it will improve as we see newer browser releases.
good stuff Robert. I own miview and can say as a non techie that questions abound. as we are now in 2013, several clients are moving to hds (they are akamai clients).
they have been pushed to mobile phones and apple by expanding client bases.
but the security question is always asked when we move away from rtmp (e) and tokens.
what security is lost.. ? is it significant or non starter..