Web-Video and Mobile
Decoding video is a rather resource intensive task. Video is composed of hundreds of thousands, if not millions of pixels appearing on your screen and changing as many as 30 times a second!
Taking even a conservative resolution of 400×240 for a web video — a resolution that is fast disappearing as we get cheaper HD-capable camcorders — and at 24 frames per second, we get 400x240x24 pixels for just a second of video! Each pixel is represented by a 24bit (3 byte) number denoting its colour. Even at this meager resolution we get a size in area of 6.6 Megabytes per second!
The only reason that it is feasible to deliver video over the web, or even in VCDs, DVDs and BluRays is because video content is highly compressed. This is not just your usual zip, rar, or 7z compression, it is a highly complicated and complex series of algorithms that manage to get your video down from the 52 Megabits per second (1 byte = 8 bits) it takes to describe video at 400×240 to as low as a 0.25 Megabits per second. A compression of over 200 times!
Such heavy compression might seem unthinkable when you are dealing with documents, software or other data files, however video intrinsically has a lot of redundancy. If you look at a sequence of video one frame at a time, you will notice that usually there are few changes between frames. Video codecs such as H.264, DivX, XviD, Theora, VP8 etc take advantage by of this fact by making sure that they only store the important, changed bits of the video between frames.
Each codec uses a variety of methods that bring down the size of the video considerably. The effects are even more pronounced when dealing with HD video, which at 1920 x 1080 at 30 frames per second and 24bit color comes to as much as 1423 Megabits per second, while an HD YouTube video could be as low as 3 Megabits per second. The H.264 codec particularly uses numerous “tricks” to achieve great compression. The decoder will then have to recompose the original video while reading from such a heavily compressed source, and will have to do so fast enough to provide the desired frame rate of the video.
With such complexity in encoding it is only natural that decoding video will be a resource intensive task. As such the number of features / tricks used to compress video is restricted based on the “profile” of the video. These define what features can be used to compress the video and thus the complexity of the video.
As such, if a video decoder claims to be compatible with videos of a particular profile, then it should be able to decode any video that has been encoded in a way that is conformant with that profile. The profiles a mobile is capable of will be different than those a BluRay player will be capable of, while computer decoder software are usually able to decode any video in the format.
While using a higher profile might result is better quality at lower sizes, the complexity of the resultant file might be too much to allow playback on lower power devices.
To allow low-power devices such a mobiles to play higher quality video content, mobile devices use hardware decoders to decode video content in H.264 thus allowing for smooth playback of such content. This dedicated hardware is much more efficient in decoding the video, as it is dedicated to the task, and will put much lesser strain on the battery than if the CPU was being used. The absence of hardware decoding support is what is plaguing Google’s latest video format WebM (VP8 codec), however, considering how new it is, this is to be expected. This problem is only temporary as such support is sure to come in the future.
Such decoders are also found in desktop graphics cards, making it possible for people with slower single / dual core processors to play full HD video flawlessly. Once again profiles come into play. The hardware decoder is rigid, designed to accelerate the decoding only up to a particular profile. Hence a video encoded using a higher profile than a desktop graphics card supports will play using the CPU. Depending on the CPU this could mean higher CPU usage, choppy playback, or inability to play the video at all. Same is the case with mobile hardware, except that video that is above what the hardware decoder supports will probably not play at all or will be very inefficient.
For example Adobe Flash Player on mobiles (and desktops) will use the hardware video decoding capabilities of the device it is playing on to accelerate playback. This results in a smooth and efficient (low battery usage) playback of the video content. Since Flash Player on mobile uses the same codecs as on the PC, it till still play videos that are not encoded for the mobile profile. However, such content will be decoded use the CPU of the mobile, which is a lot less efficient at the task. The result, the device will run hotter, use battery faster and play less smoothly on the mobile device.
While encoding video content for the desktop and mobile, it is important to keep such considerations in mind. It is not enough to be accessible to people using multiple devices, but the resultant experience is also important. Video is one of the most resource (bandwidth, CPU usage, etc) intensive aspect of the web experience and a little optimization and consideration in this department can go a long way.