Open Video Codecs and Flash

General
Theora
Xvid
OMS Video
Dirac / Schrödinger
Flash / flv / f4v

General

When a standard is open it allows for a huge adoption of it by anyone, anyone can use it and be sure that their data isn’t locked away and they have to deal with a specific company if they want to access their own content. Open Standards are what runs the Internet. The problem is that being an ‘Open Standard’ isn’t all that’s required. H.264 for instance is an Open Standard but its not royalty-free as there are patents on it, and it requires a licensing fee for implementation. While these licenses are cheap and easy to obtain for companies making them attractive, they block the formats for the non-commercial open source community. You are still allowing a 3rd party to dictate the requirements for access to your data.

This is where the much hated software patents come into it, you cannot distribute patented software in binary, precompiled form as a patent has to be applied to a physical object (thanks to a court case in America binary code somehow now counts as such, while other countries have various laws America is where Silicon valley is so we all loose out, some countries seen to be specifically making exceptions to allow patents to be applied on computers). You can distribute patent software in source code since its not an actual implementation of it. As for if you can legally compile that code for personal use various from country to country, there is some discussion on that here.

Firstly its important to have an royalty-free unencumbered codec for use in streaming video for things such as Firefox and Linux/Unix distributions to be able to legally play back these formats, patents are the reason that in order to support MP3 playback you have to install codecs (which in newer distributions is a lot easier and automatically setup). Commercial distros can afford to pay the patent license fees but this isn’t much help for for the open source community, or hobbyists, Ubuntu/Debian/Fedora/Gentoo/Arch/BSDs etc… aren’t commercial distros, they don’t charge you so they can’t pay for the codecs and if they could pay for them then the media is still in a format that is locked away accessible on the whims of the patent holder.

Since the HTML 5 draft (due to be finalized 4 years from now in 2012) included video streaming, having a decent open codec is more important now than ever before, originally the draft had mentioned the use of Ogg however Nokia and Apple raised objections concerned about hidden ‘submarine patents’, low compression ratio and lack of hardware decoders, Nokia wanting support for H.264 (which also happens to be the codec Apple is already using for iTunes/iPod video along with AAC for audio) or alternatively leaving out streaming video and letting corporations fight it out. H.264 being impossible to include in the standard.

As for the royalty-free video codecs that around around we have, Theora, DIRAC and OMS.

Ogg Theora

Firstly there is the oldest and most widely known Theora codec, often referred to as “Ogg Theora” as its contained in the Ogg container format, not to be confused with Ogg Vorbis which is an audio codec designed to be a royalty-free alternative to MP3, also lives in the Ogg container format and is often used to provide the audio for Theora videos in Ogg format.

Theora is a project of the Xiph.Org foundation (also responsible for the royalty-free codecs, FLAC for lossless audio and Speex a voice audio codec with an extremely good compression ratio), its based on VP3 which was donated to the public by its creator On2 who dropped all claims on it.

Unfortunately is seems that Theora is now out of date and has fairly bad compression when compared to other codecs. Xpih.Org are apparently working on an improved version of Theora for HTML 5 but with the binary format locked for compatibility its unclear to me if it can be improved enough to reduce file sizes and improve quality or if its just work on improving the tools around Theora.

Xvid

Xvid is apparently a royalty-free codec, originally from OpenDivX code it was forked when the DivX 4 closed source. The problem is that Xvid is based on the MPEG-4 standard which has 2 dozen companies claiming patents on it and licenses are apparently no longer being offered.

OMS Video

Sun’s Open Media Commons recently announced OMS Video, and open coded, the audio component is using the video component is based on H.261 which is out of the 17 year patent restriction, then adding newer unpatented technologies. Currently there isn’t anything from them yet code wise. Another worry is another Open Media Commons project, DReaM, its a DRM specification, as far as DRM goes it seems less evil since its designed to be open and royalty-free itself but its still DRM, in the end as long as the DRM isn’t built into OMS it shouldn’t be a problem but I have a small concern that they will use OMS as an infection vector for DReaM. The announcement and specification overview don’t mention DReaM at all other than saying its also part of Open Media Commons so its probably not an issue but worth watching. Fortunately DRM is its own worst enemy, DReaM is supposed to bring an open royalty free DRM system to allow music to interoperate but DRM seems less about protecting music and more about online music retailers locking clients to their system/devices, one someone has a whole database of DRM’d songs they will have to buy hardware that supports it for ever and keep shopping at the same place, they can never leave (at least not without breaking the DRM or loosing all their music), you can read more about why DRM sucks at the Defective By Design website.

Dirac / Schrödinger

The BBC who have been experimenting with streaming video created Dirac (wikipedia) which is designed to be completely unencumbered by using patent free technologies. Wikipedia says it is in the same range of compression as h264. There is an implementation of DIRAC called Schrödinger which has libraries, gstreamer plug-ins and is intended to get it in the Ogg container.

Flash / flv / f4v

Recently Adobe with their Open Screen Project, opened flash and the flv/fl4 format for use without license restrictions, the swf specification and the flv specifications are already published. This is great news for projects like Gnash however my main concern however is that flv has technologies using patents in it. For instance flv in Flash 9 supports AAC for audio and the Wikipedia article on ACC says:
“…a patent license is required for all manufacturers or developers of AAC codecs, that require encoding or decoding. It is for this reason FOSS implementations such as FAAC and FAAD are distributed in source form only, in order to avoid patent infringement.”. This makes it seem like even though the license restriction is removed, the open source community will benefit from having the API’s available but not be able to actual make a binary version of the flash client. You won’t be able to expect flash to be built into Firefox or shipped with Ubuntu. The real clients of Adobe will still likely need a license from Adobe unless they want to go to patent holders such as AAC and independently obtain licenses (likely to end up costing more in the end). Another format used is MP3 which has a whole load of parent issues, the MP3 decoding patents run out around 2012 and the encoding later around 2016 (Ive seen various different times but their fairly close, there is a big list of mp3 patents but it doesn’t say what is needed for decoding/encoding and whats optional, the latest is 2017), flv also uses yet another commercial proprietary codec Nellymoser.

These are just the audio codecs for the video there is H.263 since Flash 6 and as of Flash 8
VP6, I haven’t found much information on the license issues around them but they do seem to be patented. Wikipedia says “As of September 2006, an open-source implementation of the decoder is part of the libavcodec project, though producing or dealing with VP6 video streams inside libavcodec/libavformat seems to be discouraged and/or refused due to clashes between the ffmpeg’s developers and On2 technologies by a claim of Intellectual Property and Trade Secrets Infringement made by the corporation itself.”

As for Flash itself I have no idea about what other patents on the technology exist when we live in a world where anti-aliasing fonts is patented. In order for flash to really be open source friendly we would need to see it adopt patent free codecs for flv (such as DIRAC, Vorbis, Theora or OMS).