A Guide To Developing MPlayer Codecs by Mike Melanson (melanson at pcisys dot net) Introduction ------------ I've developed a number of open source decoders for the MPlayer project, for both audio and video data. As such, I feel I'm qualified to document a few notes about developing new codecs for the codebase. As always, the best way to learn how to incorporate a new codec is to study a bunch of existing code. This document is supplementary material to the code, meant to give some tips, pointers, and a general roadmap. A note about terminology: "Codec" stands for coder/decoder (or compressor/decompressor, if you prefer). The term refers to a module that can both encode and decode data. However, this document focuses primarily on incorporating decoders. Still, the terms "decoder" and "codec" are often used interchangeably. Necessary Materials ------------------- So you've decided that you want to implement a new decoder for MPlayer. There are a few things you will need: - Knowledge of the codec to be implemented: You will need to know the data format of the chunks that MPlayer will pass to you. You will need to know how to take apart the data structures inside. You will need to know the algorithmic operations that need to be performed on the data in order to reconstruct the original media. - Sample media: Preferably, lots of it. You will need media encoded in your data format and stored in a media file format that MPlayer knows how to parse (these include AVI, ASF, MOV, RM, VIVO, among others). If the encoded data is stored in a media file format that MPlayer doesn't understand, then you will either need to somehow convert the format to a media file format that the program does understand, or write your own MPlayer file demuxer that can handle the data. Writing a file demuxer is beyond the scope of this document. Try to obtain media that stresses all possible modes of a decoder. If an audio codec is known to work with both mono and stereo data, search for sample media of both types. If a video codec is known to work at 7 different bit depths, then, as painful as it may be, do what you can to obtain sample media encoded for each of the 7 bit depths. - Latest CVS snapshot: It's always useful to develop code for the very latest development version of MPlayer. Be sure to update your local CVS copy often. - General programming knowledge, working Linux development environment: I would hope that these items would go without saying, but you never know. Typical Development Cycle ------------------------- 1) Set up basic infrastructure First things first, there's a big song and dance to go through in order to let the MPlayer program know that you have a new codec to incorporate. MPlayer does not feature what some would term a "clean" codec plugin architecture. Some log this as a complaint. Personally, I think it's necessary to allow MPlayer the type of flexibility to incorporate so many open- and closed-source codecs. First, modify your local copy of codecs.conf. It may be system-shared or in your home directory. Add a new entry for your codec. If it's an open source codec, it would be a good idea to place the new entry with the rest of the open source codecs. When you're confident that you have the entry right, be sure to add it to etc/codecs.conf in your workspace. See the file codecs.conf.txt for a detailed description of the format of this file. Create a new audiocodec or videocodec block with the proper info, FOURCCs/format numbers, output formats, and a unique driver name. Remember the driver name. Next, edit the file codec-cfg.h. You will find a list of #define's that map names like AFM_MSADPCM and VFM_MSVIDC to numbers. The definitions that begin with AFM_ are audio drivers. The definitions that begin with VFM_ are video drivers. If you want to implement a new audio driver, go to the end of the AFM_ list and create a new AFM_ definition for your decoder using the next available number in the list. If you want to make a new video decoder, do the same thing to the VFM_ list. Next, edit the file codec-cfg.c. You will find an array of audio driver names and video driver names. If you are implementing a new audio codec, add your new driver name (the one you entered into the codecs.conf file) between the last non-NULL entry and the NULL at the end of the audio driver array. If you are implementing a new video codec, do the same in the video driver array. Next, create a new source file which contains the main decoding function that MPlayer will call to decode data. Eventually, you may have multiple files which comprise your decoder, but let's start simple here. Create the skeleton function for your decoder. Since you will also have to write the code to invoke the function, you can make the decoding function look however you want, with whatever parameters you feel will be necessary. Here's an example video decoder: void some_video_decoder( char *encoded, // buffer of encoded data int encoded_size, // length of encoded buffer char *decoded, // buffer where decoded data is written int width, // width of decoded frame in pixels int height, // height of decoded frame in pixels int bytes_per_pixel) // bytes/pixel in output image Here's an example audio decoder: int some_audio_decoder( unsigned short *output, // buffer where decoded 16-bit PCM samples go unsigned char *input, // encoded data int channels) // mono = 1, stereo = 2 Next, modify the Makefile so that it will compile your new source file. Next, modify either dec_audio.c or dec_video.c, depending on whether you're writing an audio or video decoder. You'll probably put the new decoder function header at the top of the file unless you've created a header file to handle it, in which case, you'll include the new header file. The dec_*.c functions are in charge of initializing codecs and then passing encoded data to the proper decoder function. The init and decode functions are big switch statements that key off of the codec definition numbers from codec-cfg.h. Your best bet in here is to examine some simple other simple decoders and clone relevant portions of the case blocks. Next, compile the project and see if you have everything correct so far. Next, you want to make sure that the encoded data is making it to your decoding function in the first place. This may sound like a trivial exercise, but there are a lot of things that can go wrong (and I've watched most of them go wrong in my experience). At the beginning of your skeleton decoder function, enter the following code: int i; for (i = 0; i < 16; i++) printf ("%02X ", input[i]); printf ("\n"); When you compile and run MPlayer, your decoder function will print the first 16 bytes of each data chunk that it receives. Open the sample media in a hex editor and reconcile what you see on the screen with what you find in the file. If the decoder is printing the first 16 bytes of each block, that's a good sign that you're ready to move on to step 2. Otherwise, you need to figure out why the data isn't getting to your decoder. Is your decoder even being invoked? If not, why not? 2) Develop the decoder Go for it. Remember to make it work, first, then make it work fast. 3) Debug and test the decoder If you're extremely lucky, the decoder will work the first time. If you're very lucky, it will work after you've reviewed your code a few times and corrected a few obvious programming mistakes. Realistically, you will write the decoder, review it many times and fix many obvious and subtle programming errors, and still have to go through an elaborate debug process in order to get the decoder to a minimally functional state. Big hint: Ask for all warnings. You know, the -Wall option in gcc? It's very useful to develop your codec while running in debug mode. In order to compile MPlayer with debug support (which includes -Wall for all gcc operations), use the --enable-debug option when configuring the project. Pay attention to all warnings and make it a goal to get rid of every single one. I'll never forget when the compiler warned me that there was no point in clamping a signed 16-bit variable within a signed 16-bit range (the calculation to be clamped was supposed to be stored in a signed 32-bit variable and then stored in the signed 16-bit variable). I sat stunned for a moment, feeling like I had just dodged a bullet as I knew that would have taken me hours to debug that kind of mistake. 4) Contribute decoder to codebase Create a patch with the "diff -u" format and email it to the MPlayer development team for approval. You will likely need to diff the following files: - Makefile - etc/codecs.conf - codec-cfg.c - codec-cfg.h - dec_audio.c -OR- dec_video.c Of course, you will need to include your newly-created file(s). If you contribute enough decoders, the development team may even grant you write privileges to the CVS repository. 5) Wait for bug reports to start rolling in You may think you're finished when you release the codec and if you're extremely lucky, you will be right. However, it's more likely that people will start throwing all kinds of oddball media at your decoder that it never counted on. Cheer up; take comfort in knowing that people are testing your code and attempting to use it as a real world application. Download the problem media that people upload to the MPlayer FTP site and get back to work, implementing fixed code that addresses the issues. Contribute more patches and encourage people to hammer on your decoder even more. This is how you make your decoder rock-solid. EOF