diff options
Diffstat (limited to 'DOCS/OUTDATED-tech/libmpcodecs.txt')
| -rw-r--r-- | DOCS/OUTDATED-tech/libmpcodecs.txt | 384 |
1 files changed, 384 insertions, 0 deletions
diff --git a/DOCS/OUTDATED-tech/libmpcodecs.txt b/DOCS/OUTDATED-tech/libmpcodecs.txt new file mode 100644 index 0000000000..b79df93d78 --- /dev/null +++ b/DOCS/OUTDATED-tech/libmpcodecs.txt @@ -0,0 +1,384 @@ +NOTE: If you want to implement a new native codec, please add it to +libavcodec. libmpcodecs is considered mostly deprecated, except for wrappers +around external libraries and codecs requiring binary support. + +The libMPcodecs API details, hints - by A'rpi +================================== + +See also: colorspaces.txt, codec-devel.txt, dr-methods.txt, codecs.conf.txt + +The VIDEO path: +=============== + + [MPlayer core] + | (1) + ______V______ (2) /~~~~~~~~~~\ (3,4) |~~~~~~| + | | -----> | vd_XXX.c | -------> | vd.c | + | dec_video | \__________/ <-(3a)-- |______| + | | -----, ,.............(3a,4a).....: + ~~~~~~~~~~~~~ (6) V V + /~~~~~~~~\ /~~~~~~~~\ (8) + | vf_X.c | --> | vf_Y.c | ----> vf_vo.c / ve_XXX.c + \________/ \________/ + | ^ + (7) | |~~~~~~| : (7a) + `-> | vf.c |...: + |______| + +Short description of video path: +1. MPlayer/MEncoder core requests the decoding of a compressed video frame: + calls dec_video.c::decode_video(). + +2. decode_video() calls the video codec previously selected in init_video(). + (vd_XXX.c file, where XXX == vfm name, see the 'driver' line of codecs.conf) + +3. The codec should initialize the output device before decoding the first + frame. It may happen in init() or at the middle of the first decode(), see + 3a. It means calling vd.c::mpcodecs_config_vo() with the image dimensions, + and the _preferred_ (meaning: internal, native, best) colorspace. + NOTE: This colorspace may not be equal to the colorspace that is actually + used. It's just a _hint_ for the colorspace matching algorithm and mainly + used as input format of the converter, but _only_ when colorspace + conversion is required, + +3a. Selecting the best output colorspace: + The vd.c::mpcodecs_config_vo() function will go through the outfmt list + defined by the 'out' lines in codecs.conf and query both vd (codec) and + vo (output device/filter/encoder) if the format is supported or not. + + For the vo, it calls the query_format() function of vf_XXX.c or ve_XXX.c. + It should return a set of feature flags, the most important ones for this + stage are: VFCAP_CSP_SUPPORTED (colorspace supported directly or by + conversion) and VFCAP_CSP_SUPPORTED_BY_HW (colorspace supported + WITHOUT any conversion). + + For the vd (codec), control() with VDCTRL_QUERY_FORMAT will be called. + If it does not implement VDCTRL_QUERY_FORMAT, (i.e. answers CONTROL_UNKNOWN + or CONTROL_NA), it is assumed to be CONTROL_TRUE (colorspace supported)! + + So, by default, if the list of supported colorspaces is constant and does + not depend on the actual file/stream header, then it is enough to list the + formats in codecs.conf ('out' field) and not implement VDCTRL_QUERY_FORMAT. + This is the case for most codecs. + + If the supported colorspace list depends on the file being decoded, list + the possible out formats (colorspaces) in codecs.conf and implement the + VDCTRL_QUERY_FORMAT to test the availability of the given colorspace for + the given video file/stream. + + The vd.c core will find the best matching colorspace, depending on the + VFCAP_CSP_SUPPORTED_BY_HW flag (see vfcap.h). If no match can be found, + it will try again with the 'scale' filter inserted between vd and vo. + If this is unsuccessful, it will fail :( + +4. Requesting buffer for the decoded frame: + The codec has to call mpcodecs_get_image() with proper imgtype & imgflag. + It will find the optimal buffering setup (preferred stride, alignment etc) + and return a pointer to the allocated and filled up mpi (mp_image_t*) struct. + The 'imgtype' controls the buffering setup, i.e. STATIC (just one buffer that + 'remembers' its content between frames), TEMP (write-only, full update), + EXPORT (memory allocation is done by the codec, not recommended) and so on. + The 'imgflags' set up the limits for the buffer, i.e. stride limitations, + readability, remembering content etc. See mp_image.h for the short + description. See dr-methods.txt for the explanation of buffer + importing and mpi imgtypes. + + Always try to implement stride support! (stride == bytes per line) + If no stride support, then stride==bytes_per_pixel*image_width. + If you have stride support in your decoder, use the mpi->stride[] value + for the byte_per_line for each plane. + Also take care of other imgflags, like MP_IMGFLAG_PRESERVE and + MP_IMGFLAG_READABLE, MP_IMGFLAG_COMMON_STRIDE and MP_IMGFLAG_COMMON_PLANE! + The file mp_image.h contains flag descriptions in comments, read it! + Ask for help on dev-eng, describing the behavior of your codec, if unsure. + +4.a. buffer allocation, vd.c::mpcodecs_get_image(): + If the requested buffer imgtype!=EXPORT, then vd.c will try to do + direct rendering, i.e. ask the next filter/vo for the buffer allocation. + It's done by calling get_image() of the vf_XXX.c file. + If it was successful, the imgflag MP_IMGFLAG_DIRECT will be set, and one + memcpy() will be saved when passing the data from vd to the next filter/vo. + See dr-methods.txt for details and examples. + +5. Decode the frame, to the mpi structure requested in 4., then return the mpi + to decvideo.c. Return NULL if the decoding failed or skipped the frame. + +6. decvideo.c::decode_video() will now pass the 'mpi' to the next filter (vf_X). + +7. The filter's (vf_X) put_image() then requests a new mpi buffer by calling + vf.c::vf_get_image(). + +7.a. vf.c::vf_get_image() will try to get direct rendering by asking the + next filter to do the buffer allocation (calls vf_Y's get_image()). + If it fails, it will fall back on normal system memory allocation. + +8. When we're past the whole filter chain (multiple filters can be connected, + even the same filter multiple times) then the last, 'leaf' filters will be + called. The only difference between leaf and non-leaf filters is that leaf + filters have to implement the whole filter API. + Currently leaf filters are: vf_vo.c (wrapper over libvo) and ve_XXX.c + (video encoders used by MEncoder). + + +Video Filters +============= + +Video filters are plugin-like code modules implementing the interface +defined in vf.h. + +Basically it means video output manipulation, i.e. these plugins can +modify the image and the image properties (size, colorspace, etc) between +the video decoders (vd.h) and the output layer (libvo or video encoders). + +The actual API is a mixture of the video decoder (vd.h) and libvo +(video_out.h) APIs. + +main differences: +- vf plugins may be "loaded" multiple times, with different parameters + and context - it's new in MPlayer, old APIs weren't reentrant. +- vf plugins don't have to implement all functions - all functions have a + 'fallback' version, so the plugins only override these if wanted. +- Each vf plugin has its own get_image context, and they can interchange + images/buffers using these get_image/put_image calls. + + +The VIDEO FILTER API: +===================== +filename: vf_FILTERNAME.c + +vf_info_t* info; + pointer to the filter description structure: + + const char *info; // description of the filter + const char *name; // short name of the filter, must be FILTERNAME + const char *author; // name and email/URL of the author(s) + const char *comment; // comment, URL to papers describing algorithm etc. + int (*open)(struct vf_instance *vf,char* args); + // pointer to the open() function: + +Sample: + +vf_info_t vf_info_foobar = { + "Universal Foo and Bar filter", + "foobar", + "Ms. Foo Bar", + "based on algorithm described at http://www.foo-bar.org", + open +}; + +The open() function: + + open() is called when the filter is appended/inserted in the filter chain. + It'll receive the handler (vf) and the optional filter parameters as + char* string. Note that encoders (ve_*) and vo wrapper (vf_vo.c) have + non-string arg, but it's specially handled by MPlayer/MEncoder. + + The open() function should fill the vf_instance_t structure with the + implemented functions' pointers (see below). + It can optionally allocate memory for its internal data (vf_priv_t) and + store the pointer in vf->priv. + + The open() function should parse (or at least check syntax of) parameters, + and fail (return 0) on error. + +Sample: + +static int open(vf_instance_t *vf, char* args) +{ + vf->query_format = query_format; + vf->config = config; + vf->put_image = put_image; + // allocate local storage: + vf->priv = malloc(sizeof(struct vf_priv_s)); + vf->priv->w = + vf->priv->h = -1; + if(args) // parse args: + if(sscanf(args, "%d:%d", &vf->priv->w, &vf->priv->h)!=2) return 0; + return 1; +} + +Functions in vf_instance: + +NOTE: All these are optional, their function pointer is either NULL or points +to a default implementation. If you implement them, don't forget to set +vf->FUNCNAME in your open() ! + + int (*query_format)(struct vf_instance *vf, unsigned int fmt); + +The query_format() function is called one or more times before the config(), +to find out the capabilities and/or support status of a given colorspace (fmt). +For the return values, see vfcap.h! +Normally, a filter should return at least VFCAP_CSP_SUPPORTED for all supported +colorspaces it accepts as input, and 0 for the unsupported ones. +If your filter does linear conversion, it should query the next filter, +and merge in its capability flags. Note: You should always ensure that the +next filter will accept at least one of your possible output colorspaces! + +Sample: + +static int query_format(struct vf_instance *vf, unsigned int fmt) +{ + switch(fmt){ + case IMGFMT_YV12: + case IMGFMT_I420: + case IMGFMT_IYUV: + case IMGFMT_422P: + return vf_next_query_format(vf,IMGFMT_YUY2) & (~VFCAP_CSP_SUPPORTED_BY_HW); + } + return 0; +} + +For the more complex case, when you have an N -> M colorspace mapping matrix, +see vf_scale or vf_format for examples. + + + int (*config)(struct vf_instance *vf, + int width, int height, int d_width, int d_height, + unsigned int flags, unsigned int outfmt); + +The config() is called to initialize/configure the filter before using it. +Its parameters are already well-known from libvo: + width, height: size of the coded image + d_width, d_height: wanted display size (usually aspect corrected w/h) + Filters should use width, height as input image dimension, but the + resizing filters (crop, expand, scale, rotate, etc) should update + d_width/d_height (display size) to preserve the correct aspect ratio! + Filters should not rely on d_width, d_height as input parameters, + the only exception is when a filter replaces some libvo functionality + (like -vf scale with -zoom, or OSD rendering with -vf expand). + flags: the "good" old libvo flag set: + 0x01 - force fullscreen (-fs) + 0x02 - allow mode switching (-vm) + 0x04 - allow software scaling (-zoom) + 0x08 - flipping (-flip) + (Usually you don't have to worry about flags, just pass it to next config.) + outfmt: the selected colorspace/pixelformat. You'll receive images in this + format. + +Sample: + +static int config(struct vf_instance *vf, + int width, int height, int d_width, int d_height, + unsigned int flags, unsigned int outfmt) +{ + // use d_width/d_height if not set by the user: + if (vf->priv->w == -1) + vf->priv->w = d_width; + if (vf->priv->h == -1) + vf->priv->h = d_height; + // initialize your filter code + ... + // OK now config the rest of the filter chain, with our output parameters: + return vf_next_config(vf,vf->priv->w,vf->priv->h,d_width,d_height,flags,outfmt); +} + + void (*uninit)(struct vf_instance *vf); + +Okay, uninit() is the simplest, it's called at the end. You can free your +private buffers etc here. + + int (*put_image)(struct vf_instance *vf, mp_image_t *mpi); + +Ah, put_image(). This is the main filter function, it should convert/filter/ +transform the image data from one format/size/color/whatever to another. +Its input parameter is an mpi (mplayer image) structure, see mp_image.h. +Your filter has to request a new image buffer for the output, using the +vf_get_image() function. NOTE: Even if you don't want to modify the image, +just pass it to the next filter, you have to either +- not implement put_image() at all - then it will be skipped +- request a new image with type==EXPORT and copy the pointers +NEVER pass the mpi as-is, it's local to the filters and may cause trouble. + +If you completely copy/transform the image, then you probably want this: + + dmpi = vf_get_image(vf->next,mpi->imgfmt, MP_IMGTYPE_TEMP, + MP_IMGFLAG_ACCEPT_STRIDE, vf->priv->w, vf->priv->h); + +It will allocate a new image and return an mp_image structure filled by +buffer pointers and stride (bytes per line) values, in size of vf->priv->w +times vf->priv->h. If your filter cannot handle stride, then leave out +MP_IMGFLAG_ACCEPT_STRIDE. Note that you can do this, but it isn't recommended, +the whole video path is designed to use strides to get optimal throughput. +If your filter allocates output image buffers, then use MP_IMGTYPE_EXPORT +and fill the returned dmpi's planes[], stride[] with your buffer parameters. +Note, it is not recommended (no direct rendering), so if you can, use +vf_get_image() for buffer allocation! +For other image types and flags see mp_image.h, it has comments. +If you are unsure, feel free to ask on the dev-eng mailing list. Please +describe the behavior of your filter, and its limitations, so we can +suggest the optimal buffer type + flags for your code. + +Now that you have the input (mpi) and output (dmpi) buffers, you can do +the conversion. If you didn't notice yet, mp_image has some useful info +fields. They may help you a lot creating if() or for() structures: + flags: MP_IMGFLAG_PLANAR, MP_IMGFLAG_YUV, MP_IMGFLAG_SWAPPED + helps you to handle various pixel formats in single code. + bpp: bits per pixel + WARNING! It's number of bits _allocated_ to store a pixel, + it is not the number of bits actually used to keep colors! + So it's 16 for both 15 and 16 bit color depth, and is 32 for + 32bpp (actually 24 bit color depth) mode! + It's 1 for 1bpp, 9 for YVU9, and is 12 for YV12 mode. Get it? + For planar formats, you also have chroma_width, chroma_height and + chroma_x_shift, chroma_y_shift too, they specify the chroma subsampling + for YUV formats: + chroma_width = luma_width >> chroma_x_shift; + chroma_height = luma_height >> chroma_y_shift; + +If you're done, call the rest of the filter chain to process your output +image: + return vf_next_put_image(vf,dmpi); + + +Ok, the rest is for advanced functionality only: + + int (*control)(struct vf_instance *vf, int request, void* data); + +You can control the filter at runtime from MPlayer/MEncoder/dec_video: +#define VFCTRL_QUERY_MAX_PP_LEVEL 4 /* test for postprocessing support (max level) */ +#define VFCTRL_SET_PP_LEVEL 5 /* set postprocessing level */ +#define VFCTRL_SET_EQUALIZER 6 /* set color options (brightness,contrast etc) */ +#define VFCTRL_GET_EQUALIZER 8 /* get color options (brightness,contrast etc) */ +#define VFCTRL_DRAW_OSD 7 +#define VFCTRL_CHANGE_RECTANGLE 9 /* Change the rectangle boundaries */ + + + void (*get_image)(struct vf_instance *vf, mp_image_t *mpi); + +This is for direct rendering support, works the same way as in libvo drivers. +It makes in-place pixel modifications possible. +If you implement it (vf->get_image!=NULL), then it will be called to do the +buffer allocation. You SHOULD check the buffer restrictions (stride, type, +readability etc) and if everything is OK, then allocate the requested buffer +using the vf_get_image() function and copying the buffer pointers. + +NOTE: You HAVE TO save the dmpi pointer, as you'll need it in put_image() +later on. It is not guaranteed that you'll get the same mpi for put_image() as +in get_image() (think of out-of-order decoding, get_image is called in decoding +order, while put_image is called for display) so the only safe place to save +it is in the mpi struct itself: mpi->priv=(void*)dmpi; + + + void (*draw_slice)(struct vf_instance *vf, unsigned char** src, + int* stride, int w,int h, int x, int y); + +It's the good old draw_slice callback, already known from libvo. +If your filter can operate on partial images, you can implement this one +to improve performance (cache utilization). + +Ah, and there are two sets of capability/requirement flags (vfcap.h type) +in vf_instance_t, used by the default query_format() implementation, and by +the automatic colorspace/stride matching code (vf_next_config()). + + // caps: + unsigned int default_caps; // used by default query_format() + unsigned int default_reqs; // used by default config() + +BTW, you should avoid using global or static variables to store filter instance +specific stuff, as filters might be used multiple times and in the future even +multiple streams might be possible. + + +The AUDIO path: +=============== +TODO!!! |