summaryrefslogtreecommitdiffstats
path: root/video/mp_image.h
Commit message (Collapse)AuthorAgeFilesLines
* mp_image: add helper for clearing regions outside of a rectanglewm42020-05-221-0/+2
| | | | Not sure if generally useful; the following commit uses it.
* video: clean up some imgfmt related stuffwm42020-05-181-0/+1
| | | | | | | | | | | | | | | | Remove the vaguely defined plane_bits and component_bits fields from struct mp_imgfmt_desc. Add weird replacements for existing uses. Remove the bytes[] field, replace uses with bpp[]. Fix some potential alignment issues in existing code. As a compromise, split mp_image_pixel_ptr() into 2 functions, because I think it's a bad idea to implicitly round, but for some callers being slightly less strict is convenient. This shouldn't really change anything. In fact, it's a 100% useless change. I'm just cleaning up what I started almost 8 years ago (see commit 00653a3eb052). With this I've decided to keep mp_imgfmt_desc, just removing the weird parts, and keeping the saner parts.
* mp_image: add some helperswm42020-05-061-0/+3
| | | | | | | | | | This is really basic for planar image data access; not sure why there weren't such helpers before. They also handle trickier formats that use bit-packing, or they would be mich simpler. (This affects only BGR4/BGR4/MONOW/MONOH, I hope whoever invented them is proud of triggering so many special cases for so little gain.)
* video: add alpha type metadatawm42020-04-241-0/+1
| | | | | | | | | This is mostly for testing. It adds passing through the metadata through the video chain. The metadata can be manipulated with vf_format. Support for zimg alpha conversion (if built with zimg after it gained alpha support) is implemented. Support premultiplied input in vo_gpu. Some things still seem to be buggy.
* video, demux: rip out unused spherical metadata codewm42019-10-171-15/+0
| | | | | | This was preparation into something that never happened. Spherical video is a shit idea anyway.
* video: remove mp_image_params.hw_flags fieldwm42019-10-171-6/+0
| | | | | | | | This was speculatively added 2 years ago in preparation for something that apparently never happened. The D3D code was added as an "example", but this too was never used/finished. No reason to keep this.
* video: generally try to align image data on 64 byteswm42019-09-191-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Generally, using x86 SIMD efficiently (or crash-free) requires aligning all data on boundaries of 16, 32, or 64 (depending on instruction set used). 64 bytes is needed or AVX-512, 32 for old AVX, 16 for SSE. Both FFmpeg and zimg usually require aligned data for this reason. FFmpeg is very unclear about alignment. Yes, it requires you to align data pointers and strides. No, it doesn't tell you how much, except sometimes (libavcodec has a legacy-looking avcodec_align_dimensions2() API function, that requires a heavy-weight AVCodecContext as argument). Sometimes, FFmpeg will take a shit on YOUR and ITS OWN alignment. For example, vf_crop will randomly reduce alignment of data pointers, depending on the crop parameters. On the other hand, some libavfilter filters or libavcodec encoders may randomly crash if they get the wrong alignment. I have no idea how this thing works at all. FFmpeg usually doesn't seem to signal alignment internal anywhere, and usually leaves it to av_malloc() etc. to allocate with proper alignment. libavutil/mem.c currently has a ALIGN define, which is set to 64 if FFmpeg is built with AVX-512 support, or as low as 16 if built without any AVX support. The really funny thing is that a normal FFmpeg build will e.g. align tiny string allocations to 64 bytes, even if the machine does not support AVX at all. For zimg use (in a later commit), we also want guaranteed alignment. Modern x86 should actually not be much slower at unaligned accesses, but that doesn't help. zimg's dumb intrinsic code apparently randomly chooses between aligned or unaligned accesses (depending on compiler, I guess), and on some CPUs these can even cause crashes. So just treat the requirement to align as a fact of life. All this means that we should probably make sure our own allocations are 64 bit aligned. This still doesn't guarantee alignment in all cases, but it's slightly better than before. This also makes me wonder whether we should always override libavcodec's buffer pool, just so we have a guaranteed alignment. Currently, we only do that if --vd-lavc-dr is used (and if that actually works). On the other hand, it always uses DR on my machine, so who cares.
* Implement backwards playbackwm42019-09-191-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | See manpage additions. This is a huge hack. You can bet there are shit tons of bugs. It's literally forcing square pegs into round holes. Hopefully, the manpage wall of text makes it clear enough that the whole shit can easily crash and burn. (Although it shouldn't literally crash. That would be a bug. It possibly _could_ start a fire by entering some sort of endless loop, not a literal one, just something where it tries to do work without making progress.) (Some obvious bugs I simply ignored for this initial version, but there's a number of potential bugs I can't even imagine. Normal playback should remain completely unaffected, though.) How this works is also described in the manpage. Basically, we demux in reverse, then we decode in reverse, then we render in reverse. The decoding part is the simplest: just reorder the decoder output. This weirdly integrates with the timeline/ordered chapter code, which also has special requirements on feeding the packets to the decoder in a non-straightforward way (it doesn't conflict, although a bugmessmass breaks correct slicing of segments, so EDL/ordered chapter playback is broken in backward direction). Backward demuxing is pretty involved. In theory, it could be much easier: simply iterating the usual demuxer output backward. But this just doesn't fit into our code, so there's a cthulhu nightmare of shit. To be specific, each stream (audio, video) is reversed separately. At least this means we can do backward playback within cached content (for example, you could play backwards in a live stream; on that note, it disables prefetching, which would lead to losing new live video, but this could be avoided). The fuckmess also meant that I didn't bother trying to support subtitles. Subtitles are a problem because they're "sparse" streams. They need to be "passively" demuxed: you don't try to read a subtitle packet, you demux audio and video, and then look whether there was a subtitle packet. This means to get subtitles for a time range, you need to know that you demuxed video and audio over this range, which becomes pretty messy when you demux audio and video backwards separately. Backward display is the most weird (and potentially buggy) part. To avoid that we need to touch a LOT of timing code, we negate all timestamps. The basic idea is that due to the navigation, all comparisons and subtractions of timestamps keep working, and you don't need to touch every single of them to "reverse" them. E.g.: bool before = pts_a < pts_b; would need to be: bool before = forward ? pts_a < pts_b : pts_a > pts_b; or: bool before = pts_a * dir < pts_b * dir; or if you, as it's implemented now, just do this after decoding: pts_a *= dir; pts_b *= dir; and then in the normal timing/renderer code: bool before = pts_a < pts_b; Consequently, we don't need many changes in the latter code. But some assumptions inhererently true for forward playback may have been broken anyway. What is mainly needed is fixing places where values are passed between positive and negative "domains". For example, seeking and timestamp user display always uses positive timestamps. The main mess is that it's not obvious which domain a given variable should or does use. Well, in my tests with a single file, it suddenly started to work when I did this. I'm honestly surprised that it did, and that I didn't have to change a single line in the timing code past decoder (just something minor to make external/cached text subtitles display). I committed it immediately while avoiding thinking about it. But there really likely are subtle problems of all sorts. As far as I'm aware, gstreamer also supports backward playback. When I looked at this years ago, I couldn't find a way to actually try this, and I didn't revisit it now. Back then I also read talk slides from the person who implemented it, and I'm not sure if and which ideas I might have taken from it. It's possible that the timestamp reversal is inspired by it, but I didn't check. (I think it claimed that it could avoid large changes by changing a sign?) VapourSynth has some sort of reverse function, which provides a backward view on a video. The function itself is trivial to implement, as VapourSynth aims to provide random access to video by frame numbers (so you just request decreasing frame numbers). From what I remember, it wasn't exactly fluid, but it worked. It's implemented by creating an index, and seeking to the target on demand, and a bunch of caching. mpv could use it, but it would either require using VapourSynth as demuxer and decoder for everything, or replacing the current file every time something is supposed to be played backwards. FFmpeg's libavfilter has reversal filters for audio and video. These require buffering the entire media data of the file, and don't really fit into mpv's architecture. It could be used by playing a libavfilter graph that also demuxes, but that's like VapourSynth but worse.
* video: remove internal stereo_out flagwm42018-04-291-2/+1
| | | | | | Also rename stereo3d to stereo_in. The only real change is that the vo_gpu OSD code now uses the actual stereo 3D mode, instead of the --video-steroe-mode value. (Why does this vo_gpu code even exist?)
* video: pass through container fps to filterswm42018-04-191-0/+2
| | | | | | | | | | | | This means vf_vapoursynth doesn't need a hack to work around the filter code, and libavfilter filters now actually get the frame_rate field on input pads set. The libavfilter doxygen says the frame_rate field is only to be set if the frame rate is known to be constant, and uses the word "must" (which probably means they really mean it?) - but ffmpeg.c sets the field to mere guesses anyway, and it looks like this normally won't lead to problems.
* mp_image: pass through unknown AVFrame side datawm42018-03-031-0/+8
| | | | | | | | | Useful for libavfilter. Somewhat risky, because we can't ensure the consistency of the unknown side data (but this is a general problem with side data, and libavfilter filters will usually get it wrong too _if_ there are conflict cases). Fixes #5569.
* mp_image: preserve AVFrame closed captions datawm42018-01-301-0/+2
| | | | | | | | | | This is preparation for a change in vd_lavc.c: it should not have to access the demuxer (to pass along closed captions), so the idea is to make them part of mp_image, and to let the layer above vd_lavc propagate the buffer. Don't bother with preserving them for mp_image->AVFrame, because we don't need this.
* vd_lavc, mp_image: remove weird mpv specific palette constantwm42017-12-011-2/+0
| | | | | Was for times when we were trying to be less dependent on libav* I guess.
* video: add mp_image_params.hw_flags and add an examplewm42017-10-161-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | It seems this will be useful for Rokchip DRM hwcontext integration. DRM hwcontexts have additional internal structure which can be different depending on the decoder, and which is not part of the generic hwcontext API. Rockchip has 1 layer, which EGL interop happens to translate to a RGB texture, while VAAPI (mapped as DRM hwcontext) will use multiple layers. Both will use sw_format=nv12, and thus are indistinguishable on the mp_image_params level. But this is needed to initialize the EGL mapping and the vo_gpu video renderer correctly. We hope that the layer count is enough to tell whether EGL will translate the data to a RGB texture (vs. 2 texture resembling raw nv12 data). For that we introduce MP_IMAGE_HW_FLAG_OPAQUE. This commit adds the flag, infrastructure to set it, and an "example" for D3D11. The D3D11 addition is quite useless at this point. But later we want to get rid of d3d11_update_image_attribs() anyway, while we still need a way to force d3d11vpp filter insertion, so maybe it has some justification (who knows). In any case it makes testing this easier. Obviously it also adds some basic support for triggering the opaque format for decoding, which will use a driver-specific format, but which is not supported in shaders. The opaque flag is not used to determine whether d3d11vpp needs to be inserted, though.
* video: drop old D3D11/DXVA2 supportwm42017-09-261-3/+0
| | | | | | | | | Now you need FFmpeg git, or something. This also gets rid of the last real use of gpu_memcpy(). libavutil does that itself. (vaapi.c still used it, but it was essentially unused, because the code path isn't really in use anymore. It wasn't even included due to the d3d-hwaccel dependency in wscript.)
* build: add preliminary LGPL modewm42017-09-211-9/+7
| | | | | | | See "Copyright" file for caveats. This changes the remaining "almost LGPL" files to LGPL, because we think that the conditions the author set for these was finally fulfilled.
* video: add metadata handling for spherical videowm42017-08-211-1/+16
| | | | | | | | | | | | | | This adds handling of spherical video metadata: retrieving it from demux_lavf and demux_mkv, passing it through filters, and adjusting it with vf_format. This does not include support for rendering this type of video. We don't expect we need/want to support the other projection types like cube maps, so we don't include that for now. They can be added later as needed. Also raise the maximum sizes of stringified image params, since they can get really long.
* vd_lavc: decode embedded ICC profilesNiklas Haas2017-08-031-0/+2
| | | | | | | | | Since these need to be refcounted, we throw them directly into struct mp_image instead of being part of mp_colorspace. Even though they would semantically make more sense in mp_colorspace, having them there is really awkward because mp_colorspace is passed around and stored a lot, and this way their lifetime is exactly tied to the lifetime of the mp_image associated with it.
* mp_image: expose some image allocation code as helpers, refactorwm42017-07-231-0/+6
| | | | | | | Refactor the image allocation code, and expose part of it as helper code. This aims towards allowing callers to easily allocate mp_image references from custom-allocated linear buffers. This is exposing only as much as what should be actually required.
* mp_image: change license to LGPL (almost)wm42017-06-161-0/+2
| | | | | | | | | | | | | | | | | | | | | | Since michael was somewhat involved in it, wait with the actual license change until the core is relicensed. Thus mark it as "Almost LGPL.". The worrisome part about mp_image.c is that it was created by cehoyos (which disagreed with LGPL) in commit f2dee327b2797. But it turns out it was a patch by someone else (who agreed with LGPL). For some reason, the patch was actually slightly modified by cehoyos for no reason (messed with the include statements), so we mess them back, just to be sure. Other than this, there were some commits that added support for new IMGFMTs over the years. Some of these were by people we didn't ask or we didn't get permission from. But since the original mp_image code was replaced by more generic code using FFmpeg pixdesc, none of these changes are left anyway. One additional change by cehoyos (115bfb976270) has been removed as well (when "direct rendering" was dropped from the filter chain).
* lavfi: use mp_image to store the filter pad formatwm42017-02-201-0/+1
| | | | | | | | | | Preparation for enabling hw filters. mp_image_params can't have an AVHWFramesContext reference (because it can't hold any allocations, and isn't meant to hold "active" data in the first place. So just use a mp_image. It has all real data removed, because that would essentially leak 1 frame once the decoder or renderer don't need it anymore.
* video: use demuxer-signaled duration for last video framewm42016-12-211-1/+1
| | | | | | | | | Helps with gif, probably does unwanted things with other formats. This doesn't handle --end quite correctly, but this could be added later. Fixes #3924.
* mp_image: dump all mp_colorspace members in verbose loggingwm42016-11-081-1/+1
| | | | | Also extend the default buffer size for formatting this string, because it can get too damn long.
* video: change hw_subfmt meaningwm42016-07-151-2/+1
| | | | | | | | | | | | | | | | | | The hw_subfmt field roughly corresponds to the field AVHWFramesContext.sw_format in ffmpeg. The ffmpeg one is of the type AVPixelFormat (instead of the underlying hardware format), so it's a good idea to switch to this too for preparation. Now the hw_subfmt field is an mp_imgfmt instead of an opaque/API- specific number. VDPAU and Direct3D11 already used mp_imgfmt, but Videotoolbox and VAAPI had to be switched. One somewhat user-visible change is that the verbose log will now always show the hw_subfmt as image format, instead of as nonsensical number. (In the end it would be good if we could switch to AVHWFramesContext completely, but the upstream API is incomplete and doesn't cover Direct3D11 and Videotoolbox.)
* mp_image: split colorimetry metadata into its own structNiklas Haas2016-07-031-5/+1
| | | | | | | | | | | | | | | | | | This has two reasons: 1. I tend to add new fields to this metadata, and every time I've done so I've consistently forgotten to update all of the dozens of places in which this colorimetry metadata might end up getting used. While most usages don't really care about most of the metadata, sometimes the intend was simply to “copy” the colorimetry metadata from one struct to another. With this being inside a substruct, those lines of code can now simply read a.color = b.color without having to care about added or removed fields. 2. It makes the type definitions nicer for upcoming refactors. In going through all of the usages, I also expanded a few where I felt that omitting the “young” fields was a bug.
* vo_opengl: refactor HDR mechanismNiklas Haas2016-05-301-0/+1
| | | | | | | | | | | | | | | | | | | | Instead of doing HDR tone mapping on an ad-hoc basis inside pass_colormanage, the reference peak of an image is now part of the image params (alongside colorspace, gamma, etc.) and tone mapping is done whenever peak_src != peak_dst. To get sensible behavior when mixing HDR and SDR content and displays, target-brightness is a generic filler for "the assumed brightness of SDR content". This gets rid of the weird display_scaled hack, sets the framework for multiple HDR functions with difference reference peaks, and allows us to (in a future commit) autodetect the right source peak from the HDR metadata. (Apart from metadata, the source peak can also be controlled via vf_format. For HDR content this adjusts the overall image brightness, for SDR content it's like simulating a different exposure)
* mp_image: don't reset pixel aspect with mp_image_set_size()wm42016-05-301-1/+1
| | | | | | | | | | | | | | | No reason to do so. See also commit 240ba92b. Since now many mp_images will never have a pixel aspect ratio set, redefine a 0/0 aspect ratio to "undefined" instead invalid. This also brings it more in line with how decoder vs. container aspect ratios are handled. Most callers seem to be fine with the new behavior. mp_image_params_valid() in particular has to be adjusted, or some things stop working due to mp_images not becoming valid after setting size and format.
* mp_image: make mp_image_copy_fields_* privatewm42016-04-151-4/+0
| | | | | Future code should always use mp_image_{to,from}_av_frame(). Everything else is way too messy and fragile.
* mp_image: add mp_image_to_av_frame()wm42016-04-151-0/+1
| | | | | What mp_image_to_av_frame_and_unref() should have been. (The _unref variant is still useful though.)
* mp_image: pass through AVHWFramesContextwm42016-04-151-0/+2
| | | | In both directions.
* mp_image: introduce a hw_subfmt fieldwm42016-04-071-0/+2
| | | | | | | | | | | For hwaccel formats, mp_image will merely point to a hardware surface handle. In these cases, the mp_image_params.imgfmt field describes the format insufficiently, because it mostly only describes the type of the hardware format, not its underlying format. Introduce hw_subfmt to describe the underlying format. It makes sense to use it with most hwaccels, though for now it will be used with the following commit only.
* video: cleanup pts/dts passing between decoder componentswm42016-01-251-0/+2
| | | | | Instead of using semi-public codec_pts/codec_dts fields in struct dec_video, pass them via mp_image fields.
* video: switch from using display aspect to sample aspectwm42015-12-191-1/+5
| | | | | | | | | | | | | | | | MPlayer traditionally always used the display aspect ratio, e.g. 16:9, while FFmpeg uses the sample (aka pixel) aspect ratio. Both have a bunch of advantages and disadvantages. Actually, it seems using sample aspect ratio is generally nicer. The main reason for the change is making mpv closer to how FFmpeg works in order to make life easier. It's also nice that everything uses integer fractions instead of floats now (except --video-aspect option/property). Note that there is at least 1 user-visible change: vf_dsize now does not set the display size, only the display aspect ratio. This is because the image_params d_w/d_h fields did not just set the display aspect, but also the size (except in encoding mode).
* video: replace vf_format outputlevels option with global optionwm42015-09-291-4/+0
| | | | | | | | | | | The vf_format suboption is replaced with --video-output-levels (a global option and property). In particular, the parameter is removed from mp_image_params. The mechanism is moved to the "video equalizer", which also handles common video output customization like brightness and contrast controls. The new code is slightly cleaner, and the top-level option is slightly more user-friendly than as vf_format sub-option.
* video: refactor GPU memcpy usagewm42015-09-251-0/+3
| | | | | | | | | | | | | | | | | Make the GPU memcpy from the dxva2 code generally useful to other parts of the player. We need to check at configure time whether SSE intrinsics work at all. (At least in this form, they won't work on clang, for example. It also won't work on non-x86.) Introduce a mp_image_copy_gpu(), and make the dxva2 code use it. Do some awkward stuff to share the existing code used by mp_image_copy(). I'm hoping that FFmpeg will sooner or later provide a function like this, so we can remove most of this again. (There is a patch, bit it's stuck in limbo since forever.) All this is used by the following commit.
* video: replace our own refcounting with libavutil'swm42015-07-051-2/+9
| | | | | | | | | | | | | | | | | | | | | | mpv had refcounted frames before libav*, so we were not using libavutil's facilities. Change this and drop our own code. Since AVFrames are not actually refcounted, and only the image data they reference, the semantics change a bit. This affects mainly mp_image_pool, which was operating on whole images instead of buffers. While we could work on AVBufferRefs instead (and use AVBufferPool), this doesn't work for use with hardware decoding, which doesn't map cleanly to FFmpeg's reference counting. But it worked out. One weird consequence is that we still need our custom image data allocation function (for normal image data), because AVFrame's uses multiple buffers. There also seems to be a timing-dependent problem with vaapi (the pool appears to be "leaking" surfaces). I don't know if this is a new problem, or whether the code changes just happened to cause it more often. Raising the number of reserved surfaces seemed to fix it, but since it appears to be timing dependent, and I couldn't find anything wrong with the code, I'm just going to assume it's not a new bug.
* mp_image: remove some unused interlacing flagswm42015-04-231-3/+0
| | | | | | MP_IMGFIELD_TOP/MP_IMGFIELD_BOTTOM were completely unused, and MP_IMGFIELD_ORDERED was always set (even though vf_vdpaupp.c strangely checked for the latter).
* Update license headersMarcin Kurczewski2015-04-131-5/+4
| | | | Signed-off-by: wm4 <wm4@nowhere>
* mp_image: remove redundant flags fieldwm42015-04-101-2/+1
| | | | | | | Because gcc (and clang) is a goddamn PITA and unnecessarily warns if the universal initializer for structs is used (like mp_image x = {}) and the first member of the struct is also a struct, move the w/h fields to the top.
* mp_image: remove redundant chroma_x/y_shift fieldswm42015-04-101-2/+0
|
* mp_image: remove redundant plane_w/h fieldswm42015-04-101-4/+2
| | | | Seems relatively painful in this case, but they are morally wrong.
* mp_image: get rid of chroma_width/height fieldswm42015-04-091-2/+0
| | | | | | | They are redundant. They were used by draw_bmp.c only, and only in a special code path that 1. used fixed image formats, and 2. had image sized perfectly aligned to chroma boundaries (so computing the chroma width/height is trivial).
* video: uninline memcpy_pic functions