summaryrefslogtreecommitdiffstats
path: root/video/decode/vd_lavc.c
Commit message (Collapse)AuthorAgeFilesLines
* vd_lavc: log if hw decoding selects a different underlying decoderwm42016-09-301-0/+3
| | | | | Less confusing to see what's going on. I think there were more than one users who got tricked by this, including myself.
* rpi: add --hwdec=rpi-copywm42016-09-301-0/+6
| | | | | | This means it can be used with normal video filters. Might help out with #3604.
* video: handle override video parameters in a better placewm42016-09-201-8/+0
| | | | | | | | This really shouldn't be in vd_lavc.c - move it to dec_video.c, where it also applies aspect overrides. This makes all overrides in one place. The previous commit contains some required changes for resetting the image parameters change detection (i.e. it's not done only on video aspect override changes).
* hwdec_cuda: Rename config variable to be more consistentPhilip Langdale2016-09-161-2/+2
| | | | | | 'cuda-gl' isn't right - you can turn this on without any GL and get some non-zero benefit (with the cuda-copy hwaccel). So 'cuda-hwaccel' seems more consistent with everything else.
* hwdec_cuda: Add trivial cuda-copy wrapperPhilip Langdale2016-09-111-0/+9
| | | | | | | | | The cuvid decoder already knows how to copy back to system memory if NV12 frames are requested, and this will happen if the decoder is used without the hwdec. For convenience, let's add a wrapper hwdec so people don't have to explicitly pick the cuvid decoder if they want this behaviour.
* hwdec/opengl: Add support for CUDA and cuvid/NvDecodePhilip Langdale2016-09-081-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Nvidia's "NvDecode" API (up until recently called "cuvid" is a cross platform, but nvidia proprietary API that exposes their hardware video decoding capabilities. It is analogous to their DXVA or VDPAU support on Windows or Linux but without using platform specific API calls. As a rule, you'd rather use DXVA or VDPAU as these are more mature and well supported APIs, but on Linux, VDPAU is falling behind the hardware capabilities, and there's no sign that nvidia are making the investments to update it. Most concretely, this means that there is no VP8/9 or HEVC Main10 support in VDPAU. On the other hand, NvDecode does export vp8/9 and partial support for HEVC Main10 (more on that below). ffmpeg already has support in the form of the "cuvid" family of decoders. Due to the design of the API, it is best exposed as a full decoder rather than an hwaccel. As such, there are decoders like h264_cuvid, hevc_cuvid, etc. These decoders support two output paths today - in both cases, NV12 frames are returned, either in CUDA device memory or regular system memory. In the case of the system memory path, the decoders can be used as-is in mpv today with a command line like: mpv --vd=lavc:h264_cuvid foobar.mp4 Doing this will take advantage of hardware decoding, but the cost of the memcpy to system memory adds up, especially for high resolution video (4K etc). To avoid that, we need an hwdec that takes advantage of CUDA's OpenGL interop to copy from device memory into OpenGL textures. That is what this change implements. The process is relatively simple as only basic device context aquisition needs to be done by us - the CUDA buffer pool is managed by the decoder - thankfully. The hwdec looks a bit like the vdpau interop one - the hwdec maintains a single set of plane textures and each output frame is repeatedly mapped into these textures to pass on. The frames are always in NV12 format, at least until 10bit output supports emerges. The only slightly interesting part of the copying process is that CUDA works by associating PBOs, so we need to define these for each of the textures. TODO Items: * I need to add a download_image function for screenshots. This would do the same copy to system memory that the decoder's system memory output does. * There are items to investigate on the ffmpeg side. There appears to be a problem with timestamps for some content. Final note: I mentioned HEVC Main10. While there is no 10bit output support, NvDecode can return dithered 8bit NV12 so you can take advantage of the hardware acceleration. This particular mode requires compiling ffmpeg with a modified header (or possibly the CUDA 8 RC) and is not upstream in ffmpeg yet. Usage: You will need to specify vo=opengl and hwdec=cuda. Note that hwdec=auto will probably not work as it will try to use vdpau first. mpv --hwdec=cuda --vo=opengl foobar.mp4 If you want to use filters that require frames in system memory, just use the decoder directly without the hwdec, as documented above.
* vd_lavc: always force milliseconds for MMALwm42016-08-291-0/+5
| | | | | This libavcodec wrapper should rescale the API timestamps to whatever it internally needs, but it doesn't yet. So restore this code.
* vd_lavc, ad_lavc: set pkt_timebase, not time_basewm42016-08-291-1/+4
| | | | | | | | | These are different AVCodecContext fields. pkt_timebase is the correct one for identifying the unit of packet/frame timestamps when decoding, while time_base is for encoding. Some decoders also overwrite the time_base field with some unrelated codec metadata. pkt_timebase does not exist in Libav, so an #if is required.
* vd_lavc: minor simplificationwm42016-08-231-3/+1
| | | | The timebase is now always valid.
* vd_lavc: remove unnecessary initializationwm42016-08-191-1/+0
| | | | This is already the default value.
* video/audio: always provide "proper" timestamps to libavcodecwm42016-08-191-4/+1
| | | | | | | | | | | | | | | | | | | Instead of passing through double float timestamps opaquely, pass real timestamps. Do so by always setting a valid timebase on the AVCodecContext for audio and video decoding. Specifically try not to round timestamps to a too coarse timebase, which could round off small adjustments to timestamps (such as for start time rebasing or demux_timeline). If the timebase is considered too coarse, make it finer. This gets rid of the need to do this specifically for some hardware decoding wrapper. The old method of passing through double timestamps was also a bit questionable. While libavcodec is not supposed to interpret timestamps at all if no timebase is provided, it was needlessly tricky. Also, it actually does compare them with AV_NOPTS_VALUE. This change will probably also reduce confusion in the future.
* videotoolbox: add --hwdec=videotoolbox-copy for h/w accelerated decoding ↵Aman Gupta2016-07-151-0/+2
| | | | with video filters
* vd_lavc: expose mastering display side data reference peakNiklas Haas2016-07-031-0/+25
| | | | | | | | | | | | | | | | | | | This greatly improves the result when decoding typical (ST.2084) HDR content, since the job of tone mapping gets significantly easier when you're only mapping from 1000 to 250, rather than 10000 to 250. The difference is so drastic that we can now even reasonably use `hdr-tone-mapping=linear` and get a very perceptually uniform result that is only slightly darker than normal. (To compensate for the extra dynamic range) Due to weird implementation details, this only seems to be present on keyframes (or something like that), so we have to cache the last seen value for the frames in between. Also, in some files the metadata is just completely broken / nonsensical, so I decided to apply a simple heuristic to detect completely broken metadata.
* mp_image: split colorimetry metadata into its own structNiklas Haas2016-07-031-4/+6
| | | | | | | | | | | | | | | | | | This has two reasons: 1. I tend to add new fields to this metadata, and every time I've done so I've consistently forgotten to update all of the dozens of places in which this colorimetry metadata might end up getting used. While most usages don't really care about most of the metadata, sometimes the intend was simply to “copy” the colorimetry metadata from one struct to another. With this being inside a substruct, those lines of code can now simply read a.color = b.color without having to care about added or removed fields. 2. It makes the type definitions nicer for upcoming refactors. In going through all of the usages, I also expanded a few where I felt that omitting the “young” fields was a bug.
* vd_lavc: hide structs behind platform flagsBen Boeckel2016-07-011-0/+4
| | | | Otherwise, warnings about them being unused appear.
* video: add --hwdec=auto-copy modewm42016-05-111-3/+9
| | | | | | | | This uses the normal autoprobing rules like "auto", but rejects anything that isn't flagged as copying data back to system memory. The chunk in command.c was dead code, so remove it instead of updating it.
* build: merge d3d11va and dxva2 hwaccel checkswm42016-05-111-5/+1
| | | | | | We don't have any reason to disable either. Both are loaded dynamically at runtime anyway. There is also no reason why dxva2 would disappear from libavcodec any time soon.
* video: refactor how VO exports hwdec device handleswm42016-05-091-11/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | The main change is with video/hwdec.h. mp_hwdec_info is made opaque (and renamed to mp_hwdec_devices). Its accessors are mainly thread-safe (or documented where not), which makes the whole thing saner and cleaner. In particular, thread-safety rules become less subtle and more obvious. The new internal API makes it easier to support multiple OpenGL interop backends. (Although this is not done yet, and it's not clear whether it ever will.) This also removes all the API-specific fields from mp_hwdec_ctx and replaces them with a "ctx" field. For d3d in particular, we drop the mp_d3d_ctx struct completely, and pass the interfaces directly. Remove the emulation checks from vaapi.c and vdpau.c; they are pointless, and the checks that matter are done on the VO layer. The d3d hardware decoders might slightly change behavior: dxva2-copy will not use the VO device anymore if the VO supports proper interop. This pretty much assumes that any in such cases the VO will not use any form of exclusive mode, which makes using the VO device in copy mode unnecessary. This is a big refactor. Some things may be untested and could be broken.
* vd_lavc: adjust D3D11VA autoprobe orderwm42016-04-271-1/+3
| | | | | | We want to prefer d3d11va over dxva2 anything. But since dxva2 copyback is more efficient than d3d11va's currently, d3d11va-copy should come last.
* vo_opengl: D3D11VA + ANGLE interopwm42016-04-271-0/+2
| | | | | | | | | | | | | | | | | | | This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
* vd_lavc: hack against videotoolbox crash on failurewm42016-04-261-1/+3
| | | | | I guess this won't ever be fixed properly in FFmpeg. Too hairy, and the alternative (using VideoToolbox as "full decoder") is too attractive.
* vd_lavc: simplify some unneeded ifdefferywm42016-04-251-8/+2
| | | | These were for ancient libavcodec versions.
* vd_lavc: make image_format hwdec field optionalwm42016-04-251-26/+24
| | | | | | | For Mediacodec in particular we don't care about the format. It can just decode to whatever it wants. The only case we would care about is it not returning an opaque format if we don't have proper interop, but libavcodec always returns non-opaque formats by default.
* vd_lavc: simplify RPI and Mediacodec wrapperswm42016-04-251-2/+12
| | | | | | | | | | | | Use the recently added lavc_suffix mechanism to select the wrapper decoder. With all hwdec callbacks being optional, and RPI/Mediacodec having only dummy callbacks, all the callbacks can be removed as well. The result is that the vd_lavc_hwdec struct for both of them is tiny. It's better to move them to vd_lavc.c directly, because they are so trivial and small.
* vd_lavc: make all hwdec callbacks optionalwm42016-04-251-2/+2
|
* vd_lavc: set AVCodecContext.time_base to forced time basewm42016-04-251-0/+3
| | | | | This is a bit sketchy, as there isn't a truly standard way to communicate the timebase.
* vd_lavc: better hwdec wrapper decoder selectionwm42016-04-251-1/+60
| | | | | | | | | | | | | | | | This is intended for cases when --hwdec needs to override the decoder implementation in use, like for example on the RPI. It does two things: 1. Allow the hwdec to indicate a decoder suffix. libavcodec by convention adds a suffix to all wrapper decoders, and here we start relying on it. While not necessarily the best idea, it's the only thing we got. libavcodec's hwaccel list is useless, because it only has the codec ID, not the associated decoder's name. 2. Make --hwdec=auto work properly. It shouldn't fail anymore, and hwdec probing should reliably work, even if a different decoder is selected with --vd. The semantics of --hwdec should dictate that it overrides the default decoder.
* vd_lavc: allow process_image() to return NULLwm42016-04-251-1/+1
| | | | In case of errors or whatever.
* vd_lavc: fix hwdec fallback if hwdec pre-initialization failswm42016-04-221-7/+7
| | | | Damn.
* vd_lavc: let hardware decoder request delaying frames explicitlywm42016-04-071-4/+1
| | | | | | | | | | Until now, the presence of the process_image() callback was used to set a delay queue with a hardcoded size. Change this to a vd_lavc_hwdec field instead, so the decoder can explicitly set this if it's really needed. Do this so process_image() can be used in the VideoToolbox glue code for something entirely unrelated.
* vd_lavc: fix codec vs. decoder confusionwm42016-04-071-13/+14
| | | | | | | | | | Some functions which expected a codec name (i.e. the name of the video format itself) were passed a decoder name. Most "native" libavcodec decoders have the same name as the codec, so this was never an issue. This should mean that e.g. using "--vd=lavc:h264_mmal --hwdec=mmal" should now actually enable native surface mode (instead of doing copy- back).
* demux_lavf, ad_lavc, ad_spdif, vd_lavc: handle FFmpeg codecpar API changewm42016-03-311-2/+1
| | | | | | | | | AVFormatContext.codec is deprecated now, and you're supposed to use AVFormatContext.codecpar instead. Handle this for all of the normal playback code. Encoding mode isn't touched.
* vd_lavc: add d3d11va hwdecKevin Mitchell2016-03-301-0/+4
| | | | | | This commit adds the d3d11va-copy hwdec mode using the ffmpeg d3d11va api. Functions in common with dxva2 are handled in a separate decode/d3d.c file. A future commit will rewrite decode/dxva2.c to share this code.
* Add a mediacodec decoder hwdec wrapperJan Ekström2016-03-251-0/+4
| | | | | Does the same thing as the rpi one - makes fallback possible by pretending that h264_mediacodec is a hwdec.
* vd_lavc: mediacodec also needs special timebase loveJan Ekström2016-03-251-1/+1
|
* ad_lavc, vd_lavc: support new Libav decoding APIwm42016-03-241-3/+18
| | | | For now only found in Libav.
* vd_lavc: slightly improve codec_timebase hackwm42016-03-241-4/+3
| | | | | | The mp_set_av_packet()/mp_pts_from_av() functions check whether the timebase is set at all (i.e. AVRational.num!=0), so there's no need to fiddle with pointers.
* vd_lavc: don't get into an endless loop if decoding failswm42016-03-181-0/+2
| | | | Regression since commit 6640b22a.
* av_common: explicitly exclude _vdpau deccoders from enumerationwm42016-03-021-11/+0
| | | | | | | | | | Completely pointless abominations that FFmpeg refuses to remove. They are ancient, long deprecated API which we can't use anymore. They confused users as well. Pretend that they don't exist. Due to the way --vd works, they can't even be forced anymore. The older hack which explicitly rejects these can be dropped as well.
* vd_lavc: simplify hwdec pixfmt checkwm42016-02-201-1/+1
| | | | | Instead of checking whether the format is a hwaccel format, check whether it's the exact format we've requested for hardware decoding.
* video: allow the decoder to consume packets partiallywm42016-02-191-0/+5
| | | | | | | | | | | | | | | | | | | | This is in preparation for a hypothetical API change in libavcodec, which would allow the decoder to return multiple video frames before accepting a new input packet. In theory, the body of the if() added to vd_lavc.c could be replaced with this code: packet->buffer += ret; packet->len -= ret; but currently this is not needed, as libavformat already outputs one frame per packet. Also, using libavcodec this way could lead to a "deadlock" if the decoder refuses to consume e.g. garbage padding, so enabling this now would introduce bugs. (Adding this now for easier testing, and for symmetry with the audio code.)
* dxva2: add interop (non-copyback) hwdec_typeKevin Mitchell2016-02-171-0/+2
| | | | | This always falls back to software decoding right now. VO support will be added in future commits.
* audio/video: expose codec info as separate fieldwm42016-02-151-4/+3
| | | | | Preparation for the timeline rewrite. The codec will be able to change, the stream header not.
* video: approximate AVI timestamps via DTS handlingwm42016-02-111-6/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Until now (and in mplayer traditionally), avi timestamps were handled with a timestamp FIFO. AVI timestamps are essentially just strictly increasing frame numbers and are not reordered like normal timestamps. Limiting the FIFO is required because frames can be dropped. To make it worse, frame dropping can't be distinguished from the decoder not returning output due to increasing the buffering required for B-frames. ("Measuring" the buffering at playback start seems like an interesting idea, but won't work as the buffering could be increased mid-playback.) Another problem are skipped frames (packets with data, but which do not contain a video frame). Besides dropped and skipped frames, there is the problem that we can't always know the delay. External decoders like MMAL are not going to tell us. (And later perhaps others, like direct VideoToolbox usage.) In general, this works not-well enough that I prefer the solution of passing through AVI timestamps as DTS. This is slightly incorrect, because most decoders treat DTS as mpeg-style timestamps, which already include a b-frame delay, and thus will be shifted by a few frames. This means there will be a problem with A/V sync in some situations. Note that the FFmpeg AVI demuxer shifts timestamps by an additional amount (which increases after the first seek!?!?), which makes the situation worse. It works well with VfW-muxed Matroska files, though. On RPI, the first X timestamps are broken until the MMAL decoder "locks on".
* vd_lavc: fix use after free in some hwdecsKevin Mitchell2016-02-061-8/+3
| | | | | | | | | | | | | | fd339e3f53996efd2dae9525990da433d1e1bf89 introduced a regression that caused segfault while uniniting dxva2 decoder (and possibly vdpau too). The problem was that it freed the avctx earlier, before calling the backend-specific uninit which referenced it. Revert some of the changes of that commit, and avoid calling flush by checking whether the codec is open instead. (Based on a PR by Kevin Mitchell.) Signed-off-by: wm4 <wm4@nowhere>
* vd_lavc: avoid calling flush on an unopened AVCodecContextwm42016-02-051-6/+9
| | | | | | | It can be "dangerous". In particular, the decoder might have failed to initialize, and is now in a broken state. avcodec_flush_buffers() is not expected to be called in this state, and could trigger undefined behavior.
* vd_lavc: remove redundant best_csp fieldwm42016-02-031-14/+3
| | | | And some other simplifications.
* vd_lavc: force microsecond timestamps on RPIwm42016-02-031-3/+8
| | | | | | | | | | Avoids "problems". In particular, it makes MMAL output a NOPTS timestamp if the input timestamp was NOPTS. Don't do it for other decoders. Ideally, we will at some point in the future switch to integer fractions for timestamps at least up until the filter layer. But this would be a larger change, and for now I'd prefer keeping the not-rounded demuxer timestamps (if we have them).
* vd_lavc: release surfaces before destroying decoderwm42016-01-301-4/+3
| | | | | | | | | | Commit b53cb8de added a delay queue for decoded frames. This is supposed to be used with copy-back decoders like dxva2-copy and vaapi-copy. Surfaces returned by them can't be referenced after uninitializing the decoders, so they have to be released before destroying the decoder. Move the flush_all() call above decoder uninit accordingly. Also move the destruction of the AVFrame used for decoding (just for being defensive - normally it doesn't hold any reference).
* vd_lavc: allow switching between hw/sw decoding any timewm42016-01-291-21/+33
| | | | | | | We just need to provide an entrypoint for it, and move the main init code to a separate function. This gets rid of the messy video chain full reinit in command.c, which completely destroyed and recreated the video state for the purpose of mid-stream hw/sw switching.
* vd_lavc: simplify an aspect of hwdec fallbackwm42016-01-291-9/+4
| | | | | | Don't give the "software_fallback_decoder" field special meaning. Alwass set it, and rename it to "decoder". Whether hw decoding is used is determined by the "hwdec" field already.
* vd_lavc: delay images before reading them backwm42016-01-251-7/+42
| | | | Facilitates hardware pipelining in particular with nvidia/dxva.
* video: cleanup pts/dts passing between decoder componentswm42016-01-251