summaryrefslogtreecommitdiffstats
path: root/video/decode
Commit message (Collapse)AuthorAgeFilesLines
* video, audio: always read all frames before getting next packetwm42018-01-012-2/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The old code tried to make sure at all times to try to read a new packet. Only once that was read, it tried to retrieve new video or audio frames the decoder might already have decoded. Change this to strictly read frames from the decoder until it signals that it wants a new packet, and only then read and feed a new packet. This is in theory nicer, follows the libavcodec recommended data flow, and and reduces the minimum latency by 1 frame. This merely requires switching the order in which those calls are done. Normally, the decoder will return only 1 frame until a new packet is required. If we would just feed it 1 packet, return DATA_AGAIN, and wait until the next frame is decoded, we would run the playloop 1 time too often for no reason (which is fine but might have some overhead). To avoid this, try to read a frame again after possibly feeding a packet. For this reason, move the feed/read code to its own functions each, instead of merely moving the code. The audio and video code for this particular thing is basically duplicated. The idea is to unify them one day, so make the change to both. (Doing this for video is the real motivation for this change, see below.) The video code change is slightly more complicated, because we have to care about the framedrop counting (which is just a heuristic, but for now considered better than nothing, and possibly considered required to warn the user of framedrops happening - maybe). Apparently this change helps with stalling streams on Android with the mediacodec wrapper and mpeg2 decoder implementations which deinterlace on decoding (and return 2 frames per packet). Based on an idea and observations by tmm1.
* demux_mkv: add hack to pass along x264 version to decoderwm42017-12-281-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | This fixes when resuming certain broken h264 files encoded by x264. See FFmpeg commit 840b41b2a643fc8f0617c0370125a19c02c6b586 about the x264 bug itself. Normally, the unregistered user data SEI (that contains the x264 version string) is informational only. But libavcodec uses it to workaround a x264 bug, which was recently fixed in both libavcodec and x264. The fact that both encoder and decoder were buggy is the reason that it was not found earlier, and there are apparently a lot of files around created by the broken decoder. If libavcodec sees the SEI, this bug can be worked around by using the old behavior. If you resume a file with mpv (i.e. seeking when the file loads), libavcodec never sees the first video packet. Consequently it has to assume the file is not broken, and never applies the workaround, resulting in garbage being played. Fix this by always feeding the first video packet to the decoder on init, and then flushing the codec (to avoid that an unwanted image is output). Flushing the codec does not remove info such as the x264 version. We also abuse the fact that the first avcodec_send_packet() always pushes the frame into the decoder (so we don't have to trigger the decoder by requsting an output frame).
* vd_lavc: add an option to explicitly workaround x264 4:4:4 bugwm42017-12-281-0/+5
| | | | | | Technically, the user could just use --vd-lavc-o with the same result. But I find it better to make this an explicit option, so we can document the ups and downs, and also avoid setting it for non-h264.
* vd_lavc: fix crash with RPI hwdecwm42017-12-281-1/+2
| | | | | | If you use vo_rpi, this could crash, because hwdec_devs is NULL. Untested. Fixes #5301.
* options: drop some previously deprecated optionswm42017-12-251-4/+0
| | | | | | | | A release has been made, so drop options deprecated for that release. Also drop some options which have been deprecated a much longer time before. Also fix a typo in client-api-changes.rst.
* Restore Libav supportwm42017-12-211-0/+2
| | | | | | | | | | | Libav has been broken due to the hwdec changes. This was always a temporary situation (depended on pending patches to be merged), although it took a bit longer. This also restores the travis config. One code change is needed in vd_lavc.c, because it checks the AV_PIX_FMT for videotoolbox (as opposed to the mpv format identifier), which is not available in Libav. Add an ifdef; the affected code is for a deprecated option anyway.
* msg: reinterpret a bunch of message levelsNiklas Haas2017-12-151-4/+4
| | | | | | | | | | | | | | | | | | | | | | I've decided that MP_TRACE means “noisy spam per frame”, whereas MP_DBG just means “more verbose debugging messages than MSGL_V”. Basically, MSGL_DBG shouldn't create spam per frame like it currently does, and MSGL_V should make sense to the end-user and provide mostly additional informational output. MP_DBG is basically what I want to make the new default for --log-file, so the cut-off point for MP_DBG is if we probably want to know if for debugging purposes but the user most likely doesn't care about on the terminal. Also, the debug callbacks for libass and ffmpeg got bumped in their verbosity levels slightly, because being external components they're a bit less relevant to mpv debugging, and a bit too over-eager in what they consider to be relevant information. I exclusively used the "try it on my machine and remove messages from MSGL_* until it does what I want it to" approach of refactoring, so YMMV.
* vd_lavc: rename --hwdec=rpi to --hwdec=mmalwm42017-12-151-4/+0
| | | | | | Annoying exception that makes no sense to keep. Normally, users or client applications will either use --hwdec=auto, or not set the option at all, which both leads to the expected result.
* vd_lavc: use libavcodec metadata for hardware decoder wrapperswm42017-12-151-25/+8
| | | | | This removes the need to keep an explicit list and to attempt to parse codec names. Needs latest FFmpeg git.
* vd_lavc: always load VO interops with non-copy hw decoderswm42017-12-111-0/+4
| | | | | | | | For METHOD_INTERNAL hwdecs (non-copy cases), make sure the VO interops are always loaded, because those decoders will output hardware pixel formats, which will need special support in vo_gpu. Otherwise, initialization will fail, complaining that it can't convert the output format to something the VO supports.
* vd_lavc: add rkmpp to the hwdec_wrappers array.LongChair2017-12-101-0/+1
| | | | Allows to get the hwdec picked up properly by mpv on rockchip devices
* vd_lavc: don't request native pixfmt with -copy and METHOD_INTERNALwm42017-12-021-0/+1
| | | | | | | | | If the codec uses AV_CODEC_HW_CONFIG_METHOD_INTERNAL, and we're using the -copy method, then don't request the native pix_fmt. It might not have a AVFrame.hw_frames_ctx set, and we couldn't read back at all. On top of that, most of those decoders probably don't provide read-back when using such opaque formats anyway, while providing separate decoding modes to decode to RAM.
* vd_lavc, vdpau, vaapi: restore emulated API avoidancewm42017-12-021-0/+11
| | | | | | This code is for trying to avoid using an emulation layer when using auto probing, so that we end up using the actual API the drivers provide. It was destroyed in the recent refactor.
* vd_lavc: simpler way to check for opque hw framewm42017-12-021-3/+1
| | | | The ->fmt shit is something I'd like to phase out.
* vd_lavc: sort -copy hwdec modes to end of listwm42017-12-021-6/+12
| | | | | | | | | | | | | | | Otherwise, if e.g. "nvdec" didn't work, but "nvdec-copy" did, it would never try "vdpau", which is actually the next non-copy mode on the autprobe list. It's really expected that it selects "vdpau". Fix this by sorting the -copy modes to the end of the final hwdec list. But we still don't want preferred -copy modes like "nvdec-copy" to be sorted after fragile non-preferred modes like "cuda", and --hwdec=auto should prefer "nvdec-copy" over it, so make sure the copying mode does not get precedence over preferred vs. non-preferred mode. Also simplify the existing auto_pos sorting condition, and fix the fallback sort order (although that doesn't matter too much).
* vd_lavc: allow forcing single implementations with --hwdecwm42017-12-021-2/+3
|
* vd_lavc: slightly simplifywm42017-12-021-34/+28
| | | | | | Factor the somewhat-duplicated code into an append function. Also fix setting the copying flag in one of the cases. This also ensures some uniformity.
* vd_lavc: coding stylewm42017-12-021-1/+1
|
* vd_lavc: fix dumb nonsensewm42017-12-011-1/+4
|
* vd_lavc, mp_image: remove weird mpv specific palette constantwm42017-12-011-4/+0
| | | | | Was for times when we were trying to be less dependent on libav* I guess.
* vd_lavc: merge redundant headerwm42017-12-012-94/+77
| | | | This is not needed anymore.
* vd_lavc: rewrite how --hwdec is handledwm42017-12-012-479/+323
| | | | | | | | | | | | | | | | Change it from explicit metadata about every hwaccel method to trying to get it from libavcodec. As shown by add_all_hwdec_methods(), this is a quite bumpy road, and a bit worse than expected. This will probably cause a bunch of regressions. In particular I didn't check all the strange decoder wrappers, which all cause some sort of special cases each. You're volunteering for beta testing by using this commit. One interesting thing is that we completely get rid of mp_hwdec_ctx in vd_lavc.c, and that HWDEC_* mostly goes away (some filters still use it, and the VO hwdec interops still have a lot of code to set it up, so it's not going away completely for now).
* vd_lavc: drop mediacodec direct rendering support temporarilywm42017-12-012-69/+8
| | | | | | | | | | | | The libavcodec mediacodec support does not conform to the new hwaccel APIs yet. It has been agreed uppon that this glue code can be deleted for now, and support for it will be restored at a later point. Readding would require that it supports the AVCodecContext.hw_device_ctx API. The hw_device_ctx would then contain the surface ID. vo_mediacodec_embed would actually perform the task of creating vo.hwdec_devs and adding a mp_hwdec_ctx, whose av_device_ref is a AVHWDeviceContext containing the android surface.
* video: move d3d.c out of decode sub directorywm42017-12-012-329/+0
| | | | | | It makes more sense to have it in the general video directory (along with vdpau.c and vaapi.c), since the decoder source files don't even access it anymore.
* vd_lavc: delete hw_d3d11va.c/hw_dxva2.c and merge leftoverswm42017-12-015-117/+39
| | | | | | Like with all hwaccels, there's little that is actually specific to decoding (which has been moved away anyway), and what is left are declarations (which will also go away soon).
* hw_dxva2: move dxva2 code to d3d.cwm42017-12-012-117/+126
| | | | This source file will disappear, so just collect the leftovers in d3d.c.
* video: refactor hw device creation for hwdec copy modeswm42017-12-015-87/+71
| | | | | | | | | | Lots of shit code for nothing. We probably could just use libavutil's code for all of this. But for now go with this, since it tends to prevent stupid terminal messages during probing (libavutil has no mechanism to selectively suppress errors specifically during probing). Ignores the "emulated" API flag (for avoiding vaapi/vdpau wrappers), but it doesn't matter that much for -copy anyway.
* vd_lavc: move entrypoint for hwframes_refinewm42017-12-016-11/+7
| | | | | | | The idea is to get rid of vd_lavc_hwdec, so special functionality like this has to go somewhere else. At this point, hwframes_refine is only needed for d3d11, and it doesn't do much, so for now the new callback has no context. In can be made more fancy if really needed.
* vd_lavc: remove process_image callbackwm42017-12-012-9/+3
| | | | Now unused.
* d3d11: move code for d3d11eglrgb hackwm42017-12-012-10/+3
|
* vo_gpu: make it possible to load multiple hwdec interop driverswm42017-12-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make the VO<->decoder interface capable of supporting multiple hwdec APIs at once. The main gain is that this simplifies autoprobing a lot. Before this change, it could happen that the VO loaded the "wrong" hwdec API, and the decoder was stuck with the choice (breaking hw decoding). With the change applied, the VO simply loads all available APIs, so autoprobing trickery is left entirely to the decoder. In the past, we were quite careful about not accidentally loading the wrong interop drivers. This was in part to make sure autoprobing works, but also because libva had this obnoxious bug of dumping garbage to stderr when using the API. libva was fixed, so this is not a problem anymore. The --opengl-hwdec-interop option is changed in various ways (again...), and renamed to --gpu-hwdec-interop. It does not have much use anymore, other than debugging. It's notable that the order in the hwdec interop array ra_hwdec_drivers[] still matters if multiple drivers support the same image formats, so the option can explicitly force one, if that should ever be necessary, or more likely, for debugging. One example are the ra_hwdec_d3d11egl and ra_hwdec_d3d11eglrgb drivers, which both support d3d11 input. vo_gpu now always loads the interop lazily by default, but when it does, it loads them all. vo_opengl_cb now always loads them when the GL context handle is initialized. I don't expect that this causes any problems. It's now possible to do things like changing between vdpau and nvdec decoding at runtime. This is also preparation for cleaning up vd_lavc.c hwdec autoprobing. It's another reason why hwdec_devices_request_all() does not take a hwdec type anymore.
* vd_lavc: prefer nvdec over vdpau with --hwdec=autowm42017-11-301-11/+13
| | | | | | | | | | | | | | | | nvdec aka cuvid aka cuda should work much better than vdpau, and support newer codecs (such as vp9), and more advanced surface formats (like 10 bit). This requires moving the d3d hwaccels in the autoprobe order, since on Windows, d3d decoding should be preferred over nvidia proprietary stuff. Users of older drivers will need to force --hwdec=vdpau, since it could happen that the vo_gpu cuda hwdec interop loads (so the vdpau interop is not loaded), but the hwdec itself doesn't work. I expect this does not break AMD (which still needs vdpau for vo_gpu interop, until libva is fixed so it can fully support AMD).
* vd_lavc: restore --hwdec-image-format and d3d11 opaque modewm42017-11-022-1/+6
| | | | | When the ifdeffery for the frame_params API was added, the new code accidentally didn't include this.
* vd_lavc: clean out more hwdec legacy codewm42017-10-313-39/+5
| | | | | | | | All this code used to be required by the old variants of the libavcodec hw decoding APIs. Almost all of that is gone, although the mediacodec API unfortunately still pulls in some old stuff (but not all of it). (mediacodec build/functionality is untested, but should work.)
* vd_lavc: remove more dead legacy codewm42017-10-314-108/+2
| | | | | | | | | | | | | | All of this was dead code and completely unused. get_buffer2_hwdec() is the biggest chunk. One unfortunate thing about it is that, while it was active, it could perform a software fallback much faster, because it didn't have to wait until a full frame is decoded (it actually decoded a full frame, but the current code has to decode many more frames due to the codec delay, because the current code waits until the API returns a decoded frame.) We should probably restore the latter, although since it's an optional optimization, and the current behavior doesn't change with the removal of this code, don't actually do anything about it.
* videotoolbox: use generic code for dummy hwdevice initwm42017-10-312-75/+19
| | | | | And move the remaining code (just 2 struct constant definitions) to vd_lavc.c.
* vd_lavc: remove dead legacy codewm42017-10-312-22/+0
|
* d3d: remove some legacy codewm42017-10-311-42/+0
| | | | See #5062.
* vd_lavc: makre sure required headers are included early enoughwm42017-10-312-1/+1
| | | | Should fix #5062.
* vd_lavc: move display mastering data stuff to mp_imagewm42017-10-303-42/+3
| | | | | | | | | | | This is where it should be. It only wasn't because of an old libavcodec bug, that returned the side data only on every IDR. This required some sort of caching, which is now dropped. (mp_image wouldn't have been able to do this kind of caching, because this code is stateless.) We don't support these old libavcodec versions anymore, which is why this is not needed anymore. Also move initialization of rotation/stereo stuff to dec_video.c.
* Bump libav* API usewm42017-10-301-154/+6
| | | | (Not tested on Windows and OSX.)
* vd_lavc: make --hwdec=nvdec-copy actually workwm42017-10-302-3/+32
| | | | | | | | | | | | This simply didn't work. Unlike cuda-copy, this is a true hwaccel, and obviously we need to provide it a device. Implement this in a relatively generic way, which can probably reused directly by videotoolbox (not doing this yet because it would require testing on OSX). Like with cuda-copy, --cuda-decode-device is ignored. We might be able to provide a more general way to select devices at some later point.
* vd_lavc: remove need for duplicated cuda GL interop backendwm42017-10-302-2/+6
| | | | | | | This is just a dumb consequence of HWDEC_ types somehow being part of both decoder and VO. Obviously, the VO should only care about supporting specific hardware surface types or providing specific device types, but until they are separated, stupid unintuitive mismatches will occur.
* Get rid of deprecated AVFrame accessorswm42017-10-301-1/+1
| | | | | | Fist we were required to use them for ABI compat. reasons (and other BS), now they're deprecated and we're supposed to access them directly again.
* vd_lavc: add support for nvdec hwaccelwm42017-10-281-0/+15
| | | | | | | | See manpage additions. (In ffmpeg-mpv and Libav, this is still called "cuvid". Libav won't work yet, because it has no frame params support yet, but this could get fixed soon.)
* vd_lavc: use avcodec_fill_hw_frames_parameters() APIwm42017-10-272-2/+112
| | | | | | | | This removes the need for codec- and API-specific knowledge in the libavcodec hardware acceleration API user. For mpv, this removes the need for vd_lavc_hwdec.pixfmt_map and a few other things. (For now, we still keep the "old" parts for the sake of supporting older Libav, and FFgarbage.)
* vd_lavc: more aggressive frame dropping for intra only codecswm42017-10-262-5/+15
| | | | | | | | Should speed up seeks. (Unfortunately it's useless for backstepping. Backstepping is like precise seeking, except we're unable to drop frames, as we can't know the previous frame if we drop it.)
* demux: get rid of demux_packet.new_segment fieldwm42017-10-241-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | The new_segment field was used to track the decoder data flow handler of timeline boundaries, which are used for ordered chapters etc. (anything that sets demuxer_desc.load_timeline). This broke seeking with the demuxer cache enabled. The demuxer is expected to set the new_segment field after every seek or segment boundary switch, so the cached packets basically contained incorrect values for this, and the decoders were not initialized correctly. Fix this by getting rid of the flag completely. Let the decoders instead compare the segment information by content, which is hopefully enough. (In theory, two segments with same information could perhaps appear in broken-ish corner cases, or in an attempt to simulate looping, and such. I preferred the simple solution over others, such as generating unique and stable segment IDs.) We still add a "segmented" field to make it explicit whether segments are used, instead of doing something silly like testing arbitrary other segment fields for validity. Cached seeking with timeline stuff is still slightly broken even with this commit: the seek logic is not aware of the overlap that segments can have, and the timestamp clamping that needs to be performed in theory to account for the fact that a packet might contain a frame that is always clipped off by segment handling. This can be fixed later.
* Add DRM_PRIME Format Handling and Display for RockChip MPP decodersLionel CHAZALLON2017-10-231-0/+7
| | | | | | | | | | | This commit allows to use the AV_PIX_FMT_DRM_PRIME newly introduced format in ffmpeg that allows decoders to provide an AVDRMFrameDescriptor struct. That struct holds dmabuf fds and information allowing zerocopy rendering using KMS / DRM Atomic. This has been tested on RockChip ROCK64 device.
* video: make previously added hwdec params mechanism more genericwm42017-10-161-4/+5
| | | | | | | | | | | | The mechanism introduced in b135af6842bf assumed AVHWFramesContext would be enough. Apparently it's not - the intended use with Rockchip (not Rokchip btw.) requires accessing actual frame data in order to access the AVDRMFrameDescriptor struct. Just pass the entire mp_image to the new function. This is more flexible, although it slightly worries me that it will be less reusable for things which require setting up mp_image_params before any real frames are processed (such as filters).
* video: properly pass through ICC datawm42017-10-161-6/+0
| | | | | | | | | | | | | | | | | | The same should happen with any other side data that matters to mpv, otherwise filters will drop it. (No, don't try to argue that mpv should use AVFrame. That won't work.) ffmpeg_garbage() is copy&paste from frame_new_side_data() in FFmpeg (roughly feed201849b8f91), because it's not public API. The name reflects my opinion about FFmpeg's API. In mp_image_to_av_frame(), change the too-fragile *new_ref = (struct mp_image){0}; into explicitly zeroing out the fields that are "transferred" to the created AVFrame.
* video: add mp_image_params.hw_flags and add an examplewm42017-10-161-1/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | It seems this will be useful for Rokchip DRM hwcontext integration. DRM hwcontexts have additional internal structure which can be different depending on the decoder, and which is not part of the generic hwcontext API. Rockchip has 1 layer, which EGL interop happens to translate to a RGB texture, while VAAPI (mapped as DRM hwcontext) will use multiple layers. Both will use sw_format=nv12, and thus are indistinguishable on the mp_image_params level. But this is needed to initialize the EGL mapping and the vo_gpu video renderer correctly. We hope that the layer count is enough to tell whether EGL will translate the data to a RGB texture (vs. 2 texture resembling raw nv12 data). For that we introduce MP_IMAGE_HW_FLAG_OPAQUE. This commit adds the flag, infrastructure to set it, and an "example" for D3D11. The D3D11 addition is quite useless at this point. But later we want to get rid of d3d11_update_image_attribs() anyway, while we still need a way to force d3d11vpp filter insertion, so maybe it has some justification (who knows). In any case it makes testing this easier. Obviously it also adds some basic support for triggering the opaque format for decoding, which will use a driver-specific format, but which is not supported in shaders. The opaque flag is not used to determine whether d3d11vpp needs to be inserted, though.
*