[FFmpeg-devel] [RFC 2/6] Add common V4L2 request API code

[Jonas Karlman]

On 2019-04-09 00:35, James Almer wrote:
> On 4/8/2019 5:12 PM, Jonas Karlman wrote:
>> Signed-off-by: Jonas Karlman <jonas at kwiboo.se>
>> ---
>>  configure                 |   8 +
>>  libavcodec/Makefile       |   1 +
>>  libavcodec/hwaccel.h      |   2 +
>>  libavcodec/v4l2_request.c | 885 ++++++++++++++++++++++++++++++++++++++
>>  libavcodec/v4l2_request.h |  65 +++
>>  5 files changed, 961 insertions(+)
>>  create mode 100644 libavcodec/v4l2_request.c
>>  create mode 100644 libavcodec/v4l2_request.h
>> +int ff_v4l2_request_uninit(AVCodecContext *avctx)
>> +{
>> +    V4L2RequestContext *ctx = avctx->internal->hwaccel_priv_data;
>> +    int ret;
>> +
>> +    av_log(avctx, AV_LOG_DEBUG, "%s: avctx=%p ctx=%p\n", __func__, avctx, ctx);
>> +
>> +    if (ctx->video_fd >= 0) {
>> +        ret = ioctl(ctx->video_fd, VIDIOC_STREAMOFF, &ctx->output_type);
>> +        if (ret < 0)
>> +            av_log(avctx, AV_LOG_ERROR, "%s: output stream off failed, %s (%d)\n", __func__, strerror(errno), errno);
>> +
>> +        ret = ioctl(ctx->video_fd, VIDIOC_STREAMOFF, &ctx->format.type);
>> +        if (ret < 0)
>> +            av_log(avctx, AV_LOG_ERROR, "%s: capture stream off failed, %s (%d)\n", __func__, strerror(errno), errno);
>> +    }
>> +
>> +    if (avctx->hw_frames_ctx) {
>> +        AVHWFramesContext *hwfc = (AVHWFramesContext*)avctx->hw_frames_ctx->data;
>> +        av_buffer_pool_reclaim(hwfc->pool);
> What's the idea behind this? Calling this function will free all the
> unused buffers currently residing in the pool, but will do nothing with
> those already in assigned to some AVBufferRef created with
> av_buffer_pool_get(). Those will in fact go back to the pool once they
> are not needed anymore.
>
> Other than immediately freeing up some memory, the new function you
> added is not going to do much. It wont be until av_buffer_pool_uninit()
> is called and all the assigned buffers if any returned to the pool that
> everything will be effectively freed.

This was added to solve an issue with seeking h264 files, during seeking a new hwaccel instance / buffer pool would be created (pps/sps changed)
and since we need to pre-allocate output/capture buffers during hwaccel init there was situations where too much CMA memory got consumed
while there was multiple hwaccel instances / buffer pools active.

This may have been a bigger issue in the beginning when we pre-allocated 20 buffers similar to vaapi hwaccel and when we had a bug in Kodi that
would hold reference on two instead of one AVFrame while waiting on next frame being decoded, in some situations (seeking too fast) this bug
resulted in Kodi having reference on AVFrame's from three different buffer pools requiring enough CMA memory for 60 capture and output buffers.

We have since then fixed the bug in Kodi and minimized the number of buffers that is pre-allocated in hwaccel.
Immediately freeing up unused buffers helps the likelihood of successful initializing a new hwaccel instance / buffer pool, especially useful for 4K content.

Regards,
Jonas