[FFmpeg-devel] [PATCH RFC v2 2/3] libavcodec: Add thumbnail output to vaapi_h264 decoder

[Li, Zhong]

> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf
> Of Mark Thompson
> Sent: Wednesday, April 10, 2019 6:27 AM
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH RFC v2 2/3] libavcodec: Add thumbnail
> output to vaapi_h264 decoder
> 
> On 08/04/2019 09:53, Zachary Zhou wrote:
> > This is sample code for reference
> >
> > HW support for decode+scaling in a single HW command (VDBOX+SFC).
> > The primary target usage is video analytics, but can be used playback,
> > transcoding, etc.
> >
> > For VAAPI -
> > https://github.com/intel/libva
> > basically, it allows multiple outputs (in different resolutions) using the
> decode context in a single call (you can search for “additional_outputs” in
> va.h).
> >
> > VAAPI sample code -
> >
> https://github.com/intel/libva-utils/commit/957a269f02b00760b7e807643c
> > 821ee26abc529b
> > ---
> >  libavcodec/avcodec.h       |   8 +++
> >  libavcodec/decode.c        |  16 +++++
> >  libavcodec/options_table.h |   4 ++
> >  libavcodec/vaapi_decode.c  | 122
> > ++++++++++++++++++++++++++++++++++---
> >  libavcodec/vaapi_decode.h  |  30 +++++++++
> >  libavcodec/vaapi_h264.c    |  13 ++++
> >  6 files changed, 185 insertions(+), 8 deletions(-)
> 
> This doesn't fit into the libavcodec API at all.  If it is desirable to have some
> sort of generic multiple-output mechanism in libavcodec, please open a
> separate discussion considering that API specifically.
> 
> For the particular hardware instance you are talking about here, can you
> explain what this mechanism actually helps with?  My understanding was
> that the VDBOX / SFC chaining existed to save a small amount of power in
> restricted playback cases going directly to the display (that is, for maximising
> playback time on battery devices).  Since you are writing everything back to
> memory immediately here anyway, I don't see how it helps compared to just
> doing the scale as a filter operation.
> 
> - Mark

I would like to see some brief test data to show the benefit of this solution (e.g: power, performance, memory, ect) in the commit message. 
The comparison target should be "vaapi decoder in libavcodec + vaapi SFC scaling in libavfilter". 

"vaapi decoder in libavcodec + vaapi SFC scaling in libavfilter" path has been supported in FFmpeg, but an interesting thing is that I found that the "fast" mode is not SFC when working with iHD driver,
Instead VA_FILTER_SCALING_DEFAULT looks like SFC will take effect. More details please see: https://github.com/intel/media-driver/issues/582 
BTW, IMHO VA_FILTER_SCALING_FAST definition in vaapi, or "fast" mode named in FFmpeg is misleading, AVS mode is possible faster then SFC on high performance GPU. So would better to change it to be another name such as "low_power".