[FFmpeg-devel] [PATCH 0/2] [RFC] Attached pics, particularly wrt Matroska

[Andreas Rheinhardt]

Hello,

there are currently a few open track issues related to Matroska attached
pics: #2270, #4286, #4591 and #7717. So I thought it worthwhile to take
a look.

The modifications to mux streams with disposition AV_DISPOSITION_ATTACHED_PIC
that contain an attached pic are easy; see the second patch.

Unfortunately, not every stream with this disposition has actually an
attached pic, although the documentation says that the picture "can also
be accessed at any time in AVStream.attached_pic." E.g. no output track
in FFmpeg currently contains an attached pic; attached pics are not
propagated from input streams.

The first commit changes this a bit and together these two commits are
enough to make remuxing attached pics from Matroska to Matroska with
FFmpeg easily possible. It is also possible to use attached pics
provided that they emanate from a demuxer that sets the attached_pic
and provided one provides the necessary filename metadata information
which containers other than Matroska don't provide on their own.

But e.g. the scenario from #4286 doesn't work because just setting the
disposition to attached pic doesn't fill the attached_pic. Therefore
video tracks without attached_pic are always treated as a normal video
track as they are now.

There is also another complication: Actually, the description of
attached_pic contains: "decoding: set by libavformat, must not be
modified by the caller." This means that my patch to ffmpeg.c is
unacceptable, isn't it? And it means that an API function to set the
attached pic is required.

I have not even looked into the complications that would arise if one
wants to do more than a simple streamcopy.

Finally, I am also wondering about how timed thumbnails should be
handled: For a track that ends up as an attachment all the normal
packets are discarded. Which is bad for timed thumbnails.

- Andreas


PS: There is btw another bug wrt to the video streams corresponding to
attached pics: If they are muxed, the whole file is read until a packet
from this stream is found; often this means that the file is read until
the end. This is not Matroska-specific and probably caused by such
streams being counted among the streams to interleave.

Andreas Rheinhardt (2):
  fftools/ffmpeg: Propagate attached_pic
  avformat/matroskaenc: Write attached_pic as attachment

 fftools/ffmpeg.c          |  8 ++++++++
 libavformat/matroskaenc.c | 27 +++++++++++++++++++++------
 2 files changed, 29 insertions(+), 6 deletions(-)

-- 
2.19.2