[FFmpeg-devel] [PATCH v2] avcodec: add AV1 frame split bitstream filter

James Almer jamrial at gmail.com
Mon Mar 25 21:58:15 EET 2019


On 3/25/2019 4:30 PM, Ronald S. Bultje wrote:
> Hi,
> 
> On Mon, Mar 25, 2019 at 12:42 PM James Almer <jamrial at gmail.com> wrote:
> 
>> On 3/25/2019 12:29 PM, James Almer wrote:
>>> +        ret = ff_cbs_read_packet(s->cbc, td, s->buffer_pkt);
>>> +        if (ret < 0) {
>>> +            av_log(ctx, AV_LOG_ERROR, "Failed to parse temporal
>> unit.\n");
>>> +            return ret;
>>
>>> +    if (split) {
>>> +        AV1RawFrameHeader *frame = NULL;
>>> +        int cur_frame_type = -1, size = 0;
>>> +
>>> +        for (i = s->cur_frame_idx; i < td->nb_units; i++) {
>>> +            CodedBitstreamUnit *unit = &td->units[i];
>>> +
>>> +            size += unit->data_size;
>>> +            if (unit->type == AV1_OBU_FRAME) {
>>> +                AV1RawOBU *obu = unit->content;
>>> +
>>> +                if (frame) {
>>> +                    ret = AVERROR_INVALIDDATA;
>>> +                    goto fail;
>>
>>> +            } else if (unit->type == AV1_OBU_FRAME_HEADER) {
>>> +                AV1RawOBU *obu = unit->content;
>>> +
>>> +                if (frame) {
>>> +                    ret = AVERROR_INVALIDDATA;
>>> +                    goto fail;
>>
>>> +            } else if (unit->type == AV1_OBU_TILE_GROUP) {
>>> +                AV1RawOBU *obu = unit->content;
>>> +                AV1RawTileGroup *group = &obu->obu.tile_group;
>>> +
>>> +                if (!frame || cur_frame_type != AV1_OBU_FRAME_HEADER) {
>>> +                    ret = AVERROR_INVALIDDATA;
>>> +                    goto fail;
>>
>> I'm not sure if we should abort and discard the packet in these cases,
>> or just pass it through.
>>
>> In all these the Temporal Unit is invalid in some form (Orphaned Tile
>> Group OBUs, Frame Headers showing up when the Tile Groups from a
>> previous Frame Header were expected, etc), but maybe it should be left
>> to the decoder to decide what to do with them.
> 
> 
> Is cur_frame_type the previous OBU's type?

cur_frame_type is set to either Frame or Frame Header based on the last
of such OBU type seen, and checked when parsing a Tile Group OBU in
order to abort if it was not a Frame Header (Tile Group OBUs must come
after a Frame Header OBU).

I only added it because in the case where i want to include all the
trailing OBUs when the last frame is the Frame OBU kind, if one of said
trailing OBUs is an orphaned Tile Group then it means the TU is broken.

> Is something like framehdr-highdynamicrangframedata-tiledata valid?

I don't know. The spec says:

"A coded video sequence consists of one or more temporal units. A
temporal unit consists of a series of OBUs starting from a temporal
delimiter, optional sequence headers, optional metadata OBUs, a sequence
of one or more frame headers, each followed by zero or more tile group
OBUs as well as optional padding OBUs."

It would seem it's not valid or at least not expected, but nothing in
practice should prevent decoding with such ordering.
It doesn't seem to be too strict about what's meant to be after a Frame
Header other than its Tile Groups (The sentence is lacking the usual
must/shall/should wording), but either way this bsf will handle that
just fine. It makes sure to split starting from a Frame Header all the
way to last Tile Group for the frame, including everything in between,
and everything after if it's the last frame. It doesn't do any reordering.


More information about the ffmpeg-devel mailing list