Change #273947

Comments

avdevice/avfoundation: wait for frame consumption to avoid dropping A/V frames
The capture callback dropped the previous frame instead of waiting for it
to be consumed, so frames were lost when avf_read_packet() fell behind;
39fbd06314 (return EAGAIN instead of waiting) made it easy to hit.

Add back a condition variable: the capture callback blocks until
avf_read_packet() takes the current frame, and the reader waits on it when
no frame is ready. An is_stopping flag set in destroy_context() wakes both
sides so teardown cannot deadlock. observed_quit is now set under the lock
and broadcast as well, so a blocked reader wakes and returns EOF when a
transport-control device stops delivering frames.

Based on a patch by Zhongxin Zhuang <zx.zhuang@hotmail.com>; here
unlock_frames() only broadcasts and unlocks, so a read no longer releases
the other stream's still-unconsumed frame.

Verified by capturing camera+mic for 8s and comparing delivered packet
counts before/after this change:

  ffmpeg -f avfoundation -pixel_format nv12 -framerate 30 -i "0:0" -t 8 \
         -vf "scale=2560:1440,hqdn3d" -c:v libx264 -preset medium \
         -c:a aac out.mp4
  ffprobe -count_packets -show_entries stream=nb_read_packets out.mp4

  slow consumer (ideal: video ~240, audio ~375)
                     video   audio
    before (EAGAIN)   174     163   (audio ~57% dropped)
    after  (condvar)  234     376   (no drops)

  fast consumer (-preset ultrafast, no filter)
    before            240     328
    after             238     376

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

Changed files

• libavdevice/avfoundation.m