When using the custom WebKitMediaSrc element (used by WebKit and able to
perform an initial seek in playbin), a stall caused by streamsynchronizer
was detected during an initial seek. The flow of events revealed that the
intertwining of the initial configuration of the streams with the reset
caused by the flush events from the seek left streamsynchronizer in an
inconsistent state:
streamsynchronizer0:sink_0 (video) events, starting before the seek:
stream-start --> Sets the stream to wait
flush-stop --> Clears the stream wait flag
caps
tag
segment
stream-collection
(buffers start to come and flow properly)
streamsynchronizer0:sink_1 (audio) events, happening after seek:
(no flush events, because the stream hadn't been initialized when the seek happened)
stream-start --> Sets the stream to wait
caps
segment
(stalled because the stream is in wait mode!)
The code in streamsynchronizer expects that all the streams are in wait
state before releasing all of them at once. The flush on the video stream
broke that assumption and that's why the audio stream is never released in
that scenario.
Avoiding the clearing of the wait flag on flush-stop isn't an actual solution
to the problem, as it creates other side effects and at least makes the
gst-editing-services/seek_with_stop test to timeout. The alternate solution
implemented in this patch consists on analyzing if the other streams different
from the one newly added (after the flush) aren't waiting (which would mean
that they've all been unlocked after all of them were waiting before), and,
in that case, mark the new stream as also not waiting.
A new test_stream_start_wait test case has been added to demonstrate this
problem. The test case creates a video stream, pushes a buffer, then
simulates a seek by pushing flush-start, flush-stop, stream-start and segment
events. Note that the flush-stop clears the video stream waiting flag.
After that, a new audio stream is created and stream-start and new segment
events are sent. Note that stream-start will set the audio stream to wait.
Then a buffer is pushed on each stream. In the failing case, the test hangs.
In the working case (after this fix), the test runs properly because the
fact of having seen a stream-start also helps to clear the wait flag.
A second new test_stream_start_wait_sparse test has also been added to prove
that this mechanism can also work with sparse streams (a special case of the
current stream-start handling code). This test behaves like the previous one,
but there's no video buffer after the seek (it'll come in the future, as the
stream is sparse, but actually never comes). The buffer after the seek in the
audio stream starts at its due time. Streamsynchronizer is able to ignore
the wait for the video stream and produce audio buffers on time.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4544>
a1a189c07c