It is racy and may cause us to accidentally keep forwarding data past
the EOS. The only reason to stop dropping would be when we encounter a
stream-start, segment, or segment-done event, either in push_one
(already queued) or in the sink pad's event function.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5766>