FaSTTUBe/GST-Tensordecoder-ov_ep
37
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

design: seeking: section breakdown, markup, readability and grammar fixes

Browse Source
This commit is contained in:
Reynaldo H. Verdejo Pinochet 2016-12-29 22:40:20 -08:00
parent 8c08738547
commit e509b30d95
1 changed files with 43 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

84
markdown/design/seeking.md
View File

@ -1,11 +1,12 @@
# Seeking # Seeking
Seeking in GStreamer means configuring the pipeline for playback of the Seeking in GStreamer means configuring the pipeline for playback of the
media between a certain start and stop time, called the playback media between a certain start and stop time, called the playback
segment. By default a pipeline will play from position 0 to the total segment. By default a pipeline will play from position 0 to the total
duration of the media at a rate of 1.0. duration of the media at a rate of 1.0.
A seek is performed by sending a seek event to the sink elements of a A seek is performed by sending a `SEEK` event to the sink elements of a
pipeline. Sending the seek event to a bin will by default forward the pipeline. Sending the `SEEK` event to a bin will by default forward the
event to all sinks in the bin. event to all sinks in the bin.
When performing a seek, the start and stop values of the segment can be When performing a seek, the start and stop values of the segment can be
@ -20,7 +21,7 @@ Feedback of the seek operation can be immediately using the
pipeline is discarded and playback starts from the new position pipeline is discarded and playback starts from the new position
immediately. immediately.
When the FLUSH flag is not set, the seek will be queued and executed as When the `FLUSH` flag is not set, the seek will be queued and executed as
soon as possible, which might be after all queues are emptied. soon as possible, which might be after all queues are emptied.
Seeking can be performed in different formats such as time, frames or Seeking can be performed in different formats such as time, frames or
@ -39,7 +40,7 @@ Non segment seeking will make the pipeline emit EOS when the configured
segment has been played. segment has been played.
Segment seeking (using the `GST_SEEK_FLAG_SEGMENT`) will not emit an Segment seeking (using the `GST_SEEK_FLAG_SEGMENT`) will not emit an
EOS at the end of the playback segment but will post a SEGMENT_DONE EOS at the end of the playback segment but will post a `SEGMENT_DONE`
message on the bus. This message is posted by the element driving the message on the bus. This message is posted by the element driving the
playback in the pipeline, typically a demuxer. After receiving the playback in the pipeline, typically a demuxer. After receiving the
message, the application can reconnect the pipeline or issue other seek message, the application can reconnect the pipeline or issue other seek
@ -81,8 +82,8 @@ media is immediately played after the seek call returns.
This seek type is typically performed after issuing segment seeks to This seek type is typically performed after issuing segment seeks to
finish the playback of the pipeline. finish the playback of the pipeline.
Performing a non-flushing seek in a PAUSED pipeline blocks until the Performing a non-flushing seek in a `PAUSED` pipeline blocks until the
pipeline is set to playing again since all data passing is blocked in pipeline is set to playing again, since all data passing is blocked in
the prerolled sinks. the prerolled sinks.
### segment seeking with FLUSH ### segment seeking with FLUSH
@ -93,14 +94,13 @@ This seek is typically performed when starting seamless looping.
This seek is typically performed when continuing seamless looping. This seek is typically performed when continuing seamless looping.
Demuxer/parser behaviour and `SEEK_FLAG_KEY_UNIT` and ## `KEY_UNIT` and `ACCURATE` flags
`SEEK_FLAG_ACCURATE`
This section aims to explain the behaviour expected by an element with This section aims to explain the behaviour expected by an element with
regard to the KEY_UNIT and ACCURATE seek flags using the example of a regard to the `KEY_UNIT` and `ACCURATE` seek flags, using a parser
parser or demuxer. or demuxer as an example.
#### DEFAULT BEHAVIOUR: ### DEFAULT BEHAVIOUR:
When a seek to a certain position is requested, the demuxer/parser will When a seek to a certain position is requested, the demuxer/parser will
do two things (ignoring flushing and segment seeks, and simplified for do two things (ignoring flushing and segment seeks, and simplified for
@ -114,17 +114,17 @@ To ensure that the data corresponding to the requested seek position can
actually be decoded, a demuxer or parser needs to start pushing data actually be decoded, a demuxer or parser needs to start pushing data
from a keyframe/keyunit at or before the requested seek position. from a keyframe/keyunit at or before the requested seek position.
Unless requested differently (via the KEY_UNIT flag), the start of the Unless requested differently (via the `KEY_UNIT` flag), the start of the
segment event should be the requested seek position. segment event should be the requested seek position.
So by default a demuxer/parser will then start pushing data from So by default a demuxer/parser will then start pushing data from
position DATA and send a segment event with start position SEG_START, position DATA and send a segment event with start position `SEG_START`,
and DATA ⇐ SEG_START. and `DATA ⇐ SEG_START`.
If DATA < SEG_START, a well-behaved video decoder will start decoding If `DATA < SEG_START`, a well-behaved video decoder will start decoding
frames from DATA, but take into account the segment configured by the frames from DATA, but take into account the segment configured by the
demuxer via the segment event, and only actually output decoded video demuxer via the segment event, and only actually output decoded video
frames from SEG_START onwards, dropping all decoded frames that are frames from `SEG_START` onwards, dropping all decoded frames that are
before the segment start and adjusting the timestamp/duration of the before the segment start and adjusting the timestamp/duration of the
buffer that overlaps the segment start ("clipping"). A buffer that overlaps the segment start ("clipping"). A
not-so-well-behaved video decoder will start decoding frames from DATA not-so-well-behaved video decoder will start decoding frames from DATA
@ -132,52 +132,52 @@ and push decoded video frames out starting from position DATA, in which
case the frames that are before the configured segment start will case the frames that are before the configured segment start will
usually be dropped/clipped downstream (e.g. by the video sink). usually be dropped/clipped downstream (e.g. by the video sink).
#### GST_SEEK_FLAG_KEY_UNIT: ### `GST_SEEK_FLAG_KEY_UNIT`
If the KEY_UNIT flag is specified, the demuxer/parser should adjust the If the `KEY_UNIT` flag is specified, the demuxer/parser should adjust the
segment start to the position of the key frame closest to the requested segment start to the position of the key frame closest to the requested
seek position and then start pushing out data from there. The nearest seek position and then start pushing out data from there. The nearest
key frame may be before or after the requested seek position, but many key frame may be before or after the requested seek position, but many
implementations will only look for the closest keyframe before the implementations will only look for the closest keyframe before the
requested position. requested position.
Most media players and thumbnailers do (and should be doing) KEY_UNIT Most media players and thumbnailers do (and should be doing) `KEY_UNIT`
seeks by default, for performance reasons, to ensure almost-instant seeks by default, for performance reasons, to ensure almost-instant
responsiveness when scrubbing (dragging the seek slider in PAUSED or responsiveness when scrubbing (dragging the seek slider in `PAUSED` or
PLAYING mode). This works well for most media, but results in suboptimal `PLAYING` mode). This works well for most media, but results in suboptimal
behaviour for a small number of *odd* files (e.g. files that only have behaviour for a small number of *odd* files (e.g. files that only have
one keyframe at the very beginning, or only a few keyframes throughout one keyframe at the very beginning, or only a few keyframes throughout
the entire stream). At the time of writing, a solution for this still the entire stream). At the time of writing, a solution for this still
needs to be found, but could be implemented demuxer/parser-side, e.g. needs to be found, but could be implemented demuxer/parser-side, e.g.
make demuxers/parsers ignore the KEY_UNIT flag if the position make demuxers/parsers ignore the `KEY_UNIT` flag if the position
adjustment would be larger than 1/10th of the duration or somesuch. adjustment would be larger than 1/10th of the duration or somesuch.
Flags can be used to influence snapping direction for those cases where Flags can be used to influence snapping direction for those cases where
it matters. SNAP_BEFORE will select the preceding position to the seek it matters. `SNAP_BEFORE` will select the preceding position to the seek
target, and SNAP_AFTER will select the following one. If both flags are target, and `SNAP_AFTER` will select the following one. If both flags are
set, the nearest one to the seek target will be used. If none of these set, the nearest one to the seek target will be used. If none of these
flags are set, the seeking implemention is free to select whichever it flags are set, the seeking implemention is free to select whichever it
wants. wants.
#### Summary: #### Summary:
- if the KEY_UNIT flag is **not** specified, the demuxer/parser - if the `KEY_UNIT` flag is **not** specified, the demuxer/parser
should start pushing data from a key unit preceding the seek should start pushing data from a key unit preceding the seek
position (or from the seek position if that falls on a key unit), position (or from the seek position if that falls on a key unit),
and the start of the new segment should be the requested seek and the start of the new segment should be the requested seek
position. position.
- if the KEY_UNIT flag is specified, the demuxer/parser should start - if the `KEY_UNIT` flag is specified, the demuxer/parser should start
pushing data from the key unit nearest the seek position (or from pushing data from the key unit nearest the seek position (or from
the seek position if that falls on a key unit), and the start of the the seek position if that falls on a key unit), and the start of the
new segment should be adjusted to the position of that key unit new segment should be adjusted to the position of that key unit
which was nearest the requested seek position (ie. the new segment which was nearest the requested seek position (ie. the new segment
start should be the position from which data is pushed). start should be the position from which data is pushed).
### GST_SEEK_FLAG_ACCURATE: ### `GST_SEEK_FLAG_ACCURATE`
If the ACCURATE flag is specified in a seek request, the demuxer/parser If the `ACCURATE` flag is specified in a seek request, the demuxer/parser
is asked to do whatever it takes (!) to make sure that the position is asked to do whatever it takes (!) to make sure the position
seeked to is accurate in relation to the beginning of the stream. This seeked to is accurate in relation to the beginning of the stream. This
means that it is not acceptable to just approximate the position (e.g. means that it is not acceptable to just approximate the position (e.g.
using an average bitrate). The achieved position must be exact. In the using an average bitrate). The achieved position must be exact. In the
@ -185,30 +185,30 @@ worst case, the demuxer or parser needs to push data from the beginning
of the file and let downstream clip everything before the requested of the file and let downstream clip everything before the requested
segment start. segment start.
The ACCURATE flag does not affect what the segment start should be in The `ACCURATE` flag does not affect what the segment start should be in
relation to the requested seek position. Only the KEY_UNIT flag (or its relation to the requested seek position. Only the `KEY_UNIT` flag (or its
absence) has any effect on that. absence) has any effect on that.
Video editors and frame-stepping applications usually use the ACCURATE Video editors and frame-stepping applications usually use the `ACCURATE`
flag. flag.
#### Summary: #### Summary:
- if the ACCURATE flag is **not** specified, it is up to the - if the `ACCURATE` flag is **not** specified, it is up to the
demuxer/parser to decide how exact the seek should be. If the flag demuxer/parser to decide how exact the seek should be. In this case,
is not specified, the expectation is that the demuxer/parser does a the expectation is that the demuxer/parser does a
resonable best effort attempt, trading speed for accuracy. In the resonable best effort attempt, trading speed for accuracy. In the
absence of an index, the seek position may be approximated. absence of an index, the seek position may be approximated.
- if the ACCURATE flag is specified, absolute accuracy is required, - if the `ACCURATE` flag **is** specified, absolute accuracy is required,
and speed is of no concern. It is not acceptable to just approximate and speed is of no concern. It is not acceptable to just approximate
the seek position in that case. the seek position in that case.
- the ACCURATE flag does not imply that the segment starts at the - the `ACCURATE` flag does not imply that the segment starts at the
requested seek position or should be adjusted to the nearest requested seek position or should be adjusted to the nearest
keyframe, only the KEY_UNIT flag determines that. keyframe, only the `KEY_UNIT` flag determines that.
### ACCURATE and KEY_UNIT combinations: ### `ACCURATE` and `KEY_UNIT` combinations:
All combinations of these two flags are valid: All combinations of these two flags are valid:
@ -216,14 +216,14 @@ All combinations of these two flags are valid:
from preceding key frame (or earlier), feel free to approximate the from preceding key frame (or earlier), feel free to approximate the
seek position seek position
- only KEY_UNIT specified: segment starts from position of nearest - only `KEY_UNIT` specified: segment starts from position of nearest
keyframe, send data from nearest keyframe, feel free to approximate keyframe, send data from nearest keyframe, feel free to approximate
the seek position the seek position
- only ACCURATE specified: segment starts at seek position, send data - only `ACCURATE` specified: segment starts at seek position, send data
from preceding key frame (or earlier), do not approximate the seek from preceding key frame (or earlier), do not approximate the seek
position under any circumstances position under any circumstances
- ACCURATE | KEY_UNIT specified: segment starts from position of - `ACCURATE | KEY_UNIT` specified: segment starts from position of
nearest keyframe, send data from nearest key frame, do not nearest keyframe, send data from nearest key frame, do not
approximate the seek position under any circumstances approximate the seek position under any circumstances