Source code
Revision control
Copy as Markdown
Other Tools
# AudioStream
`AudioStream` owns a single cubeb stream and turns the backend's raw frame
counter into a media-time clock. It is owned by `AudioSink`, which is wrapped by
[AudioSinkWrapper](AudioSinkWrapper.md), and that wrapper is what
`MediaDecoderStateMachine` reads the playback position from. So the clock
described here is what the video sink synchronises against.
Two responsibilities live here and are worth keeping apart: handing decoded
audio to the backend, on the backend's realtime callback thread, and reporting a
media position, on the state machine's thread. `AudioClock` and its
`FrameHistory` are the bridge between them.
## The two frame cursors
A backend exposes two frame counts, and they do not mean the same thing.
- The **write cursor** is what the data callback has handed over. It moves in
one jump per callback, then sits still.
- The **play cursor** is what the device has actually output, read through
`cubeb_stream_get_position`. It rises continuously, driven by the hardware
clock.
The gap between them is audio handed over but not yet heard. This document calls
it the **unplayed** audio.
```
frames ---------------------------------------------------------------->
P W
| |
..... played ......|........... unplayed ...........|
|<------------------------------>|
| handed to the backend, |
| not yet heard |
P = play cursor (cubeb_stream_get_position)
W = write cursor (what the data callback has handed over)
unplayed = W - P
```
The unplayed audio is the cushion that lets a slightly late callback still find
audio queued ahead of it, so it does not reach zero during healthy playback. Its
size is the device's output latency plus the part of the last callback the
backend has not yet interpolated past, so it jumps up by one callback on each
callback and slides back down to the latency in between.
Two consequences matter below:
- The play cursor normally stays at or below the write cursor, because it
reports what the device has consumed and a device cannot consume what nobody
handed it. How it is derived varies: some backends compute it as the frames
handed over minus the latency they report, others return an OS or device
counter. Either way it is an estimate rather than a hardware readout, and some
backends extrapolate from the wall clock since the last callback, so a late
callback or a device xrun can report it above the write cursor. The frame
history can also account for less than the device has played, because a
callback that found the queue full keeps its frames on the audio thread until
a later call collects them.
- The unplayed audio belongs to the position that was current when it was handed
over, not to anything that happens afterwards.
## The frame history
`FrameHistory` maps a play-cursor value to a media time. It is a piecewise
mapping rather than a single division, because silence carries no media time and
the playback rate can change mid-stream.
| Member | Unit | Meaning |
|-----------------|--------------|------------------------------------|
| `mBaseOffset` | frames | Where the retained history starts |
| `mBasePosition` | microseconds | The media time at `mBaseOffset` |
| `mChunks` | both | Segments from `mBaseOffset` upward |
The first two are one anchor in two units: when the device had output
`mBaseOffset` frames, the media was at `mBasePosition`. `GetPosition` starts
there and walks the chunks up to the play cursor.
Per chunk, `servicedFrames` counts frames carrying media time and `totalFrames`
counts every frame output, so the difference is silence inserted on an underrun.
The clamp `min(delta, servicedFrames)` holds the clock still while that silence
is audible, and each chunk's `rate` is the output rate in force when it was
recorded, so a rate change maps correctly on both sides of it.
### The conserved identity
`mBaseOffset` starts at zero and folding a chunk away adds its `totalFrames` to
it, so this sum is invariant:
```
mBaseOffset + sum of every retained chunk's totalFrames == the write cursor
```
A query normally lands inside the retained chunks, because the play cursor stays
at or below the write cursor. The exception is a callback still stranded on the
audio thread, which leaves the history short. Keeping the identity true is the
whole job: break it and the walk runs off the end, at which point `GetPosition`
falls back to `mBasePosition` alone, which is the serviced total, so the
reported position stops following the device between callbacks and jumps
instead.
## Rebasing a stream reused across a seek
When the stream is kept alive across a seek rather than destroyed, its frame
counter keeps climbing while media time has to restart, so `RebaseLive` moves
the anchor instead of resetting the counter.
Media time zero belongs at the write cursor, not the play cursor, because
everything below the write cursor was handed over before the seek. The anchor
cannot simply be set to the write cursor: the history stores `playCursor -
mBaseOffset` in a `uint32_t` and asserts the difference is not negative, so an
anchor above the play cursor trips that assertion in a debug build and wraps in
an opt one. The anchor therefore stays at the play cursor and the unplayed audio
is carried across as a chunk that services nothing:
```
frames: P W
| |
|<---------- unplayed ---------->|
| one chunk, totalFrames = |
| unplayed, servicedFrames |
| = 0 |
media time: 0 .............................. 0 ------> counts up from here
```
`totalFrames` keeps the identity intact so queries stay inside the chunks, and
`servicedFrames` of zero means draining that audio advances no media time, so
the clock holds at the seek target until the first post-seek frame is heard.
Both come from one subtraction, `W - P`.
`W` is a counter the clock keeps itself, bumped on the audio thread, not a sum
over the retained chunks: the history is only fed when the owner thread drains
the callback queue, and nothing drains it during a seek. It is read after
synchronising with that thread, since an early read understates the gap and lets
the position lead the audio for the rest of playback.
The subtraction needs both cursors, and the audio it describes has to be audio
that will still be played:
| Situation | Carried |
|---------------------------------------|----------|
| Stream kept running across the seek | `W - P` |
| Backend restarted, queue discarded | nothing |
| Position query failed, no `P` | nothing |
| `P` reported above `W` | nothing |
| Backend drained since the resume | nothing |
Carrying nothing is how the clock behaved before the carry existed. On the rows
where the anchor is still a real play cursor, the identity then holds against
the frames the device will play rather than against the counter, which still
counts the discarded ones.
The failed-query row is weaker, and deliberately so. With no cursor to anchor
at, the rebase anchors at zero, which is neither cursor, so the identity is left
broken and the reported position degrades to the serviced total: it stops
following the device between callbacks. That is tolerated because the query only
fails on a stream that is already dead, after which the sink pins the position
to the last good value and this mapping is never consulted.
A rebase also has to neutralise callback information that has not reached the
history yet. The reader counts the frames of every item it takes off the queue,
and a rebase records the count it accounted for, so an item landing at or below
that point is dropped rather than appended. Without that, an item stranded on
the audio thread during the seek would be applied afterwards and, being
underrun-only, would merge into the carried chunk and extend the window that
advances no media time by the whole stranded silence.
The unplayed audio may be leftover pre-seek audio, silence produced during the
seek, or a mixture. Either way it was handed over before the seek and is not
part of the new position.
## Threading
- The data callback runs on the backend's realtime thread and must not block or
allocate.
- `UpdateFrameHistory` is called from that callback; `GetPosition` from the
state machine's thread.
- On macOS the position read is deliberately lock free. Callback information
reaches the reader through a single-producer single-consumer queue, applied by
`AudioClock::ApplyQueuedCallbackInfo`, which both `GetPosition` and `Rebase`
call before touching the history. Anything mutating the history must run on
that reader thread and touch only reader-side state; the callback may still be
running.
- Elsewhere a mutex guards the history on both sides.