Copyright © 2013 W3C® (MIT, ERCIM, Keio, Beihang), All Rights Reserved. W3C liability, trademark and document use rules apply.
If you wish to make comments or file bugs regarding this document in a manner that is tracked by the W3C, please submit them via our public bug database.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
The working groups maintains a list of all bug reports that the editors have not yet tried to address. This draft highlights some of the pending issues that are still to be discussed in the working group. No decision has been taken on the outcome of these issues including whether they are valid.
Implementors should be aware that this specification is not stable. Implementors who are not taking part in the discussions are likely to find the specification changing out from under them in incompatible ways. Vendors interested in implementing this specification before it eventually reaches the Candidate Recommendation stage should join the mailing list mentioned below and take part in the discussions.
This document was published by the HTML Working Group as an Editor's Draft. If you wish to make comments regarding this document, please send them to public-html-media@w3.org (subscribe, archives). All comments are welcome.
Publication as an Editor's Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.
This specification allows JavaScript to dynamically construct media streams for <audio> and <video>. It defines objects that allow JavaScript to pass media segments to an HTMLMediaElement [HTML5]. A buffering model is also included to describe how the user agent acts when different media segments are appended at different times. Byte stream specifications used with these extensions are available in the byte stream format registry.
This specification was designed with the following goals in mind:
This specification defines:
The track buffers that provide coded frames for the enabled
audioTracks
, the selected
videoTracks
, and the
"showing"
or "hidden"
textTracks
. All these tracks are associated with
SourceBuffer
objects in the activeSourceBuffers
list.
A presentation timestamp range used to filter out coded frames while appending. The append window represents a single
continuous time range with a single start time and end time. Coded frames with presentation timestamp within this range are allowed to be appended
to the SourceBuffer
while coded frames outside this range are filtered out. The append window start and end times are controlled by
the appendWindowStart
and appendWindowEnd
attributes respectively.
A unit of media data that has a presentation timestamp, a decode timestamp, and a coded frame duration.
The duration of a coded frame. For video and text, the duration indicates how long the video frame or text should be displayed. For audio, the duration represents the sum of all the samples contained within the coded frame. For example, if an audio frame contained 441 samples @44100Hz the frame duration would be 100 milliseconds.
A group of coded frames that are adjacent and have monotonically increasing decode timestamps without any gaps. Discontinuities detected by the
coded frame processing algorithm and abort()
calls trigger the start of a new coded frame group.
The decode timestamp indicates the latest time at which the frame needs to be decoded assuming instantaneous decoding and rendering of this and any dependant frames (this is equal to the presentation timestamp of the earliest frame, in presentation order, that is dependant on this frame). If frames can be decoded out of presentation order, then the decode timestamp must be present in or derivable from the byte stream. The user agent must run the end of stream algorithm with the error parameter set to "decode"
if this is not the case. If frames cannot be decoded out of presentation order and a decode timestamp is not present in the byte stream, then the decode timestamp is equal to the presentation timestamp.
The delay between a frame's presentation time and the actual time it was displayed, in microseconds & rounded to the nearest display refresh interval. This delay is always greater than or equal to zero since frames must never be displayed before their presentation time. Non-zero delays are a sign of playback jitter and possible loss of A/V sync.
A sequence of bytes that contain all of the initialization information required to decode a sequence of media segments. This includes codec initialization data, Track ID mappings for multiplexed segments, and timestamp offsets (e.g. edit lists).
The byte stream format specifications in the byte stream format registry contain format specific examples.
A sequence of bytes that contain packetized & timestamped media data for a portion of the media timeline. Media segments are always associated with the most recently appended initialization segment.
The byte stream format specifications in the byte stream format registry contain format specific examples.
A MediaSource object URL is a unique Blob URI [FILE-API] created by createObjectURL()
. It is used to attach a MediaSource
object to an HTMLMediaElement.
These URLs are the same as a Blob URI, except that anything in the definition of that feature that refers to File and Blob objects is hereby extended to also apply to MediaSource
objects.
The origin of the MediaSource object URL is specified by the Origin of Blob URIs [FILE-API].
For example, the origin of the MediaSource object URL affects the way that the media element is consumed by canvas.
The parent media source of a SourceBuffer
object is the MediaSource
object that created it.
The presentation start time is the earliest time point in the presentation and specifies the initial playback position and earliest possible position. All presentations created using this specification have a presentation start time of 0.
The presentation interval of a coded frame is the time interval from its presentation timestamp to the presentation timestamp plus the coded frame's duration. For example, if a coded frame has a presentation timestamp of 10 seconds and a coded frame duration of 100 milliseconds, then the presentation interval would be [10-10.1). Note that the start of the range is inclusive, but the end of the range is exclusive.
The order that coded frames are rendered in the presentation. The presentation order is achieved by ordering coded frames in monotonically increasing order by their presentation timestamps.
A reference to a specific time in the presentation. The presentation timestamp in a coded frame indicates when the frame must be rendered.
A position in a media segment where decoding and continuous playback can begin without relying on any previous data in the segment. For video this tends to be the location of I-frames. In the case of audio, most audio frames can be treated as a random access point. Since video tracks tend to have a more sparse distribution of random access points, the location of these points are usually considered the random access points for multiplexed streams.
The specific byte stream format specification that describes the format of the byte stream accepted by a SourceBuffer
instance. The
byte stream format specification, for a SourceBuffer
object, is selected based on the type passed to the
addSourceBuffer()
call that created the object.
A byte stream format specific structure that provides the Track ID, codec configuration, and other metadata for a single track. Each track description inside a single initialization segment has a unique Track ID. The user agent must run the end of stream algorithm with the error parameter set to "decode"
if the Track ID is not unique within the initialization segment .
A Track ID is a byte stream format specific identifier that marks sections of the byte stream as being part of a specific track. The Track ID in a track description identifies which sections of a media segment belong to that track.
The MediaSource object represents a source of media data for an HTMLMediaElement. It keeps track of the readyState
for this source as well as a list of SourceBuffer
objects that can be used to add media data to the presentation. MediaSource objects are created by the web application and then attached to an HTMLMediaElement. The application uses the SourceBuffer
objects in sourceBuffers
to add media data to this source. The HTMLMediaElement fetches this media data from the MediaSource
object when it is needed during playback.
enum ReadyState {
"closed",
"open",
"ended"
};
Enumeration description | |
---|---|
closed | Indicates the source is not currently attached to a media element. |
open |
The source has been opened by a media element and is ready for data to be appended to the SourceBuffer objects in sourceBuffers .
|
ended |
The source is still attached to a media element, but endOfStream() has been called.
|
enum EndOfStreamError {
"network",
"decode"
};
Enumeration description | |
---|---|
network |
Terminates playback and signals that a network error has occured. Note JavaScript applications should use this status code to terminate playback with a network error. For example, if a network error occurs while fetching media data. |
decode |
Terminates playback and signals that a decoding error has occured. Note JavaScript applications should use this status code to terminate playback with a decode error. For example, if a parsing error occurs while processing out-of-band media data. |
[Constructor]
interface MediaSource : EventTarget {
readonly attribute SourceBufferList
sourceBuffers;
readonly attribute SourceBufferList
activeSourceBuffers;
readonly attribute ReadyState
readyState;
attribute unrestricted double duration;
SourceBuffer
addSourceBuffer (DOMString type);
void removeSourceBuffer (SourceBuffer
sourceBuffer);
void endOfStream (optional EndOfStreamError
error);
static boolean isTypeSupported (DOMString type);
};
activeSourceBuffers
of type SourceBufferList
, readonly Contains the subset of sourceBuffers
that are providing the
selected video track, the
enabled audio tracks, and the
"showing" or "hidden" text tracks.
The Changes to selected/enabled track state section describes how this attribute gets updated.
duration
of type unrestricted double, Allows the web application to set the presentation duration. The duration is initially set to NaN when the MediaSource
object is created.
On getting, run the following steps:
readyState
attribute is "closed"
then return NaN and abort these steps.On setting, run the following steps:
INVALID_ACCESS_ERR
exception and abort these steps.readyState
attribute is not "open"
then throw an INVALID_STATE_ERR
exception and abort these steps.updating
attribute equals true on any SourceBuffer
in sourceBuffers
, then throw an INVALID_STATE_ERR
exception and abort these steps.appendBuffer()
, appendStream()
and endOfStream()
can update the duration under certain circumstances.
readyState
of type ReadyState
, readonly Indicates the current state of the MediaSource
object. When the MediaSource
is created readyState
must be set to "closed"
.
sourceBuffers
of type SourceBufferList
, readonly SourceBuffer
objects associated with this MediaSource
. When readyState
equals "closed"
this list will be empty. Once readyState
transitions to "open"
SourceBuffer objects can be added to this list by using addSourceBuffer()
.
addSourceBuffer
Adds a new SourceBuffer
to sourceBuffers
.
Implementations must support at least 1 MediaSource object with the following SourceBuffer
configurations. MediaSource objects must
support each of the configurations below, but they are only required to support one configuration at a time.
Supporting multiple configurations at once or additional configurations is a quality of implementation issue.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
type | DOMString | ✘ | ✘ |
SourceBuffer
When this method is invoked, the user agent must run the following steps:
INVALID_ACCESS_ERR
exception and abort these steps.SourceBuffer
objects in sourceBuffers
, then throw a NOT_SUPPORTED_ERR
exception and abort these steps.QUOTA_EXCEEDED_ERR
exception and abort these steps.
For example, a user agent may throw a QUOTA_EXCEEDED_ERR
exception if the media element has reached the
HAVE_METADATA
readyState. This can occur if the user agent's media engine does not support adding more tracks during
playback.
readyState
attribute is not in the "open"
state then throw an INVALID_STATE_ERR
exception and abort these steps.SourceBuffer
object and associated resources.sourceBuffers
and queue a task to fire a simple event named addsourcebuffer
at sourceBuffers
.endOfStream
Signals the end of the stream.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
error |
| ✘ | ✔ |
void
When this method is invoked, the user agent must run the following steps:
readyState
attribute is not in the "open"
state then throw an INVALID_STATE_ERR
exception and abort these steps.updating
attribute equals true on any SourceBuffer
in sourceBuffers
, then throw an INVALID_STATE_ERR
exception and abort these steps.isTypeSupported
, staticCheck to see whether the MediaSource
is capable of creating SourceBuffer
objects for the the specified MIME type.
If true is returned from this method, it only indicates that the MediaSource
implementation is capable of creating SourceBuffer
objects for the specified MIME type. An addSourceBuffer()
call may still fail if sufficient resources are not available to support the addition of a new SourceBuffer
.
This method returning true implies that HTMLMediaElement.canPlayType() will return "maybe" or "probably" since it does not make sense for a MediaSource
to support a type the HTMLMediaElement knows it cannot play.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
type | DOMString | ✘ | ✘ |
boolean
When this method is invoked, the user agent must run the following steps:
removeSourceBuffer
Removes a SourceBuffer
from sourceBuffers
.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
sourceBuffer |
| ✘ | ✘ |
void
When this method is invoked, the user agent must run the following steps:
INVALID_ACCESS_ERR
exception and abort these steps.sourceBuffers
then throw a NOT_FOUND_ERR
exception and abort these steps.updating
attribute equals true, then run the following steps:
updating
attribute to false.abort
at sourceBuffer.updateend
at sourceBuffer.AudioTrackList
object returned by sourceBuffer.audioTracks
.AudioTrackList
object returned by the audioTracks
attribute on the HTMLMediaElement.AudioTrack
object in the SourceBuffer audioTracks list, run the following steps:
sourceBuffer
attribute on the AudioTrack
object to null.enabled
attribute on the AudioTrack
object is true, then
set the removed enabled audio track flag to true.AudioTrack
object from the HTMLMediaElement audioTracks list.removetrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the HTMLMediaElement audioTracks list.AudioTrack
object from the SourceBuffer audioTracks list.removetrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the SourceBuffer audioTracks list.change
at the
HTMLMediaElement audioTracks list.VideoTrackList
object returned by sourceBuffer.videoTracks
.VideoTrackList
object returned by the videoTracks
attribute on the HTMLMediaElement.VideoTrack
object in the SourceBuffer videoTracks list, run the following steps:
sourceBuffer
attribute on the VideoTrack
object to null.selected
attribute on the VideoTrack
object is true, then
set the removed selected video track flag to true.VideoTrack
object from the HTMLMediaElement videoTracks list.removetrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the HTMLMediaElement videoTracks list.VideoTrack
object from the SourceBuffer videoTracks list.removetrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the SourceBuffer videoTracks list.change
at the
HTMLMediaElement videoTracks list.TextTrackList
object returned by sourceBuffer.textTracks
.TextTrackList
object returned by the textTracks
attribute on the HTMLMediaElement.TextTrack
object in the SourceBuffer textTracks list, run the following steps:
sourceBuffer
attribute on the TextTrack
object to null.mode
attribute on the TextTrack
object is set to "showing"
or
"hidden"
, then set the removed enabled text track flag to true.TextTrack
object from the HTMLMediaElement textTracks list.removetrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the HTMLMediaElement textTracks list.TextTrack
object from the SourceBuffer textTracks list.removetrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the SourceBuffer textTracks list.change
at the
HTMLMediaElement textTracks list.activeSourceBuffers
, then remove sourceBuffer from activeSourceBuffers
and
queue a task to fire a simple event named removesourcebuffer
at the SourceBufferList
returned by activeSourceBuffers
.sourceBuffers
and queue a task to fire a simple event named removesourcebuffer
at
the SourceBufferList
returned by sourceBuffers
.Event name | Interface | Dispatched when... |
---|---|---|
sourceopen |
Event |
readyState transitions from "closed" to "open" or from "ended" to "open" . |
sourceended |
Event |
readyState transitions from "open" to "ended" . |
sourceclose |
Event |
readyState transitions from "open" to "closed" or "ended" to "closed" . |
A MediaSource
object can be attached to a media element by assigning a MediaSource object URL to the media element src
attribute or the src attribute of a <source> inside a media element. A MediaSource object URL is created by passing a MediaSource object to createObjectURL()
.
If the resource fetch algorithm absolute URL matches the MediaSource object URL, run the following steps right before the "Perform a potentially CORS-enabled fetch" step in the resource fetch algorithm.
readyState
is NOT set to "closed"
readyState
attribute to "open"
.sourceopen
at the MediaSource
.appendBuffer()
and appendStream()
. References to
HTTP in the resource fetch algorithm do not apply because the HTMLMediaElement does not fetch media data via HTTP when a MediaSource
is attached.The following steps are run in any case where the media element is going to transition to NETWORK_EMPTY and queue a task to fire a simple event named emptied at the media element. These steps must be run right before the transition.
readyState
attribute to "closed"
.duration
attribute to NaN.SourceBuffer
objects from activeSourceBuffers
.removesourcebuffer
at activeSourceBuffers
.SourceBuffer
objects from sourceBuffers
.removesourcebuffer
at sourceBuffers
.sourceclose
at the MediaSource
.Run the following steps as part of the "Wait until the user agent has established whether or not the media data for the new playback position is available, and, if it is, until it has decoded enough data to play back that position" step of the seek algorithm:
SourceBuffer
object in activeSourceBuffers
.
activeSourceBuffers
is missing media segments for the new playback position
HTMLMediaElement.readyState
attribute to HAVE_METADATA
.appendBuffer()
or an appendStream()
call causes the coded frame processing algorithm to set
the HTMLMediaElement.readyState
attribute to a value greater than HAVE_METADATA
.
The web application can use buffered
to determine what the media element needs to resume playback.
The following steps are periodically run during playback to make sure that all of the SourceBuffer
objects in activeSourceBuffers
have enough data to ensure uninterrupted playback. Appending new segments and changes to activeSourceBuffers
also cause these steps to run because they affect the conditions that trigger state transitions.
Having enough data to ensure uninterrupted playback is an implementation specific condition where the user agent
determines that it currently has enough data to play the presentation without stalling for a meaningful period of time. This condition is
constantly evaluated to determine when to transition the media element into and out of the HAVE_ENOUGH_DATA
ready state.
These transitions indicate when the user agent believes it has enough data buffered or it needs more data respectively.
An implementation may choose to use bytes buffered, time buffered, the append rate, or any other metric it sees fit to
determine when it has enough data. The metrics used may change during playback so web applications should only rely on the value of
HTMLMediaElement.readyState
to determine whether more data is needed or not.
When the media element needs more data, the user agent should transition it from HAVE_ENOUGH_DATA
to
HAVE_FUTURE_DATA
early enough for a web application to be able to respond without causing an interruption in playback.
For example, transitioning when the current playback position is 500ms before the end of the buffered data gives the application roughly
500ms to append more data before playback stalls.
buffered
for all objects in activeSourceBuffers
do not contain TimeRanges
for the current playback position:HTMLMediaElement.readyState
attribute to HAVE_METADATA
.HAVE_METADATA
, then queue a task to fire a simple event named loadedmetadata
at the media element.buffered
for all objects in activeSourceBuffers
contain TimeRanges
that include the current playback position and enough data to ensure uninterrupted playback:HTMLMediaElement.readyState
attribute to HAVE_ENOUGH_DATA
.canplaythrough
at the media element.HAVE_CURRENT_DATA
.buffered
for at least one object in activeSourceBuffers
contains a TimeRange
that includes the current playback position but not enough data to ensure uninterrupted playback:HTMLMediaElement.readyState
attribute to HAVE_FUTURE_DATA
.HTMLMediaElement.readyState
was less than HAVE_FUTURE_DATA
, then queue a task to fire a simple event named canplay
at the media element.HAVE_CURRENT_DATA
.buffered
for at least one object in activeSourceBuffers
contains a TimeRange
that ends at the current playback position and does not have a range covering the time immediately after the current position:HTMLMediaElement.readyState
attribute to HAVE_CURRENT_DATA
.HAVE_CURRENT_DATA
, then queue a task to fire a simple event named loadeddata
at the media element.During playback activeSourceBuffers
needs to be updated if the selected video track, the enabled audio tracks, or a text track mode changes. When one or more of these changes occur the following steps need to be followed.
SourceBuffer
associated with the previously selected video track is not associated with any other enabled tracks, run the following steps:
SourceBuffer
from activeSourceBuffers
.removesourcebuffer
at activeSourceBuffers
SourceBuffer
associated with the newly selected video track is not already in activeSourceBuffers
, run the following steps:
SourceBuffer
to activeSourceBuffers
.addsourcebuffer
at activeSourceBuffers
SourceBuffer
associated with this track is not associated with any other enabled or selected track, then run the following steps:SourceBuffer
associated with the audio track from activeSourceBuffers
removesourcebuffer
at activeSourceBuffers
SourceBuffer
associated with this track is not already in activeSourceBuffers
, then run the following steps:
SourceBuffer
associated with the audio track to activeSourceBuffers
addsourcebuffer
at activeSourceBuffers
SourceBuffer
associated with this track is not associated with any other enabled or selected track, then run the following steps:SourceBuffer
associated with the text track from activeSourceBuffers
removesourcebuffer
at activeSourceBuffers
SourceBuffer
associated with this track is not already in activeSourceBuffers
, then run the following steps:
SourceBuffer
associated with the text track to activeSourceBuffers
addsourcebuffer
at activeSourceBuffers
Follow these steps when duration
needs to change to a new duration.
duration
is equal to new duration, then return.duration
.
duration
to new duration.remove(new duration, old duration)
on all objects in sourceBuffers
.
This preserves audio frames and text cues that start before and end after the duration
.
duration
, then run the following steps:
SourceBuffer
objects in sourceBuffers
.duration
to new duration.media controller duration
to new duration and run the HTMLMediaElement duration change algorithm.This algorithm gets called when the application signals the end of stream via an endOfStream()
call or an algorithm needs to
signal a decode error. This algorithm takes an error parameter that indicates whether an error will be signalled.
readyState
attribute value to "ended"
.sourceended
at the MediaSource
.SourceBuffer
objects in sourceBuffers
.This allows the duration to properly reflect the end of the appended media segments. For example, if the duration was explicitly set to 10 seconds and only media segments for 0 to 5 seconds were appended before endOfStream() was called, then the duration will get updated to 5 seconds.
"network"
HTMLMediaElement.readyState
attribute equals HAVE_NOTHING
HTMLMediaElement.readyState
attribute is greater than HAVE_NOTHING
"decode"
HTMLMediaElement.readyState
attribute equals HAVE_NOTHING
HTMLMediaElement.readyState
attribute is greater than HAVE_NOTHING
INVALID_ACCESS_ERR
exception.enum AppendMode {
"segments",
"sequence"
};
Enumeration description | |
---|---|
segments |
The timestamps in the media segment determine where the coded frames are placed in the presentation. Media segments can be appended in any order. |
sequence |
Media segments will be treated as adjacent in time independent of the timestamps in the media segment. Coded frames in a new media segment will be placed immediately after the coded
frames in the previous media segment. The |
interface SourceBuffer : EventTarget {
attribute AppendMode
mode;
readonly attribute boolean updating;
readonly attribute TimeRanges buffered;
attribute double timestampOffset;
readonly attribute AudioTrackList audioTracks;
readonly attribute VideoTrackList videoTracks;
readonly attribute TextTrackList textTracks;
attribute double appendWindowStart;
attribute unrestricted double appendWindowEnd;
void appendBuffer (ArrayBuffer data);
void appendBuffer (ArrayBufferView data);
void appendStream (Stream stream, [EnforceRange] optional unsigned long long maxSize);
void abort ();
void remove (double start, double end);
};
appendWindowEnd
of type unrestricted double, The presentation timestamp for the end of the append window. This attribute is initially set to positive Infinity.
On getting, Return the initial value or the last value that was successfully set.
On setting, run the following steps:
sourceBuffers
attribute of the parent media source, then throw an
INVALID_STATE_ERR
exception and abort these steps.updating
attribute equals true, then throw an INVALID_STATE_ERR
exception and abort these steps.INVALID_ACCESS_ERR
and abort these steps.appendWindowStart
then throw an INVALID_ACCESS_ERR
exception and abort these
steps.appendWindowStart
of type double, The presentation timestamp for the start of the append window. This attribute is initially set to 0.
On getting, Return the initial value or the last value that was successfully set.
On setting, run the following steps:
sourceBuffers
attribute of the parent media source, then throw an
INVALID_STATE_ERR
exception and abort these steps.updating
attribute equals true, then throw an INVALID_STATE_ERR
exception and abort these steps.appendWindowEnd
then throw an INVALID_ACCESS_ERR
exception
and abort these steps.audioTracks
of type AudioTrackList, readonly AudioTrack
objects created by this object.
buffered
of type TimeRanges, readonly Indicates what TimeRanges
are buffered in the SourceBuffer
.
When the attribute is read the following steps must occur:
sourceBuffers
attribute of the parent media source then throw an INVALID_STATE_ERR
exception and abort these steps.mode
of type AppendMode
, Controls how a sequence of media segments are handled. This attribute is initially set to "segments"
when the object is created.
On getting, Return the initial value or the last value that was successfully set.
On setting, run the following steps:
"segments"
or "sequence"
, then throw an INVALID_ACCESS_ERR
exception and abort these
steps.sourceBuffers
attribute of the parent media source, then throw an INVALID_STATE_ERR
exception and
abort these steps.updating
attribute equals true, then throw an INVALID_STATE_ERR
exception and abort these steps.If the readyState
attribute of the parent media source is in the "ended"
state then run the following steps:
readyState
attribute of the parent media source to "open"
sourceopen
at the parent media source.INVALID_STATE_ERR
and abort these steps."sequence"
, then set the group start timestamp to the highest presentation end timestamp.textTracks
of type TextTrackList, readonly TextTrack
objects created by this object.
timestampOffset
of type double, Controls the offset applied to timestamps inside subsequent media segments that are appended to this SourceBuffer
. The timestampOffset
is initially set to 0 which indicates that no offset is being applied.
On getting, Return the initial value or the last value that was successfully set.
On setting, run the following steps:
sourceBuffers
attribute of the parent media source, then throw an INVALID_STATE_ERR
exception and abort these steps.updating
attribute equals true, then throw an INVALID_STATE_ERR
exception and abort these steps.If the readyState
attribute of the parent media source is in the "ended"
state then run the following steps:
readyState
attribute of the parent media source to "open"
sourceopen
at the parent media source.INVALID_STATE_ERR
and abort these steps.mode
attribute equals "sequence"
, then set the group start timestamp to new timestamp offset.updating
of type boolean, readonly Indicates whether the asynchronous continuation of an appendBuffer()
, appendStream()
, or remove()
operation is still being processed. This attribute is initially set to false when the object is created.
videoTracks
of type VideoTrackList, readonly VideoTrack
objects created by this object.
abort
Aborts the current segment and resets the segment parser.
void
When this method is invoked, the user agent must run the following steps:
sourceBuffers
attribute of the parent media source then throw an INVALID_STATE_ERR
exception and abort these steps.readyState
attribute of the parent media source is not in the "open"
state then throw an INVALID_STATE_ERR
exception and abort these steps.updating
attribute equals true, then run the following steps:
updating
attribute to false.abort
at this SourceBuffer
object.updateend
at this SourceBuffer
object.appendWindowStart
to 0.appendWindowEnd
to positive Infinity.appendBuffer
Appends the segment data in an ArrayBuffer[TYPED-ARRAYS] to the source buffer.
The steps for this method are the same as the ArrayBufferView version of appendBuffer()
.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
data | ArrayBuffer | ✘ | ✘ |
void
appendBuffer
Appends the segment data in an ArrayBufferView[TYPED-ARRAYS] to the source buffer.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
data | ArrayBufferView | ✘ | ✘ |
void
When this method is invoked, the user agent must run the following steps:
INVALID_ACCESS_ERR
exception and abort these steps.updating
attribute to true.updatestart
at this SourceBuffer
object.appendStream
Appends segment data to the source buffer from a Stream
[STREAMS-API].
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
stream | Stream | ✘ | ✘ | |
maxSize | unsigned long long | ✘ | ✔ |
void
When this method is invoked, the user agent must run the following steps:
INVALID_ACCESS_ERR
exception and abort these steps.updating
attribute to true.updatestart
at this SourceBuffer
object.remove
Removes media for a specific time range.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
start | double | ✘ | ✘ | |
end | double | ✘ | ✘ |
void
When this method is invoked, the user agent must run the following steps:
duration
, then throw an INVALID_ACCESS_ERR
exception and abort these steps.INVALID_ACCESS_ERR
exception and abort these steps.sourceBuffers
attribute of the parent media source then throw an INVALID_STATE_ERR
exception and abort these steps.updating
attribute equals true, then throw an INVALID_STATE_ERR
exception and abort these steps.If the readyState
attribute of the parent media source is in the "ended"
state then run
the following steps:
readyState
attribute of the parent media source to "open"
sourceopen
at the parent media source .updating
attribute to true.updatestart
at this SourceBuffer
object.updating
attribute to false.update
at this SourceBuffer
object.updateend
at this SourceBuffer
object.A track buffer stores the track descriptions and coded frames for an individual
track. The track buffer is updated as initialization segments and media segments are appended to the
SourceBuffer
.
Each track buffer has a last decode timestamp variable that stores the decode timestamp of the last coded frame appended in the current coded frame group. The variable is initially unset to indicate that no coded frames have been appended yet.
Each track buffer has a last frame duration variable that stores the coded frame duration of the last coded frame appended in the current coded frame group. The variable is initially unset to indicate that no coded frames have been appended yet.
Each track buffer has a highest presentation timestamp variable that stores the highest presentation timestamp encountered in a coded frame appended in the current coded frame group. The variable is initially unset to indicate that no coded frames have been appended yet.
Each track buffer has a need random access point flag variable that keeps track of whether the track buffer is waiting for a random access point coded frame. The variable is initially set to true to indicate that random access point coded frame is needed before anything can be added to the track buffer.
Event name | Interface | Dispatched when... |
---|---|---|
updatestart |
Event |
updating transitions from false to true. |
update |
Event |
The append or remove has successfully completed. updating transitions from true to false. |
updateend |
Event |
The append or remove has ended. |
error |
Event |
An error occurred during the append. updating transitions from true to false. |
abort |
Event |
The append or remove was aborted by an abort() call. updating transitions from true to false. |
All SourceBuffer objects have an internal append state variable that keeps track of the high-level segment parsing state. It is initially set to WAITING_FOR_SEGMENT and can transition to the following states as data is appended.
Append state name | Description |
---|---|
WAITING_FOR_SEGMENT | Waiting for the start of an initialization segment or media segment to be appended. |
PARSING_INIT_SEGMENT | Currently parsing an initialization segment. |
PARSING_MEDIA_SEGMENT | Currently parsing a media segment. |
The input buffer is a byte buffer that is used to hold unparsed bytes across appendBuffer()
and appendStream()
calls. The buffer is empty when the SourceBuffer object is created.
The buffer full flag keeps track of whether appendBuffer()
or
appendStream()
is allowed to accept more bytes. It is set to false when the SourceBuffer object is created and gets updated
as data is appended and removed.
The group start timestamp variable keeps track of the starting timestamp for a new
coded frame group in the "sequence"
mode.
It is unset when the SourceBuffer object is created and gets updated when the mode
attribute equals "sequence"
and the
timestampOffset
attribute is set, or the coded frame processing algorithm runs.
The highest presentation end timestamp variable stores the highest presentation end timestamp encountered in the current coded frame group. It is set to 0 when the SourceBuffer object is created and gets updated by the coded frame processing algorithm.
When the segment parser loop algorithm is invoked, run the following steps:
"decode"
and abort this algorithm.If the append state equals WAITING_FOR_SEGMENT, then run the following steps:
If the append state equals PARSING_INIT_SEGMENT, then run the following steps:
If the append state equals PARSING_MEDIA_SEGMENT, then run the following steps:
"decode"
and abort this algorithm.The frequency at which the coded frame processing algorithm is run is implementation-specific. The coded frame processing algorithm may be called when the input buffer contains the complete media segment or it may be called multiple times as complete coded frames are added to the input buffer.
SourceBuffer
is full and cannot accept more media data, then set the buffer full flag to true.When the parser state needs to be reset, run the following steps:
When an error occurs during an append, run the following steps:
updating
attribute to false.error
at this SourceBuffer
object.updateend
at this SourceBuffer
object.When an append operation begins, the follow steps are run to validate and prepare the SourceBuffer
.
SourceBuffer
has been removed from the sourceBuffers
attribute of the parent media source then throw an INVALID_STATE_ERR
exception and abort these steps.updating
attribute equals true, then throw an INVALID_STATE_ERR
exception and abort these steps.If the readyState
attribute of the parent media source is in the "ended"
state then run the following steps:
readyState
attribute of the parent media source to "open"
sourceopen
at the parent media source .If the buffer full flag equals true, then throw a QUOTA_EXCEEDED_ERR
exception and abort these step.
This is the signal that the implementation was unable to evict enough data to accomodate the append or the append is too big. The web
application should use remove()
to explicitly free up space and/or reduce the size of the append.
When appendBuffer()
is called, the following steps are run to process the appended data.
updating
attribute to false.update
at this SourceBuffer
object.updateend
at this SourceBuffer
object.When a Stream
[STREAMS-API] is passed to appendStream()
, the following steps are run to transfer data from the
Stream
to the SourceBuffer
. This algorithm is initialized with the stream and maxSize parameters
from the appendStream()
call.
If the buffer full flag equals true, then run the append error algorithm and abort this algorithm.
The web application should use remove()
to free up space in the SourceBuffer
.
updating
attribute to false.update
at this SourceBuffer
object.updateend
at this SourceBuffer
object.The following steps are run when the segment parser loop successfully parses a complete initialization segment:
Each SourceBuffer object has an internal first initialization segment flag that tracks whether the first initialization segment has been appended. This flag is set to false when the SourceBuffer is created and updated by the algorithm below.
duration
attribute if it currently equals NaN:
"decode"
and abort these steps."decode"
and abort these steps.
If the first initialization segment flag is false, then run the following steps:
"decode"
and abort these steps.
User agents may consider codecs, that would otherwise be supported, as "not supported" here if the codecs were not
specified in the type parameter passed to addSourceBuffer()
.
For example, MediaSource.isTypeSupported('video/webm;codecs="vp8,vorbis"') may return true, but if
addSourceBuffer()
was called with 'video/webm;codecs="vp8"' and a Vorbis track appears in the
initialization segment, then the user agent may use this step to trigger a decode error.
For each audio track in the initialization segment, run following steps:
AudioTrack
object.id
property on new audio track.
If audioTracks
.length
equals 0, then run
the following steps:
enabled
property on new audio track to true.audioTracks
attribute on this SourceBuffer
object.addtrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the AudioTrackList
object referenced by the
audioTracks
attribute on this SourceBuffer
object.audioTracks
attribute on the HTMLMediaElement.addtrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the AudioTrackList
object referenced by the audioTracks
attribute on the HTMLMediaElement.For each video track in the initialization segment, run following steps:
VideoTrack
object.id
property on new video track.
If videoTracks
.length
equals 0, then run
the following steps:
selected
property on new video track to true.videoTracks
attribute on this SourceBuffer
object.addtrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the VideoTrackList
object referenced by the videoTracks
attribute
on this SourceBuffer
object.videoTracks
attribute on the HTMLMediaElement.addtrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the VideoTrackList
object referenced by the videoTracks
attribute on the
HTMLMediaElement.For each text track in the initialization segment, run following steps:
TextTrack
object with its properties populated with the appropriate
information from the initialization segment.mode
property on new text track equals "showing"
or
"hidden"
, then set active track flag to true.
textTracks
attribute on this SourceBuffer
object.addtrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at textTracks
attribute
on this SourceBuffer
object.textTracks
attribute on the HTMLMediaElement.addtrack
, that does not bubble and is not cancelable, and that uses the TrackEvent
interface, at the TextTrackList
object referenced by the textTracks
attribute on the
HTMLMediaElement.SourceBuffer
to activeSourceBuffers
.addsourcebuffer
at activeSourceBuffers
If the HTMLMediaElement.readyState
attribute is HAVE_NOTHING
, then run the following steps:
sourceBuffers
have first initialization segment flag set to false, then abort
these steps.HTMLMediaElement.readyState
attribute to HAVE_METADATA
.loadedmetadata
at the media element.HTMLMediaElement.readyState
attribute is greater than
HAVE_CURRENT_DATA
, then set the HTMLMediaElement.readyState
attribute to HAVE_METADATA
.
When complete coded frames have been parsed by the segment parser loop then the following steps are run:
For each coded frame in the media segment run the following steps:
Special processing may be needed to determine the presentation and decode timestamps for timed text frames since this information may not be explicilty present in the underlying format or may be dependent on the order of the frames. Some metadata text tracks, like MPEG2-TS PSI data, may only have implied timestamps. Format specific rules for these situations should be in the byte stream format specifications or in separate extension specifications.
Implementations don't have to internally store timestamps in a double precision floating point representation. This representation is used here because it is the represention for timestamps in the HTML spec. The intention here is to make the behavior clear without adding unnecessary complexity to the algorithm to deal with the fact that adding a timestampOffset may cause a timestamp rollover in the underlying timestamp representation used by the byte stream format. Implementations can use any internal timestamp representation they wish, but the addition of timestampOffset should behave in a similar manner to what would happen if a double precision floating point representation was used.
mode
equals "sequence"
and group start timestamp is set, then run the following steps:
timestampOffset
equal to group start timestamp - presentation timestamp.If timestampOffset
is not 0, then run the following steps:
timestampOffset
to the presentation timestamp.timestampOffset
to the decode timestamp."decode"
, and abort these steps.mode
equals "segments"
:mode
equals "sequence"
:appendWindowStart
, then set the need random access point flag to true, drop the
coded frame, and jump to the top of the loop to start processing the next coded frame.
Some implementations may choose to collect some of these coded frames that are outside the append window and use them
to generate a splice at the first coded frame that has a presentation timestamp greater than or equal to appendWindowStart
even if
that frame is not a random access point. Supporting this requires multiple decoders or faster than real-time decoding so for now
this behavior will not be a normative requirement.
appendWindowEnd
, then set the need random access point flag to true, drop the
coded frame, and jump to the top of the loop to start processing the next coded frame.
This is to compensate for minor errors in frame timestamp computations that can appear when converting back and forth between double precision floating point numbers and rationals. This tolerance allows a frame to replace an existing one as long as it is within 1 microsecond of the existing frame's start time. Frames that come slightly before an existing frame are handled by the removal step below.
For example if an I-frame is removed in the previous step, then all P-frames & B-frames that depend on that I-frame should be removed from track buffer. This makes sure that decode dependencies are properly maintained during overlaps.
Removing all coded frames until the next random access point is a conservative estimate of the decoding dependencies since it assumes all frames between the removed frames and the next random access point depended on the frames that were removed.
The greater than check is needed because bidirectional prediction between coded frames can cause presentation timestamp to not be monotonically increasing eventhough the decode timestamps are monotonically increasing.
If the HTMLMediaElement.readyState
attribute is HAVE_METADATA
and the new coded frames cause all objects in activeSourceBuffers
to have media data for the current playback position, then run the following steps:
HTMLMediaElement.readyState
attribute to HAVE_CURRENT_DATA
.HAVE_CURRENT_DATA
, then queue a task to fire a simple event named loadeddata
at the media element.If the HTMLMediaElement.readyState
attribute is HAVE_CURRENT_DATA
and the new coded frames cause all objects in activeSourceBuffers
to have media data beyond the current playback position, then run the following steps:
HTMLMediaElement.readyState
attribute to HAVE_FUTURE_DATA
.canplay
at the media element.If the HTMLMediaElement.readyState
attribute is HAVE_FUTURE_DATA
and the new coded frames cause all objects in activeSourceBuffers
to have enough data to ensure uninterrupted playback, then run the following steps:
HTMLMediaElement.readyState
attribute to HAVE_ENOUGH_DATA
.canplaythrough
at the media element.duration
, then run the duration change algorithm with new duration set to the maximum of the current duration and the highest end timestamp reported by HTMLMediaElement.buffered
.Follow these steps when coded frames for a specific time range need to be removed from the SourceBuffer:
For each track buffer in this source buffer, run the following steps:
duration
If this track buffer has a random access point timestamp that is greater than or equal to end, then update remove end timestamp to that random access point timestamp.
Random access point timestamps can be different across tracks because the dependencies between coded frames within a track are usually different than the dependencies in another track.
If this object is in activeSourceBuffers
, the current playback position is greater than or equal to
start and less than the remove end timestamp, and HTMLMediaElement.readyState
is greater than
HAVE_METADATA
, then set the HTMLMediaElement.readyState
attribute to HAVE_METADATA
and stall playback.
This transition occurs because media data for the current position has been removed. Playback cannot progress until media for the current playback position is appended or the selected/enabled tracks change.
This algorithm is run to free up space in this source buffer when new data is appended.
Implementations may use different methods for selecting removal ranges so web applications should not depend on a
specific behavior. The web application can use the buffered
attribute to observe whether portions of the buffered data have been evicted.
Follow these steps when the coded frame processing algorithm needs to generate a splice frame for two overlapping audio coded frames:
For example, given the following values:
presentation timestamp and decode timestamp are updated to 10.0125 since 10.01255 is closer to 10 + 100/8000 (10.0125) than 10 + 101/8000 (10.012625)
Some implementations may apply fades to/from silence to coded frames on either side of the inserted silence to make the transition less jarring.
This is intended to allow new coded frame to be added to the track buffer as if overlapped frame had not been in the track buffer to begin with.
If the new coded frame is less than 5 milliseconds in duration, then coded frames that are appended after the new coded frame will be needed to properly render the splice.
See the audio splice rendering algorithm for details on how this splice frame is rendered.
The following steps are run when a spliced frame, generated by the audio splice frame algorithm, needs to be rendered by the media element:
Here is a graphical representation of this algorithm.
Follow these steps when the coded frame processing algorithm needs to generate a splice frame for two overlapping timed text coded frames:
This is intended to allow new coded frame to be added to the track buffer as if it hadn't overlapped any frames in track buffer to begin with.
SourceBufferList is a simple container object for SourceBuffer
objects. It provides read-only array access and fires events when the list is modified.
interface SourceBufferList : EventTarget {
readonly attribute unsigned long length;
getter SourceBuffer (unsigned long index);
};
length
of type unsigned long, readonly Indicates the number of SourceBuffer
objects in the list.
SourceBuffer
Allows the SourceBuffer objects in the list to be accessed with an array operator (i.e. []).
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
index | unsigned long | ✘ | ✘ |
getter
When this method is invoked, the user agent must run the following steps:
length
attribute then return undefined and abort these steps.SourceBuffer
object in the list.Event name | Interface | Dispatched when... |
---|---|---|
addsourcebuffer |
Event |
When a SourceBuffer is added to the list. |
removesourcebuffer |
Event |
When a SourceBuffer is removed from the list. |
interface VideoPlaybackQuality {
readonly attribute DOMHighResTimeStamp creationTime;
readonly attribute unsigned long totalVideoFrames;
readonly attribute unsigned long droppedVideoFrames;
readonly attribute unsigned long corruptedVideoFrames;
readonly attribute double totalFrameDelay;
};
corruptedVideoFrames
of type unsigned long, readonly The total number of corrupted frames that have been detected.
creationTime
of type DOMHighResTimeStamp, readonly The timestamp returned by Performance.now() when this object was created.
droppedVideoFrames
of type unsigned long, readonly The total number of frames dropped predecode or dropped because the frame missed its display deadline.
totalFrameDelay
of type double, readonly The sum of all displayed frame delays for all displayed frames. (i.e., Frames included in the totalVideoFrames
count, but not in the droppedVideoFrames
count.
totalVideoFrames
of type unsigned long, readonly The total number of frames that would have been displayed if no frames are dropped.
This section specifies extensions to the URL[FILE-API] object definition.
partial interface URL {
static DOMString createObjectURL (MediaSource
mediaSource);
};
createObjectURL
, staticCreates URLs for MediaSource
objects.
This algorithm is intended to mirror the behavior of the createObjectURL()[FILE-API] method with autoRevoke set to true.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
mediaSource |
| ✘ | ✘ |
DOMString
When this method is invoked, the user agent must run the following steps:
This section specifies what existing attributes on the HTMLMediaElement
must return when a MediaSource
is attached to the element.
The HTMLMediaElement.seekable attribute returns a new static normalized TimeRanges object created based on the following steps:
duration
equals NaNTimeRanges
object.duration
equals positive InfinityHTMLMediaElement.buffered
attribute.duration
.The HTMLMediaElement.buffered
attribute returns a new static normalized TimeRanges object created based on the following steps:
activeSourceBuffers
.length equals 0 then return an empty TimeRanges
object and abort these steps.buffered
for each SourceBuffer
object in activeSourceBuffers
.TimeRange
object containing a single range from 0 to highest end time.SourceBuffer
object in activeSourceBuffers
run the following steps:
buffered
attribute on the current SourceBuffer
.readyState
is "ended"
, then set the end time on the last range in source ranges to
highest end time.This section specifies new attributes and internal state that are being added to the HTMLVideoElement
.
Each HTMLVideoElement
will maintain a total video frame count variable that keeps
track of the total number of frames that have been displayed and dropped. This variable is initialized to 0 when the
element is created and whenever the media element load algorithm is invoked. It is incremented when a video frame is displayed
or when the dropped video frame count is incremented.
Each HTMLVideoElement
will maintain a dropped video frame count variable that keeps
track of the total number of frames that have been dropped. This variable is initialized to 0 when the
element is created and whenever the media element load algorithm is invoked. It is incremented when a video frame is dropped
predecode or when a frame is decoded but dropped because it missed a display deadline.
Each HTMLVideoElement
will maintain a corrupted video frame count variable that keeps
track of the total number of corrupted frames detected. This variable is initialized to 0 when the element is created and whenever the
media element load algorithm is invoked. It is incremented when a corrupted video frame is detected by the decoder. It is up to
the implementation to determine whether to display or drop a corrupted frame. Whichever choice is made, the total video frame count
and dropped video frame count must be updated appropriately.
Each HTMLVideoElement
will maintain a displayed frame delay sum variable that keeps
track of the sum of all displayed frame delays. This variable is initialized to 0 when the element is created and whenever the
media element load algorithm is invoked. When a frame is displayed, its displayed frame delay is computed and added
to this variable.
partial interface HTMLVideoElement {
VideoPlaybackQuality
getVideoPlaybackQuality ();
};
getVideoPlaybackQuality
Provides the current the playback quality metrics.
VideoPlaybackQuality
When this method is invoked, the user agent must run the following steps:
VideoPlaybackQuality
.creationTime
to the value returned by a call to Performance.now().totalVideoFrames
to the current value of the total video frame count.droppedVideoFrames
to the current value of the dropped video frame count.corruptedVideoFrames
to the current value of the corrupted video frame count.totalFrameDelay
to the current value of the displayed frame delay sum.This section specifies extensions to the HTML AudioTrack
definition.
partial interface AudioTrack {
attribute DOMString kind;
attribute DOMString language;
readonly attribute SourceBuffer
? sourceBuffer;
};
kind
of type DOMString, Allows the web application to get and update the track kind
.
On getting, return the current value of the attribute. This is either the value provided when this object was created or the value provided on the last successful set operation.
On setting, run the following steps:
sourceBuffer
attribute on this track is not null, then
queue a task to fire a simple event named change
at
sourceBuffer
.audioTracks
.
change
at the AudioTrackList
object referenced by the audioTracks
attribute on the
HTMLMediaElement.
language
of type DOMString, Allows the web application to get and update the track language
.
On getting, return the current value of the attribute. This is either the value provided when this object was created or the value provided on the last successful set operation.
On setting, run the following steps:
sourceBuffer
attribute on this track is not null, then
queue a task to fire a simple event named change
at
sourceBuffer
.audioTracks
.
change
at the AudioTrackList
object referenced by the audioTracks
attribute on the
HTMLMediaElement.
sourceBuffer
of type SourceBuffer
, readonly , nullableReturns the SourceBuffer
that created this track. Returns null if this track was not created by a SourceBuffer
or the SourceBuffer
has been removed from the sourceBuffers
attribute of its parent media source.
This section specifies extensions to the HTML VideoTrack
definition.
partial interface VideoTrack {
attribute DOMString kind;
attribute DOMString language;
readonly attribute SourceBuffer
? sourceBuffer;
};
kind
of type DOMString, Allows the web application to get and update the track kind
.
On getting, return the current value of the attribute. This is either the value provided when this object was created or the value provided on the last successful set operation.
On setting, run the following steps:
sourceBuffer
attribute on this track is not null, then
queue a task to fire a simple event named change
at
sourceBuffer
.videoTracks
.
change
at the VideoTrackList
object referenced by the videoTracks
attribute on the
HTMLMediaElement.
language
of type DOMString, Allows the web application to get and update the track language
.
On getting, return the current value of the attribute. This is either the value provided when this object was created or the value provided on the last successful set operation.
On setting, run the following steps:
sourceBuffer
attribute on this track is not null, then
queue a task to fire a simple event named change
at
sourceBuffer
.videoTracks
.
change
at the VideoTrackList
object referenced by the videoTracks
attribute on the
HTMLMediaElement.
sourceBuffer
of type SourceBuffer
, readonly , nullableReturns the SourceBuffer
that created this track. Returns null if this track was not created by a SourceBuffer
or the SourceBuffer
has been removed from the sourceBuffers
attribute of its parent media source.
This section specifies extensions to the HTML TextTrack
definition.
partial interface TextTrack {
attribute DOMString kind;
attribute DOMString language;
readonly attribute SourceBuffer
? sourceBuffer;
};
kind
of type DOMString, Allows the web application to get and update the track kind
.
On getting, return the current value of the attribute. This is either the value provided when this object was created or the value provided on the last successful set operation.
On setting, run the following steps:
sourceBuffer
attribute on this track is not null, then
queue a task to fire a simple event named change
at
sourceBuffer
.textTracks
.
change
at the TextTrackList
object referenced by the textTracks
attribute on the
HTMLMediaElement.
language
of type DOMString, Allows the web application to get and update the track language
.
On getting, return the current value of the attribute. This is either the value provided when this object was created or the value provided on the last successful set operation.
On setting, run the following steps:
sourceBuffer
attribute on this track is not null, then
queue a task to fire a simple event named change
at
sourceBuffer
.textTracks
.
change
at the TextTrackList
object referenced by the textTracks
attribute on the
HTMLMediaElement.
sourceBuffer
of type SourceBuffer
, readonly , nullableReturns the SourceBuffer
that created this track. Returns null if this track was not created by a SourceBuffer
or the SourceBuffer
has been removed from the sourceBuffers
attribute of its parent media source.
The bytes provided through appendBuffer()
and appendStream()
for a SourceBuffer
form a logical byte stream. The format and
semantics of these byte streams are defined in byte stream format specifications.
The byte stream format registry provides mappings between a MIME type that may be passed to addSourceBuffer()
or
isTypeSupported()
and the byte stream format expected by a SourceBuffer
created with that MIME type. Implementations are encouraged to register
mappings for byte stream formats they support to facilitate interoperability. The byte stream format registry is the authoritative source for these
mappings. If an implementation claims to support a MIME type listed in the registry, its SourceBuffer
implementation must conform to the
byte stream format specification listed in the registry entry.
The byte stream format specifications in the registry are not intended to define new storage formats. They simply outline the subset of existing storage format structures that implementations of this specification will accept.
Byte stream format parsing and validation is implemented in the segment parser loop algorithm.
This section provides general requirements for all byte stream format specifications:
"decode"
when any of the following conditions are met:
The number and type of tracks are not consistent.
For example, if the first initialization segment has 2 audio tracks and 1 video track, then all initialization segments that follow it in the byte stream must describe 2 audio tracks and 1 video track.
Codecs changes across initialization segments.
For example, a byte stream that starts with an initialization segment that specifies a single AAC track and later contains an initialization segment that specifies a single AMR-WB track is not allowed. Support for multiple codecs is handled with multiple SourceBuffer
objects.
Video frame size changes. The user agent must support seamless playback.
This will cause the <video> display region to change size if the web application does not use CSS or HTML attributes (width/height) to constrain the element size.
Audio channel count changes. The user agent may support this seamlessly and could trigger downmixing.
This is a quality of implementation issue because changing the channel count may require reinitializing the audio device, resamplers, and channel mixers which tends to be audible.
buffered
attribute.
This is intended to simplify switching between audio streams where the frame boundaries don't always line up across encodings (e.g. Vorbis).
"decode"
when any combination of an initialization segment and any contiguous sequence of media segments satisfies the
following conditions:
For example, if I1 is associated with M1, M2, M3 then the above must hold for all the combinations I1+M1, I1+M2, I1+M1+M2, I1+M2+M3, etc.
Byte stream specifications must at a minimum define constraints which ensure that the above requirements hold. Additional constraints may be defined, for example to simplify implementation.
Example use of the Media Source Extensions
<script> function onSourceOpen(videoTag, e) { var mediaSource = e.target; if (mediaSource.sourceBuffers.length > 0) return; var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"'); videoTag.addEventListener('seeking', onSeeking.bind(videoTag, mediaSource)); videoTag.addEventListener('progress', onProgress.bind(videoTag, mediaSource)); var initSegment = GetInitializationSegment(); if (initSegment == null) { // Error fetching the initialization segment. Signal end of stream with an error. mediaSource.endOfStream("network"); return; } // Append the initialization segment. var firstAppendHandler = function(e) { var sourceBuffer = e.target; sourceBuffer.removeEventListener('updateend', firstAppendHandler); // Append some initial media data. appendNextMediaSegment(mediaSource); }; sourceBuffer.addEventListener('updateend', firstAppendHandler); sourceBuffer.appendBuffer(initSegment); } function appendNextMediaSegment(mediaSource) { if (mediaSource.readyState == "closed") return; // If we have run out of stream data, then signal end of stream. if (!HaveMoreMediaSegments()) { mediaSource.endOfStream(); return; } // Make sure the previous append is not still pending. if (mediaSource.sourceBuffers[0].updating) return; var mediaSegment = GetNextMediaSegment(); if (!mediaSegment) { // Error fetching the next media segment. mediaSource.endOfStream("network"); return; } // NOTE: If mediaSource.readyState == “ended”, this appendBuffer() call will // cause mediaSource.readyState to transition to "open". The web application // should be prepared to handle multiple “sourceopen” events. mediaSource.sourceBuffers[0].appendBuffer(mediaSegment); } function onSeeking(mediaSource, e) { var video = e.target; if (mediaSource.readyState == "open") { // Abort current segment append. mediaSource.sourceBuffers[0].abort(); } // Notify the media segment loading code to start fetching data at the // new playback position. SeekToMediaSegmentAt(video.currentTime); // Append a media segment from the new playback position. appendNextMediaSegment(mediaSource); } function onProgress(mediaSource, e) { appendNextMediaSegment(mediaSource); } </script> <video id="v" autoplay> </video> <script> var video = document.getElementById('v'); var mediaSource = new MediaSource(); mediaSource.addEventListener('sourceopen', onSourceOpen.bind(this, video)); video.src = window.URL.createObjectURL(mediaSource); </script>
Version | Comment |
---|---|
02 December 2013 |
|
14 November 2013 |
|
04 November 2013 |
|
29 October 2013 |
|
28 October 2013 |
|
15 October 2013 |
|
04 October 2013 |
|
26 July 2013 |
|
18 July 2013 |
|
02 July 2013 |
|
05 June 2013 |
|
01 June 2013 |
|
13 May 2013 |
|
06 May 2013 |
|
02 May 2013 |
|
30 April 2013 |
|
24 April 2013 |
|
08 April 2013 |
|
26 March 2013 |
|
12 March 2013 |
|
05 March 2013 |
|
25 February 2013 |
|
19 February 2013 |
|
05 February 2013 |
|
31 January 2013 |
|
30 January 2013 |
|
15 January 2013 | Replace setTrackInfo() and getSourceBuffer() with AudioTrack, VideoTrack, and TextTrack extensions. |
04 January 2013 |
|
14 December 2012 | Pubrules, Link Checker, and Markup Validation fixes. |
13 December 2012 |
|
08 December 2012 |
|
06 December 2012 |
|
28 November 2012 |
|
09 November 2012 | Converted document to ReSpec. |
18 October 2012 | Refactored SourceBuffer.append() & added SourceBuffer.remove(). |
8 October 2012 |
|
1 October 2012 | Fixed various addsourcebuffer & removesourcebuffer bugs and allow append() in ended state. |
13 September 2012 | Updated endOfStream() behavior to change based on the value of HTMLMediaElement.readyState. |
24 August 2012 |
|
22 August 2012 |
|
17 August 2012 | Minor editorial fixes. |
09 August 2012 | Change presentation start time to always be 0 instead of using format specific rules about the first media segment appended. |
30 July 2012 | Added SourceBuffer.timestampOffset and MediaSource.duration. |
17 July 2012 | Replaced SourceBufferList.remove() with MediaSource.removeSourceBuffer(). |
02 July 2012 | Converted to the object-oriented API |
26 June 2012 | Converted to Editor's draft. |
0.5 | Minor updates before proposing to W3C HTML-WG. |
0.4 | Major revision. Adding source IDs, defining buffer model, and clarifying byte stream formats. |
0.3 | Minor text updates. |
0.2 | Updates to reflect initial WebKit implementation. |
0.1 | Initial Proposal |