Attaching to a media element

A MediaSource object can be attached to a media element by assigning a [=MediaSource object URL=] to the media element {{HTMLMediaElement/src}} attribute or the src attribute of a <source> inside a media element. A [=MediaSource object URL=] is created by passing a MediaSource object to {{URL/createObjectURL()}}.

If the was invoked with a media provider object that is a MediaSource object or a URL record whose object is a MediaSource object, then let mode be local, skip the first step in the (which may otherwise set mode to remote) and add the steps and clarifications below to the section of the .

The 's first step is expected to eventually align with selecting local mode for URL records whose objects are media provider objects. The intent is that if the HTMLMediaElement's src attribute or selected child <source>'s src attribute is a blob: URL matching a [=MediaSource object URL=] when the respective src attribute was last changed, then that MediaSource object is used as the media provider object and current media resource in the local mode logic in the . This also means that the remote mode logic that includes observance of any preload attribute is skipped when a MediaSource object is attached. Even with that eventual change to [[HTML]], the execution of the following steps at the beginning of the local mode logic is still required when the current media resource is a MediaSource object.

Relative to the action which triggered the media element's resource selection algorithm, these steps are asynchronous. The resource fetch algorithm is run after the task that invoked the resource selection algorithm is allowed to continue and a stable state is reached. Implementations may delay the steps in the "Otherwise" clause, below, until the MediaSource object is ready for use.

If {{MediaSource/readyState}} is NOT set to {{ReadyState/""closed""}}

Run the steps of the 's .

Otherwise

1. Set the media element's to false.

2. If the {{MediaSource}} was constructed in a {{DedicatedWorkerGlobalScope}}, then setup worker attachment communication and open the {{MediaSource}}:

   1. Set {{HTMLMediaElement/[[channel with worker]]}} to be a new {{MessageChannel}}.
   2. Set {{HTMLMediaElement/[[port to worker]]}} to the {{MessageChannel/port1}} value of {{HTMLMediaElement/[[channel with worker]]}}.
   3. Execute [=StructuredSerializeWithTransfer=] with the {{MessageChannel/port2}} of {{HTMLMediaElement/[[channel with worker]]}} as both the value and the sole member of the |transferList|, and let the result be |serialized port2:MessagePort|.
   4. [=Queue a task=] on the {{MediaSource}}'s {{DedicatedWorkerGlobalScope}} that will
      1. Execute [=StructuredDeserializeWithTransfer=] with |serialized port2| and {{DedicatedWorkerGlobalScope}}'s [=environment settings object/realm=], and set {{MediaSource/[[port to main]]}} to be the resulting deserialized clone of the transferred {{MessageChannel/port2}} value of {{HTMLMediaElement/[[channel with worker]]}}.
      2. Set the {{MediaSource/readyState}} attribute to {{ReadyState/""open""}}.
      3. [=Queue a task=] to [=fire an event=] named {{sourceopen}} at the MediaSource.

   Otherwise, the {{MediaSource}} was constructed in a {{Window}}:

   1. Set {{HTMLMediaElement/[[channel with worker]]}} null.
   2. Set {{HTMLMediaElement/[[port to worker]]}} null.
   3. Set {{MediaSource/[[port to main]]}} null.
   4. Set the {{MediaSource/readyState}} attribute to {{ReadyState/""open""}}.
   5. [=Queue a task=] to [=fire an event=] named {{sourceopen}} at the MediaSource.

3. Continue the by running the remaining steps, with these clarifications:
   1. Text in the or the that refers to "the download", "bytes received", or "whenever new data for the current media resource becomes available" refers to data passed in via {{SourceBuffer/appendBuffer()}} or {{SourceBuffer/appendEncodedChunks()}}.
   2. References to HTTP in the and the do not apply because the HTMLMediaElement does not fetch media data via HTTP when a MediaSource is attached.

An attached MediaSource does not use the remote mode steps in the , so the media element will not fire "suspend" events. Though future versions of this specification will likely remove "progress" and "stalled" events from a media element with an attached MediaSource, user agents conforming to this version of the specification may still fire these two events as these [[HTML]] references changed after implementations of this specification stabilized.

Detaching from a media element

The following steps are run in any case where the media element is going to transition to {{HTMLMediaElement/NETWORK_EMPTY}} and [=queue a task=] to [=fire an event=] named [=HTMLMediaElement/emptied=] at the media element. These steps SHOULD be run right before the transition.

1. If the {{MediaSource}} was constructed in a {{DedicatedWorkerGlobalScope}}:

   1. Notify the {{MediaSource}} using an internal detach message posted to {{HTMLMediaElement/[[port to worker]]}}.
   2. Set {{HTMLMediaElement/[[port to worker]]}} null.
   3. Set {{HTMLMediaElement/[[channel with worker]]}} null.
   4. The implicit message handler for this detach notification runs the remainder of these steps in the {{DedicatedWorkerGlobalScope}} {{MediaSource}}.

   Otherwise, the {{MediaSource}} was constructed in a {{Window}}:

   Continue the remainder of these steps on the {{Window}} {{MediaSource}}.

2. Set {{MediaSource/[[port to main]]}} null.
3. Set the {{MediaSource/readyState}} attribute to {{ReadyState/""closed""}}.
4. Update {{MediaSource/duration}} to NaN.
5. Remove all the SourceBuffer objects from {{MediaSource/activeSourceBuffers}}.
6. [=Queue a task=] to [=fire an event=] named {{removesourcebuffer}} at {{MediaSource/activeSourceBuffers}}.
7. Remove all the SourceBuffer objects from {{MediaSource/sourceBuffers}}.
8. [=Queue a task=] to [=fire an event=] named {{removesourcebuffer}} at {{MediaSource/sourceBuffers}}.
9. [=Queue a task=] to [=fire an event=] named {{sourceclose}} at the MediaSource.

Going forward, this algorithm is intended to be externally called and run in any case where the attached MediaSource, if any, must be detached from the media element. It MAY be called on HTMLMediaElement [[HTML]] operations like load() and resource fetch algorithm failures in addition to, or in place of, when the media element transitions to {{HTMLMediaElement/NETWORK_EMPTY}}. Resource fetch algorithm failures are those which abort either the resource fetch algorithm or the resource selection algorithm, with the exception that the "Final step" [[HTML]] is not considered a failure that triggers detachment.

Seeking

Run the following steps as part of the "Wait until the user agent has established whether or not the media data for the new playback position is available, and, if it is, until it has decoded enough data to play back that position" step of the :

1. The media element looks for [=media segments=] containing the |new playback position:double| in each SourceBuffer object in {{MediaSource/activeSourceBuffers}}. Any position within a {{TimeRanges}} in the current value of the {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} attribute has all necessary media segments buffered for that position.

   If |new playback position| is not in any {{TimeRanges}} of {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}}

   1. If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is greater than {{HTMLMediaElement/HAVE_METADATA}}, then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_METADATA}}.

      Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

   2. The media element waits until an {{SourceBuffer/appendBuffer()}} or {{SourceBuffer/appendEncodedChunks()}} call causes the [=coded frame processing=] algorithm to set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to a value greater than {{HTMLMediaElement/HAVE_METADATA}}.

      The web application can use {{SourceBuffer/buffered}} and {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} to determine what the media element needs to resume playback.

   Otherwise

   Continue

   If the {{MediaSource/readyState}} attribute is {{ReadyState/""ended""}} and the |new playback position| is within a {{TimeRanges}} currently in {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}}, then the seek operation must continue to completion here even if one or more currently selected or enabled track buffers' largest range end timestamp is less than |new playback position|. This condition should only occur due to logic in {{SourceBuffer/buffered}} when {{MediaSource/readyState}} is {{ReadyState/""ended""}}.

2. The media element resets all decoders and initializes each one with data from the appropriate [=initialization segment=].
3. The media element feeds [=coded frames=] from the [=active track buffers=] into the decoders starting with the closest [=random access point=] before the |new playback position|.
4. Resume the at the "Await a stable state" step.

SourceBuffer Monitoring

The following steps are periodically run during playback to make sure that all of the SourceBuffer objects in {{MediaSource/activeSourceBuffers}} have [=enough data to ensure uninterrupted playback=]. Changes to {{MediaSource/activeSourceBuffers}} also cause these steps to run because they affect the conditions that trigger state transitions.

Having enough data to ensure uninterrupted playback is an implementation specific condition where the user agent determines that it currently has enough data to play the presentation without stalling for a meaningful period of time. This condition is constantly evaluated to determine when to transition the media element into and out of the {{HTMLMediaElement/HAVE_ENOUGH_DATA}} ready state. These transitions indicate when the user agent believes it has enough data buffered or it needs more data respectively.

An implementation MAY choose to use bytes buffered, time buffered, the append rate, or any other metric it sees fit to determine when it has enough data. The metrics used MAY change during playback so web applications SHOULD only rely on the value of {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} to determine whether more data is needed or not.

When the media element needs more data, the user agent SHOULD transition it from {{HTMLMediaElement/HAVE_ENOUGH_DATA}} to {{HTMLMediaElement/HAVE_FUTURE_DATA}} early enough for a web application to be able to respond without causing an interruption in playback. For example, transitioning when the current playback position is 500ms before the end of the buffered data gives the application roughly 500ms to append more data before playback stalls.

If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute equals {{HTMLMediaElement/HAVE_NOTHING}}:

1. Abort these steps.

If {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} does not contain a {{TimeRanges}} for the current playback position:

1. Set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_METADATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

2. Abort these steps.

If {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} contains a {{TimeRanges}} that includes the current playback position and [=enough data to ensure uninterrupted playback=]:

1. Set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_ENOUGH_DATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

2. Playback may resume at this point if it was previously suspended by a transition to {{HTMLMediaElement/HAVE_CURRENT_DATA}}.
3. Abort these steps.

If {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} contains a {{TimeRanges}} that includes the current playback position and some time beyond the current playback position, then run the following steps:

1. Set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_FUTURE_DATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

2. Playback may resume at this point if it was previously suspended by a transition to {{HTMLMediaElement/HAVE_CURRENT_DATA}}.
3. Abort these steps.

If {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} contains a {{TimeRanges}} that ends at the current playback position and does not have a range covering the time immediately after the current position:

1. Set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_CURRENT_DATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

2. Playback is suspended at this point since the media element doesn't have enough data to advance the .
3. Abort these steps.

Changes to selected/enabled track state

During playback {{MediaSource/activeSourceBuffers}} needs to be updated if the {{VideoTrack/selected}} video track, the {{AudioTrack/enabled}} audio track(s), or a text track {{TextTrack/mode}} changes. When one or more of these changes occur the following steps need to be followed. Also, when {{MediaSource}} was constructed in a {{DedicatedWorkerGlobalScope}}, then each change that occurs to a {{Window}} mirror of a track created previously by the implicit handler for the internal create track mirror message MUST also be made to the corresponding {{DedicatedWorkerGlobalScope}} track using an internal update track state message posted to {{HTMLMediaElement/[[port to worker]]}} whose implicit handler makes the change and runs the following steps. Likewise, each change that occurs to a {{DedicatedWorkerGlobalScope}} track MUST also be made to the corresponding {{Window}} mirror of the track using an internal update track state message posted to {{MediaSource/[[port to main]]}} whose implicit handler makes the change to the mirror.

If the selected video track changes, then run the following steps:

1. If the SourceBuffer associated with the previously selected video track is not associated with any other enabled tracks, run the following steps:
   1. Remove the SourceBuffer from {{MediaSource/activeSourceBuffers}}.
   2. [=Queue a task=] to [=fire an event=] named {{removesourcebuffer}} at {{MediaSource/activeSourceBuffers}}
2. If the SourceBuffer associated with the newly selected video track is not already in {{MediaSource/activeSourceBuffers}}, run the following steps:
   1. Add the SourceBuffer to {{MediaSource/activeSourceBuffers}}.
   2. [=Queue a task=] to [=fire an event=] named {{addsourcebuffer}} at {{MediaSource/activeSourceBuffers}}

If an audio track becomes disabled and the SourceBuffer associated with this track is not associated with any other enabled or selected track, then run the following steps:

1. Remove the SourceBuffer associated with the audio track from {{MediaSource/activeSourceBuffers}}
2. [=Queue a task=] to [=fire an event=] named {{removesourcebuffer}} at {{MediaSource/activeSourceBuffers}}

If an audio track becomes enabled and the SourceBuffer associated with this track is not already in {{MediaSource/activeSourceBuffers}}, then run the following steps:

1. Add the SourceBuffer associated with the audio track to {{MediaSource/activeSourceBuffers}}
2. [=Queue a task=] to [=fire an event=] named {{addsourcebuffer}} at {{MediaSource/activeSourceBuffers}}

If a text track {{TextTrack/mode}} becomes and the SourceBuffer associated with this track is not associated with any other enabled or selected track, then run the following steps:

1. Remove the SourceBuffer associated with the text track from {{MediaSource/activeSourceBuffers}}
2. [=Queue a task=] to [=fire an event=] named {{removesourcebuffer}} at {{MediaSource/activeSourceBuffers}}

If a text track {{TextTrack/mode}} becomes or and the SourceBuffer associated with this track is not already in {{MediaSource/activeSourceBuffers}}, then run the following steps:

1. Add the SourceBuffer associated with the text track to {{MediaSource/activeSourceBuffers}}
2. [=Queue a task=] to [=fire an event=] named {{addsourcebuffer}} at {{MediaSource/activeSourceBuffers}}

Duration change

Follow these steps when {{MediaSource/duration}} needs to change to a |new duration:unrestricted double|.

1. If the current value of {{MediaSource/duration}} is equal to |new duration|, then return.
2. If |new duration| is less than the highest [=presentation timestamp=] of any buffered [=coded frames=] for all SourceBuffer objects in {{MediaSource/sourceBuffers}}, then throw an {{InvalidStateError}} exception and abort these steps.

   Duration reductions that would truncate currently buffered media are disallowed. When truncation is necessary, use {{SourceBuffer/remove()}} to reduce the buffered range before updating {{MediaSource/duration}}.

3. Let |highest end time:unrestricted double| be the largest [=track buffer ranges=] end time across all the [=track buffers=] across all SourceBuffer objects in {{MediaSource/sourceBuffers}}.
4. If |new duration| is less than |highest end time|, then

   This condition can occur because the [=coded frame removal=] algorithm preserves coded frames that start before the start of the removal range.

   1. Update |new duration| to equal |highest end time|.
5. Update {{MediaSource/duration}} to |new duration|.
6. Update the {{HTMLMediaElement/duration}} to |new duration|.

7. If the {{MediaSource}} was constructed in a {{DedicatedWorkerGlobalScope}}:

   Post an internal duration change message to {{MediaSource//[[port to main]]}} whose implicit handler in {{Window}} runs the .

   Otherwise:

   Run the .

End of stream

This algorithm gets called when the application signals the end of stream via an {{MediaSource/endOfStream()}} call or an algorithm needs to signal a decode error. This algorithm takes an |error:EndOfStreamError| parameter that indicates whether an error will be signalled.

1. Change the {{MediaSource/readyState}} attribute value to {{ReadyState/""ended""}}.
2. [=Queue a task=] to [=fire an event=] named {{sourceended}} at the MediaSource.

3. If |error| is not set

   1. Run the [=duration change=] algorithm with |new duration:unrestricted double| set to the largest [=track buffer ranges=] end time across all the [=track buffers=] across all SourceBuffer objects in {{MediaSource/sourceBuffers}}.

      This allows the duration to properly reflect the end of the appended media segments. For example, if the duration was explicitly set to 10 seconds and only media segments for 0 to 5 seconds were appended before endOfStream() was called, then the duration will get updated to 5 seconds.

   2. Notify the media element that it now has all of the media data.

   If |error| is set to {{EndOfStreamError/""network""}}

   If the {{MediaSource}} was constructed in a {{DedicatedWorkerGlobalScope}}:

   Post an internal network error message to {{MediaSource//[[port to main]]}} whose implicit handler in {{Window}} runs the steps in the following "otherwise" case.

   Otherwise:

   If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute equals {{HTMLMediaElement/HAVE_NOTHING}}

   Run the steps of the 's .

   If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is greater than {{HTMLMediaElement/HAVE_NOTHING}}

   Run the "If the connection is interrupted after some media data has been received, causing the user agent to give up trying to fetch the resource" steps of the 's .

   If |error| is set to {{EndOfStreamError/""decode""}}

   If the {{MediaSource}} was constructed in a {{DedicatedWorkerGlobalScope}}:

   Post an internal decode error message to {{MediaSource/[[port to main]]}} whose implicit handler in {{Window}} runs the steps in the following "otherwise" case.

   Otherwise:

   If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute equals {{HTMLMediaElement/HAVE_NOTHING}}

   Run the "If the media data can be fetched but is found by inspection to be in an unsupported format, or can otherwise not be rendered at all" steps of the 's .

   If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is greater than {{HTMLMediaElement/HAVE_NOTHING}}

   Run the steps of the 's .

Segment Parser Loop

Each {{SourceBuffer}} object has an [[\append state]] internal slot that keeps track of the high-level segment parsing state. It is initially set to [=WAITING_FOR_SEGMENT=] and can transition to the following states as data is appended.

Each {{SourceBuffer}} object has an [[\input buffer]] internal slot that is a byte buffer that holds unparsed bytes across {{SourceBuffer/appendBuffer()}} calls. The buffer is empty when the {{SourceBuffer}} object is created.

Each {{SourceBuffer}} object has a [[\buffer full flag]] internal slot that keeps track of whether {{SourceBuffer/appendBuffer()}} is allowed to accept more bytes or {{SourceBuffer/appendEncodedChunks()}} is allowed to accept more encoded chunks. It is set to false when the {{SourceBuffer}} object is created and gets updated as data is appended and removed.

Each {{SourceBuffer}} object has a [[\group start timestamp]] internal slot that keeps track of the starting timestamp for a new [=coded frame group=] in the {{AppendMode/""sequence""}} mode. It is unset when the SourceBuffer object is created and gets updated when the {{SourceBuffer/mode}} attribute equals {{AppendMode/""sequence""}} and the {{SourceBuffer/timestampOffset}} attribute is set, or the [=coded frame processing=] algorithm runs.

Each {{SourceBuffer}} object has a [[\group end timestamp]] internal slot that stores the highest [=coded frame end timestamp=] across all [=coded frames=] in the current [=coded frame group=]. It is set to 0 when the SourceBuffer object is created and gets updated by the [=coded frame processing=] algorithm.

The {{SourceBuffer/[[group end timestamp]]}} stores the highest [=coded frame end timestamp=] across all [=track buffers=] in a SourceBuffer. Therefore, care should be taken in setting the {{SourceBuffer/mode}} attribute when appending multiplexed segments in which the timestamps are not aligned across tracks.

Each {{SourceBuffer}} object has a [[\generate timestamps flag]] internal slot that is a boolean that keeps track of whether timestamps need to be generated for the [=coded frames=] passed to the [=coded frame processing=] algorithm. This flag is set by {{MediaSource/addSourceBuffer()}} when the {{SourceBuffer}} object is created and is updated by {{SourceBuffer/changeType()}}.

When the segment parser loop algorithm is invoked, run the following steps:

1. Loop Top: If the {{SourceBuffer/[[input buffer]]}} is empty, then jump to the need more data step below.
2. If the {{SourceBuffer/[[input buffer]]}} contains bytes that violate the [=SourceBuffer byte stream format specification=], then run the [=append error=] algorithm and abort this algorithm.
3. Remove any bytes that the [=byte stream format specifications=] say MUST be ignored from the start of the {{SourceBuffer/[[input buffer]]}}.

4. If the {{SourceBuffer/[[append state]]}} equals [=WAITING_FOR_SEGMENT=], then run the following steps:

   1. If the beginning of the {{SourceBuffer/[[input buffer]]}} indicates the start of an [=initialization segment=], set the {{SourceBuffer/[[append state]]}} to [=PARSING_INIT_SEGMENT=].
   2. If the beginning of the {{SourceBuffer/[[input buffer]]}} indicates the start of a [=media segment=], set {{SourceBuffer/[[append state]]}} to [=PARSING_MEDIA_SEGMENT=].
   3. Jump to the loop top step above.

5. If the {{SourceBuffer/[[append state]]}} equals [=PARSING_INIT_SEGMENT=], then run the following steps:

   1. If the {{SourceBuffer/[[input buffer]]}} does not contain a complete [=initialization segment=] yet, then jump to the need more data step below.
   2. Run the [=initialization segment received=] algorithm.
   3. Remove the [=initialization segment=] bytes from the beginning of the {{SourceBuffer/[[input buffer]]}}.
   4. Set {{SourceBuffer/[[append state]]}} to [=WAITING_FOR_SEGMENT=].
   5. Jump to the loop top step above.

6. If the {{SourceBuffer/[[append state]]}} equals [=PARSING_MEDIA_SEGMENT=], then run the following steps:

   1. If the {{SourceBuffer/[[first initialization segment received flag]]}} is false or the {{SourceBuffer/[[pending initialization segment for changeType flag]]}} is true, then run the [=append error=] algorithm and abort this algorithm.
   2. If the {{SourceBuffer/[[input buffer]]}} contains one or more complete [=coded frames=], then run the [=coded frame processing=] algorithm.

      The frequency at which the coded frame processing algorithm is run is implementation-specific. The coded frame processing algorithm MAY be called when the input buffer contains the complete media segment or it MAY be called multiple times as complete coded frames are added to the input buffer.

   3. If this {{SourceBuffer}} is full and cannot accept more media data, then set the {{SourceBuffer/[[buffer full flag]]}} to true.
   4. If the {{SourceBuffer/[[input buffer]]}} does not contain a complete [=media segment=], then jump to the need more data step below.
   5. Remove the [=media segment=] bytes from the beginning of the {{SourceBuffer/[[input buffer]]}}.
   6. Set {{SourceBuffer/[[append state]]}} to [=WAITING_FOR_SEGMENT=].
   7. Jump to the loop top step above.
7. Need more data: Return control to the calling algorithm.

Reset Parser State

When the parser state needs to be reset, run the following steps:

1. If the {{SourceBuffer/[[append state]]}} equals [=PARSING_MEDIA_SEGMENT=] and the {{SourceBuffer/[[input buffer]]}} contains some complete [=coded frames=], then run the [=coded frame processing=] algorithm until all of these complete [=coded frames=] have been processed.
2. Unset the [=last decode timestamp=] on all [=track buffers=].
3. Unset the [=last frame duration=] on all [=track buffers=].
4. Unset the [=highest end timestamp=] on all [=track buffers=].
5. Set the [=need random access point flag=] on all [=track buffers=] to true.
6. If the {{SourceBuffer/mode}} attribute equals {{AppendMode/""sequence""}}, then set the {{SourceBuffer/[[group start timestamp]]}} to the {{SourceBuffer/[[group end timestamp]]}}
7. Remove all bytes from the {{SourceBuffer/[[input buffer]]}}.
8. If {{SourceBuffer/[[input webcodecs configs and chunks]]}} contains any {{EncodedChunks}}, remove them from {{SourceBuffer/[[input webcodecs configs and chunks]]}}, but retain any {{AudioDecoderConfig}} or {{VideoDecoderConfig}} that may be in that internal slot.
   Keeping an unprocessed WebCodecs config in this internal slot lets future {{SourceBuffer/appendEncodedChunks()}} reuse the config in the [=chunks append=] algorithm because there is no other way to retain such an unprocessed config, except perhaps by the app calling {{SourceBuffer/changeType()}} with the config again.
9. Set {{SourceBuffer/[[append state]]}} to [=WAITING_FOR_SEGMENT=].

Append Error

This algorithm is called when an error occurs during an append.

1. Run the [=reset parser state=] algorithm.
2. Set the {{SourceBuffer/updating}} attribute to false.

3. If this {{SourceBuffer}} has a {{Promise}} in {{SourceBuffer/[[pending append chunks promise]]}}:

   Reject that {{Promise}} with an {{AbortError}} {{DOMException}} and unset {{SourceBuffer/[[pending append chunks promise]]}}.

   Otherwise:

   1. [=Queue a task=] to [=fire an event=] named {{error}} at this {{SourceBuffer}} object.
   2. [=Queue a task=] to [=fire an event=] named {{updateend}} at this {{SourceBuffer}} object.

4. Run the [=end of stream=] algorithm with the |error:EndOfStreamError| parameter set to {{EndOfStreamError/""decode""}}.

Prepare Append

When an append operation begins, the following steps are run to validate and prepare the SourceBuffer.

1. If the SourceBuffer has been removed from the {{MediaSource/sourceBuffers}} attribute of the [=parent media source=] then throw an {{InvalidStateError}} exception and abort these steps.
2. If the {{SourceBuffer/updating}} attribute equals true, then throw an {{InvalidStateError}} exception and abort these steps.
3. Let |recent element error:boolean| be determined as follows:

   If the {{MediaSource}} was constructed in a {{Window}}

   Let |recent element error| be true if the {{HTMLMediaElement}}.{{HTMLMediaElement/error}} attribute is not null. If that attribute is null, then let |recent element error| be false.

   Otherwise

   Let |recent element error| be the value resulting from the steps for the {{Window}} case, but run on the {{Window}} {{HTMLMediaElement}} on any change to its {{HTMLMediaElement/error}} attribute and communicated by using {{HTMLMediaElement/[[port to worker]]}} implicit messages. If such a message has not yet been received, then let |recent element error| be false.

4. If |recent element error| is true, then throw an {{InvalidStateError}} exception and abort these steps.

5. If the {{MediaSource/readyState}} attribute of the [=parent media source=] is in the {{ReadyState/""ended""}} state then run the following steps:

   1. Set the {{MediaSource/readyState}} attribute of the [=parent media source=] to {{ReadyState/""open""}}
   2. [=Queue a task=] to [=fire an event=] named {{sourceopen}} at the [=parent media source=].
6. Run the [=coded frame eviction=] algorithm.

7. If the {{SourceBuffer/[[buffer full flag]]}} equals true, then throw a {{QuotaExceededError}} exception and abort these steps.

   This is the signal that the implementation was unable to evict enough data to accommodate the append or the append is too big. The web application SHOULD use {{SourceBuffer/remove()}} to explicitly free up space and/or reduce the size of the append.

Buffer Append

When {{SourceBuffer/appendBuffer()}} is called, the following steps are run to process the appended data.

1. If this {{SourceBuffer}} is currently configured to expect processing of WebCodecs {{EncodedChunks}}, then run the [=append error=] algorithm and abort this algorithm.
   The expectation of processing an appended bytestream versus appended {{EncodedChunks}} is based on the most recently successful execution of the initial {{MediaSource/addSourceBuffer()}} that created this {{SourceBuffer}} or potentially a more recent {{SourceBuffer/changeType()}} call that may have changed the expectation.
2. Run the [=segment parser loop=] algorithm.
3. If the [=segment parser loop=] algorithm in the previous step was aborted, then abort this algorithm.
4. Set the {{SourceBuffer/updating}} attribute to false.
5. [=Queue a task=] to [=fire an event=] named {{update}} at this SourceBuffer object.
6. [=Queue a task=] to [=fire an event=] named {{updateend}} at this SourceBuffer object.

Chunks Append

Each {{SourceBuffer}} object has a [[\pending append chunks promise]] internal slot that stores the promise necessary to communicate the completion of asynchronous steps begun by a call to {{SourceBuffer/appendEncodedChunks()}}. If there is no asynchronous {{SourceBuffer/appendEncodedChunks()}} operation in progress, then this slot is unset.

Each {{SourceBuffer}} object has an [[\input webcodecs configs and chunks]] internal slot that stores a queue of unprocessed {{EncodedChunks}} and at most one {{SourceBufferConfig}}. The {{SourceBufferConfig}} is added if the initial call to {{MediaSource/addSourceBuffer()}} that created this object was provided a {{SourceBufferConfig}}, or if there is a more recent successful call to {{SourceBuffer/changeType()}} on this {{SourceBuffer}} that provided a {{SourceBufferConfig}}. The contents of this slot are processed by the [=chunks append=] algorithm to buffer the necessary decoder configuration and coded frames into the underlying [=track buffer=].

When the chunks append algorithm is invoked, run the following steps to process the appended {{EncodedChunks}} relative to the current {{SourceBufferConfig}}:

1. If this {{SourceBuffer}} is currently configured to expect processing of an appended bytestream, then run the [=append error=] algorithm and abort this algorithm.
   The expectation of processing an appended bytestream versus appended {{EncodedChunks}} is based on the most recently successful execution of the initial {{MediaSource/addSourceBuffer()}} that created this {{SourceBuffer}} or potentially a more recent {{SourceBuffer/changeType()}} call that may have changed the expectation.
2. If there is a {{SourceBufferConfig}} in this {{SourceBuffer}}'s {{SourceBuffer/[[input webcodecs configs and chunks]]}}, then run the following steps:
   1. Let |config:SourceBufferConfig| be the {{SourceBufferConfig}} from {{SourceBuffer/[[input webcodecs configs and chunks]]}}.
      There must be precisely one {{AudioDecoderConfig}} or {{VideoDecoderConfig}} in |config|; this condition was already enforced by the {{MediaSource/addSourceBuffer()}} or {{SourceBuffer/changeType()}} call that placed |config| into {{SourceBuffer/[[input webcodecs configs and chunks]]}}.
   2. Remove the {{SourceBufferConfig}} from {{SourceBuffer/[[input webcodecs configs and chunks]]}}.
   3. Run the [=initialization segment received=] algorithm with |config| denoting a single track configuration. If that algorithm aborts, then abort this algorithm.
3. In the order in which they were added to {{SourceBuffer/[[input webcodecs configs and chunks]]}}, prepare the {{EncodedChunks}} for coded frame processing by running the following steps for each chunk:
   1. Let |chunk:EncodedAudioChunk or EncodedVideoChunk| be the next encoded chunk being prepared, dequeued (removed) from the front of {{SourceBuffer/[[input webcodecs configs and chunks]]}}.
   2. If |chunk| is an {{EncodedAudioChunk}} but the most recently processed |config| by this object was a {{VideoDecoderConfig}}, or if |chunk| is an {{EncodedVideoChunk}} but the most recently processed |config| by this object was an {{AudioDecoderConfig}}, then run the [=append error=] algorithm and abort this algorithm.
   3. Create a coded frame from |chunk|. Treat the frame as a random access point iff |chunk|'s type is "key". Set the frame's decode timestamp to be 0.
      Using a constant decode timestamp disables the [=coded frame processing=] algorithm's discontinuity detection. Note that other mechanisms for signalling discontinuous sequences of frames survive this simplification, with the most reliable yet lightweight being calling {{SourceBuffer/abort()}} when {{SourceBuffer/updating}} is false. A more heavyweight option would be to call {{SourceBuffer/changeType()}}. Also note that, though WebCodecs does not define a decode timestamp attribute for encoded chunks, if reliable buffering of chunks into MSE needs real decode timestamps, this spec may be refined to improve that support.
4. Run the [=coded frame processing=] algorithm with the frames in order from the previous step.
   This algorithm, unlike the [=segment parser loop=] algorithm, deterministically consumes and clears all of its input state and processes all corresponding coded frames. Even the steps which might abort this algorithm early do so after the [=append error=] algorithm clears any remaining input state and rejects the promise.
5. Set the {{SourceBuffer/updating}} attribute to false.
6. Resolve the {{SourceBuffer/[[pending append chunks promise]]}} and unset that internal slot.

Range Removal

Follow these steps when a caller needs to initiate a JavaScript visible range removal operation that blocks other SourceBuffer updates:

1. Let |start:double| equal the starting [=presentation timestamp=] for the removal range, in seconds measured from [=presentation start time=].
2. Let |end:unrestricted double| equal the end [=presentation timestamp=] for the removal range, in seconds measured from [=presentation start time=].
3. Set the {{SourceBuffer/updating}} attribute to true.
4. [=Queue a task=] to [=fire an event=] named {{updatestart}} at this SourceBuffer object.
5. Return control to the caller and run the rest of the steps asynchronously.
6. Run the [=coded frame removal=] algorithm with |start| and |end| as the start and end of the removal range.
7. Set the {{SourceBuffer/updating}} attribute to false.
8. [=Queue a task=] to [=fire an event=] named {{update}} at this SourceBuffer object.
9. [=Queue a task=] to [=fire an event=] named {{updateend}} at this SourceBuffer object.

Initialization Segment Received

The following steps are run when the [=segment parser loop=] successfully parses a complete [=initialization segment=] or the [=chunks append=] algorithm handles a {{SourceBufferConfig}}:

Each SourceBuffer object has a [[\first initialization segment received flag]] internal slot that tracks whether the first [=initialization segment=] has been appended and received by this algorithm. This flag is set to false when the SourceBuffer is created and updated by the algorithm below.

Each SourceBuffer object has a [[\pending initialization segment for changeType flag]] internal slot that tracks whether an [=initialization segment=] is needed since the most recent {{SourceBuffer/changeType()}}. This flag is set to false when the SourceBuffer is created, set to true by {{SourceBuffer/changeType()}} and reset to false by the algorithm below.

1. Update the {{MediaSource/duration}} attribute if it currently equals NaN:

   If the initialization segment contains a duration:

   Run the [=duration change=] algorithm with |new duration:unrestricted double| set to the duration in the initialization segment.

   Otherwise:

   Run the [=duration change=] algorithm with |new duration| set to positive Infinity.

2. If the [=initialization segment=] has no audio, video, or text tracks, then run the [=append error=] algorithm and abort these steps.
3. If the {{SourceBuffer/[[first initialization segment received flag]]}} is true, then run the following steps:
   1. Verify the following properties. If any of the checks fail then run the [=append error=] algorithm and abort these steps.
      • The number of audio, video, and text tracks match what was in the first [=initialization segment=].
      • If more than one track for a single type are present (e.g., 2 audio tracks), then the [=Track IDs=] match the ones in the first [=initialization segment=].
      • The codecs for each track are supported by the user agent.

        User agents MAY consider codecs, that would otherwise be supported, as "not supported" here if the codecs were not specified in |type:DOMString| parameter passed to (a) the most recently successful {{SourceBuffer/changeType()}} on this {{SourceBuffer}} object, or (b) if no successful {{SourceBuffer/changeType()}} has yet occurred on this object, the {{MediaSource/addSourceBuffer()}} that created this {{SourceBuffer}} object. For example, if the most recently successful {{SourceBuffer/changeType()}} was called with 'video/webm' or 'video/webm; codecs="vp8"', and a video track containing vp9 appears in the initialization segment, then the user agent MAY use this step to trigger a decode error even if the other two properties' checks, above, pass. Implementations are encouraged to trigger error in such cases only when the codec is indeed not supported or the other two properties' checks fail. Web authors are encouraged to use {{SourceBuffer/changeType()}}, {{MediaSource/addSourceBuffer()}} and {{MediaSource/isTypeSupported()}} with precise codec parameters to more proactively detect user agent support. {{SourceBuffer/changeType()}} is required if the {{SourceBuffer}} object's bytestream format is changing.

   2. Add the appropriate [=track descriptions=] from this [=initialization segment=] to each of the [=track buffers=].
   3. Set the [=need random access point flag=] on all track buffers to true.
4. Let |active track flag:boolean| equal false.

5. If the {{SourceBuffer/[[first initialization segment received flag]]}} is false, then run the following steps:

   1. If the [=initialization segment=] contains tracks with codecs the user agent does not support, then run the [=append error=] algorithm and abort these steps.

      User agents MAY consider codecs, that would otherwise be supported, as "not supported" here if the codecs were not specified in |type:DOMString| parameter passed to (a) the most recently successful {{SourceBuffer/changeType()}} on this {{SourceBuffer}} object, or (b) if no successful {{SourceBuffer/changeType()}} has yet occurred on this object, the {{MediaSource/addSourceBuffer()}} that created this {{SourceBuffer}} object. For example, MediaSource.isTypeSupported('video/webm;codecs="vp8,vorbis"') may return true, but if {{MediaSource/addSourceBuffer()}} was called with 'video/webm;codecs="vp8"' and a Vorbis track appears in the [=initialization segment=], then the user agent MAY use this step to trigger a decode error. Implementations are encouraged to trigger error in such cases only when the codec is indeed not supported. Web authors are encouraged to use {{SourceBuffer/changeType()}}, {{MediaSource/addSourceBuffer()}} and {{MediaSource/isTypeSupported()}} with precise codec parameters to more proactively detect user agent support. {{SourceBuffer/changeType()}} is required if the {{SourceBuffer}} object's bytestream format is changing.

   2. For each audio track in the [=initialization segment=], run following steps:

      1. Let |audio byte stream track ID| be the [=Track ID=] for the current track being processed.
      2. Let |audio language:DOMString| be a BCP 47 language tag for the language specified in the [=initialization segment=] for this track or an empty string if no language info is present.
      3. If |audio language| equals the 'und' BCP 47 value, then assign an empty string to |audio language|.
      4. Let |audio label:DOMString| be a label specified in the [=initialization segment=] for this track or an empty string if no label info is present.
      5. Let |audio kinds:DOMString sequence| be a sequence of kind strings specified in the [=initialization segment=] for this track or a sequence with a single empty string element in it if no kind information is provided.
      6. For each value in |audio kinds|, run the following steps:
         1. Let |current audio kind:DOMString| equal the value from |audio kinds| for this iteration of the loop.
         2. Let |new audio track:AudioTrack| be a new {{AudioTrack}} object.
         3. Generate a unique ID and assign it to the {{AudioTrack/id}} property on |new audio track|.
         4. Assign |audio language| to the {{AudioTrack/language}} property on |new audio track|.
         5. Assign |audio label| to the {{AudioTrack/label}} property on |new audio track|.
         6. Assign |current audio kind| to the {{AudioTrack/kind}} property on |new audio track|.

         7. If this {{SourceBuffer}} object's {{SourceBuffer/audioTracks}}.{{AudioTrackList/length}} equals 0, then run the following steps:

            1. Set the {{AudioTrack/enabled}} property on |new audio track| to true.
            2. Set |active track flag| to true.
         8. Add |new audio track| to the {{SourceBuffer/audioTracks}} attribute on this SourceBuffer object.

            This should trigger {{AudioTrackList}} [[HTML]] logic to [=queue a task=] to [=fire an event=] named [=AudioTrackList/addtrack=] using {{TrackEvent}} with the {{TrackEvent/track}} attribute initialized to |new audio track|, at the {{AudioTrackList}} object referenced by the {{SourceBuffer/audioTracks}} attribute on this SourceBuffer object.

         9. If the [=parent media source=] was constructed in a {{DedicatedWorkerGlobalScope}}:

            Post an internal create track mirror message to {{MediaSource/[[port to main]]}} whose implicit handler in {{Window}} runs the following steps:

            1. Let |mirrored audio track:AudioTrack| be a new {{AudioTrack}} object.
            2. Assign the same property values to |mirrored audio track| as were determined for |new audio track|.
            3. Add |mirrored audio track| to the {{HTMLMediaElement/audioTracks}} attribute on the HTMLMediaElement.

            Otherwise:

            Add |new audio track| to the {{HTMLMediaElement/audioTracks}} attribute on the HTMLMediaElement.

            This should trigger {{AudioTrackList}} [[HTML]] logic to [=queue a task=] to [=fire an event=] named [=AudioTrackList/addtrack=] using {{TrackEvent}} with the {{TrackEvent/track}} attribute initialized to |mirrored audio track| or |new audio track|, at the {{AudioTrackList}} object referenced by the {{HTMLMediaElement/audioTracks}} attribute on the HTMLMediaElement.

      7. Create a new [=track buffer=] to store [=coded frames=] for this track.
      8. Add the [=track description=] for this track to the [=track buffer=].

   3. For each video track in the [=initialization segment=], run following steps:

      1. Let |video byte stream track ID| be the [=Track ID=] for the current track being processed.
      2. Let |video language:DOMString| be a BCP 47 language tag for the language specified in the [=initialization segment=] for this track or an empty string if no language info is present.
      3. If |video language| equals the 'und' BCP 47 value, then assign an empty string to |video language|.
      4. Let |video label:DOMString| be a label specified in the [=initialization segment=] for this track or an empty string if no label info is present.
      5. Let |video kinds:DOMString sequence| be a sequence of kind strings specified in the [=initialization segment=] for this track or a sequence with a single empty string element in it if no kind information is provided.
      6. For each value in |video kinds|, run the following steps:
         1. Let |current video kind:DOMString| equal the value from |video kinds| for this iteration of the loop.
         2. Let |new video track:VideoTrack| be a new {{VideoTrack}} object.
         3. Generate a unique ID and assign it to the {{VideoTrack/id}} property on |new video track|.
         4. Assign |video language| to the {{VideoTrack/language}} property on |new video track|.
         5. Assign |video label| to the {{VideoTrack/label}} property on |new video track|.
         6. Assign |current video kind| to the {{VideoTrack/kind}} property on |new video track|.

         7. If this {{SourceBuffer}} object's {{SourceBuffer/videoTracks}}.{{VideoTrackList/length}} equals 0, then run the following steps:

            1. Set the {{VideoTrack/selected}} property on |new video track| to true.
            2. Set |active track flag| to true.
         8. Add |new video track| to the {{SourceBuffer/videoTracks}} attribute on this SourceBuffer object.

            This should trigger {{VideoTrackList}} [[HTML]] logic to [=queue a task=] to [=fire an event=] named [=VideoTrackList/addtrack=] using {{TrackEvent}} with the {{TrackEvent/track}} attribute initialized to |new video track|, at the {{VideoTrackList}} object referenced by the {{SourceBuffer/videoTracks}} attribute on this SourceBuffer object.

         9. If the [=parent media source=] was constructed in a {{DedicatedWorkerGlobalScope}}:

            Post an internal create track mirror message to {{MediaSource/[[port to main]]}} whose implicit handler in {{Window}} runs the following steps:

            1. Let |mirrored video track:VideoTrack| be a new {{VideoTrack}} object.
            2. Assign the same property values to |mirrored video track| as were determined for |new video track|.
            3. Add |mirrored video track| to the {{HTMLMediaElement/videoTracks}} attribute on the HTMLMediaElement.

            Otherwise:

            Add |new video track| to the {{HTMLMediaElement/videoTracks}} attribute on the HTMLMediaElement.

            This should trigger {{VideoTrackList}} [[HTML]] logic to [=queue a task=] to [=fire an event=] named [=VideoTrackList/addtrack=] using {{TrackEvent}} with the {{TrackEvent/track}} attribute initialized to |mirrored video track| or |new video track|, at the {{VideoTrackList}} object referenced by the {{HTMLMediaElement/videoTracks}} attribute on the HTMLMediaElement.

      7. Create a new [=track buffer=] to store [=coded frames=] for this track.
      8. Add the [=track description=] for this track to the [=track buffer=].

   4. For each text track in the [=initialization segment=], run following steps:

      1. Let |text byte stream track ID| be the [=Track ID=] for the current track being processed.
      2. Let |text language:DOMString| be a BCP 47 language tag for the language specified in the [=initialization segment=] for this track or an empty string if no language info is present.
      3. If |text language| equals the 'und' BCP 47 value, then assign an empty string to |text language|.
      4. Let |text label:DOMString| be a label specified in the [=initialization segment=] for this track or an empty string if no label info is present.
      5. Let |text kinds:DOMString sequence| be a sequence of kind strings specified in the [=initialization segment=] for this track or a sequence with a single empty string element in it if no kind information is provided.
      6. For each value in |text kinds|, run the following steps:
         1. Let |current text kind:DOMString| equal the value from |text kinds| for this iteration of the loop.
         2. Let |new text track:TextTrack| be a new {{TextTrack}} object.
         3. Generate a unique ID and assign it to the {{TextTrack/id}} property on |new text track|.
         4. Assign |text language| to the {{TextTrack/language}} property on |new text track|.
         5. Assign |text label| to the {{TextTrack/label}} property on |new text track|.
         6. Assign |current text kind| to the {{TextTrack/kind}} property on |new text track|.
         7. Populate the remaining properties on |new text track| with the appropriate information from the [=initialization segment=].
         8. If the {{TextTrack/mode}} property on |new text track| equals or , then set |active track flag| to true.
         9. Add |new text track| to the {{SourceBuffer/textTracks}} attribute on this SourceBuffer object.

            This should trigger {{TextTrackList}} [[HTML]] logic to [=queue a task=] to [=fire an event=] named [=TextTrackList/addtrack=] using {{TrackEvent}} with the {{TrackEvent/track}} attribute initialized to |new text track|, at the {{TextTrackList}} object referenced by the {{SourceBuffer/textTracks}} attribute on this SourceBuffer object.

         10. If the [=parent media source=] was constructed in a {{DedicatedWorkerGlobalScope}}:

             Post an internal create track mirror message to {{MediaSource/[[port to main]]}} whose implicit handler in {{Window}} runs the following steps:

             1. Let |mirrored text track:TextTrack| be a new {{TextTrack}} object.
             2. Assign the same property values to |mirrored text track| as were determined for |new text track|.
             3. Add |mirrored text track| to the {{HTMLMediaElement/textTracks}} attribute on the HTMLMediaElement.

             Otherwise:

             Add |new text track| to the {{HTMLMediaElement/textTracks}} attribute on the HTMLMediaElement.

             This should trigger {{TextTrackList}} [[HTML]] logic to [=queue a task=] to [=fire an event=] named [=TextTrackList/addtrack=] using {{TrackEvent}} with the {{TrackEvent/track}} attribute initialized to |mirrored text track| or |new text track|, at the {{TextTrackList}} object referenced by the {{HTMLMediaElement/textTracks}} attribute on the HTMLMediaElement.

      7. Create a new [=track buffer=] to store [=coded frames=] for this track.
      8. Add the [=track description=] for this track to the [=track buffer=].
   5. If |active track flag| equals true, then run the following steps:
      1. Add this SourceBuffer to {{MediaSource/activeSourceBuffers}}.
      2. [=Queue a task=] to [=fire an event=] named {{addsourcebuffer}} at {{MediaSource/activeSourceBuffers}}
   6. Set {{SourceBuffer/[[first initialization segment received flag]]}} to true.
6. Set {{SourceBuffer/[[pending initialization segment for changeType flag]]}} to false.
7. If the |active track flag| equals true, then run the following steps:

   If the [=parent media source=] was constructed in a {{DedicatedWorkerGlobalScope}}:

   Post an internal active track added message to {{MediaSource/[[port to main]]}} whose implicit handler in {{Window}} runs the following step:

   1. If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is greater than {{HTMLMediaElement/HAVE_CURRENT_DATA}}, then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_METADATA}}.

   Otherwise:

   If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is greater than {{HTMLMediaElement/HAVE_CURRENT_DATA}}, then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_METADATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

8. If each object in {{MediaSource/sourceBuffers}} of the [=parent media source=] has {{SourceBuffer/[[first initialization segment received flag]]}} equal to true, then run the following steps:

   If the [=parent media source=] was constructed in a {{DedicatedWorkerGlobalScope}}:

   Post an internal sourcebuffers ready message to {{MediaSource/[[port to main]]}} whose implicit handler in {{Window}} runs the following step:

   1. If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is {{HTMLMediaElement/HAVE_NOTHING}}, then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_METADATA}}.

   Otherwise:

   If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is {{HTMLMediaElement/HAVE_NOTHING}}, then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_METADATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement. If transition from {{HTMLMediaElement/HAVE_NOTHING}} to {{HTMLMediaElement/HAVE_METADATA}} occurs, it should trigger HTMLMediaElement logic to [=queue a task=] to [=fire an event=] named [=HTMLMediaElement/loadedmetadata=] at the media element.

Coded Frame Processing

When complete [=coded frames=] have been parsed by the [=segment parser loop=] or emitted by the [=chunks append=] algorithm, then the following steps are run:

1. For each [=coded frame=] run the following steps:

   1. Loop Top:

      If {{SourceBuffer/[[generate timestamps flag]]}} equals true:

      1. Let |presentation timestamp:double| equal 0.
      2. Let |decode timestamp:double| equal 0.

      Otherwise:

      1. Let |presentation timestamp| be a double precision floating point representation of the coded frame's [=presentation timestamp=] in seconds.

         Special processing may be needed to determine the presentation and decode timestamps for timed text frames since this information may not be explicitly present in the underlying format or may be dependent on the order of the frames. Some metadata text tracks, like MPEG2-TS PSI data, may only have implied timestamps. Format specific rules for these situations SHOULD be in the [=byte stream format specifications=] or in separate extension specifications.

      2. Let |decode timestamp| be a double precision floating point representation of the coded frame's decode timestamp in seconds.

         Implementations don't have to internally store timestamps in a double precision floating point representation. This representation is used here because it is the representation for timestamps in the HTML spec. The intention here is to make the behavior clear without adding unnecessary complexity to the algorithm to deal with the fact that adding a timestampOffset may cause a timestamp rollover in the underlying timestamp representation used by the byte stream format. Implementations can use any internal timestamp representation they wish, but the addition of timestampOffset SHOULD behave in a similar manner to what would happen if a double precision floating point representation was used.

   2. Let |frame duration:double| be a double precision floating point representation of the [=coded frame duration|coded frame's duration=] in seconds.
   3. If {{SourceBuffer/mode}} equals {{AppendMode/""sequence""}} and {{SourceBuffer/[[group start timestamp]]}} is set, then run the following steps:
      1. Set {{SourceBuffer/timestampOffset}} equal to {{SourceBuffer/[[group start timestamp]]}} minus |presentation timestamp|.
      2. Set {{SourceBuffer/[[group end timestamp]]}} equal to {{SourceBuffer/[[group start timestamp]]}}.
      3. Set the [=need random access point flag=] on all [=track buffers=] to true.
      4. Unset {{SourceBuffer/[[group start timestamp]]}}.

   4. If {{SourceBuffer/timestampOffset}} is not 0, then run the following steps:

      1. Add {{SourceBuffer/timestampOffset}} to the |presentation timestamp|.
      2. Add {{SourceBuffer/timestampOffset}} to the |decode timestamp|.
   5. Let |track buffer| equal the [=track buffer=] that the coded frame will be added to.

   6. If [=last decode timestamp=] for |track buffer| is set and |decode timestamp| is less than [=last decode timestamp=]:

      OR

      If [=last decode timestamp=] for |track buffer| is set and the difference between |decode timestamp| and [=last decode timestamp=] is greater than 2 times [=last frame duration=]:

      1. If {{SourceBuffer/mode}} equals {{AppendMode/""segments""}}:

         Set {{SourceBuffer/[[group end timestamp]]}} to |presentation timestamp|.

         If {{SourceBuffer/mode}} equals {{AppendMode/""sequence""}}:

         Set {{SourceBuffer/[[group start timestamp]]}} equal to the {{SourceBuffer/[[group end timestamp]]}}.

      2. Unset the [=last decode timestamp=] on all [=track buffers=].
      3. Unset the [=last frame duration=] on all [=track buffers=].
      4. Unset the [=highest end timestamp=] on all [=track buffers=].
      5. Set the [=need random access point flag=] on all [=track buffers=] to true.
      6. Jump to the Loop Top step above to restart processing of the current [=coded frame=].

      Otherwise:

      Continue.

   7. Let |frame end timestamp:double| equal the sum of |presentation timestamp| and |frame duration|.
   8. If |presentation timestamp| is less than {{SourceBuffer/appendWindowStart}}, then set the [=need random access point flag=] to true, drop the coded frame, and jump to the top of the loop to start processing the next coded frame.

      Some implementations MAY choose to collect some of these coded frames with |presentation timestamp| less than {{SourceBuffer/appendWindowStart}} and use them to generate a splice at the first coded frame that has a [=presentation timestamp=] greater than or equal to {{SourceBuffer/appendWindowStart}} even if that frame is not a [=random access point=]. Supporting this requires multiple decoders or faster than real-time decoding so for now this behavior will not be a normative requirement.

   9. If |frame end timestamp| is greater than {{SourceBuffer/appendWindowEnd}}, then set the [=need random access point flag=] to true, drop the coded frame, and jump to the top of the loop to start processing the next coded frame.

      Some implementations MAY choose to collect coded frames with |presentation timestamp| less than {{SourceBuffer/appendWindowEnd}} and |frame end timestamp| greater than {{SourceBuffer/appendWindowEnd}} and use them to generate a splice across the portion of the collected coded frames within the append window at time of collection, and the beginning portion of later processed frames which only partially overlap the end of the collected coded frames. Supporting this requires multiple decoders or faster than real-time decoding so for now this behavior will not be a normative requirement. In conjunction with collecting coded frames that span {{SourceBuffer/appendWindowStart}}, implementations MAY thus support gapless audio splicing.

   10. If the [=need random access point flag=] on |track buffer| equals true, then run the following steps:
       1. If the coded frame is not a [=random access point=], then drop the coded frame and jump to the top of the loop to start processing the next coded frame.
       2. Set the [=need random access point flag=] on |track buffer| to false.
   11. Let |spliced audio frame| be an unset variable for holding audio splice information
   12. Let |spliced timed text frame| be an unset variable for holding timed text splice information
   13. If [=last decode timestamp=] for |track buffer| is unset and |presentation timestamp| falls within the [=presentation interval=] of a [=coded frame=] in |track buffer|, then run the following steps:
       1. Let |overlapped frame| be the [=coded frame=] in |track buffer| that matches the condition above.

       2. If |track buffer| contains audio [=coded frames=]:

          Run the [=audio splice frame=] algorithm and if a splice frame is returned, assign it to |spliced audio frame|.

          If |track buffer| contains video [=coded frames=]:

          1. Let |remove window timestamp:double| equal the |overlapped frame| [=presentation timestamp=] plus 1 microsecond.
          2. If the |presentation timestamp| is less than the |remove window timestamp|, then remove |overlapped frame| from |track buffer|.

             This is to compensate for minor errors in frame timestamp computations that can appear when converting back and forth between double precision floating point numbers and rationals. This tolerance allows a frame to replace an existing one as long as it is within 1 microsecond of the existing frame's start time. Frames that come slightly before an existing frame are handled by the removal step below.

          If |track buffer| contains timed text [=coded frames=]:

          Run the [=text splice frame=] algorithm and if a splice frame is returned, assign it to |spliced timed text frame|.

   14. Remove existing coded frames in |track buffer|:

       If [=highest end timestamp=] for |track buffer| is not set:

       Remove all [=coded frames=] from |track buffer| that have a [=presentation timestamp=] greater than or equal to |presentation timestamp| and less than |frame end timestamp|.

       If [=highest end timestamp=] for |track buffer| is set and less than or equal to |presentation timestamp|:

       Remove all [=coded frames=] from |track buffer| that have a [=presentation timestamp=] greater than or equal to [=highest end timestamp=] and less than |frame end timestamp|.

   15. Remove all possible decoding dependencies on the [=coded frames=] removed in the previous two steps by removing all [=coded frames=] from |track buffer| between those frames removed in the previous two steps and the next [=random access point=] after those removed frames.

       Removing all [=coded frames=] until the next [=random access point=] is a conservative estimate of the decoding dependencies since it assumes all frames between the removed frames and the next random access point depended on the frames that were removed.

   16. If |spliced audio frame| is set:

       Add |spliced audio frame| to the |track buffer|.

       If |spliced timed text frame| is set:

       Add |spliced timed text frame| to the |track buffer|.

       Otherwise:

       Add the [=coded frame=] with the |presentation timestamp|, |decode timestamp|, and |frame duration| to the |track buffer|.

   17. Set [=last decode timestamp=] for |track buffer| to |decode timestamp|.
   18. Set [=last frame duration=] for |track buffer| to |frame duration|.
   19. If [=highest end timestamp=] for |track buffer| is unset or |frame end timestamp| is greater than [=highest end timestamp=], then set [=highest end timestamp=] for |track buffer| to |frame end timestamp|.

       The greater than check is needed because bidirectional prediction between coded frames can cause |presentation timestamp| to not be monotonically increasing even though the decode timestamps are monotonically increasing.

   20. If |frame end timestamp| is greater than {{SourceBuffer/[[group end timestamp]]}}, then set {{SourceBuffer/[[group end timestamp]]}} equal to |frame end timestamp|.
   21. If {{SourceBuffer/[[generate timestamps flag]]}} equals true, then set {{SourceBuffer/timestampOffset}} equal to |frame end timestamp|.

2. If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is {{HTMLMediaElement/HAVE_METADATA}} and the new [=coded frames=] cause {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} to have a {{TimeRanges}} for the current playback position, then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_CURRENT_DATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

3. If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is {{HTMLMediaElement/HAVE_CURRENT_DATA}} and the new [=coded frames=] cause {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} to have a {{TimeRanges}} that includes the current playback position and some time beyond the current playback position, then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_FUTURE_DATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

4. If the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute is {{HTMLMediaElement/HAVE_FUTURE_DATA}} and the new [=coded frames=] cause {{HTMLMediaElement}}.{{HTMLMediaElement/buffered}} to have a {{TimeRanges}} that includes the current playback position and [=enough data to ensure uninterrupted playback=], then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_ENOUGH_DATA}}.

   Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

5. If the result of coded frame processing buffers data beyond the current {{MediaSource/duration}}, then run the [=duration change=] algorithm with |new duration:unrestricted double| set to the maximum of the current duration and the {{SourceBuffer/[[group end timestamp]]}}.

Coded Frame Removal

Follow these steps when [=coded frames=] for a specific time range need to be removed from the SourceBuffer:

1. Let |start:double| be the starting [=presentation timestamp=] for the removal range.
2. Let |end:unrestricted double| be the end [=presentation timestamp=] for the removal range.

3. For each [=track buffer=] in this {{SourceBuffer}}, run the following steps:

   1. Let |remove end timestamp:unrestricted double| be the current value of {{MediaSource/duration}}

   2. If this [=track buffer=] has a [=random access point=] timestamp that is greater than or equal to |end|, then update |remove end timestamp| to that random access point timestamp.

      Random access point timestamps can be different across tracks because the dependencies between [=coded frames=] within a track are usually different than the dependencies in another track.

   3. Remove all media data, from this [=track buffer=], that contain starting timestamps greater than or equal to |start| and less than the |remove end timestamp|.

      1. For each removed frame, if the frame has a [=decode timestamp=] equal to the [=last decode timestamp=] for the frame's track, run the following steps:

         If {{SourceBuffer/mode}} equals {{AppendMode/""segments""}}:

         Set {{SourceBuffer/[[group end timestamp]]}} to [=presentation timestamp=].

         If {{SourceBuffer/mode}} equals {{AppendMode/""sequence""}}:

         Set {{SourceBuffer/[[group start timestamp]]}} equal to the {{SourceBuffer/[[group end timestamp]]}}.

      2. Unset the [=last decode timestamp=] on all [=track buffers=].
      3. Unset the [=last frame duration=] on all [=track buffers=].
      4. Unset the [=highest end timestamp=] on all [=track buffers=].
      5. Set the [=need random access point flag=] on all [=track buffers=] to true.
   4. Remove all possible decoding dependencies on the [=coded frames=] removed in the previous step by removing all [=coded frames=] from this [=track buffer=] between those frames removed in the previous step and the next [=random access point=] after those removed frames.

      Removing all [=coded frames=] until the next [=random access point=] is a conservative estimate of the decoding dependencies since it assumes all frames between the removed frames and the next random access point depended on the frames that were removed.

   5. If this object is in {{MediaSource/activeSourceBuffers}}, the is greater than or equal to |start| and less than the |remove end timestamp|, and {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} is greater than {{HTMLMediaElement/HAVE_METADATA}}, then set the {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} attribute to {{HTMLMediaElement/HAVE_METADATA}} and stall playback.

      Per [[HTML]] logic, {{HTMLMediaElement}}.{{HTMLMediaElement/readyState}} changes may trigger events on the HTMLMediaElement.

      This transition occurs because media data for the current position has been removed. Playback cannot progress until media for the is appended or the selected/enabled tracks change.

4. If the {{SourceBuffer/[[buffer full flag]]}} equals true and this object is ready to accept more bytes, then set the {{SourceBuffer/[[buffer full flag]]}} to false.

Coded Frame Eviction

This algorithm is run to free up space in this {{SourceBuffer}} when new data is appended. New data is either the bytes being appended via {{SourceBuffer/appendBuffer()}} or the WebCodecs {{EncodedChunks}} being appended via {{SourceBuffer/appendEncodedChunks()}}.

1. Let |new data:BufferSource or EncodedChunks| equal the data that is about to be appended to this SourceBuffer.

   Need to recognize step here that implementations MAY decide to set {{SourceBuffer/[[buffer full flag]]}} true here if it predicts that processing |new data| in addition to any existing bytes in {{SourceBuffer/[[input buffer]]}} would exceed the capacity of the {{SourceBuffer}}. Such a step enables more proactive push-back from implementations before accepting |new data| which would overflow resources, for example. In practice, at least one implementation already does this.

2. If the {{SourceBuffer/[[buffer full flag]]}} equals false, then abort these steps.
3. Let |removal ranges:normalized TimeRanges| equal a list of presentation time ranges that can be evicted from the presentation to make room for the |new data|.

   Implementations MAY use different methods for selecting |removal ranges| so web applications SHOULD NOT depend on a specific behavior. The web application can use the {{SourceBuffer/buffered}} attribute to observe whether portions of the buffered data have been evicted.

4. For each range in |removal ranges|, run the [=coded frame removal=] algorithm with |start:double| and |end:unrestricted double| equal to the removal range start and end timestamp respectively.

Audio Splice Frame

Follow these steps when the [=coded frame processing=] algorithm needs to generate a splice frame for two overlapping audio [=coded frames=]:

1. Let |track buffer| be the [=track buffer=] that will contain the splice.
2. Let |new coded frame| be the new [=coded frame=], that is being added to |track buffer|, which triggered the need for a splice.
3. Let |presentation timestamp:double| be the [=presentation timestamp=] for |new coded frame|.
4. Let |decode timestamp:double| be the decode timestamp for |new coded frame|.
5. Let |frame duration:double| be the [=coded frame duration=] of |new coded frame|.
6. Let |overlapped frame| be the [=coded frame=] in |track buffer| with a [=presentation interval=] that contains |presentation timestamp|.
7. Update |presentation timestamp| and |decode timestamp| to the nearest audio sample timestamp based on sample rate of the audio in |overlapped frame|. If a timestamp is equidistant from both audio sample timestamps, then use the higher timestamp (e.g., floor(x * sample_rate + 0.5) / sample_rate).

   For example, given the following values:

   • The [=presentation timestamp=] of |overlapped frame| equals 10.
   • The sample rate of |overlapped frame| equals 8000 Hz
   • |presentation timestamp| equals 10.01255
   • |decode timestamp| equals 10.01255

   |presentation timestamp| and |decode timestamp| are updated to 10.0125 since 10.01255 is closer to 10 + 100/8000 (10.0125) than 10 + 101/8000 (10.012625)

8. If the user agent does not support crossfading then run the following steps:
   1. Remove |overlapped frame| from |track buffer|.
   2. Add a silence frame to |track buffer| with the following properties:
      • The [=presentation timestamp=] set to the |overlapped frame| [=presentation timestamp=].
      • The [=decode timestamp=] set to the |overlapped frame| [=decode timestamp=].
      • The [=coded frame duration=] set to difference between |presentation timestamp| and the |overlapped frame| [=presentation timestamp=].

      Some implementations MAY apply fades to/from silence to coded frames on either side of the inserted silence to make the transition less jarring.

   3. Return to caller without providing a splice frame.

      This is intended to allow |new coded frame| to be added to the |track buffer| as if |overlapped frame| had not been in the |track buffer| to begin with.

9. Let |frame end timestamp:double| equal the sum of |presentation timestamp| and |frame duration|.
10. Let |splice end timestamp:double| equal the sum of |presentation timestamp| and the splice duration of 5 milliseconds.
11. Let |fade out coded frames| equal |overlapped frame| as well as any additional frames in |track buffer| that have a [=presentation timestamp=] greater than |presentation timestamp| and less than |splice end timestamp|.
12. Remove all the frames included in |fade out coded frames| from |track buffer|.
13. Return a splice frame with the following properties:
    • The [=presentation timestamp=] set to the |overlapped frame| [=presentation timestamp=].
    • The [=decode timestamp=] set to the |overlapped frame| [=decode timestamp=].
    • The [=coded frame duration=] set to difference between |frame end timestamp| and the |overlapped frame| [=presentation timestamp=].
    • The fade out coded frames equals |fade out coded frames|.
    • The fade in coded frame equals |new coded frame|.

      If the |new coded frame| is less than 5 milliseconds in duration, then coded frames that are appended after the |new coded frame| will be needed to properly render the splice.

    • The splice timestamp equals |presentation timestamp|.

    See the [=audio splice rendering=] algorithm for details on how this splice frame is rendered.

Audio Splice Rendering

The following steps are run when a spliced frame, generated by the [=audio splice frame=] algorithm, needs to be rendered by the media element:

1. Let |fade out coded frames| be the [=coded frames=] that are faded out during the splice.
2. Let |fade in coded frames| be the [=coded frames=] that are faded in during the splice.
3. Let |presentation timestamp:double| be the [=presentation timestamp=] of the first coded frame in |fade out coded frames|.
4. Let |end timestamp:double| be the sum of the [=presentation timestamp=] and the [=coded frame duration=] of the last frame in |fade in coded frames|.
5. Let |splice timestamp:double| be the [=presentation timestamp=] where the splice starts. This corresponds with the [=presentation timestamp=] of the first frame in |fade in coded frames|.
6. Let |splice end timestamp:double| equal |splice timestamp| plus five milliseconds.
7. Let |fade out samples| be the samples generated by decoding |fade out coded frames|.
8. Trim |fade out samples| so that it only contains samples between |presentation timestamp| and |splice end timestamp|.
9. Let |fade in samples| be the samples generated by decoding |fade in coded frames|.
10. If |fade out samples| and |fade in samples| do not have a common sample rate and channel layout, then convert |fade out samples| and |fade in samples| to a common sample rate and channel layout.
11. Let |output samples| be a buffer to hold the output samples.
12. Apply a linear gain fade out with a starting gain of 1 and an ending gain of 0 to the samples between |splice timestamp| and |splice end timestamp| in |fade out samples|.
13. Apply a linear gain fade in with a starting gain of 0 and an ending gain of 1 to the samples between |splice timestamp| and |splice end timestamp| in |fade in samples|.
14. Copy samples between |presentation timestamp| to |splice timestamp| from |fade out samples| into |output samples|.
15. For each sample between |splice timestamp| and |splice end timestamp|, compute the sum of a sample from |fade out samples| and the corresponding sample in |fade in samples| and store the result in |output samples|.
16. Copy samples between |splice end timestamp| to |end timestamp| from |fade in samples| into |output samples|.
17. Render |output samples|.

Here is a graphical representation of this algorithm.

Text Splice Frame

Follow these steps when the [=coded frame processing=] algorithm needs to generate a splice frame for two overlapping timed text [=coded frames=]:

1. Let |track buffer| be the [=track buffer=] that will contain the splice.
2. Let |new coded frame| be the new [=coded frame=], that is being added to |track buffer|, which triggered the need for a splice.
3. Let |presentation timestamp:double| be the [=presentation timestamp=] for |new coded frame|
4. Let |decode timestamp:double| be the decode timestamp for |new coded frame|.
5. Let |frame duration:double| be the [=coded frame duration=] of |new coded frame|.
6. Let |frame end timestamp:double| equal the sum of |presentation timestamp| and |frame duration|.
7. Let |first overlapped frame| be the [=coded frame=] in |track buffer| with a [=presentation interval=] that contains |presentation timestamp|.
8. Let |overlapped presentation timestamp:double| be the [=presentation timestamp=] of the |first overlapped frame|.
9. Let |overlapped frames| equal |first overlapped frame| as well as any additional frames in |track buffer| that have a [=presentation timestamp=] greater than |presentation timestamp| and less than |frame end timestamp|.
10. Remove all the frames included in |overlapped frames| from |track buffer|.
11. Update the [=coded frame duration=] of the |first overlapped frame| to |presentation timestamp| minus |overlapped presentation timestamp|.
12. Add |first overlapped frame| to the |track buffer|.
13. Return to caller without providing a splice frame.

    This is intended to allow |new coded frame| to be added to the |track buffer| as if it hadn't overlapped any frames in |track buffer| to begin with.