Introduction

The two main components in the MediaStream API are the MediaStreamTrack and MediaStream interfaces. The MediaStreamTrack object represents media of a single type that originates from one media source in the User Agent, e.g. video produced by a web camera. A MediaStream is used to group several MediaStreamTrack objects into one unit that can be recorded or rendered in a media element.

Each MediaStream can contain zero or more MediaStreamTrack objects. All tracks in a MediaStream are intended to be synchronized when rendered. This is not a hard requirement, since it might not be possible to synchronize tracks from sources that have different clocks. Different MediaStream objects do not need to be synchronized.

While the intent is to synchronize tracks, it could be better in some circumstances to permit tracks to lose synchronization. In particular, when tracks are remotely sourced and real-time [[WEBRTC10]], it can be better to allow loss of synchronization than to accumulate delays or risk glitches and other artifacts. Implementations are expected to understand the implications of choices regarding synchronization of playback and the effect that these have on user perception.

A single MediaStreamTrack can represent multi-channel content, such as stereo or 5.1 audio or stereoscopic video, where the channels have a well defined relationship to each other. Information about channels might be exposed through other APIs, such as [[WEBAUDIO]], but this specification provides no direct access to channels.

A MediaStream object has an input and an output that represent the combined input and output of all the object's tracks. The output of the MediaStream controls how the object is rendered, e.g., what is saved if the object is recorded to a file or what is displayed if the object is used in a video element. A single MediaStream object can be attached to multiple different outputs at the same time.

A new MediaStream object can be created from existing media streams or tracks using the MediaStream() constructor. The constructor argument can either be an existing MediaStream object, in which case all the tracks of the given stream are added to the new MediaStream object, or an array of MediaStreamTrack objects. The latter form makes it possible to compose a stream from different source streams.

Both MediaStream and MediaStreamTrack objects can be cloned. A cloned MediaStream contains clones of all member tracks from the original stream. A cloned MediaStreamTrack has a set of constraints that is independent of the instance it is cloned from, which allows media from the same source to have different constraints applied for different consumers. The MediaStream object is also used in contexts outside getUserMedia, such as [[WEBRTC10]].

MediaStream

The MediaStream() constructor composes a new stream out of existing tracks. It takes an optional argument of type MediaStream or an array of MediaStreamTrack objects. When the constructor is invoked, the User Agent must run the following steps:

1. Let stream be a newly constructed MediaStream object.

2. Initialize stream's id attribute to a newly generated value.

3. If the constructor's argument is present, construct a set of tracks, tracks based on the type of argument:

   • A MediaStream object:

     Let tracks be a set containing all the MediaStreamTrack objects in the MediaStream track set.

   • A sequence of MediaStreamTrack objects:

     Let tracks be a set containing all the MediaStreamTrack objects in the provided sequence.

4. Add each track in tracks to stream.

5. Return stream.

The tracks of a MediaStream are stored in a track set. The track set MUST contain the MediaStreamTrack objects that correspond to the tracks of the stream. The relative order of the tracks in the set is User Agent defined and the API will never put any requirements on the order. The proper way to find a specific MediaStreamTrack object in the set is to look it up by its id.

An object that reads data from the output of a MediaStream is referred to as a MediaStream consumer. The list of MediaStream consumers currently include media elements (such as <video> and <audio>) [[HTML5]], Web Real-Time Communications (WebRTC; RTCPeerConnection) [[WEBRTC10]], media recording (MediaRecorder) [[mediastream-recording]], image capture (ImageCapture) [[image-capture]], and web audio (MediaStreamAudioSourceNode) [[WEBAUDIO]].

MediaStream consumers must be able to handle tracks being added and removed. This behavior is specified per consumer.

A MediaStream object is said to be active when it has at least one MediaStreamTrack that has not ended. A MediaStream that does not have any tracks or only has tracks that are ended is inactive.

To add a track to a MediaStream, the User Agent MUST run the following steps:

1. Let track be the MediaStreamTrack in question and stream the MediaStream object to which track is to be added.

2. If track is already in stream's track set, then abort these steps.

3. Add track to stream's track set.

To remove a track from a MediaStream, the User Agent MUST run the following steps:

1. Let track be the MediaStreamTrack in question and stream the MediaStream object from which track is to be removed.

2. If track is not in stream's track set, then abort these steps.

3. Remove track from stream's track set.

The User Agent may update a MediaStream's track set in response to, for example, an external event. This specification does not specify any such cases, but other specifications using the MediaStream API may. One such example is the WebRTC 1.0 [[WEBRTC10]] specification where the track set of a MediaStream, received from another peer, can be updated as a result of changes to the media session.

When the User Agent initiates adding a track to a MediaStream, with the exception of initializing a newly created MediaStream with tracks, the User Agent MUST queue a task that runs the following steps:

1. Let track be the MediaStreamTrack in question and stream the MediaStream object to which track is to be added.

2. Add track to stream.

3. If the operation in the previous step was aborted prematurely, then abort these steps.

4. Fire a track event named addtrack with track at stream.

When the User Agent initiates removing a track from a MediaStream, the User Agent MUST queue a task that runs the following steps:

1. Let track be the MediaStreamTrack in question and stream the MediaStream object to which track is to be added.

2. Remove track from stream.

3. If the operation in the previous step was aborted prematurely, then abort these steps.

4. Fire a track event named removetrack with track at stream.

Constructor()

See the MediaStream constructor algorithm

Constructor(MediaStream stream)

See the MediaStream constructor algorithm

Constructor(sequence<MediaStreamTrack> tracks)

See the MediaStream constructor algorithm

readonly attribute DOMString id

When a MediaStream object is created, the User Agent MUST generate an identifier string, and MUST initialize the object's id attribute to that string. A good practice is to use a UUID [[rfc4122]], which is 36 characters long in its canonical form. To avoid fingerprinting, implementations SHOULD use the forms in section 4.4 or 4.5 of RFC 4122 when generating UUIDs.

The id attribute MUST return the value to which it was initialized when the object was created.

sequence<MediaStreamTrack> getAudioTracks()

Returns a sequence of MediaStreamTrack objects representing the audio tracks in this stream.

The getAudioTracks() method MUST return a sequence that represents a snapshot of all the MediaStreamTrack objects in this stream's track set whose kind is equal to "audio". The conversion from the track set to the sequence is User Agent defined and the order does not have to be stable between calls.

sequence<MediaStreamTrack> getVideoTracks()

Returns a sequence of MediaStreamTrack objects representing the video tracks in this stream.

The getVideoTracks() method MUST return a sequence that represents a snapshot of all the MediaStreamTrack objects in this stream's track set whose kind is equal to "video". The conversion from the track set to the sequence is User Agent defined and the order does not have to be stable between calls.

sequence<MediaStreamTrack> getTracks()

Returns a sequence of MediaStreamTrack objects representing all the tracks in this stream.

The getTracks() method MUST return a sequence that represents a snapshot of all the MediaStreamTrack objects in this stream's track set, regardless of kind. The conversion from the track set to the sequence is User Agent defined and the order does not have to be stable between calls.

MediaStreamTrack? getTrackById(DOMString trackId)

The getTrackById() method MUST return either a MediaStreamTrack object from this stream's track set whose id is equal to trackId, or null, if no such track exists.

void addTrack(MediaStreamTrack track)

Adds the given MediaStreamTrack to this MediaStream.

When the addTrack() method is invoked, the User Agent MUST add the track, specified by the method's first argument, to this MediaStream.

void removeTrack(MediaStreamTrack track)

Removes the given MediaStreamTrack object from this MediaStream.

When the removeTrack() method is invoked, the User Agent MUST remove the track, specified by the method's first argument, from this MediaStream.

MediaStream clone()

Clones the given MediaStream and all its tracks.

When the clone() method is invoked, the User Agent MUST run the following steps:

1. Let streamClone be a newly constructed MediaStream object.

2. Initialize streamClone's id attribute to a newly generated value.

3. Clone each track in this MediaStream object and add the result to streamClone's track set.

4. Return streamClone.

readonly attribute boolean active

The active attribute MUST return true if this MediaStream is active and false otherwise.

attribute EventHandler onaddtrack

The event type of this event handler is addtrack.

attribute EventHandler onremovetrack

The event type of this event handler is removetrack.

MediaStreamTrack

A MediaStreamTrack object represents a media source in the User Agent. Several MediaStreamTrack objects can represent the same media source, e.g., when the user chooses the same camera in the UI shown by two consecutive calls to getUserMedia() .

The data from a MediaStreamTrack object does not necessarily have a canonical binary form; for example, it could just be "the video currently coming from the user's video camera". This allows User Agents to manipulate media in whatever fashion is most suitable on the user's platform.

A script can indicate that a MediaStreamTrack object no longer needs its source with the stop() method. When all tracks using a source have been stopped or ended by some other means, the source is stopped. Unless there is a stored permission for the source in question, the given permission is revoked and the User Agent SHOULD also remove the "permission granted" indicator for the source. If the data is being generated from a live source (e.g., a microphone or camera), then the User Agent SHOULD remove any active "on-air" indicator for that source. An implementation may use a per-source reference count to keep track of source usage, but the specifics are out of scope for this specification.

To clone a track the User Agent MUST run the following steps:

1. Let track be the MediaStreamTrack object to be cloned.

2. Let trackClone be a newly constructed MediaStreamTrack object.

3. Initialize trackClone's id attribute to a newly generated value.

4. Initialize trackClone's kind, label, readyState, and enabled attributes by copying the corresponding values from track.

5. Let trackClone's underlying source be the source of track.

6. Set trackClone's constraints to the active constrains of track.

7. Return trackClone.

MediaStreamTrackEvent

The addtrack and removetrack events use the MediaStreamTrackEvent interface.

The addtrack and removetrack events notify the script that the track set of a MediaStream has been updated by the User Agent.

Firing a track event named e with a MediaStreamTrack track means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the MediaStreamTrackEvent interface with the track attribute set to track, MUST be created and dispatched at the given target.

Constructor(DOMString type, MediaStreamTrackEventInit eventInitDict)

Constructs a new MediaStreamTrackEvent.

[SameObject] readonly attribute MediaStreamTrack track

The track attribute represents the MediaStreamTrack object associated with the event.

required MediaStreamTrack track

ECMAScript 6 Terminology

The following terms used in this section are defined in [[!ES6]].

OverconstrainedError Object

The following interface is defined for cases when an OverconstrainedError is raised as an event:

Constructor(DOMString type, OverconstrainedErrorEventInit eventInitDict)

Constructs a new OverconstrainedErrorEvent.

readonly attribute OverconstrainedError? error

The OverconstrainedError describing the error that triggered the event (if any).

OverconstrainedError? error = null

The OverconstrainedError describing the error associated with the event (if any)

ErrorEvent

The following interface is defined for cases when an event is raised that could have been caused by an error:

Firing an error event named e with an Error error means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the ErrorEvent interface with the error attribute set to error, MUST be created and dispatched at the given target. If no Error object is specified, the error attribute defaults to null.

Constructor(DOMString type, ErrorEventInit eventInitDict)

Constructs a new ErrorEvent.

readonly attribute Error? error

If the event was raised because of an error, this attribute may be set to that error object.

Error? error = null

If the event was raised because of an error, this attribute may be set to that error object.

Error names

There is a known issue with this table of error names, because MediaStreamError has been removed but is still needed in the legacy, callback-based getUserMedia interface. This issue will be fixed through resolution of PR #216.

The table below lists the error names defined in this specification.

NavigatorUserMedia

[SameObject] readonly attribute MediaDevices mediaDevices

Returns the MediaDevices object associated with this Navigator object.

MediaDevices

The MediaDevices object is the entry point to the API used to examine and get access to media devices available to the User Agent.

When a new media input or output device is made available, the User Agent MUST queue a task that fires a simple event named devicechange at the MediaDevices object.

attribute EventHandler ondevicechange

The event type of this event handler is devicechange.

Promise<sequence<MediaDeviceInfo>> enumerateDevices ()

Collects information about the User Agent's available media input and output devices.

This method returns a promise. The promise will be fulfilled with a sequence of MediaDeviceInfo dictionaries representing the User Agent's available media input and output devices if enumeration is successful.

Elements of this sequence that represent input devices will be of type InputDeviceInfo which extends MediaDeviceInfo.

Camera and microphone sources should be enumerable. Specifications that add additional types of source will provide recommendations about whether the source type should be enumerable.

When the enumerateDevices() method is called, the User Agent must run the following steps:

1. Let p be a new promise.

2. Run the following steps in parallel:

   1. Let resultList be an empty list.

   2. If this method has been called previously within this browsing session, let oldList be the list of MediaDeviceInfo objects that was produced at that call (resultList); otherwise, let oldList be an empty list.

   3. Probe the User Agent for available media devices, and run the following sub steps for each discovered device, device:

      1. If device is represented by a MediaDeviceInfo object in oldList, append that object to resultList, abort these steps and continue with the next device (if any).

      2. Let deviceInfo be a new MediaDeviceInfo object to represent device.

      3. If device belongs to the same physical device as a device already represented in oldList or resultList, initialize deviceInfo's groupId member to the groupId value of the existing MediaDeviceInfo object. Otherwise, let deviceInfo's groupId member be a newly generated unique identifier.

      4. Append deviceInfo to resultList.

   4. If none of the local devices are attached to an active MediaStreamTrack in the current browsing context, and if no persistent permission to access these local devices has been granted to the page's origin, let filteredList be a copy of resultList, and all its elements, where the label member is the empty string.

   5. If filteredList is a non-empty list, then resolve p with filteredList. Otherwise, resolve p with resultList.

3. Return p.

Since this method returns persistent information across browsing sessions and origins via the number and grouping of media capture devices, it adds to the fingerprinting surface exposed by the user agent.

Once authorization has been granted to one of the capture devices, it provides additional persistent cross-origin information via the human readable labels associated with available capture devices, which furhter adds to the fingerprinting surface.

Device Info

readonly attribute DOMString deviceId

A unique identifier for the represented device.

All enumerable devices have an identifier that MUST be unique to the page's origin. As long as no local device has been attached to an active MediaStreamTrack in a page from this origin, and no persistent permission to access local devices has been granted to this origin, then this identifier MUST NOT be reusable after the end of the current browsing session.

However, if any local devices have been attached to an active MediaStreamTrack in a page from this origin, or persistent permission to access local devices has been granted to this origin, then this identifier MUST be persisted across browsing sessions.

Unique and stable identifiers let the application save, identify the availability of, and directly request specific sources.

This identifier MUST be un-guessable by applications of other origins to prevent the identifier being used to correlate the same user across different origins.

Since deviceId may persist across browsing sessions and to reduce its potential as a fingerprinting mechanism, deviceId is to be treated as other persistent storage mechanisms such as cookies [[COOKIES]]. User agents MUST reset per-origin device identifiers when other persistent storages are cleared.

readonly attribute MediaDeviceKind kind

Describes the kind of the represented device.

readonly attribute DOMString label

A label describing this device (for example "External USB Webcam"). If the device has no associated label, then this attribute MUST return the empty string.

readonly attribute DOMString groupId

Returns the group identifier of the represented device. Two devices have the same group identifier if they belong to the same physical device; for example a monitor with a built-in camera and microphone.

Since groupId may persist across browsing sessions and to reduce its potential as a fingerprinting mechanism, groupId is to be treated as other persistent storage mechanisms such as cookies [[COOKIES]]. User agents MUST reset per-origin device identifiers when other persistent storages are cleared.

serializer = { attribute }

audioinput

Represents an audio input device; for example a microphone.

audiooutput

Represents an audio output device; for example a pair of headphones.

videoinput

Represents a video input device; for example a webcam.

Input-specific Device Info

MediaTrackCapabilities getCapabilities()

Returns a MediaTrackCapabilities object describing the primary audio or video track of a device's MediaStream (according to its kind value), in the absence of any user-supplied constraints. These capabilities MUST be identical to those that would have been obtained by calling getCapabilities() on the first MediaStreamTrack of this type in a MediaStream returned by getUserMedia({deviceId: id}) where id is the value of the deviceId attribute of this MediaDeviceInfo.

If no access has been granted to any local devices and this InputDeviceInfo has been filtered with respect to unique identifying information (see above description of enumerateDevices() result), then this method returns null.

NavigatorUserMedia Interface Extensions

void getUserMedia(MediaStreamConstraints constraints, NavigatorUserMediaSuccessCallback successCallback, NavigatorUserMediaErrorCallback errorCallback)

Prompts the user for permission to use their Web cam or other video or audio input.

The constraints argument is a dictionary of type MediaStreamConstraints.

The successCallback will be invoked with a suitable MediaStream object as its argument if the user accepts valid tracks as described in getUserMedia() on MediaDevices.

The errorCallback will be invoked if there is a failure in finding valid tracks or if the user denies permission, as described in getUserMedia() on MediaDevices.

When the getUserMedia() method is called, the User Agent MUST run the following steps:

1. Let constraints be the method's first argument.

2. Let successCallback be the callback indicated by the method's second argument.

3. Let errorCallback be the callback indicated by the method's third argument.

4. Run the steps specified by the getUserMedia() algorithm with constraints as the argument, and let p be the resulting promise.

5. Upon fulfillment of p with value stream, run the following step:

   1. Invoke successCallback with stream as the argument.

6. Upon rejection of p with reason r, run the following step:

   1. Invoke errorCallback with r as the argument.

MediaDevices Interface Extensions

The getSupportedConstraints method is provided to allow the application to determine which constraints the User Agent recognizes.

MediaTrackSupportedConstraints getSupportedConstraints()

Returns a dictionary whose members are the constrainable properties known to the User Agent. A supported constrainable property MUST be represented and any constrainable properties not supported by the User Agent MUST NOT be present in the returned dictionary. The values returned represent what the browser implements and will not change during a browsing session.

Promise<MediaStream> getUserMedia( MediaStreamConstraints constraints)

Prompts the user for permission to use their Web cam or other video or audio input.

The constraints argument is a dictionary of type MediaStreamConstraints.

This method returns a promise. The promise will be fulfilled with a suitable MediaStream object if the user accepts valid tracks as described below.

The promise will be rejected if there is a failure in finding valid tracks or if the user denies permission, as described below.

When the getUserMedia() method is called, the User Agent MUST run the following steps:

1. Let constraints be the method's first argument.

2. Let requestedMediaTypes be the set of media types in constraints with either a dictionary value or a value of "true".

3. If requestedMediaTypes is the empty set, return a promise rejected with a TypeError.

4. Let p be a new promise.

5. Run the following steps in parallel:

   1. Let finalSet be an (initially) empty set.

   2. For each media type T in requestedMediaTypes,

      1. For each possible source for that media type, construct an unconstrained MediaStreamTrack with that source as its source.

         Call this set of tracks the candidateSet.

         If candidateSet is the empty set, reject p with a new DOMException object whose name attribute has the value NotFoundError and abort these steps.

      2. If the value of the T entry of constraints is "true", set CS to the empty constraint set (no constraint). Otherwise, continue with CS set to the value of the T entry of constraints.

      3. Run the SelectSettings algorithm on each track in CandidateSet with CS as the constraint set. If the algorithm does not return undefined, add the track to finalSet. This eliminates devices unable to satisfy the constraints, by verifying that at least one settings dictionary exists that satisfies the constraints.

      If finalSet is the empty set, let constraint be any required constraint whose fitness distance was infinity for all settings dictionaries examined while executing the SelectSettings algorithm, let message be either undefined or an informative human-readable message, and reject p with a new OverconstrainedError created by calling OverconstrainedError(constraint, message), then abort these steps.

      This error gives information about what the underlying device is not capable of producing, before the user has given any authorization to any device, and can thus be used as a fingerprinting surface.

   3. Optionally, e.g., based on a previously-established user preference, for security reasons, or due to platform limitations, jump to the step labeled Permission Failure below.

   4. Let originIdentifier be the [[!HTML51]] top-level browsing context's origin.

   5. If the current [[!HTML51]] browsing context is a [[!HTML51]] nested browsing context whose origin is different from originIdentifier, let originIdentifier be the result of combining originIdentifier and the current browsing context's origin.

   6. Prompt the user in a User Agent specific manner for permission to provide the origin, identified by originIdentifier, with a MediaStream object representing a media stream.

      The provided media MUST include precisely one track of each media type in requestedMediaTypes from the finalSet. The decision of which tracks to choose from the finalSet is completely up to the User Agent and may be determined by asking the user. Once selected, the source of a MediaStreamTrack MUST NOT change.

      The User Agent MAY use the value of the computed "fitness distance" from the SelectSettings algorithm, or any other internally-available information about the devices, as an input to the selection algorithm.

      User Agents are encouraged to default to using the user's primary or system default camera and/or microphone (when possible) to generate the media stream. User Agents MAY allow users to use any media source, including pre-recorded media files.

      If the user grants permission to use local recording devices, User Agents are encouraged to include a prominent indicator that the devices are "hot" (i.e. an "on-air" or "recording" indicator), as well as a "device accessible" indicator indicating that the page has been granted access to the source.

      If the user denies permission, jump to the step labeled Permission Failure below. If the user never responds, this algorithm stalls on this step.

      If the user grants permission but a hardware error such as an OS/program/webpage lock prevents access, reject p with a new DOMException object whose name attribute has the value NotReadableError and abort these steps.

      If the user grants permission but device access fails for any reason other than those listed above, reject p with a new DOMException object whose name attribute has the value AbortError and abort these steps.

   7. Let stream be the MediaStream object for which the user granted permission.

   8. Run the ApplyConstraints() algorithm on all tracks in stream with the appropriate constraints.

   9. Resolve p with stream and abort these steps.

   10. Permission Failure: Reject p with a new DOMException object whose name attribute has the value SecurityError.

6. Return p.

MediaStreamConstraints

The MediaStreamConstraints dictionary is used to instruct the User Agent what sort of MediaStreamTracks to include in the MediaStream returned by getUserMedia().

(boolean or MediaTrackConstraints) video = false

If true, it requests that the returned MediaStream contain a video track. If a Constraints structure is provided, it further specifies the nature and settings of the video Track. If false, the MediaStream MUST NOT contain a video Track.

(boolean or MediaTrackConstraints) audio = false

If true, it requests that the returned MediaStream contain an audio track. If a Constraints structure is provided, it further specifies the nature and settings of the audio Track. If false, the MediaStream MUST NOT contain an audio Track.

NavigatorUserMediaSuccessCallback

MediaStream stream

MediaStream object representing the stream to which the user granted permission as described in the navigator.getUserMedia() algorithm.

NavigatorUserMediaErrorCallback

MediaStreamError error

Error in obtaining a MediaStream as described in the failure steps of the navigator.getUserMedia() algorithm.

Implementation Suggestions

Interface Definition

Due to the limitations of the interface definition language used in this specification, it is not possible for other interfaces to inherit or implement ConstrainablePattern. Therefore the WebIDL definitions given are only templates to be copied. Each interface that wishes to make use of the functionality defined here will have to provide its own copy of the WebIDL for the functions and interfaces given here. However it can refer to the semantics defined here, which will not change. See MediaStreamTrack Interface Definition for an example of this.

When the User Agent is no longer able to satisfy the requiredConstraints from the currently valid Constraints, the User Agent MUST queue a task that fires an OverconstrainedErrorEvent, initialized as described in the following paragraph, at the constrainable object. The event firing task MAY also be used to update the constrainable object as a result of the overconstrained situation.

The OverconstrainedErrorEvent references an OverconstrainedError whose constraint attribute is set to one of the requiredConstraints that can no longer be satisfied. The message attribute of the OverconstrainedError SHOULD contain a string that is useful for debugging. The conditions under which this error might occur are platform and application-specific. For example, the user might physically manipulate a camera in a way that makes it impossible to provide a resolution or frameRate that satisfies the constraints. The User Agent MAY take other actions as a result of the overconstrained situation.

Capabilities getCapabilities()

The getCapabilities() method returns the dictionary of the names of the constrainable properties that the object supports.

It is possible that the underlying hardware may not exactly map to the range defined for the constrainable property. Where this is possible, the entry SHOULD define how to translate and scale the hardware's setting onto the values defined for the property. For example, suppose that a hypothetical fluxCapacitance property ranges from -10 (min) to 10 (max), but there are common hardware devices that support only values of "off" "medium" and "full". The constrainable property definition might specify that for such hardware, the User Agent should map the range value of -10 to "off", 10 to "full", and 0 to "medium". It might also indicate that given a ConstraintSet imposing a strict value of 3, the User Agent should attempt to set the value of "medium" on the hardware, and that getSettings() should return a fluxCapacitance of 0, since that is the value defined as corresponding to "medium".

Constraints getConstraints()

The getConstraints method returns the Constraints that were the argument to the most recent successful call of applyConstraints(), maintaining the order in which they were specified. Note that some of the optional ConstraintSets returned may not be currently satisfied. To check which ConstraintSets are currently in effect, the application should use getSettings.

Settings getSettings()

The getSettings() method returns the current settings of all the constrainable properties of the object, whether they are platform defaults or have been set by applyConstraints(). Note that the actual setting of a property MUST be a single value.

Promise<void> applyConstraints()

optional Constraints constraints

A new constraint structure to apply to this object.

The applyConstraints() algorithm for applying constraints is stated below. Here are some preliminary definitions that are used in the statement of the algorithm:

We use the term settings dictionary for the set of values that might be applied as settings to the object.

For string valued constraints, we define "==" below to be true if one of the values in the sequence is exactly the same as the value being compared against.

We define the fitness distance between a settings dictionary and a constraint set CS as the sum, for each constraint provided for a constraint name in CS, of the following values:

1. If the constraint is not supported by the browser, the fitness distance is 0.

2. If the constraint is required ('min', 'max', or 'exact'), and the settings dictionary's value for the constraint does not satisfy the constraint, the fitness distance is positive infinity.

3. If no ideal value is specified, the fitness distance is 0.
4. For all positive numeric non-required constraints (such as height, width, frameRate, aspectRatio, sampleRate and sampleSize), the fitness distance is the result of the formula

   (actual == ideal) ? 0 : |actual -
                     ideal|/max(|actual|,|ideal|)
   

5. For all string and enum non-required constraints (sourceId, groupId, facingMode, echoCancellation), the fitness distance is the result of the formula

   (actual == ideal) ? 0 : 1
   

More definitions:

• We refer to each element of a ConstraintSet (other than the special term 'advanced') as a 'constraint' since it is intended to constrain the acceptable settings for the given property from the full list or range given in the corresponding Capability of the ConstrainablePattern object to a value that is within the range or list of values it specifies.
• We refer to the "effective Capability" C of an object O as the possibly proper subset of the possible values of C (as returned by getCapabilities) taking into consideration environmental limitations and/or restrictions placed by other constraints. For example given a ConstraintSet that constrains the aspectRatio, height, and width properties, the values assigned to any two of the properties limit the effective Capability of the third. The set of effective Capabilities may be platform dependent. For example, on a resource-limited device it may not be possible to set properties P1 and P2 both to 'high', while on another less limited device, this may be possible.
• A settings dictionary, which is a set of values for the constrainable properties of an object O, satisfies ConstraintSet CS if the fitness distance between the set and CS is less than infinity.
• A set of ConstraintSets CS1...CSn (n >= 1) can be satisfied by an object O if it is possible to find a settings dictionary of O that satisfies CS1...CSn simultaneously.
• To apply a set of ConstraintSets CS1...CSn to object O is to choose such a sequence of values that satisfy CS1...CSn and assign them as the settings for the properties of O.

We define the SelectSettings algorithm as follows:

1. Each constraint specifies one or more values (or a range of values) for its property. A property MAY appear more than once in the list of 'advanced' ConstraintSets. If an empty object or list has been given as the value for a constraint, it MUST be interpreted as if the constraint were not specified (in other words, an empty constraint == no constraint).

   Note that unknown properties are discarded by WebIDL, which means that unknown/unsupported required constraints will silently disappear. To avoid this being a surprise, application authors are expected to first use the getSupportedConstraints() method as shown in the Examples below.

2. Let object be the ConstrainablePattern object on which this algorithm is applied. Let copy be an unconstrained copy of object (i.e., copy should behave as if it were object with all ConstraintSets removed.)

3. For every possible settings dictionary of copy compute its fitness distance, treating bare values of properties as ideal values. Let candidates be the set of settings dictionaries for which the fitness distance is finite.

4. If candidates is empty, return undefined as the result of the SelectSettings() algorithm.

5. Iterate over the 'advanced' ConstraintSets in newConstraints in the order in which they were specified. For each ConstraintSet:

   1. compute the fitness distance between it and each settings dictionary in candidates, treating bare values of properties as exact.

   2. If the fitness distance is finite for one or more settings dictionaries in candidates, keep those settings dictionaries in candidates, discarding others.

      If the fitness distance is infinite for all settings dictionaries in candidates, ignore this ConstraintSet.

6. Select one settings dictionary from candidates, and return it as the result of the SelectSettings() algorithm. The UA SHOULD use the one with the smallest fitness distance, as calculated in step 3.

When applyConstraints is called, the User Agent MUST run the following steps:

1. Let p be a new promise.

2. Let newContraints be the argument to this function.

3. Run the following steps in parallel:

   1. Let successfulSettings be the result of running the SelectSettings algorithm with newConstraints as the constraint set.

   2. If successfulSettings is undefined, let failedConstraint be any required constraint whose fitness distance was infinity for all settings dictionaries examined while executing the SelectSettings algorithm, let message be either undefined or an informative human-readable message, reject p with a new OverconstrainedError created by calling OverconstrainedError(failedConstraint, message), and abort these steps. The existing constraints remain in effect in this case.

   3. In a single operation, remove the existing constraints from object, apply newConstraints, and apply successfulSettings as the current settings.
   4. Finally, resolve p. From this point on until applyConstraints() is called successfully again, getConstraints() must return the newConstraints that were passed as an argument to this call.

4. Return p.

Any implementation that has the same result as the algorithm above is an allowed implementation. For instance, the implementation may choose to keep track of the maximum and minimum values for a setting that are OK under the constraints considered, rather than keeping track of all possible values for the setting.

When picking a settings dictionary, the UA can use any information available to it. Examples of such information may be whether the selection is done as part of device selection in getUserMedia, whether the energy usage of the camera varies between the settings dictionaries, or whether using a settings dictionary will cause the device driver to apply resampling.

The User Agent MAY choose new settings for the constrainable properties of the object at any time. When it does so it MUST attempt to satisfy the current Constraints, in the manner described in the algorithm above.

attribute EventHandler onoverconstrained

The event type of this event handler is overconstrained.

An example of Constraints that could be passed into applyConstraints() or returned as a value of constraints is below. It uses the constrainable properties defined for MediaStreamTrack.

var supports = navigator.mediaDevices.getSupportedConstraints();
if(!supports["facingMode"]) {
  // Treat like an error.
}
var constraints = {
  width: {
    min: 640
  },
  height: {
    min: 480
  },
  advanced: [{
      width: 650
    }, {
      width: {
        min: 650
      }
    }, {
      frameRate: 60
    }, {
      width: {
        max: 800
      }
    }, {
      facingMode: "user"
    }]
};

Here is another example, specifically for a video track where I must have a particular camera and have separate preferences for the width and height:

var supports = navigator.mediaDevices.getSupportedConstraints();
if(!supports["deviceId"]) {
  // Treat like an error.
}
var constraints = {
  deviceId: {exact: "20983-20o198-109283-098-09812"},
  advanced: [{
      width: {
        min: 800,
        max: 1200
      }
    }, {
      height: {
        min: 600
      }
    }]
};

And here's one for an audio track:

var supports = navigator.mediaDevices.getSupportedConstraints();
if(!supports["deviceId"] || !supports["volume"]) {
  // Treat like an error.
}
var constraints = {
  advanced: [{
      deviceId: "64815-wi3c89-1839dk-x82-392aa"
    }, {
      volume: 0.5
    }]
};

Here's an example of use of ideal:

var supports = navigator.mediaDevices.getSupportedConstraints();
if(!supports["aspectRatio"] || !supports["facingMode"]) {
  // Treat like an error.
}
var gotten = navigator.mediaDevices.getUserMedia({
  video: {
    width: {min: 320, ideal: 1280, max: 1920},
    height: {min: 240, ideal: 720, max: 1080},
    frameRate: 30,     // Shorthand for ideal.
    // facingMode: "environment" would be optional.
    facingMode: {exact: "environment"}
  }});

Here's an example of "I want 720p, but I can accept up to 1080p and down to VGA.":

var supports = navigator.mediaDevices.getSupportedConstraints();
if(!supports["width"] || !supports["height"]) {
  // Treat like an error.
}
var gotten = navigator.mediaDevices.getUserMedia({video: {
  width: {min: 640, ideal: 1280, max: 1920},
  height: {min: 480, ideal: 720, max: 1080},
}});

Here's an example of "I want a front-facing camera and it must be VGA.":

var supports = navigator.mediaDevices.getSupportedConstraints();
if(!supports["width"] || !supports["height"] || !supports["facingMode"]) {
  // Treat like an error.
}
var gotten = navigator.mediaDevices.getUserMedia({video: {
  facingMode: {exact: "user"},
  width: {exact: 640},
  height: {exact: 480}
}});

Types for Constrainable Properties

The syntax for the specification of the set of legal values depends on the type of the values. In addition to the standard atomic types (boolean, long, double, DOMString), legal values include lists of any of the atomic types, plus min-max ranges, as defined below.

List values MUST be interpreted as disjunctions. For example, if a property 'facingMode' for a camera is defined as having legal values ["left", "right", "user", "environment"], this means that 'facingMode' can have the values "left", "right", "environment", and "user". Similarly Constraints restricting 'facingMode' to ["user", "left", "right"] would mean that the User Agent should select a camera (or point the camera, if that is possible) so that "facingMode" is either "user", "left", or "right". This Constraint would thus request that the camera not be facing away from the user, but would allow the User Agent to allow the user to choose other directions.

double max

The maximum legal value of this property.

double min

The minimum value of this Property.

double exact

The exact required value for this property.

double ideal

The ideal (target) value for this property.

long max

The maximum legal value of this property.

long min

The minimum value of this property.

long exact

The exact required value for this property.

long ideal

The ideal (target) value for this property.

boolean exact

The exact required value for this property.

boolean ideal

The ideal (target) value for this property.

(DOMString or sequence<DOMString>) exact

The exact required value for this property.

(DOMString or sequence<DOMString>) ideal

The ideal (target) value for this property.

Capabilities

Capabilities is a dictionary containing one or more key-value pairs, where each key MUST be a constrainable property, and each value MUST be a subset of the set of values allowed for that property. The exact syntax of the value expression depends on the type of the property. The Capabilities dictionary specifies the subset of the constrainable properties and values that the User Agent supports. Note that a User Agent MAY support only a subset of the properties defined in the Web platform, and MAY support a subset of the set values for those properties that it does support. Note that Capabilities are returned from the User Agent to the application, and cannot be specified by the application. However, the application can control the Settings that the User Agent chooses for constrainable properties by means of Constraints.

An example of a Capabilities dictionary is shown below. This example is not very realistic in that a browser would actually be required to support more constrainable properties than just these.

{
  frameRate: {
    min: 1.0,
    max: 60.0
  },
  facingMode: [ "user", "left" ]
}

The next example below points out that capabilities for range values provide ranges for individual constrainable properties, not combinations. This is particularly relevant for video width and height, since the ranges for width and height are reported separately. In the example, if the User Agent can only provide 640x480 and 800x600 resolutions the relevant capabilities returned would be:

{
  width: {
    min: 640,
    max: 800
  },
  height: {
    min: 480,
    max: 600
  },
  aspectRatio: 1.3333333333
}

Note in the example above that the aspectRatio would make clear that arbitrary combination of widths and heights are not possible, although it would still suggest that more than two resolutions were available.

Settings

Settings is a dictionary containing one or more key-value pairs. It MUST contain each key returned in getCapabilities(). There MUST be a single value for each key and the value MUST be a member of the set defined for that property by getCapabilities(). The Settings dictionary contains the actual values that the User Agent has chosen for the object's constrainable properties. The exact syntax of the value depends on the type of the property.

A conforming User Agent MUST support all the constrainable properties defined in this specification.

An example of a Settings dictionary is shown below. This example is not very realistic in that a browser would actually be required to support more constrainable properties than just these.

{
  frameRate: 30.0,
  facingMode: "user"
}

Constraints and ConstraintSet

Due to the limitations of WebIDL, interfaces implementing the Constrainable Pattern cannot simply subclass Constraints and ConstraintSet as they are defined here. Instead they must provide their own definitions that follow this pattern. See MediaTrackConstraints for an example of this.

Each member of a ConstraintSet corresponds to a constrainable property and specifies a subset of the property's legal Capability values. Applying a ConstraintSet instructs the User Agent to restrict the settings of the corresponding constrainable properties to the specified values or ranges of values. A given property MAY occur both in the basic Constraints set and in the advanced ConstraintSets list, and MAY occur at most once in each ConstraintSet in the advanced list.

sequence<ConstraintSet> advanced

This is the list of ConstraintSets that the User Agent MUST attempt to satisfy, in order, skipping only those that cannot be satisfied. The order of these ConstraintSets is significant. In particular, when they are passed as an argument to applyConstraints, the User Agent MUST try to satisfy them in the order that is specified. Thus if optional ConstraintSets C1 and C2 can be satisfied individually, but not together, then whichever of C1 and C2 is first in this list will be satisfied, and the other will not. The User Agent MUST attempt to satisfy all optional ConstraintSets in the list, even if some cannot be satisfied. Thus, in the preceding example, if optional constraint C3 is specified after C1 and C2, the User Agent will attempt to satisfy C3 even though C2 cannot be satisfied. Note that a given property name may occur only once in each ConstraintSet but may occur in more than one ConstraintSet.