Problem with adaptation sets that contain representations with different codecs

[barbibulle]

With an MPD that contains an audio adaptation set where there are two representations, each with a different codec, the player can play each of the representations by itself, but cannot switch from one to the other. I suspect that it doesn't reset the audio decoder, so feeding one elementary stream into a session setup with a different codec is likely to cause trouble.
The DASH spec doesn't prohibit mixing codecs in the same adaptation set, and it would seem like the right thing to do if you want to offer a stream with a choice of AAC or EAC-3 (better than having two adaptation sets, one for each codec).

Here's an example:
http://www.bok.net/tmp/dash/dual_codec/stream_aac.mpd

the same data with just the AAC codec:
http://www.bok.net/tmp/dash/dual_codec/stream.mpd

and with just the EAC-3 codec (plays fine on Safari on El Capitan)
http://www.bok.net/tmp/dash/dual_codec/stream_ec3.mpd

[joeyparrish]

The DASH spec allows for many things that are not a good fit for MSE. MSE does not allow changing codecs like this, so we do not support changing codecs across Representations. There is an expectation in Shaka that Representations in an AdaptationSet have the same (or decoder-compatible) codec.

[barbibulle]

Ok, that makes sense.
But we get the same error when signaling the two codec representations in different adaptation sets:
The player throws a Shaka Error MEDIA.VIDEO_ERROR when selecting the second adaptation set from the drop-down menu in the sample player UI.

Here's an example:
http://www.bok.net/tmp/dash/dual_codec_2/stream.mpd

or with just the AAC adaptation set (works by itself)
http://www.bok.net/tmp/dash/dual_codec_2/stream_aac.mpd

or with just the E-AC3 adaptation set (works by itself)
http://www.bok.net/tmp/dash/dual_codec_2/stream_ec3.mpd

[joeyparrish]

In what browser do you see more than one audio track for the dual-codec MPD? My Chromebook only shows one (AAC), since EAC-3 is not supported by the browser. If I query MSE in the JS console:

MediaSource.isTypeSupported('audio/mp4; codecs="mp4a.40.2"')
true
MediaSource.isTypeSupported('audio/mp4; codecs="ec-3"')
false

[barbibulle]

Safari on OSX El Capitan supports ec-3

[joeyparrish]

If the two AdaptationSets are both playable, but incompatible with one another at a codec level, Shaka should filter out one once the other has been selected. I'll look into it.

[joeyparrish]

@barbibulle This should be fixed now. We will not permit switching between incompatible codecs once one has been chosen, so you should only see one audio track now.