1*d9f75844SAndroid Build Coastguard Worker<?% config.freshness.owner = 'alessiob' %?> 2*d9f75844SAndroid Build Coastguard Worker<?% config.freshness.reviewed = '2021-04-21' %?> 3*d9f75844SAndroid Build Coastguard Worker 4*d9f75844SAndroid Build Coastguard Worker# The WebRTC Audio Mixer Module 5*d9f75844SAndroid Build Coastguard Worker 6*d9f75844SAndroid Build Coastguard WorkerThe WebRTC audio mixer module is responsible for mixing multiple incoming audio 7*d9f75844SAndroid Build Coastguard Workerstreams (sources) into a single audio stream (mix). It works with 10 ms frames, 8*d9f75844SAndroid Build Coastguard Workerit supports sample rates up to 48 kHz and up to 8 audio channels. The API is 9*d9f75844SAndroid Build Coastguard Workerdefined in 10*d9f75844SAndroid Build Coastguard Worker[`api/audio/audio_mixer.h`](https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/audio/audio_mixer.h) 11*d9f75844SAndroid Build Coastguard Workerand it includes the definition of 12*d9f75844SAndroid Build Coastguard Worker[`AudioMixer::Source`](https://source.chromium.org/search?q=symbol:AudioMixer::Source%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h), 13*d9f75844SAndroid Build Coastguard Workerwhich describes an incoming audio stream, and the definition of 14*d9f75844SAndroid Build Coastguard Worker[`AudioMixer`](https://source.chromium.org/search?q=symbol:AudioMixer%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h), 15*d9f75844SAndroid Build Coastguard Workerwhich operates on a collection of 16*d9f75844SAndroid Build Coastguard Worker[`AudioMixer::Source`](https://source.chromium.org/search?q=symbol:AudioMixer::Source%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h) 17*d9f75844SAndroid Build Coastguard Workerobjects to produce a mix. 18*d9f75844SAndroid Build Coastguard Worker 19*d9f75844SAndroid Build Coastguard Worker## AudioMixer::Source 20*d9f75844SAndroid Build Coastguard Worker 21*d9f75844SAndroid Build Coastguard WorkerA source has different characteristic (e.g., sample rate, number of channels, 22*d9f75844SAndroid Build Coastguard Workermuted state) and it is identified by an SSRC[^1]. 23*d9f75844SAndroid Build Coastguard Worker[`AudioMixer::Source::GetAudioFrameWithInfo()`](https://source.chromium.org/search?q=symbol:AudioMixer::Source::GetAudioFrameWithInfo%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h) 24*d9f75844SAndroid Build Coastguard Workeris used to retrieve the next 10 ms chunk of audio to be mixed. 25*d9f75844SAndroid Build Coastguard Worker 26*d9f75844SAndroid Build Coastguard Worker[^1]: A synchronization source (SSRC) is the source of a stream of RTP packets, 27*d9f75844SAndroid Build Coastguard Worker identified by a 32-bit numeric SSRC identifier carried in the RTP header 28*d9f75844SAndroid Build Coastguard Worker so as not to be dependent upon the network address (see 29*d9f75844SAndroid Build Coastguard Worker [RFC 3550](https://tools.ietf.org/html/rfc3550#section-3)). 30*d9f75844SAndroid Build Coastguard Worker 31*d9f75844SAndroid Build Coastguard Worker## AudioMixer 32*d9f75844SAndroid Build Coastguard Worker 33*d9f75844SAndroid Build Coastguard WorkerThe interface allows to add and remove sources and the 34*d9f75844SAndroid Build Coastguard Worker[`AudioMixer::Mix()`](https://source.chromium.org/search?q=symbol:AudioMixer::Mix%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h) 35*d9f75844SAndroid Build Coastguard Workermethod allows to generates a mix with the desired number of channels. 36*d9f75844SAndroid Build Coastguard Worker 37*d9f75844SAndroid Build Coastguard Worker## WebRTC implementation 38*d9f75844SAndroid Build Coastguard Worker 39*d9f75844SAndroid Build Coastguard WorkerThe interface is implemented in different parts of WebRTC: 40*d9f75844SAndroid Build Coastguard Worker 41*d9f75844SAndroid Build Coastguard Worker* [`AudioMixer::Source`](https://source.chromium.org/search?q=symbol:AudioMixer::Source%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h): 42*d9f75844SAndroid Build Coastguard Worker [`audio/audio_receive_stream.h`](https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/audio/audio_receive_stream.h) 43*d9f75844SAndroid Build Coastguard Worker* [`AudioMixer`](https://source.chromium.org/search?q=symbol:AudioMixer%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h): 44*d9f75844SAndroid Build Coastguard Worker [`modules/audio_mixer/audio_mixer_impl.h`](https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_mixer/audio_mixer_impl.h) 45*d9f75844SAndroid Build Coastguard Worker 46*d9f75844SAndroid Build Coastguard Worker[`AudioMixer`](https://source.chromium.org/search?q=symbol:AudioMixer%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h) 47*d9f75844SAndroid Build Coastguard Workeris thread-safe. The output sample rate of the generated mix is automatically 48*d9f75844SAndroid Build Coastguard Workerassigned depending on the sample rate of the sources; whereas the number of 49*d9f75844SAndroid Build Coastguard Workeroutput channels is defined by the caller[^2]. Samples from the non-muted sources 50*d9f75844SAndroid Build Coastguard Workerare summed up and then a limiter is used to apply soft-clipping when needed. 51*d9f75844SAndroid Build Coastguard Worker 52*d9f75844SAndroid Build Coastguard Worker[^2]: [`audio/utility/channel_mixer.h`](https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/audio/utility/channel_mixer.h) 53*d9f75844SAndroid Build Coastguard Worker is used to mix channels in the non-trivial cases - i.e., if the number of 54*d9f75844SAndroid Build Coastguard Worker channels for a source or the mix is greater than 3. 55