1# Audio Device Module (ADM) 2 3<?% config.freshness.owner = 'henrika' %?> 4<?% config.freshness.reviewed = '2021-04-12' %?> 5 6## Overview 7 8The ADM is responsible for driving input (microphone) and output (speaker) audio 9in WebRTC and the API is defined in [audio_device.h][19]. 10 11Main functions of the ADM are: 12 13* Initialization and termination of native audio libraries. 14* Registration of an [AudioTransport object][16] which handles audio callbacks 15 for audio in both directions. 16* Device enumeration and selection (only for Linux, Windows and Mac OSX). 17* Start/Stop physical audio streams: 18 * Recording audio from the selected microphone, and 19 * playing out audio on the selected speaker. 20* Level control of the active audio streams. 21* Control of built-in audio effects (Audio Echo Cancelation (AEC), Audio Gain 22 Control (AGC) and Noise Suppression (NS)) for Android and iOS. 23 24ADM implementations reside at two different locations in the WebRTC repository: 25`/modules/audio_device/` and `/sdk/`. The latest implementations for [iOS][20] 26and [Android][21] can be found under `/sdk/`. `/modules/audio_device/` contains 27older versions for mobile platforms and also implementations for desktop 28platforms such as [Linux][22], [Windows][23] and [Mac OSX][24]. This document is 29focusing on the parts in `/modules/audio_device/` but implementation specific 30details such as threading models are omitted to keep the descriptions as simple 31as possible. 32 33By default, the ADM in WebRTC is created in [`WebRtcVoiceEngine::Init`][1] but 34an external implementation can also be injected using 35[`rtc::CreatePeerConnectionFactory`][25]. An example of where an external ADM is 36injected can be found in [PeerConnectionInterfaceTest][26] where a so-called 37[fake ADM][29] is utilized to avoid hardware dependency in a gtest. Clients can 38also inject their own ADMs in situations where functionality is needed that is 39not provided by the default implementations. 40 41## Background 42 43This section contains a historical background of the ADM API. 44 45The ADM interface is old and has undergone many changes over the years. It used 46to be much more granular but it still contains more than 50 methods and is 47implemented on several different hardware platforms. 48 49Some APIs are not implemented on all platforms, and functionality can be spread 50out differently between the methods. 51 52The most up-to-date implementations of the ADM interface are for [iOS][27] and 53for [Android][28]. 54 55Desktop version are not updated to comply with the latest 56[C++ style guide](https://chromium.googlesource.com/chromium/src/+/main/styleguide/c++/c++.md) 57and more work is also needed to improve the performance and stability of these 58versions. 59 60## WebRtcVoiceEngine 61 62[`WebRtcVoiceEngine`][2] does not utilize all methods of the ADM but it still 63serves as the best example of its architecture and how to use it. For a more 64detailed view of all methods in the ADM interface, see [ADM unit tests][3]. 65 66Assuming that an external ADM implementation is not injected, a default - or 67internal - ADM is created in [`WebRtcVoiceEngine::Init`][1] using 68[`AudioDeviceModule::Create`][4]. 69 70Basic initialization is done using a utility method called 71[`adm_helpers::Init`][5] which calls fundamental ADM APIs like: 72 73* [`AudiDeviceModule::Init`][6] - initializes the native audio parts required 74 for each platform. 75* [`AudiDeviceModule::SetPlayoutDevice`][7] - specifies which speaker to use 76 for playing out audio using an `index` retrieved by the corresponding 77 enumeration method [`AudiDeviceModule::PlayoutDeviceName`][8]. 78* [`AudiDeviceModule::SetRecordingDevice`][9] - specifies which microphone to 79 use for recording audio using an `index` retrieved by the corresponding 80 enumeration method which is [`AudiDeviceModule::RecordingDeviceName`][10]. 81* [`AudiDeviceModule::InitSpeaker`][11] - sets up the parts of the ADM needed 82 to use the selected output device. 83* [`AudiDeviceModule::InitMicrophone`][12] - sets up the parts of the ADM 84 needed to use the selected input device. 85* [`AudiDeviceModule::SetStereoPlayout`][13] - enables playout in stereo if 86 the selected audio device supports it. 87* [`AudiDeviceModule::SetStereoRecording`][14] - enables recording in stereo 88 if the selected audio device supports it. 89 90[`WebRtcVoiceEngine::Init`][1] also calls 91[`AudiDeviceModule::RegisterAudioTransport`][15] to register an existing 92[AudioTransport][16] implementation which handles audio callbacks in both 93directions and therefore serves as the bridge between the native ADM and the 94upper WebRTC layers. 95 96Recorded audio samples are delivered from the ADM to the `WebRtcVoiceEngine` 97(who owns the `AudioTransport` object) via 98[`AudioTransport::RecordedDataIsAvailable`][17]: 99 100``` 101int32_t RecordedDataIsAvailable(const void* audioSamples, size_t nSamples, size_t nBytesPerSample, 102 size_t nChannels, uint32_t samplesPerSec, uint32_t totalDelayMS, 103 int32_t clockDrift, uint32_t currentMicLevel, bool keyPressed, 104 uint32_t& newMicLevel) 105``` 106 107Decoded audio samples ready to be played out are are delivered by the 108`WebRtcVoiceEngine` to the ADM, via [`AudioTransport::NeedMorePlayoutData`][18]: 109 110``` 111int32_t NeedMorePlayData(size_t nSamples, size_t nBytesPerSample, size_t nChannels, int32_t samplesPerSec, 112 void* audioSamples, size_t& nSamplesOut, 113 int64_t* elapsed_time_ms, int64_t* ntp_time_ms) 114``` 115 116Audio samples are 16-bit [linear PCM](https://wiki.multimedia.cx/index.php/PCM) 117using regular interleaving of channels within each sample. 118 119`WebRtcVoiceEngine` also owns an [`AudioState`][30] member and this class is 120used has helper to start and stop audio to and from the ADM. To initialize and 121start recording, it calls: 122 123* [`AudiDeviceModule::InitRecording`][31] 124* [`AudiDeviceModule::StartRecording`][32] 125 126and to initialize and start playout: 127 128* [`AudiDeviceModule::InitPlayout`][33] 129* [`AudiDeviceModule::StartPlayout`][34] 130 131Finally, the corresponding stop methods [`AudiDeviceModule::StopRecording`][35] 132and [`AudiDeviceModule::StopPlayout`][36] are called followed by 133[`AudiDeviceModule::Terminate`][37]. 134 135[1]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/media/engine/webrtc_voice_engine.cc;l=314;drc=f7b1b95f11c74cb5369fdd528b73c70a50f2e206 136[2]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/media/engine/webrtc_voice_engine.h;l=48;drc=d15a575ec3528c252419149d35977e55269d8a41 137[3]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/audio_device_unittest.cc;l=1;drc=d15a575ec3528c252419149d35977e55269d8a41 138[4]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=46;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738e 139[5]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/media/engine/adm_helpers.h;drc=2222a80e79ae1ef5cb9510ec51d3868be75f47a2 140[6]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=62;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 141[7]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=77;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 142[8]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=69;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 143[9]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=79;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 144[10]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=72;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 145[11]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=99;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 146[12]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=101;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 147[13]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=130;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 148[14]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=133;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 149[15]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=59;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 150[16]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device_defines.h;l=34;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 151[17]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device_defines.h;l=36;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 152[18]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device_defines.h;l=48;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 153[19]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738es 154[20]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/sdk/objc/native/api/audio_device_module.h;drc=76443eafa9375374d9f1d23da2b913f2acac6ac2 155[21]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/sdk/android/src/jni/audio_device/audio_device_module.h;drc=bbeb10925eb106eeed6143ccf571bc438ec22ce1 156[22]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/linux/;drc=d15a575ec3528c252419149d35977e55269d8a41 157[23]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/win/;drc=d15a575ec3528c252419149d35977e55269d8a41 158[24]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/mac/;drc=3b68aa346a5d3483c3448852d19d91723846825c 159[25]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/create_peerconnection_factory.h;l=45;drc=09ceed2165137c4bea4e02e8d3db31970d0bf273 160[26]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/pc/peer_connection_interface_unittest.cc;l=692;drc=2efb8a5ec61b1b87475d046c03d20244f53b14b6 161[27]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/sdk/objc/native/api/audio_device_module.h;drc=76443eafa9375374d9f1d23da2b913f2acac6ac2 162[28]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/sdk/android/src/jni/audio_device/audio_device_module.h;drc=bbeb10925eb106eeed6143ccf571bc438ec22ce1 163[29]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/pc/test/fake_audio_capture_module.h;l=42;drc=d15a575ec3528c252419149d35977e55269d8a41 164[30]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/audio/audio_state.h;drc=d15a575ec3528c252419149d35977e55269d8a41 165[31]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=87;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738e 166[32]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=94;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738e 167[33]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=84;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738e 168[34]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=91;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738e 169[35]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=95;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738e 170[36]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=92;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738e 171[37]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_device/include/audio_device.h;l=63;drc=eb8c4ca608486add9800f6bfb7a8ba3cf23e738e 172