1<?% config.freshness.reviewed = '2021-04-12' %?> 2 3# PeerConnection level framework fixture architecture 4 5## Overview 6 7The main implementation of 8[`webrtc::webrtc_pc_e2e::PeerConnectionE2EQualityTestFixture`][1] is 9[`webrtc::webrtc_pc_e2e::PeerConnectionE2EQualityTest`][2]. Internally it owns 10the next main pieces: 11 12* [`MediaHelper`][3] - responsible for adding audio and video tracks to the 13 peers. 14* [`VideoQualityAnalyzerInjectionHelper`][4] and 15 [`SingleProcessEncodedImageDataInjector`][5] - used to inject video quality 16 analysis and properly match captured and rendered video frames. You can read 17 more about it in 18 [DefaultVideoQualityAnalyzer](default_video_quality_analyzer.md) section. 19* [`AudioQualityAnalyzerInterface`][6] - used to measure audio quality metrics 20* [`TestActivitiesExecutor`][7] - used to support [`ExecuteAt(...)`][8] and 21 [`ExecuteEvery(...)`][9] API of `PeerConnectionE2EQualityTestFixture` to run 22 any arbitrary action during test execution timely synchronized with a test 23 call. 24* A vector of [`QualityMetricsReporter`][10] added by the 25 `PeerConnectionE2EQualityTestFixture` user. 26* Two peers: Alice and Bob represented by instances of [`TestPeer`][11] 27 object. 28 29Also it keeps a reference to [`webrtc::TimeController`][12], which is used to 30create all required threads, task queues, task queue factories and time related 31objects. 32 33## TestPeer 34 35Call participants are represented by instances of `TestPeer` object. 36[`TestPeerFactory`][13] is used to create them. `TestPeer` owns all instances 37related to the `webrtc::PeerConnection`, including required listeners and 38callbacks. Also it provides an API to do offer/answer exchange and ICE candidate 39exchange. For this purposes internally it uses an instance of 40[`webrtc::PeerConnectionWrapper`][14]. 41 42The `TestPeer` also owns the `PeerConnection` worker thread. The signaling 43thread for all `PeerConnection`'s is owned by 44`PeerConnectionE2EQualityTestFixture` and shared between all participants in the 45call. The network thread is owned by the network layer (it maybe either emulated 46network provided by [Network Emulation Framework][24] or network thread and 47`rtc::NetworkManager` provided by user) and provided when peer is added to the 48fixture via [`AddPeer(...)`][15] API. 49 50## GetStats API based metrics reporters 51 52`PeerConnectionE2EQualityTestFixture` gives the user ability to provide 53different `QualityMetricsReporter`s which will listen for `PeerConnection` 54[`GetStats`][16] API. Then such reporters will be able to report various metrics 55that user wants to measure. 56 57`PeerConnectionE2EQualityTestFixture` itself also uses this mechanism to 58measure: 59 60* Audio quality metrics 61* Audio/Video sync metrics (with help of [`CrossMediaMetricsReporter`][17]) 62 63Also framework provides a [`StatsBasedNetworkQualityMetricsReporter`][18] to 64measure network related WebRTC metrics and print debug raw emulated network 65statistic. This reporter should be added by user via 66[`AddQualityMetricsReporter(...)`][19] API if requried. 67 68Internally stats gathering is done by [`StatsPoller`][20]. Stats are requested 69once per second for each `PeerConnection` and then resulted object is provided 70into each stats listener. 71 72## Offer/Answer exchange 73 74`PeerConnectionE2EQualityTest` provides ability to test Simulcast and SVC for 75video. These features aren't supported by P2P call and in general requires a 76Selective Forwarding Unit (SFU). So special logic is applied to mimic SFU 77behavior in P2P call. This logic is located inside [`SignalingInterceptor`][21], 78[`QualityAnalyzingVideoEncoder`][22] and [`QualityAnalyzingVideoDecoder`][23] 79and consist of SDP modification during offer/answer exchange and special 80handling of video frames from unrelated Simulcast/SVC streams during decoding. 81 82### Simulcast 83 84In case of Simulcast we have a video track, which internally contains multiple 85video streams, for example low resolution, medium resolution and high 86resolution. WebRTC client doesn't support receiving an offer with multiple 87streams in it, because usually SFU will keep only single stream for the client. 88To bypass it framework will modify offer by converting a single track with three 89video streams into three independent video tracks. Then sender will think that 90it send simulcast, but receiver will think that it receives 3 independent 91tracks. 92 93To achieve such behavior some extra tweaks are required: 94 95* MID RTP header extension from original offer have to be removed 96* RID RTP header extension from original offer is replaced with MID RTP header 97 extension, so the ID that sender uses for RID on receiver will be parsed as 98 MID. 99* Answer have to be modified in the opposite way. 100 101Described modifications are illustrated on the picture below. 102 103 104 105The exchange will look like this: 106 1071. Alice creates an offer 1082. Alice sets offer as local description 1093. Do described offer modification 1104. Alice sends modified offer to Bob 1115. Bob sets modified offer as remote description 1126. Bob creates answer 1137. Bob sets answer as local description 1148. Do reverse modifications on answer 1159. Bob sends modified answer to Alice 11610. Alice sets modified answer as remote description 117 118Such mechanism put a constraint that RTX streams are not supported, because they 119don't have RID RTP header extension in their packets. 120 121### SVC 122 123In case of SVC the framework will update the sender's offer before even setting 124it as local description on the sender side. Then no changes to answer will be 125required. 126 127`ssrc` is a 32 bit random value that is generated in RTP to denote a specific 128source used to send media in an RTP connection. In original offer video track 129section will look like this: 130 131``` 132m=video 9 UDP/TLS/RTP/SAVPF 98 100 99 101 133... 134a=ssrc-group:FID <primary ssrc> <retransmission ssrc> 135a=ssrc:<primary ssrc> cname:... 136.... 137a=ssrc:<retransmission ssrc> cname:... 138.... 139``` 140 141To enable SVC for such video track framework will add extra `ssrc`s for each SVC 142stream that is required like this: 143 144``` 145a=ssrc-group:FID <Low resolution primary ssrc> <Low resolution retransmission ssrc> 146a=ssrc:<Low resolution primary ssrc> cname:... 147.... 148a=ssrc:<Low resolution retransmission ssrc> cname:.... 149... 150a=ssrc-group:FID <Medium resolution primary ssrc> <Medium resolution retransmission ssrc> 151a=ssrc:<Medium resolution primary ssrc> cname:... 152.... 153a=ssrc:<Medium resolution retransmission ssrc> cname:.... 154... 155a=ssrc-group:FID <High resolution primary ssrc> <High resolution retransmission ssrc> 156a=ssrc:<High resolution primary ssrc> cname:... 157.... 158a=ssrc:<High resolution retransmission ssrc> cname:.... 159... 160``` 161 162The next line will also be added to the video track section of the offer: 163 164``` 165a=ssrc-group:SIM <Low resolution primary ssrc> <Medium resolution primary ssrc> <High resolution primary ssrc> 166``` 167 168It will tell PeerConnection that this track should be configured as SVC. It 169utilize WebRTC Plan B offer structure to achieve SVC behavior, also it modifies 170offer before setting it as local description which violates WebRTC standard. 171Also it adds limitations that on lossy networks only top resolution streams can 172be analyzed, because WebRTC won't try to restore low resolution streams in case 173of loss, because it still receives higher stream. 174 175### Handling in encoder/decoder 176 177In the encoder, the framework for each encoded video frame will propagate 178information requried for the fake SFU to know if it belongs to an interesting 179simulcast stream/spatial layer of if it should be "discarded". 180 181On the decoder side frames that should be "discarded" by fake SFU will be auto 182decoded into single pixel images and only the interesting simulcast 183stream/spatial layer will go into real decoder and then will be analyzed. 184 185[1]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/test/peerconnection_quality_test_fixture.h;l=55;drc=484acf27231d931dbc99aedce85bc27e06486b96 186[2]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/peer_connection_quality_test.h;l=44;drc=6cc893ad778a0965e2b7a8e614f3c98aa81bee5b 187[3]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/media/media_helper.h;l=27;drc=d46db9f1523ae45909b4a6fdc90a140443068bc6 188[4]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/analyzer/video/video_quality_analyzer_injection_helper.h;l=38;drc=79020414fd5c71f9ec1f25445ea5f1c8001e1a49 189[5]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/analyzer/video/single_process_encoded_image_data_injector.h;l=40;drc=79020414fd5c71f9ec1f25445ea5f1c8001e1a49 190[6]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/test/audio_quality_analyzer_interface.h;l=23;drc=20f45823e37fd7272aa841831c029c21f29742c2 191[7]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/test_activities_executor.h;l=28;drc=6cc893ad778a0965e2b7a8e614f3c98aa81bee5b 192[8]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/test/peerconnection_quality_test_fixture.h;l=439;drc=484acf27231d931dbc99aedce85bc27e06486b96 193[9]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/test/peerconnection_quality_test_fixture.h;l=445;drc=484acf27231d931dbc99aedce85bc27e06486b96 194[10]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/test/peerconnection_quality_test_fixture.h;l=413;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 195[11]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/test_activities_executor.h;l=28;drc=6cc893ad778a0965e2b7a8e614f3c98aa81bee5b 196[12]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/test_activities_executor.h;l=28;drc=6cc893ad778a0965e2b7a8e614f3c98aa81bee5b 197[13]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/test_peer_factory.h;l=46;drc=0ef4a2488a466a24ab97b31fdddde55440d451f9 198[14]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/pc/peer_connection_wrapper.h;l=47;drc=5ab79e62f691875a237ea28ca3975ea1f0ed62ec 199[15]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/test/peerconnection_quality_test_fixture.h;l=459;drc=484acf27231d931dbc99aedce85bc27e06486b96 200[16]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/peer_connection_interface.h;l=886;drc=9438fb3fff97c803d1ead34c0e4f223db168526f 201[17]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/cross_media_metrics_reporter.h;l=29;drc=9d777620236ec76754cfce19f6e82dd18e52d22c 202[18]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/cross_media_metrics_reporter.h;l=29;drc=9d777620236ec76754cfce19f6e82dd18e52d22c 203[19]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/api/test/peerconnection_quality_test_fixture.h;l=450;drc=484acf27231d931dbc99aedce85bc27e06486b96 204[20]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/stats_poller.h;l=52;drc=9b526180c9e9722d3fc7f8689da6ec094fc7fc0a 205[21]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/sdp/sdp_changer.h;l=79;drc=ee558dcca89fd8b105114ededf9e74d948da85e8 206[22]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/analyzer/video/quality_analyzing_video_encoder.h;l=54;drc=79020414fd5c71f9ec1f25445ea5f1c8001e1a49 207[23]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/test/pc/e2e/analyzer/video/quality_analyzing_video_decoder.h;l=50;drc=79020414fd5c71f9ec1f25445ea5f1c8001e1a49 208[24]: /test/network/g3doc/index.md 209
