xref: /aosp_15_r20/external/webrtc/docs/native-code/rtp-hdrext/video-layers-allocation00/README.md (revision d9f758449e529ab9291ac668be2861e7a55c2422) 
1# Video Layers Allocation
2
3The goal of this extension is for a video sender to provide information about
4the target bitrate, resolution and frame rate of each scalability layer in order
5to aid a selective forwarding middlebox to decide which layer to relay.
6
7**Name:** "Video layers allocation version 0"
8
9**Formal name:**
10<http://www.webrtc.org/experiments/rtp-hdrext/video-layers-allocation00>
11
12**Status:** This extension is defined here to allow for experimentation.
13
14In a conference scenario, a video from a single sender may be received by
15several recipients with different downlink bandwidth constraints and UI
16requirements. To allow this, a sender can send video with several scalability
17layers and a middle box can choose a layer to relay for each receiver.
18
19This extension support temporal layers, multiple spatial layers sent on a single
20rtp stream (SVC), or independent spatial layers sent on multiple rtp streams
21(simulcast).
22
23## RTP header extension format
24
25### Data layout
26
27```
28//                           +-+-+-+-+-+-+-+-+
29//                           |RID| NS| sl_bm |
30//                           +-+-+-+-+-+-+-+-+
31// Spatial layer bitmask     |sl0_bm |sl1_bm |
32//   up to 2 bytes           |---------------|
33//   when sl_bm == 0         |sl2_bm |sl3_bm |
34//                           +-+-+-+-+-+-+-+-+
35// Number of temporal layers |#tl|#tl|#tl|#tl|
36// per spatial layer         |   |   |   |   |
37//                           +-+-+-+-+-+-+-+-+
38//  Target bitrate in kpbs   |               |
39//   per temporal layer      :      ...      :
40//    leb128 encoded         |               |
41//                           +-+-+-+-+-+-+-+-+
42// Resolution and framerate  |               |
43// 5 bytes per spatial layer + width-1 for   +
44//      (optional)           | rid=0, sid=0  |
45//                           +---------------+
46//                           |               |
47//                           + height-1 for  +
48//                           | rid=0, sid=0  |
49//                           +---------------+
50//                           | max framerate |
51//                           +-+-+-+-+-+-+-+-+
52//                           :      ...      :
53//                           +-+-+-+-+-+-+-+-+
54```
55
56RID: RTP stream index this allocation is sent on, numbered from 0. 2 bits.
57
58NS: Number of RTP streams minus one. 2 bits, thus allowing up-to 4 RTP streams.
59
60sl_bm: BitMask of the active Spatial Layers when same for all RTP streams or 0
61otherwise. 4 bits, thus allows up to 4 spatial layers per RTP streams.
62
63slX_bm: BitMask of the active Spatial Layers for RTP stream with index=X.
64When NS < 2, takes one byte, otherwise uses two bytes. Zero-padded to byte
65alignment.
66
67\#tl: 2-bit value of number of temporal layers-1, thus allowing up-to 4 temporal
68layers. Values are stored in ascending order of spatial id. Zero-padded to byte
69alignment.
70
71Target bitrate in kbps. Values are stored using leb128 encoding [1]. One value per
72temporal layer. Values are stored in (RTP stream id, spatial id, temporal id)
73ascending order. All bitrates are total required bitrate to receive the
74corresponding layer, i.e. in simulcast mode they include only corresponding
75spatial layers, in full-svc all lower spatial layers are included. All lower
76temporal layers are also included.
77
78Resolution and framerate. Optional. Presence is inferred from the rtp header
79extension size. Encoded (width - 1), 16-bit, (height - 1), 16-bit, max frame
80rate 8-bit per spatial layer per RTP stream. Values are stored in (RTP stream
81id, spatial id) ascending order.
82
83An empty layer allocation (i.e nothing sent on ssrc) is encoded as
84special case with a single 0 byte.
85
86[1] https://aomediacodec.github.io/av1-spec/#leb128
87