1*61c4878aSAndroid Build Coastguard Worker.. _seed-0107: 2*61c4878aSAndroid Build Coastguard Worker 3*61c4878aSAndroid Build Coastguard Worker============================ 4*61c4878aSAndroid Build Coastguard Worker0107: Pigweed Communications 5*61c4878aSAndroid Build Coastguard Worker============================ 6*61c4878aSAndroid Build Coastguard Worker.. seed:: 7*61c4878aSAndroid Build Coastguard Worker :number: 107 8*61c4878aSAndroid Build Coastguard Worker :name: Communications 9*61c4878aSAndroid Build Coastguard Worker :status: Accepted 10*61c4878aSAndroid Build Coastguard Worker :proposal_date: 2023-07-19 11*61c4878aSAndroid Build Coastguard Worker :cl: 157090 12*61c4878aSAndroid Build Coastguard Worker :authors: Wyatt Hepler 13*61c4878aSAndroid Build Coastguard Worker :facilitator: Carlos Chinchilla 14*61c4878aSAndroid Build Coastguard Worker 15*61c4878aSAndroid Build Coastguard Worker------- 16*61c4878aSAndroid Build Coastguard WorkerSummary 17*61c4878aSAndroid Build Coastguard Worker------- 18*61c4878aSAndroid Build Coastguard WorkerPigweed does not currently offer an end-to-end solution for network 19*61c4878aSAndroid Build Coastguard Workercommunications. This SEED proposes that Pigweed adopt a new sockets API as its 20*61c4878aSAndroid Build Coastguard Workerprimary networking abstraction. The sockets API will be backed by a new, 21*61c4878aSAndroid Build Coastguard Workerlightweight embedded-focused network protocol stack, inspired by the Internet 22*61c4878aSAndroid Build Coastguard Workerprotocol suite (TCP/IP). It will also support full TCP/IP via an open source 23*61c4878aSAndroid Build Coastguard Workerembedded TCP/IP stack or OS sockets. The new communications APIs will support 24*61c4878aSAndroid Build Coastguard Workerasynchronous use and zero-copy transmission. 25*61c4878aSAndroid Build Coastguard Worker 26*61c4878aSAndroid Build Coastguard WorkerThis work is comprised of the following subareas: 27*61c4878aSAndroid Build Coastguard Worker 28*61c4878aSAndroid Build Coastguard Worker- `Sockets API`_ 29*61c4878aSAndroid Build Coastguard Worker- `Network protocol stack`_ 30*61c4878aSAndroid Build Coastguard Worker- `Async`_ API pattern 31*61c4878aSAndroid Build Coastguard Worker- `Buffer management`_ system 32*61c4878aSAndroid Build Coastguard Worker 33*61c4878aSAndroid Build Coastguard WorkerThe Pigweed team will revisit :ref:`pw_rpc <module-pw_rpc>` after deploying the 34*61c4878aSAndroid Build Coastguard Workersockets API and network protocol stack. 35*61c4878aSAndroid Build Coastguard Worker 36*61c4878aSAndroid Build Coastguard Worker---------- 37*61c4878aSAndroid Build Coastguard WorkerBackground 38*61c4878aSAndroid Build Coastguard Worker---------- 39*61c4878aSAndroid Build Coastguard WorkerPigweed's primary communications system is :ref:`pw_rpc <module-pw_rpc>`. pw_rpc 40*61c4878aSAndroid Build Coastguard Workermakes it possible to call functions and exchange data with a remote system. 41*61c4878aSAndroid Build Coastguard WorkerRequests and responses are encoded as protobufs. pw_rpc was initially deployed 42*61c4878aSAndroid Build Coastguard Workerto a system with its own network protocol stack, so the Pigweed team did not 43*61c4878aSAndroid Build Coastguard Workerinvest in building a network stack of its own. 44*61c4878aSAndroid Build Coastguard Worker 45*61c4878aSAndroid Build Coastguard WorkerThe TCP/IP model, as described in `RFC 1122 46*61c4878aSAndroid Build Coastguard Worker<https://datatracker.ietf.org/doc/html/rfc1122>`_, organizes communications 47*61c4878aSAndroid Build Coastguard Workersystems and protocols into four layers: Application, Transport, Internet (or 48*61c4878aSAndroid Build Coastguard WorkerNetwork), and Link (or Network access). Pigweed's current communications 49*61c4878aSAndroid Build Coastguard Workerofferings fit into the TCP/IP model as follows: 50*61c4878aSAndroid Build Coastguard Worker 51*61c4878aSAndroid Build Coastguard Worker+-----------------------+-----------------------------+ 52*61c4878aSAndroid Build Coastguard Worker| TCP/IP Model | Pigweed Modules | 53*61c4878aSAndroid Build Coastguard Worker+=======================+=============================+ 54*61c4878aSAndroid Build Coastguard Worker| Application | | :ref:`module-pw_transfer` | 55*61c4878aSAndroid Build Coastguard Worker| | | :ref:`module-pw_rpc` | 56*61c4878aSAndroid Build Coastguard Worker+-----------------------+-----------------------------+ 57*61c4878aSAndroid Build Coastguard Worker| Transport | | :ref:`module-pw_router` | 58*61c4878aSAndroid Build Coastguard Worker+-----------------------+ | :ref:`module-pw_hdlc` | 59*61c4878aSAndroid Build Coastguard Worker| Internet / Network | | 60*61c4878aSAndroid Build Coastguard Worker+-----------------------+ | 61*61c4878aSAndroid Build Coastguard Worker| Link / Network access | | 62*61c4878aSAndroid Build Coastguard Worker+-----------------------+-----------------------------+ 63*61c4878aSAndroid Build Coastguard Worker 64*61c4878aSAndroid Build Coastguard WorkerNotably, Pigweed provides little functionality below the application layer. The 65*61c4878aSAndroid Build Coastguard Workerpw_router and pw_hdlc modules only implement a subset of features needed at 66*61c4878aSAndroid Build Coastguard Workertheir layer in the communications stack. 67*61c4878aSAndroid Build Coastguard Worker 68*61c4878aSAndroid Build Coastguard WorkerChallenges deploying pw_rpc 69*61c4878aSAndroid Build Coastguard Worker=========================== 70*61c4878aSAndroid Build Coastguard Workerpw_rpc is application-layer communications module. It relies on a network layer 71*61c4878aSAndroid Build Coastguard Workerto send packets between endpoints and doesn't provide any networking features 72*61c4878aSAndroid Build Coastguard Workeritself. When initially developing pw_rpc, the Pigweed team focused its limited 73*61c4878aSAndroid Build Coastguard Workerresources solely on this application-layer feature, which made it possible to 74*61c4878aSAndroid Build Coastguard Workerdeploy pw_rpc quickly to systems with existing networks. 75*61c4878aSAndroid Build Coastguard Worker 76*61c4878aSAndroid Build Coastguard Workerpw_rpc has been deployed to many projects with great results. However, since 77*61c4878aSAndroid Build Coastguard WorkerPigweed does not provide a network stack, deploying pw_rpc to systems without 78*61c4878aSAndroid Build Coastguard Workerexisting stacks can be challenging. These systems have to develop their own 79*61c4878aSAndroid Build Coastguard Workersolutions to transmit and route pw_rpc packets. 80*61c4878aSAndroid Build Coastguard Worker 81*61c4878aSAndroid Build Coastguard WorkerAs an example, one project based its network communications on Pigweed's 82*61c4878aSAndroid Build Coastguard Worker:ref:`module-pw_hdlc` module. It used HDLC in a way more similar to IP, 83*61c4878aSAndroid Build Coastguard Workerproviding network-level addressing and features like quality-of-service. Source 84*61c4878aSAndroid Build Coastguard Workerand destination addresses and ports were packed into the HDLC address field to 85*61c4878aSAndroid Build Coastguard Workerfacilitate routing and multiplexing. The :ref:`module-pw_router` module was 86*61c4878aSAndroid Build Coastguard Workerdeveloped to support static routing tables for HDLC frames through nodes in the 87*61c4878aSAndroid Build Coastguard Workersystem, and the :ref:`pw_transfer RPC service <module-pw_transfer>` was 88*61c4878aSAndroid Build Coastguard Workerdeveloped to provide reliable delivery of data. 89*61c4878aSAndroid Build Coastguard Worker 90*61c4878aSAndroid Build Coastguard WorkerLearning from custom network stacks 91*61c4878aSAndroid Build Coastguard Worker----------------------------------- 92*61c4878aSAndroid Build Coastguard WorkerTeams want to use Pigweed to build cool devices. Their goal isn't to build a 93*61c4878aSAndroid Build Coastguard Workernetwork protocol stack, but they need one to use features like pw_rpc and 94*61c4878aSAndroid Build Coastguard Workerpw_transfer. Given this, teams have little incentive to make the enormous time 95*61c4878aSAndroid Build Coastguard Workerinvestment to develop a robust, reusable network stack. The practical approach 96*61c4878aSAndroid Build Coastguard Workeris to assemble the minimum viable network stack from what's available. 97*61c4878aSAndroid Build Coastguard Worker 98*61c4878aSAndroid Build Coastguard WorkerThe Pigweed team has seen a few teams create custom network stacks for pw_rpc. 99*61c4878aSAndroid Build Coastguard WorkerWhile these projects were successful, their network stacks were not their 100*61c4878aSAndroid Build Coastguard Workerprimary focus. As a result, they had some shortcomings, including the following: 101*61c4878aSAndroid Build Coastguard Worker 102*61c4878aSAndroid Build Coastguard Worker- **Byte stuffing memory overhead** -- HDLC is a low-level protocol. It uses 103*61c4878aSAndroid Build Coastguard Worker `byte stuffing 104*61c4878aSAndroid Build Coastguard Worker <https://en.wikipedia.org/wiki/High-Level_Data_Link_Control#Asynchronous_framing>`_ 105*61c4878aSAndroid Build Coastguard Worker to ensure frame integrity across unreliable links. Byte stuffing makes sense 106*61c4878aSAndroid Build Coastguard Worker on the wire, but not in memory. Storing byte stuffed frames requires double 107*61c4878aSAndroid Build Coastguard Worker the memory to account for worst-case byte stuffing. Some projects use HDLC 108*61c4878aSAndroid Build Coastguard Worker frames as network layer packets, so they are buffered in memory for routing, 109*61c4878aSAndroid Build Coastguard Worker which requires more memory than necessary. 110*61c4878aSAndroid Build Coastguard Worker- **HDLC protocol overhead** -- HDLC's frame recovery and integrity features are 111*61c4878aSAndroid Build Coastguard Worker not needed across all links. For example, these features are unnecessary for 112*61c4878aSAndroid Build Coastguard Worker Bluetooth. However, when projects use HDLC for both the network and link 113*61c4878aSAndroid Build Coastguard Worker layers, it has to be used across all links. 114*61c4878aSAndroid Build Coastguard Worker- **pw_transfer at the application layer** -- :ref:`pw_transfer 115*61c4878aSAndroid Build Coastguard Worker <module-pw_transfer>` supports reliable data transfers with :ref:`pw_rpc 116*61c4878aSAndroid Build Coastguard Worker <module-pw_rpc>`. It required significant investment to develop, but since it 117*61c4878aSAndroid Build Coastguard Worker is layered on top of pw_rpc, it has additional overhead and limited 118*61c4878aSAndroid Build Coastguard Worker reusability. 119*61c4878aSAndroid Build Coastguard Worker- **Custom routing** -- Some network nodes have multiple routes between them. 120*61c4878aSAndroid Build Coastguard Worker Projects have had to write custom, non-portable logic to handle routing. 121*61c4878aSAndroid Build Coastguard Worker- **pw_rpc channel IDs in routing** -- Some projects used pw_rpc channel IDs as 122*61c4878aSAndroid Build Coastguard Worker a network addresses. Channel IDs were assigned for the whole network ahead of 123*61c4878aSAndroid Build Coastguard Worker time. This has several downsides: 124*61c4878aSAndroid Build Coastguard Worker 125*61c4878aSAndroid Build Coastguard Worker - Requires nodes to have knowledge of the global channel ID assignments 126*61c4878aSAndroid Build Coastguard Worker and routes between them, which can be difficult to keep in sync. 127*61c4878aSAndroid Build Coastguard Worker - Implies that all traffic is pw_rpc packets. 128*61c4878aSAndroid Build Coastguard Worker - Requires decoding pw_rpc packets at lower levels of the network stack. 129*61c4878aSAndroid Build Coastguard Worker - Complicates runtime assignment of channel IDs. 130*61c4878aSAndroid Build Coastguard Worker 131*61c4878aSAndroid Build Coastguard Worker- **Flow control** -- Projects' communications stacks have not supported flow 132*61c4878aSAndroid Build Coastguard Worker control. The network layer simply has to drop packets it cannot process. 133*61c4878aSAndroid Build Coastguard Worker There is no mechanism to tell the producer to slow down or wait for the 134*61c4878aSAndroid Build Coastguard Worker receiver to be ready. 135*61c4878aSAndroid Build Coastguard Worker- **Accounting for the MTU** -- HDLC and pw_rpc have variable overheads, so it 136*61c4878aSAndroid Build Coastguard Worker is difficult to know how much memory to allocate for RPC payloads. If packets 137*61c4878aSAndroid Build Coastguard Worker are not sized properly with respect to the maximum transmission unit (MTU), 138*61c4878aSAndroid Build Coastguard Worker packets may be silently dropped. 139*61c4878aSAndroid Build Coastguard Worker 140*61c4878aSAndroid Build Coastguard WorkerProblem summary 141*61c4878aSAndroid Build Coastguard Worker=============== 142*61c4878aSAndroid Build Coastguard WorkerThese are the key issues of Pigweed's communications offerings based on the 143*61c4878aSAndroid Build Coastguard Workerteam's experiences deploying pw_rpc. 144*61c4878aSAndroid Build Coastguard Worker 145*61c4878aSAndroid Build Coastguard Worker**No cohesive full stack solution** 146*61c4878aSAndroid Build Coastguard Worker 147*61c4878aSAndroid Build Coastguard WorkerPigweed only provides a handful of communications modules. They were not 148*61c4878aSAndroid Build Coastguard Workerdesigned to work together, and there is not enough to assemble a functioning 149*61c4878aSAndroid Build Coastguard Workernetwork stack. Some projects have to create bespoke network protocols with 150*61c4878aSAndroid Build Coastguard Workerlimited reusability. 151*61c4878aSAndroid Build Coastguard Worker 152*61c4878aSAndroid Build Coastguard Worker**Layering violations** 153*61c4878aSAndroid Build Coastguard Worker 154*61c4878aSAndroid Build Coastguard Workerpw_transfer runs on top of pw_rpc instead of the transport layer, which adds 155*61c4878aSAndroid Build Coastguard Workeroverhead and prevents its use independent of pw_rpc. Using pw_rpc channels for 156*61c4878aSAndroid Build Coastguard Workerrouting ties the network to pw_rpc. Projects often use pw_hdlc for multiple 157*61c4878aSAndroid Build Coastguard Workernetwork layers, which brings the encoding's overhead higher up the stack and 158*61c4878aSAndroid Build Coastguard Workeracross links that do not need it. 159*61c4878aSAndroid Build Coastguard Worker 160*61c4878aSAndroid Build Coastguard Worker**Inefficiency** 161*61c4878aSAndroid Build Coastguard Worker 162*61c4878aSAndroid Build Coastguard WorkerReliable data transfer requires pw_transfer, which runs on top of pw_rpc. This 163*61c4878aSAndroid Build Coastguard Workeradds additional overhead and requires more CPU-intensive decoding operations. 164*61c4878aSAndroid Build Coastguard WorkerUsing pw_rpc channel IDs in lower layers of the network requires expensive 165*61c4878aSAndroid Build Coastguard Workervarint decodes, even when the packets are bound for other nodes. 166*61c4878aSAndroid Build Coastguard Worker 167*61c4878aSAndroid Build Coastguard Worker**Missing features** 168*61c4878aSAndroid Build Coastguard Worker 169*61c4878aSAndroid Build Coastguard WorkerEach project has to develop its own version of common features, including: 170*61c4878aSAndroid Build Coastguard Worker 171*61c4878aSAndroid Build Coastguard Worker- **Addressing** -- There are no standard addressing schemes available to 172*61c4878aSAndroid Build Coastguard Worker Pigweed users. 173*61c4878aSAndroid Build Coastguard Worker- **Routing** -- Projects must implement their own logic for routing packets, 174*61c4878aSAndroid Build Coastguard Worker which can be complex. 175*61c4878aSAndroid Build Coastguard Worker- **Flow control** -- There is no way for the receiver to signal that it is ready 176*61c4878aSAndroid Build Coastguard Worker for more data or that it cannot receive any more, either at the protocol or 177*61c4878aSAndroid Build Coastguard Worker API level anywhere in the stack. Flow control is a crucial feature for 178*61c4878aSAndroid Build Coastguard Worker realistic networks with limited resources. 179*61c4878aSAndroid Build Coastguard Worker- **Connections** -- Connections ensure the recipient is listening to 180*61c4878aSAndroid Build Coastguard Worker transmissions, and detect when the other end is no longer communicating. 181*61c4878aSAndroid Build Coastguard Worker pw_transfer maintains a connection, but it sits atop pw_rpc, so cannot be used 182*61c4878aSAndroid Build Coastguard Worker elsewhere. 183*61c4878aSAndroid Build Coastguard Worker- **Quality of service (QoS)** -- Projects have developed basic QoS features in 184*61c4878aSAndroid Build Coastguard Worker HDLC, but there is no support in upstream Pigweed. Every project has to 185*61c4878aSAndroid Build Coastguard Worker develop its own custom implementation. 186*61c4878aSAndroid Build Coastguard Worker 187*61c4878aSAndroid Build Coastguard Worker----- 188*61c4878aSAndroid Build Coastguard WorkerGoals 189*61c4878aSAndroid Build Coastguard Worker----- 190*61c4878aSAndroid Build Coastguard WorkerThis SEED proposes a new communications system for Pigweed with the following 191*61c4878aSAndroid Build Coastguard Workergoals: 192*61c4878aSAndroid Build Coastguard Worker 193*61c4878aSAndroid Build Coastguard Worker- **Practical end-to-end solution** -- Pigweed provides a full suite of APIs 194*61c4878aSAndroid Build Coastguard Worker and protocols that support simple and complex networking use cases. 195*61c4878aSAndroid Build Coastguard Worker- **Robust, stable, and reliable** -- Pigweed communications "just work", even 196*61c4878aSAndroid Build Coastguard Worker under high load. The networking stack is thoroughly tested in both single and 197*61c4878aSAndroid Build Coastguard Worker multithreaded environments, with functional, load, fuzz, and performance 198*61c4878aSAndroid Build Coastguard Worker testing. Projects can easily test their own deployments with Pigweed tooling. 199*61c4878aSAndroid Build Coastguard Worker- **Cohesive, yet modular** -- The network stack is holistically designed, but 200*61c4878aSAndroid Build Coastguard Worker modular. It is organized into layers that can be exchanged and configured 201*61c4878aSAndroid Build Coastguard Worker independently. Layering simplifies the stack, decouples protocol 202*61c4878aSAndroid Build Coastguard Worker implementations, and maximizes flexibility within a cohesive system. 203*61c4878aSAndroid Build Coastguard Worker- **Efficient & performant** -- Pigweed’s network stack minimizes code size and 204*61c4878aSAndroid Build Coastguard Worker CPU usage. It provides for high throughput, low latency data transmission. 205*61c4878aSAndroid Build Coastguard Worker Memory allocation is configurable and adaptable to a project’s needs. 206*61c4878aSAndroid Build Coastguard Worker- **Usable & easy to learn** -- Pigweed’s communications systems are backed by 207*61c4878aSAndroid Build Coastguard Worker thorough and up-to-date documentation. Getting started is easy using 208*61c4878aSAndroid Build Coastguard Worker Pigweed's tutorials and examples. 209*61c4878aSAndroid Build Coastguard Worker 210*61c4878aSAndroid Build Coastguard Worker-------- 211*61c4878aSAndroid Build Coastguard WorkerProposal 212*61c4878aSAndroid Build Coastguard Worker-------- 213*61c4878aSAndroid Build Coastguard WorkerPigweed will unify its communications systems under a common sockets API. This 214*61c4878aSAndroid Build Coastguard Workerentails the following: 215*61c4878aSAndroid Build Coastguard Worker 216*61c4878aSAndroid Build Coastguard Worker- **Sockets API** -- Pigweed will introduce a `sockets 217*61c4878aSAndroid Build Coastguard Worker API`_ to serve as its common networking interface. 218*61c4878aSAndroid Build Coastguard Worker- **Lightweight protocol stack** -- Pigweed will provide a custom, 219*61c4878aSAndroid Build Coastguard Worker :ref:`lightweight network protocol stack <seed-0107-network-stack>` inspired 220*61c4878aSAndroid Build Coastguard Worker by IPv6, with UDP, TCP, and SCTP-like transport protocols. 221*61c4878aSAndroid Build Coastguard Worker- **TCP/IP integration** -- Pigweed will offer sockets implementations for OS 222*61c4878aSAndroid Build Coastguard Worker sockets and an existing `embedded TCP/IP stack`_. 223*61c4878aSAndroid Build Coastguard Worker- **Async** -- Pigweed will establish a new pattern for `async`_ programming and 224*61c4878aSAndroid Build Coastguard Worker use it in its networking APIs. 225*61c4878aSAndroid Build Coastguard Worker- **Zero copy** -- Pigweed will develop a new `buffer management`_ system to 226*61c4878aSAndroid Build Coastguard Worker enable zero-copy networking. 227*61c4878aSAndroid Build Coastguard Worker 228*61c4878aSAndroid Build Coastguard WorkerThese features fit fit into the TCP/IP model as follows: 229*61c4878aSAndroid Build Coastguard Worker 230*61c4878aSAndroid Build Coastguard Worker+-------------------------------------+-------------------------------------+ 231*61c4878aSAndroid Build Coastguard Worker| TCP/IP Model | Future Pigweed Comms Stack | 232*61c4878aSAndroid Build Coastguard Worker+=====================================+=====================================+ 233*61c4878aSAndroid Build Coastguard Worker| Application | | *Various modules including* | 234*61c4878aSAndroid Build Coastguard Worker| | | *pw_rpc and pw_transfer.* | 235*61c4878aSAndroid Build Coastguard Worker| | | 236*61c4878aSAndroid Build Coastguard Worker| | | 237*61c4878aSAndroid Build Coastguard Worker| | | 238*61c4878aSAndroid Build Coastguard Worker+-------------------------------------+-------------------------------------+ 239*61c4878aSAndroid Build Coastguard Worker| .. rst-class:: pw-text-center-align | .. rst-class:: pw-text-center-align | 240*61c4878aSAndroid Build Coastguard Worker| | | 241*61c4878aSAndroid Build Coastguard Worker| **OS Sockets** | **Pigweed Sockets** | 242*61c4878aSAndroid Build Coastguard Worker+-------------------------------------+-------------------------------------+ 243*61c4878aSAndroid Build Coastguard Worker| Transport | | UDP-like unreliable protocol | 244*61c4878aSAndroid Build Coastguard Worker| | | TCP-like reliable protocol | 245*61c4878aSAndroid Build Coastguard Worker| | | SCTP-like reliable protocol | 246*61c4878aSAndroid Build Coastguard Worker+-------------------------------------+-------------------------------------+ 247*61c4878aSAndroid Build Coastguard Worker| Network / Internet | | IPv6-like protocol | 248*61c4878aSAndroid Build Coastguard Worker+-------------------------------------+-------------------------------------+ 249*61c4878aSAndroid Build Coastguard Worker| Network access / Link | | HDLC | 250*61c4878aSAndroid Build Coastguard Worker| | | others | 251*61c4878aSAndroid Build Coastguard Worker+-------------------------------------+-------------------------------------+ 252*61c4878aSAndroid Build Coastguard Worker 253*61c4878aSAndroid Build Coastguard WorkerSockets API 254*61c4878aSAndroid Build Coastguard Worker=========== 255*61c4878aSAndroid Build Coastguard WorkerThe new sockets API will become the primary networking abstraction in Pigweed. 256*61c4878aSAndroid Build Coastguard WorkerThe API will support the following: 257*61c4878aSAndroid Build Coastguard Worker 258*61c4878aSAndroid Build Coastguard Worker- Creating sockets for bidirectional communications with other nodes in the 259*61c4878aSAndroid Build Coastguard Worker network. 260*61c4878aSAndroid Build Coastguard Worker- Opening and closing connections for connection-oriented socket types. 261*61c4878aSAndroid Build Coastguard Worker- Sending and receiving data, optionally :ref:`asynchronously 262*61c4878aSAndroid Build Coastguard Worker <seed-0107-async>`. 263*61c4878aSAndroid Build Coastguard Worker- Reporting errors. 264*61c4878aSAndroid Build Coastguard Worker 265*61c4878aSAndroid Build Coastguard WorkerThe sockets API will support runtime polymorphism. In C++, it will be a virtual 266*61c4878aSAndroid Build Coastguard Workerinterface. 267*61c4878aSAndroid Build Coastguard Worker 268*61c4878aSAndroid Build Coastguard Worker**Rationale** 269*61c4878aSAndroid Build Coastguard Worker 270*61c4878aSAndroid Build Coastguard WorkerA network socket represents a bidirectional communications channel with another 271*61c4878aSAndroid Build Coastguard Workernode, which could be local or across the Internet. Network sockets form the API 272*61c4878aSAndroid Build Coastguard Workerbetween an application and the network. 273*61c4878aSAndroid Build Coastguard Worker 274*61c4878aSAndroid Build Coastguard WorkerSockets are a proven, well-understood concept. Socket APIs such as Berkeley / 275*61c4878aSAndroid Build Coastguard WorkerPOSIX sockets are familiar to anyone with Internet programming experience. 276*61c4878aSAndroid Build Coastguard Worker 277*61c4878aSAndroid Build Coastguard WorkerSockets APIs hide the details of the network protocol stack. A socket provides 278*61c4878aSAndroid Build Coastguard Workerwell-defined semantics for a communications channel, but applications do not 279*61c4878aSAndroid Build Coastguard Workerneed to know how data is sent and received. The same API can be used to exchange 280*61c4878aSAndroid Build Coastguard Workerdata with another process on the same machine or with a device across the world. 281*61c4878aSAndroid Build Coastguard Worker 282*61c4878aSAndroid Build Coastguard Worker.. admonition:: Sockets SEEDs 283*61c4878aSAndroid Build Coastguard Worker 284*61c4878aSAndroid Build Coastguard Worker The Pigweed sockets API is described in SEED-0116. The sockets API is based 285*61c4878aSAndroid Build Coastguard Worker on ``pw_channel``, which is proposed in SEED-0114. 286*61c4878aSAndroid Build Coastguard Worker 287*61c4878aSAndroid Build Coastguard WorkerSocket types 288*61c4878aSAndroid Build Coastguard Worker------------ 289*61c4878aSAndroid Build Coastguard WorkerPigweed's sockets API will support the following sockets types. 290*61c4878aSAndroid Build Coastguard Worker 291*61c4878aSAndroid Build Coastguard Worker.. list-table:: 292*61c4878aSAndroid Build Coastguard Worker :header-rows: 1 293*61c4878aSAndroid Build Coastguard Worker 294*61c4878aSAndroid Build Coastguard Worker * - Berkeley socket type 295*61c4878aSAndroid Build Coastguard Worker - Internet protocol 296*61c4878aSAndroid Build Coastguard Worker - Payload type 297*61c4878aSAndroid Build Coastguard Worker - Connection-oriented 298*61c4878aSAndroid Build Coastguard Worker - Guaranteed, ordered delivery 299*61c4878aSAndroid Build Coastguard Worker - Description 300*61c4878aSAndroid Build Coastguard Worker * - ``SOCK_DGRAM`` 301*61c4878aSAndroid Build Coastguard Worker - UDP 302*61c4878aSAndroid Build Coastguard Worker - Datagram 303*61c4878aSAndroid Build Coastguard Worker - ❌ 304*61c4878aSAndroid Build Coastguard Worker - ❌ 305*61c4878aSAndroid Build Coastguard Worker - Unreliable datagram 306*61c4878aSAndroid Build Coastguard Worker * - ``SOCK_STREAM`` 307*61c4878aSAndroid Build Coastguard Worker - TCP 308*61c4878aSAndroid Build Coastguard Worker - Byte stream 309*61c4878aSAndroid Build Coastguard Worker - ✅ 310*61c4878aSAndroid Build Coastguard Worker - ✅ 311*61c4878aSAndroid Build Coastguard Worker - Reliable byte stream 312*61c4878aSAndroid Build Coastguard Worker * - ``SOCK_SEQPACKET`` 313*61c4878aSAndroid Build Coastguard Worker - SCTP 314*61c4878aSAndroid Build Coastguard Worker - Datagram 315*61c4878aSAndroid Build Coastguard Worker - ✅ 316*61c4878aSAndroid Build Coastguard Worker - ✅ 317*61c4878aSAndroid Build Coastguard Worker - Reliable datagram 318*61c4878aSAndroid Build Coastguard Worker 319*61c4878aSAndroid Build Coastguard WorkerRaw sockets (``SOCK_RAW``) may be supported in the future if required. 320*61c4878aSAndroid Build Coastguard Worker``SOCK_CONN_DGRAM`` (unreliable connection-oriented datagram) sockets are 321*61c4878aSAndroid Build Coastguard Workeruncommon and will not be supported. 322*61c4878aSAndroid Build Coastguard Worker 323*61c4878aSAndroid Build Coastguard WorkerThe socket's semantics will be expressed in the sockets API, e.g. with a 324*61c4878aSAndroid Build Coastguard Workerdifferent interface or class for each type. Instances of the connection-oriented 325*61c4878aSAndroid Build Coastguard Workersocket types will be generated from a "listener" object. 326*61c4878aSAndroid Build Coastguard Worker 327*61c4878aSAndroid Build Coastguard WorkerPigweed's sockets API will draw inspiration from modern type safe APIs like 328*61c4878aSAndroid Build Coastguard WorkerRust's `std::net sockets <https://doc.rust-lang.org/std/net/index.html>`_, 329*61c4878aSAndroid Build Coastguard Workerrather than traditional APIs like POSIX sockets or Winsock. Pigweed sockets will 330*61c4878aSAndroid Build Coastguard Workermap trivially to these APIs and implementations will be provided upstream. 331*61c4878aSAndroid Build Coastguard Worker 332*61c4878aSAndroid Build Coastguard WorkerUsing the sockets API 333*61c4878aSAndroid Build Coastguard Worker--------------------- 334*61c4878aSAndroid Build Coastguard WorkerThe Pigweed sockets API will provide the interface between applications and the 335*61c4878aSAndroid Build Coastguard Workernetwork. Any application can open a socket to communicate across the network. 336*61c4878aSAndroid Build Coastguard WorkerA future revision of ``pw_rpc`` will use the sockets API in place of its current 337*61c4878aSAndroid Build Coastguard Worker``Channel`` API. 338*61c4878aSAndroid Build Coastguard Worker 339*61c4878aSAndroid Build Coastguard WorkerThe sockets API will support both synchronous and :ref:`asynchronous 340*61c4878aSAndroid Build Coastguard Worker<seed-0107-async>` use. The synchronous API may be built using the async API. 341*61c4878aSAndroid Build Coastguard WorkerIt will also support :ref:`zero-copy <seed-0107-buffers>` data transmission. 342*61c4878aSAndroid Build Coastguard Worker 343*61c4878aSAndroid Build Coastguard WorkerAddressing 344*61c4878aSAndroid Build Coastguard Worker---------- 345*61c4878aSAndroid Build Coastguard WorkerThe Pigweed sockets API will be aware of addresses. Addresses are used to refer 346*61c4878aSAndroid Build Coastguard Workerto nodes in a network, including the socket's own node. With TCP/IP, the socket 347*61c4878aSAndroid Build Coastguard Workeraddress includes an IP address and a port number. 348*61c4878aSAndroid Build Coastguard Worker 349*61c4878aSAndroid Build Coastguard WorkerThe POSIX sockets API supports different domains through address family 350*61c4878aSAndroid Build Coastguard Workerconstants such as ``AF_INET``, ``AF_INET6``, and ``AF_UNIX``. Addresses in these 351*61c4878aSAndroid Build Coastguard Workerfamilies are specified or accessed in various socket operations. Because the 352*61c4878aSAndroid Build Coastguard Workeraddress format is not specified by the API, working with addresses is not type 353*61c4878aSAndroid Build Coastguard Workersafe. 354*61c4878aSAndroid Build Coastguard Worker 355*61c4878aSAndroid Build Coastguard WorkerPigweed sockets will approach addressing differently, but details are yet to be 356*61c4878aSAndroid Build Coastguard Workerdetermined. Possible approaches include: 357*61c4878aSAndroid Build Coastguard Worker 358*61c4878aSAndroid Build Coastguard Worker- Use IPv6 addresses exclusively. Systems with other addressing schemes map 359*61c4878aSAndroid Build Coastguard Worker these into IPv6 for use with Pigweed APIs. 360*61c4878aSAndroid Build Coastguard Worker- Provide a polymorphic address class so sockets can work with addresses 361*61c4878aSAndroid Build Coastguard Worker generically. 362*61c4878aSAndroid Build Coastguard Worker- Avoid addresses in the base sockets API. Instead, use implementation specific 363*61c4878aSAndroid Build Coastguard Worker derived classes to access addresses. 364*61c4878aSAndroid Build Coastguard Worker 365*61c4878aSAndroid Build Coastguard WorkerNetwork protocol stack 366*61c4878aSAndroid Build Coastguard Worker====================== 367*61c4878aSAndroid Build Coastguard WorkerThe sockets API will be backed by a network protocol stack. Pigweed will provide 368*61c4878aSAndroid Build Coastguard Workersockets implementations for following network protocol stacks: 369*61c4878aSAndroid Build Coastguard Worker 370*61c4878aSAndroid Build Coastguard Worker* Third party embedded TCP/IP stack, most likely `lwIP 371*61c4878aSAndroid Build Coastguard Worker <https://savannah.nongnu.org/projects/lwip/>`_. 372*61c4878aSAndroid Build Coastguard Worker* Operating system TCP/IP stack via POSIX sockets or `Winsock 373*61c4878aSAndroid Build Coastguard Worker <https://learn.microsoft.com/en-us/windows/win32/winsock/windows-sockets-start-page-2>`_. 374*61c4878aSAndroid Build Coastguard Worker* Custom :ref:`lightweight network protocol stack <seed-0107-network-stack>`. 375*61c4878aSAndroid Build Coastguard Worker 376*61c4878aSAndroid Build Coastguard WorkerEmbedded TCP/IP stack 377*61c4878aSAndroid Build Coastguard Worker--------------------- 378*61c4878aSAndroid Build Coastguard WorkerPigweed will provide a sockets implementation for an embedded TCP/IP stack such 379*61c4878aSAndroid Build Coastguard Workeras `lwIP <https://savannah.nongnu.org/projects/lwip/>`_. 380*61c4878aSAndroid Build Coastguard Worker 381*61c4878aSAndroid Build Coastguard WorkerThe sockets integration will be structured to avoid unnecessary dependencies on 382*61c4878aSAndroid Build Coastguard Workernetwork stack features. For example, if a system is using IPv6 exclusively, the 383*61c4878aSAndroid Build Coastguard Workerintegration won't require IPv4 support, and the TCP/IP stack can be configured 384*61c4878aSAndroid Build Coastguard Workerwithout it. 385*61c4878aSAndroid Build Coastguard Worker 386*61c4878aSAndroid Build Coastguard Worker**Rationale** 387*61c4878aSAndroid Build Coastguard Worker 388*61c4878aSAndroid Build Coastguard WorkerThe Internet protocol suite, or TCP/IP, is informed by decades of research and 389*61c4878aSAndroid Build Coastguard Workerpractical experience. It is much more than IP, TCP, and UDP; it's an alphabet 390*61c4878aSAndroid Build Coastguard Workersoup of protocols that address a myriad of use cases and challenges. 391*61c4878aSAndroid Build Coastguard WorkerImplementing a functional TCP/IP stack is no small task. At time of writing, 392*61c4878aSAndroid Build Coastguard WorkerlwIP has about as many lines of C as Pigweed has C++ (excluding tests). 393*61c4878aSAndroid Build Coastguard Worker 394*61c4878aSAndroid Build Coastguard WorkerThe Pigweed team does not plan to implement a full TCP/IP stack. This is a major 395*61c4878aSAndroid Build Coastguard Workerundertaking, and there are already established open source embedded TCP/IP 396*61c4878aSAndroid Build Coastguard Workerstacks. Projects needing the full power of TCP/IP can use an embedded stack like 397*61c4878aSAndroid Build Coastguard Worker`lwIP <https://savannah.nongnu.org/projects/lwip/>`_. 398*61c4878aSAndroid Build Coastguard Worker 399*61c4878aSAndroid Build Coastguard WorkerChoosing between embedded TCP/IP and :ref:`Pigweed's stack <seed-0107-network-stack>` 400*61c4878aSAndroid Build Coastguard Worker^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 401*61c4878aSAndroid Build Coastguard WorkerlwIP's `website <https://savannah.nongnu.org/projects/lwip/>`_ states that it 402*61c4878aSAndroid Build Coastguard Workerrequires tens of KB of RAM and about 40 KB of ROM. Using lwIP means using the 403*61c4878aSAndroid Build Coastguard Workersame TCP/IP protocols that run the Internet. These protocols are feature rich, 404*61c4878aSAndroid Build Coastguard Workerbut have more overhead than is necessary for local communications within a small 405*61c4878aSAndroid Build Coastguard Workerembedded system. 406*61c4878aSAndroid Build Coastguard Worker 407*61c4878aSAndroid Build Coastguard WorkerProjects that can afford the resource requirements and protocol overhead of 408*61c4878aSAndroid Build Coastguard WorkerTCP/IP should use it. These projects can set up a local IPv4 or IPv6 network 409*61c4878aSAndroid Build Coastguard Workerand use it for communications behind the Pigweed sockets API. Projects that 410*61c4878aSAndroid Build Coastguard Workercannot afford full TCP/IP can opt for Pigweed's :ref:`custom protocol stack 411*61c4878aSAndroid Build Coastguard Worker<seed-0107-network-stack>`. Pigweed's custom stack will not have the depth of 412*61c4878aSAndroid Build Coastguard Workerfeatures and tooling of TCP/IP does, but will be sufficient for many systems. 413*61c4878aSAndroid Build Coastguard Worker 414*61c4878aSAndroid Build Coastguard WorkerTCP/IP socket types 415*61c4878aSAndroid Build Coastguard Worker^^^^^^^^^^^^^^^^^^^ 416*61c4878aSAndroid Build Coastguard WorkerWith an embedded TCP/IP stack, the Pigweed sockets API will be implemented as 417*61c4878aSAndroid Build Coastguard Workerfollows: 418*61c4878aSAndroid Build Coastguard Worker 419*61c4878aSAndroid Build Coastguard Worker- Unreliable datagram (``SOCK_DGRAM``) -- UDP 420*61c4878aSAndroid Build Coastguard Worker- Reliable byte stream (``SOCK_STREAM``) -- TCP 421*61c4878aSAndroid Build Coastguard Worker- Reliable datagram (``SOCK_SEQPACKET``) -- Lightweight framing over TCP. This 422*61c4878aSAndroid Build Coastguard Worker will be semantically similar to `SCTP 423*61c4878aSAndroid Build Coastguard Worker <https://datatracker.ietf.org/doc/html/rfc9260>`_, but integrations will not 424*61c4878aSAndroid Build Coastguard Worker use SCTP since it is not widely supported. 425*61c4878aSAndroid Build Coastguard Worker 426*61c4878aSAndroid Build Coastguard Worker.. _seed-0107-network-stack: 427*61c4878aSAndroid Build Coastguard Worker 428*61c4878aSAndroid Build Coastguard WorkerPigweed's custom network protocol stack 429*61c4878aSAndroid Build Coastguard Worker--------------------------------------- 430*61c4878aSAndroid Build Coastguard WorkerPigweed will develop a custom, lightweight network protocol stack. 431*61c4878aSAndroid Build Coastguard Worker 432*61c4878aSAndroid Build Coastguard WorkerThis new protocol stack will be designed for small devices with relatively 433*61c4878aSAndroid Build Coastguard Workersimple networks. It will scale to several interconnected cores that interface 434*61c4878aSAndroid Build Coastguard Workerwith a few external devices (e.g. over USB or Bluetooth). Depending on project 435*61c4878aSAndroid Build Coastguard Workerrequirements, it may or may not support dynamic network host configuration (e.g. 436*61c4878aSAndroid Build Coastguard WorkerDHCP or SLAAC). 437*61c4878aSAndroid Build Coastguard Worker 438*61c4878aSAndroid Build Coastguard WorkerPigweed's network protocol stack will be a strict subset of TCP/IP. This will 439*61c4878aSAndroid Build Coastguard Workerinclude minimal, reduced overhead versions of UDP, TCP, and IPv6. Portions of 440*61c4878aSAndroid Build Coastguard Workerother protocols such as ICMPv6 may be implemented as required. 441*61c4878aSAndroid Build Coastguard Worker 442*61c4878aSAndroid Build Coastguard Worker**Rationale** 443*61c4878aSAndroid Build Coastguard Worker 444*61c4878aSAndroid Build Coastguard WorkerTCP/IP is too large and complex for some embedded systems. Systems for which 445*61c4878aSAndroid Build Coastguard WorkerTCP/IP is unnecessary can use Pigweed's lightweight embedded network protocol 446*61c4878aSAndroid Build Coastguard Workerstack. 447*61c4878aSAndroid Build Coastguard Worker 448*61c4878aSAndroid Build Coastguard WorkerTransport layer 449*61c4878aSAndroid Build Coastguard Worker^^^^^^^^^^^^^^^ 450*61c4878aSAndroid Build Coastguard WorkerPigweed will provide transport layer protocols that implement the semantics of 451*61c4878aSAndroid Build Coastguard Worker``SOCK_DGRAM``, ``SOCK_STREAM``, and ``SOCK_SEQPACKET``-like sockets. 452*61c4878aSAndroid Build Coastguard Worker 453*61c4878aSAndroid Build Coastguard Worker- ``SOCK_DRAM``-like sockets will be backed by a UDP-like protocol. This will 454*61c4878aSAndroid Build Coastguard Worker add source and destination ports to the IP-style packets for multiplexing on 455*61c4878aSAndroid Build Coastguard Worker top of the network layer. 456*61c4878aSAndroid Build Coastguard Worker- ``SOCK_STREAM``-like sockets will be backed by a TCP-like protocol that uses 457*61c4878aSAndroid Build Coastguard Worker network layer packets to implement a reliable byte stream. It will be based on 458*61c4878aSAndroid Build Coastguard Worker TCP, but will not implement all of its features. The :ref:`module-pw_transfer` 459*61c4878aSAndroid Build Coastguard Worker module may serve as a starting point for the new protocol implementation. 460*61c4878aSAndroid Build Coastguard Worker- ``SOCK_SEQPACKET``-like sockets will be implemented with a simple 461*61c4878aSAndroid Build Coastguard Worker message-oriented protocol on top of the TCP-like protocol. Applications like 462*61c4878aSAndroid Build Coastguard Worker pw_rpc will use ``SOCK_SEQPACKET`` sockets. 463*61c4878aSAndroid Build Coastguard Worker 464*61c4878aSAndroid Build Coastguard WorkerNetwork layer 465*61c4878aSAndroid Build Coastguard Worker^^^^^^^^^^^^^ 466*61c4878aSAndroid Build Coastguard WorkerPigweed will create a new network-layer protocol closely based on IPv6. Details 467*61c4878aSAndroid Build Coastguard Workerare still to be determined, but the protocol is intended to be a strict subset 468*61c4878aSAndroid Build Coastguard Workerof IPv6 and related protocols (e.g. ICMP, NDP) as needed. If a need arises, it 469*61c4878aSAndroid Build Coastguard Workeris met by implementing the associated IP suite protocol. Packets will use 470*61c4878aSAndroid Build Coastguard Workercompressed version of an IPv6 header (e.g. omit fields, use smaller addresses). 471*61c4878aSAndroid Build Coastguard Worker 472*61c4878aSAndroid Build Coastguard WorkerThis protocol will provide: 473*61c4878aSAndroid Build Coastguard Worker 474*61c4878aSAndroid Build Coastguard Worker- Unreliable packet delivery between source and destination. 475*61c4878aSAndroid Build Coastguard Worker- Routing based on the source and destination addresses. 476*61c4878aSAndroid Build Coastguard Worker- Quality of service (e.g. via the traffic class field). 477*61c4878aSAndroid Build Coastguard Worker 478*61c4878aSAndroid Build Coastguard WorkerPackets may be routed at this layer independently of the link layer. Wire format 479*61c4878aSAndroid Build Coastguard Workerdetails stay on the wire. 480*61c4878aSAndroid Build Coastguard Worker 481*61c4878aSAndroid Build Coastguard WorkerNetwork access / link layer 482*61c4878aSAndroid Build Coastguard Worker^^^^^^^^^^^^^^^^^^^^^^^^^^^ 483*61c4878aSAndroid Build Coastguard WorkerPigweed's network stack will interact with the link layer through a generic 484*61c4878aSAndroid Build Coastguard Workerinterface. This will allow Pigweed to send network packets with any protocol 485*61c4878aSAndroid Build Coastguard Workerover any physical interface. 486*61c4878aSAndroid Build Coastguard Worker 487*61c4878aSAndroid Build Coastguard WorkerPigweed already provides minimal support for one link layer protocol, HDLC. 488*61c4878aSAndroid Build Coastguard WorkerOther protocols (e.g. COBS, PPP) may be implemented. Some hardware interfaces 489*61c4878aSAndroid Build Coastguard Worker(e.g. Bluetooth, USB) may not require an additional link-layer protocol. 490*61c4878aSAndroid Build Coastguard Worker 491*61c4878aSAndroid Build Coastguard WorkerLanguage support 492*61c4878aSAndroid Build Coastguard Worker---------------- 493*61c4878aSAndroid Build Coastguard WorkerPigweed today is primarily C++, but it supports Rust, C, Python, TypeScript, and 494*61c4878aSAndroid Build Coastguard WorkerJava to varying extents. 495*61c4878aSAndroid Build Coastguard Worker 496*61c4878aSAndroid Build Coastguard WorkerPigweed’s communications stack will be developed in either C++ or Rust to start, 497*61c4878aSAndroid Build Coastguard Workerbut it will be ported to all supported languages in time. The stack may have C 498*61c4878aSAndroid Build Coastguard WorkerAPIs to facilitate interoperability between C++ and Rust. 499*61c4878aSAndroid Build Coastguard Worker 500*61c4878aSAndroid Build Coastguard Worker.. admonition:: Network protocol stack SEED 501*61c4878aSAndroid Build Coastguard Worker 502*61c4878aSAndroid Build Coastguard Worker Pigweed's network protocol stack will be explored in an upcoming SEED. 503*61c4878aSAndroid Build Coastguard Worker 504*61c4878aSAndroid Build Coastguard Worker.. _seed-0107-async: 505*61c4878aSAndroid Build Coastguard Worker 506*61c4878aSAndroid Build Coastguard WorkerAsync 507*61c4878aSAndroid Build Coastguard Worker===== 508*61c4878aSAndroid Build Coastguard WorkerPigweed will develop a model for asynchronous programming and use it in its 509*61c4878aSAndroid Build Coastguard Workernetworking APIs, including sockets. Sockets will also support synchronous 510*61c4878aSAndroid Build Coastguard Workeroperations, but these may be implemented in terms of the asynchronous API. 511*61c4878aSAndroid Build Coastguard Worker 512*61c4878aSAndroid Build Coastguard WorkerThe Pigweed async model has not been designed yet. The :ref:`pw_async 513*61c4878aSAndroid Build Coastguard Worker<module-pw_async>` module has a task dispatcher, but the pattern for async APIs 514*61c4878aSAndroid Build Coastguard Workerhas not been established. Further exploration is needed, but C++20 coroutines 515*61c4878aSAndroid Build Coastguard Workermay be used for Pigweed async APIs where supported. 516*61c4878aSAndroid Build Coastguard Worker 517*61c4878aSAndroid Build Coastguard Worker**Rationale** 518*61c4878aSAndroid Build Coastguard Worker 519*61c4878aSAndroid Build Coastguard WorkerSynchronous APIs require the thread to block while an operation completes. The 520*61c4878aSAndroid Build Coastguard Workerthread and its resources cannot be used by the system until the task completes. 521*61c4878aSAndroid Build Coastguard WorkerAsync APIs allow a single thread to handle multiple simultaneous tasks. The 522*61c4878aSAndroid Build Coastguard Workerthread advances tasks until they need to wait for an external operation to 523*61c4878aSAndroid Build Coastguard Workercomplete, then switches to another task to avoid blocking. 524*61c4878aSAndroid Build Coastguard Worker 525*61c4878aSAndroid Build Coastguard WorkerThreads are expensive in embedded systems. Each thread requires significant 526*61c4878aSAndroid Build Coastguard Workermemory for its stack and kernel structures for bookkeeping. They occupy this 527*61c4878aSAndroid Build Coastguard Workermemory all the time, even when they are not running. Furthermore, context 528*61c4878aSAndroid Build Coastguard Workerswitches between threads can take significant CPU time. 529*61c4878aSAndroid Build Coastguard Worker 530*61c4878aSAndroid Build Coastguard WorkerAsynchronous programming avoids these downsides. Many asynchronous threads run 531*61c4878aSAndroid Build Coastguard Workeron a single thread. Fewer threads are needed, and the resources of one thread 532*61c4878aSAndroid Build Coastguard Workerare shared by multiple tasks. Since asynchronous systems run within one thread, 533*61c4878aSAndroid Build Coastguard Workerno thread context switches occur. 534*61c4878aSAndroid Build Coastguard Worker 535*61c4878aSAndroid Build Coastguard WorkerNetworking involves many asynchronous tasks. For example, waiting for data to be 536*61c4878aSAndroid Build Coastguard Workersent through a network interface, for a connection request, or for data to 537*61c4878aSAndroid Build Coastguard Workerarrive on one or more interfaces are all operations that benefit from 538*61c4878aSAndroid Build Coastguard Workerasynchronous APIs. Network protocols themselves are heavily asynchronous. 539*61c4878aSAndroid Build Coastguard Worker 540*61c4878aSAndroid Build Coastguard Worker.. admonition:: Async SEED 541*61c4878aSAndroid Build Coastguard Worker 542*61c4878aSAndroid Build Coastguard Worker Pigweed's async pattern is proposed in :ref:`SEED-0112 <seed-0112>`. 543*61c4878aSAndroid Build Coastguard Worker 544*61c4878aSAndroid Build Coastguard Worker.. _seed-0107-buffers: 545*61c4878aSAndroid Build Coastguard Worker 546*61c4878aSAndroid Build Coastguard WorkerBuffer management 547*61c4878aSAndroid Build Coastguard Worker================= 548*61c4878aSAndroid Build Coastguard WorkerPigweed's networking APIs will support zero-copy data transmission. Applications 549*61c4878aSAndroid Build Coastguard Workerwill be able to request a buffer from a socket. When one is available, they fill 550*61c4878aSAndroid Build Coastguard Workerit with data for transmission. 551*61c4878aSAndroid Build Coastguard Worker 552*61c4878aSAndroid Build Coastguard WorkerPigweed will develop a general purpose module for allocating and managing 553*61c4878aSAndroid Build Coastguard Workerbuffers. This will be used to implement zero-copy features for Pigweed's 554*61c4878aSAndroid Build Coastguard Workernetworking stack. 555*61c4878aSAndroid Build Coastguard Worker 556*61c4878aSAndroid Build Coastguard WorkerAs an example, zero-copy buffer allocation could work as follows: 557*61c4878aSAndroid Build Coastguard Worker 558*61c4878aSAndroid Build Coastguard Worker- The user requests a buffer from a socket. 559*61c4878aSAndroid Build Coastguard Worker- The network protocol layer under the socket requests a buffer from the next 560*61c4878aSAndroid Build Coastguard Worker lower layer. 561*61c4878aSAndroid Build Coastguard Worker- The bottom protocol layer allocates a buffer. 562*61c4878aSAndroid Build Coastguard Worker- Each layer reserves part of the buffer for its headers or footers. 563*61c4878aSAndroid Build Coastguard Worker- The remaining buffer is provided to the user to populate with their payload. 564*61c4878aSAndroid Build Coastguard Worker- When the user is done, the buffer is released. Each layer of the network stack 565*61c4878aSAndroid Build Coastguard Worker processes the buffer as necessary. 566*61c4878aSAndroid Build Coastguard Worker- Finally, at the lowest layer, the final buffer is sent over the hardware 567*61c4878aSAndroid Build Coastguard Worker interface. 568*61c4878aSAndroid Build Coastguard Worker 569*61c4878aSAndroid Build Coastguard WorkerZero-copy APIs will be :ref:`asynchronous <seed-0107-async>`. 570*61c4878aSAndroid Build Coastguard Worker 571*61c4878aSAndroid Build Coastguard Worker**Rationale** 572*61c4878aSAndroid Build Coastguard Worker 573*61c4878aSAndroid Build Coastguard WorkerNetworking involves transmitting large amounts of data. Copying network traffic 574*61c4878aSAndroid Build Coastguard Workercan result in substantial CPU usage, particularly in nodes that route traffic to 575*61c4878aSAndroid Build Coastguard Workerother nodes. 576*61c4878aSAndroid Build Coastguard Worker 577*61c4878aSAndroid Build Coastguard WorkerA buffer management system that minimizes copying saves precious CPU cycles and 578*61c4878aSAndroid Build Coastguard Workerpower on constrained systems. 579*61c4878aSAndroid Build Coastguard Worker 580*61c4878aSAndroid Build Coastguard Worker.. admonition:: Buffer management SEED 581*61c4878aSAndroid Build Coastguard Worker 582*61c4878aSAndroid Build Coastguard Worker Pigweed's buffer management system is proposed in :ref:`SEED-0109 583*61c4878aSAndroid Build Coastguard Worker <seed-0109>`. 584*61c4878aSAndroid Build Coastguard Worker 585*61c4878aSAndroid Build Coastguard WorkerVectored I/O 586*61c4878aSAndroid Build Coastguard Worker------------ 587*61c4878aSAndroid Build Coastguard WorkerVectored or scatter/gather I/O allows users to serialize data from multiple 588*61c4878aSAndroid Build Coastguard Workerbuffers into a single output stream, or vice versa. For Pigweed's networking 589*61c4878aSAndroid Build Coastguard WorkerAPIs, this could be used to, for example, store a packet header in one buffer 590*61c4878aSAndroid Build Coastguard Workerand packet contents in one or more other buffers. These isolated chunks are 591*61c4878aSAndroid Build Coastguard Workerserialized in order to the network interface. 592*61c4878aSAndroid Build Coastguard Worker 593*61c4878aSAndroid Build Coastguard WorkerVectored I/O minimizes copying, but is complex. Additionally, simple DMA engines 594*61c4878aSAndroid Build Coastguard Workermay only operate on a single DMA buffer. Thus, vectored I/O could require 595*61c4878aSAndroid Build Coastguard Workereither: 596*61c4878aSAndroid Build Coastguard Worker 597*61c4878aSAndroid Build Coastguard Worker- a copy into the DMA engine's buffer, which costs CPU time and memory, or 598*61c4878aSAndroid Build Coastguard Worker- multiple, small DMAs, which involves extra interrupts and CPU time. 599*61c4878aSAndroid Build Coastguard Worker 600*61c4878aSAndroid Build Coastguard WorkerVectored I/O may be supported in Pigweed's communications stack, depending on 601*61c4878aSAndroid Build Coastguard Workerproject requirements. 602*61c4878aSAndroid Build Coastguard Worker 603*61c4878aSAndroid Build Coastguard Worker---------- 604*61c4878aSAndroid Build Coastguard WorkerNext steps 605*61c4878aSAndroid Build Coastguard Worker---------- 606*61c4878aSAndroid Build Coastguard WorkerPigweed's communications revamp will proceed loosely as follows: 607*61c4878aSAndroid Build Coastguard Worker 608*61c4878aSAndroid Build Coastguard Worker* Write SEEDs to explore existing solutions, distill requirements, and propose 609*61c4878aSAndroid Build Coastguard Worker new Pigweed features for these areas: 610*61c4878aSAndroid Build Coastguard Worker 611*61c4878aSAndroid Build Coastguard Worker - Sockets API (SEED-0116) 612*61c4878aSAndroid Build Coastguard Worker - Async pattern (:ref:`SEED-0112 <seed-0112>`). 613*61c4878aSAndroid Build Coastguard Worker - Buffer management (:ref:`SEED-0109 <seed-0109>`) 614*61c4878aSAndroid Build Coastguard Worker - Network protocol stack 615*61c4878aSAndroid Build Coastguard Worker 616*61c4878aSAndroid Build Coastguard Worker* Implement the Sockets API. 617*61c4878aSAndroid Build Coastguard Worker 618*61c4878aSAndroid Build Coastguard Worker - Document, integrate, and deploy the async programming pattern for Pigweed. 619*61c4878aSAndroid Build Coastguard Worker - Develop and test Pigweed's buffer management system. 620*61c4878aSAndroid Build Coastguard Worker - Use these features in the sockets API. If necessary, the synchronous, 621*61c4878aSAndroid Build Coastguard Worker copying API could be implemented first. 622*61c4878aSAndroid Build Coastguard Worker 623*61c4878aSAndroid Build Coastguard Worker* Deploy the sockets API for TCP/IP. 624*61c4878aSAndroid Build Coastguard Worker 625*61c4878aSAndroid Build Coastguard Worker - Implement and unit test sockets for TCP/IP with POSIX and Winsock sockets. 626*61c4878aSAndroid Build Coastguard Worker - Implement and unit test sockets for an embedded TCP/IP stack. 627*61c4878aSAndroid Build Coastguard Worker 628*61c4878aSAndroid Build Coastguard Worker* Develop a test suite for Pigweed network communications. 629*61c4878aSAndroid Build Coastguard Worker 630*61c4878aSAndroid Build Coastguard Worker - Create integration tests for networks with multiple nodes that cover basic 631*61c4878aSAndroid Build Coastguard Worker operation, high load, and packet loss. 632*61c4878aSAndroid Build Coastguard Worker - Write performance tests against the sockets API to measure network stack 633*61c4878aSAndroid Build Coastguard Worker performance. 634*61c4878aSAndroid Build Coastguard Worker 635*61c4878aSAndroid Build Coastguard Worker* Develop Pigweed's lightweight network protocol stack. 636*61c4878aSAndroid Build Coastguard Worker 637*61c4878aSAndroid Build Coastguard Worker - Test the lightweight network protocol stack on hardware and in a simulated 638*61c4878aSAndroid Build Coastguard Worker environment. 639*61c4878aSAndroid Build Coastguard Worker - Write fuzz tests for the protocol stack. 640*61c4878aSAndroid Build Coastguard Worker - Write performance tests for the protocol stack. 641*61c4878aSAndroid Build Coastguard Worker 642*61c4878aSAndroid Build Coastguard Worker* Revisit other communications systems, including pw_rpc and pw_transfer. 643