xref: /aosp_15_r20/external/pigweed/seed/0107.rst (revision 61c4878ac05f98d0ceed94b57d316916de578985)
1.. _seed-0107:
2
3============================
40107: Pigweed Communications
5============================
6.. seed::
7   :number: 107
8   :name: Communications
9   :status: Accepted
10   :proposal_date: 2023-07-19
11   :cl: 157090
12   :authors: Wyatt Hepler
13   :facilitator: Carlos Chinchilla
14
15-------
16Summary
17-------
18Pigweed does not currently offer an end-to-end solution for network
19communications. This SEED proposes that Pigweed adopt a new sockets API as its
20primary networking abstraction. The sockets API will be backed by a new,
21lightweight embedded-focused network protocol stack, inspired by the Internet
22protocol suite (TCP/IP). It will also support full TCP/IP via an open source
23embedded TCP/IP stack or OS sockets. The new communications APIs will support
24asynchronous use and zero-copy transmission.
25
26This work is comprised of the following subareas:
27
28- `Sockets API`_
29- `Network protocol stack`_
30- `Async`_ API pattern
31- `Buffer management`_ system
32
33The Pigweed team will revisit :ref:`pw_rpc <module-pw_rpc>` after deploying the
34sockets API and network protocol stack.
35
36----------
37Background
38----------
39Pigweed's primary communications system is :ref:`pw_rpc <module-pw_rpc>`. pw_rpc
40makes it possible to call functions and exchange data with a remote system.
41Requests and responses are encoded as protobufs. pw_rpc was initially deployed
42to a system with its own network protocol stack, so the Pigweed team did not
43invest in building a network stack of its own.
44
45The TCP/IP model, as described in `RFC 1122
46<https://datatracker.ietf.org/doc/html/rfc1122>`_, organizes communications
47systems and protocols into four layers: Application, Transport, Internet (or
48Network), and Link (or Network access). Pigweed's current communications
49offerings fit into the TCP/IP model as follows:
50
51+-----------------------+-----------------------------+
52| TCP/IP Model          | Pigweed Modules             |
53+=======================+=============================+
54| Application           | | :ref:`module-pw_transfer` |
55|                       | | :ref:`module-pw_rpc`      |
56+-----------------------+-----------------------------+
57| Transport             | | :ref:`module-pw_router`   |
58+-----------------------+ | :ref:`module-pw_hdlc`     |
59| Internet / Network    |                             |
60+-----------------------+                             |
61| Link / Network access |                             |
62+-----------------------+-----------------------------+
63
64Notably, Pigweed provides little functionality below the application layer. The
65pw_router and pw_hdlc modules only implement a subset of features needed at
66their layer in the communications stack.
67
68Challenges deploying pw_rpc
69===========================
70pw_rpc is application-layer communications module. It relies on a network layer
71to send packets between endpoints and doesn't provide any networking features
72itself. When initially developing pw_rpc, the Pigweed team focused its limited
73resources solely on this application-layer feature, which made it possible to
74deploy pw_rpc quickly to systems with existing networks.
75
76pw_rpc has been deployed to many projects with great results. However, since
77Pigweed does not provide a network stack, deploying pw_rpc to systems without
78existing stacks can be challenging. These systems have to develop their own
79solutions to transmit and route pw_rpc packets.
80
81As an example, one project based its network communications on Pigweed's
82:ref:`module-pw_hdlc` module. It used HDLC in a way more similar to IP,
83providing network-level addressing and features like quality-of-service. Source
84and destination addresses and ports were packed into the HDLC address field to
85facilitate routing and multiplexing. The :ref:`module-pw_router` module was
86developed to support static routing tables for HDLC frames through nodes in the
87system, and the :ref:`pw_transfer RPC service <module-pw_transfer>` was
88developed to provide reliable delivery of data.
89
90Learning from custom network stacks
91-----------------------------------
92Teams want to use Pigweed to build cool devices. Their goal isn't to build a
93network protocol stack, but they need one to use features like pw_rpc and
94pw_transfer. Given this, teams have little incentive to make the enormous time
95investment to develop a robust, reusable network stack. The practical approach
96is to assemble the minimum viable network stack from what's available.
97
98The Pigweed team has seen a few teams create custom network stacks for pw_rpc.
99While these projects were successful, their network stacks were not their
100primary focus. As a result, they had some shortcomings, including the following:
101
102- **Byte stuffing memory overhead** -- HDLC is a low-level protocol. It uses
103  `byte stuffing
104  <https://en.wikipedia.org/wiki/High-Level_Data_Link_Control#Asynchronous_framing>`_
105  to ensure frame integrity across unreliable links. Byte stuffing makes sense
106  on the wire, but not in memory. Storing byte stuffed frames requires double
107  the memory to account for worst-case byte stuffing. Some projects use HDLC
108  frames as network layer packets, so they are buffered in memory for routing,
109  which requires more memory than necessary.
110- **HDLC protocol overhead** -- HDLC's frame recovery and integrity features are
111  not needed across all links. For example, these features are unnecessary for
112  Bluetooth. However, when projects use HDLC for both the network and link
113  layers, it has to be used across all links.
114- **pw_transfer at the application layer** -- :ref:`pw_transfer
115  <module-pw_transfer>` supports reliable data transfers with :ref:`pw_rpc
116  <module-pw_rpc>`. It required significant investment to develop, but since it
117  is layered on top of pw_rpc, it has additional overhead and limited
118  reusability.
119- **Custom routing** -- Some network nodes have multiple routes between them.
120  Projects have had to write custom, non-portable logic to handle routing.
121- **pw_rpc channel IDs in routing** -- Some projects used pw_rpc channel IDs as
122  a network addresses. Channel IDs were assigned for the whole network ahead of
123  time. This has several downsides:
124
125  - Requires nodes to have knowledge of the global channel ID assignments
126    and routes between them, which can be difficult to keep in sync.
127  - Implies that all traffic is pw_rpc packets.
128  - Requires decoding pw_rpc packets at lower levels of the network stack.
129  - Complicates runtime assignment of channel IDs.
130
131- **Flow control** -- Projects' communications stacks have not supported flow
132  control. The network layer simply has to drop packets it cannot process.
133  There is no mechanism to tell the producer to slow down or wait for the
134  receiver to be ready.
135- **Accounting for the MTU** -- HDLC and pw_rpc have variable overheads, so it
136  is difficult to know how much memory to allocate for RPC payloads. If packets
137  are not sized properly with respect to the maximum transmission unit (MTU),
138  packets may be silently dropped.
139
140Problem summary
141===============
142These are the key issues of Pigweed's communications offerings based on the
143team's experiences deploying pw_rpc.
144
145**No cohesive full stack solution**
146
147Pigweed only provides a handful of communications modules. They were not
148designed to work together, and there is not enough to assemble a functioning
149network stack. Some projects have to create bespoke network protocols with
150limited reusability.
151
152**Layering violations**
153
154pw_transfer runs on top of pw_rpc instead of the transport layer, which adds
155overhead and prevents its use independent of pw_rpc. Using pw_rpc channels for
156routing ties the network to pw_rpc. Projects often use pw_hdlc for multiple
157network layers, which brings the encoding's overhead higher up the stack and
158across links that do not need it.
159
160**Inefficiency**
161
162Reliable data transfer requires pw_transfer, which runs on top of pw_rpc. This
163adds additional overhead and requires more CPU-intensive decoding operations.
164Using pw_rpc channel IDs in lower layers of the network requires expensive
165varint decodes, even when the packets are bound for other nodes.
166
167**Missing features**
168
169Each project has to develop its own version of common features, including:
170
171- **Addressing** -- There are no standard addressing schemes available to
172  Pigweed users.
173- **Routing** -- Projects must implement their own logic for routing packets,
174  which can be complex.
175- **Flow control** -- There is no way for the receiver to signal that it is ready
176  for more data or that it cannot receive any more, either at the protocol or
177  API level anywhere in the stack. Flow control is a crucial feature for
178  realistic networks with limited resources.
179- **Connections** -- Connections ensure the recipient is listening to
180  transmissions, and detect when the other end is no longer communicating.
181  pw_transfer maintains a connection, but it sits atop pw_rpc, so cannot be used
182  elsewhere.
183- **Quality of service (QoS)** -- Projects have developed basic QoS features in
184  HDLC, but there is no support in upstream Pigweed. Every project has to
185  develop its own custom implementation.
186
187-----
188Goals
189-----
190This SEED proposes a new communications system for Pigweed with the following
191goals:
192
193- **Practical end-to-end solution** -- Pigweed provides a full suite of APIs
194  and protocols that support simple and complex networking use cases.
195- **Robust, stable, and reliable** -- Pigweed communications "just work", even
196  under high load. The networking stack is thoroughly tested in both single and
197  multithreaded environments, with functional, load, fuzz, and performance
198  testing. Projects can easily test their own deployments with Pigweed tooling.
199- **Cohesive, yet modular** -- The network stack is holistically designed, but
200  modular. It is organized into layers that can be exchanged and configured
201  independently. Layering simplifies the stack, decouples protocol
202  implementations, and maximizes flexibility within a cohesive system.
203- **Efficient & performant** -- Pigweed’s network stack minimizes code size and
204  CPU usage. It provides for high throughput, low latency data transmission.
205  Memory allocation is configurable and adaptable to a project’s needs.
206- **Usable & easy to learn** -- Pigweed’s communications systems are backed by
207  thorough and up-to-date documentation. Getting started is easy using
208  Pigweed's tutorials and examples.
209
210--------
211Proposal
212--------
213Pigweed will unify its communications systems under a common sockets API. This
214entails the following:
215
216- **Sockets API** -- Pigweed will introduce a `sockets
217  API`_ to serve as its common networking interface.
218- **Lightweight protocol stack** -- Pigweed will provide a custom,
219  :ref:`lightweight network protocol stack <seed-0107-network-stack>` inspired
220  by IPv6, with UDP, TCP, and SCTP-like transport protocols.
221- **TCP/IP integration** -- Pigweed will offer sockets implementations for OS
222  sockets and an existing `embedded TCP/IP stack`_.
223- **Async** -- Pigweed will establish a new pattern for `async`_ programming and
224  use it in its networking APIs.
225- **Zero copy** -- Pigweed will develop a new `buffer management`_ system to
226  enable zero-copy networking.
227
228These features fit fit into the TCP/IP model as follows:
229
230+-------------------------------------+-------------------------------------+
231| TCP/IP Model                        | Future Pigweed Comms Stack          |
232+=====================================+=====================================+
233| Application                         | | *Various modules including*       |
234|                                     | | *pw_rpc and pw_transfer.*         |
235|                                     |                                     |
236|                                     |                                     |
237|                                     |                                     |
238+-------------------------------------+-------------------------------------+
239| .. rst-class:: pw-text-center-align | .. rst-class:: pw-text-center-align |
240|                                     |                                     |
241|    **OS Sockets**                   |    **Pigweed Sockets**              |
242+-------------------------------------+-------------------------------------+
243| Transport                           | | UDP-like unreliable protocol      |
244|                                     | | TCP-like reliable protocol        |
245|                                     | | SCTP-like reliable protocol       |
246+-------------------------------------+-------------------------------------+
247| Network / Internet                  | | IPv6-like protocol                |
248+-------------------------------------+-------------------------------------+
249| Network access / Link               | | HDLC                              |
250|                                     | | others                            |
251+-------------------------------------+-------------------------------------+
252
253Sockets API
254===========
255The new sockets API will become the primary networking abstraction in Pigweed.
256The API will support the following:
257
258- Creating sockets for bidirectional communications with other nodes in the
259  network.
260- Opening and closing connections for connection-oriented socket types.
261- Sending and receiving data, optionally :ref:`asynchronously
262  <seed-0107-async>`.
263- Reporting errors.
264
265The sockets API will support runtime polymorphism. In C++, it will be a virtual
266interface.
267
268**Rationale**
269
270A network socket represents a bidirectional communications channel with another
271node, which could be local or across the Internet. Network sockets form the API
272between an application and the network.
273
274Sockets are a proven, well-understood concept. Socket APIs such as Berkeley /
275POSIX sockets are familiar to anyone with Internet programming experience.
276
277Sockets APIs hide the details of the network protocol stack. A socket provides
278well-defined semantics for a communications channel, but applications do not
279need to know how data is sent and received. The same API can be used to exchange
280data with another process on the same machine or with a device across the world.
281
282.. admonition:: Sockets SEEDs
283
284   The Pigweed sockets API is described in SEED-0116. The sockets API is based
285   on ``pw_channel``, which is proposed in SEED-0114.
286
287Socket types
288------------
289Pigweed's sockets API will support the following sockets types.
290
291.. list-table::
292   :header-rows: 1
293
294   * - Berkeley socket type
295     - Internet protocol
296     - Payload type
297     - Connection-oriented
298     - Guaranteed, ordered delivery
299     - Description
300   * - ``SOCK_DGRAM``
301     - UDP
302     - Datagram
303     - ❌
304     - ❌
305     - Unreliable datagram
306   * - ``SOCK_STREAM``
307     - TCP
308     - Byte stream
309     - ✅
310     - ✅
311     - Reliable byte stream
312   * - ``SOCK_SEQPACKET``
313     - SCTP
314     - Datagram
315     - ✅
316     - ✅
317     - Reliable datagram
318
319Raw sockets (``SOCK_RAW``) may be supported in the future if required.
320``SOCK_CONN_DGRAM`` (unreliable connection-oriented datagram) sockets are
321uncommon and will not be supported.
322
323The socket's semantics will be expressed in the sockets API, e.g. with a
324different interface or class for each type. Instances of the connection-oriented
325socket types will be generated from a "listener" object.
326
327Pigweed's sockets API will draw inspiration from modern type safe APIs like
328Rust's `std::net sockets <https://doc.rust-lang.org/std/net/index.html>`_,
329rather than traditional APIs like POSIX sockets or Winsock. Pigweed sockets will
330map trivially to these APIs and implementations will be provided upstream.
331
332Using the sockets API
333---------------------
334The Pigweed sockets API will provide the interface between applications and the
335network. Any application can open a socket to communicate across the network.
336A future revision of ``pw_rpc`` will use the sockets API in place of its current
337``Channel`` API.
338
339The sockets API will support both synchronous and :ref:`asynchronous
340<seed-0107-async>` use. The synchronous API may be built using the async API.
341It will also support :ref:`zero-copy <seed-0107-buffers>` data transmission.
342
343Addressing
344----------
345The Pigweed sockets API will be aware of addresses. Addresses are used to refer
346to nodes in a network, including the socket's own node. With TCP/IP, the socket
347address includes an IP address and a port number.
348
349The POSIX sockets API supports different domains through address family
350constants such as ``AF_INET``, ``AF_INET6``, and ``AF_UNIX``. Addresses in these
351families are specified or accessed in various socket operations. Because the
352address format is not specified by the API, working with addresses is not type
353safe.
354
355Pigweed sockets will approach addressing differently, but details are yet to be
356determined. Possible approaches include:
357
358- Use IPv6 addresses exclusively. Systems with other addressing schemes map
359  these into IPv6 for use with Pigweed APIs.
360- Provide a polymorphic address class so sockets can work with addresses
361  generically.
362- Avoid addresses in the base sockets API. Instead, use implementation specific
363  derived classes to access addresses.
364
365Network protocol stack
366======================
367The sockets API will be backed by a network protocol stack. Pigweed will provide
368sockets implementations for following network protocol stacks:
369
370* Third party embedded TCP/IP stack, most likely `lwIP
371  <https://savannah.nongnu.org/projects/lwip/>`_.
372* Operating system TCP/IP stack via POSIX sockets or `Winsock
373  <https://learn.microsoft.com/en-us/windows/win32/winsock/windows-sockets-start-page-2>`_.
374* Custom :ref:`lightweight network protocol stack <seed-0107-network-stack>`.
375
376Embedded TCP/IP stack
377---------------------
378Pigweed will provide a sockets implementation for an embedded TCP/IP stack such
379as `lwIP <https://savannah.nongnu.org/projects/lwip/>`_.
380
381The sockets integration will be structured to avoid unnecessary dependencies on
382network stack features. For example, if a system is using IPv6 exclusively, the
383integration won't require IPv4 support, and the TCP/IP stack can be configured
384without it.
385
386**Rationale**
387
388The Internet protocol suite, or TCP/IP, is informed by decades of research and
389practical experience. It is much more than IP, TCP, and UDP; it's an alphabet
390soup of protocols that address a myriad of use cases and challenges.
391Implementing a functional TCP/IP stack is no small task. At time of writing,
392lwIP has about as many lines of C as Pigweed has C++ (excluding tests).
393
394The Pigweed team does not plan to implement a full TCP/IP stack. This is a major
395undertaking, and there are already established open source embedded TCP/IP
396stacks. Projects needing the full power of TCP/IP can use an embedded stack like
397`lwIP <https://savannah.nongnu.org/projects/lwip/>`_.
398
399Choosing between embedded TCP/IP and :ref:`Pigweed's stack <seed-0107-network-stack>`
400^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
401lwIP's `website <https://savannah.nongnu.org/projects/lwip/>`_ states that it
402requires tens of KB of RAM and about 40 KB of ROM. Using lwIP means using the
403same TCP/IP protocols that run the Internet. These protocols are feature rich,
404but have more overhead than is necessary for local communications within a small
405embedded system.
406
407Projects that can afford the resource requirements and protocol overhead of
408TCP/IP should use it. These projects can set up a local IPv4 or IPv6 network
409and use it for communications behind the Pigweed sockets API. Projects that
410cannot afford full TCP/IP can opt for Pigweed's :ref:`custom protocol stack
411<seed-0107-network-stack>`. Pigweed's custom stack will not have the depth of
412features and tooling of TCP/IP does, but will be sufficient for many systems.
413
414TCP/IP socket types
415^^^^^^^^^^^^^^^^^^^
416With an embedded TCP/IP stack, the Pigweed sockets API will be implemented as
417follows:
418
419- Unreliable datagram (``SOCK_DGRAM``) -- UDP
420- Reliable byte stream (``SOCK_STREAM``) -- TCP
421- Reliable datagram (``SOCK_SEQPACKET``) -- Lightweight framing over TCP. This
422  will be semantically similar to `SCTP
423  <https://datatracker.ietf.org/doc/html/rfc9260>`_, but integrations will not
424  use SCTP since it is not widely supported.
425
426.. _seed-0107-network-stack:
427
428Pigweed's custom network protocol stack
429---------------------------------------
430Pigweed will develop a custom, lightweight network protocol stack.
431
432This new protocol stack will be designed for small devices with relatively
433simple networks. It will scale to several interconnected cores that interface
434with a few external devices (e.g. over USB or Bluetooth). Depending on project
435requirements, it may or may not support dynamic network host configuration (e.g.
436DHCP or SLAAC).
437
438Pigweed's network protocol stack will be a strict subset of TCP/IP. This will
439include minimal, reduced overhead versions of UDP, TCP, and IPv6. Portions of
440other protocols such as ICMPv6 may be implemented as required.
441
442**Rationale**
443
444TCP/IP is too large and complex for some embedded systems. Systems for which
445TCP/IP is unnecessary can use Pigweed's lightweight embedded network protocol
446stack.
447
448Transport layer
449^^^^^^^^^^^^^^^
450Pigweed will provide transport layer protocols that implement the semantics of
451``SOCK_DGRAM``, ``SOCK_STREAM``, and ``SOCK_SEQPACKET``-like sockets.
452
453- ``SOCK_DRAM``-like sockets will be backed by a UDP-like protocol. This will
454  add source and destination ports to the IP-style packets for multiplexing on
455  top of the network layer.
456- ``SOCK_STREAM``-like sockets will be backed by a TCP-like protocol that uses
457  network layer packets to implement a reliable byte stream. It will be based on
458  TCP, but will not implement all of its features. The :ref:`module-pw_transfer`
459  module may serve as a starting point for the new protocol implementation.
460- ``SOCK_SEQPACKET``-like sockets will be implemented with a simple
461  message-oriented protocol on top of the TCP-like protocol. Applications like
462  pw_rpc will use ``SOCK_SEQPACKET`` sockets.
463
464Network layer
465^^^^^^^^^^^^^
466Pigweed will create a new network-layer protocol closely based on IPv6. Details
467are still to be determined, but the protocol is intended to be a strict subset
468of IPv6 and related protocols (e.g. ICMP, NDP) as needed. If a need arises, it
469is met by implementing the associated IP suite protocol. Packets will use
470compressed version of an IPv6 header (e.g. omit fields, use smaller addresses).
471
472This protocol will provide:
473
474- Unreliable packet delivery between source and destination.
475- Routing based on the source and destination addresses.
476- Quality of service (e.g. via the traffic class field).
477
478Packets may be routed at this layer independently of the link layer. Wire format
479details stay on the wire.
480
481Network access / link layer
482^^^^^^^^^^^^^^^^^^^^^^^^^^^
483Pigweed's network stack will interact with the link layer through a generic
484interface. This will allow Pigweed to send network packets with any protocol
485over any physical interface.
486
487Pigweed already provides minimal support for one link layer protocol, HDLC.
488Other protocols (e.g. COBS, PPP) may be implemented. Some hardware interfaces
489(e.g. Bluetooth, USB) may not require an additional link-layer protocol.
490
491Language support
492----------------
493Pigweed today is primarily C++, but it supports Rust, C, Python, TypeScript, and
494Java to varying extents.
495
496Pigweed’s communications stack will be developed in either C++ or Rust to start,
497but it will be ported to all supported languages in time. The stack may have C
498APIs to facilitate interoperability between C++ and Rust.
499
500.. admonition:: Network protocol stack SEED
501
502   Pigweed's network protocol stack will be explored in an upcoming SEED.
503
504.. _seed-0107-async:
505
506Async
507=====
508Pigweed will develop a model for asynchronous programming and use it in its
509networking APIs, including sockets. Sockets will also support synchronous
510operations, but these may be implemented in terms of the asynchronous API.
511
512The Pigweed async model has not been designed yet. The :ref:`pw_async
513<module-pw_async>` module has a task dispatcher, but the pattern for async APIs
514has not been established. Further exploration is needed, but C++20 coroutines
515may be used for Pigweed async APIs where supported.
516
517**Rationale**
518
519Synchronous APIs require the thread to block while an operation completes. The
520thread and its resources cannot be used by the system until the task completes.
521Async APIs allow a single thread to handle multiple simultaneous tasks. The
522thread advances tasks until they need to wait for an external operation to
523complete, then switches to another task to avoid blocking.
524
525Threads are expensive in embedded systems. Each thread requires significant
526memory for its stack and kernel structures for bookkeeping. They occupy this
527memory all the time, even when they are not running. Furthermore, context
528switches between threads can take significant CPU time.
529
530Asynchronous programming avoids these downsides. Many asynchronous threads run
531on a single thread. Fewer threads are needed, and the resources of one thread
532are shared by multiple tasks. Since asynchronous systems run within one thread,
533no thread context switches occur.
534
535Networking involves many asynchronous tasks. For example, waiting for data to be
536sent through a network interface, for a connection request, or for data to
537arrive on one or more interfaces are all operations that benefit from
538asynchronous APIs. Network protocols themselves are heavily asynchronous.
539
540.. admonition:: Async SEED
541
542   Pigweed's async pattern is proposed in :ref:`SEED-0112 <seed-0112>`.
543
544.. _seed-0107-buffers:
545
546Buffer management
547=================
548Pigweed's networking APIs will support zero-copy data transmission. Applications
549will be able to request a buffer from a socket. When one is available, they fill
550it with data for transmission.
551
552Pigweed will develop a general purpose module for allocating and managing
553buffers. This will be used to implement zero-copy features for Pigweed's
554networking stack.
555
556As an example, zero-copy buffer allocation could work as follows:
557
558- The user requests a buffer from a socket.
559- The network protocol layer under the socket requests a buffer from the next
560  lower layer.
561- The bottom protocol layer allocates a buffer.
562- Each layer reserves part of the buffer for its headers or footers.
563- The remaining buffer is provided to the user to populate with their payload.
564- When the user is done, the buffer is released. Each layer of the network stack
565  processes the buffer as necessary.
566- Finally, at the lowest layer, the final buffer is sent over the hardware
567  interface.
568
569Zero-copy APIs will be :ref:`asynchronous <seed-0107-async>`.
570
571**Rationale**
572
573Networking involves transmitting large amounts of data. Copying network traffic
574can result in substantial CPU usage, particularly in nodes that route traffic to
575other nodes.
576
577A buffer management system that minimizes copying saves precious CPU cycles and
578power on constrained systems.
579
580.. admonition:: Buffer management SEED
581
582   Pigweed's buffer management system is proposed in :ref:`SEED-0109
583   <seed-0109>`.
584
585Vectored I/O
586------------
587Vectored or scatter/gather I/O allows users to serialize data from multiple
588buffers into a single output stream, or vice versa. For Pigweed's networking
589APIs, this could be used to, for example, store a packet header in one buffer
590and packet contents in one or more other buffers. These isolated chunks are
591serialized in order to the network interface.
592
593Vectored I/O minimizes copying, but is complex. Additionally, simple DMA engines
594may only operate on a single DMA buffer. Thus, vectored I/O could require
595either:
596
597- a copy into the DMA engine's buffer, which costs CPU time and memory, or
598- multiple, small DMAs, which involves extra interrupts and CPU time.
599
600Vectored I/O may be supported in Pigweed's communications stack, depending on
601project requirements.
602
603----------
604Next steps
605----------
606Pigweed's communications revamp will proceed loosely as follows:
607
608* Write SEEDs to explore existing solutions, distill requirements, and propose
609  new Pigweed features for these areas:
610
611  - Sockets API (SEED-0116)
612  - Async pattern (:ref:`SEED-0112 <seed-0112>`).
613  - Buffer management (:ref:`SEED-0109 <seed-0109>`)
614  - Network protocol stack
615
616* Implement the Sockets API.
617
618  - Document, integrate, and deploy the async programming pattern for Pigweed.
619  - Develop and test Pigweed's buffer management system.
620  - Use these features in the sockets API. If necessary, the synchronous,
621    copying API could be implemented first.
622
623* Deploy the sockets API for TCP/IP.
624
625  - Implement and unit test sockets for TCP/IP with POSIX and Winsock sockets.
626  - Implement and unit test sockets for an embedded TCP/IP stack.
627
628* Develop a test suite for Pigweed network communications.
629
630  - Create integration tests for networks with multiple nodes that cover basic
631    operation, high load, and packet loss.
632  - Write performance tests against the sockets API to measure network stack
633    performance.
634
635* Develop Pigweed's lightweight network protocol stack.
636
637  - Test the lightweight network protocol stack on hardware and in a simulated
638    environment.
639  - Write fuzz tests for the protocol stack.
640  - Write performance tests for the protocol stack.
641
642* Revisit other communications systems, including pw_rpc and pw_transfer.
643