1 .. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
13 For details regarding the user-facing interface refer to the TLS
18  * Software crypto mode (``TLS_SW``) - CPU handles the cryptography.
24  * Packet-based NIC offload mode (``TLS_HW``) - the NIC handles crypto
28    (``ethtool`` flags ``tls-hw-tx-offload`` and ``tls-hw-rx-offload``).
29  * Full TCP NIC offload mode (``TLS_HW_RECORD``) - mode of operation where
33    abilities or QoS and packet scheduling (``ethtool`` flag ``tls-hw-record``).
35 The operation mode is selected automatically based on device configuration,
36 offload opt-in or opt-out on per-connection basis is not currently supported.
38 TX  section in Kernel TLS operation
39 --
47 Instead packets reach a device driver, the driver will mark the packets
49 and send them to the device for encryption and transmission.
52 --
54 On the receive side, if the device handled decryption and authentication
60 If device decrypted all the segments of the record the decryption is skipped,
63 .. kernel-figure::  tls-offload-layers.svg
70 Device configuration
73 During driver initialization device sets the ``NETIF_F_HW_TLS_RX`` and
80 (note that it is done twice, once for RX and once for TX direction,
82 network device is offload-capable and attempts the offload. In case offload
89 .. code-block:: c
98 to retrieve the connection 5-tuple and socket family (IPv4 vs IPv6).
107 TX  section in Device configuration
108 --
110 After TX state is installed, the stack guarantees that the first segment
114 TX offload being fully initialized does not imply that all segments passing
121 --
130 At the minimum the device maintains the following state for each connection, in
140 Assuming segments are received in order, the device should be able to perform
142 to be possible, the device has to keep a small amount of segment-to-segment
151 in order the device should be able to handle them separately and make
154 TX  section in Normal operation
155 --
160 Both the device and the driver maintain expected TCP sequence numbers
162 once the packet reaches the device.
164 a connection identifier (note that a 5-tuple lookup is insufficient to identify
166 and hands them to the device. The device identifies the packet as requiring
168 The device performs encryption and authentication of the record data.
172 --
175 and packet transformation functions) the device validates the Layer 4
176 checksum and performs a 5-tuple lookup to find any TLS connection the packet
177 may belong to (technically a 4-tuple
178 lookup is sufficient - IP addresses and TCP port numbers, as the protocol
179 is always TCP). If the packet is matched to a connection, the device confirms
182 The device leaves the record framing unmodified, the stack takes care of record
183 decapsulation. Device indicates successful handling of TLS offload in the
184 per-packet context (descriptor) passed to the host.
189 and non-decrypted segments do not get coalesced (e.g. by GRO or socket layer)
195 In presence of packet drops or network packet reordering, the device may lose
200 added to the device table and are in TLS_HW mode. For example,
205 TX  section in Resync handling
206 --
209 in similar ways to the receive side-retransmissions - local drops
217 the device with enough information to perform cryptographic operations.
219 segment has to be passed to the device as part of the packet context,
220 together with its TCP sequence number and TLS record number. The device
228 with the previous stream state - assuming that the out of order segment
239 and doesn't change device state. If the segment is in the future, it
240 may imply a local drop, the driver asks the stack to sync the device
245 .. code-block:: c
253 .. code-block:: c
263 --
266 In particular the device should not lose synchronization
269 .. kernel-figure::  tls-offload-reorder-good.svg
270    :alt:	reorder of non-header segment
273    Reorder of non-header segment
279 Segment 2 was dropped so 3 arrives out of order. The device knows
283 The device can, however, collect the authentication algorithm's state
286 of expected window of the device so it's passed as is without special
288 spanning segments 1, 2 and 3. The device did not get out of sync,
294 .. kernel-figure::  tls-offload-reorder-bad.svg
301 Device can only detect that segment 4 also contains a TLS header
303 the device will lose synchronization with the stream.
308 When the device gets out of sync and the stream reaches TCP sequence
310 the device starts scanning for a known header pattern. For example
313 the device continues attempting parsing headers at expected locations
318 When the header is matched the device sends a confirmation request
321 The kernel confirms the guessed location was correct and tells the device
322 the record sequence number. Meanwhile, the device had been parsing
323 and counting all records since the just-confirmed one, it adds the number
325 At this point the device is in sync and can resume decryption at next
328 In a pathological case the device may latch onto a sequence of matching
331 restart scan. Given how unlikely falsely-matching stream is, however,
338 Stack-driven resynchronization
347 The stack waits for the socket to drain and informs the device about
357 TX  section in Error handling
358 --
361 device than the selected TLS offload device. The stack will handle
365 fully acknowledged, so if skbs reach the wrong device they can be handled
368 Any device TLS offload handling error on the transmission side must result
370 due to a bug in the stack or the device, reached the device and can't
374 --
376 If the device encounters any problems with TLS offload on the receive
386 driver if device provides precise error is sufficient).
388 The Linux networking stack does not provide a way of reporting per-packet
409 --------------------
411 The number of connections device can support can be exposed via
415 -------------------------------
419 Overload of the cryptographic subsystem of the device should not have
420 significant performance impact on non-offloaded streams.
425 Following minimum set of TLS-related statistics should be reported
428  * ``rx_tls_decrypted_packets`` - number of successfully decrypted RX packets
430  * ``rx_tls_decrypted_bytes`` - number of TLS payload bytes in RX packets
432  * ``rx_tls_ctx`` - number of TLS RX HW offload contexts added to device for
434  * ``rx_tls_del`` - number of TLS RX HW offload contexts deleted from device
436  * ``rx_tls_resync_req_pkt`` - number of received TLS packets with a resync
438  * ``rx_tls_resync_req_start`` - number of times the TLS async resync request
440  * ``rx_tls_resync_req_end`` - number of times the TLS async resync request
441     properly ended with providing the HW tracked tcp-seq.
442  * ``rx_tls_resync_req_skip`` - number of times the TLS async resync request
444  * ``rx_tls_resync_res_ok`` - number of times the TLS resync response call to
446  * ``rx_tls_resync_res_skip`` - number of times the TLS resync response call to
448  * ``rx_tls_err`` - number of RX packets which were part of a TLS stream
450  * ``tx_tls_encrypted_packets`` - number of TX packets passed to the device
452  * ``tx_tls_encrypted_bytes`` - number of TLS payload bytes in TX packets
453    passed to the device for encryption.
454  * ``tx_tls_ctx`` - number of TLS TX HW offload contexts added to device for
456  * ``tx_tls_ooo`` - number of TX packets which were part of a TLS stream
458  * ``tx_tls_skip_no_sync_data`` - number of TX packets which were part of
459    a TLS stream and arrived out-of-order, but skipped the HW offload routine
462  * ``tx_tls_drop_no_sync_data`` - number of TX packets which were part of
465  * ``tx_tls_drop_bypass_req`` - number of TX packets which were part of a TLS
474 5-tuple matching limitations
475 ----------------------------
477 The device can only recognize received packets based on the 5-tuple
482 any intermediate software device, therefore a 5-tuple match may
483 consistently miss at the device level. In such cases the device
484 should still be able to perform TX offload (encryption) and should
488 ------------
495 ---------------
497 A device is permitted to perform packet reordering for consecutive
502 -----------------------------------------------------
505 transparently. Enabling device TLS offload should not cause any difference
509 ----------------------------
511 For the purpose of simplifying TLS offload, the device should not modify any
514 The device should not depend on any packet headers beyond what is strictly
518 -------------
525 TLS device features
526 -------------------
528 Drivers should ignore the changes to the TLS device feature flags.
530 TLS device feature flags only control adding of new TLS connection
534 offload. Hence, TLS TX device feature flag requires TX csum offload being set.
535 Disabling the latter implies clearing the former. Disabling TX checksum offload
538 Similarly, device-offloaded TLS decryption implies doing RXCSUM. If the user
539 does not want to enable RX csum offload, TLS RX device feature is disabled