+++ /dev/null
-.. SPDX-License-Identifier: GPL-2.0
-
-.. _ethernet-flow-control:
-
-=====================
-Ethernet Flow Control
-=====================
-
-This document is a practical guide to Ethernet Flow Control in Linux, covering
-what it is, how it works, and how to configure it.
-
-What is Flow Control?
-=====================
-
-Flow control is a mechanism to prevent a fast sender from overwhelming a
-slow receiver with data, which would cause buffer overruns and dropped packets.
-The receiver can signal the sender to temporarily stop transmitting, giving it
-time to process its backlog.
-
-Standards references
-====================
-
-Ethernet flow control mechanisms are specified across consolidated IEEE base
-standards; some originated as amendments:
-
-- Collision-based flow control is part of CSMA/CD in **IEEE 802.3**
- (half-duplex).
-- Link-wide PAUSE is defined in **IEEE 802.3 Annex 31B**
- (originally **802.3x**).
-- Priority-based Flow Control (PFC) is defined in **IEEE 802.1Q Clause 36**
- (originally **802.1Qbb**).
-
-In the remainder of this document, the consolidated clause numbers are used.
-
-How It Works: The Mechanisms
-============================
-
-The method used for flow control depends on the link's duplex mode.
-
-.. note::
- The user-visible ``ethtool`` pause API described in this document controls
- **link-wide PAUSE** (IEEE 802.3 Annex 31B) only. It does not control the
- collision-based behavior that exists on half-duplex links.
-
-1. Half-Duplex: Collision-Based Flow Control
---------------------------------------------
-On half-duplex links, a device cannot send and receive simultaneously, so PAUSE
-frames are not used. Flow control is achieved by leveraging the CSMA/CD
-(Carrier Sense Multiple Access with Collision Detection) protocol itself.
-
-* **How it works**: To inhibit incoming data, a receiving device can force a
- collision on the line. When the sending station detects this collision, it
- terminates its transmission, sends a "jam" signal, and then executes the
- "Collision backoff and retransmission" procedure as defined in IEEE 802.3,
- Section 4.2.3.2.5. This algorithm makes the sender wait for a random
- period before attempting to retransmit. By repeatedly forcing collisions,
- the receiver can effectively throttle the sender's transmission rate.
-
-.. note::
- While this mechanism is part of the IEEE standard, there is currently no
- generic kernel API to configure or control it. Drivers should not enable
- this feature until a standardized interface is available.
-
-.. warning::
- On shared-medium networks (e.g. 10BASE2, or twisted-pair networks using a
- hub rather than a switch) forcing collisions inhibits traffic **across the
- entire shared segment**, not just a single point-to-point link. Enabling
- such behavior is generally undesirable.
-
-2. Full-Duplex: Link-wide PAUSE (IEEE 802.3 Annex 31B)
-------------------------------------------------------
-On full-duplex links, devices can send and receive at the same time. Flow
-control is achieved by sending a special **PAUSE frame**, defined by IEEE
-802.3 Annex 31B. This mechanism pauses all traffic on the link and is therefore
-called *link-wide PAUSE*.
-
-* **What it is**: A standard Ethernet frame with a globally reserved
- destination MAC address (``01-80-C2-00-00-01``). This address is in a range
- that standard IEEE 802.1D-compliant bridges do not forward. However, some
- unmanaged or misconfigured bridges have been reported to forward these
- frames, which can disrupt flow control across a network.
-
-* **How it works**: The frame contains a MAC Control opcode for PAUSE
- (``0x0001``) and a ``pause_time`` value, telling the sender how long to
- wait before sending more data frames. This time is specified in units of
- "pause quantum", where one quantum is the time it takes to transmit 512 bits.
- For example, one pause quantum is 51.2 microseconds on a 10 Mbit/s link,
- and 512 nanoseconds on a 1 Gbit/s link. A ``pause_time`` of zero indicates
- that the transmitter can resume transmission, even if a previous non-zero
- pause time has not yet elapsed.
-
-* **Who uses it**: Any full-duplex link, from 10 Mbit/s to multi-gigabit speeds.
-
-3. Full-Duplex: Priority-based Flow Control (PFC) (IEEE 802.1Q Clause 36)
--------------------------------------------------------------------------
-Priority-based Flow Control is an enhancement to the standard PAUSE mechanism
-that allows flow control to be applied independently to different classes of
-traffic, identified by their priority level.
-
-* **What it is**: PFC allows a receiver to pause traffic for one or more of the
- 8 standard priority levels without stopping traffic for other priorities.
- This is critical in data center environments for protocols that cannot
- tolerate packet loss due to congestion (e.g., Fibre Channel over Ethernet
- or RoCE).
-
-* **How it works**: PFC uses a specific PAUSE frame format. It shares the same
- globally reserved destination MAC address (``01-80-C2-00-00-01``) as legacy
- PAUSE frames but uses a unique opcode (``0x0101``). The frame payload
- contains two key fields:
-
- - **``priority_enable_vector``**: An 8-bit mask where each bit corresponds to
- one of the 8 priorities. If a bit is set to 1, it means the pause time
- for that priority is active.
- - **``time_vector``**: A list of eight 2-octet fields, one for each priority.
- Each field specifies the ``pause_time`` for its corresponding priority,
- measured in units of ``pause_quanta`` (the time to transmit 512 bits).
-
-.. note::
- When PFC is enabled for at least one priority on a port, the standard
- **link-wide PAUSE** (IEEE 802.3 Annex 31B) must be disabled for that port.
- The two mechanisms are mutually exclusive (IEEE 802.1Q Clause 36).
-
-Configuring Flow Control
-========================
-
-Link-wide PAUSE and Priority-based Flow Control are configured with different
-tools.
-
-Configuring Link-wide PAUSE with ``ethtool`` (IEEE 802.3 Annex 31B)
--------------------------------------------------------------------
-Use ``ethtool -a <interface>`` to view and ``ethtool -A <interface>`` to change
-the link-wide PAUSE settings.
-
-.. code-block:: bash
-
- # View current link-wide PAUSE settings
- ethtool -a eth0
-
- # Enable RX and TX pause, with autonegotiation
- ethtool -A eth0 autoneg on rx on tx on
-
-**Key Configuration Concepts**:
-
-* **Pause Autoneg vs Generic Autoneg**: ``ethtool -A ... autoneg {on,off}``
- controls **Pause Autoneg** (Annex 31B) only. It is independent from the
- **Generic link autonegotiation** configured with ``ethtool -s``. A device can
- have Generic autoneg **on** while Pause Autoneg is **off**, and vice versa.
-
-* **If Pause Autoneg is off** (``-A ... autoneg off``): the device will **not**
- advertise pause in the PHY. The MAC PAUSE state is **forced** according to
- ``rx``/``tx`` and does not depend on partner capabilities or resolution.
- Ensure the peer is configured complementarily for PAUSE to be effective.
-
-* **If generic autoneg is off** but **Pause Autoneg is on**, the pause policy
- is **remembered** by the kernel and applied later when Generic autoneg is
- enabled again.
-
-* **Autonegotiation Mode**: The PHY will *advertise* the ``rx`` and ``tx``
- capabilities. The final active state is determined by what both sides of the
- link agree on. See the "PHY (Physical Layer Transceiver)" section below,
- especially the *Resolution* subsection, for details of the negotiation rules.
-
-* **Forced Mode**: This mode is necessary when autonegotiation is not used or
- not possible. This includes links where one or both partners have
- autonegotiation disabled, or in setups without a PHY (e.g., direct
- MAC-to-MAC connections). The driver bypasses PHY advertisement and
- directly forces the MAC into the specified ``rx``/``tx`` state. The
- configuration on both sides of the link must be complementary. For
- example, if one side is set to ``tx on`` ``rx off``, the link partner must be
- set to ``tx off`` ``rx on`` for flow control to function correctly.
-
-Configuring PFC with ``dcb`` (IEEE 802.1Q Clause 36)
-----------------------------------------------------
-PFC is part of the Data Center Bridging (DCB) subsystem and is managed with the
-``dcb`` tool (iproute2). Some deployments use ``dcbtool`` (lldpad) instead; this
-document shows ``dcb(8)`` examples.
-
-**Viewing PFC Settings**:
-
-.. code-block:: text
-
- $ dcb pfc show dev eth0
- pfc-cap 8 macsec-bypass off delay 4096
- prio-pfc 0:off 1:off 2:off 3:off 4:off 5:off 6:on 7:on
-
-This shows the PFC state (on/off) for each priority (0-7).
-
-**Changing PFC Settings**:
-
-.. code-block:: bash
-
- # Enable PFC on priorities 6 and 7, leaving others as they are
- $ dcb pfc set dev eth0 prio-pfc 6:on 7:on
-
- # Disable PFC for all priorities except 6 and 7
- $ dcb pfc set dev eth0 prio-pfc all:off 6:on 7:on
-
-Monitoring Flow Control
-=======================
-
-The standard way to check if flow control is actively being used is to view the
-pause-related statistics.
-
-**Monitoring Link-wide PAUSE**:
-Use ``ethtool --include-statistics -a <interface>``.
-
-.. code-block:: text
-
- $ ethtool --include-statistics -a eth0
- Pause parameters for eth0:
- ...
- Statistics:
- tx_pause_frames: 0
- rx_pause_frames: 0
-
-**Monitoring PFC**:
-PFC statistics (sent and received frames per priority) are available
-through the ``dcb`` tool.
-
-.. code-block:: text
-
- $ dcb pfc show dev eth0 requests indications
- requests 0:0 1:0 2:0 3:1024 4:2048 5:0 6:0 7:0
- indications 0:0 1:0 2:0 3:512 4:4096 5:0 6:0 7:0
-
-The ``requests`` counters track transmitted PFC frames (TX), and the
-``indications`` counters track received PFC frames (RX).
-
-Link-wide PAUSE Autonegotiation Details
-=======================================
-
-The autonegotiation process for link-wide PAUSE is managed by the PHY and
-involves advertising capabilities and resolving the outcome.
-
-* Terminology (link-wide PAUSE):
-
- - **Symmetric pause**: both directions are paused when requested (TX+RX
- enabled).
- - **Asymmetric pause**: only one direction is paused (e.g., RX-only or
- TX-only).
-
- In IEEE 802.3 advertisement/resolution, symmetric/asymmetric are encoded
- using two bits (Pause/Asym) and resolved per the standard truth tables
- below.
-
-* **Advertisement**: The PHY advertises the MAC's flow control capabilities.
- This is done using two bits in the advertisement register: "Symmetric
- Pause" (Pause) and "Asymmetric Pause" (Asym). These bits should be
- interpreted as a combined value, not as independent flags. The kernel
- converts the user's ``rx`` and ``tx`` settings into this two-bit value as
- follows:
-
- .. code-block:: text
-
- tx rx | Pause Asym
- -------+-------------
- 0 0 | 0 0
- 0 1 | 1 1
- 1 0 | 0 1
- 1 1 | 1 0
-
-* **Resolution**: After negotiation, the PHY reports the link partner's
- advertised Pause and Asym bits. The final flow control mode is determined
- by the combination of the local and partner advertisements, according to
- the IEEE 802.3 standard:
-
- .. code-block:: text
-
- Local Device | Link Partner | Result
- Pause Asym | Pause Asym |
- -------------------+--------------------+---------
- 0 X | 0 X | Disabled
- 0 1 | 1 0 | Disabled
- 0 1 | 1 1 | TX only
- 1 0 | 0 X | Disabled
- 1 X | 1 X | TX + RX
- 1 1 | 0 1 | RX only
-
- It is important to note that the advertised bits reflect the *current
- configuration* of the MAC, which may not represent its full hardware
- capabilities.
-
-Kernel Policy: "Set and Trust"
-==============================
-
-The ethtool pause API is defined as a **wish policy** for
-IEEE 802.3 link-wide PAUSE only. A user request is always accepted
-as the preferred configuration, but it may not be possible to apply
-it in all link states.
-
-Key constraints:
-
-- Link-wide PAUSE is not valid on half-duplex links.
-- Link-wide PAUSE cannot be used together with Priority-based Flow Control
- (PFC, IEEE 802.1Q Clause 36).
-- If autonegotiation is active and the link is currently down, the future
- mode is not yet known.
-
-Because of these constraints, the kernel stores the requested setting
-and applies it only when the link is in a compatible state.
-
-Implications for userspace:
-
-1. Set once (the "wish"): the requested Rx/Tx PAUSE policy is
- remembered even if it cannot be applied immediately.
-2. Applied conditionally: when the link comes up, the kernel enables
- PAUSE only if the active mode allows it.
-
-Component Roles in Flow Control
-===============================
-
-The configuration of flow control involves several components, each with a
-distinct role.
-
-The MAC (Media Access Controller)
----------------------------------
-The MAC is the hardware component that actually sends and receives PAUSE
-frames. Its capabilities define the upper limit of what the driver can support.
-For link-wide PAUSE, MACs can vary in their support for symmetric (both
-directions) or asymmetric (independent TX/RX) flow control.
-
-For PFC, the MAC must be capable of generating and interpreting the
-priority-based PAUSE frames and managing separate pause states for each
-traffic class.
-
-Many MACs also implement automatic PAUSE frame transmission based on the fill
-level of their internal RX FIFO. This is typically configured with two
-thresholds:
-
-* **FLOW_ON (High Water Mark)**: When the RX FIFO usage reaches this
- threshold, the MAC automatically transmits a PAUSE frame to stop the sender.
-
-* **FLOW_OFF (Low Water Mark)**: When the RX FIFO usage drops below this
- threshold, the MAC transmits a PAUSE frame with a quantum of zero to tell
- the sender it can resume transmission.
-
-The PHY (Physical Layer Transceiver)
-------------------------------------
-The PHY's role is distinct for each flow control mechanism:
-
-* **Link-wide PAUSE**: During the autonegotiation process, the PHY is
- responsible for advertising the device's flow control capabilities. See the
- "Link-wide PAUSE Autonegotiation Details" section for more information.
-
-* **Half-Duplex Collision-Based Flow Control**: The PHY is fundamental to the
- CSMA/CD process. It performs carrier sensing (checking if the line is idle)
- and collision detection, which is the mechanism leveraged to throttle the
- sender.
-
-* **Priority-based Flow Control (PFC)**: The PHY is not directly involved in
- negotiating PFC capabilities. Its role is to establish the physical link.
- PFC negotiation happens at a higher layer via the Data Center Bridging
- Capability Exchange Protocol (DCBX).
-
-User Space Interface
-====================
-The primary user space tools are ``ethtool`` for link-wide PAUSE and ``dcb`` for
-PFC. They communicate with the kernel to configure the network device driver
-and underlying hardware.
-
-**Link-wide PAUSE Netlink Interface (``ethtool``)**
-
-See the ethtool Netlink spec (``Documentation/netlink/specs/ethtool.yaml``)
-for the authoritative definition of the Pause control and Pause statistics
-attributes. The generated UAPI is in
-``include/uapi/linux/ethtool_netlink_generated.h``.
-
-**PFC Netlink Interface (``dcb``)**
-
-The authoritative definitions for DCB/PFC netlink attributes and commands are in
-``include/uapi/linux/dcbnl.h``. See also the ``dcb(8)`` manual page and the DCB
-subsystem documentation for userspace configuration details.
-
projects / thirdparty / kernel / stable.git / commitdiff
