xsk: Add launch time hardware offload support to XDP Tx metadata

Extend the XDP Tx metadata framework so that user can requests launch time
hardware offload, where the Ethernet device will schedule the packet for
transmission at a pre-determined time called launch time. The value of
launch time is communicated from user space to Ethernet driver via
launch_time field of struct xsk_tx_metadata.

Suggested-by: Stanislav Fomichev <sdf@fomichev.me>
Signed-off-by: Song Yoong Siang <yoong.siang.song@intel.com>
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Link: https://patch.msgid.link/20250216093430.957880-2-yoong.siang.song@intel.com

diff --git a/Documentation/netlink/specs/netdev.yaml b/Documentation/netlink/specs/netdev.yaml
index 288923e965aed39ff30032ef540462046c9f2e4e..5f6f84630b16e4d737b593e8cd1f39cd70be261c 100644 (file)
--- a/Documentation/netlink/specs/netdev.yaml
+++ b/Documentation/netlink/specs/netdev.yaml
@@ -70,6 +70,10 @@ definitions:
         name: tx-checksum
         doc:
           L3 checksum HW offload is supported by the driver.
+      -
+        name: tx-launch-time-fifo
+        doc:
+          Launch time HW offload is supported by the driver.
   -
     name: queue-type
     type: enum

diff --git a/Documentation/networking/xsk-tx-metadata.rst b/Documentation/networking/xsk-tx-metadata.rst
index e76b0cfc32f7d062b1c0343cd023336c0369c7f7..df53a10ccac34b38415f47654c4c8d457e0977a1 100644 (file)
--- a/Documentation/networking/xsk-tx-metadata.rst
+++ b/Documentation/networking/xsk-tx-metadata.rst
@@ -50,6 +50,10 @@ The flags field enables the particular offload:
   checksum. ``csum_start`` specifies byte offset of where the checksumming
   should start and ``csum_offset`` specifies byte offset where the
   device should store the computed checksum.
+- ``XDP_TXMD_FLAGS_LAUNCH_TIME``: requests the device to schedule the
+  packet for transmission at a pre-determined time called launch time. The
+  value of launch time is indicated by ``launch_time`` field of
+  ``union xsk_tx_metadata``.
 
 Besides the flags above, in order to trigger the offloads, the first
 packet's ``struct xdp_desc`` descriptor should set ``XDP_TX_METADATA``
@@ -65,6 +69,63 @@ In this case, when running in ``XDK_COPY`` mode, the TX checksum
 is calculated on the CPU. Do not enable this option in production because
 it will negatively affect performance.
 
+Launch Time
+===========
+
+The value of the requested launch time should be based on the device's PTP
+Hardware Clock (PHC) to ensure accuracy. AF_XDP takes a different data path
+compared to the ETF queuing discipline, which organizes packets and delays
+their transmission. Instead, AF_XDP immediately hands off the packets to
+the device driver without rearranging their order or holding them prior to
+transmission. Since the driver maintains FIFO behavior and does not perform
+packet reordering, a packet with a launch time request will block other
+packets in the same Tx Queue until it is sent. Therefore, it is recommended
+to allocate separate queue for scheduling traffic that is intended for
+future transmission.
+
+In scenarios where the launch time offload feature is disabled, the device
+driver is expected to disregard the launch time request. For correct
+interpretation and meaningful operation, the launch time should never be
+set to a value larger than the farthest programmable time in the future
+(the horizon). Different devices have different hardware limitations on the
+launch time offload feature.
+
+stmmac driver
+-------------
+
+For stmmac, TSO and launch time (TBS) features are mutually exclusive for
+each individual Tx Queue. By default, the driver configures Tx Queue 0 to
+support TSO and the rest of the Tx Queues to support TBS. The launch time
+hardware offload feature can be enabled or disabled by using the tc-etf
+command to call the driver's ndo_setup_tc() callback.
+
+The value of the launch time that is programmed in the Enhanced Normal
+Transmit Descriptors is a 32-bit value, where the most significant 8 bits
+represent the time in seconds and the remaining 24 bits represent the time
+in 256 ns increments. The programmed launch time is compared against the
+PTP time (bits[39:8]) and rolls over after 256 seconds. Therefore, the
+horizon of the launch time for dwmac4 and dwxlgmac2 is 128 seconds in the
+future.
+
+igc driver
+----------
+
+For igc, all four Tx Queues support the launch time feature. The launch
+time hardware offload feature can be enabled or disabled by using the
+tc-etf command to call the driver's ndo_setup_tc() callback. When entering
+TSN mode, the igc driver will reset the device and create a default Qbv
+schedule with a 1-second cycle time, with all Tx Queues open at all times.
+
+The value of the launch time that is programmed in the Advanced Transmit
+Context Descriptor is a relative offset to the starting time of the Qbv
+transmission window of the queue. The Frst flag of the descriptor can be
+set to schedule the packet for the next Qbv cycle. Therefore, the horizon
+of the launch time for i225 and i226 is the ending time of the next cycle
+of the Qbv transmission window of the queue. For example, when the Qbv
+cycle time is set to 1 second, the horizon of the launch time ranges
+from 1 second to 2 seconds, depending on where the Qbv cycle is currently
+running.
+
 Querying Device Capabilities
 ============================
 
@@ -74,6 +135,7 @@ Refer to ``xsk-flags`` features bitmask in
 
 - ``tx-timestamp``: device supports ``XDP_TXMD_FLAGS_TIMESTAMP``
 - ``tx-checksum``: device supports ``XDP_TXMD_FLAGS_CHECKSUM``
+- ``tx-launch-time-fifo``: device supports ``XDP_TXMD_FLAGS_LAUNCH_TIME``
 
 See ``tools/net/ynl/samples/netdev.c`` on how to query this information.
 

diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h
index bfe625b55d55d74e3484d46c151c1c984c7107ca..a58ae7589d12127c2b66e86c3679c5eb5053490a 100644 (file)
--- a/include/net/xdp_sock.h
+++ b/include/net/xdp_sock.h
@@ -110,11 +110,16 @@ struct xdp_sock {
  *     indicates position where checksumming should start.
  *     csum_offset indicates position where checksum should be stored.
  *
+ * void (*tmo_request_launch_time)(u64 launch_time, void *priv)
+ *     Called when AF_XDP frame requested launch time HW offload support.
+ *     launch_time indicates the PTP time at which the device can schedule the
+ *     packet for transmission.
  */
 struct xsk_tx_metadata_ops {
        void    (*tmo_request_timestamp)(void *priv);
        u64     (*tmo_fill_timestamp)(void *priv);
        void    (*tmo_request_checksum)(u16 csum_start, u16 csum_offset, void *priv);
+       void    (*tmo_request_launch_time)(u64 launch_time, void *priv);
 };
 
 #ifdef CONFIG_XDP_SOCKETS
@@ -162,6 +167,11 @@ static inline void xsk_tx_metadata_request(const struct xsk_tx_metadata *meta,
        if (!meta)
                return;
 
+       if (ops->tmo_request_launch_time)
+               if (meta->flags & XDP_TXMD_FLAGS_LAUNCH_TIME)
+                       ops->tmo_request_launch_time(meta->request.launch_time,
+                                                    priv);
+
        if (ops->tmo_request_timestamp)
                if (meta->flags & XDP_TXMD_FLAGS_TIMESTAMP)
                        ops->tmo_request_timestamp(priv);

diff --git a/include/net/xdp_sock_drv.h b/include/net/xdp_sock_drv.h
index 15086dcf51d850f548ea65c7f6f764407c956772..513c8e9704f657f05101f83127d66960407ac291 100644 (file)
--- a/include/net/xdp_sock_drv.h
+++ b/include/net/xdp_sock_drv.h
@@ -216,6 +216,7 @@ xsk_buff_raw_get_ctx(const struct xsk_buff_pool *pool, u64 addr)
 #define XDP_TXMD_FLAGS_VALID ( \
                XDP_TXMD_FLAGS_TIMESTAMP | \
                XDP_TXMD_FLAGS_CHECKSUM | \
+               XDP_TXMD_FLAGS_LAUNCH_TIME | \
        0)
 
 static inline bool

diff --git a/include/uapi/linux/if_xdp.h b/include/uapi/linux/if_xdp.h
index 42ec5ddaab8dc89eaccab29d689be208b24c89ed..42869770776ec068341bb666bda090ab3c99837e 100644 (file)
--- a/include/uapi/linux/if_xdp.h
+++ b/include/uapi/linux/if_xdp.h
@@ -127,6 +127,12 @@ struct xdp_options {
  */
 #define XDP_TXMD_FLAGS_CHECKSUM                        (1 << 1)
 
+/* Request launch time hardware offload. The device will schedule the packet for
+ * transmission at a pre-determined time called launch time. The value of
+ * launch time is communicated via launch_time field of struct xsk_tx_metadata.
+ */
+#define XDP_TXMD_FLAGS_LAUNCH_TIME             (1 << 2)
+
 /* AF_XDP offloads request. 'request' union member is consumed by the driver
  * when the packet is being transmitted. 'completion' union member is
  * filled by the driver when the transmit completion arrives.
@@ -142,6 +148,10 @@ struct xsk_tx_metadata {
                        __u16 csum_start;
                        /* Offset from csum_start where checksum should be stored. */
                        __u16 csum_offset;
+
+                       /* XDP_TXMD_FLAGS_LAUNCH_TIME */
+                       /* Launch time in nanosecond against the PTP HW Clock */
+                       __u64 launch_time;
                } request;
 
                struct {

diff --git a/include/uapi/linux/netdev.h b/include/uapi/linux/netdev.h
index 6c6ee183802d335c03bcf3a0daad71909026c721..b97ff8bbb0c672169bd1bede162b015d6326afd7 100644 (file)
--- a/include/uapi/linux/netdev.h
+++ b/include/uapi/linux/netdev.h
@@ -59,10 +59,13 @@ enum netdev_xdp_rx_metadata {
  *   by the driver.
  * @NETDEV_XSK_FLAGS_TX_CHECKSUM: L3 checksum HW offload is supported by the
  *   driver.
+ * @NETDEV_XSK_FLAGS_TX_LAUNCH_TIME_FIFO: Launch time HW offload is supported
+ *   by the driver.
  */
 enum netdev_xsk_flags {
        NETDEV_XSK_FLAGS_TX_TIMESTAMP = 1,
        NETDEV_XSK_FLAGS_TX_CHECKSUM = 2,
+       NETDEV_XSK_FLAGS_TX_LAUNCH_TIME_FIFO = 4,
 };
 
 enum netdev_queue_type {

diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c
index c18bb53d13fd81fe4e414ac79d3cd16f734d7176..55bb58d623ace095891f4cdb7b0ae1c5e655b3e5 100644 (file)
--- a/net/core/netdev-genl.c
+++ b/net/core/netdev-genl.c
@@ -53,6 +53,8 @@ XDP_METADATA_KFUNC_xxx
                        xsk_features |= NETDEV_XSK_FLAGS_TX_TIMESTAMP;
                if (netdev->xsk_tx_metadata_ops->tmo_request_checksum)
                        xsk_features |= NETDEV_XSK_FLAGS_TX_CHECKSUM;
+               if (netdev->xsk_tx_metadata_ops->tmo_request_launch_time)
+                       xsk_features |= NETDEV_XSK_FLAGS_TX_LAUNCH_TIME_FIFO;
        }
 
        if (nla_put_u32(rsp, NETDEV_A_DEV_IFINDEX, netdev->ifindex) ||

diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c
index 89d2bef964698434be6c7f994e278602821f651c..41093ea63700cd4eb74f99be83aae87f2e8fca66 100644 (file)
--- a/net/xdp/xsk.c
+++ b/net/xdp/xsk.c
@@ -742,6 +742,9 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs,
                                                goto free_err;
                                }
                        }
+
+                       if (meta->flags & XDP_TXMD_FLAGS_LAUNCH_TIME)
+                               skb->skb_mstamp_ns = meta->request.launch_time;
                }
        }
 

diff --git a/tools/include/uapi/linux/if_xdp.h b/tools/include/uapi/linux/if_xdp.h
index 42ec5ddaab8dc89eaccab29d689be208b24c89ed..42869770776ec068341bb666bda090ab3c99837e 100644 (file)
--- a/tools/include/uapi/linux/if_xdp.h
+++ b/tools/include/uapi/linux/if_xdp.h
@@ -127,6 +127,12 @@ struct xdp_options {
  */
 #define XDP_TXMD_FLAGS_CHECKSUM                        (1 << 1)
 
+/* Request launch time hardware offload. The device will schedule the packet for
+ * transmission at a pre-determined time called launch time. The value of
+ * launch time is communicated via launch_time field of struct xsk_tx_metadata.
+ */
+#define XDP_TXMD_FLAGS_LAUNCH_TIME             (1 << 2)
+
 /* AF_XDP offloads request. 'request' union member is consumed by the driver
  * when the packet is being transmitted. 'completion' union member is
  * filled by the driver when the transmit completion arrives.
@@ -142,6 +148,10 @@ struct xsk_tx_metadata {
                        __u16 csum_start;
                        /* Offset from csum_start where checksum should be stored. */
                        __u16 csum_offset;
+
+                       /* XDP_TXMD_FLAGS_LAUNCH_TIME */
+                       /* Launch time in nanosecond against the PTP HW Clock */
+                       __u64 launch_time;
                } request;
 
                struct {

diff --git a/tools/include/uapi/linux/netdev.h b/tools/include/uapi/linux/netdev.h
index 6c6ee183802d335c03bcf3a0daad71909026c721..b97ff8bbb0c672169bd1bede162b015d6326afd7 100644 (file)
--- a/tools/include/uapi/linux/netdev.h
+++ b/tools/include/uapi/linux/netdev.h
@@ -59,10 +59,13 @@ enum netdev_xdp_rx_metadata {
  *   by the driver.
  * @NETDEV_XSK_FLAGS_TX_CHECKSUM: L3 checksum HW offload is supported by the
  *   driver.
+ * @NETDEV_XSK_FLAGS_TX_LAUNCH_TIME_FIFO: Launch time HW offload is supported
+ *   by the driver.
  */
 enum netdev_xsk_flags {
        NETDEV_XSK_FLAGS_TX_TIMESTAMP = 1,
        NETDEV_XSK_FLAGS_TX_CHECKSUM = 2,
+       NETDEV_XSK_FLAGS_TX_LAUNCH_TIME_FIFO = 4,
 };
 
 enum netdev_queue_type {