net/mlx5e: Fix publication race for priv->channel_stats[]
mlx5e_channel_stats_alloc() publishes a new entry to
priv->channel_stats[] and then increments priv->stats_nch as a
publication token, but neither store carries any memory barrier:
priv->channel_stats[ix] = kvzalloc_node(...);
if (!priv->channel_stats[ix])
return -ENOMEM;
priv->stats_nch++;
Concurrent readers compute the loop bound from priv->stats_nch and
then dereference priv->channel_stats[i] using plain accesses, e.g.
for (i = 0; i < priv->stats_nch; i++) {
struct mlx5e_channel_stats *cs = priv->channel_stats[i];
... cs->rq.packets ...
}
On weakly-ordered architectures (ARM, PowerPC, RISC-V) the writes to
channel_stats[ix] and stats_nch may become visible to other CPUs out
of program order. A reader can observe stats_nch == N while still
seeing channel_stats[N-1] == NULL, leading to a NULL pointer
dereference in the channel_stats loop.
This has been observed in production on BlueField-3 DPUs (arm64),
where ovs-vswitchd queries netdev statistics over netlink during NIC
bringup, racing mlx5e_open_channel() -> mlx5e_channel_stats_alloc()
on another CPU:
Unable to handle kernel NULL pointer dereference at virtual address 0x840
Hardware name: BlueField-3 DPU
pc : mlx5e_fold_sw_stats64+0x30/0x180 [mlx5_core]
Call trace:
mlx5e_fold_sw_stats64+0x30/0x180 [mlx5_core]
dev_get_stats+0x50/0xc0
ovs_vport_get_stats+0x38/0xac [openvswitch]
ovs_vport_cmd_fill_info+0x194/0x290 [openvswitch]
ovs_vport_cmd_get+0xbc/0x10c [openvswitch]
genl_family_rcv_msg_doit+0xd0/0x160
genl_rcv_msg+0xec/0x1f0
netlink_rcv_skb+0x64/0x130
genl_rcv+0x40/0x60
netlink_unicast+0x2fc/0x370
netlink_sendmsg+0x1dc/0x454
...
__arm64_sys_sendmsg+0x2c/0x40
Add mlx5e_stats_nch_write() and mlx5e_stats_nch_read() helpers in en.h
that wrap the smp_store_release()/smp_load_acquire() pair on stats_nch.
The release/acquire pair establishes the contract:
stats_nch == N => channel_stats[0..N-1] are visible and non-NULL.
Publish the stats_nch increment via mlx5e_stats_nch_write() in the
writer (mlx5e_channel_stats_alloc()), and read stats_nch via
mlx5e_stats_nch_read() in all readers: mlx5e RX/TX queue stats,
mlx5e_get_base_stats(), ethtool channels stats, IPoIB stats, the
sw_stats fold and the HV VHCA stats agent.
Fixes: fa691d0c9c08 ("net/mlx5e: Allocate per-channel stats dynamically at first usage")
Signed-off-by: Feng Liu <feliu@nvidia.com>
Reviewed-by: Eran Ben Elisha <eranbe@nvidia.com>
Reviewed-by: Cosmin Ratiu <cratiu@nvidia.com>
Reviewed-by: Nimrod Oren <noren@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
Link: https://patch.msgid.link/20260630115151.729219-4-tariqt@nvidia.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>