Merge branch 'net-mlx5e-save-per-channel-async-icosq-in-default'

Merge branch 'net-mlx5e-save-per-channel-async-icosq-in-default'

Tariq Toukan says:

====================
net/mlx5e: Save per-channel async ICOSQ in default

This series by William reduces the default number of SQs in a channel
from 3 down to 2, by not creating the async ICOSQ (asynchronous
internal-communication-operations send-queue).

This significantly improves the latency of channel configuration
operations, like interface up (create channels), interface down (destroy
channels), and channels reconfiguration (create new set, destroy old
one).

This reduces the per-channel memory usage, saves hardware resources, in
addition to the improved latency.

This significantly speeds up the setup/config stage on systems with high
number of channels or many netdevs, in particular systems with hundreds
or K's of SFs.

The two remaining default SQs per channel after this series:
1 TXQ SQ (for traffic), and 1 ICOSQ (for internal communication
operations with the device).

Perf numbers:
NIC: Connect-X7.
Test: Latency of interface up + down operations.

Measured 20% speedup.
Saving ~0.36 sec for 248 channels (~1.45 msec per channel).
====================

Link: https://patch.msgid.link/1768376800-1607672-1-git-send-email-tariqt@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>