ksmbd: fix multichannel binding and enforce channel limit

ksmbd: fix multichannel binding and enforce channel limit

A signed multichannel SESSION_SETUP binding request can require multiple
authentication rounds. ksmbd excludes SESSION_SETUP from the signed
request check and tries to sign every binding response with the channel
signing key. The channel does not exist for
STATUS_MORE_PROCESSING_REQUIRED, so that response is sent unsigned.
Clients reject it with STATUS_ACCESS_DENIED.

The final channel signing key also needs the key exported by the binding
authentication context. Keep that key in the channel instead of
overwriting the established session key, and use the session signing key
for intermediate and failed binding responses. Retain the binding session
reference until an error response has been signed and sent.

Limit a session to 32 channels while holding the channel lock. Return
STATUS_INSUFFICIENT_RESOURCES for an additional binding, matching the
server limit expected by clients.

This fixes smb2.multichannel.generic.num_channels, which previously
failed the first binding with STATUS_ACCESS_DENIED and returned the same
status instead of STATUS_INSUFFICIENT_RESOURCES for channel 33.

Fixes: f5a544e3bab7 ("ksmbd: add support for SMB3 multichannel")
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Steve French <stfrench@microsoft.com>