smb: server: let smb_direct_cm_handler() call ib_drain_qp() after smb_direct_disconne...

All handlers triggered by ib_drain_qp() should already see the
broken connection.

smb_direct_cm_handler() is called under a mutex of the rdma_cm,
we should make sure ib_drain_qp() and all rdma layer logic completes
and unlocks the mutex.

It means free_transport() will also already see the connection
as SMBDIRECT_SOCKET_DISCONNECTED, so we need to call
crdma_[un]lock_handler(sc->rdma.cm_id) around
ib_drain_qp(), rdma_destroy_qp(), ib_free_cq() and ib_dealloc_pd().

Otherwise we free resources while the ib_drain_qp() within
smb_direct_cm_handler() is still running.

We have to unlock before rdma_destroy_id() as it locks again.

Fixes: 141fa9824c0f ("ksmbd: call ib_drain_qp when disconnected")
Fixes: 4c564f03e23b ("smb: server: make use of common smbdirect_socket")
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Acked-by: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Steve French <stfrench@microsoft.com>

diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index e70fc447e815a32b0abbd43653602614ce7d13cb..7d86553fcc7cdc0c9b256d828e243091eba47425 100644 (file)
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -466,6 +466,9 @@ static void free_transport(struct smb_direct_transport *t)
        disable_delayed_work_sync(&sc->idle.timer_work);
        disable_work_sync(&sc->idle.immediate_work);
 
+       if (sc->rdma.cm_id)
+               rdma_lock_handler(sc->rdma.cm_id);
+
        if (sc->ib.qp) {
                ib_drain_qp(sc->ib.qp);
                sc->ib.qp = NULL;
@@ -494,8 +497,10 @@ static void free_transport(struct smb_direct_transport *t)
                ib_free_cq(sc->ib.recv_cq);
        if (sc->ib.pd)
                ib_dealloc_pd(sc->ib.pd);
-       if (sc->rdma.cm_id)
+       if (sc->rdma.cm_id) {
+               rdma_unlock_handler(sc->rdma.cm_id);
                rdma_destroy_id(sc->rdma.cm_id);
+       }
 
        smb_direct_destroy_pools(sc);
        ksmbd_conn_free(KSMBD_TRANS(t)->conn);
@@ -1724,10 +1729,10 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
        }
        case RDMA_CM_EVENT_DEVICE_REMOVAL:
        case RDMA_CM_EVENT_DISCONNECTED: {
-               ib_drain_qp(sc->ib.qp);
-
                sc->status = SMBDIRECT_SOCKET_DISCONNECTED;
                smb_direct_disconnect_rdma_work(&sc->disconnect_work);
+               if (sc->ib.qp)
+                       ib_drain_qp(sc->ib.qp);
                break;
        }
        case RDMA_CM_EVENT_CONNECT_ERROR: {