From: Oliver Kurth <okurth@vmware.com>
Date: Fri, 12 Jun 2020 03:43:21 +0000 (-0700)
Subject: Code cleanup to address a Coverity issue.
X-Git-Tag: stable-11.2.0~146
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=9afd238cddc0cb0511d8daa903b4f5c9a52b8dc3;p=thirdparty%2Fopen-vm-tools.git

Code cleanup to address a Coverity issue.

Coverity reports a "dereference after NULL check" in
BkdoorChannelStart.  However, at the point of dereference
it's known that chan->inStarted is TRUE, which means chan->in
is guaranteed to be non-NULL, so it's not a bug.

Still, given that an input channel, if present, must have been
started before calling BkdoorChannelStart, it's possible to do
some code cleanup that will also get Coverity to stop reporting
the issue.  Change what's currently a test into an ASSERT, test
chan->in rather than chan->inStarted, and add comments to make
it clearer what's going on.
---

diff --git a/open-vm-tools/lib/rpcChannel/bdoorChannel.c b/open-vm-tools/lib/rpcChannel/bdoorChannel.c
index b153d52b2..6144bf919 100644
--- a/open-vm-tools/lib/rpcChannel/bdoorChannel.c
+++ b/open-vm-tools/lib/rpcChannel/bdoorChannel.c
@@ -54,19 +54,25 @@ typedef struct BackdoorChannel {
 static gboolean
 BkdoorChannelStart(RpcChannel *chan)
 {
-   gboolean ret = TRUE;
+   gboolean ret;
    BackdoorChannel *bdoor = chan->_private;
 
 #if defined(NEED_RPCIN)
-   ret = chan->in == NULL || chan->inStarted;
-   if (ret) {
-      ret = RpcOut_start(bdoor->out);
-      if (!ret) {
-         if (chan->inStarted) {
-            RpcIn_stop(chan->in);
-            chan->inStarted = FALSE;
-         }
-      }
+   /*
+    * If the RpcIn channel exists, it should have been started before
+    * calling this routine.
+    */
+   ASSERT(chan->in == NULL || chan->inStarted);
+
+   ret = RpcOut_start(bdoor->out);
+   if (!ret && chan->in != NULL) {
+      /*
+       * If the output channel failed to start, stop the input channel
+       * if there is one.
+       */
+
+      RpcIn_stop(chan->in);
+      chan->inStarted = FALSE;
    }
 #else
    ret = RpcOut_start(bdoor->out);