virSocketAddrPrefixToNetmask: Prevent undefined behaviour on bitshifts on signed...

Shifting bits into the sign bit is undefined behaviour in C although
both gcc and clang handle it as expected.

Since the value is used as unsigned convert it to unsigned int. For code
readability use 'if' statement instead of a ternary.

Closes: https://gitlab.com/libvirt/libvirt/-/issues/785
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>

diff --git a/src/util/virsocketaddr.c b/src/util/virsocketaddr.c
index a2e6701670bdc9c75906f7290e0a3e1236f21931..f53768878ed6247e5f1b9df53775173be645679d 100644 (file)
--- a/src/util/virsocketaddr.c
+++ b/src/util/virsocketaddr.c
@@ -1144,12 +1144,14 @@ virSocketAddrPrefixToNetmask(unsigned int prefix,
     netmask->data.stor.ss_family = AF_UNSPEC; /* assume failure */
 
     if (family == AF_INET) {
-        int ip;
+        unsigned int ip = 0;
 
         if (prefix > 32)
             return -1;
 
-        ip = prefix ? ~((1 << (32 - prefix)) - 1) : 0;
+        if (prefix > 0)
+            ip = ~((1U << (32 - prefix)) - 1);
+
         netmask->data.inet4.sin_addr.s_addr = htonl(ip);
         netmask->data.stor.ss_family = AF_INET;
         netmask->len = sizeof(struct sockaddr_in);