doh: fix undefined behaviour and open up for gcc and clang optimization

The undefined behaviour is annoying when running fuzzing with
sanitizers. The codegen is the same, but the meaning is now not up for
dispute. See https://cppinsights.io/s/516a2ff4

By incrementing the pointer first, both gcc and clang recognize this as
a bswap and optimizes it to a single instruction.  See
https://godbolt.org/z/994Zpx

Closes #4350

diff --git a/lib/doh.c b/lib/doh.c
index e84c9b0ade1267453f9cf389c6ee5b06e380f9a4..6f06d0a35ccb7146d07c1ef81ced3b7d891b490e 100644 (file)
--- a/lib/doh.c
+++ b/lib/doh.c
@@ -432,8 +432,14 @@ static unsigned short get16bit(unsigned char *doh, int index)
 
 static unsigned int get32bit(unsigned char *doh, int index)
 {
-  return (doh[index] << 24) | (doh[index + 1] << 16) |
-    (doh[index + 2] << 8) | doh[index + 3];
+   /* make clang and gcc optimize this to bswap by incrementing
+      the pointer first. */
+   doh += index;
+
+   /* avoid undefined behaviour by casting to unsigned before shifting
+      24 bits, possibly into the sign bit. codegen is same, but
+      ub sanitizer won't be upset */
+  return ( (unsigned)doh[0] << 24) | (doh[1] << 16) |(doh[2] << 8) | doh[3];
 }
 
 static DOHcode store_a(unsigned char *doh, int index, struct dohentry *d)