Fix setbuf use for VMS C

The VMS C RTL has setbuf() working for short pointers only, probably
the FILE pointer will always be in P0 (the lower 4GB).  Fortunately,
this only generates a warning about possible data loss (doesn't apply
in this case) that we can simply turn off.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5789)

diff --git a/crypto/rand/randfile.c b/crypto/rand/randfile.c
index fa6f49e1b5d57787ff30b23920ca0bd035f9b3e8..99a3f1424c0162ad11102a4dcece50c414d3f1f6 100644 (file)
--- a/crypto/rand/randfile.c
+++ b/crypto/rand/randfile.c
@@ -99,6 +99,17 @@ int RAND_load_file(const char *file, long bytes)
 
     if (!S_ISREG(sb.st_mode) && bytes < 0)
         bytes = 256;
+#endif
+    /*
+     * On VMS, setbuf() will only take 32-bit pointers, and a compilation
+     * with /POINTER_SIZE=64 will give off a MAYLOSEDATA2 warning here.
+     * However, we trust that the C RTL will never give us a FILE pointer
+     * above the first 4 GB of memory, so we simply turn off the warning
+     * temporarily.
+     */
+#if defined(OPENSSL_SYS_VMS) && defined(__DECC)
+# pragma environment save
+# pragma message disable maylosedata2
 #endif
     /*
      * Don't buffer, because even if |file| is regular file, we have
@@ -106,6 +117,9 @@ int RAND_load_file(const char *file, long bytes)
      * contents lying around?
      */
     setbuf(in, NULL);
+#if defined(OPENSSL_SYS_VMS) && defined(__DECC)
+# pragma environment restore
+#endif
 
     for ( ; ; ) {
         if (bytes > 0)