[libc] Fix memcmp() to return proper values

Fix memcmp() to return proper standard positive/negative values for
unequal comparisons.  Current implementation is backwards (i.e. the
functions are returning negative when should be positive and
vice-versa).

Currently most consumers of these functions only check the return value
for ==0 or !=0 and so we can safely change the implementation without
breaking things.

However, there is one call that checks the polarity of this function,
and that is prf_sha1() for wireless WPA 4-way handshake.  Due to the
incorrect memcmp() polarity, the WPA handshake creates an incorrect
PTK, and the handshake would fail after step 2.  Undoubtedly, the AP
noticed the supplicant failed the mic check.  This commit fixes that
issue.

Similar to commit 3946aa9 ("[libc] Fix strcmp()/strncmp() to return
proper values").

Signed-off-by: Michael Bazzinotti <bazz@bazz1.com>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>

diff --git a/src/core/string.c b/src/core/string.c
index c35015b595206ac1815a4871ad92182cb2403241..188fe08646b3ffd3d21cd678ca8b7a81fdce8298 100644 (file)
--- a/src/core/string.c
+++ b/src/core/string.c
@@ -116,7 +116,7 @@ int memcmp ( const void *first, const void *second, size_t len ) {
        int diff;
 
        while ( len-- ) {
-               diff = ( *(second_bytes++) - *(first_bytes++) );
+               diff = ( *(first_bytes++) - *(second_bytes++) );
                if ( diff )
                        return diff;
        }

diff --git a/src/tests/string_test.c b/src/tests/string_test.c
index a66501da303c12a420f04e84e82e72ca75ed8dd1..88a730aecfeb31213f384acd063ad6b9561eff91 100644 (file)
--- a/src/tests/string_test.c
+++ b/src/tests/string_test.c
@@ -109,6 +109,7 @@ static void string_test_exec ( void ) {
        ok ( memcmp ( "", "", 0 ) == 0 );
        ok ( memcmp ( "Foo", "Foo", 3 ) == 0 );
        ok ( memcmp ( "Foo", "Bar", 3 ) != 0 );
+       ok ( memcmp ( "abc", "def", 3 ) < 0 );
 
        /* Test strstr() */
        {