patch 9.2.0168: invalid pointer casting in string_convert() arguments
Problem: invalid pointer casting in string_convert() arguments
Solution: Use a temporary local int variable (James McCoy)
string_convert()/string_convert_ext() accept an "int *lenp" parameter,
however a few call sites were taking the address of a possibly larger
type (long, size_t) and casting it as an int * when calling these
functions.
On big-endian platforms, this passes the (likely) zeroed high bytes of
the known length through to string_convert(). This indicates it received
an empty string and returns an allocated empty string rather than
converting the input. This is exhibited by test failures like
From test_blob.vim:
Found errors in Test_blob2str_multi_byte_encodings():
command line..script src/testdir/runtest.vim[636]..function RunTheTest[63]..Test_blob2str_multi_byte_encodings line 2: Expected ['Hello'] but got ['']
command line..script src/testdir/runtest.vim[636]..function RunTheTest[63]..Test_blob2str_multi_byte_encodings line 3: Expected ['Hello'] but got ['']
command line..script srctestdir/runtest.vim[636]..function RunTheTest[63]..Test_blob2str_multi_byte_encodings line 6: Expected ['Hello'] but got ['']
Instead, use a temporary local int variable as the in/out variable for
string_convert() and assign the result back to the larger typed length
variable post-conversion.
closes: #19672
Signed-off-by: James McCoy <jamessan@debian.org> Signed-off-by: Christian Brabandt <cb@256bit.org>