liblzma: Fix infinite loop in LZMA encoder init with dict_size >= 2 GiB.

liblzma: Fix infinite loop in LZMA encoder init with dict_size >= 2 GiB.

The encoder doesn't support dictionary sizes larger than 1536 MiB.
This is validated, for example, when calculating the memory usage
via lzma_raw_encoder_memusage(). It is also enforced by the LZ
part of the encoder initialization. However, LZMA encoder with
LZMA_MODE_NORMAL did an unsafe calculation with dict_size before
such validation and that results in an infinite loop if dict_size
was 2 << 30 or greater.