Change the hex literal processing so that only the SQL parser understands

Change the hex literal processing so that only the SQL parser understands
hex literals.  Casting and coercing string literals into numeric values does
not understand hexadecimal integers.  This preserves backwards compatibility.
Also:  Throw an error on any hex literal that is too big to fit into 64 bits.

FossilOrigin-Name: 6c6f0de59bf96b79c8ace8c9bfe48c7a6a306a50