macro: rework IN_SET a bit

macro: rework IN_SET a bit

This makes use of assert_cc() to guard against missing CASE macros,
instead of a manual implementation that might result in a static
variable to be allocated.

More importantly though this changes the base type for the array used to
determine the number of arguments for the compile time check from "int"
to "long double". This is done in order to avoid warnings from "ubsan"
that possibly large constants are assigned to small types. "long double"
hopefully isn't vulnerable to that.

Fixes: #10332