detect/byte_test: Allow nbytes value to be a variable

detect/byte_test: Allow nbytes value to be a variable

Issue: 6144

This commit allows the byte_test keyword to accept an existing
variable name for a value (the value may still be specified directly as
an integer).

All nbytes values are subject to the same value constraints as before
- 23 if included with string
- 8 otherwise