The debounce bit sets a bypass on the debounce rather than enabling it,
as such the current polarity of the debounce is set incorrectly. Invert
the polarity to correct this.
Fixes: d5282a539297 ("pinctrl: cs42l43: Add support for the cs42l43") Signed-off-by: Charles Keepax <ckeepax@opensource.cirrus.com> Reviewed-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com> Signed-off-by: Linus Walleij <linusw@kernel.org>