client_dict_add_timeout() adds the timeout thinking that the dict no longer
has any work to do, so it can be freed soon. However, if other work is
added before that, the timeout just keeps getting called over and over
again. This is especially bad with the default idle timeout of 0.