BUG/MINOR: backend: check that the mux installed properly

BUG/MINOR: backend: check that the mux installed properly

The return value from conn_install_mux() was not checked, so if an
inconsistency happens in the code, or a memory allocation fails while
initializing the mux, we can crash while using an uninitialized mux.
In practice the code inconsistency does not really happen since we
cannot configure such a situation, except during development, but
the out of memory condition could definitely happen.

This should be backported to 1.8 (the code is a bit different there,
there are two calls to conn_install_mux()).