device-manager: Only wait for device timeout for framebuffer devices

device-manager: Only wait for device timeout for framebuffer devices

At the moment we ignore any udev events that come in before the
device timeout that are not drm devices.

That is because we don't want to use framebuffer devices as anything
but a last resort fallback option.

In the near future we're going to be handling input events from udev
as well. Those will obviously need to be handled right away, just like
drm devices.

This commit makes the check only defer framebuffer devices and not
anything that isn't a drm device.