Set x11 as backend for gtk3 of open-vm-tools

Open-vm-tools defaults to gtk3, if available.  For distros where wayland is
the default display server, plugins will try to use wayland as the backend
of gtk3.  As a result, gdk_display_get_default() returns a wayland display,
and gdk_display_get_default_group() also returns a wayland window.  Then,
applying GDK_WINDOW_XID() on the wayland window will result in crashes.

As x11-specific code is widely used in tools plugins, as a compromized
solution, force plugins to run on XWayland, which is the compatible mode
of wayland for legacy X11 clients.

gdk_set_allowed_backends() only applies when flag GTK3 is defined,
and the gtk version is over 3.10.

diff --git a/open-vm-tools/services/plugins/desktopEvents/x11Lock.c b/open-vm-tools/services/plugins/desktopEvents/x11Lock.c
index 572234ac1b5b2d36702979daa961d89e9dc37297..613b69463fc47502bf96ebfe802998366f1b480c 100644 (file)
--- a/open-vm-tools/services/plugins/desktopEvents/x11Lock.c
+++ b/open-vm-tools/services/plugins/desktopEvents/x11Lock.c
@@ -1,5 +1,5 @@
 /*********************************************************
- * Copyright (C) 2010-2016 VMware, Inc. All rights reserved.
+ * Copyright (C) 2010-2018 VMware, Inc. All rights reserved.
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU Lesser General Public License as published
@@ -364,6 +364,16 @@ X11Lock_Init(ToolsAppCtx *ctx,
    g_set_prgname(VMUSER_TITLE);
    argv[0] = VMUSER_TITLE;
 
+#if GTK_MAJOR_VERSION > 3 || (GTK_MAJOR_VERSION == 3 && GTK_MINOR_VERSION >= 10)
+   /*
+    * On recent distros, Wayland is the default display server. If the obtained
+    * display or window is a wayland one, applying X11 specific functions on them
+    * will result in crashes. Before migrating the X11 specific code to Wayland,
+    * force using X11 as the backend of Gtk+3. gdk_set_allowed_backends() is
+    * introduced since Gtk+3.10 and Wayland is supported from Gtk+3.10.
+    */
+   gdk_set_allowed_backends("x11");
+#endif
    /* XXX: is calling gtk_init() multiple times safe? */
    gtk_init(&argc, (char ***) &argv);
 

diff --git a/open-vm-tools/services/plugins/dndcp/copyPasteDnDX11.cpp b/open-vm-tools/services/plugins/dndcp/copyPasteDnDX11.cpp
index fffdc402634b0881a80d3550d4a7bd019e12d9ca..4c2f3d8d56399140ceecac3e8ce7c965220adf49 100644 (file)
--- a/open-vm-tools/services/plugins/dndcp/copyPasteDnDX11.cpp
+++ b/open-vm-tools/services/plugins/dndcp/copyPasteDnDX11.cpp
@@ -1,5 +1,5 @@
 /*********************************************************
- * Copyright (C) 2010-2016 VMware, Inc. All rights reserved.
+ * Copyright (C) 2010-2018 VMware, Inc. All rights reserved.
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU Lesser General Public License as published
@@ -211,6 +211,18 @@ gboolean
 CopyPasteDnDX11::Init(ToolsAppCtx *ctx)
 {
    TRACE_CALL();
+
+#if GTK_MAJOR_VERSION > 3 || (GTK_MAJOR_VERSION == 3 && GTK_MINOR_VERSION >= 10)
+   /*
+    * On recent distros, Wayland is the default display server. If the obtained
+    * display or window is a wayland one, applying X11 specific functions on them
+    * will result in crashes. Before migrating the X11 specific code to Wayland,
+    * force using X11 as the backend of Gtk+3. gdk_set_allowed_backends() is
+    * introduced since Gtk+3.10 and Wayland is supported from Gtk+3.10.
+    */
+   gdk_set_allowed_backends("x11");
+#endif
+
    CopyPasteDnDWrapper *wrapper = CopyPasteDnDWrapper::GetInstance();
 
    ASSERT(ctx);