Oliver Kurth [Thu, 10 May 2018 23:38:11 +0000 (16:38 -0700)]
Remove debug display of CPUID vendor string seen in VmCheck_IsVirtualWorld
The "Debug" message containing vendor string returned by the CPUID
instruction will appear on STDERR for stand-alone programs such
as vmware-checkvm. This unexpected output is at best noise and may
adversely affect existing scripts.
Oliver Kurth [Mon, 30 Apr 2018 21:04:13 +0000 (14:04 -0700)]
[DeployPkg] Replace Log() references with g_log()
* Modified the code to replace a couple of Log() references with g_debug()
so that they are properly logged with 'deplogPkg' domain instead of the
generic 'vmsvc' domain in the tools log files.
Oliver Kurth [Tue, 24 Apr 2018 00:08:18 +0000 (17:08 -0700)]
[Wayland DnD] Part2: Open the uinput device file with root permission.
The device file /dev/uinput (or /dev/input/uinput) can only be accessed
by root account, so the dndcp plugin may failed to open this device file
if the current user is not root account.
A way to fix this issue is opening this device file with root account,
then pass the file descriptor to the sub task which is started with the
current account. An example for this solution is blockVM file system
device file.
This patch is part of the new feature 'Wayland support in Linux guest'.
Oliver Kurth [Tue, 24 Apr 2018 00:08:18 +0000 (17:08 -0700)]
[Wayland DnD] Part1: Add 'fakeMouse' module which is used to simulate the
mouse motion under Wayland
For Linux guest with X11, the VMTools uses the X system APIs to simulate
the mouse motion, but these X11 System APIs do not work in Wayland. Need
to pick up another method to simulate the mouse motion.
Another way to simulate the mouse motion is using uinput module.
uinput is a kernel module that makes it possible to emulate input devices
from userspace. By writing to /dev/uinput (or /dev/input/uinput) device,
a process can create a virtual input device with specific capabilities.
Once this virtual device is created, the process can send events through
it, that will be delivered to userspace and in-kernel consumers.
Here is the link which contains more information about the uinput module:
Oliver Kurth [Tue, 24 Apr 2018 00:08:17 +0000 (17:08 -0700)]
Bump up the version requirement for glib and friends to 2.34.0
For open-vm-tools, we are bumping up the minimum version requirement
to 2.34.0 for glib and its friends. Modified the configure.ac
with the proper new version checks. Also, updated the
error message with the proper version.
Oliver Kurth [Tue, 24 Apr 2018 00:08:17 +0000 (17:08 -0700)]
Common source file changes that are not directly related to open-vm-tools.
Build vmtools and vgauth for Windows with xmlsec instead of
xml-security-c. It will still be possible to build with xml-security-c
either by commenting out the definition of USE_XMLSEC_FOR_WINDOWS in
bora-vmsoft/vgauth/make/inc/defs.mk or adding a make command line
argument of USE_XMLSEC_FOR_WINDOWS=0.
Oliver Kurth [Tue, 24 Apr 2018 00:08:17 +0000 (17:08 -0700)]
Update bora-vmsoft/install/Source/README
The content is stale. Since the file is used internally and is
not published, rather than maintain it in two places, point to
the current README file and Release Notes on github.
Oliver Kurth [Tue, 24 Apr 2018 00:08:15 +0000 (17:08 -0700)]
FreeBSD vmmemctl: Fold in open-vm-tools pull requests #125 and #140
This change incorporates two open-vm-tools pull requests that provide
code changes that allows the existing vmmemctl.ko driver to be compiled
on FreeBSD 10, 11 or 12.
Pull request #125:
An optional argument has been added to the sysctl_add_oid() function
in FreeBSD 12.x. All releases have a SYSCTL_ADD_OID() macro that does
not require the additional argument.
Pull request #140:
Functions and macros to lock and unlock memory pages and objects have
changed between the FBSD 10 & 11. The pull request provides a set C
preprocessor directives to select the appropriate code based on the
FreeBSD release being targeted.
The code change as been tidied up by using the release version checks
to define LOCK and UNLOCK macros for memory pages and objects at the
beginning of the source file. The code logic is easier to follow
without the abundance of #if, #else and #endif statements scattered
about.
Ed Schouten: https://github.com/vmware/open-vm-tools/pull/125
Steve Wills: https://github.com/vmware/open-vm-tools/pull/140
Oliver Kurth [Fri, 23 Mar 2018 22:05:35 +0000 (15:05 -0700)]
Add missing memory constraints for vm_atomic RMW (Read/Modify/Write) instructions
There are two inseparable concepts involved when dealing with atomics:
1. Atomicity of the access itself
2. Ordering of the access with respect to other reads&writes (from the view of
other processors).
Our Read-Modify-Write functions are all meant to provide the highest level of
ordering guarantee: Sequential Consistency, which means no reordering of reads
or writes across the access. We properly implement that on ARM, and on
x86/x64 at the hardware layer. But, on x86/x64 we needed to tell the compiler
(it must flush out any pending reads/writes that are currently hiding in
registers)
Side Note: we do *not* change the pure Read and pure Write functions, only
the Read/Modify/Write ones. On both ARM and x86/x64, vm_atomic functions like
Atomic_WriteN provide no (re)ordering guarantees today (at the hardware layer
on ARM or at the compiler layer on x86/x64). This is because some callers
didn't need or want such guarantees - as such, Atomic_WriteN by itself is
*not* sufficient to, say, release a lock. Making these remaining atomic
weapons safe by default will require us to first
1. add new unordered atomic equivalents of Atomic_ReadN/TestBitN and
Atomic_WriteN (in C11 terminology, acquire/release and relaxed)
2. Scan the tree and switch appropriate callers to the new functions
*and only then*
3. Strengthen the defaults, affecting only callers who needed the stronger
defaults to be correct. ... but that would be a separate change in the
future.
Codegen differences (vmm.vmm64):
--------------------------------
The function that uncovered this was ST_HandleCrossCall, which invoked
Atomic_And64.
original w/o explicit compiler mem barrier in ST_HandleCrossCall:
...
lock and QWORD PTR [rdx+0x0],rax
movsxd rax,DWORD PTR [rip+offset] <-- load reordered after Atomic_And64
lea rax,[rax+rax*2]
lea rdi,[rax*8+0x0]
with "memory" constraint, it's now identical to code w/an explicit
compiler mem barrier (which was the workaround):
...
mov edx,DWORD PTR [rip+offset] <-- load emitted prior to Atomic_And64
...
lock and QWORD PTR [rsi+0x0],rax
movsxd rax,edx <-- ... and used after Atomic_And64
lea rax,[rax+rax*2]
lea rdi,[rax*8+0x0]
Given that these are tiny fractions of the .text section, this suggests the
problem was quite rare, which is why it escaped our attention until now.
(Testing of vmx's lib/lock, lib/sync, lib/vprobe, and lib/misc saw no
differences with GCC 4.4, 6.4, 7.1 or Clang)
Oliver Kurth [Fri, 23 Mar 2018 22:05:35 +0000 (15:05 -0700)]
vmcheck.c VmCheck_IsVirtualWorld(): Always check for a working backdoor.
The specific checks for Xen and VirtualPC hypervisors currently happen
only if the VMware hypervisor is not detected. The test for a working
VMware backdoor is then done to avoid all other hypervisors.
In the case where running on a VMware hypervisor and the backdoor
channel has been disabled such as with
monitor_control.restrict_backdoor = "TRUE"
vmtoolsd, vmware-toolbox-cmd and vmware-checkvm will crash when
attempting to get the version number of the installed VMware Tools.
Added an additional test to detect Linux KVM with the existing tests
for the Xen hypervisor and Microsoft Virtual PC. Avoid checking for
a working backdoor if a non VMware hypervisor is seen.
Microsoft Hv checking to be added later and is tracked in a separate PR.
Oracle VirtualBox provides no unique CPUID vendor signature string.
With the change for VMware headers to utilize the standard types
defined in C99 standard headers, the compilation of FreeBSD kernel
modules must specifically add /usr/include to the compilation
options.
CFLAGS += -isystem /usr/include
This change updates the Makefiles's for FreeBSD kernel modules and
drivers.
Oliver Kurth [Fri, 23 Mar 2018 22:05:35 +0000 (15:05 -0700)]
FreeBSD: Improper use of sysconf() for getpwent buffer size leads to
vmtoolsd crash.
On FreeBSD, sysconf(_SC_GETPW_R_SIZE_MAX) can return -1 if it has
no hard limit ultimately resulting in an incorrect buffer size.
This change is adapting the sysconf() ifixes done elsewhere to
bora-vmsoft/services/plugins/vix/vixTools.c and updating the
open-vm-tools AUTHORS file to share credit for the pull request.
Oliver Kurth [Fri, 23 Mar 2018 21:57:12 +0000 (14:57 -0700)]
[lib/file]: remove useless heap allocation
FileIO_AtomicUpdateEx allocates the argument to the "swap" ioctl
on the heap. This argument is a struct which contains a single int
fd... there is no need to heap-allocate it, the stack is just fine.
Oliver Kurth [Fri, 23 Mar 2018 21:57:12 +0000 (14:57 -0700)]
bora/lib: Use _exit when in the context of a signal handler
There are a few cases in bora/lib where exit(3) is used in the
context of a signal handler. It was recently noticed that exit(3) is
not async-signal-safe, and _exit(2) should be used instead.
This patch changes a call from exit(3) to _exit(2) in bora/lib.
Oliver Kurth [Fri, 23 Mar 2018 21:57:12 +0000 (14:57 -0700)]
Hgfs FUSE Client: enable the FUSE client for RHEL 7, Ubuntu 14.04 and others
The RHEL 7.x releases are all kernel 3.10 based. Currently the Hgfs FUSE
client will not be enabled for these earlier kernel versions, enabled only
for 4.0 kernels and later. This means for default tools install on plaftorms
running those kernel versions the Hgfs kernel client must be installed and
used for Shared Folders. The consequence of this means that the tar tools
installer must be run to provide the Hgfs kernel client on top of the OVT
install.
To remove the need to install the tar tools we must enable support for the
Hgfs FUSE client, which just requires the kernel version check to be modified
from 4.0.0 to 3.10.0.
Oliver Kurth [Tue, 6 Mar 2018 18:38:43 +0000 (10:38 -0800)]
Enhancements for rpcChannel library.
rpcChannel.c file contains the code for building rpcChannel library. A good
amount of the same code is duplicated in rpcChannelSimple.c file to
build a simplified version (no glib dependency) of the rpcChannel library.
It's recommended to remove the duplication and maintain only one single
file which can be used to build both 'glib-dependent rpcChannel' and
'glib independent rpcChannel' library.
This changeset implements the following changes:
* Removed rpcChannelSimple.c
* Renamed I_USE_SIMPLE_RPC make variable to USE_RPCI_ONLY. If this
flag is specified, rpcChannel library will be built using only RPCI.
No dependencies on rpcIn will be added. This inturn will remove
dependencies on dynxdr, xdr and glib.
* Modified the RpcChannelInt structure to have only the necessary
attributes when USE_RPCI_ONLY is defined.
* Modified a bunch of functions into '#if defined(NEED_RPCIN)' block
to make available only when necessary.
* Modified guestrpc.h to provide the declarations for few RpcChannel_*
functions only when needed.
* Modified rpcChannel library to implement necessary stub files / functions
when it has to be built 'with rpci only' setting.
* Removed various copies of 'glib_stubs.c' maintained by different
consumers of rpcChannel (appmonitorlib, vmGuestLib, 'imgcust' components).
* Modified the make files of various consumers (vmGuestLib, appmonitorlib,
imgcust components [linux-pkg-deployer, guestcustutil, UnitTest]) to
remove the dependencies for dynxdr library.
* Did some code refactoring.
- Removed some dead code in some make files.
Oliver Kurth [Tue, 6 Mar 2018 18:38:43 +0000 (10:38 -0800)]
Implement a new function VMTools_GetTimeAsString.
* Implemented a new function VMTools_GetTimeAsString in
vmtools library which returns a properly formatted UTC timestamp
information. This function can be used by different modules / plugins
in 'VMware Tools' to add the timestamp information in log files.
Example of UTC timestamp information: "2018-02-22T21:17:38.517Z"
* Modified vmtools logging module to call the new function and
prepend the UTC timestamp information in every log message.
* Modified deployPkgLog.c to use the new function.
* Removed references / definition of System_GetTimeAsString since
it is no longer used anywhere in the code.
Oliver Kurth [Tue, 6 Mar 2018 18:38:42 +0000 (10:38 -0800)]
Add timestamp information to each deploypkg log message.
* Modified deployPkgLog.c file to prepend UTC timestamp information
to each log message coming from 'deploypkg' module in 'VMware Tools'.
* Used bunch of glib functions to get the current UTC time. As per
the glib manual, these glib functions are available since 2.26. 2.26 was
released around 8 years ago which is kind of very old. Having the glib
dependency on 2.26 is OK.
* Did some minor code refactoring. In DeployPkg.c file, at all
call sites for DeployPkgLog_Log, replaced the hard coded values with
the proper LogLevel enums.
Oliver Kurth [Tue, 6 Mar 2018 18:38:42 +0000 (10:38 -0800)]
vm_basic_types.h: enable stdint.h for c99
C99 mode (a.k.a. __STDC_VERSION__ >= 199901L) guarantees
the stdint.h header will be present, which means we can use it
for standard types instead of rolling our own.
Oliver Kurth [Tue, 6 Mar 2018 18:38:41 +0000 (10:38 -0800)]
Don't lose errno/lastError when logging
guestOps need to see errno/lastError in order to return the proper
error to VIX/guestOps APIs. Logging (at any layer) can cause these
to be overwritten, so make sure they're preserved.
Oliver Kurth [Tue, 27 Feb 2018 03:23:18 +0000 (19:23 -0800)]
Enhancing CAF listener preconfigured check logic to update the preconfigured flag at runtime
1. Preconfigure listener and start listener upon tunnel enabled logic are
invoked in two different threads.
2. There is a case where preconfigure logic is performed after the
pre-configured flag is computed in the enable-listener thread.
The current patch invalidates the flag in the start listener thread and
should guarantee the preconfigured status is up-to-date.
Oliver Kurth [Tue, 27 Feb 2018 03:22:04 +0000 (19:22 -0800)]
CAF MA Performance Issues
CAF MA Performance Issues
Vmware CAF ManagementAgentHost service was polling every 5 secs to check if Guest Network "Tunnel(port 6672)" was enabled to preconfigure & bring up Vmware CAF CommAmqpListener service.
This resulted in more CPU Utilization as opposed to previous release in CAF & caused regression in vmtools 10.2.0.
We are increasing the Poll rate to 5 mins, which fixed the issue & is giving us the same CPU/memory utilizations as in vmtools 10.1.0.
We were checking for the different stages of listener preconfiguration status from the listenerpreConfigure.txt files in the polling time interval, which caused increased IO operations.
To fix that we have stored the value of listener preconfiguration status from the files into a variable locally & using the same at all places instead of file access.
Oliver Kurth [Mon, 26 Feb 2018 20:35:36 +0000 (12:35 -0800)]
Skip specified file systems when doing a quiesced snapshot on Linux
Add a tools.conf setting "excludedFileSystems" that specifies one or
more file system mount points to be skipped over when performing a
quiesced snapshot on Linux guests. The value of excludedFileSystems
is a comma-separated list of glob-style patterns as recognized by the
glib routines described here:
With this change, when performing a quiesced snapshot, the sync driver
freeze routine removes from the list of mount points it is to process
any path that matches a pattern in the excludedFileSystems list.
In the course of testing the change, a bug was found in SyncDriverFreeze
in which it returned SD_ERROR rather than FALSE when the path list is
empty.
This change also includes some whitespace cleanup in syncDriverWin32.c.
Oliver Kurth [Mon, 26 Feb 2018 20:35:36 +0000 (12:35 -0800)]
Skip davfs mount points during quiescing
The davfs2 is a Linux filesystem that allows mounting a Web Distributed
Authoring and Versioning (WebDAV) network resource as a local filesystem.
On some Linux releases, the FREEZE ioctl() is rejected and quiescing
these filesystems is not attempted. On others, however, the FREEZE is
not rejected and there exists the potential of a quiescing deadlock
if the davfs2 attempts to write to its cache in /var which has already
been quiesced.
This fix is to avoid a potential deadlock by excluding davfs2 mounts
as the networking filesystem they are. Since davfs2 may be implemented
on the Linux fuse (default) or coda filesystem kernel module,
the detection of these network mounts must be based upon detection
of the URL prefix of the mounted device name.
The static function SyncDriverIsRemoteFSType() is being updated to
not only exclude remote filesystems based on filesystem type but also
to exclude davfs2 remote filesystems by the "http://" or "https://"
URL prefix on the device name. The function name is being changed
to SyncDriverIsRemoteFS().
Oliver Kurth [Mon, 26 Feb 2018 20:35:36 +0000 (12:35 -0800)]
Use a unique random temp directory for vmtoolsd on Linux.
The temporay directory currently used by vmtoolsd and its plugins
on Linux is of the form /tmp/vmware-<user>. Since it is used to
upload VMware Tools upgrade and GOS customization scripts and commands,
that name predictability may make it susceptible to attack. This
change adds a new function File_GetSafeRandomTmpDir() in
bora/lib/file/fileTempPosix.c to both add the PID to the user name
and add a random number suffix to the temp directory path.
/tmp/vmware-<user>_<pid>-nnnnnn
VMware Tools commands and plugins are being updated to use this random
temp directory.
Oliver Kurth [Mon, 26 Feb 2018 20:35:35 +0000 (12:35 -0800)]
open-vm-tools: ignore with/without gtk2/3 options when building without X
When building without X support, we should ignore the options to enable/disable
gtk2 and gtk3. This was reported in gthub issue
https://github.com/vmware/open-vm-tools/issues/228 .
Oliver Kurth [Mon, 26 Feb 2018 20:35:35 +0000 (12:35 -0800)]
Ignore ENXIO errors with SyncDriver
A quiesced snapshot fails when the target VM has a bind mount. The
problem is that the syncDriver gets an ENXIO error when it tries to
open the mount point. To fix the problem, let the quiesce operation
skip a mount point when opening it results in an ENXIO error.
This change is based on pull request
https://github.com/vmware/open-vm-tools/pull/218 .
Oliver Kurth [Mon, 26 Feb 2018 20:35:35 +0000 (12:35 -0800)]
open-vm-tools: use pkg-config for building with icu
icu uses pkg-config in recent versions, and no longer ships with
icu-config in most recent versions. Make the configure script
use pkg-config for icu if available, and icu-config if not. For
pkg-config we need to specify 'icui18n' which returns the same result
as icu-config did to link with icui18n.
Oliver Kurth [Mon, 26 Feb 2018 20:29:08 +0000 (12:29 -0800)]
lib/file: add File_MakeSafeTempSubdir
File_MakeSafeTempDir and File_GetSafeTmpDir are typically used to create
"safe" temp dirs on Windows and Linux. These are dirs in a known-secure
location with known-secure security attributes that only allow access to
the current user. These APIs work fine. However, on Windows if one creates
a subdir inside the safe dir, the subdir may be given an unexpected
(and overly restrictive) set of default DACLs on it in unusual situations.
The solution is to introduce a new API specifically to allow a caller to
create a subdir within an existing safe dir that ensures the DACLs
are exactly what are needed.
Oliver Kurth [Mon, 26 Feb 2018 20:29:07 +0000 (12:29 -0800)]
Skip specified file systems when doing a quiesced snapshot on Linux
Add a tools.conf setting "excludedFileSystems" that specifies one or
more file system mount points to be skipped over when performing a
quiesced snapshot on Linux guests. The value of excludedFileSystems
is a comma-separated list of glob-style patterns as recognized by the
glib routines described here:
With this change, when performing a quiesced snapshot, the sync driver
freeze routine removes from the list of mount points it is to process
any path that matches a pattern in the excludedFileSystems list.
In the course of testing the change, a bug was found in SyncDriverFreeze
in which it returned SD_ERROR rather than FALSE when the path list is
empty.
This change also includes some whitespace cleanup in syncDriverWin32.c.
Oliver Kurth [Mon, 26 Feb 2018 20:29:07 +0000 (12:29 -0800)]
lib/file: File_GetSafeTmpDir is not aware of credentials changes
Some applications may masquerade their use (change their EUID).
Futhermore, each thread in an application can have a separate EUID.
Anytime a application or thread asked for its safe temporary directory,
it needs to get the same result regardless of how many times it asks.
Change File_GetSafeTmpDir to cache the EUID associated with the cached
values. If there is an EUID change, invalidate the existing cache
entries and recompute them. The recomputation process is stable, in
that it will obtain the same return any time it is called.
This way we get the benefit of the cache (performance), ensured
correctness (for applications that do not masquerade), and correctness
for those applications that do masquerade.