Oliver Kurth [Fri, 23 Mar 2018 21:57:12 +0000 (14:57 -0700)]
[lib/file]: remove useless heap allocation
FileIO_AtomicUpdateEx allocates the argument to the "swap" ioctl
on the heap. This argument is a struct which contains a single int
fd... there is no need to heap-allocate it, the stack is just fine.
Oliver Kurth [Fri, 23 Mar 2018 21:57:12 +0000 (14:57 -0700)]
bora/lib: Use _exit when in the context of a signal handler
There are a few cases in bora/lib where exit(3) is used in the
context of a signal handler. It was recently noticed that exit(3) is
not async-signal-safe, and _exit(2) should be used instead.
This patch changes a call from exit(3) to _exit(2) in bora/lib.
Oliver Kurth [Fri, 23 Mar 2018 21:57:12 +0000 (14:57 -0700)]
Hgfs FUSE Client: enable the FUSE client for RHEL 7, Ubuntu 14.04 and others
The RHEL 7.x releases are all kernel 3.10 based. Currently the Hgfs FUSE
client will not be enabled for these earlier kernel versions, enabled only
for 4.0 kernels and later. This means for default tools install on plaftorms
running those kernel versions the Hgfs kernel client must be installed and
used for Shared Folders. The consequence of this means that the tar tools
installer must be run to provide the Hgfs kernel client on top of the OVT
install.
To remove the need to install the tar tools we must enable support for the
Hgfs FUSE client, which just requires the kernel version check to be modified
from 4.0.0 to 3.10.0.
Oliver Kurth [Tue, 6 Mar 2018 18:38:43 +0000 (10:38 -0800)]
Enhancements for rpcChannel library.
rpcChannel.c file contains the code for building rpcChannel library. A good
amount of the same code is duplicated in rpcChannelSimple.c file to
build a simplified version (no glib dependency) of the rpcChannel library.
It's recommended to remove the duplication and maintain only one single
file which can be used to build both 'glib-dependent rpcChannel' and
'glib independent rpcChannel' library.
This changeset implements the following changes:
* Removed rpcChannelSimple.c
* Renamed I_USE_SIMPLE_RPC make variable to USE_RPCI_ONLY. If this
flag is specified, rpcChannel library will be built using only RPCI.
No dependencies on rpcIn will be added. This inturn will remove
dependencies on dynxdr, xdr and glib.
* Modified the RpcChannelInt structure to have only the necessary
attributes when USE_RPCI_ONLY is defined.
* Modified a bunch of functions into '#if defined(NEED_RPCIN)' block
to make available only when necessary.
* Modified guestrpc.h to provide the declarations for few RpcChannel_*
functions only when needed.
* Modified rpcChannel library to implement necessary stub files / functions
when it has to be built 'with rpci only' setting.
* Removed various copies of 'glib_stubs.c' maintained by different
consumers of rpcChannel (appmonitorlib, vmGuestLib, 'imgcust' components).
* Modified the make files of various consumers (vmGuestLib, appmonitorlib,
imgcust components [linux-pkg-deployer, guestcustutil, UnitTest]) to
remove the dependencies for dynxdr library.
* Did some code refactoring.
- Removed some dead code in some make files.
Oliver Kurth [Tue, 6 Mar 2018 18:38:43 +0000 (10:38 -0800)]
Implement a new function VMTools_GetTimeAsString.
* Implemented a new function VMTools_GetTimeAsString in
vmtools library which returns a properly formatted UTC timestamp
information. This function can be used by different modules / plugins
in 'VMware Tools' to add the timestamp information in log files.
Example of UTC timestamp information: "2018-02-22T21:17:38.517Z"
* Modified vmtools logging module to call the new function and
prepend the UTC timestamp information in every log message.
* Modified deployPkgLog.c to use the new function.
* Removed references / definition of System_GetTimeAsString since
it is no longer used anywhere in the code.
Oliver Kurth [Tue, 6 Mar 2018 18:38:42 +0000 (10:38 -0800)]
Add timestamp information to each deploypkg log message.
* Modified deployPkgLog.c file to prepend UTC timestamp information
to each log message coming from 'deploypkg' module in 'VMware Tools'.
* Used bunch of glib functions to get the current UTC time. As per
the glib manual, these glib functions are available since 2.26. 2.26 was
released around 8 years ago which is kind of very old. Having the glib
dependency on 2.26 is OK.
* Did some minor code refactoring. In DeployPkg.c file, at all
call sites for DeployPkgLog_Log, replaced the hard coded values with
the proper LogLevel enums.
Oliver Kurth [Tue, 6 Mar 2018 18:38:42 +0000 (10:38 -0800)]
vm_basic_types.h: enable stdint.h for c99
C99 mode (a.k.a. __STDC_VERSION__ >= 199901L) guarantees
the stdint.h header will be present, which means we can use it
for standard types instead of rolling our own.
Oliver Kurth [Tue, 6 Mar 2018 18:38:41 +0000 (10:38 -0800)]
Don't lose errno/lastError when logging
guestOps need to see errno/lastError in order to return the proper
error to VIX/guestOps APIs. Logging (at any layer) can cause these
to be overwritten, so make sure they're preserved.
Oliver Kurth [Tue, 27 Feb 2018 03:23:18 +0000 (19:23 -0800)]
Enhancing CAF listener preconfigured check logic to update the preconfigured flag at runtime
1. Preconfigure listener and start listener upon tunnel enabled logic are
invoked in two different threads.
2. There is a case where preconfigure logic is performed after the
pre-configured flag is computed in the enable-listener thread.
The current patch invalidates the flag in the start listener thread and
should guarantee the preconfigured status is up-to-date.
Oliver Kurth [Tue, 27 Feb 2018 03:22:04 +0000 (19:22 -0800)]
CAF MA Performance Issues
CAF MA Performance Issues
Vmware CAF ManagementAgentHost service was polling every 5 secs to check if Guest Network "Tunnel(port 6672)" was enabled to preconfigure & bring up Vmware CAF CommAmqpListener service.
This resulted in more CPU Utilization as opposed to previous release in CAF & caused regression in vmtools 10.2.0.
We are increasing the Poll rate to 5 mins, which fixed the issue & is giving us the same CPU/memory utilizations as in vmtools 10.1.0.
We were checking for the different stages of listener preconfiguration status from the listenerpreConfigure.txt files in the polling time interval, which caused increased IO operations.
To fix that we have stored the value of listener preconfiguration status from the files into a variable locally & using the same at all places instead of file access.
Oliver Kurth [Mon, 26 Feb 2018 20:35:36 +0000 (12:35 -0800)]
Skip specified file systems when doing a quiesced snapshot on Linux
Add a tools.conf setting "excludedFileSystems" that specifies one or
more file system mount points to be skipped over when performing a
quiesced snapshot on Linux guests. The value of excludedFileSystems
is a comma-separated list of glob-style patterns as recognized by the
glib routines described here:
With this change, when performing a quiesced snapshot, the sync driver
freeze routine removes from the list of mount points it is to process
any path that matches a pattern in the excludedFileSystems list.
In the course of testing the change, a bug was found in SyncDriverFreeze
in which it returned SD_ERROR rather than FALSE when the path list is
empty.
This change also includes some whitespace cleanup in syncDriverWin32.c.
Oliver Kurth [Mon, 26 Feb 2018 20:35:36 +0000 (12:35 -0800)]
Skip davfs mount points during quiescing
The davfs2 is a Linux filesystem that allows mounting a Web Distributed
Authoring and Versioning (WebDAV) network resource as a local filesystem.
On some Linux releases, the FREEZE ioctl() is rejected and quiescing
these filesystems is not attempted. On others, however, the FREEZE is
not rejected and there exists the potential of a quiescing deadlock
if the davfs2 attempts to write to its cache in /var which has already
been quiesced.
This fix is to avoid a potential deadlock by excluding davfs2 mounts
as the networking filesystem they are. Since davfs2 may be implemented
on the Linux fuse (default) or coda filesystem kernel module,
the detection of these network mounts must be based upon detection
of the URL prefix of the mounted device name.
The static function SyncDriverIsRemoteFSType() is being updated to
not only exclude remote filesystems based on filesystem type but also
to exclude davfs2 remote filesystems by the "http://" or "https://"
URL prefix on the device name. The function name is being changed
to SyncDriverIsRemoteFS().
Oliver Kurth [Mon, 26 Feb 2018 20:35:36 +0000 (12:35 -0800)]
Use a unique random temp directory for vmtoolsd on Linux.
The temporay directory currently used by vmtoolsd and its plugins
on Linux is of the form /tmp/vmware-<user>. Since it is used to
upload VMware Tools upgrade and GOS customization scripts and commands,
that name predictability may make it susceptible to attack. This
change adds a new function File_GetSafeRandomTmpDir() in
bora/lib/file/fileTempPosix.c to both add the PID to the user name
and add a random number suffix to the temp directory path.
/tmp/vmware-<user>_<pid>-nnnnnn
VMware Tools commands and plugins are being updated to use this random
temp directory.
Oliver Kurth [Mon, 26 Feb 2018 20:35:35 +0000 (12:35 -0800)]
open-vm-tools: ignore with/without gtk2/3 options when building without X
When building without X support, we should ignore the options to enable/disable
gtk2 and gtk3. This was reported in gthub issue
https://github.com/vmware/open-vm-tools/issues/228 .
Oliver Kurth [Mon, 26 Feb 2018 20:35:35 +0000 (12:35 -0800)]
Ignore ENXIO errors with SyncDriver
A quiesced snapshot fails when the target VM has a bind mount. The
problem is that the syncDriver gets an ENXIO error when it tries to
open the mount point. To fix the problem, let the quiesce operation
skip a mount point when opening it results in an ENXIO error.
This change is based on pull request
https://github.com/vmware/open-vm-tools/pull/218 .
Oliver Kurth [Mon, 26 Feb 2018 20:35:35 +0000 (12:35 -0800)]
open-vm-tools: use pkg-config for building with icu
icu uses pkg-config in recent versions, and no longer ships with
icu-config in most recent versions. Make the configure script
use pkg-config for icu if available, and icu-config if not. For
pkg-config we need to specify 'icui18n' which returns the same result
as icu-config did to link with icui18n.
Oliver Kurth [Mon, 26 Feb 2018 20:29:08 +0000 (12:29 -0800)]
lib/file: add File_MakeSafeTempSubdir
File_MakeSafeTempDir and File_GetSafeTmpDir are typically used to create
"safe" temp dirs on Windows and Linux. These are dirs in a known-secure
location with known-secure security attributes that only allow access to
the current user. These APIs work fine. However, on Windows if one creates
a subdir inside the safe dir, the subdir may be given an unexpected
(and overly restrictive) set of default DACLs on it in unusual situations.
The solution is to introduce a new API specifically to allow a caller to
create a subdir within an existing safe dir that ensures the DACLs
are exactly what are needed.
Oliver Kurth [Mon, 26 Feb 2018 20:29:07 +0000 (12:29 -0800)]
Skip specified file systems when doing a quiesced snapshot on Linux
Add a tools.conf setting "excludedFileSystems" that specifies one or
more file system mount points to be skipped over when performing a
quiesced snapshot on Linux guests. The value of excludedFileSystems
is a comma-separated list of glob-style patterns as recognized by the
glib routines described here:
With this change, when performing a quiesced snapshot, the sync driver
freeze routine removes from the list of mount points it is to process
any path that matches a pattern in the excludedFileSystems list.
In the course of testing the change, a bug was found in SyncDriverFreeze
in which it returned SD_ERROR rather than FALSE when the path list is
empty.
This change also includes some whitespace cleanup in syncDriverWin32.c.
Oliver Kurth [Mon, 26 Feb 2018 20:29:07 +0000 (12:29 -0800)]
lib/file: File_GetSafeTmpDir is not aware of credentials changes
Some applications may masquerade their use (change their EUID).
Futhermore, each thread in an application can have a separate EUID.
Anytime a application or thread asked for its safe temporary directory,
it needs to get the same result regardless of how many times it asks.
Change File_GetSafeTmpDir to cache the EUID associated with the cached
values. If there is an EUID change, invalidate the existing cache
entries and recompute them. The recomputation process is stable, in
that it will obtain the same return any time it is called.
This way we get the benefit of the cache (performance), ensured
correctness (for applications that do not masquerade), and correctness
for those applications that do masquerade.
Oliver Kurth [Mon, 26 Feb 2018 20:29:06 +0000 (12:29 -0800)]
Skip davfs mount points during quiescing (part 2)
Refactor the change to skip davfs mount points. Capture the length
of the static URL prefixes at compile time and avoid repetitive
strlen() calls when checking each active mount point.
Oliver Kurth [Mon, 26 Feb 2018 20:29:06 +0000 (12:29 -0800)]
Skip davfs mount points during quiescing
The davfs2 is a Linux filesystem that allows mounting a Web Distributed
Authoring and Versioning (WebDAV) network resource as a local filesystem.
On some Linux releases, the FREEZE ioctl() is rejected and quiescing
these filesystems is not attempted. On others, however, the FREEZE is
not rejected and there exists the potential of a quiescing deadlock
if the davfs2 attempts to write to its cache in /var which has already
been quiesced.
This fix is to avoid a potential deadlock by excluding davfs2 mounts
as the networking filesystem they are. Since davfs2 may be implemented
on the Linux fuse (default) or coda filesystem kernel module,
the detection of these network mounts must be based upon detection
of the URL prefix of the mounted device name.
The static function SyncDriverIsRemoteFSType() is being updated to
not only exclude remote filesystems based on filesystem type but also
to exclude davfs2 remote filesystems by the "http://" or "https://"
URL prefix on the device name. The function name is being changed
to SyncDriverIsRemoteFS().
Oliver Kurth [Mon, 26 Feb 2018 20:29:06 +0000 (12:29 -0800)]
vm_basic_types.h: scrub FMTxx specifiers
Reduce the amount of magic in vm_basic_types.h... nowadays, any
system we support (including Tools) DEFINITELY supports C99-style
specifiers, with two exceptions: pre-vs2015 MSVC, and pre-10.13
macOS kernels.
Rip out all the pre-C99 junk. Add a few comments explaining exactly
what the caveats here are.
Oliver Kurth [Mon, 26 Feb 2018 20:29:05 +0000 (12:29 -0800)]
Remove uses of header file safetime.h
- (many places) replace "safetime.h" with <time.h> and/or <sys/stat.h>,
occasionally re-ordering this header to top of file
- (many places) remove "safetime.h" as no time functions are used
Oliver Kurth [Thu, 8 Feb 2018 00:39:26 +0000 (16:39 -0800)]
Use a unique random temp directory for vmtoolsd on Linux.
The temporay directory currently used by vmtoolsd and its plugins
on Linux is of the form /tmp/vmware-<user>. Since it is used to
upload VMware Tools upgrade and GOS customization scripts and commands,
that name predictability may make it susceptible to attack. This
change adds a new function File_GetSafeRandomTmpDir() in
bora/lib/file/fileTempPosix.c to both add the PID to the user name
and add a random number suffix to the temp directory path.
/tmp/vmware-<user>_<pid>-nnnnnn
VMware Tools commands and plugins are being updated to use this random
temp directory.
Oliver Kurth [Thu, 8 Feb 2018 00:39:26 +0000 (16:39 -0800)]
open-vm-tools: ignore with/without gtk2/3 options when building without X
When building without X support, we should ignore the options to enable/disable
gtk2 and gtk3. This was reported in gthub issue
https://github.com/vmware/open-vm-tools/issues/228 .
Oliver Kurth [Wed, 7 Feb 2018 00:32:40 +0000 (16:32 -0800)]
open-vm-tools: use pkg-config for building with icu
icu uses pkg-config in recent versions, and no longer ships with
icu-config in most recent versions. Make the configure script
use pkg-config for icu if available, and icu-config if not. For
pkg-config we need to specify 'icui18n' which returns the same result
as icu-config did to link with icui18n.
Oliver Kurth [Wed, 7 Feb 2018 00:32:40 +0000 (16:32 -0800)]
Ignore ENXIO errors with SyncDriver
A quiesced snapshot fails when the target VM has a bind mount. The
problem is that the syncDriver gets an ENXIO error when it tries to
open the mount point. To fix the problem, let the quiesce operation
skip a mount point when opening it results in an ENXIO error.
This change is based on pull request
https://github.com/vmware/open-vm-tools/pull/218 .
Oliver Kurth [Wed, 7 Feb 2018 00:32:39 +0000 (16:32 -0800)]
open-vm-tools: use ENV{DEVTYPE}=="disk" in udev scsi rule
Fix udev scsi rule to not print warnings on boot.
The rule now checks for the device type. The previous fix by
trying to check for a valid timeout field did not work as expected.
This change uses the rule from Mike Latimer at
https://github.com/vmware/open-vm-tools/pull/216
Oliver Kurth [Wed, 7 Feb 2018 00:32:38 +0000 (16:32 -0800)]
guestOS identification: Tweak handling the LSB
The multi-standard handing code dealt with the LSB strictly. This
causes some issues with older distros that are not LSB compliant but
happened to work due to a specific behavior.
Return the LSB to "LSB and old way" and provide some specific coverage
of detaults to ensure coverage.
Oliver Kurth [Wed, 7 Feb 2018 00:32:38 +0000 (16:32 -0800)]
Restore "ufs" to the list of known filesystems
A previous change, 5074636, to remove "ufs" from the kernel also removed
"ufs" from the known list of Linux filesystems. As a result, the "ufs"
filesystems on FreeBSD and Solaris VMs cannot be seen and therefore
no disk information is available in vmsvc/get.guest output.
Oliver Kurth [Wed, 7 Feb 2018 00:32:37 +0000 (16:32 -0800)]
GuestOS: Support the os-release standard
While bringing up Amazon Linux as a guest, it was discovered that
Amazon Linux was not LSB compilant. The LSB standard defines a way
to uniquely determine which distro one is running on. Since our code
was based on the LSB standard, we made some changes to explicitly
detect Amazon Linux and handle it as a special case.
Amazon let us know that there is a "new" standard - os-release - which
they are compliant with. One must explicitly install Amazon Linux in a
special way to obtain the LSB compliance.
Researching the "new" standard, it turned out to not be that different
than the LSB standard. Further research showed that some distros (e.g.
Ubuntu) are releasing with both standards.
This change refactors the existing code, removes the explicit Amazon
Linux checking, and accepts the os-release standard. Amazon Linux then
"just falls out".