Oliver Kurth [Mon, 23 Oct 2017 21:21:22 +0000 (14:21 -0700)]
Add a poll flag for one-shot callbacks in AsyncSocket
The original change had an issue. When the read and write callbacks
fire in parallel, both would release the reference to the SharedStream,
triggering its destructor. Later when the read callback tried to
re-register itself, it would hit a boost exception. To avoid that,
delay the reset of CallbackFD::stream to the end of Invoke so that the
SharedStream object will not be destroyed as long as the callback is
re-registered.
Oliver Kurth [Mon, 23 Oct 2017 21:21:21 +0000 (14:21 -0700)]
Fix a segfault when log file isn't set and conf is reloaded.
When vmsvc.handler=file, but vmsvc.data isn't set, vmtoolsd crashed
on reloading the config file. This was caused by using a NULL value
as an argument to a strcmp().
This change fixes this by using g_strcmp0 which handles NULL pointers
gracefully, and setting confData to a default value before calling
g_strcmp0().
This also fixes a case where a change in the log file would be ignored.
Oliver Kurth [Mon, 23 Oct 2017 21:21:21 +0000 (14:21 -0700)]
Changes to header files vmnet_defs.h and vmxnet3_defs.h needed to
integrate ENS capabilities into existing Uplink capabilities.
The changes are not directly applicable to open-vm-tools.
For the I[Vmdb]Poll support in AsyncTCPSocket, inBlockingRecv has been
used to avoid conflict with AsyncSocketDoOneMsg. There is an issue
with where inBlocking Recv is decremented, which is right after poll,
where the thread would drop the lock. The lock may also be dropped
during AsyncTCPSocketFillRecvBuffer, and for that there is a separate
inDoOneMsg flag. This change unifies the two flags and makes
inBlockingRecv cover both functions that may drop the lock. This fixes
the race that would allow AsyncTCPSocketRecvCallback to run when
AsyncTCPSocketDoOneMsg is in progress. A couple places where we
previously checked inBlockingRecv to report an error now have to also
check inRecvLoop (TRUE during AsyncTCPSocketFillRecvBuffer) -- those
checks cannot be enforced during FillRecvBuffer because the client
callback is allowed to change the callback or cancel it. The lone
usage of inDoOneMsg is switched to inBlockingRecv.
Oliver Kurth [Mon, 23 Oct 2017 21:21:20 +0000 (14:21 -0700)]
guestOS: Uniquely run time identify Photon
Currently VMware Photon 2 is identified at run time as a Linux 4.x kernel.
This is correct, but the Photon distro has an LSB compliant identification
file.
Attempt to check the LSB compliant identification file for Photon. If
present, identify the guest as "vmware-photon".
Oliver Kurth [Mon, 23 Oct 2017 21:21:20 +0000 (14:21 -0700)]
Fix ASSERT of registering the same callback twice
The previous change for Poll introduced a bug in AsyncSocket.
When re-registering the recv callback, it needs to check whether
there is a concurrent blocking recv (which means the send callback
has detected a disconnection and is draining the pipe). The recv
callback should act like a no-op in that case (other than to
release the reference).
Oliver Kurth [Mon, 23 Oct 2017 21:21:19 +0000 (14:21 -0700)]
Hgfs Protocol: clean up the Hgfs protocol operation capabilities
This change cleans up the operation capability and its flags which
currently are not clearly associated with the type to which they refer.
The name doesn't help in that regard either.
The capability and flags are used in the Hgfs session creation by
both the server and client to dictate the protocol operation versions
to be used during the lifetime of that session. The flags denote
which operations are currently used and how they are expected to be
used.
The hosted file locking code tosses a random number generator and
creates an directory name (D name) and member name (M name) that both
use the random number.
If the D name cannot be created we try again until we can create one.
The assumption is other lockers are racing with the code. The random
number generator makes the chance of collisons small and soon we get
a D name.
Once a D name can be created, we check if the M name exists. If it
does, we remove the D name and try again until we get a unique D and
M name. Once the M name is created, the D name is discarded.
If we're unable to remove the D name, we could land up filling the
locking directory with garbage that cannot be cleaned up. Fail if
we're unable to remove the D entry.
Oliver Kurth [Mon, 23 Oct 2017 21:21:19 +0000 (14:21 -0700)]
Disable lib/string wprintf functions for non-Windows builds
The lib/string wprintf functions currently are enabled only
for Windows and Linux builds, but Windows is the only
platform that uses them. Disable them for Linux, too.
Oliver Kurth [Mon, 23 Oct 2017 21:21:18 +0000 (14:21 -0700)]
Remove Str_Mbscpy and Str_Mbscat
Since we assume that all non-wide-char strings are UTF-8, the
implementations of Str_Mbscpy and Str_Mbscat are basically
identical to those of Str_Strcpy and Str_Strcat. We thus do
not need the Str_Mbs versions.
Oliver Kurth [Mon, 23 Oct 2017 21:21:18 +0000 (14:21 -0700)]
lib/file: FileIO_AtomicUpdateEx ASSERT
ASSERT(File_IsSameFile(dirName, dstDirName)) fired when NFS went
down at just the right moment. It was not obvious that the ASSERT
fired due to an NFS failure.
File_IsSameFile correctly returns errno upon failure. Check the
errno value to properly detect and return an error.
Oliver Kurth [Fri, 15 Sep 2017 18:23:57 +0000 (11:23 -0700)]
Fix processing RPC data in TimeSync RPC handler
RPC data is whitespace delimited string and any leading whitespaces need to
be ignoreed before processing data. The data here indicates whether timeSync
should do a backward sync on this RPC callback.
Oliver Kurth [Fri, 15 Sep 2017 18:23:56 +0000 (11:23 -0700)]
Add support for 64-bit inodes in FreeBSD 12
FreeBSD 12 is switching to 64-bit inodes which will cause some ABI
breakage in user code. This fix modifies the getdirentries() syscall
used in hgfsServerLinux.c for that FBSD 12 64-bit inodes.
The fix was provided by Josh Paetzel in open-vm-tools pull request
https://github.com/vmware/open-vm-tools/pull/190
Oliver Kurth [Fri, 15 Sep 2017 18:23:56 +0000 (11:23 -0700)]
Add support to check if a native guest time sync service is running.
This change adds support to VMware tools timesync to identify if a guest
native time synchronization service is running. If a native sync service is
running, then a resync command is issued. If native time sync service is not
running, we fall back on legacy VMware tools onetime sync to do guest time
correction.
We also post a callback to do legacy VMware tools onetime sync after a
timeout period. A timeout can be configured if guest native resync command
fails to sync guest time.
Oliver Kurth [Fri, 15 Sep 2017 18:23:56 +0000 (11:23 -0700)]
lib/file: Continue simplifying and optimizing
Since we're UTF8 only, there are simplifications and
optimizations that can be done to lib/file, particularly in the
routines that look for directory separators and ASCII characters like
'.'. We don't need to use the unicode library to deal with these
type of code points.
This is the second in a sequence of changes to improve lib/file.
Oliver Kurth [Fri, 15 Sep 2017 18:23:56 +0000 (11:23 -0700)]
Fix lib/string unit tests
Updates to lib/string.str.c and its unit tests.
* On Linux, Str_Vsnwprintf uses vswprintf from glibc[1].
However, glibc's vswprintf implementation appears to be
broken by not NUL-terminating the destination if truncation
occurs. Make Str_Vsnwprintf explicitly NUL-terminate along
this path.
* The unit tests expected that Str_Vsnwprintf use %s for
wchar_t* arguments (and %S for char* ones). That is a
Microsoft-ism (necessary for TCHAR-based printf functions
to use the same format strings with or without _UNICODE).
Our bsd_vsnwprintf function instead uses %s for char* (and
%S for wchar_t*). Existing callers to Str_...wprintf use
use the BSD implementation and expect the BSD behavior
(which is consistent with standard C in this regard), so
go with the existing behavior and adjust the tests.
* Some of the tests passed int arguments for floating-point
format specifiers. This is undefined behavior.
* Some of the tests had more format specifiers than
arguments. More undefined behavior.
* Finally, the existing Str_...printf tests are
fundamentally flawed: they compare output against the
system's printf functions. However, the whole point of
having our own is to have consistent behavior across
platforms and to work around bugs; the output therefore
cannot be expected to consistently match. Instead rewrite
the tests to compare Str_...printf output against fixed
strings.[2] Additionally, enable these tests only if
HAS_BSD_PRINTF/HAS_BSD_WPRINTF are enabled; I see no point
in testing the system versions (which won't have consistent
output), and IMO the BSD implementations are the only ones
we should be supporting.
[1] I do not understand why HAS_BSD_WPRINTF is enabled on
Linux only for < gcc 2.96 (and not at all for macOS).
This probably should be fixed, but not as part of this
change. (I also don't know why HAS_BSD_WPRINTF is
distinct from HAS_BSD_PRINTF.)
[2] bsd_vsnwprintf is not consistent with bsd_vsnprintf for
all format specifiers. I plan to deal with this later,
but for now, make the expected strings match the actual
output so we have a starting point where everything
passes.
Oliver Kurth [Fri, 15 Sep 2017 18:23:55 +0000 (11:23 -0700)]
lib/file: File_ReplaceExtension fails in the root directory
On Linux, File_ReplaceExtension returns an incorrect path name for a
file in the root directory. The code, as written, doesn't differentiate
"test.txt" from "/test.txt". Fix this.
Oliver Kurth [Fri, 15 Sep 2017 18:23:55 +0000 (11:23 -0700)]
Update __IS_FREE_BSD__ macro in vm_basic_defs.h
The __IS_FREEBSD__ macro in vm_basic_defs.h uses "defined" in
the definition. That is undefined behavior and the FreeBSD default
compiler, clang (LLVM), warns about this pervasively misused
construct. With all warnings equivalent to an error, builds fail.
The object-like macro __IS_FREEBSD__ has been rewritten in a manner
acceptable to the current C standard.
Oliver Kurth [Fri, 15 Sep 2017 18:23:54 +0000 (11:23 -0700)]
Make TimeUtil_GetTimeFormat avoid a null pointer dereference if ctime_r fails
ctime_r potentially could return NULL on failure. Instead of
blindly dereferencing the result, TimeUtil_GetTimeFormat
should check for that and also return NULL (which callers
should already expect on failure).
Oliver Kurth [Fri, 15 Sep 2017 18:23:53 +0000 (11:23 -0700)]
Add vmx config options for guest time service resync.
VMware tools timeSync can be enabled in a guest that also has a native
time service running. This can lead to race conditions when both the time
services are trying to correct system time. Particularly, when both of
these services tries to do a step correction on certain events, guest
time behaves in an undesirable manner that could cause guest
applications to fail. This change introduces a couple of vmx config options
that can be used to eliminate or atleast reduce the chance of race
condition between VMware tools timeSync and guest native time service
(like w32time or NTP).
The basic idea is to utilize native time service to resynchronize and
recalibrate its parameters and force to do a step correction if
necessary. VMware tools timeSync would send a service request to native
guest time service when it needs to do a step correction, rather than
correcting system time itself. This helps in having a single service
responsible for correcting guest system time and eliminates/reduces race
condition.
"time.synchronize.guest.resync"=<TRUE|FALSE> controls whether this new
method of guest time synchronization is enabled or not.
There might be situations where native guest time service resync request
is not successful. This could happen if the native time service is not
configured correctly or if there is an intermittent network outage, etc.
We would like to fall back to using legacy way of VMware tool timeSync
onetime correction in such failure scenarios.
"time.synchronize.guest.resync.timeout>=0" vmx option allows us to achieve
that. Tools timeSync would initiate a resync request, then wait for
'timeout' period of time and would fall back of legacy timeSync onetime
sync behavior. This allows us to correct guest time even in case of
native guest time service failure.
Oliver Kurth [Fri, 15 Sep 2017 18:23:52 +0000 (11:23 -0700)]
Hgfs OSX Server: Create and use server callback table for change notifications
There are now three different callbacks into the Hgfs server from the file
change notification component. So we now pass these in a table to the nofify
module when it is initialized instead of ad hoc. Previously, the same
callback is passed for every subscriber watch event and therefore duplicated
and stored in each subscriber.
Note that the feature is currently switched off by default until these changes
are complete. A user will explicitly have to enable the VMX file setting and
would cause a VMX assert to be triggered.
Details:
- The server now creates a table for its notification callbacks and passes
it to the notify initialization function.
- The server now includes the async thread registration callbacks previously
ifdef'd out.
- The Notify_Init function APIs are updated to reflect the new argument for
callbacks.
- The HgfsNotify_AddSubscriber argument does not include the callback as it
is now passed once at the initialization time.
- The HgfsNotifyEventReceiveCb type has been modified now it is an entry in
the callback table.
- The subscriber callback function call is now validated to be non-NULL
before calling.
- The Mac notify module adds the server callback table to its FS event
context which is passed to the event generating submodule for the Mac
implementation.
- The Mac notify module adds its own FS event callbacks for thread register
and unregister called from the FS event module when registering and
unregistering its FS event thread.
Oliver Kurth [Fri, 15 Sep 2017 18:23:52 +0000 (11:23 -0700)]
Re-gen rabbitproxy key if its perms are bad
Make sure nothing has played with our keyfile permissions.
The old (9.10) VB script that generated the key left it world-readable,
and it's also possible someone could mess up the perms.
This change checks the perms, and if they're wrong, regenerates
the cert & key.
Oliver Kurth [Fri, 15 Sep 2017 18:23:52 +0000 (11:23 -0700)]
GuestInfo: Add option to exclude network interfaces
VMs running docker can have hundred of virtual network interfaces.
Also, docker creates one virtual network interface 'docker0' with
an IPv4 address. This address is useless outside the VM, and should
not be propagated via GuestInfo, or even as the primary network
address.
This change adds on option for tools.conf to exclude specific
network interface names using patterns with wildcards. This makes
it easy to exclude virtual docker interface names with the option
exclude-nics=docker*,veth*
For convenience, the option is set to docker*,veth* by default,
except for Windows, where it is "vEthernet*". To
send information of all interfaces, including docker* and veth*,
the option needs to be set to an empty value: exclude-nics=
Oliver Kurth [Fri, 15 Sep 2017 18:23:51 +0000 (11:23 -0700)]
Tools Hgfs Plugin: Refresh Hgfs drives on client start
In order to get Explorer to see that VMware Shared Fodlers
drives are available when the feature is enabled, we touch
each drive that is mapped as a VMware Shared Folder. This
is to prevent users being confused when they have the feature
enabled but Explorer creates the user's drive mappings before
the tools service has initialized the Hgfs plugin and started
the Hgfs client. In this situation the drives are marked
as red (disconnected) until the user clicks on the mapped
drive when Explorer will refresh the connction and set it
to green (connected).
Details:
- Main service will usually create otheriwse open a named event
to synchronize with the user service.
- Main service will start the HGFS client redirector (as before
this is not new).
- Main service will set the synchronization event to signalled.
- User service will usually open the named event (or create it
if first).
- User service will enumerate all the mapped network drives and,
if they are provided by the VMware Shared Folder redirector,
reconnect them.
- The named event will be created with Administrators group all
access and Everyone with modify state to signal it and synchronize
to wait on it.
Oliver Kurth [Fri, 15 Sep 2017 18:23:51 +0000 (11:23 -0700)]
Emphatically close virtual disk.
Sometimes I/O stack modules return EBUSY during teardown.
This may be due to an external process opening files at
inopportune moments. The workaround is to retry the affected
operations.
Oliver Kurth [Fri, 15 Sep 2017 18:23:51 +0000 (11:23 -0700)]
lib/file/filePosix.c: Implement File_GetMountPath() for POSIX
- The change implements File_GetMountPath() for POSIX,
using realpath(3) and readlink(2).
Changes to common header files; not applicable to open-vm-tools.
Oliver Kurth [Fri, 15 Sep 2017 18:23:51 +0000 (11:23 -0700)]
VThreadBase: Switch to "stable" thread IDs
Now that lib/thread no longer has a dependency on small-numbered
VThreadIDs, and now that lib/thread has ceeded control of assigning
VThreadIDs to threads, we can make VThreadBase really simple.