Oliver Kurth [Tue, 19 Feb 2019 20:51:31 +0000 (12:51 -0800)]
Include vmware/tools/log.h to define g_info.
A recent change added a call to g_info from syncManifest.c. This
in turn is causing open-vm-tools builds to fail because g_info is
not available on SLES 12sp1.
To fix the problem, include vmware/tools/log.h in syncManifest.c.
log.h defines g_info as a macro.
Oliver Kurth [Tue, 19 Feb 2019 20:51:31 +0000 (12:51 -0800)]
Always send VMBACKUP_EVENT_GENERIC_MANIFEST during quiesced snapshots.
vSphere 6.7 added a host-side interface that allows VMTools to send
a "generic" backup manifest during a quiesced snapshot on Linux guests.
VMTools 10.2.0 or later tries to notify the host of the backup manifest
file through a vmbackup event message VMBACKUP_EVENT_GENERIC_MANIFEST.
If the host is unable to field the message, then VMTools logs the
failure and then continues with the quiesced snapshot in the older
fashion, without the backup manifest.
An earlier change attempted to reduce the amount of logging done when
running on older hosts that don't support VMBACKUP_EVENT_GENERIC_MANIFEST
by detecting when sending VMBACKUP_EVENT_GENERIC_MANIFEST fails and
not sending the message again for subsequent quiesced snapshots.
However, subsequent stress testing has uncovered problems with this
approach when running on newer hosts; specifically, errors may sometimes
be encountered on newer hosts when sending VMBACKUP_EVENT_GENERIC_MANIFEST.
Therefore this change backs out that earlier change.
Note that the need to solve the problem that that earlier change was
intended to solve has been reduced because support for
VMBACKUP_EVENT_GENERIC_MANIFEST has been backported to vSphere 6.5
P03, which is available, and vSphere 6.0 P08, which is scheduled for
release later this year. ESXi 5.5 is out of general support.
This change also addresses an issue that surfaced when testing on a
host without support for VMBACKUP_EVENT_GENERIC_MANIFEST.
If VMTools fails to send VMBACKUP_EVENT_GENERIC_MANIFEST, the quiesced
snapshot operation will be aborted rather than continuing as it should.
To address this, create a new function, VmBackup_SendEventNoAbort,
which does not abort the quiesced snapshot on failure, and call that
function rather than VmBackup_SendEvent when sending
VMBACKUP_EVENT_GENERIC_MANIFEST.
Oliver Kurth [Tue, 19 Feb 2019 20:51:31 +0000 (12:51 -0800)]
Linux DeployPkg should provide a configurable timeout for the spanning
customization process. Right now it is hard-coded to 100 seconds.
Sources to provide "timeout" value:
1. Clients such as vCenter and SRM can pack timeout value in cab header
via API func "DeployPkg_SetProcessTimeout(uint16 timeout)". this timeout
value will be implemented during the package deployment process.
2. Package deployment engines:
- tools deployPkg plugin:
Add "process-timeout" in tools.conf. The tools deployPkg plugin will use
that value to control the time period of the package deployment.
- linuxDeployPkg:
Add "-t <timeout>" as optional argument in linuxDeployPkg
Usage: ./linuxDeployPkg -d <cabfile> -t <timeout> --skip-reboot
3. Default value in deployPkg is 100s. If both the client and package
deployment engines don't provid this value, then the default value of
100s will be used.
4. The "timeout" value from clients will overwrite the value from the
deployment engines when both of them provid this value.
Oliver Kurth [Tue, 19 Feb 2019 20:51:30 +0000 (12:51 -0800)]
Use the LSB and os-release data to find the best guest identification
Historically, we have the LSB to describe a Linux distro. More
recently, the os-release standard was added. Change our guest
identification code to attempt to use both data - when available - and
make a value judgement as to which is better.
Oliver Kurth [Tue, 19 Feb 2019 20:51:30 +0000 (12:51 -0800)]
[Part 2] GuestLib support for 64bit memory shares.
MEMSCHED_SHARES_MAX is currently set to MAX_PPN and will not fit in
32-bit soon. 'memshares' property which is currently available in GuestSDK
is an unsigned 32bit value. Starting from ESX 7.0, this value is available
only if it can be fit in an unsigned 32-bit data type.
'memShares64' is the new unsigned 64-bit property added in GuestSDK for the
memory shares.
Added necessary APIs to retrieve the memshares64 value. The new API
VMGuestLib_GetMemShares64 will first try to retrieve the memShares64 property.
If the underlying host is an older one, then the API will return the memShares
property.
Updated the vmGuestLibTest code to retrieve and print the new memShares64 value.
Oliver Kurth [Tue, 19 Feb 2019 20:51:30 +0000 (12:51 -0800)]
Update seamless window's copyright year as 1998-2019
1. Changing the COPYRIGHT_YEARS in vm_legal.h to 1998-2019
2. COPYRIGHT_YEARS is used by UTF8_COPYRIGHT_STRING_BASE and
UTF16_COPYRIGHT_STRING for win32 UTF-16 version.
Oliver Kurth [Tue, 19 Feb 2019 20:51:30 +0000 (12:51 -0800)]
[Part 1] GuestLib support for 64bit memory shares.
MEMSCHED_SHARES_MAX is currently set to MAX_PPN and will not fit in 32-bit soon.
Changed the datatype of 'memshares' property to uint64 for the future support.
'memshares' is exposed in GuestLib as unsigned 32-bit value.
Modified the ESX code to set this property only if the vaulue can be fit in
a uint32 type. Else, the value is not set and 'valid' bit set is set to FALSE.
Added a new property "memShares64" (as uint64) in GuestLib for the new
64-bit memory shares.
This CLN implements only the necessary changes on ESX side.
Oliver Kurth [Tue, 19 Feb 2019 20:51:29 +0000 (12:51 -0800)]
Security issue with the intermediate staging directory used for DnD and CnP
Problem:
/tmp/VMwareDnD is a staging directory used for DnD and CnP. It should be
a regular directory, but malicious code or user may create the /tmp/VMwareDnD
as a symbolic link which points to the home directory of the current desktop
user before the staging directory is created. Later when the desktop user
initiates a DnD or CnP operation, the permission of that user's home directory
will be changed allowing the malicious user full access to the desktop user's
home directory.
Solution:
Do not set the permission of the staging directory if that directory already
exists and has the wrong permission. The permission of the directory will
be 1777 if it is created by VMTools. If that is not the case, the directory
has been created or modified by malicious code or user; just cancel the host
to guest DnD or CnP operation.
Oliver Kurth [Wed, 30 Jan 2019 01:24:44 +0000 (17:24 -0800)]
Fix CentOS 7.6 detection
The version information in /etc/centos-release has 3 parts,
in particular for CentOS 7.6:
CentOS Linux release 7.6.1810 (Core)
This was misidentified as CentOS 6 because the substring "6."
was matched before matching "7.". This change fixes this by
requiring a space before the major version.
Oliver Kurth [Wed, 30 Jan 2019 01:24:44 +0000 (17:24 -0800)]
Include vmware/tools/log.h to define g_info.
A recent change added a call to g_info from syncManifest.c. This
in turn is causing open-vm-tools builds to fail because g_info is
not available on SLES 12sp1.
To fix the problem, include vmware/tools/log.h in syncManifest.c.
log.h defines g_info as a macro.
Oliver Kurth [Wed, 30 Jan 2019 01:24:44 +0000 (17:24 -0800)]
Always send VMBACKUP_EVENT_GENERIC_MANIFEST during quiesced snapshots.
vSphere 6.7 added a host-side interface that allows VMTools to send
a "generic" backup manifest during a quiesced snapshot on Linux guests.
VMTools 10.2.0 or later tries to notify the host of the backup manifest
file through a vmbackup event message VMBACKUP_EVENT_GENERIC_MANIFEST.
If the host is unable to field the message, then VMTools logs the
failure and then continues with the quiesced snapshot in the older
fashion, without the backup manifest.
An earlier change attempted to reduce the amount of logging done when
running on older hosts that don't support VMBACKUP_EVENT_GENERIC_MANIFEST
by detecting when sending VMBACKUP_EVENT_GENERIC_MANIFEST fails and
not sending the message again for subsequent quiesced snapshots.
However, subsequent stress testing has uncovered problems with this
approach when running on newer hosts; specifically, errors may sometimes
be encountered on newer hosts when sending VMBACKUP_EVENT_GENERIC_MANIFEST.
Therefore this change backs out that earlier change.
Note that the need to solve the problem that that earlier change was
intended to solve has been reduced because support for
VMBACKUP_EVENT_GENERIC_MANIFEST has been backported to vSphere 6.5
P03, which is available, and vSphere 6.0 P08, which is scheduled for
release later this year. ESXi 5.5 is out of general support.
This change also addresses an issue that surfaced when testing on a
host without support for VMBACKUP_EVENT_GENERIC_MANIFEST.
If VMTools fails to send VMBACKUP_EVENT_GENERIC_MANIFEST, the quiesced
snapshot operation will be aborted rather than continuing as it should.
To address this, create a new function, VmBackup_SendEventNoAbort,
which does not abort the quiesced snapshot on failure, and call that
function rather than VmBackup_SendEvent when sending
VMBACKUP_EVENT_GENERIC_MANIFEST.
Oliver Kurth [Tue, 29 Jan 2019 22:03:19 +0000 (14:03 -0800)]
Fix possible security issue with the permissions of the intermediate
staging directory and path
/tmp/VMwareDnD is a staging directory used for DnD and CnP. It should be
a regular directory, but malicious code or user may create the /tmp/VMwareDnD
as a symbolic link which points elsewhere on the system. This may provide
user access to user B's files.
Do not set the permission of the root directory if the root directory
already exists and has the wrong permission. The permission of the directory
must be 1777 if it is created by the VMToolsi. If not, then the directory
has been created or modified by malicious code or user, so just cancel the
host to guest DnD or CnP operation.
Oliver Kurth [Tue, 29 Jan 2019 22:03:19 +0000 (14:03 -0800)]
Bail out vmtoolsd early when there are RPC errors.
VMX state machine could give up quiescing operation for various
reasons when vmtoolsd is busy performing necessary state transitions.
Once VMX gives up quiescing operation, there is no point in
vmtoolsd continuing with it. vmtoolsd should also give up the
operation asap. vmtoolsd can detect VMX state machine change
when it gets errors sending VMBACKUP_PROTOCOL_EVENT_SET RPC.
RPC errors are only used as a trigger to abort the operation.
We ignore the RPC errors that might occur after aborting the
operation.
Oliver Kurth [Tue, 29 Jan 2019 22:03:19 +0000 (14:03 -0800)]
Don't send a backup manifest when aborting a Linux quiesced snapshot.
When taking a Linux quiesced snapshot, communication failures between
VMX and VMTools may result in VMTools sending a genericManifest event
message after the quiesced snapshot operation has been aborted. If
this happens, VMX will send an error back to VMTools, which in turn
causes VMTools not to send genericManifest messages on subsequent
quiesced snapshots even if the host supports such messages.
One aspect of the implementation that gives rise to this behavior is
the use of the sync provider's snapshotDone function to undo a
quiescing operation. Specifically, if VMTools aborts a quiesced
snapshot when the file system is quiesced, the quiescing must be
undone. Currently, this is handled by calling the sync provider's
snapshotDone function. This is the same function that is called to
complete the quiescing snapshot protocol when it is successful. In
some respects this makes sense, since in either case snapshotDone
unquiesces the file system. However, architecturally and conceptually,
it seems useful to distinguish between the action to be taken in the
successful case versus the aborting case. It's also useful to do so
in practice, because the successful case sends the genericManifest
event to notify the host there is a backup manifest file, while the
aborting case should not do that.
To address the issue, add an "undo" function for the Linux sync
provider. The undo function is called instead of snapshotDone as
part of aborting a quiesced snapshot in which the file system is
quiesced at the time of the abort.
Oliver Kurth [Tue, 29 Jan 2019 22:03:18 +0000 (14:03 -0800)]
General code clean-up.
bora/lib/misc/posixPosix.c:
Treat local variables "len" consistently as "size_t" type in
Posix_Getmntent_r()
bora-vmsoft/apps/toolbox-cmd/toolboxcmd-shrink.c:
Improve readability of error handling logic in ShrinkDoWipeAndShrink()
and remove another line of dead code.
bora-vmsoft/lib/procMgr/procMgrPosix.c:
Recent discussions about setting "errno" to ENOENT when either
getpwuid_r() or getpwnam_r() return 0 (success) if there is no passwd
entry for the user have vacillated.
Since none of the current callers of the routines in procMgrPosix.c
query the contents of "errno", the current consensus is to simply log
a warning message along with reason for the failure.
Oliver Kurth [Tue, 29 Jan 2019 22:03:18 +0000 (14:03 -0800)]
Fix NULL pointer dereference and remove three lines of dead code.
QbitLogic has run [Code]Ai on the open-vm-tools source code and
and notified VMware of a potential NULL pointer reference and four
pieces of dead code.
See https://github.com/vmware/open-vm-tools/pull/247
bora-vmsoft/services/vmtoolsd/pluginMgr.c
The proposed fix conditionally emitted a log message. That fix has
been reworked to consistently emit a log message with conditional
content.
bora/lib/misc/posixPosix.c
bora-vmsoft/apps/toolbox-cmd/toolboxcmd-shrink.c
Remove one line of dead code from each source file.
bora-vmsoft/lib/procMgr/procMgrPosix.c:
Suggested dead code fix actually pointed to a bug where errno
was not getting updated as intended.
bora-vmsoft/services/plugins/guestInfo/perfMonLinux.c:
Suggested dead code is no longer dead; change not applicable.
Oliver Kurth [Wed, 16 Jan 2019 22:53:06 +0000 (14:53 -0800)]
Add WarningToHost() and WarningToGuest() functions
This change is needed to address the privacy and security changes that
are required so that vmware library warnings can be forwarded to the
host side selectively.
For instance, if a warning message is sent to VMX, the user name must be
stripped from the message.
Oliver Kurth [Wed, 16 Jan 2019 22:53:05 +0000 (14:53 -0800)]
Bail out of vmtoolsd early when there are RPC errors.
VMX state machine could give up quiescing operation for various
reasons when vmtoolsd is busy performing necessary state transitions.
Once VMX gives up quiescing operation, there is no point in
vmtoolsd continuing with it. vmtoolsd should also give up the
operation asap. The vmtoolsd process can detect VMX state machine
change when it gets errors sending VMBACKUP_PROTOCOL_EVENT_SET RPC.
RPC errors are only used as a trigger to abort the operation.
We ignore the RPC errors that might occur after aborting the
operation.
Oliver Kurth [Wed, 16 Jan 2019 22:53:05 +0000 (14:53 -0800)]
Don't send a backup manifest when aborting a Linux quiesced snapshot.
When taking a Linux quiesced snapshot, communication failures between
VMX and VMTools may result in VMTools sending a genericManifest event
message after the quiesced snapshot operation has been aborted. If
this happens, VMX will send an error back to VMTools, which in turn
causes VMTools not to send genericManifest messages on subsequent
quiesced snapshots even if the host supports such messages.
One aspect of the implementation that gives rise to this behavior is
the use of the sync provider's snapshotDone function to undo a
quiescing operation. Specifically, if VMTools aborts a quiesced
snapshot when the file system is quiesced, the quiescing must be
undone. Currently, this is handled by calling the sync provider's
snapshotDone function. This is the same function that is called to
complete the quiescing snapshot protocol when it is successful. In
some respects this makes sense, since in either case snapshotDone
unquiesces the file system. However, architecturally and conceptually,
it seems useful to distinguish between the action to be taken in the
successful case versus the aborting case. It's also useful to do so
in practice, because the successful case sends the genericManifest
event to notify the host there is a backup manifest file, while the
aborting case should not do that.
To address the issue, add an "undo" function for the Linux sync
provider. The undo function is called instead of snapshotDone as
part of aborting a quiesced snapshot in which the file system is
quiesced at the time of the abort.
Recent discussions about setting "errno" to ENOENT when either
getpwuid_r() or getpwnam_r() return 0 (success) if there is no passwd
entry for the user have vacillated.
Since none of the current callers of the routines in procMgrPosix.c
query the contents of "errno", the current consensus is to simply log
a warning message along with reason for the failure.
Oliver Kurth [Tue, 18 Dec 2018 21:19:48 +0000 (13:19 -0800)]
General code clean-up.
bora/lib/misc/posixPosix.c:
Treat local variables "len" consistently as "size_t" type in
Posix_Getmntent_r()
bora-vmsoft/apps/toolbox-cmd/toolboxcmd-shrink.c:
Improve readability of error handling logic in ShrinkDoWipeAndShrink()
and remove another line of dead code.
Both gcc and clang (on Mac) warn for pointer-incompatible
assignments. It appears that gcc's -Wwarn-strings isn't working
properly, because it produces no warnings for the following
assignment:
char *potentialSegV = ".rodata-based string";
However, clang does.
This change addresses three problems in bora/lib that produce build
failures when -Wincompatible-pointer-types-discards-qualifiers is
enabled when using clang (one wonders how many other such potential
problems exist?)
o bsd_vsnprintf.c
The 'cp' pointer is assigned the address of a constant string.
Since there are writes through 'cp', the quickest solution is to
declare a local, non-const, array variable that holds the string
"(null)".
o unicodeSimpleTypes.c
This change, the easiest of the three, marks the 'char *' field as
'const'.
Oliver Kurth [Tue, 18 Dec 2018 21:19:47 +0000 (13:19 -0800)]
Fix NULL pointer dereference and remove three lines of dead code.
QbitLogic has run [Code]Ai on the open-vm-tools source code and
and notified VMware of a potential NULL pointer reference and four
pieces of dead code.
See https://github.com/vmware/open-vm-tools/pull/247
bora-vmsoft/services/vmtoolsd/pluginMgr.c
The proposed fix conditionally emitted a log message. That fix has
been reworked to consistently emit a log message with conditional
content.
bora/lib/misc/posixPosix.c
bora-vmsoft/apps/toolbox-cmd/toolboxcmd-shrink.c
Remove one line of dead code from each source file.
bora-vmsoft/lib/procMgr/procMgrPosix.c:
Suggested dead code fix actually pointed to a bug where errno
was not getting updated as intended.
bora-vmsoft/services/plugins/guestInfo/perfMonLinux.c:
Suggested dead code is no longer dead; change not applicable.
Oliver Kurth [Tue, 18 Dec 2018 21:19:46 +0000 (13:19 -0800)]
lib/file: Improve File_ListDirectory and File_WalkDirectory*
The File_WalkDirectory implementation has an initial latency (before
file names are available) that is unnecessary. Using the unicode
library, when not necessary, adds a huge amount of memory usage and
wastes CPU time.
These routines duplicate quite a bit of code.
Rewrite File_ListDirectory to use File_WalkDirectory*.
Rewrite File_WalkDirectory*:
- Return files as they are discovered, not after the entire directory
contents are parsed. If one decides to stop early, they don't have
to pay the price for the entire directory contents.
- Preserve the protection from duplicate file names. While Windows
locks a directory during a content tranverse, POSIXen does nott. Code
is here in the lib/file for quite some time so that the numerous
callers do not have to handle dealing with dups.
- Avoid using the unicode library on platforms where we can, this avoid
most of the eggregious memory usage.
- Don't use Dynbuf... Handling things directly is more efficient.