Peter Krempa [Wed, 24 Feb 2021 15:40:45 +0000 (16:40 +0100)]
tests: qemucapabilities: Update qemu caps for object-add qapification
qemu qapified object-add, which means that it's introspectable via
query-qmp-schema. Update the qemu-6.0 capabilities to commit v5.2.0-3205-g92566947b3
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Peter Krempa [Fri, 12 Mar 2021 14:44:19 +0000 (15:44 +0100)]
qemu: command: Use JSON for QAPIfied -object directly
Skip the lossy conversion to legacy commandline arguments by using the
JSON props directly when -object is QAPIfied. This avoids issues with
conversion of bitmaps and also allows validation of the generated JSON
against the QMP schema in the tests.
Since the new approach is triggered by a qemu capability the code
from 'virQEMUBuildObjectCommandlineFromJSON' in util/virqemu.c was moved
to 'qemuBuildObjectCommandlineFromJSON' in qemu/qemu_command.c which has
the virQEMUCaps type.
Some functions needed to be modified to propagate qemuCaps.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Peter Krempa [Mon, 30 Nov 2020 16:08:46 +0000 (17:08 +0100)]
qemuMonitorCreateObjectPropsWrap: Open-code in qemuBuildMemoryBackendProps
There's just one caller left. Since qemuBuildMemoryBackendProps is too
complex to be modified for now, just move the adding of 'id' and 'qom'
type directly into the function.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Starting from qemu-6.0 the parameters of -object/object-add are formally
described by the QAPI schema. Additionally this changes the nesting of
the properties as the 'props' nested object will be flattened to the
parent.
We'll need to detect whether qemu switched to this new approach to
generate the objects with proper nesting and also allow testing.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Tue, 23 Mar 2021 10:47:39 +0000 (11:47 +0100)]
domain_conf: Separate virDomainOS clear into a function
The virDomainDefFree() function frees individual members of
virDomainDef struct. The function is already long enough, move
code that handles def->os member into a separate function.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Andrea Bolognani [Mon, 22 Mar 2021 11:02:23 +0000 (12:02 +0100)]
ci: Drop FreeBSD 11 build
FreeBSD 12 was released in December 2018, so according to our
platform support policy we can now drop support for the previous
major release. It would be going EOL in September anyway.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Jiri Denemark [Fri, 19 Mar 2021 21:42:25 +0000 (22:42 +0100)]
qemu: Update asyncOwnerAPI when entering async job phase
In case an async job spans multiple APIs (e.g., incoming migration) the
API that started the job is recorded as the asyncOwnerAPI even though it
is no longer running and the owner thread is updated properly to the one
currently handling the job. Let's also update asyncOwnerAPI to make it
more obvious which is the current (or the most recent) API involved in
the job.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Attempting to set the memlock limit might fail if we're running
in a containerized environment where CAP_SYS_RESOURCE is not
available, and if the limit is already high enough there's no
point in trying to raise it anyway.
Now that we've implemented a fallback for the function that
obtains the information from /proc, there is no reason we would
get a failure unless there's something seriously wrong with the
environment we're running in, in which case we're better off
reporting the issue to the user rather than pretending
everything is fine.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Calling prlimit() requires elevated privileges, specifically
CAP_SYS_RESOURCE, and getrlimit() only works for the current
process which is too limiting for our needs; /proc/$pid/limits,
on the other hand, can be read by any process, so implement
parsing that file as a fallback for when prlimit() fails.
This is useful in containerized environments.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Andrea Bolognani [Fri, 19 Mar 2021 13:24:19 +0000 (14:24 +0100)]
syntax-check: Run flake8 on all Python scripts
Currenty we only check files that end in .py, but we have at
least a couple of scripts that don't have that suffix and we
nonetheless want to keep compliant with the code style.
Extend the sc_flake8 syntax-check rule so that any file that
contains a Python 3 shebang is fed to flake8 too.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Andrea Bolognani [Fri, 19 Mar 2021 13:27:28 +0000 (14:27 +0100)]
gitignore: Ignore __pycache__ directory
Unfortunately running Python scripts causes this directory to
be created in the *source* tree, and there doesn't seem to be
a way to prevent that from happening.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Andrea Bolognani [Fri, 19 Mar 2021 15:04:37 +0000 (16:04 +0100)]
ci: Drop prefix from Dockerfiles
Since the string "ci" is already contained in the path, it
seems unnecessary to include it into the filename too: in fact,
we only do that for Dockerfiles and not for files in ci/cirrus,
even though those are generated the very same way.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Implement "<os firmware='efi'>" support for bhyve driver.
As there are not really lot of options, try to find
"BHYVE_UEFI.fd" firmware which is installed by the
sysutils/uefi-edk2-bhyve FreeBSD port.
If not found, just use the first found firmware
in the firmwares directory (which is configurable via
config file).
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Peter Krempa [Wed, 17 Mar 2021 15:30:31 +0000 (16:30 +0100)]
qemu: backup: Restore security label on backup disk store image on VM termination
When the backup job is terminated normally the security label is
restored by the blockjob finishing handler.
If the VM dies or is destroyed that wouldn't happen as the blockjob
handler wouldn't be called.
Restore the security label on disk store where we remember that the job
was running at the point when 'qemuBackupJobTerminate' was called.
Not resetting the security label means that we also leak the xattr
attributes remembering the label which prevents any further use of the
file, which is a problem for block devices.
This also requires that the call to 'qemuBackupJobTerminate' from
'qemuProcessStop' happens only after 'vm->pid' was reset as otherwise
the security subdrivers attempt to enter the process namespace which
fails if the process isn't running any more.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1939082 Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Peter Krempa [Thu, 18 Mar 2021 14:11:55 +0000 (15:11 +0100)]
qemu: backup: Enable full backup support
qemuBackupBegin can take a full backup of the disks (excluding any
operations with bitmaps) without the need to wait for the
blockdev-reopen support in qemu.
Add a check that no checkpoint creation is required and the disk backup
mode isn't VIR_DOMAIN_BACKUP_DISK_BACKUP_MODE_INCREMENTAL.
Call to virDomainBackupAlignDisks is moved earlier as it initializes the
disk backup mode if not present in user config.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Erik Skultety [Mon, 15 Mar 2021 14:42:13 +0000 (15:42 +0100)]
ci: util: Add a registry checker for stale images
This function checks whether there are any stale Docker images in the
registry that can be purged. Since we're pulling available container
images from our GitLab registry with the 'list-images' action, it
could happen that we'd list old (already unsupported) images and make
them available for the user to consume and run a build in them.
Naturally, the build will most likely fail leaving the user confused.
Signed-off-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Erik Skultety [Tue, 16 Mar 2021 08:47:23 +0000 (09:47 +0100)]
ci: helper: Rewrite image listing to Python
The corresponding Bash script is dropped.
After this patch's rewrite, the Makefile's original image listing
target remains intact only to notify the user to use the Python helper
instead.
Signed-off-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Pavel Hrdina [Tue, 16 Mar 2021 12:07:26 +0000 (13:07 +0100)]
conf: introduce support for firmware auto-selection feature filtering
When the firmware auto-selection was introduced it always picked first
usable firmware based on the JSON descriptions on the host. It is
possible to add/remove/change the JSON files but it will always be for
the whole host.
This patch introduces support for configuring the auto-selection per VM
by adding users an option to limit what features they would like to have
available in the firmware.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Pavel Hrdina [Tue, 16 Mar 2021 12:05:08 +0000 (13:05 +0100)]
docs: improve description of secure attribute for loader element
The original text was not explaining what this attribute actually
controls and could have been interpreted as a control switch for the
Secure boot feature in firmwares.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com> Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This is because the initial population of the device list is done in a
separate initialization thread. If we attempt to exit libvirtd before
this init thread has completed, we'll try to free the device list while
accessing it from the other thread. In order to guarantee that this
init thread is not accessing the device list when we're cleaning up the
nodedev driver, make it joinable and wait for it to finish before
proceding with the cleanup. This is similar to how we handle the udev
event handler thread.
The separate initialization thread was added in commit 9f0ae0b1.
virxml: Fix possible memory leak in virXMLNodeContentString()
Previously, if xml node passed to the virXMLNodeContentString()
was not of type XML_ELEMENT_NODE, @ret could have caused a memory
leak because xmlNodeGetContent() works for other types of nodes
as well.
Signed-off-by: Kristina Hanicova <khanicov@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Hao Wang [Thu, 18 Mar 2021 09:00:29 +0000 (17:00 +0800)]
NEWS: Document domain memory dirty rate APIs
Signed-off-by: Hao Wang <wanghao232@huawei.com> Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We don't need to go to the trouble of telling users about existance of
insecure SASL mechanisms only to then say that they shouldn't be used.
We should only tell people about the GSSAPI mechanism for TCP sockets.
For the SCRAM mechanism we should be telling people about the SHA256
variant only, and also warning that the password database stores the
passwords in clear text.
Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
util: tell users that memory locking ulimit is too low for BPF
If running libvirtd via systemd, it gets a 64 MB memlock limit, but if
running from the shell it will only get 64 KB on a Fedora 33 system.
The latter low limit causes any attempt to use BPF to fail and it is
not obvious why.
This improves the error message thus:
# virsh -c lxc:/// start sh
error: Failed to start domain 'sh'
error: internal error: guest failed to start: Failure in libvirt_lxc startup: failed to initialize device BPF map; locked memory limit for libvirtd probably needs to be raised: Operation not permitted
Reviewed-by: Pavel Hrdina <phrdina@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
src: ensure GSource background unref happens in correct event loop
The g_idle_add function adds a callback to the primary GMainContext.
To workaround the GSource unref bugs, we need to add our callbacks
to the GMainContext that is associated with the GSource being
unref'd. Thus code using the per-VM virEventThread must use its
private GMainContext.
Reviewed-by: Martin Kletzander <mkletzan@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Peter Krempa [Tue, 16 Mar 2021 09:19:45 +0000 (10:19 +0100)]
docs/compiling: Add note on how to check minimum meson version
Give guidance on how to check minimum meson version for a given package.
Resolves: https://gitlab.com/libvirt/libvirt/-/issues/140 Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Michal Privoznik [Mon, 15 Mar 2021 14:47:04 +0000 (15:47 +0100)]
qemu_process: Use accessor for def->mem.total_memory
When connecting to the monitor, a timeout is calculated that is
bigger the more memory guest has (because QEMU has to allocate
and possibly zero out the memory and what not, empirically
deducted). However, when computing the timeout the @total_memory
mmember is accessed directly even though
virDomainDefGetMemoryTotal() should have been used.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Michal Privoznik [Mon, 15 Mar 2021 17:22:22 +0000 (18:22 +0100)]
meson: Don't check for addr2line
In the past, we used to have this oomtrace.pl script that
attempted to print the stack trace of where an OOM error
occurred and it used addr2line for that. But since v5.8.0-rc1~189
we don't really care about OOM anymore and the script is long
gone so there's no need to check for addr2line program either.
Fixes: 2c52ecd96086b4643b99b4570b5823d40ce2787b Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Andrea Bolognani [Fri, 12 Mar 2021 16:34:23 +0000 (17:34 +0100)]
ci: Discourage users from using the Makefile directly
We now wrap all its important functionality with the much more
user-friendly ci/helper script, and the long term plan is for
the Makefile to disappear completely.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Andrea Bolognani [Tue, 16 Feb 2021 16:21:49 +0000 (17:21 +0100)]
ci: Add helper script
This is intended to be perform a number of CI-related operations
that are currently implemented in various different scripts
written in various different programming languages.
Eventually, all existing functionality will be reimplemented in
Python and made available through this single entry point; for
now, let's start with a very basic skeleton.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Erik Skultety [Fri, 12 Mar 2021 10:53:13 +0000 (11:53 +0100)]
docs: html.in: Convert api to rst
There were a number of occurrences where we used nested inline markup
(verbatim + refs) which is currently not possible with RST syntax [1].
There is a possible workaround involving substitution definitions like
Substitutions cannot be made generic, hence we cannot create a template
for substitution and use a single template everywhere, so we'd end up
with a lot of clutter and convolution. Therefore, we can make an
exception and just link the data type without further style markup.
Pavel Hrdina [Mon, 15 Mar 2021 13:10:45 +0000 (14:10 +0100)]
qemu_driver: fix setting vcpu_quota if not all vCPUs are online
When switching to g_autoptr this was incorrectly changed from
'continue;' into 'return -1;' resulting into an error when user tries
to set vcpu_quota of running VM:
error: An error occurred, but the cause is unknown
Fixes: e4a8bbfaf2b4cdd741bb441873bb730f9134b714 Signed-off-by: Pavel Hrdina <phrdina@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
Michal Privoznik [Thu, 11 Mar 2021 14:37:07 +0000 (15:37 +0100)]
lib: Put some variable declarations on individual lines
In short, virXXXPtr type is going away. With big bang. And to
help us rewrite the code with a sed script, it's better if each
variable is declared on its own line.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Michal Privoznik [Thu, 11 Mar 2021 07:29:42 +0000 (08:29 +0100)]
syntax-check: Fix and rename virSecurity rule
The aim of virSecurity rule is to discourage from using plain
virSecurityManager*() APIs within QEMU driver in favor of their
qemuSecurity*() counterparts. The reason is simple: namespaces;
virSecurityManager*() needs additional
virSecurityManagerTransactionCommit() call to enter given
namespace and do its work from there. And that's exactly what
those qemuSecurity*() wrappers do.
To help us ensure correctness (from this POV), we have a
syntax-check rule that forbids any occurrence of
"virSecurityManager" string under src/qemu/ (except for
qemu_security of course).
But with if we want to remove virSecurityManagerPtr type, then we
have to allow "virSecurityManager *". Therefore, change the rule
so that no call of a function with "virSecurityManager" prefix is
allowed. And also change the name to better reflect what is going
on.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
projects / thirdparty / libvirt.git / log
