Fiona Ebner [Fri, 23 May 2025 07:02:11 +0000 (09:02 +0200)]
hw/core/qdev-properties-system: Add missing return in set_drive_helper()
Currently, changing the 'drive' property of e.g. a scsi-hd object will
result in an assertion failure if the aio context of the block node
it's replaced with doesn't match the current aio context:
The problematic scenario is already detected, but a 'return' statement
was missing.
Cc: qemu-stable@nongnu.org Fixes: d1a58c176a ("qdev: allow setting drive property for realized device") Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250523070211.280498-1-f.ebner@proxmox.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Thu, 29 May 2025 20:31:47 +0000 (16:31 -0400)]
iotests: fix 240
Commit 2e8e18c2e463 ("virtio-scsi: add iothread-vq-mapping parameter")
removed the limitation that virtio-scsi devices must successfully set
the AioContext on their BlockBackends. This was made possible thanks to
the QEMU multi-queue block layer.
This change broke qemu-iotests 240, which checks that adding a
virtio-scsi device with a drive that is already in another AioContext
will fail.
Update the test to take the relaxed behavior into account. I considered
removing this test case entirely, but the code coverage still seems
valuable.
Fixes: 2e8e18c2e463 ("virtio-scsi: add iothread-vq-mapping parameter") Reported-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com>
Message-ID: <20250529203147.180338-1-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:59 +0000 (17:10 +0200)]
block/io: remove duplicate GLOBAL_STATE_CODE() in bdrv_do_drained_end()
Both commit ab61335025 ("block: drain from main loop thread in
bdrv_co_yield_to_drain()") and commit d05ab380db ("block: Mark drain
related functions GRAPH_RDLOCK") introduced a GLOBAL_STATE_CODE()
macro in bdrv_do_drained_end(). The assertion of being in the main
thread cannot change here, so keep only the earlier instance.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-23-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Andrey Drobyshev [Fri, 30 May 2025 15:10:58 +0000 (17:10 +0200)]
iotests/graph-changes-while-io: add test case with removal of lower snapshot
This case is catching potential deadlock which takes place when job-dismiss
is issued when I/O requests are processed in a separate iothread.
See https://mail.gnu.org/archive/html/qemu-devel/2025-04/msg04421.html
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
[FE: re-use top image and rename snap1->mid as suggested by Kevin Wolf
remove image file after test as suggested by Kevin Wolf
add type annotation for function argument to make mypy happy] Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-22-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:57 +0000 (17:10 +0200)]
iotests/graph-changes-while-io: remove image file after test
Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-21-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:56 +0000 (17:10 +0200)]
block: mark bdrv_drained_begin() and friends as GRAPH_UNLOCKED
All of bdrv_drain_all_begin(), bdrv_drain_all() and
bdrv_drained_begin() poll and are not allowed to be called with the
block graph lock held. Mark the function as such.
Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-20-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:55 +0000 (17:10 +0200)]
blockdev: drain while unlocked in external_snapshot_action()
This is in preparation to mark bdrv_drained_begin() as GRAPH_UNLOCKED.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-19-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:54 +0000 (17:10 +0200)]
blockdev: drain while unlocked in internal_snapshot_action()
This is in preparation to mark bdrv_drained_begin() as GRAPH_UNLOCKED.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-18-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:53 +0000 (17:10 +0200)]
block: move drain outside of quorum_del_child()
The quorum_del_child() callback runs under the graph lock, so it is
not allowed to drain. It is only called as the .bdrv_del_child()
callback, which is only called in the bdrv_del_child() function, which
also runs under the graph lock.
The bdrv_del_child() function is called by qmp_x_blockdev_change().
A drained section was already introduced there by commit "block: move
drain out of quorum_add_child()".
This finally finishes moving out the drain to places that are not
under the graph lock started in "block: move draining out of
bdrv_change_aio_context() and mark GRAPH_RDLOCK".
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-17-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:52 +0000 (17:10 +0200)]
block: move drain outside of bdrv_root_unref_child()
This is part of resolving the deadlock mentioned in commit "block:
move draining out of bdrv_change_aio_context() and mark GRAPH_RDLOCK".
bdrv_root_unref_child() is called by:
1. blk_remove_bs(), where a drained section is introduced.
2. bdrv_unref_child(), which runs under the graph lock, so the drain
will be moved further up to its callers.
3. block_job_remove_all_bdrv(), where a drained section is introduced.
For all callers of bdrv_unref_child() and its generated
bdrv_co_unref_child() coroutine variant, a drained section is
introduced, they are not explicilty listed here. The caller
quorum_del_child() holds the graph lock, so it is not actually allowed
to drain. This will be addressed in the next commit.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-16-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:51 +0000 (17:10 +0200)]
block: move drain outside of quorum_add_child()
This is part of resolving the deadlock mentioned in commit "block:
move draining out of bdrv_change_aio_context() and mark GRAPH_RDLOCK".
The quorum_add_child() callback runs under the graph lock, so it is
not allowed to drain. It is only called as the .bdrv_add_child()
callback, which is only called in the bdrv_add_child() function, which
also runs under the graph lock.
The bdrv_add_child() function is called by qmp_x_blockdev_change(),
where a drained section is introduced.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-15-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:50 +0000 (17:10 +0200)]
block: move drain outside of bdrv_attach_child()
This is part of resolving the deadlock mentioned in commit "block:
move draining out of bdrv_change_aio_context() and mark GRAPH_RDLOCK".
The function bdrv_attach_child() runs under the graph lock, so it is
not allowed to drain. It is called by:
1. replication_start()
2. quorum_add_child()
3. bdrv_open_child_common()
4. Throughout test-bdrv-graph-mod.c and test-bdrv-drain.c unit tests.
In all callers, a drained section is introduced.
The function quorum_add_child() runs under the graph lock, so it is
not actually allowed to drain. This will be addressed by the following
commit.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-14-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:49 +0000 (17:10 +0200)]
block: move drain outside of bdrv_root_attach_child()
This is part of resolving the deadlock mentioned in commit "block:
move draining out of bdrv_change_aio_context() and mark GRAPH_RDLOCK".
The function bdrv_root_attach_child() runs under the graph lock, so it
is not allowed to drain. It is called by:
1. blk_insert_bs(), where a drained section is introduced.
2. block_job_add_bdrv(), which holds the graph lock itself.
block_job_add_bdrv() is called by:
1. mirror_start_job()
2. stream_start()
3. commit_start()
4. backup_job_create()
5. block_job_create()
6. In the test_blockjob_common_drain_node() unit test
In all callers, a drained section is introduced.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-13-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:48 +0000 (17:10 +0200)]
block: move drain outside of bdrv_set_backing_hd_drained()
This is part of resolving the deadlock mentioned in commit "block:
move draining out of bdrv_change_aio_context() and mark GRAPH_RDLOCK".
The function bdrv_set_backing_hd_drained() holds the graph lock, so it
is not allowed to drain. It is called by:
1. bdrv_set_backing_hd(), where a drained section is introduced,
replacing the previously present bs-specific drains.
2. stream_prepare(), where a drained section is introduced replacing
the previously present bs-specific drains.
The drain_bs variable in bdrv_set_backing_hd_drained() is now
superfluous and thus dropped.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-12-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:47 +0000 (17:10 +0200)]
block: move drain outside of bdrv_attach_child_common(_abort)()
This is part of resolving the deadlock mentioned in commit "block:
move draining out of bdrv_change_aio_context() and mark GRAPH_RDLOCK".
The function bdrv_attach_child_common_abort() is used only as the
abort callback in bdrv_attach_child_common_drv transactions, so the
tran_finalize() calls of such transactions need to be in drained
sections too.
All code paths are covered:
The bdrv_attach_child_common_drv transactions are only used in
bdrv_attach_child_common(), so it is enough to check callers of
bdrv_attach_child_common() following the transactions.
bdrv_attach_child_common() is called by:
1. bdrv_attach_child_noperm(), which does not finalize the
transaction yet.
2. bdrv_root_attach_child(), where a drained section is introduced.
bdrv_attach_child_noperm() is called by:
1. bdrv_attach_child(), where a drained section is introduced.
2. bdrv_set_file_or_backing_noperm(), which does not finalize the
transaction yet.
3. bdrv_append(), where a drained section is introduced.
bdrv_set_file_or_backing_noperm() is called by:
1. bdrv_set_backing_hd_drained(), where a drained section is
introduced.
2. bdrv_reopen_parse_file_or_backing(), which does not finalize the
transaction yet. Draining the old child bs currently happens under
the graph lock there. This is replaced with an assertion, because
the drain will be moved further up to the caller.
bdrv_reopen_parse_file_or_backing() is called by:
1. bdrv_reopen_prepare(), which does not finalize the transaction yet.
bdrv_reopen_prepare() is called by:
1. bdrv_reopen_multiple(), which does finalize the transaction. It is
called after bdrv_reopen_queue(), which starts a drained section.
The drained section ends, when bdrv_reopen_queue_free() is called
at the end of bdrv_reopen_multiple().
This resolves all code paths.
The functions bdrv_set_backing_hd_drained(), bdrv_attach_child() and
bdrv_root_attach_child() run under the graph lock, so they are not
actually allowed to drain. This will be addressed in the following
commits.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250530151125.955508-11-f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:46 +0000 (17:10 +0200)]
block: move drain outside of bdrv_try_change_aio_context()
This is part of resolving the deadlock mentioned in commit "block:
move draining out of bdrv_change_aio_context() and mark GRAPH_RDLOCK".
Convert the function to a _locked() version that has to be called with
the graph lock held and add a convenience wrapper that has to be
called with the graph unlocked, which drains and takes the lock
itself. Since bdrv_try_change_aio_context() is global state code, the
wrapper is too.
Callers are adapted to use the appropriate variant, depending on
whether the caller already holds the lock. In the
test_set_aio_context() unit test, prior drains can be removed, because
draining already happens inside the new wrapper.
Note that bdrv_attach_child_common_abort(), bdrv_attach_child_common()
and bdrv_root_unref_child() hold the graph lock and are not actually
allowed to drain either. This will be addressed in the following
commits.
Functions like qmp_blockdev_mirror() query the nodes to act on before
draining and locking. In theory, draining could invalidate those nodes.
This kind of issue is not addressed by these commits.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-10-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:45 +0000 (17:10 +0200)]
block: move drain outside of bdrv_change_aio_context() and mark GRAPH_RDLOCK
This is in preparation to mark bdrv_drained_begin() as GRAPH_UNLOCKED.
Note that even if bdrv_drained_begin() were already marked as
GRAPH_UNLOCKED, TSA would not complain about the instance in
bdrv_change_aio_context() before this change, because it is preceded
by a bdrv_graph_rdunlock_main_loop() call. It is not correct to
release the lock here, and in case the caller holds a write lock, it
wouldn't actually release the lock.
In combination with block-stream, there is a deadlock that can happen
because of this [0]. In particular, it can happen that
main thread IO thread
1. acquires write lock
in blk_co_do_preadv_part():
2. have non-zero blk->in_flight
3. try to acquire read lock
4. begin drain
Steps 3 and 4 might be switched. Draining will poll and get stuck,
because it will see the non-zero in_flight counter. But the IO thread
will not make any progress either, because it cannot acquire the read
lock.
After this change, all paths to bdrv_change_aio_context() drain:
bdrv_change_aio_context() is called by:
1. bdrv_child_cb_change_aio_ctx() which is only called via the
change_aio_ctx() callback, see below.
2. bdrv_child_change_aio_context(), see below.
3. bdrv_try_change_aio_context(), where a drained section is
introduced.
The change_aio_ctx() callback is called by:
1. bdrv_attach_child_common_abort(), where a drained section is
introduced.
2. bdrv_attach_child_common(), where a drained section is introduced.
3. bdrv_parent_change_aio_context(), see below.
bdrv_child_change_aio_context() is called by:
1. bdrv_change_aio_context(), i.e. recursive, so being in a drained
section is invariant.
2. child_job_change_aio_ctx(), which is only called via the
change_aio_ctx() callback, see above.
bdrv_parent_change_aio_context() is called by:
1. bdrv_change_aio_context(), i.e. recursive, so being in a drained
section is invariant.
This resolves all code paths. Note that bdrv_attach_child_common()
and bdrv_attach_child_common_abort() hold the graph write lock and
callers of bdrv_try_change_aio_context() might too, so they are not
actually allowed to drain either. This will be addressed in the
following commits.
More granular draining is not trivially possible, because
bdrv_change_aio_context() can recursively call itself e.g. via
bdrv_child_change_aio_context().
Reported-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-9-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:44 +0000 (17:10 +0200)]
block: mark bdrv_child_change_aio_context() GRAPH_RDLOCK
This is a small step in preparation to mark bdrv_drained_begin() as
GRAPH_UNLOCKED. More concretely, it is in preparation to move the
drain out of bdrv_change_aio_context() and marking that function as
GRAPH_RDLOCK.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-8-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:43 +0000 (17:10 +0200)]
block: mark change_aio_ctx() callback and instances as GRAPH_RDLOCK(_PTR)
This is a small step in preparation to mark bdrv_drained_begin() as
GRAPH_UNLOCKED. More concretely, it is in preparation to move the
drain out of bdrv_change_aio_context() and marking that function as
GRAPH_RDLOCK.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-7-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:42 +0000 (17:10 +0200)]
block: mark bdrv_parent_change_aio_context() GRAPH_RDLOCK
This is a small step in preparation to mark bdrv_drained_begin() as
GRAPH_UNLOCKED. More concretely, it allows marking the
change_aio_ctx() callback GRAPH_RDLOCK_PTR, which is the next step.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-6-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:41 +0000 (17:10 +0200)]
block: move drain outside of read-locked bdrv_inactivate_recurse()
This is in preparation to mark bdrv_drained_begin() as GRAPH_UNLOCKED.
More granular draining is not trivially possible, because
bdrv_inactivate_recurse() can recursively call itself.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-5-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:40 +0000 (17:10 +0200)]
block/snapshot: move drain outside of read-locked bdrv_snapshot_delete()
This is in preparation to mark bdrv_drained_begin() as GRAPH_UNLOCKED.
More granular draining is not trivially possible, because
bdrv_snapshot_delete() can recursively call itself.
The return value of bdrv_all_delete_snapshot() changes from -1 to
-errno propagated from failed sub-calls. This is fine for the existing
callers of bdrv_all_delete_snapshot().
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-4-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:39 +0000 (17:10 +0200)]
block: move drain outside of read-locked bdrv_reopen_queue_child()
This is in preparation to mark bdrv_drained_begin() as GRAPH_UNLOCKED.
More granular draining is not trivially possible, because
bdrv_reopen_queue_child() can recursively call itself.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-3-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Fiona Ebner [Fri, 30 May 2025 15:10:38 +0000 (17:10 +0200)]
block: remove outdated comments about AioContext locking
AioContext locking was removed in commit b49f4755c7 ("block: remove
AioContext locking").
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20250530151125.955508-2-f.ebner@proxmox.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* tag 'pull-qapi-2025-06-03' of https://repo.or.cz/qemu/armbru:
qapi: Improve documentation around job state @concluded
qapi: Tidy up references to job state CONCLUDED
qapi: Mention both job-cancel and block-job-cancel in doc comments
qapi: Refer to job-FOO instead of deprecated block-job-FOO in docs
qapi: Spell JSON null correctly in blockdev-reopen documentation
qapi: Use proper markup instead of CAPS for emphasis in doc comments
qapi: Fix capitalization in doc comments
qapi: Correct spelling of QEMU in doc comments
qapi: Drop a problematic (Since: 2.11) from query-hotpluggable-cpus
qapi: Avoid breaking lines within (since X.Y)
qapi: Move (since X.Y) to end of description
qapi: Tidy up whitespace in doc comments
qapi: Tidy up run-together sentences in doc comments
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
qapi: Improve documentation around job state @concluded
We use "the query list" in a few places. It's not entirely obvious
what that means. It's actually the output of query-jobs or
query-block-jobs.
Documentation of @auto-dismiss talks about the job disappearing from
the query list when it reaches state @concluded. This is less than
precise. The job doesn't merely disappear from the query list, it
disappears, period.
Documentation of JobStatus @concluded explains "the job will remain in
the query list until it is dismissed". Again less than precise. It
remains in state @concluded until dismissed.
Rephrase without use of "the query list" for clarity and precision.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-14-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
When talking about the job state machine, we refer to the states like
READY, ABORTING, CONCLUDED, and so forth. Except in two places, where
we use JOB_STATUS_CONCLUDED. Replace by CONCLUDED for consistency.
We should arguably use the JobStatus enum values instead. Left for
another day.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-13-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
qapi: Mention both job-cancel and block-job-cancel in doc comments
Several doc comments mention block-job-cancel where the more generic
job-cancel would also work. Adjust them to mention both.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-12-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
qapi: Refer to job-FOO instead of deprecated block-job-FOO in docs
We deprecated several block-job-FOO commands in commit b836bf2ab68 (qapi/block-core: deprecate some block-job- APIs). Update
the doc comments to refer to their replacements instead.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-11-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
qapi: Spell JSON null correctly in blockdev-reopen documentation
The doc comment misspells JSON null as NULL. Fix that.
Cc: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-10-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
qapi: Use proper markup instead of CAPS for emphasis in doc comments
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-9-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-8-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Improve awkward phrasing in migrate-incoming While there.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-7-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
qapi: Drop a problematic (Since: 2.11) from query-hotpluggable-cpus
There is a (Since: 2.11) in a query-hotpluggable-cpus example.
Versioning information ought to be in the command description, not
examples. The command description is basically empty (there is a TODO
about it).
What exactly didn't work before 2.11 is not quite clear from the
documentation. The example was added in commit 4dc3b151882 (s390x:
implement query-hotpluggable-cpus), which suggests the command failed
for the s390x target until then. This was almost eight years ago, and
I doubt anyone still cares about this detail. Simply delete
the problematic (Since: 2.11).
Cc: David Hildenbrand <david@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-6-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-5-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
By convention, we put (since X.Y) at the end of the description. Move
the ones that somehow ended up in the middle of the description to the
end.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-4-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-3-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
qapi: Tidy up run-together sentences in doc comments
Fixes: a937b6aa739f (qapi: Reformat doc comments to conform to current conventions) Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250527073916.1243024-2-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Tanish Desai [Wed, 28 May 2025 19:25:28 +0000 (19:25 +0000)]
trace/simple: seperate hot paths of tracing fucntions
This change improves performance by moving the hot path of the trace_vhost_commit()(or any other trace function) logic to the header file.
Previously, even when the trace event was disabled, the function call chain:-
trace_vhost_commit()(Or any other trace function) → _nocheck__trace_vhost_commit() → _simple_trace_vhost_commit()
incurred a significant function prologue overhead before checking the trace state.
Disassembly of _simple_trace_vhost_commit() (from the .c file) showed that 11 out of the first 14 instructions were prologue-related, including:
0x10 stp x29, x30, [sp, #-64]! Prologue: allocates 64-byte frame and saves old FP (x29) & LR (x30)
0x14 adrp x3, trace_events_enabled_count Prologue: computes page-base of the trace-enable counter
0x18 adrp x2, __stack_chk_guard Important (maybe prolog don't know?)(stack-protector): starts up the stack-canary load
0x1c mov x29, sp Prologue: sets new frame pointer
0x20 ldr x3, [x3] Prologue: loads the actual trace-enabled count
0x24 stp x19, x20, [sp, #16] Prologue: spills callee-saved regs used by this function (x19, x20)
0x28 and w20, w0, #0xff Tracepoint setup: extracts the low-8 bits of arg0 as the “event boolean”
0x2c ldr x2, [x2] Prologue (cont’d): completes loading of the stack-canary value
0x30 and w19, w1, #0xff Tracepoint setup: extracts low-8 bits of arg1
0x34 ldr w0, [x3] Important: loads the current trace-enabled flag from memory
0x38 ldr x1, [x2] Prologue (cont’d): reads the canary
0x3c str x1, [sp, #56] Prologue (cont’d): writes the canary into the new frame
0x40 mov x1, #0 Prologue (cont’d): zeroes out x1 for the upcoming branch test
0x44 cbnz w0, 0x88 Important: if tracing is disabled (w0==0) skip the heavy path entirely
The trace-enabled check happens after the prologue. This is wasteful when tracing is disabled, which is often the case in production.
To optimize this:
_nocheck__trace_vhost_commit() is now fully inlined in the .h file with
the hot path.It checks trace_event_get_state() before calling into _simple_trace_vhost_commit(), which remains in .c.
This avoids calling into the .c function altogether when the tracepoint is disabled, thereby skipping unnecessary prologue instructions.
This results in better performance by removing redundant instructions in the tracing fast path.
Signed-off-by: Tanish Desai <tanishdesai37@gmail.com>
Message-id: 20250528192528.3968-1-tanishdesai37@gmail.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi [Mon, 2 Jun 2025 18:52:44 +0000 (14:52 -0400)]
Merge tag 'for_upstream' of https://git.kernel.org/pub/scm/virt/kvm/mst/qemu into staging
virtio,pci,pc: features, fixes, tests
vhost will now no longer set a call notifier if unused
some work towards loongarch testing based on bios-tables-test
some core pci work for SVM support in vtd
vhost vdpa init has been optimized for response time to QMP
A couple more fixes
* tag 'for_upstream' of https://git.kernel.org/pub/scm/virt/kvm/mst/qemu: (26 commits)
hw/i386/pc_piix: Fix RTC ISA IRQ wiring of isapc machine
vdpa: move memory listener register to vhost_vdpa_init
vdpa: move iova_tree allocation to net_vhost_vdpa_init
vdpa: reorder listener assignment
vdpa: add listener_registered
vdpa: set backend capabilities at vhost_vdpa_init
vdpa: reorder vhost_vdpa_set_backend_cap
vdpa: check for iova tree initialized at net_client_start
vhost: Don't set vring call if guest notifier is unused
tests/qtest/bios-tables-test: Use MiB macro rather hardcode value
tests/data/uefi-boot-images: Add ISO image for LoongArch system
uefi-test-tools:: Add LoongArch64 support
pci: Add a PCI-level API for PRI
pci: Add a pci-level API for ATS
pci: Add a pci-level initialization function for IOMMU notifiers
memory: Store user data pointer in the IOMMU notifiers
pci: Add an API to get IOMMU's min page size and virtual address width
pci: Cache the bus mastering status in the device
pcie: Helper functions to check to check if PRI is enabled
pcie: Add a helper to declare the PRI capability for a pcie device
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Bernhard Beschow [Mon, 26 May 2025 20:38:20 +0000 (22:38 +0200)]
hw/i386/pc_piix: Fix RTC ISA IRQ wiring of isapc machine
Commit 56b1f50e3c10 ("hw/i386/pc: Wire RTC ISA IRQs in south bridges")
attempted to refactor RTC IRQ wiring which was previously done in
pc_basic_device_init() but forgot about the isapc machine. Fix this by
wiring in the code section dedicated exclusively to the isapc machine.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2961 Fixes: 56b1f50e3c10 ("hw/i386/pc: Wire RTC ISA IRQs in south bridges")
cc: qemu-stable Signed-off-by: Bernhard Beschow <shentey@gmail.com> Reviewed-by: Mark Cave-Ayland <mark.caveayland@nutanix.com>
Message-Id: <20250526203820.1853-1-shentey@gmail.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Eugenio Pérez [Thu, 22 May 2025 14:58:39 +0000 (10:58 -0400)]
vdpa: move memory listener register to vhost_vdpa_init
Current memory operations like pinning may take a lot of time at the
destination. Currently they are done after the source of the migration is
stopped, and before the workload is resumed at the destination. This is a
period where neigher traffic can flow, nor the VM workload can continue
(downtime).
We can do better as we know the memory layout of the guest RAM at the
destination from the moment that all devices are initializaed. So
moving that operation allows QEMU to communicate the kernel the maps
while the workload is still running in the source, so Linux can start
mapping them.
As a small drawback, there is a time in the initialization where QEMU
cannot respond to QMP etc. By some testing, this time is about
0.2seconds. This may be further reduced (or increased) depending on the
vdpa driver and the platform hardware, and it is dominated by the cost
of memory pinning.
This matches the time that we move out of the called downtime window.
The downtime is measured as the elapsed trace time between the last
vhost_vdpa_suspend on the source and the last vhost_vdpa_set_vring_enable_one
on the destination. In other words, from "guest CPUs freeze" to the
instant the final Rx/Tx queue-pair is able to start moving data.
Using ConnectX-6 Dx (MLX5) NICs in vhost-vDPA mode with 8 queue-pairs,
the series reduces guest-visible downtime during back-to-back live
migrations by more than half:
- 39G VM: 4.72s -> 2.09s (-2.63s, ~56% improvement)
- 128G VM: 14.72s -> 5.83s (-8.89s, ~60% improvement)
Tested-by: Lei Yang <leiyang@redhat.com> Reviewed-by: Si-Wei Liu <si-wei.liu@oracle.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <20250522145839.59974-8-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Eugenio Pérez [Thu, 22 May 2025 14:58:38 +0000 (10:58 -0400)]
vdpa: move iova_tree allocation to net_vhost_vdpa_init
As we are moving to keep the mapping through all the vdpa device life
instead of resetting it at VirtIO reset, we need to move all its
dependencies to the initialization too. In particular devices with
x-svq=on need a valid iova_tree from the beginning.
Simplify the code also consolidating the two creation points: the first
data vq in case of SVQ active and CVQ start in case only CVQ uses it.
Tested-by: Lei Yang <leiyang@redhat.com> Reviewed-by: Si-Wei Liu <si-wei.liu@oracle.com> Acked-by: Jason Wang <jasowang@redhat.com> Suggested-by: Si-Wei Liu <si-wei.liu@oracle.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <20250522145839.59974-7-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Eugenio Pérez [Thu, 22 May 2025 14:58:37 +0000 (10:58 -0400)]
vdpa: reorder listener assignment
Since commit f6fe3e333f ("vdpa: move memory listener to
vhost_vdpa_shared") this piece of code repeatedly assign
shared->listener members. This was not a problem as it was not used
until device start.
However next patches move the listener registration to this
vhost_vdpa_init function. When the listener is registered it is added
to an embedded linked list, so setting its members again will cause
memory corruption to the linked list node.
Do the right thing and only set it in the first vdpa device.
Tested-by: Lei Yang <leiyang@redhat.com> Reviewed-by: Si-Wei Liu <si-wei.liu@oracle.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <20250522145839.59974-6-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Eugenio Pérez [Thu, 22 May 2025 14:58:36 +0000 (10:58 -0400)]
vdpa: add listener_registered
Check if the listener has been registered or not, so it needs to be
registered again at start.
Tested-by: Lei Yang <leiyang@redhat.com> Reviewed-by: Si-Wei Liu <si-wei.liu@oracle.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <20250522145839.59974-5-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Eugenio Pérez [Thu, 22 May 2025 14:58:35 +0000 (10:58 -0400)]
vdpa: set backend capabilities at vhost_vdpa_init
The backend does not reset them until the vdpa file descriptor is closed
so there is no harm in doing it only once.
This allows the destination of a live migration to premap memory in
batches, using VHOST_BACKEND_F_IOTLB_BATCH.
Tested-by: Lei Yang <leiyang@redhat.com> Reviewed-by: Si-Wei Liu <si-wei.liu@oracle.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <20250522145839.59974-4-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Eugenio Pérez [Thu, 22 May 2025 14:58:34 +0000 (10:58 -0400)]
vdpa: reorder vhost_vdpa_set_backend_cap
It will be used directly by vhost_vdpa_init.
Tested-by: Lei Yang <leiyang@redhat.com> Reviewed-by: Si-Wei Liu <si-wei.liu@oracle.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <20250522145839.59974-3-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Eugenio Pérez [Thu, 22 May 2025 14:58:33 +0000 (10:58 -0400)]
vdpa: check for iova tree initialized at net_client_start
To map the guest memory while it is migrating we need to create the
iova_tree, as long as the destination uses x-svq=on. Checking to not
override it.
The function vhost_vdpa_net_client_stop clear it if the device is
stopped. If the guest starts the device again, the iova tree is
recreated by vhost_vdpa_net_data_start_first or vhost_vdpa_net_cvq_start
if needed, so old behavior is kept.
Tested-by: Lei Yang <leiyang@redhat.com> Reviewed-by: Si-Wei Liu <si-wei.liu@oracle.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <20250522145839.59974-2-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Huaitong Han [Thu, 22 May 2025 10:05:48 +0000 (18:05 +0800)]
vhost: Don't set vring call if guest notifier is unused
The vring call fd is set even when the guest does not use MSI-X (e.g., in the
case of virtio PMD), leading to unnecessary CPU overhead for processing
interrupts.
The commit 96a3d98d2c("vhost: don't set vring call if no vector") optimized the
case where MSI-X is enabled but the queue vector is unset. However, there's an
additional case where the guest uses INTx and the INTx_DISABLED bit in the PCI
config is set, meaning that no interrupt notifier will actually be used.
In such cases, the vring call fd should also be cleared to avoid redundant
interrupt handling.
Fixes: 96a3d98d2c("vhost: don't set vring call if no vector") Reported-by: Zhiyuan Yuan <yuanzhiyuan@chinatelecom.cn> Signed-off-by: Jidong Xia <xiajd@chinatelecom.cn> Signed-off-by: Huaitong Han <hanht2@chinatelecom.cn>
Message-Id: <20250522100548.212740-1-hanht2@chinatelecom.cn> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Bibo Mao [Tue, 20 May 2025 13:01:53 +0000 (21:01 +0800)]
tests/qtest/bios-tables-test: Use MiB macro rather hardcode value
Replace 1024 * 1024 with MiB macro.
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
Message-Id: <20250520130158.767083-4-maobibo@loongson.cn> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Bibo Mao [Tue, 20 May 2025 13:01:52 +0000 (21:01 +0800)]
tests/data/uefi-boot-images: Add ISO image for LoongArch system
To test ACPI tables, edk2 needs to be booted with a disk image having
EFI partition. This image is created using UefiTestToolsPkg.
The image is generated with the following command:
make -f tests/uefi-test-tools/Makefile
Signed-off-by: Bibo Mao <maobibo@loongson.cn> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20250520130158.767083-3-maobibo@loongson.cn> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Bibo Mao [Tue, 20 May 2025 13:01:51 +0000 (21:01 +0800)]
uefi-test-tools:: Add LoongArch64 support
Add support to build bios-tables-test iso image for LoongArch system.
Signed-off-by: Bibo Mao <maobibo@loongson.cn> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20250520130158.767083-2-maobibo@loongson.cn> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
A device can send a PRI request to the IOMMU using pci_pri_request_page.
The PRI response is sent back using the notifier managed with
pci_pri_register_notifier and pci_pri_unregister_notifier.
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com> Co-authored-by: Ethan Milon <ethan.milon@eviden.com>
Message-Id: <20250520071823.764266-12-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Devices implementing ATS can send translation requests using
pci_ats_request_translation. The invalidation events are sent
back to the device using the iommu notifier managed with
pci_iommu_register_iotlb_notifier / pci_iommu_unregister_iotlb_notifier.
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com> Co-authored-by: Ethan Milon <ethan.milon@eviden.com>
Message-Id: <20250520071823.764266-11-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pci: Add a pci-level initialization function for IOMMU notifiers
This is meant to be used by ATS-capable devices.
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-10-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
memory: Store user data pointer in the IOMMU notifiers
This will help developers of ATS-capable devices to track a state.
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-9-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pci: Add an API to get IOMMU's min page size and virtual address width
This kind of information is needed by devices implementing ATS in order
to initialize their translation cache.
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-8-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
The cached is_master value is necessary to know if a device is
allowed to issue ATS/PRI requests or not as these operations do not go
through the master_enable memory region.
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-7-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pcie: Helper functions to check to check if PRI is enabled
pri_enabled can be used to check whether the capability is present and
enabled on a PCIe device
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-6-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pcie: Add a helper to declare the PRI capability for a pcie device
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-5-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
ats_enabled checks whether the capability is
present or not. If so, we read the configuration space to get
the status of the feature (enabled or not).
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-4-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pcie: Helper functions to check if PASID is enabled
pasid_enabled checks whether the capability is
present or not. If so, we read the configuration space to get
the status of the feature (enabled or not).
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-3-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pcie: Add helper to declare PASID capability for a pcie device
Signed-off-by: Clement Mathieu--Drif <clement.mathieu--drif@eviden.com>
Message-Id: <20250520071823.764266-2-clement.mathieu--drif@eviden.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Vasant Hegde [Fri, 16 May 2025 10:05:35 +0000 (15:35 +0530)]
hw/i386/amd_iommu: Fix xtsup when vcpus < 255
If vCPUs > 255 then x86 common code (x86_cpus_init()) call kvm_enable_x2apic().
But if vCPUs <= 255 then the common code won't calls kvm_enable_x2apic().
This is because commit 8c6619f3e692 ("hw/i386/amd_iommu: Simplify non-KVM
checks on XTSup feature") removed the call to kvm_enable_x2apic when xtsup
is "on", which break things when guest is booted with x2apic mode and
there are <= 255 vCPUs.
Fix this by adding back kvm_enable_x2apic() call when xtsup=on.
Sairaj Kodilkar [Fri, 16 May 2025 10:05:34 +0000 (15:35 +0530)]
hw/i386/amd_iommu: Fix device setup failure when PT is on.
Commit c1f46999ef506 ("amd_iommu: Add support for pass though mode")
introduces the support for "pt" flag by enabling nodma memory when
"pt=off". This allowed VFIO devices to successfully register notifiers
by using nodma region.
But, This also broke things when guest is booted with the iommu=nopt
because, devices bypass the IOMMU and use untranslated addresses (IOVA) to
perform DMA reads/writes to the nodma memory region, ultimately resulting in
a failure to setup the devices in the guest.
Fix the above issue by always enabling the amdvi_dev_as->iommu memory region.
But this will once again cause VFIO devices to fail while registering the
notifiers with AMD IOMMU memory region.
Fixes: c1f46999ef506 ("amd_iommu: Add support for pass though mode") Signed-off-by: Sairaj Kodilkar <sarunkod@amd.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Message-Id: <20250516100535.4980-2-sarunkod@amd.com> Fixes: c1f46999ef506 ("amd_iommu: Add support for pass though mode") Signed-off-by: Sairaj Kodilkar <sarunkod@amd.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Yuri Benditovich [Thu, 15 May 2025 06:32:37 +0000 (09:32 +0300)]
virtio: check for validity of indirect descriptors
virtio processes indirect descriptors even if the respected
feature VIRTIO_RING_F_INDIRECT_DESC was not negotiated.
If qemu is used with reduced set of features to emulate the
hardware device that does not support indirect descriptors,
the will probably trigger problematic flows on the hardware
setup but do not reveal the mistake on qemu.
Add LOG_GUEST_ERROR for such case. This will issue logs with
'-d guest_errors' in the command line
Stefan Hajnoczi [Fri, 30 May 2025 15:41:21 +0000 (11:41 -0400)]
Merge tag 'pull-target-arm-20250530-2' of https://git.linaro.org/people/pmaydell/qemu-arm into staging
target-arm queue:
* hw/arm: Add GMAC devices to NPCM8XX SoC
* hw/arm: Add missing psci_conduit to NPCM8XX SoC boot info
* docs/interop: convert text files to restructuredText
* target/arm: Some minor refactorings
* tests/functional: Add a test for the Stellaris arm machines
* hw/block: Drop unused nand.c
* tag 'pull-target-arm-20250530-2' of https://git.linaro.org/people/pmaydell/qemu-arm:
hw/block: Drop unused nand.c
tests/functional: Add a test for the Stellaris arm machines
target/arm/hvf: Include missing 'cpu-qom.h' header
target/arm/kvm: Include missing 'cpu-qom.h' header
target/arm/qmp: Include missing 'cpu.h' header
target/arm/cpu-features: Include missing 'cpu.h' header
hw/arm/boot: Include missing 'system/memory.h' header
target/arm/cpregs: Include missing 'target/arm/cpu.h' header
target/arm: Only link with zlib when TCG is enabled
target/arm/hvf_arm: Avoid using poisoned CONFIG_HVF definition
target/arm/tcg-stubs: compile file once (system)
docs/interop: convert text files to restructuredText
hw/arm: Add missing psci_conduit to NPCM8XX SoC boot info
tests/qtest: Migrate GMAC test from 7xx to 8xx
hw/arm: Add GMAC devices to NPCM8XX SoC
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi [Fri, 30 May 2025 15:41:13 +0000 (11:41 -0400)]
Merge tag 'pull-request-2025-05-30' of https://gitlab.com/thuth/qemu into staging
* Functional tests improvements
* Endianness improvements/clean-ups for the Microblaze machines
* Remove obsolete -2.4 and -2.5 i440fx and q35 machine types and related code
Stefan Hajnoczi [Fri, 30 May 2025 15:41:07 +0000 (11:41 -0400)]
Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging
* target/i386/kvm: Intel TDX support
* target/i386/emulate: more lflags cleanups
* meson: remove need for explicit listing of dependencies in hw_common_arch and
target_common_arch
* rust: small fixes
* hpet: Reorganize register decoding to be more similar to Rust code
* target/i386: fixes for AMD models
* target/i386: new EPYC-Turin CPU model
* tag 'for-upstream' of https://gitlab.com/bonzini/qemu: (77 commits)
target/i386/tcg/helper-tcg: fix file references in comments
target/i386: Add support for EPYC-Turin model
target/i386: Update EPYC-Genoa for Cache property, perfmon-v2, RAS and SVM feature bits
target/i386: Add couple of feature bits in CPUID_Fn80000021_EAX
target/i386: Update EPYC-Milan CPU model for Cache property, RAS, SVM feature bits
target/i386: Update EPYC-Rome CPU model for Cache property, RAS, SVM feature bits
target/i386: Update EPYC CPU model for Cache property, RAS, SVM feature bits
rust: make declaration of dependent crates more consistent
docs: Add TDX documentation
i386/tdx: Validate phys_bits against host value
i386/tdx: Make invtsc default on
i386/tdx: Don't treat SYSCALL as unavailable
i386/tdx: Fetch and validate CPUID of TD guest
target/i386: Print CPUID subleaf info for unsupported feature
i386: Remove unused parameter "uint32_t bit" in feature_word_description()
i386/cgs: Introduce x86_confidential_guest_check_features()
i386/tdx: Define supported KVM features for TDX
i386/tdx: Add XFD to supported bit of TDX
i386/tdx: Add supported CPUID bits relates to XFAM
i386/tdx: Add supported CPUID bits related to TD Attributes
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
* tag 'pull-nbd-2025-05-29' of https://repo.or.cz/qemu/ericb:
iotests: Filter out ZFS in several tests
iotests: Improve mirror-sparse on ext4 and xfs
iotests: Use disk_usage in more places
nbd: Set unix socket send buffer on Linux
nbd: Set unix socket send buffer on macOS
io: Add helper for setting socket send buffer size
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
tests/unit/test-util-sockets: fix mem-leak on error object
The test fails with --enable-asan as the error struct is never freed.
In the case where the test expects a success but it fails, let's also
report the error for debugging (it will be freed internally).
Fixes 316e8ee8d6 ("util/qemu-sockets: Refactor inet_parse() to use QemuOpts")
Signed-off-by: Matheus Tavares Bernardino <matheus.bernardino@oss.qualcomm.com> Reviewed-by: Juraj Marcin <jmarcin@redhat.com>
Message-ID: <518d94c7db20060b2a086cf55ee9bffab992a907.1748280011.git.matheus.bernardino@oss.qualcomm.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
hw/net/vmxnet3: Merge DeviceRealize in InstanceInit
Simplify merging vmxnet3_realize() within vmxnet3_instance_init(),
removing the need for device_class_set_parent_realize().
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-20-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
VMXNET3_COMPAT_FLAG_DISABLE_PCIE was only used by the
hw_compat_2_5[] array, via the 'x-disable-pcie=on' property.
We removed all machines using that array, lets remove all the
code around VMXNET3_COMPAT_FLAG_DISABLE_PCIE.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-19-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
VMXNET3_COMPAT_FLAG_OLD_MSI_OFFSETS was only used by the
hw_compat_2_5[] array, via the 'x-old-msi-offsets=on' property.
We removed all machines using that array, lets remove all the
code around VMXNET3_COMPAT_FLAG_OLD_MSI_OFFSETS.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-18-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Simplify replacing pvscsi_realize() by pvscsi_instance_init(),
removing the need for device_class_set_parent_realize().
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-17-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
PVSCSI_COMPAT_DISABLE_PCIE_BIT was only used by the
hw_compat_2_5[] array, via the 'x-disable-pcie=on' property.
We removed all machines using that array, lets remove all the
code around PVSCSI_COMPAT_DISABLE_PCIE_BIT, including the now
unused PVSCSIState::compat_flags field.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-16-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
PVSCSI_COMPAT_OLD_PCI_CONFIGURATION was only used by the
hw_compat_2_5[] array, via the 'x-old-pci-configuration=on'
property. We removed all machines using that array, lets remove
all the code around PVSCSI_COMPAT_OLD_PCI_CONFIGURATION.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-15-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
The hw_compat_2_5[] array was only used by the pc-q35-2.5 and
pc-i440fx-2.5 machines, which got removed. Remove it.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-13-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-12-philmd@linaro.org>
[thuth: Fix error from check_patch.pl wrt to an empty "for" loop] Signed-off-by: Thomas Huth <thuth@redhat.com>
hw/i386/x86: Remove X86MachineClass::save_tsc_khz field
The X86MachineClass::save_tsc_khz boolean was only used
by the pc-q35-2.5 and pc-i440fx-2.5 machines, which got
removed. Remove it and simplify tsc_khz_needed().
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-11-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
hw/i386/pc: Remove deprecated pc-q35-2.5 and pc-i440fx-2.5 machines
These machines has been supported for a period of more than 6 years.
According to our versioned machine support policy (see commit ce80c4fa6ff "docs: document special exception for machine type
deprecation & removal") they can now be removed.
Remove the now unused empty pc_compat_2_5[] array.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-10-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
VIRTIO_PCI_FLAG_DISABLE_PCIE was only used by the hw_compat_2_4[]
array, via the 'x-disable-pcie=false' property. We removed all
machines using that array, lets remove all the code around
VIRTIO_PCI_FLAG_DISABLE_PCIE (see commit 9a4c0e220d8 for similar
VIRTIO_PCI_FLAG_* enum removal).
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-9-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
VIRTIO_PCI_FLAG_MIGRATE_EXTRA was only used by the
hw_compat_2_4[] array, via the 'migrate-extra=true'
property. We removed all machines using that array,
lets remove all the code around VIRTIO_PCI_FLAG_MIGRATE_EXTRA.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Message-ID: <20250512083948.39294-8-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
E1000_FLAG_MAC was only used by the hw_compat_2_4[] array,
via the 'extra_mac_registers=off' property. We removed all
machines using that array, lets remove all the code around
E1000_FLAG_MAC, including the MAC_ACCESS_FLAG_NEEDED enum,
similarly to commit fa4ec9ffda7 ("e1000: remove old
compatibility code").
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Message-ID: <20250512083948.39294-7-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
The hw_compat_2_4[] array was only used by the pc-q35-2.4 and
pc-i440fx-2.4 machines, which got removed. Remove it.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-6-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
The pc_compat_2_4[] array was only used by the pc-q35-2.4
and pc-i440fx-2.4 machines, which got removed. Remove it.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-4-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
hw/i386/pc: Remove PCMachineClass::broken_reserved_end field
The PCMachineClass::broken_reserved_end field was only used
by the pc-q35-2.4 and pc-i440fx-2.4 machines, which got removed.
Remove it and simplify pc_memory_init().
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-3-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
hw/i386/pc: Remove deprecated pc-q35-2.4 and pc-i440fx-2.4 machines
These machines has been supported for a period of more than 6 years.
According to our versioned machine support policy (see commit ce80c4fa6ff "docs: document special exception for machine type
deprecation & removal") they can now be removed.
Remove the qtest in test-x86-cpuid-compat.c file.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Message-ID: <20250512083948.39294-2-philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Thomas Huth [Thu, 15 May 2025 13:20:19 +0000 (15:20 +0200)]
docs: Deprecate the qemu-system-microblazeel binary
The (former big-endian only) binary qemu-system-microblaze can
handle both endiannesses nowadays, so we don't need the separate
qemu-system-microblazeel binary for little endian anymore. Let's
deprecate it to avoid unnecessary compilation and test time in
the future.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250515132019.569365-5-thuth@redhat.com>
Thomas Huth [Thu, 15 May 2025 13:20:18 +0000 (15:20 +0200)]
hw/microblaze: Remove the big-endian variants of ml605 and xlnx-zynqmp-pmu
Both machines were added with little-endian in mind only (the
"endianness" CPU property was hard-wired to "true", see commits 133d23b3ad1 and a88bbb006a52), so the variants that showed up
on the big endian target likely never worked. We deprecated these
non-working machine variants two releases ago, and so far nobody
complained, so it should be fine now to disable them. Hard-wire
the machines to little endian now.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250515132019.569365-4-thuth@redhat.com>
Thomas Huth [Thu, 15 May 2025 13:20:17 +0000 (15:20 +0200)]
tests/functional: Test both microblaze s3adsp1800 endianness variants
Now that the endianness of the petalogix-s3adsp1800 can be configured,
we should test that the cross-endianness also works as expected, thus
test the big endian variant on the little endian target and vice versa.
(based on an original idea from Philippe Mathieu-Daudé)
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250515132019.569365-3-thuth@redhat.com>
Thomas Huth [Thu, 15 May 2025 13:20:16 +0000 (15:20 +0200)]
hw/microblaze: Add endianness property to the petalogix_s3adsp1800 machine
Since the microblaze target can now handle both endianness, big and
little, we should provide a config knob for the user to select the
desired endianness.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250515132019.569365-2-thuth@redhat.com>
Thomas Huth [Wed, 21 May 2025 14:37:32 +0000 (16:37 +0200)]
tests/functional/test_mem_addr_space: Use set_machine() to select the machine
By using self.set_machine() the tests get properly skipped in case
the machine has not been compiled into the QEMU binary, e.g. when
"configure" has been run with "--without-default-devices".
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250521143732.140711-1-thuth@redhat.com>
Thomas Huth [Thu, 22 May 2025 08:02:08 +0000 (10:02 +0200)]
tests/functional/test_mips_malta: Re-enable the check for the PCI host bridge
The problem with the PCI bridge has been fixed in commit e5894fd6f411c1
("hw/pci-host/gt64120: Fix endianness handling"), so we can enable the
corresponding test again.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250522080208.205489-1-thuth@redhat.com>
Thomas Huth [Wed, 21 May 2025 14:51:12 +0000 (16:51 +0200)]
tests/functional/test_sparc64_tuxrun: Explicitly set the 'sun4u' machine
Use self.set_machine() to set the machine instead of relying on the
default machine of the binary. This way the test can be skipped in
case the machine has not been compiled into the QEMU binary.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250521145112.142222-1-thuth@redhat.com>
Eric Blake [Fri, 23 May 2025 16:27:23 +0000 (11:27 -0500)]
iotests: Filter out ZFS in several tests
Fiona reported that ZFS makes sparse file testing awkward, since:
- it has asynchronous allocation (not even 'fsync $file' makes du see
the desired size; it takes the slower 'fsync -f $file' which is not
appropriate for the tests)
- for tests of fully allocated files, ZFS with compression enabled
still reports smaller disk usage
Add a new _require_disk_usage that quickly probes whether an attempt
to create a sparse 5M file shows as less than 1M usage, while the same
file with -o preallocation=full shows as more than 4M usage without
sync, which should filter out ZFS behavior. Then use it in various
affected tests.
This does not add the new filter on all tests that Fiona is seeing ZFS
failures on, but only those where I could quickly spot that there is
at least one place where the test depends on the output of 'du -b' or
'stat -c %b'.
Eric Blake [Fri, 23 May 2025 16:27:22 +0000 (11:27 -0500)]
iotests: Improve mirror-sparse on ext4 and xfs
Fiona reported that an ext4 filesystem on top of LVM can sometimes
report over-allocation to du (based on the heuristics the filesystem
is making while observing the contents being mirrored); even though
the contents and actual size matched, about 50% of the time the size
reported by disk_usage was too large by 4k, failing the test. In
auditing other iotests, this is a common problem we've had to deal
with.
Meanwhile, Markus reported that an xfs filesystem reports disk usage
at a default granularity of 1M (so the sparse file occupies 3M, since
it has just over 2M data).
Reported-by: Fiona Ebner <f.ebner@proxmox.com> Reported-by: Markus Armbruster <armbru@redhat.com> Fixes: c0ddcb2c ("tests: Add iotest mirror-sparse for recent patches") Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Fiona Ebner <f.ebner@proxmox.com> Tested-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20250523163041.2548675-7-eblake@redhat.com>
[eblake: Also fix xfs issue] Signed-off-by: Eric Blake <eblake@redhat.com>
Nir Soffer [Sat, 17 May 2025 20:11:54 +0000 (23:11 +0300)]
nbd: Set unix socket send buffer on Linux
Like macOS we have similar issue on Linux. For TCP socket the send
buffer size is 2626560 bytes (~2.5 MiB) and we get good performance.
However for unix socket the default and maximum buffer size is 212992
bytes (208 KiB) and we see poor performance when using one NBD
connection, up to 4 times slower than macOS on the same machine.
Tracing shows that for every 2 MiB payload (qemu uses 2 MiB io size), we
do 1 recvmsg call with TCP socket, and 10 recvmsg calls with unix
socket.
Fixing this issue requires changing the maximum send buffer size (the
receive buffer size is ignored). This can be done using:
With this we can set the socket buffer size to 2 MiB. With the defaults
the value requested by qemu is clipped to the maximum size and has no
effect.
I tested on 2 machines:
- Fedora 42 VM on MacBook Pro M2 Max
- Dell PowerEdge R640 (Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz)
On the older Dell machine we see very little improvement, up to 1.03
higher throughput. On the M2 machine we see up to 2.67 times higher
throughput. The following results are from the M2 machine.
Reading from qemu-nbd with qemu-img convert. In this test buffer size of
4m is optimal (2.28 times faster).
Signed-off-by: Nir Soffer <nirsof@gmail.com>
Message-ID: <20250517201154.88456-4-nirsof@gmail.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
projects / thirdparty / qemu.git / log
