]> git.ipfire.org Git - thirdparty/qemu.git/log
thirdparty/qemu.git
3 months agoMerge tag 'pull-qapi-2025-03-11' of https://repo.or.cz/qemu/armbru into staging
Stefan Hajnoczi [Tue, 11 Mar 2025 23:49:54 +0000 (07:49 +0800)] 
Merge tag 'pull-qapi-2025-03-11' of https://repo.or.cz/qemu/armbru into staging

QAPI patches patches for 2025-03-11

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmfQCnkSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTsJ0P/jcXiyFxjcbXN/3a6+iuPPqlviiWPAKG
# db2aHn2divceFEf7hUrwqjiJIPLDxaq6iJy71bjPUDkE8wAEdsf2zD7ryHo+sGcO
# rWaSaHmonn0QHvqcvkGGrbmTH+Ezl1RpP8XVGfG2lmHbjPQ3+EYnRwML6jC8dnvR
# C7qkyQ+qxmdV2lWb4MalgABKZToZ2aqnI9lr9KzHmN+55i2OxJrhECUKDHcgtG2i
# Pqc1GLGmmQ4Wj+4z0PyvKYZS4LP/90eH8bNyeA6TVsPHxgG79pencct7DOHxhc8q
# hHQ1TaqcBeWFQ7tndLMNDnHjm9XpAzMuew87xMTo6R450JxiSn+AkioTE0L563hy
# SjeXmIQ8COZbHsuSKlFJcV1OS1c/mJbwpkxptyaMLjTt2Lp9geFs39WKWHcs8pCN
# EmWSdvoqmP7D4bp1hXAVSPIIvJ7L2NwnM8ONH0KmRD5uMQrjiHsfvyWHAVnT10yu
# 8822hjlJp7l3B1QCi19mTlkiztCFScjb3Se8A+jScP5iX0q9C4H4t+tAw2m4UY1V
# pvn4xFxV82CvR3uQI0OMTKhp0/eEfvBioA1PEXOegPH5cS/L7YFF59mta1dCnaL7
# 0JRRCsTAnwAAAXoEteGqF1/6tXBdOnroL0OvHXJQVb2HH5c5YTnuxMiQywcP6Jty
# wt1vl42jfTj1
# =Gt4B
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 11 Mar 2025 18:03:37 HKT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* tag 'pull-qapi-2025-03-11' of https://repo.or.cz/qemu/armbru: (61 commits)
  scripts/qapi/backend: Clean up create_backend()'s failure mode
  MAINTAINERS: Add jsnow as maintainer for Sphinx documentation
  docs: add qapi-domain syntax documentation
  docs: enable qapidoc transmogrifier for QEMU QMP Reference
  docs: disambiguate cross-references
  qapi/parser: add undocumented stub members to all_sections
  docs/qapidoc: generate entries for undocumented members
  docs/qapidoc: Add "the members of" pointers
  docs/qapidoc: add intermediate output debugger
  docs/qapidoc: process @foo into ``foo``
  docs/qapidoc: implement transmogrify() method
  docs/qapidoc: add visit_entity()
  docs/qapidoc: add visit_sections() method
  docs/qapidoc: add visit_member() method
  docs/qapidoc: add visit_returns() method
  docs/qapidoc: prepare to record entity being transmogrified
  docs/qapidoc: add visit_feature() method
  docs/qapidoc: add add_field() and generate_field() helper methods
  docs/qapidoc: add format_type() method
  docs/qapidoc: add visit_errors() method
  ...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
3 months agoMerge tag 'tracing-pull-request' of https://gitlab.com/stefanha/qemu into staging
Stefan Hajnoczi [Tue, 11 Mar 2025 23:47:54 +0000 (07:47 +0800)] 
Merge tag 'tracing-pull-request' of https://gitlab.com/stefanha/qemu into staging

Pull request

A tracing cleanup.

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAmfPpaMACgkQnKSrs4Gr
# c8iC2wf/WuKijQF2eQ6R5kVY/z3H+8eg1oR3MaeRgnzFDf5Dp9H4JxNEPXssdC7p
# Dg0mXL2FhdaaQcZ9VAuyEJGtGkcbNzpXixLto3+d1SNK4fWv1VlPASp8GiDkKxpt
# nGhChUUVXLIv/wRX/eOVEuBFrUdDl/2Ri/3dMij0cZsa361KiSIygHQqF3QyspIr
# crU9B1+7ti38x/Zem+J+Wrb4VHRgJk29QUqLnH4w9j4p3LtE5cfUndlTnx28Xwkl
# bZ45XCnEu2GabaSrOmGGiAyC89w6iuxxwsnlVqg0g8fyxpUbzfhsh70FCalKfgWo
# TetDo7penusK2CBlWbrCA5BKMF29Tg==
# =2HWS
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 11 Mar 2025 10:53:23 HKT
# gpg:                using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [ultimate]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [ultimate]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* tag 'tracing-pull-request' of https://gitlab.com/stefanha/qemu:
  trace/control-target: cleanup headers and make compilation unit common

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
3 months agoscripts/qapi/backend: Clean up create_backend()'s failure mode
Markus Armbruster [Tue, 11 Mar 2025 06:53:52 +0000 (07:53 +0100)] 
scripts/qapi/backend: Clean up create_backend()'s failure mode

create_backend()'s caller catches QAPIError, and returns non-zero exit
code on catch.  The caller's caller passes the exit code to
sys.exit().

create_backend() doesn't care: it reports errors to stderr and
sys.exit()s.

Change it to raise QAPIError instead.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250311065352.992307-1-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
3 months agoMAINTAINERS: Add jsnow as maintainer for Sphinx documentation
John Snow [Tue, 11 Mar 2025 03:43:01 +0000 (23:43 -0400)] 
MAINTAINERS: Add jsnow as maintainer for Sphinx documentation

Since I've just about rewritten the entirety of the QAPI documentation
system, it's probably fair that I be the contact point for if it goes
awry.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-64-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs: add qapi-domain syntax documentation
John Snow [Tue, 11 Mar 2025 03:43:00 +0000 (23:43 -0400)] 
docs: add qapi-domain syntax documentation

Who documents the documentation?

Me, I guess.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-63-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs: enable qapidoc transmogrifier for QEMU QMP Reference
John Snow [Tue, 11 Mar 2025 03:42:59 +0000 (23:42 -0400)] 
docs: enable qapidoc transmogrifier for QEMU QMP Reference

We are not enabling the transmogrifier for QSD or QGA yet because we
don't (yet) have a way to create separate indices, and all of the
definitions will bleed together, which isn't so nice.

For now, QMP is better than nothing at all!

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-62-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs: disambiguate cross-references
John Snow [Tue, 11 Mar 2025 03:42:58 +0000 (23:42 -0400)] 
docs: disambiguate cross-references

The next patch will engage the qapidoc transmogrifier, which creates a
lot of cross-reference targets. Some of the existing targets
("migration", "qom", "replay") will become ambiguous as a result. Nail
them down more explicitly to prevent ambiguous cross-reference warnings.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-61-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agoqapi/parser: add undocumented stub members to all_sections
John Snow [Tue, 11 Mar 2025 03:42:57 +0000 (23:42 -0400)] 
qapi/parser: add undocumented stub members to all_sections

Parser and doc generator cooperate on generating stub documentation for
undocumented members.  The parser makes up an ArgSection with an empty
description, and the doc generator makes up a description.

Right now, the made-up ArgSections go into doc.args.  However, the new
doc generator uses .all_sections, not .args.  So put them into
.all_sections, too.

Insert them right after existing 'member' sections.  If there are none,
insert directly after the leading section.

Doesn't affect the old generator, because that one doesn't use
.all_sections.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-60-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message rewritten]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: generate entries for undocumented members
John Snow [Tue, 11 Mar 2025 03:42:56 +0000 (23:42 -0400)] 
docs/qapidoc: generate entries for undocumented members

Presently, we never have any empty text entries for members. The next
patch will explicitly generate such sections, so enable support for it
in advance.

The parser will generate placeholder sections to indicate undocumented
members, but it's the qapidoc generator that's responsible for deciding
what to do with that stub section.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-59-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Tweak the stub section text]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: Add "the members of" pointers
John Snow [Tue, 11 Mar 2025 03:42:55 +0000 (23:42 -0400)] 
docs/qapidoc: Add "the members of" pointers

Add "the members of ..." pointers to Members and Arguments lists where
appropriate, with clickable cross-references - so it's a slight
improvement over the old system :)

This patch is meant to be a temporary solution until we can review and
merge the inliner.

The implementation of this patch is a little bit of a hack: Sphinx is
not designed to allow you to mix fields of different "type"; i.e. mixing
member descriptions and free-form text under the same heading. To
accomplish this with a minimum of hackery, we technically document a
"dummy field" and then just strip off the documentation for that dummy
field in a post-processing step. We use the "q_dummy" variable for this
purpose, then strip it back out before final processing. If this
processing step should fail, you'll see warnings for a bad
cross-reference. (So if you don't see any, it must be working!)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-58-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add intermediate output debugger
John Snow [Tue, 11 Mar 2025 03:42:54 +0000 (23:42 -0400)] 
docs/qapidoc: add intermediate output debugger

Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
will produce .ir files (one for each qapidoc directive) that write the
generated rst file to disk to allow for easy debugging and verification
of the generated document.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-57-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: process @foo into ``foo``
John Snow [Tue, 11 Mar 2025 03:42:53 +0000 (23:42 -0400)] 
docs/qapidoc: process @foo into ``foo``

Add support for the special QAPI doc syntax to process @references as
``preformatted text``. At the moment, there are no actual
cross-references for individual members, so there is nothing to link
against. For now, process it identically to how we did in the old
qapidoc system.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-56-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: implement transmogrify() method
John Snow [Tue, 11 Mar 2025 03:42:52 +0000 (23:42 -0400)] 
docs/qapidoc: implement transmogrify() method

This is the true top-level processor for the new transmogrifier;
responsible both for generating the intermediate rST and then running
the nested parse on that generated document to produce the final
docutils tree that is then - very finally - postprocessed by sphinx for
final rendering to HTML &c.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-55-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Use the opportunity to move the __version__ assignment to where
PEP 8 wants it]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_entity()
John Snow [Tue, 11 Mar 2025 03:42:51 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_entity()

Finally, the core entry method for a qapi entity.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-54-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_sections() method
John Snow [Tue, 11 Mar 2025 03:42:50 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_sections() method

Implement the actual main dispatch method that processes and handles the
list of doc sections for a given QAPI entity.

Process doc sections in strict source order. This is good; reordering
doc text is undesirable. Improvement over the old doc generator, which
can reorder doc comments that don't adhere to (largely unspoken)
conventions.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-53-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
[Commit message extended]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_member() method
John Snow [Tue, 11 Mar 2025 03:42:49 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_member() method

This method is used for generating the "members" of a wide variety of
things, including structs, unions, enums, alternates, etc. The field
name it uses to do so is dependent on the type of entity the "member"
belongs to.

Currently, IF conditionals for individual members are not handled or
rendered, a small regression from the prior documentation
generator. This will be fixed in a future patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-52-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_returns() method
John Snow [Tue, 11 Mar 2025 03:42:48 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_returns() method

Generates :return: fields for explicit returns statements. Note that
this does not presently handle undocumented returns, which is handled in
a later commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-51-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: prepare to record entity being transmogrified
John Snow [Tue, 11 Mar 2025 03:42:47 +0000 (23:42 -0400)] 
docs/qapidoc: prepare to record entity being transmogrified

Prepare to keep a record of which entity we're working on documenting
for the purposes of being able to change certain generative features
conditionally and create stronger assertions.

If you find yourself asking: "Wait, but where does the current entity
actually get recorded?!", you're right! That part comes with the
visit_entity() implementation, which gets added later.

This patch is front-loaded for the sake of type checking in the
forthcoming commits before visit_entity() is ready to be added.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-50-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_feature() method
John Snow [Tue, 11 Mar 2025 03:42:46 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_feature() method

This adds a simple ":feat name: lorem ipsum ..." line to the generated
rST document, so at the moment it's only for "top level" features.

Features not attached directly to a QAPI definition are not currently
handled! This is a small regression over the prior documentation
generator that will be addressed in a future patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-49-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add add_field() and generate_field() helper methods
John Snow [Tue, 11 Mar 2025 03:42:45 +0000 (23:42 -0400)] 
docs/qapidoc: add add_field() and generate_field() helper methods

These are simple rST generation methods that assist in getting the types
and formatting correct for a field list entry. add_field() is a more
raw, direct call while generate_field() is intended to be used for
generating the correct field from a member object.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-48-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add format_type() method
John Snow [Tue, 11 Mar 2025 03:42:44 +0000 (23:42 -0400)] 
docs/qapidoc: add format_type() method

This method is responsible for generating a type name for a given member
with the correct annotations for the QAPI domain. Features and enums do
not *have* types, so they return None. Everything else returns the type
name with a "?" suffix if that type is optional, and ensconced in
[brackets] if it's an array type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-47-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_errors() method
John Snow [Tue, 11 Mar 2025 03:42:43 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_errors() method

Notably, this method does not currently address the formatting issues
present with the "errors" section in QAPIDoc and just vomits the text
verbatim into the rST doc, with somewhat inconsistent results.

To be addressed in a future patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-46-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_paragraph() method
John Snow [Tue, 11 Mar 2025 03:42:42 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_paragraph() method

This transforms "formerly known as untagged sections" into our pure
intermediate rST format. These sections are already pure rST, so this
method doesn't do a whole lot except ensure appropriate newlines.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-45-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add preamble() method
John Snow [Tue, 11 Mar 2025 03:42:41 +0000 (23:42 -0400)] 
docs/qapidoc: add preamble() method

This method adds the options/preamble to each definition block. Notably,
:since: and :ifcond: are added, as are any "special features" such as
:deprecated: and :unstable:.

If conditionals, if attached to special features, are currently
unhandled in this patch and will be addressed at a future date. We
currently do not have any if conditionals attached to special features.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-44-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_freeform() method
John Snow [Tue, 11 Mar 2025 03:42:40 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_freeform() method

Add the transmogrifier implementation for converting freeform doc blocks
to rST.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-43-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agoqapi/source: allow multi-line QAPISourceInfo advancing
John Snow [Tue, 11 Mar 2025 03:42:39 +0000 (23:42 -0400)] 
qapi/source: allow multi-line QAPISourceInfo advancing

This is for the sake of the new rST generator (the "transmogrifier") so
we can advance multiple lines on occasion while keeping the
generated<-->source mappings accurate.

next_line now simply takes an optional n parameter which chooses the
number of lines to advance.

The next patch will use this when converting section syntax in free-form
documentation to more traditional rST section header syntax, which does
not always line up 1:1 for line counts.

For example:

```
 ##
 # = Section     <-- Info is pointing here, "L1"
 #
 # Lorem Ipsum
 ##
```

would be transformed to rST as:

```
=======        <-- L1
Section        <-- L1
=======        <-- L1
               <-- L2
Lorem Ipsum    <-- L3
```

After consuming the single "Section" line from the source, we want to
advance the source pointer to the next non-empty line which requires
jumping by more than one line.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20250311034303.75779-42-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add visit_module() method
John Snow [Tue, 11 Mar 2025 03:42:38 +0000 (23:42 -0400)] 
docs/qapidoc: add visit_module() method

This method annotates the start of a new module, crediting the source
location to the first line of the module file.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-41-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add transmogrifier class stub
John Snow [Tue, 11 Mar 2025 03:42:37 +0000 (23:42 -0400)] 
docs/qapidoc: add transmogrifier class stub

Add the beginnings of the Transmogrifier class by adding the rST
conversion helpers that will be used to build the virtual rST document.

This version of the class does not actually "do anything" yet; each
individual feature is added one-at-a-time in the forthcoming commits.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-40-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: Fix static typing on qapidoc.py
John Snow [Tue, 11 Mar 2025 03:42:35 +0000 (23:42 -0400)] 
docs/qapidoc: Fix static typing on qapidoc.py

Now that the legacy code is factored out, fix up the typing on the
remaining code in qapidoc.py. Add a type ignore to qapi_legacy.py to
prevent the errors there from bleeding out into qapidoc.py.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-38-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: split old implementation into qapidoc_legacy.py
John Snow [Tue, 11 Mar 2025 03:42:34 +0000 (23:42 -0400)] 
docs/qapidoc: split old implementation into qapidoc_legacy.py

This is being done primarily to be able to type check and delint the new
implementation without needing to worry about fixing up the old
implementation.

I'm adding the new implementation into the existing file instead of into
a new file so that when the dust settles, qapidoc.py will contain the
full history of development on this generative module.

This patch *should* be pure motion, give or take the import statements.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-37-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapidoc: add transmogrifier stub
John Snow [Tue, 11 Mar 2025 03:42:33 +0000 (23:42 -0400)] 
docs/qapidoc: add transmogrifier stub

This commit adds a stubbed option to the qapi-doc directive that opts-in
to the new rST generator; the implementation of which will follow in
subsequent commits.

Once all QAPI documents have been converted, this option and the old
qapidoc implementation can be dropped.

Note that moving code outside of the try...except block has no impact
because the code moved outside of that block does not ever raise a
QAPIError.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-36-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agoqapi/schema: add __repr__ to QAPIDoc.Section
John Snow [Tue, 11 Mar 2025 03:42:32 +0000 (23:42 -0400)] 
qapi/schema: add __repr__ to QAPIDoc.Section

Makes debugging far more pleasant when you can just print(section) and
get something reasonable to display.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-35-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agoqapi: clean up encoding of section kinds
John Snow [Tue, 11 Mar 2025 03:42:31 +0000 (23:42 -0400)] 
qapi: clean up encoding of section kinds

We have several kinds of sections, and to tell them apart, we use
Section attribute @tag and also the section object's Python type:

              type        @tag
    untagged  Section     None
    @foo:     ArgSection  'foo'
    Returns:  Section     'Returns'
    Errors:   Section     'Errors'
    Since:    Section     'Since'
    TODO:     Section     'TODO'

Note:

* @foo can be a member or a feature description, depending on context.

* tag == 'Since' can be a Since: section or a member or feature
  description.  If it's a Section, it's the former, and if it's an
  ArgSection, it's the latter.

Clean this up as follows.  Move the member or feature name to new
ArgSection attribute @name, and replace @tag by enum @kind like this:

              type         kind     name
    untagged  Section      PLAIN
    @foo:     ArgSection   MEMBER   'foo'   if member or argument
              ArgSection   FEATURE  'foo'   if feature
    Returns:  Section      RETURNS
    Errors:   Section      ERRORS
    Since:    Section      SINCE
    TODO:     Section      TODO

The qapi-schema tests are updated to account for the new section names;
"TODO" becomes "Todo" and `None` becomes "Plain" there.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-34-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agoqapi/parser: adjust info location for doc body section
John Snow [Tue, 11 Mar 2025 03:42:30 +0000 (23:42 -0400)] 
qapi/parser: adjust info location for doc body section

Instead of using the info object for the doc block as a whole (which
always points to the very first line of the block), update the info
pointer for each call to ensure_untagged_section when the existing
section is otherwise empty. This way, Sphinx error information will
match precisely to where the text actually starts.

For example, this patch will move the info pointer for the "Hello!"
untagged section ...

> ##       <-- from here ...
> # Hello! <-- ... to here.
> ##

This doesn't seem to improve error reporting now. It will with the
forthcoming QAPI doc transmogrifier.

If I stick bad rST into qapi/block-core.json like this:

>  ##
>  # @SnapshotInfo:
>  #
> +# rST syntax error: *ahh!
> +#
>  # @id: unique shapshot id
>  #
>  # @name: user chosen name

The existing code's error message will point to the beginning of the doc
comment, which is less than helpful. The transmogrifier's message will
point to the erroneous line, but to accomplish this, it needs this
patch.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-33-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: Fix error context reporting in Sphinx 5.x and 6.x
John Snow [Tue, 11 Mar 2025 03:42:29 +0000 (23:42 -0400)] 
docs/qapi-domain: Fix error context reporting in Sphinx 5.x and 6.x

Sphinx 5.3.0 to Sphinx 6.2.0 has a bug where nested content in an
ObjectDescription content block has its error position reported
incorrectly due to an oversight when they added nested section support
to this directive.

(This bug is present in Sphinx's own Python and C domains; test it
yourself by creating a py:func directive and creating a syntax error in
the directive's content block. The reporting will be incorrect.)

To avoid overriding and re-implementing the entirety of the run()
method, a workaround is employed where we parse the content block
ourselves in before_content(), then null the content block to make
Sphinx's own parsing a no-op. Then, in transform_content (which occurs
after Sphinx's nested parse), we simply swap our own parsed content tree
back in for Sphinx's.

It appears a little tricky, but it's the nicest solution I can find.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-32-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: warn when QAPI domain xrefs fail to resolve
John Snow [Tue, 11 Mar 2025 03:42:28 +0000 (23:42 -0400)] 
docs/qapi-domain: warn when QAPI domain xrefs fail to resolve

This patch adds a warning (which is a build failure under our current
build settings) whenever a QAPI cross-reference fails to resolve.

This applies to any cross-references of the form :qapi:{role}:`foo`,
which covers all of the automatically generated references by the qapi
domain, and any such references that are manually written into the
documentation rst files.

Cross-references of the form `foo` do not use this system, but are
already configured to issue a warning (Again, a build failure) if the
cross-reference isn't found anywhere.

Adds warnings that look like the following:

docs/qapi/index.rst:48: WARNING: qapi:type reference target not found: 'footype' [ref.qapi]
docs/qapi/index.rst:50: WARNING: qapi:mod reference target not found: 'foomod' [ref.qapi]

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-31-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add XREF compatibility goop for Sphinx < 4.1
John Snow [Tue, 11 Mar 2025 03:42:27 +0000 (23:42 -0400)] 
docs/qapi-domain: add XREF compatibility goop for Sphinx < 4.1

Sphinx < 4.1 handles cross-references ... differently. Factor out and
isolate the compatibility goop we need to make cross references work
properly in old versions of Sphinx.

Yes, it's ugly. Yes, it works. No, I don't want to talk about
it.

Understand that this patch exists because of the overflowing love in my
heart.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-30-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add CSS styling
John Snow [Tue, 11 Mar 2025 03:42:26 +0000 (23:42 -0400)] 
docs/qapi-domain: add CSS styling

Improve the general look and feel of generated QAPI docs.

Attempt to limit line lengths to offer a more comfortable measure on
maximized windows, and improve some margin and spacing for field lists.

Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-29-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add type cross-refs to field lists
John Snow [Tue, 11 Mar 2025 03:42:25 +0000 (23:42 -0400)] 
docs/qapi-domain: add type cross-refs to field lists

This commit, finally, adds cross-referencing support to various field
lists; modeled tightly after Sphinx's own Python domain code.

Cross-referencing support is added to type names provided to :arg:,
:memb:, :returns: and :choice:.

:feat:, :error: and :value:, which do not take type names, do not
support this syntax.

The general syntax is simple:

:arg TypeName ArgName: Lorem Ipsum ...

The domain will transform TypeName into :qapi:type:`TypeName` in this
basic case, and also apply the ``literal`` decoration to indicate that
this is a type cross-reference.

For optional arguments, the special "?" suffix is used. Because "*" has
special meaning in rST that would cause parsing errors, we elect to use
"?" instead. The special syntax processing strips this character from
the end of any type name argument and will append ", optional" to the
rendered output, applying the cross-reference only to the actual type
name.

The intent here is that the actual syntax in doc-blocks need not change;
but e.g. qapidoc.py will need to process and transform "@arg foo lorem
ipsum" into ":arg type? foo: lorem ipsum" based on the schema
information. Therefore, nobody should ever actually witness this
intermediate syntax unless they are writing manual documentation or the
doc transmogrifier breaks.

For array arguments, type names can similarly be surrounded by "[]",
which are stripped off and then re-appended outside of the
cross-reference.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-28-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add warnings for malformed field lists
John Snow [Tue, 11 Mar 2025 03:42:24 +0000 (23:42 -0400)] 
docs/qapi-domain: add warnings for malformed field lists

Normally, Sphinx will silently fall back to its standard field list
processing if it doesn't match one of your defined fields. A lot of the
time, that's not what we want - we want to be warned if we goof
something up.

For instance, the canonical argument field list form is:

:arg type name: descr

This form is captured by Sphinx and transformed so that the field label
will become "Arguments:". It's possible to omit the type name and descr
and still have it be processed correctly. However, if you omit the type
name, Sphinx no longer recognizes it:

:arg: this is not recognized.

This will turn into an arbitrary field list entry whose label is "Arg:",
and it otherwise silently fails. You may also see failures for doing
things like using :values: instead of :value:, or :errors: instead of
:error:, and so on. It's also case sensitive, and easy to trip up.

Add a validator that guarantees all field list entries that are the
direct child of an ObjectDescription use only recognized forms of field
lists, and emit a warning (treated as error by default in most build
configurations) whenever we detect one that is goofed up.

However, there's still benefit to allowing arbitrary fields -- they are
after all not a Sphinx invention, but perfectly normal docutils
syntax. Create an allow list for known spellings we don't mind letting
through, but warn against anything else.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-27-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add :ifcond: directive option
John Snow [Tue, 11 Mar 2025 03:42:23 +0000 (23:42 -0400)] 
docs/qapi-domain: add :ifcond: directive option

Add a special :ifcond: option that allows us to annotate the
definition-level conditionals.

The syntax of the argument is currently undefined, but it's possible we
can apply better formatting in the future. Currently, we just display
the ifcond string as preformatted text.

Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-26-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add :unstable: directive option
John Snow [Tue, 11 Mar 2025 03:42:22 +0000 (23:42 -0400)] 
docs/qapi-domain: add :unstable: directive option

Although "unstable" is a feature (and *will* appear in the features
list), add a special :unstable: option to generate an eye-catch that
makes this information very hard to miss.

The forthcoming Transmogrifier in qapidoc.py will add this option
whenever it detects that the features list attached to a definition
contains the "unstable" entry.

Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-25-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add :deprecated: directive option
John Snow [Tue, 11 Mar 2025 03:42:21 +0000 (23:42 -0400)] 
docs/qapi-domain: add :deprecated: directive option

Although "deprecated" is a feature (and *will* appear in the features
list), add a special :deprecated: option to generate an eye-catch that
makes this information very hard to miss.

The forthcoming Transmogrifier in qapidoc.py will add this option
whenever it detects that the features list attached to a definition
contains the "deprecated" entry.

P.S., I outsourced the CSS ;)

Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-24-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add qapi:object directive
John Snow [Tue, 11 Mar 2025 03:42:20 +0000 (23:42 -0400)] 
docs/qapi-domain: add qapi:object directive

Adds the .. qapi:object:: directive, object, and :qapi:obj:`name`
cross-referencing role. This directive is meant to document both structs
and unions.

As per usual, QAPI cross-referencing for types in the member field list
will be added in a forthcoming commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-23-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add qapi:event directive
John Snow [Tue, 11 Mar 2025 03:42:19 +0000 (23:42 -0400)] 
docs/qapi-domain: add qapi:event directive

Adds the .. qapi:event:: directive, object, and :qapi:event:`name`
cross-referencing role.

Adds the :memb type name: field list syntax for documenting event data
members. As this syntax and phrasing will be shared with Structs and
Unions as well, add the field list definition to a shared abstract
class.

As per usual, QAPI cross-referencing for types in the member field list
will be added in a forthcoming commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-22-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add qapi:alternate directive
John Snow [Tue, 11 Mar 2025 03:42:18 +0000 (23:42 -0400)] 
docs/qapi-domain: add qapi:alternate directive

Add the .. qapi:alternate:: directive, object, and qapi:alt:`name`
cross-reference role.

Add the "Alternatives:" field list for describing alternate choices. Like
other field lists that reference QAPI types, a forthcoming commit will
add cross-referencing support to this field.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-21-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add qapi:enum directive
John Snow [Tue, 11 Mar 2025 03:42:17 +0000 (23:42 -0400)] 
docs/qapi-domain: add qapi:enum directive

Add the .. qapi:enum:: directive, object, and :qapi:enum:`name`
cross-reference role.

Add the :value name: field list for documenting Enum values.

Of note, also introduce a new "type" role that is intended to be used by
other QAPI object directives to cross-reference arbitrary QAPI type
names, but will exclude commands, events, and modules from
consideration.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-20-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add "Return:" field lists
John Snow [Tue, 11 Mar 2025 03:42:16 +0000 (23:42 -0400)] 
docs/qapi-domain: add "Return:" field lists

Add "Return:" field list syntax to QAPI Commands.

Like "Arguments:" and "Errors:", the type name isn't currently processed
for cross-referencing, but this will be addressed in a forthcoming
commit.

The syntax of the new field is:

:return TypeName: description
   description cont'd

This patch adds "Return" as a GroupedField, which means that multiple
return values can be annotated - this is only done because Sphinx does
not support mandatory type arguments to Ungrouped fields. Because we
want to cross-reference this type information later, we want to make the
type argument mandatory. As a result, you can technically add multiple
:return: fields, though I'm not aware of any circumstance in which you'd
need or want to. Recommendation: "Don't do that, then." The forthcoming
QAPIDoc transmogrifier does not, in fact, ever "do that".

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-19-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add "Errors:" field lists
John Snow [Tue, 11 Mar 2025 03:42:15 +0000 (23:42 -0400)] 
docs/qapi-domain: add "Errors:" field lists

``:error: descr`` can now be used to document error conditions. The
format of the description is not defined here; so the ability to name
specific types is left to the document writer.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-18-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add "Features:" field lists
John Snow [Tue, 11 Mar 2025 03:42:14 +0000 (23:42 -0400)] 
docs/qapi-domain: add "Features:" field lists

Add support for Features field lists. There is no QAPI-specific
functionality here, but this could be changed if desired (if we wanted
the feature names to link somewhere, for instance.)

This feature list doesn't have any restrictions, so it can be used to
document object-wide features or per-member features as deemed
appropriate. It's essentially free-form text.

The syntax for this field is:

:feat name: description
    description cont'd

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-17-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add "Arguments:" field lists
John Snow [Tue, 11 Mar 2025 03:42:13 +0000 (23:42 -0400)] 
docs/qapi-domain: add "Arguments:" field lists

This adds special rendering for Sphinx's typed info field lists.

This patch does not add any QAPI-aware markup, rendering, or
cross-referencing for the type names, yet. That feature requires a
subclass to TypedField which will happen in its own commit quite a bit
later in this series; after all the basic fields and objects have been
established first.

The syntax for this field is:

:arg type name: description
   description cont'd

You can omit the type or the description. You should not omit the name;
if you do so, it degenerates into a "normal field list" entry, and
probably isn't what you want.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-16-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add :since: directive option
John Snow [Tue, 11 Mar 2025 03:42:12 +0000 (23:42 -0400)] 
docs/qapi-domain: add :since: directive option

Add a little special markup for registering "Since:" information. Adding
it as an option instead of generic content lets us hoist the information
into the Signature bar, optionally put it in the index, etc.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-15-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add qapi:command directive
John Snow [Tue, 11 Mar 2025 03:42:11 +0000 (23:42 -0400)] 
docs/qapi-domain: add qapi:command directive

This commit adds a stubbed version of QAPICommand that utilizes the
QAPIObject class, the qapi:command directive, the :qapi:cmd:
cross-reference role, and the "command" object type in the QAPI object
registry.

They don't do anything *particularly* interesting yet, but that will
come in forthcoming commits.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-14-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add QAPIObject class
John Snow [Tue, 11 Mar 2025 03:42:10 +0000 (23:42 -0400)] 
docs/qapi-domain: add QAPIObject class

This patch adds another abstract class that describes "a QAPI
thingie". The main difference here is that this class will be generating
visible documentation, unlike the QAPIDescription class.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-13-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add qapi:module directive
John Snow [Tue, 11 Mar 2025 03:42:09 +0000 (23:42 -0400)] 
docs/qapi-domain: add qapi:module directive

This adds the qapi:module directive, which just notes the current module
being documented and performs a nested parse of the content block, if
present.

This code is based pretty heavily on Sphinx's PyModule directive, but
with unnecessary features excised.

For example:

.. qapi:module:: block-core

   Hello, and welcome to block-core!
   =================================

   lorem ipsum, dolor sit amet ...

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-12-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: Add QAPIDescription abstract class
John Snow [Tue, 11 Mar 2025 03:42:08 +0000 (23:42 -0400)] 
docs/qapi-domain: Add QAPIDescription abstract class

This class is a generic, top-level directive for documenting some kind
of QAPI thingamajig that we expect to go into the Index. This class
doesn't do much by itself, and it isn't yet associated with any
particular directive.

handle_signature(), _object_hierarchy_parts() and _toc_entry_name() are
defined in the base class. get_index_text() and add_target_and_index()
are new methods defined here; they are based heavily on the layout and
format of the Python domain's general object class.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-11-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add compatibility node classes
John Snow [Tue, 11 Mar 2025 03:42:07 +0000 (23:42 -0400)] 
docs/qapi-domain: add compatibility node classes

Sphinx prior to v4.0 uses different classes for rendering elements of
documentation objects; add some compatibility classes to use the right
node classes conditionally.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-10-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add QAPI xref roles
John Snow [Tue, 11 Mar 2025 03:42:06 +0000 (23:42 -0400)] 
docs/qapi-domain: add QAPI xref roles

Add domain-specific cross-reference syntax. As of this commit, that
means new :qapi:any:`block-core` referencing syntax.

The :any: role will find anything registered to the QAPI domain,
including modules, commands, events, etc.

Creating the cross-references is powered by the QAPIXRefRole class;
resolving them is handled by QAPIDomain.resolve_xref().

QAPIXrefRole is based heavily on Sphinx's own PyXrefRole, with
modifications necessary for QAPI features.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-9-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add resolve_any_xref()
John Snow [Tue, 11 Mar 2025 03:42:05 +0000 (23:42 -0400)] 
docs/qapi-domain: add resolve_any_xref()

Add the ability to resolve cross-references using the `any`
cross-reference syntax. Adding QAPI-specific cross-reference roles will
be added in a forthcoming commit, and will share the same find_obj()
helper.

(There's less code needed for the generic cross-reference resolver, so
it comes first in this series.)

Once again, this code is based very heavily on sphinx.domains.python.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-8-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add QAPI index
John Snow [Tue, 11 Mar 2025 03:42:04 +0000 (23:42 -0400)] 
docs/qapi-domain: add QAPI index

Use the QAPI object registry to generate a special index just for QAPI
definitions. The index can show entries both by definition type and all
together, alphabetically.

The index can be linked from anywhere in the QEMU manual by using the
reference `qapi-index`.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-7-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/qapi-domain: add QAPI domain object registry
John Snow [Tue, 11 Mar 2025 03:42:03 +0000 (23:42 -0400)] 
docs/qapi-domain: add QAPI domain object registry

This is the first step towards QAPI domain cross-references and a QAPI
reference index.

This patch just creates the object registry, and updates the
merge_domaindata stub method now that we have actual data we may need to
merge.

Note that how to handle merge conflict resolution is unhandled, as the
Sphinx python domain itself does not handle it either. I do not know how
to intentionally trigger it, so I've left an assertion instead if it
should ever come up ...

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-6-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/sphinx: add compat.py module and nested_parse helper
John Snow [Tue, 11 Mar 2025 03:42:02 +0000 (23:42 -0400)] 
docs/sphinx: add compat.py module and nested_parse helper

Create a compat module that handles sphinx cross-version compatibility
issues. For the inaugural function, add a nested_parse_with_titles()
helper that handles differences in line number tracking for nested
directive body parsing.

Spoilers: there are more cross-version hacks to come throughout the
series.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-5-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agodocs/sphinx: create QAPI domain extension stub
John Snow [Tue, 11 Mar 2025 03:42:01 +0000 (23:42 -0400)] 
docs/sphinx: create QAPI domain extension stub

A Sphinx domain is a collection of directive and role extensions meant
to facilitate the documentation of a specific language. For instance,
Sphinx ships with "python" and "cpp" domains. This patch introduces a
stub for the "qapi" language domain.

Please see https://www.sphinx-doc.org/en/master/usage/domains/index.html
for more information.

This stub doesn't really do anything yet, we'll get to it brick-by-brick
in the forthcoming commits to keep the series breezy and the git history
informative.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-4-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
3 months agotrace/control-target: cleanup headers and make compilation unit common
Pierrick Bouvier [Wed, 5 Mar 2025 05:07:16 +0000 (21:07 -0800)] 
trace/control-target: cleanup headers and make compilation unit common

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250305050716.3460989-1-pierrick.bouvier@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
3 months agoMerge tag 'migration-20250310-pull-request' of https://gitlab.com/farosas/qemu into...
Stefan Hajnoczi [Tue, 11 Mar 2025 01:32:07 +0000 (09:32 +0800)] 
Merge tag 'migration-20250310-pull-request' of https://gitlab.com/farosas/qemu into staging

Migration pull request

- Fix use-after-free in incoming migration
- Improve cpr migration blocker for volatile ram
- Fix RDMA migration

# -----BEGIN PGP SIGNATURE-----
#
# iQJEBAABCAAuFiEEqhtIsKIjJqWkw2TPx5jcdBvsMZ0FAmfPaCAQHGZhcm9zYXNA
# c3VzZS5kZQAKCRDHmNx0G+wxnQy9EADRp/6GaSzoqWgafU8DGM5Q69HyKiZ888DZ
# 7qXqJeH3c95nvOnIw2BMhUYX4t8kkAbUcWlr7L8KCjZT/6N/d1/Z5fimqymRkw4x
# +8kDyADv5FY0339aMLf3qBbIAQj/gvPvg8H+e+hXfokZqoYgLXZ0eqNAz8MjIcyN
# +A+waEBMLNvTgZyTQl2TbCvb+mbRial8u8C9BIoILhn/gNuoMX7lbt0tq41HZwe0
# l3v16jnXlsDvQUXp99bGySomRgkcYqdAt+HWHLje3frT/Ap8dGaUJKlpgJ8DXJiA
# fV1reKihJdj37q9GSG8cR02W+ATBesiecufV4TUPNQYQzTdxn3fOMwdc3Pck074D
# YAQxFT20OPou+NRxjYoHT/GqFUY36/2qBJpt7TY3ramdklHJhXpRyedK4rppTZNn
# pC3lnbpA/LHRmfD1Nh0CRmqZpbV+qW1BWEgMwk4qui46BxYWHxKHFpxAuwlJQmcw
# RxY8qPhIXQM03tiTgIddBNDZLoVqRoUP7YpzR7MMa1rz0T5inNFMcNGm72WpKODE
# rzpw4ezXO7+D4/QmMq3PoPfhFv3QFnH6jaGj8JkJM378KLvh4fQ0woXtDKFl4Tbq
# 1oBZ17WUv6aHr75b+KMyKJNLinvMu5WF5WoRYIt1lNXaqk7I494yvIjtRrimWZIS
# Z5Q0tpUmpw==
# =yEH0
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 11 Mar 2025 06:30:56 HKT
# gpg:                using RSA key AA1B48B0A22326A5A4C364CFC798DC741BEC319D
# gpg:                issuer "farosas@suse.de"
# gpg: Good signature from "Fabiano Rosas <farosas@suse.de>" [unknown]
# gpg:                 aka "Fabiano Almeida Rosas <fabiano.rosas@suse.com>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: AA1B 48B0 A223 26A5 A4C3  64CF C798 DC74 1BEC 319D

* tag 'migration-20250310-pull-request' of https://gitlab.com/farosas/qemu:
  migration: Prioritize RDMA in ram_save_target_page()
  migration: ram block cpr blockers
  migration: Fix UAF for incoming migration on MigrationState

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
3 months agoMerge tag 'pull-xen-20250310' of https://xenbits.xen.org/git-http/people/aperard...
Stefan Hajnoczi [Tue, 11 Mar 2025 01:31:36 +0000 (09:31 +0800)] 
Merge tag 'pull-xen-20250310' of https://xenbits.xen.org/git-http/people/aperard/qemu-dm into staging

Xen queue:

* xen/passthrough: use gsi to map pirq when dom0 is PVH
* Fix missing xenstore node from xen-block backend
* Fix xen mapcache extraneous invalidate

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEE+AwAYwjiLP2KkueYDPVXL9f7Va8FAmfO+nEACgkQDPVXL9f7
# Va+QYggA9dmxMGDO05UEd2ZPv/Goub37Le44qBN4oeXizVRZgGUs2w9ETBXhPZus
# 34aI8CTID4fcH4rgF4LgJ4XuyOxYwP1ot8EpDHQg+ji2nyHeMpAyePTfubprq17U
# APN6Qqefd9X+TX+W9zUS5jV/AXO+apGX+tmVkVexFuy4gSRGSVCPoibHePtoLH9G
# 3rSREjdEx7ByY6ieCV5x3zHPp5tmnLWeHpNCVc5x6NplBslQduBz6vOqLNWB1LKO
# 3a/lYcvTn9PIla1zpvGNbeTsPv2lcdx3SccThcZmyTv2PDm1kzyUOIo1lSIP6bb3
# LjCl3dm1mfxAGEaZ+//rsRhTH8d5ew==
# =K79y
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 10 Mar 2025 22:42:57 HKT
# gpg:                using RSA key F80C006308E22CFD8A92E7980CF5572FD7FB55AF
# gpg: Good signature from "Anthony PERARD <anthony.perard@gmail.com>" [unknown]
# gpg:                 aka "Anthony PERARD <anthony.perard@vates.tech>" [unknown]
# gpg:                 aka "Anthony PERARD <anthony@xenproject.org>" [unknown]
# gpg:                 aka "Anthony PERARD <anthony.perard@citrix.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 5379 2F71 024C 600F 778A  7161 D8D5 7199 DF83 42C8
#      Subkey fingerprint: F80C 0063 08E2 2CFD 8A92  E798 0CF5 572F D7FB 55AF

* tag 'pull-xen-20250310' of https://xenbits.xen.org/git-http/people/aperard/qemu-dm:
  xen: No need to flush the mapcache for grants
  hw/xen: Add "mode" parameter to xen-block devices
  xen/passthrough: use gsi to map pirq when dom0 is PVH

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
3 months agoMerge tag 'net-pull-request' of https://github.com/jasowang/qemu into staging
Stefan Hajnoczi [Tue, 11 Mar 2025 01:26:40 +0000 (09:26 +0800)] 
Merge tag 'net-pull-request' of https://github.com/jasowang/qemu into staging

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEIV1G9IJGaJ7HfzVi7wSWWzmNYhEFAmfO1zkACgkQ7wSWWzmN
# YhET+wf+PkaGeFTNUrOtWpl35fSMKlmOVbb1fkPfuhVBmeY2Vh1EIN3OjqnzdV0F
# wxpuk+wwmFiuV1n6RNuMHQ0nz1mhgsSlZh93N5rArC/PUr3iViaT0cb82RjwxhaI
# RODBhhy7V9WxEhT9hR8sCP2ky2mrKgcYbjiIEw+IvFZOVQa58rMr2h/cbAb/iH4l
# 7T9Wba03JBqOS6qgzSFZOMxvqnYdVjhqXN8M6W9ngRJOjPEAkTB6Evwep6anRjcM
# mCUOgkf2sgQwKve8pYAeTMkzXFctvTc/qCU4ZbN8XcoKVVxe2jllGQqdOpMskPEf
# slOuINeW5M0K5gyjsb/huqcOTfDI2A==
# =/Y0+
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 10 Mar 2025 20:12:41 HKT
# gpg:                using RSA key 215D46F48246689EC77F3562EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>" [full]
# Primary key fingerprint: 215D 46F4 8246 689E C77F  3562 EF04 965B 398D 6211

* tag 'net-pull-request' of https://github.com/jasowang/qemu:
  tap-linux: Open ipvtap and macvtap
  Revert "hw/net/net_tx_pkt: Fix overrun in update_sctp_checksum()"
  util/iov: Do not assert offset is in iov
  net: move backend cleanup to NIC cleanup
  net: parameterize the removing client from nc list

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
3 months agoMerge tag 'pull-10.0-for-softfreeze-100325-3' of https://gitlab.com/stsquad/qemu...
Stefan Hajnoczi [Tue, 11 Mar 2025 01:26:14 +0000 (09:26 +0800)] 
Merge tag 'pull-10.0-for-softfreeze-100325-3' of https://gitlab.com/stsquad/qemu into staging

functional and tcg tests, plugins and MAINTAINERS

  - update and expand aarch64 GPU tests
  - fix build dependence for plugins
  - update libvirt-ci to vulkan-tools
  - allow plugin tests to run on non-POSIX systems
  - tweak test/vm times
  - mark test-vma as linux only
  - various compiler fixes for tcg tests
  - add gitlab build unit tracker
  - error out early on stalled RME tests
  - compile core plugin code once
  - update MAINTAINERS

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmfOwjcACgkQ+9DbCVqe
# KkRkcwgAlRTflCqYlZxdlo4CiOSXaHAFr8yfwWq138LJQRQH530JZnz1lZtxTbEM
# pXT7ixnuJQDMybCQJmvUlK5UTUkZhGS3VuAR1VeM2J8/3VXYzf5sFjZ7yko9mA8S
# 2FX8vdfbko8/J31+lKccA0tpbHyi2AbMR+mO8xj6KZQoePwmHoRmhgH7p7LE35YO
# 8ytaOjMwTKF5fReVK+tlcrVJHFMdGsGNwtsnB2FhhVjI56fQqyM5hGXfOING2Fx3
# iZH3rjzfDB4SWbBqaRsMgH9RXjuB9Eo4v0Qs5ve5SjDyzRJk+/CbbBJ4oRt9hurJ
# bA+CYZuNLGBf8Z/mUeYMavA7rxT5rw==
# =qobU
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 10 Mar 2025 18:43:03 HKT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-10.0-for-softfreeze-100325-3' of https://gitlab.com/stsquad/qemu: (31 commits)
  MAINTAINERS: remove widely sanctioned entities
  plugins/core: make a single build unit
  plugins/api: build only once
  plugins/api: split out time control helpers
  plugins/api: split out the vaddr/hwaddr helpers
  plugins/api: split out binary path/start/end/entry code
  plugins/loader: compile loader only once
  plugins/plugin.h: include queue.h
  plugins/api: clean-up the includes
  include/qemu: plugin-memory.h doesn't need cpu-defs.h
  plugins/loader: populate target_name with target_name()
  plugins/api: use qemu_target_page_mask() to get value
  tests/functional: add boot error detection for RME tests
  gitlab: add a new build_unit job to track build size
  tests/tcg: Suppress compiler false-positive warning on sha1.c
  tests/tcg: enable -fwrapv for test-i386-bmi
  tests/tcg: fix constraints in test-i386-adcox
  tests/tcg: add message to _Static_assert in test-avx
  tests/tcg: mark test-vma as a linux-only test
  tests/vm: bump timeout for shutdown
  ...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
3 months agomigration: Prioritize RDMA in ram_save_target_page()
Li Zhijian [Wed, 5 Mar 2025 06:28:20 +0000 (14:28 +0800)] 
migration: Prioritize RDMA in ram_save_target_page()

Address an error in RDMA-based migration by ensuring RDMA is prioritized
when saving pages in `ram_save_target_page()`.

Previously, the RDMA protocol's page-saving step was placed after other
protocols due to a refactoring in commit bc38dc2f5f3. This led to migration
failures characterized by unknown control messages and state loading errors
destination:
(qemu) qemu-system-x86_64: Unknown control message QEMU FILE
qemu-system-x86_64: error while loading state section id 1(ram)
qemu-system-x86_64: load of migration failed: Operation not permitted
source:
(qemu) qemu-system-x86_64: RDMA is in an error state waiting migration to abort!
qemu-system-x86_64: failed to save SaveStateEntry with id(name): 1(ram): -1
qemu-system-x86_64: rdma migration: recv polling control error!
qemu-system-x86_64: warning: Early error. Sending error.
qemu-system-x86_64: warning: rdma migration: send polling control error

RDMA migration implemented its own protocol/method to send pages to
destination side, hand over to RDMA first to prevent pages being saved by
other protocol.

Fixes: bc38dc2f5f3 ("migration: refactor ram_save_target_page functions")
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
Message-ID: <20250305062825.772629-2-lizhijian@fujitsu.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
3 months agomigration: ram block cpr blockers
Steve Sistare [Thu, 27 Feb 2025 14:48:01 +0000 (06:48 -0800)] 
migration: ram block cpr blockers

Unlike cpr-reboot mode, cpr-transfer mode cannot save volatile ram blocks
in the migration stream file and recreate them later, because the physical
memory for the blocks is pinned and registered for vfio.  Add a blocker
for volatile ram blocks.

Also add a blocker for RAM_GUEST_MEMFD.  Preserving guest_memfd may be
sufficient for CPR, but it has not been tested yet.

Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-ID: <1740667681-257312-1-git-send-email-steven.sistare@oracle.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
3 months agomigration: Fix UAF for incoming migration on MigrationState
Peter Xu [Thu, 20 Feb 2025 13:24:59 +0000 (08:24 -0500)] 
migration: Fix UAF for incoming migration on MigrationState

On the incoming migration side, QEMU uses a coroutine to load all the VM
states.  Inside, it may reference MigrationState on global states like
migration capabilities, parameters, error state, shared mutexes and more.

However there's nothing yet to make sure MigrationState won't get
destroyed (e.g. after migration_shutdown()).  Meanwhile there's also no API
available to remove the incoming coroutine in migration_shutdown(),
avoiding it to access the freed elements.

There's a bug report showing this can happen and crash dest QEMU when
migration is cancelled on source.

When it happens, the dest main thread is trying to cleanup everything:

  #0  qemu_aio_coroutine_enter
  #1  aio_dispatch_handler
  #2  aio_poll
  #3  monitor_cleanup
  #4  qemu_cleanup
  #5  qemu_default_main

Then it found the migration incoming coroutine, schedule it (even after
migration_shutdown()), causing crash:

  #0  __pthread_kill_implementation
  #1  __pthread_kill_internal
  #2  __GI_raise
  #3  __GI_abort
  #4  __assert_fail_base
  #5  __assert_fail
  #6  qemu_mutex_lock_impl
  #7  qemu_lockable_mutex_lock
  #8  qemu_lockable_lock
  #9  qemu_lockable_auto_lock
  #10 migrate_set_error
  #11 process_incoming_migration_co
  #12 coroutine_trampoline

To fix it, take a refcount after an incoming setup is properly done when
qmp_migrate_incoming() succeeded the 1st time.  As it's during a QMP
handler which needs BQL, it means the main loop is still alive (without
going into cleanups, which also needs BQL).

Releasing the refcount now only until the incoming migration coroutine
finished or failed.  Hence the refcount is valid for both (1) setup phase
of incoming ports, mostly IO watches (e.g. qio_channel_add_watch_full()),
and (2) the incoming coroutine itself (process_incoming_migration_co()).

Note that we can't unref in migration_incoming_state_destroy(), because
both qmp_xen_load_devices_state() and load_snapshot() will use it without
an incoming migration.  Those hold BQL so they're not prone to this issue.

PS: I suspect nobody uses Xen's command at all, as it didn't register yank,
hence AFAIU the command should crash on master when trying to unregister
yank in migration_incoming_state_destroy()..  but that's another story.

Also note that in some incoming failure cases we may not always unref the
MigrationState refcount, which is a trade-off to keep things simple.  We
could make it accurate, but it can be an overkill.  Some examples:

  - Unlike most of the rest protocols, socket_start_incoming_migration()
    may create net listener after incoming port setup sucessfully.
    It means we can't unref in migration_channel_process_incoming() as a
    generic path because socket protocol might keep using MigrationState.

  - For either socket or file, multiple IO watches might be created, it
    means logically each IO watch needs to take one refcount for
    MigrationState so as to be 100% accurate on ownership of refcount taken.

In general, we at least need per-protocol handling to make it accurate,
which can be an overkill if we know incoming failed after all.  Add a short
comment to explain that when taking the refcount in qmp_migrate_incoming().

Bugzilla: https://issues.redhat.com/browse/RHEL-69775
Tested-by: Yan Fu <yafu@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Message-ID: <20250220132459.512610-1-peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
3 months agoxen: No need to flush the mapcache for grants
Stefano Stabellini [Thu, 6 Feb 2025 19:49:15 +0000 (20:49 +0100)] 
xen: No need to flush the mapcache for grants

On IOREQ_TYPE_INVALIDATE we need to invalidate the mapcache for regular
mappings. Since recently we started reusing the mapcache also to keep
track of grants mappings. However, there is no need to remove grant
mappings on IOREQ_TYPE_INVALIDATE requests, we shouldn't do that. So
remove the function call.

Fixes: 9ecdd4bf08 (xen: mapcache: Add support for grant mappings)
Cc: qemu-stable@nongnu.org
Reported-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Message-Id: <20250206194915.3357743-2-edgar.iglesias@gmail.com>
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
3 months agohw/xen: Add "mode" parameter to xen-block devices
David Woodhouse [Fri, 7 Feb 2025 14:37:24 +0000 (14:37 +0000)] 
hw/xen: Add "mode" parameter to xen-block devices

Block devices don't work in PV Grub (0.9x) if there is no mode specified. It
complains: "Error ENOENT when reading the mode"

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Message-Id: <20250207143724.30792-2-dwmw2@infradead.org>
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
3 months agoxen/passthrough: use gsi to map pirq when dom0 is PVH
Jiqian Chen [Wed, 6 Nov 2024 06:14:18 +0000 (14:14 +0800)] 
xen/passthrough: use gsi to map pirq when dom0 is PVH

In PVH dom0, when passthrough a device to domU, QEMU code
xen_pt_realize->xc_physdev_map_pirq wants to use gsi, but in current codes
the gsi number is got from file /sys/bus/pci/devices/<sbdf>/irq, that is
wrong, because irq is not equal with gsi, they are in different spaces, so
pirq mapping fails.

To solve above problem, use new interface of Xen, xc_pcidev_get_gsi to get
gsi and use xc_physdev_map_pirq_gsi to map pirq when dom0 is PVH.

Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Acked-by: Anthony PERARD <anthony@xenproject.org>
Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Message-Id: <20241106061418.3655304-1-Jiqian.Chen@amd.com>
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
3 months agoMAINTAINERS: remove widely sanctioned entities
Alex Bennée [Tue, 4 Mar 2025 22:24:38 +0000 (22:24 +0000)] 
MAINTAINERS: remove widely sanctioned entities

The following organisations appear on the US sanctions list:

  Yadro: https://sanctionssearch.ofac.treas.gov/Details.aspx?id=41125
  ISPRAS: https://sanctionssearch.ofac.treas.gov/Details.aspx?id=50890

As a result maintainers interacting with such entities would face
legal risk in a number of jurisdictions. To reduce the risk of
inadvertent non-compliance remove entries from these organisations
from the MAINTAINERS file.

Mark the pcf8574 system as orphaned until someone volunteers to step
up as a maintainer. Add myself as a second reviewer to record/replay
so I can help with what odd fixes I can.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-32-alex.bennee@linaro.org>

3 months agoplugins/core: make a single build unit
Alex Bennée [Tue, 4 Mar 2025 22:24:37 +0000 (22:24 +0000)] 
plugins/core: make a single build unit

Trim through the includes and remove everything not needed for the
core. Only include tcg-op-common.h to remove the need to
TARGET_LONG_BITS and move the build unit into the common set.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-31-alex.bennee@linaro.org>

3 months agoplugins/api: build only once
Alex Bennée [Tue, 4 Mar 2025 22:24:36 +0000 (22:24 +0000)] 
plugins/api: build only once

Now all the softmmu/user-mode stuff has been split out we can build
this compilation unit only once.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-30-alex.bennee@linaro.org>

3 months agoplugins/api: split out time control helpers
Alex Bennée [Tue, 4 Mar 2025 22:24:35 +0000 (22:24 +0000)] 
plugins/api: split out time control helpers

These are only usable in system mode where we control the timer. For
user-mode make them NOPs.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-29-alex.bennee@linaro.org>

3 months agoplugins/api: split out the vaddr/hwaddr helpers
Alex Bennée [Tue, 4 Mar 2025 22:24:34 +0000 (22:24 +0000)] 
plugins/api: split out the vaddr/hwaddr helpers

These only work for system-mode and are NOPs for user-mode.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-28-alex.bennee@linaro.org>

3 months agoplugins/api: split out binary path/start/end/entry code
Alex Bennée [Tue, 4 Mar 2025 22:24:33 +0000 (22:24 +0000)] 
plugins/api: split out binary path/start/end/entry code

To move the main api.c to a single build compilation object we need to
start splitting out user and system specific code. As we need to grob
around host headers we move these particular helpers into the *-user
mode directories.

The binary/start/end/entry helpers are all NOPs for system mode.

While using the plugin-api.c.inc trick means we build for both
linux-user and bsd-user the BSD user-mode command line is still
missing -plugin. This can be enabled once we have reliable check-tcg
tests working for the BSDs.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-27-alex.bennee@linaro.org>

3 months agoplugins/loader: compile loader only once
Alex Bennée [Tue, 4 Mar 2025 22:24:32 +0000 (22:24 +0000)] 
plugins/loader: compile loader only once

There is very little in loader that is different between builds save
for a tiny user/system mode difference in the plugin_info structure.
Create two new files, user and system to hold mode specific helpers
and move loader into common_ss.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-26-alex.bennee@linaro.org>

3 months agoplugins/plugin.h: include queue.h
Alex Bennée [Tue, 4 Mar 2025 22:24:31 +0000 (22:24 +0000)] 
plugins/plugin.h: include queue.h

Headers should bring in what they need so don't rely on getting
queue.h by side effects. This will help with clean-ups in the
following patches.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-25-alex.bennee@linaro.org>

3 months agoplugins/api: clean-up the includes
Alex Bennée [Tue, 4 Mar 2025 22:24:30 +0000 (22:24 +0000)] 
plugins/api: clean-up the includes

Thanks to re-factoring and clean-up work (especially to exec-all) we
no longer need such broad headers for the api.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-24-alex.bennee@linaro.org>

3 months agoinclude/qemu: plugin-memory.h doesn't need cpu-defs.h
Alex Bennée [Tue, 4 Mar 2025 22:24:29 +0000 (22:24 +0000)] 
include/qemu: plugin-memory.h doesn't need cpu-defs.h

hwaddr is a fixed size on all builds.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-23-alex.bennee@linaro.org>

3 months agoplugins/loader: populate target_name with target_name()
Alex Bennée [Tue, 4 Mar 2025 22:24:28 +0000 (22:24 +0000)] 
plugins/loader: populate target_name with target_name()

We have a function we can call for this, lets not rely on macros that
stop us building once.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-22-alex.bennee@linaro.org>

3 months agoplugins/api: use qemu_target_page_mask() to get value
Alex Bennée [Tue, 4 Mar 2025 22:24:27 +0000 (22:24 +0000)] 
plugins/api: use qemu_target_page_mask() to get value

Requiring TARGET_PAGE_MASK to be defined gets in the way of building
this unit once. qemu_target_page_mask() will tell us what it is.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-21-alex.bennee@linaro.org>

3 months agotests/functional: add boot error detection for RME tests
Pierrick Bouvier [Tue, 4 Mar 2025 22:24:26 +0000 (22:24 +0000)] 
tests/functional: add boot error detection for RME tests

It was identified that those tests randomly fail with a synchronous
exception at boot (reported by EDK2).
While we solve this problem, report failure immediately so tests don't
timeout in CI.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20250303185745.2504842-1-pierrick.bouvier@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-20-alex.bennee@linaro.org>

3 months agogitlab: add a new build_unit job to track build size
Alex Bennée [Tue, 4 Mar 2025 22:24:25 +0000 (22:24 +0000)] 
gitlab: add a new build_unit job to track build size

We want to reduce the total number of build units in the system to get
on our way to a single binary. It will help to have some numbers so
lets add a job to gitlab to track our progress.

Cc: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-19-alex.bennee@linaro.org>

3 months agotests/tcg: Suppress compiler false-positive warning on sha1.c
Peter Maydell [Tue, 4 Mar 2025 22:24:24 +0000 (22:24 +0000)] 
tests/tcg: Suppress compiler false-positive warning on sha1.c

GCC versions at least 12 through 15 incorrectly report a warning
about code in sha1.c:

tests/tcg/multiarch/sha1.c:161:13: warning: ‘SHA1Transform’ reading 64 bytes from a region of size 0 [-Wstringop-overread]
  161 |             SHA1Transform(context->state, &data[i]);
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is a piece of stock library code for doing SHA1 which we've
simply copied, rather than writing ourselves. The bug has been
reported to upstream GCC (about a different library's use of this
code):
 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106709

For our test case, since this isn't our original code and there isn't
actually a bug in it, suppress the incorrect warning rather than
trying to modify the code to work around the compiler issue.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2328
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20250227141343.1675415-1-peter.maydell@linaro.org>
[AJB: -Wno-unknown-warning-option for clang's sake]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-18-alex.bennee@linaro.org>

3 months agotests/tcg: enable -fwrapv for test-i386-bmi
Alex Bennée [Tue, 4 Mar 2025 22:24:23 +0000 (22:24 +0000)] 
tests/tcg: enable -fwrapv for test-i386-bmi

We allow things like:

  tests/tcg/i386/test-i386-bmi2.c:124:35: warning: shifting a negative signed value is undefined [-Wshift-negative-value]
      assert(result == (mask & ~(-1 << 30)));

in the main code, so allow it for the test.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-17-alex.bennee@linaro.org>

3 months agotests/tcg: fix constraints in test-i386-adcox
Alex Bennée [Tue, 4 Mar 2025 22:24:22 +0000 (22:24 +0000)] 
tests/tcg: fix constraints in test-i386-adcox

Clang complains:

  clang -O2 -m64 -mcx16 /home/alex/lsrc/qemu.git/tests/tcg/i386/test-i386-adcox.c -o test-i386-adcox -static
  /home/alex/lsrc/qemu.git/tests/tcg/i386/test-i386-adcox.c:32:26: error: invalid input constraint '0' in asm
          : "r" ((REG)-1), "0" (flags), "1" (out_adcx), "2" (out_adox));
                           ^
  /home/alex/lsrc/qemu.git/tests/tcg/i386/test-i386-adcox.c:57:26: error: invalid input constraint '0' in asm
          : "r" ((REG)-1), "0" (flags), "1" (out_adcx), "2" (out_adox));
                           ^
  2 errors generated.

Pointing out a numbered input constraint can't point to a read/write
output [1]. Convert to a read-only input constraint to allow this.

[1] https://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20101101/036036.html

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-16-alex.bennee@linaro.org>

3 months agotests/tcg: add message to _Static_assert in test-avx
Alex Bennée [Tue, 4 Mar 2025 22:24:21 +0000 (22:24 +0000)] 
tests/tcg: add message to _Static_assert in test-avx

In preparation for enabling clang and avoiding:

  error: '_Static_assert' with no message is a C2x extension [-Werror,-Wc2x-extensions]

let us just add the message to silence the warning.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-15-alex.bennee@linaro.org>

3 months agotests/tcg: mark test-vma as a linux-only test
Alex Bennée [Tue, 4 Mar 2025 22:24:20 +0000 (22:24 +0000)] 
tests/tcg: mark test-vma as a linux-only test

The main multiarch tests should compile for any POSIX system, however
test-vma's usage of MAP_NORESERVE makes it a linux-only test. Simply
moving the source file is enough for the build logic to skip on BSD's.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-14-alex.bennee@linaro.org>

3 months agotests/vm: bump timeout for shutdown
Alex Bennée [Tue, 4 Mar 2025 22:24:19 +0000 (22:24 +0000)] 
tests/vm: bump timeout for shutdown

On my fairly beefy machine the timeout was triggering leaving a
corrupted disk image due to power being pulled before the disk had
synced. Triple the timeout to avoid this.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-13-alex.bennee@linaro.org>

3 months agolibvirt-ci: bump to latest for vulkan-tools
Alex Bennée [Tue, 4 Mar 2025 22:24:18 +0000 (22:24 +0000)] 
libvirt-ci: bump to latest for vulkan-tools

The alpine baseline has also been updated in the meantime so we need
to address that while we are at it.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-12-alex.bennee@linaro.org>

3 months agotests/functional: Allow running TCG plugins tests on non-Linux/BSD hosts
Philippe Mathieu-Daudé [Tue, 4 Mar 2025 22:24:17 +0000 (22:24 +0000)] 
tests/functional: Allow running TCG plugins tests on non-Linux/BSD hosts

Not all platforms use the '.so' suffix for shared libraries,
which is how plugins are built. Use the recently introduced
dso_suffix() helper to get the proper host suffix.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2804
Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250220080215.49165-4-philmd@linaro.org>
[AJB: moved plugin_file into testcase.py]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-11-alex.bennee@linaro.org>

3 months agotests/functional: Introduce the dso_suffix() helper
Philippe Mathieu-Daudé [Tue, 4 Mar 2025 22:24:16 +0000 (22:24 +0000)] 
tests/functional: Introduce the dso_suffix() helper

Introduce a helper to get the default shared library
suffix used on the host.

Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250220080215.49165-3-philmd@linaro.org>
[AJB: dropped whitespace cmd.py damage]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-10-alex.bennee@linaro.org>

3 months agoplugins: add explicit dependency in functional tests
Pierrick Bouvier [Tue, 4 Mar 2025 22:24:15 +0000 (22:24 +0000)] 
plugins: add explicit dependency in functional tests

./tests/functional/test_aarch64_tcg_plugins.py needs to have plugin
libinsn built. However, it's not listed as a dependency, so meson can't
know it needs to be built.

Thus, we keep track of all plugins, and add them as an explicit
dependency.

Fixes: 4c134d07b9e ("tests: add a new set of tests to exercise plugins")
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-9-alex.bennee@linaro.org>

3 months agotests/functional: update the aarch64_virg_gpu images
Alex Bennée [Tue, 4 Mar 2025 22:24:14 +0000 (22:24 +0000)] 
tests/functional: update the aarch64_virg_gpu images

Update to the most recent aarch64_virt_gpu image. The principle
differences are:

  - target a v8.0 baseline CPU
  - latest vkmark (2025.1)
  - actually uses the rootfs (previously was initrd)
  - rootfs includes more testing tools for interactive use

See README.md in https://fileserver.linaro.org/s/ce5jXBFinPxtEdx for
details about the image creation and the buildroot config.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-8-alex.bennee@linaro.org>

3 months agotests/functional: expand tests to cover virgl
Alex Bennée [Tue, 4 Mar 2025 22:24:13 +0000 (22:24 +0000)] 
tests/functional: expand tests to cover virgl

Add two more test modes using glmark2-wayland to exercise the OpenGL
pass-through modes with virgl. Virgl can run with or without the
hostmem blob support. To avoid repeating ourselves too much we make
the initial pass a simple --validate pass.

We might want to eventually add more directed tests and individual
features later on but the glmark/vkmark tests are a good general
smoke test for accelerated 3D.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-7-alex.bennee@linaro.org>