]> git.ipfire.org Git - thirdparty/u-boot.git/log
thirdparty/u-boot.git
2 months agopatman: Add tests for Cseries
Simon Glass [Sat, 10 May 2025 11:05:17 +0000 (13:05 +0200)] 
patman: Add tests for Cseries

Add various tests for the Cseries functionality, including both direct
and via-cmdline variants.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Implement the upstream subcommand
Simon Glass [Sat, 10 May 2025 11:05:16 +0000 (13:05 +0200)] 
patman: Implement the upstream subcommand

Add a command to allow managing the upstream tree. This is very basic
so far, only allowing setting the name and URL. Further work may allow
checking whether series apply cleaning on the upstream tree, etc.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Implement the series subcommand
Simon Glass [Sat, 10 May 2025 11:05:15 +0000 (13:05 +0200)] 
patman: Implement the series subcommand

Add a command to support management of series. These are local branches
which may have been sent to the mailing list, so may appear on the
patchwork server.

The new functionality uses a database, stored in a local file.

Various operations are supported:
   - add a new series
   - send a series and automatically link it with patchwork
   - 'increment' the version, to prepare to send an updated series
   - gather review/test tags from patchwork
   - check on progress, i.e. patches which are received new tags or
     comments
   - show comments on patches and cover letters

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Update send function to return whether it sent
Simon Glass [Sat, 10 May 2025 11:05:14 +0000 (13:05 +0200)] 
patman: Update send function to return whether it sent

Indicate whether 'git send-email' was actually called, so that we don't
bother waiting for patchwork to receive our series if it wasn't.

The 'git send-email' seems to always return a code of 0 even if nothing
was sent, so we cannot use clues there. Ideally we would watch the
output to determine which patches were sent and which not, but that is
left for another day.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Support aliases for commands and subcommands
Simon Glass [Sat, 10 May 2025 11:05:13 +0000 (13:05 +0200)] 
patman: Support aliases for commands and subcommands

It is laborious to type long commands, so add some aliases to speed up
use of patman. For now, allow 'pw' for patchwork and 'st' for status.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Implement the patchwork subcommand
Simon Glass [Sat, 10 May 2025 11:05:12 +0000 (13:05 +0200)] 
patman: Implement the patchwork subcommand

Add a command to allow setting and getting the patchwork project. This
is needed so that patman can use the correct ID when talking to the
patchwork server.

To support testing, allow passing in the test database, patchwork
object and Cseries object. Fake versions can then easily be provided for
certain test cases.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Improve Series support for patchwork links
Simon Glass [Sat, 10 May 2025 11:05:11 +0000 (13:05 +0200)] 
patman: Improve Series support for patchwork links

Update Series with a way to better manage the Series-links lines in
patches. Use this in the 'status' subcommand instead of the existing
primitive method of expecting a link without a version prefix.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Introduce basic database support in Series
Simon Glass [Sat, 10 May 2025 11:05:10 +0000 (13:05 +0200)] 
patman: Introduce basic database support in Series

This class manages a series, i.e. a group of patches with a possible
cover letter. Add some properties for recording basic patchwork info,
including the database ID.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Support returning the parsers to use
Simon Glass [Sat, 10 May 2025 11:05:09 +0000 (13:05 +0200)] 
patman: Support returning the parsers to use

The parsing tests need to be able to try invalid arguments to make sure
that these are handled correctly. Provide a way to return the parsers
being used, as well as to pass in the parsers to use. This feature is
needed in test_series_no_subcmd(), for example.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Use -c to show patch comments
Simon Glass [Sat, 10 May 2025 11:05:08 +0000 (13:05 +0200)] 
patman: Use -c to show patch comments

We chose -C as the flag to enable showing comments because -c was used
in 'patman send' to specify the patch count. Now that the 'send' parser
is separated we don't need to do this and it is OK to use the same flag
in a different subcommand.

We want to have a flag for cover-letter comments, so it makes most sense
to have -C for that and -c for patch comments. Update the latter.

Put this in a function since the new 'series' command will add this flag
too.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Enhance implementation of file-based defaults
Simon Glass [Sat, 10 May 2025 11:05:07 +0000 (13:05 +0200)] 
patman: Enhance implementation of file-based defaults

Patman allows defaults for its command-line flags to be read from a
file. The implementation of this does not fully work with subcommands,
since we don't want a default for those.

Also, it relies on being able to parse any sort of cmdline in order to
figure out the options that are available. But in some cases, the
cmdline may not parse, e.g. if there are required options, or
conflicting options.

Add a class which can be safely used to parse any cmdline, since it
allows execution to continue even when errors are obtained. Use this to
determine the defaults, being careful to skip sub/commands.

Another way to handle all of this would be to maintain the defaults
separately and insert them into the parser manually. For now, I'm not
sure which is best.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Enhance patchwork interface to support Cseries
Simon Glass [Sat, 10 May 2025 11:05:06 +0000 (13:05 +0200)] 
patman: Enhance patchwork interface to support Cseries

Add various new requests to the Patchwork class, so we can obtain the
required information. This includes cover letters and comments.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Update status command support cover-letter info
Simon Glass [Sat, 10 May 2025 11:05:05 +0000 (13:05 +0200)] 
patman: Update status command support cover-letter info

Add support to the status module for reading and supporting cover
letters, including comments. Plumb this through to the patchwork module.
The actual support in the latter is not yet integrated.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add a Cseries class
Simon Glass [Sat, 10 May 2025 11:05:04 +0000 (13:05 +0200)] 
patman: Add a Cseries class

This is the main class for dealing with series, across branches and the
database.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add a helper for managing cseries
Simon Glass [Sat, 10 May 2025 11:05:03 +0000 (13:05 +0200)] 
patman: Add a helper for managing cseries

Add a module which includes helper functions for dealing with Cseries
objects.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add a simple database implementation
Simon Glass [Sat, 10 May 2025 11:05:02 +0000 (13:05 +0200)] 
patman: Add a simple database implementation

For recording series information, patman needs a database. Add a module
which uses sqlite3 for this. It has a basic schema, enough to support a
series subcommand.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Avoid touching the system or local config
Simon Glass [Sat, 10 May 2025 11:05:01 +0000 (13:05 +0200)] 
patman: Avoid touching the system or local config

Set the git variables to ensure that any local gitconfig is not used or
affected by the tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add a fourth series for testing
Simon Glass [Sat, 10 May 2025 11:05:00 +0000 (13:05 +0200)] 
patman: Add a fourth series for testing

Create one more series to use for testing. This one has four separate
patches.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Move common test code into a new module
Simon Glass [Sat, 10 May 2025 11:04:59 +0000 (13:04 +0200)] 
patman: Move common test code into a new module

The func_test file is quite large. In order to allow new tests to be
added to a separate file, move the common test code into a separate
class, to be inherited by other classes.

Drop unnecessary imports in func_test

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add a test for collecting a PATCH prefix
Simon Glass [Sat, 10 May 2025 11:04:58 +0000 (13:04 +0200)] 
patman: Add a test for collecting a PATCH prefix

Add one more test for completeness, since this occurs commonly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Use python3 with /usr/bin/env
Simon Glass [Sat, 10 May 2025 11:04:57 +0000 (13:04 +0200)] 
patman: Use python3 with /usr/bin/env

If python2 is available then we must specify python3 here. Update the
test to handle this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Add more functions to gitutil
Simon Glass [Sat, 10 May 2025 11:04:56 +0000 (13:04 +0200)] 
u_boot_pylib: Add more functions to gitutil

Add functions for checking a branch, showing a commit, etc. to support
the new functionality.

Git version 2.34.1 ignores --stat if --quiet is given, so adjust the
args so that this performs as expected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Deal with git safe-directory warning
Simon Glass [Sat, 10 May 2025 11:04:55 +0000 (13:04 +0200)] 
patman: Deal with git safe-directory warning

When running tests where the .git directory is not owned by the current
user, various warnings are produced and the tests fail. This happens in
CI.

For patman itself, modify the gitutil.get_top_level() function to return
None in this case. Ensure that the warning is not shown, since it creates
about 1000 lines of output.

For checkpatch, the same warning is produced even though --no-tree is
given. Suppress that as well.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Correct pylint warnings in gitutil
Simon Glass [Sat, 10 May 2025 11:04:54 +0000 (13:04 +0200)] 
u_boot_pylib: Correct pylint warnings in gitutil

Correct various pylint warnings in this file.

The remaining ones are three functions with too many arguments (R0913
and R0918) and use of global (W0603).

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Support collecting all lines in the commit message
Simon Glass [Thu, 8 May 2025 04:38:49 +0000 (06:38 +0200)] 
patman: Support collecting all lines in the commit message

In some cases we want to collect all lines in the commit message so that
the commit can be recreated with the same message as before, or perhaps
with light filtering.

Add support for this.

Series-to: u-boot
Cover-letter:
patman: Minor improvements to prepare for series handling
This series includes a number of internal improvements to patman:
- Tidy-up of parsing
- Adjust how tests create the git tree
- Support for creating patches in a different git directory
- Faster determination of the upstream branch
- Ability to collect the body of a commit message as a series of lines
END

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Tidy up function comments in the series module
Simon Glass [Thu, 8 May 2025 04:35:42 +0000 (06:35 +0200)] 
patman: Tidy up function comments in the series module

This module is missing some comments, so add them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Move -H out of the send command
Simon Glass [Thu, 8 May 2025 04:30:14 +0000 (06:30 +0200)] 
patman: Move -H out of the send command

This is the help for the whole of patman, so move it to the start of the
control function, rather than being inside 'patman send'.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Move arguments for sent into the correct parser
Simon Glass [Thu, 8 May 2025 04:21:33 +0000 (06:21 +0200)] 
patman: Move arguments for sent into the correct parser

Most of the arguments for the main parser are actually arguments for the
'send' parser. Move them there, in a separate function.

Fix a pylint warning for -D and the imports while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Split subparsers into their own functions
Simon Glass [Thu, 8 May 2025 04:09:46 +0000 (06:09 +0200)] 
patman: Split subparsers into their own functions

Simplify the main parser by moving subparser code into separate
functions. Fix a few pylint warnings while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add tests for help
Simon Glass [Thu, 8 May 2025 04:01:28 +0000 (06:01 +0200)] 
patman: Add tests for help

Add a few tests to make sure that help is provided when requested.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Split parser creation from parsing
Simon Glass [Thu, 8 May 2025 03:58:10 +0000 (05:58 +0200)] 
patman: Split parser creation from parsing

Tests may want to parse their own arguments. Refactor the parser code to
support this and allow settings to receive arguments as well.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Correct some pylint and asyncio issues
Simon Glass [Thu, 8 May 2025 03:36:20 +0000 (05:36 +0200)] 
patman: Correct some pylint and asyncio issues

Correct some pylint warnings in this file. Make use of the existing
asyncio event loop, instead of creating a new one, since this also
destroys it afterwards, making it unavailable for tests which want to
share an event loop. Use tools.write_file() to avoid a warning about
encoding.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Use git to set up the test config
Simon Glass [Thu, 8 May 2025 03:02:07 +0000 (05:02 +0200)] 
patman: Use git to set up the test config

At present the branch information is dropped when writing the
configuration. It is easier to get git to set up the config anyway, so
update the test to do this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Clean up creation of the git tree
Simon Glass [Thu, 8 May 2025 03:23:41 +0000 (05:23 +0200)] 
patman: Clean up creation of the git tree

The test starts with the HEAD pointing to the wrong place, so that the
created files appear to be deleted. Fix this by resetting the tree
before tests start. Add a check that the tree is clean.

Update pygit2 so that the enums are available.

2 months agopatman: Use variables for series ID and title
Simon Glass [Thu, 8 May 2025 03:26:16 +0000 (05:26 +0200)] 
patman: Use variables for series ID and title

Rather than hard-coding these values in the sample patches, use
variables so that we can refer to these in tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Use .git as the git directory
Simon Glass [Thu, 8 May 2025 03:13:35 +0000 (05:13 +0200)] 
patman: Use .git as the git directory

In tests, the 'git' directory is a subdirectory of the temporary
directory. Rename it to '.git' so that git will automatically find it
when git operations are done in the temporary directory. Set up the
config before the first git operation, so that this works correctly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Use itr instead of lines for iterator
Simon Glass [Thu, 8 May 2025 02:58:49 +0000 (04:58 +0200)] 
patman: Use itr instead of lines for iterator

The variables 'lines' is used to hold a list of output lines within a
test, but also to hold an iterator through those lines. Use 'itr' for
the latter, to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Correct use of str in code
Simon Glass [Wed, 7 May 2025 16:08:52 +0000 (18:08 +0200)] 
patman: Correct use of str in code

Since str() is a reserved function we should not use it as a variable.
Fix this in the send module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Allow setting a git directory when sending
Simon Glass [Wed, 7 May 2025 16:06:52 +0000 (18:06 +0200)] 
patman: Allow setting a git directory when sending

Support specifying the git-directory when creating and sending patches.
This will allow better testing of this functionality, since we can use a
test directory.

For count_commits_to_branch() support an end commit while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Allow setting the current directory when sending
Simon Glass [Wed, 7 May 2025 16:02:47 +0000 (18:02 +0200)] 
patman: Allow setting the current directory when sending

Plumb a current-working-directory (cwd) through from send all the way to
the command gitutil libraries. This will allow better testing of this
functionality, since we can use a test directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add all files to __init__.py
Simon Glass [Wed, 7 May 2025 15:50:20 +0000 (17:50 +0200)] 
patman: Add all files to __init__.py

Some files are missing from the __all__ list, so add then. Reformat the
list to use more of the width of each line.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Speed up determining the upstream branch
Simon Glass [Thu, 8 May 2025 02:30:55 +0000 (04:30 +0200)] 
u_boot_pylib: Speed up determining the upstream branch

Use --decorate to quickly detect the upstream branch, since this is much
faster than using 'git name-rev' on every possible commit.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Provide directories to gitutil functions
Simon Glass [Thu, 8 May 2025 02:22:18 +0000 (04:22 +0200)] 
u_boot_pylib: Provide directories to gitutil functions

For testing it is useful to be able to set the current directory used
for git operations, as well as the git-repo directory. Update some of
the functions to support this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Tidy up quoting of cc and to
Simon Glass [Thu, 8 May 2025 02:38:30 +0000 (04:38 +0200)] 
u_boot_pylib: Tidy up quoting of cc and to

The current approach to calling 'git send-email' puts double quotes
around each email address to ensure that it will pass the shell
correctly. This is a bit cumbersome and requires using a shell to sort
it all out.

Drop the quotes and use command.run() instead, to simplify things. This
will also make it possible to (later) set the current directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Support a fatal level in tout
Simon Glass [Sun, 11 May 2025 14:18:20 +0000 (16:18 +0200)] 
u_boot_pylib: Support a fatal level in tout

It is convenient to be able to print a message and exit. Add a new
'fatal' level to support this.

Update some assumptions about the level, so that the tools continue to
work as now.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Update Series.ShowActions() to pass alias
Simon Glass [Mon, 7 Apr 2025 10:51:47 +0000 (22:51 +1200)] 
patman: Update Series.ShowActions() to pass alias

Instead of using settings.alias pass this value in. This allows tests to
work without using settings.alias

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Pass aliases to Series.MakeCcFile()
Simon Glass [Mon, 7 Apr 2025 10:51:46 +0000 (22:51 +1200)] 
patman: Pass aliases to Series.MakeCcFile()

Rather than accessing settings directly, pass the aliases in, so that
we can do the same from tests. With further work this will allow the
tests to work without using settings.alias

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Pass the alias dict into gitutil.email_patches()
Simon Glass [Mon, 7 Apr 2025 10:51:45 +0000 (22:51 +1200)] 
patman: Pass the alias dict into gitutil.email_patches()

Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Pass the alias dict into gitutil.build_email_list()
Simon Glass [Mon, 7 Apr 2025 10:51:44 +0000 (22:51 +1200)] 
patman: Pass the alias dict into gitutil.build_email_list()

Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Untangle settings from gitutil
Simon Glass [Mon, 7 Apr 2025 10:51:43 +0000 (22:51 +1200)] 
patman: Untangle settings from gitutil

The gitutil module is supposed to be independent from patman but one
piece was missed in the series which separated them.

Move the settings setup out of gitutil

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Drop the old status code
Simon Glass [Tue, 29 Apr 2025 13:22:27 +0000 (07:22 -0600)] 
patman: Drop the old status code

Now that we have the new asyncio implementation, drop the unused
ThreadPoolExecutor code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Use a pager when outputting status
Simon Glass [Tue, 29 Apr 2025 13:22:26 +0000 (07:22 -0600)] 
patman: Use a pager when outputting status

The pager makes it easier to review large number of comments. Add it
around the output.

Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Provide an option to run in single-threaded mode
Simon Glass [Tue, 29 Apr 2025 13:22:25 +0000 (07:22 -0600)] 
patman: Provide an option to run in single-threaded mode

Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Switch over to asyncio
Simon Glass [Tue, 29 Apr 2025 13:22:24 +0000 (07:22 -0600)] 
patman: Switch over to asyncio

Now that all the pieces are in place, switch over from using an executor
to using asyncio.

While we are here, import defaultdict to avoid needing to specify its
module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Use defaultdict directly
Simon Glass [Tue, 29 Apr 2025 13:22:23 +0000 (07:22 -0600)] 
patman: Use defaultdict directly

Import defaultdict to avoid needing to specify its module in multiple
places. Fix an extra blank line while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add more information to Patch
Simon Glass [Tue, 29 Apr 2025 13:22:22 +0000 (07:22 -0600)] 
patman: Add more information to Patch

The cover letter has some information on each patch, so allow this to be
stored in the Patch object.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Show patches in yellow
Simon Glass [Tue, 29 Apr 2025 13:22:21 +0000 (07:22 -0600)] 
patman: Show patches in yellow

When comments are shown below patches it can be hard to see the patch
subject. Use yellow instead of blue, since it stands out better.

Pass the colour object into show_responses() rather than creating a new
one, since that is tidier.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Simplify test_find_new_responses()
Simon Glass [Tue, 29 Apr 2025 13:22:20 +0000 (07:22 -0600)] 
patman: Simplify test_find_new_responses()

This test uses the find_new_responses() function which combines
accessing patchwork and processing the results.

Since the test is aimed at testing processing, adjust it to call
process_reviews() instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Adjust how the fake request() function is provided
Simon Glass [Tue, 29 Apr 2025 13:22:19 +0000 (07:22 -0600)] 
patman: Adjust how the fake request() function is provided

Instead of passing the URL and function to each call, put the fake
into the Patchwork object instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Split out the processing of find_new_responses()
Simon Glass [Tue, 29 Apr 2025 13:22:18 +0000 (07:22 -0600)] 
patman: Split out the processing of find_new_responses()

This function does patchwork calls and then processes the results. Move
the processing out so we can (later) test it separately.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Split up check_and_show_status()
Simon Glass [Tue, 29 Apr 2025 13:22:17 +0000 (07:22 -0600)] 
patman: Split up check_and_show_status()

This function has three phases:

- collecting things from patchwork
- doing some processing
- showing the results to the user / creating a branch

Refactor into two functions so we can eventually have the patchwork part
fully separated out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Rename check_patchwork_status()
Simon Glass [Tue, 29 Apr 2025 13:22:16 +0000 (07:22 -0600)] 
patman: Rename check_patchwork_status()

This function actually shows the status and does some other things.

Rename it to better reflect its purpose.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Add reading of series and patch status
Simon Glass [Tue, 29 Apr 2025 13:22:15 +0000 (07:22 -0600)] 
patman: Add reading of series and patch status

Expand the patchwork module so that it can match the current
requirements of the 'patman status' command, i.e. reading the state of a
series and the patches associated with it.

Since the format of each patchwork response is a little tricky to
understand, add examples in comments at the top of each function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Move Patch and Review to patchwork module
Simon Glass [Tue, 29 Apr 2025 13:22:14 +0000 (07:22 -0600)] 
patman: Move Patch and Review to patchwork module

These relate to information obtained from the patchwork server, so move
their definition into the new patchwork module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Create a module for handling patchwork
Simon Glass [Tue, 29 Apr 2025 13:22:13 +0000 (07:22 -0600)] 
patman: Create a module for handling patchwork

At present the patchwork implementation is very simple, just consisting
of a function which calls the REST API.

We want to create a fake patchwork for use in tests. So start a new
module to encapsulate communication with the patchwork server.

Use asyncio since it is easier to handle lots of concurrent requests
from different parts of the code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agoCI: Add a dependency file for patman
Simon Glass [Tue, 29 Apr 2025 13:22:12 +0000 (07:22 -0600)] 
CI: Add a dependency file for patman

Now that patman has an unusual dependency, add a requirements.txt file
and use it in CI

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Move code for sending into its own module
Simon Glass [Tue, 29 Apr 2025 13:22:11 +0000 (07:22 -0600)] 
patman: Move code for sending into its own module

The control module includes much of the implementation of patman's
'send' feature. As a first step to separating this out, move the sending
code into its own file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Support extra test features
Simon Glass [Tue, 29 Apr 2025 13:22:10 +0000 (07:22 -0600)] 
patman: Support extra test features

Provide support for the -X flag, which preserves the working directory
used by tests. Also support -N which shows captured output for tests.

Finally, allow selection of a particular test to run.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Tidy up the start-up code
Simon Glass [Tue, 29 Apr 2025 13:22:09 +0000 (07:22 -0600)] 
patman: Tidy up the start-up code

Much of this was written before the other Python tools and they have
evolved. Make a few updates:

- Rather than calling sys.exit(), return the exit code from the control
  module and use it in __main__
- Set up tout as it is used in some places
- We now have quite a few tests, so update the comment about that

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Rename test_basic() in test_checkpatch
Simon Glass [Tue, 29 Apr 2025 13:22:08 +0000 (07:22 -0600)] 
patman: Rename test_basic() in test_checkpatch

This currently has the same name as a test in func_test.py so it isn't
possible to select one or the other with 'patman test test_basic': both
are executed.

Rename this one to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Correct a stray quote
Simon Glass [Tue, 29 Apr 2025 13:22:07 +0000 (07:22 -0600)] 
patman: Correct a stray quote

A quote character was added a few years ago, but was not intended. Drop
it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agotools: Plumb in capture control
Simon Glass [Tue, 29 Apr 2025 13:22:06 +0000 (07:22 -0600)] 
tools: Plumb in capture control

Add control of capturing output into u_boot_pylib and the tools which
use it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Allow disabling colour output
Simon Glass [Tue, 29 Apr 2025 13:22:05 +0000 (07:22 -0600)] 
u_boot_pylib: Allow disabling colour output

When running tests there is no situation in which we want ANSI output as
it makes it much harder to see what is going on in logs, tests, etc.

Provide a way to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Avoid concurrent execution of only one test
Simon Glass [Tue, 29 Apr 2025 13:22:04 +0000 (07:22 -0600)] 
u_boot_pylib: Avoid concurrent execution of only one test

There is no point in spinning up multiple processes if there is only one
test to execute. Add a check for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Add a pager
Simon Glass [Tue, 29 Apr 2025 13:22:03 +0000 (07:22 -0600)] 
u_boot_pylib: Add a pager

It is useful to have a pager when outputting a lot of text. Add support
for this in the terminal library, making use of a context manager.

Also add a function to indicate whether the output device is a terminal
or not, while we are here, to avoid duplicating this code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Improve ANSI-colour output with backgrounds
Simon Glass [Tue, 29 Apr 2025 13:22:02 +0000 (07:22 -0600)] 
u_boot_pylib: Improve ANSI-colour output with backgrounds

The current implementation does not handle background colours very well:

- It outputs an incorrect code in some cases, leading to wrong colours
- Some functions lack a control for the background

Tidy this up so that background colours can be used in more places.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agou_boot_pylib: Allow control of capturing
Simon Glass [Tue, 29 Apr 2025 13:22:01 +0000 (07:22 -0600)] 
u_boot_pylib: Allow control of capturing

Tests often capture output so they can check it. This means that if the
test fails it is not easy to see what the output actually was.

Add a -N flag which writes out the output after it has been captured.
This is not a perfect solution but it is simple and seems to work well
in practice.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agobinman: Correct a typo in the toolpath comment
Simon Glass [Tue, 29 Apr 2025 13:22:00 +0000 (07:22 -0600)] 
binman: Correct a typo in the toolpath comment

Fix the 'ist' typo in the comment for TestFunctional

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Move capture_sys_output() into terminal and rename
Simon Glass [Tue, 29 Apr 2025 13:21:59 +0000 (07:21 -0600)] 
patman: Move capture_sys_output() into terminal and rename

This function is sometimes useful outside tests. Also it can affect how
terminal output is done, e.g. whether ANSI characters should be emitted
or not.

Move it out of the test_util package and into terminal.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agopatman: Move all non-test logic into control module
Simon Glass [Tue, 29 Apr 2025 13:21:58 +0000 (07:21 -0600)] 
patman: Move all non-test logic into control module

It is easier for tests if the top-level control logic is all in one
module. Create a new do_patman() function to handle this. Move the
existing code into it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agobinman: x86: Write skip-at-start when end-at-4gb is used
Simon Glass [Sun, 16 Feb 2025 02:02:37 +0000 (19:02 -0700)] 
binman: x86: Write skip-at-start when end-at-4gb is used

The end-at-4gb property implies a value for skip-at-start so add it into
the output FDT so that U-Boot can read it.

Now that skip-at-start is implemented, we can drop the workarounds used
in the x86 code to obtain the correct image-pos value.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agobinman: Read the skip-at-start value on startup
Simon Glass [Sun, 16 Feb 2025 02:02:36 +0000 (19:02 -0700)] 
binman: Read the skip-at-start value on startup

This value provides an offset for all image-pos values in the image.
Read it on startup so that we can take account of it when calculating
positions.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agobuildman: Add a way to build a particular target
Simon Glass [Tue, 17 Dec 2024 13:26:16 +0000 (06:26 -0700)] 
buildman: Add a way to build a particular target

At present buildman only supports building the default target. Generally
this is what is wanted, but in some cases boards erroneously have a
different target for product extra files.

Add a --target option to help. Also add a comment indicating which
letters are free for new options.

Signed-off-by: Simon Glass <sjg@chromium.org>
2 months agoMerge tag 'v2025.07-rc3' into next
Tom Rini [Mon, 26 May 2025 22:07:19 +0000 (16:07 -0600)] 
Merge tag 'v2025.07-rc3' into next

Prepare v2025.07-rc3

2 months agoPrepare v2025.07-rc3 v2025.07-rc3
Tom Rini [Mon, 26 May 2025 22:02:19 +0000 (16:02 -0600)] 
Prepare v2025.07-rc3

Signed-off-by: Tom Rini <trini@konsulko.com>
2 months agoMerge branch 'master' of git://source.denx.de/u-boot-usb
Tom Rini [Mon, 26 May 2025 00:01:54 +0000 (18:01 -0600)] 
Merge branch 'master' of git://source.denx.de/u-boot-usb

2 months agoMerge tag 'efi-2025-07-rc3-3' of https://source.denx.de/u-boot/custodians/u-boot-efi
Tom Rini [Sun, 25 May 2025 15:14:07 +0000 (09:14 -0600)] 
Merge tag 'efi-2025-07-rc3-3' of https://source.denx.de/u-boot/custodians/u-boot-efi

Pull request efi-2025-07-rc3-3

CI:

* https://source.denx.de/u-boot/custodians/u-boot-efi/-/pipelines/26313

UEFI:

* run dhcp if an http boot option is selected
* separate device path into its own header and add it to the API docs
* rename END to EFI_DP_END
* make x86 SMBIOS tables usable in the EFI context

2 months agousb: ulpi: Clean up how we enable support
Tom Rini [Sat, 15 Mar 2025 01:27:36 +0000 (19:27 -0600)] 
usb: ulpi: Clean up how we enable support

The way we enable ULPI support today isn't something that should work.
The "optional" keyword in a choice statement is not a documented
feature. To make this work in a supported way, make USB_ULPI something
we ask about if USB_HOST is set. Next, we move the choice of what
viewer to use to be after the framework portion and to depend on that.
We then borrow a few words from the top-level README to make the help
text here clearer. Finally we make the Qualcomm driver select ULPI as
it's required and we make the tegra driver not duplicate a check that
Kconfig now handles for us.

Signed-off-by: Tom Rini <trini@konsulko.com>
2 months agousb: ulpi: Remove unused omap-ulpi-viewport driver
Tom Rini [Sat, 15 Mar 2025 01:27:35 +0000 (19:27 -0600)] 
usb: ulpi: Remove unused omap-ulpi-viewport driver

The last platform to enable this driver was removed in 2019. Remove this
unused code and documentation now.

Signed-off-by: Tom Rini <trini@konsulko.com>
2 months agousb: dwc3: core: Fix timeout check
Varadarajan Narayanan [Wed, 15 Jan 2025 06:20:44 +0000 (11:50 +0530)] 
usb: dwc3: core: Fix timeout check

dwc3_core_init loops 'timeout' times to check if the IP block is out
of reset using 'while (timeout--)'. If there is some issue and
the block doesn't come out of reset, the loop will run till
'timeout' becomes zero and the post decrement operator would set
timeout to 0xffffffff. Though the IP block is not out reset, the
subsequent if check 'if !timeout' would fail as timeout is not
equal to zero and the function proceeds with the initialization.

Use poll API instead to resolve this.

Signed-off-by: Varadarajan Narayanan <quic_varada@quicinc.com>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
Reviewed-by: Caleb Connolly <caleb.connolly@linaro.org>
2 months agoefi_loader: Run dhcp if an http boot option is selected
Ilias Apalodimas [Fri, 23 May 2025 13:04:04 +0000 (16:04 +0300)] 
efi_loader: Run dhcp if an http boot option is selected

The EFI boot manager relies on having an IP address before trying to
boot an EFI HTTP(s) boot entry. However, defining it as a boot or
pre-boot command is not always the right answer since it will
unconditionally add delay to the board boot, even if we don't boot
over the network.

So let's do a DHCP request from the boot manager, if 'ipaddr' is
empty and fail early if we don't have an address.

Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Tested-by: Michal Simek <michal.simek@amd.com>
2 months agox86: efi_loader: Ensure the SMBIOS tables are sent via EFI
Simon Glass [Fri, 23 May 2025 16:55:04 +0000 (17:55 +0100)] 
x86: efi_loader: Ensure the SMBIOS tables are sent via EFI

The EFI-loader code has not been fully converted to use bloblist, so
relies on the SMBIOS-table address being set in global_data.

Set this up in write_tables() so that the SMBIOS tables are actually
available.

Enable the command for x86 QEMU so that the SMBIOS tests actually run.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: 83ce35d6ebb ("emulation: Use bloblist to hold tables")
Reported-by: Niklas Sombert <niklas.sombert@uni-duesseldorf.de>
Tested-by: Niklas Sombert <niklas.sombert@uni-duesseldorf.de>
Tested-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
2 months agoefi: Rename END to EFI_DP_END
Simon Glass [Sat, 24 May 2025 17:28:23 +0000 (11:28 -0600)] 
efi: Rename END to EFI_DP_END

This exported symbol has a very generic name. Rename it to indicate that
it relates to EFI and device-paths.

Fix checkpatch warnings related to use of multiple assignments.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
2 months agoefi: Include device-path functions in the EFI API docs
Simon Glass [Sat, 24 May 2025 17:28:22 +0000 (11:28 -0600)] 
efi: Include device-path functions in the EFI API docs

Include these function so they can be browsed in the API docs. Exclude
END since it causes a warning, which becomes an error:

   ./include/efi_device_path.h:22: warning: cannot understand function
      prototype: 'const struct efi_device_path END; '

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
2 months agoefi_loader: Separate device path into its own header
Simon Glass [Sat, 24 May 2025 17:28:21 +0000 (11:28 -0600)] 
efi_loader: Separate device path into its own header

These functions are useful for the EFI app. As a first step towards
making these available outside lib/efi_loader, create a separate header
file and include it where needed. Add proper comments to the functions,
since many are missing at present.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
2 months agonet: phy: Do not do CL22 phy reset before ethernet phy driver probe
SkyLake.Huang [Mon, 14 Oct 2024 07:06:11 +0000 (15:06 +0800)] 
net: phy: Do not do CL22 phy reset before ethernet phy driver probe

Remove unnecessary CL22 phy reset before ethernet phy driver
probe. Lots of ethernet phys requires driver to load firmware.
Before that, CL22 phy reset may lead to malfunction.

Signed-off-by: SkyLake.Huang <skylake.huang@mediatek.com>
2 months agonet: tftp: don't call stop callback from UDP handler
Alexander Sverdlin [Mon, 13 May 2024 06:26:35 +0000 (08:26 +0200)] 
net: tftp: don't call stop callback from UDP handler

Contrary to doc/develop/driver-model/ethernet.rst contract, eth_ops
.free_pkt can be called after .stop, there are several error paths in TFTP,
for instance:

eth_halt() <= tftp_handler() <= net_process_received_packet() <= eth_rx()
...
am65_cpsw_free_pkt() <= eth_rx()

Which results in (deliberately "tftpboot"ing non-existing file):

TFTP error: 'File not found' (1)
Not retrying...
am65_cpsw_nuss_port ethernet@8000000port@1: RX dma free_pkt failed -22

Avoid the DMA error message (and follow the documentation) by deferring
eth_halt() until net_loop() calls net_start_again() and only do
eth_halt_state_only() instead.

Fixes: aafda38fb266 ("Add error codes/handling for TFTP-server")
Signed-off-by: Alexander Sverdlin <alexander.sverdlin@siemens.com>
2 months agoMerge patch series "BOOTP/DHCPv4 enhancements"
Tom Rini [Fri, 23 May 2025 17:29:55 +0000 (11:29 -0600)] 
Merge patch series "BOOTP/DHCPv4 enhancements"

Sean Edmond <seanedmond@microsoft.com> says:

In our datacenter application, a single DHCP server is servicing 36000+ clients.
Improvements are required to the DHCPv4 retransmission behavior to align with
RFC and ensure less pressure is exerted on the server:
- retransmission backoff interval maximum is configurable
  (environment variable bootpretransmitperiodmax)
- initial retransmission backoff interval is configurable
  (environment variable bootpretransmitperiodinit)
- transaction ID is kept the same for each BOOTP/DHCPv4 request
  (not recreated on each retry)

For our application we'll use:
- bootpretransmitperiodmax=16000
- bootpretransmitperiodinit=2000

A new configuration BOOTP_RANDOM_XID has been added to enable a randomized
BOOTP/DHCPv4 transaction ID.

Enhance DHCPv4 sending/parsing option 209 (PXE config file).  A previous
patch was accepted.  A new patch fixes a possible double free() and
addresses latest review comments.

Link: https://lore.kernel.org/r/20240509023918.2504185-1-seanedmond@microsoft.com
2 months agonet: bootp: add config option BOOTP_RANDOM_XID
Sean Edmond [Thu, 9 May 2024 02:39:03 +0000 (19:39 -0700)] 
net: bootp: add config option BOOTP_RANDOM_XID

The new config option BOOTP_RANDOM_XID will randomize the transaction ID
for each new BOOT/DHCPv4 exchange.

Signed-off-by: Sean Edmond <seanedmond@microsoft.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2 months agonet: bootp: BOOTP/DHCPv4 retransmission improvements
Sean Edmond [Thu, 9 May 2024 02:39:02 +0000 (19:39 -0700)] 
net: bootp: BOOTP/DHCPv4 retransmission improvements

This patch introduces 3 improvements to align with RFC 951:
- retransmission backoff interval maximum is configurable
- initial retranmission backoff interval is configurable
- transaction ID is kept the same for each BOOTP/DHCPv4 request

In applications where thousands of nodes are serviced by a single DHCP
server, maximizing the retransmission backoff interval at 2 seconds (the
current u-boot default) exerts high pressure on the DHCP server and
network layer.

RFC 951 “7.2. Client Retransmission Strategy” states that the
retransmission backoff interval should be limited to 60 seconds.  This
patch allows the interval to be configurable using the environment
variable "bootpretransmitperiodmax"

The initial retranmission backoff period defaults to 250ms, which is
also too small for these scenarios with many clients.  This patch makes
the initial retransmission interval to be configurable using the
environment variable "bootpretransmitperiodinit".

Also, on a retransmission it is not expected for the transaction ID to
change (only the 'secs' field should be updated). Let's save the
transaction ID and use the same transaction ID for each BOOTP/DHCPv4
exchange.

Signed-off-by: Sean Edmond <seanedmond@microsoft.com>