]> git.ipfire.org Git - thirdparty/mkosi.git/log
thirdparty/mkosi.git
6 years agoCall pytest from ci/semaphore.sh 330/head
Adrian Freihofer [Mon, 25 Feb 2019 21:49:08 +0000 (22:49 +0100)] 
Call pytest from ci/semaphore.sh

6 years agoAdd unit tests for argument and config file parsing
Adrian Freihofer [Sun, 20 Jan 2019 10:07:30 +0000 (11:07 +0100)] 
Add unit tests for argument and config file parsing

This adds 53 tests for the parse_args function. Just run pytest-3
in the root directory of this project to execute all tests.

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoman page: cover ! in lists
Adrian Freihofer [Sun, 24 Feb 2019 11:54:01 +0000 (12:54 +0100)] 
man page: cover ! in lists

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoparse_args: handle kernel_cmd as space separated list
Adrian Freihofer [Sat, 2 Mar 2019 14:18:23 +0000 (15:18 +0100)] 
parse_args: handle kernel_cmd as space separated list

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoparse_args: Support [foo,bar] syntax
Adrian Freihofer [Sat, 2 Mar 2019 14:47:20 +0000 (15:47 +0100)] 
parse_args: Support [foo,bar] syntax

For comma separated lists also the list syntax is common in
ini files.

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoparse_args: major refactoring and improvements
Adrian Freihofer [Sun, 24 Feb 2019 15:49:24 +0000 (16:49 +0100)] 
parse_args: major refactoring and improvements

The man page describes the powerful configuration approach
based on a mkosi.default file and several file in a
mkosi.default.d folder as well as command line arguments.
This does not work for many parameters. The problem is that
argparse is called before config files are loaded.
For some values the implementation basically looks like:
  a = False   # argparse lines
  ...
  if a is None:  # config file loading, lower priority
      a = True

This new implementation delegates loading of mkosi.default files
to python's argparse.ArgumentParser. According to docummentation
ArgumentParser supports loading of settings from files by
overloading the convert_arg_line_to_args function. The function
gets one line from the config file and returns the corresponding
command line counterpart. For example WithTest=yes found in a
defaults file would be converted to ['--with-tests', 'yes'].

Unfortunately this line by line conversion is not really compatible
with python's configparser module which works file wise. To stay
with ArgumentParser and ConfigParser the new ArgumentParserMkosi
overloads the undocumented _read_args_from_files function of the
ArgumentParser. This function is called for each @ prefixed
command line argument.

The new implementation supports unlimited number of mkosi.default
files. The files are loaded according their alphabetical priority.
Command line arguments are processed with highest priority.

The second problem solved by this patch is data type conversion
for settings loaded from mkosi.default files. Since ArgumentParser
handles all settings, data type conversion is always done by
ArgumentParser.

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoparse_args: Add compatibility options, set defaults
Adrian Freihofer [Sun, 24 Feb 2019 15:24:40 +0000 (16:24 +0100)] 
parse_args: Add compatibility options, set defaults

Support (or prepare to support) WithTests and BuildDirectory
arguments as stated in the man page.

Ensure repositories is always a list.

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoparse_args: verb arg has no attributes
Adrian Freihofer [Wed, 27 Feb 2019 21:21:43 +0000 (22:21 +0100)] 
parse_args: verb arg has no attributes

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoparse_args: Improve handling of boolean arguments
Adrian Freihofer [Sun, 13 Jan 2019 11:55:21 +0000 (12:55 +0100)] 
parse_args: Improve handling of boolean arguments

First step to handle boolean configuration parameters from
mkosi.default files as well as from command line arguments. The
implementation supports multiple definitions of one argument. The
last value gets priority. Examples:
  --foo=yes --foo=no  --> args.foo=False
  --foo               --> args.foo=True
  --without-foo       --> args.foo=False
  --foo --without-foo --> args.foo=False

Purpose:
Let argparse import mkosi.default files in a future commit.

Make parse_boolean function case-insensitive.

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoUse constants for commands
Adrian Freihofer [Sun, 24 Feb 2019 12:42:29 +0000 (13:42 +0100)] 
Use constants for commands

Refactoring without changes

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoMove ArgumentParser creation to function
Adrian Freihofer [Sun, 24 Feb 2019 12:26:03 +0000 (13:26 +0100)] 
Move ArgumentParser creation to function

Refactoring without changes

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agogitignore: __pycache__, PyDev
Adrian Freihofer [Tue, 22 Jan 2019 23:14:09 +0000 (00:14 +0100)] 
gitignore: __pycache__, PyDev

- ingnore __pycache__ in subfolders
- ignore PyDev project and temporary files

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoparse_args: Support removing list entries with ! prefix
Adrian Freihofer [Thu, 24 Jan 2019 23:26:16 +0000 (00:26 +0100)] 
parse_args: Support removing list entries with ! prefix

Entries added handled by the ListAction class may be prefixed with
! to remove an entry if it is already in the list.
!* removes all entries from the list.

Examples:
- mkosi -p httpd -p !httpd
  has no effect

- mkosi -p vi -p emacs -p nano -p "!*,httpd"
  Only the httpd package will be added to the image.

Note: This function becomes even more useful if the configuration
      files are also processed by argparse. This will be the case
      with a subsequent commit.

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoAdd COMMANDS section to summary
Adrian Freihofer [Sun, 13 Jan 2019 12:21:32 +0000 (13:21 +0100)] 
Add COMMANDS section to summary

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoMerge pull request #326 from poettering/xbootldr
Lennart Poettering [Thu, 7 Mar 2019 11:10:19 +0000 (12:10 +0100)] 
Merge pull request #326 from poettering/xbootldr

support for "xbootldr" partitions

6 years agoMerge pull request #331 from llucax/btrfs-progs
Zbigniew Jędrzejewski-Szmek [Thu, 28 Feb 2019 15:56:09 +0000 (16:56 +0100)] 
Merge pull request #331 from llucax/btrfs-progs

Debian: Use btrfs-progs instead of btrfs-tools

6 years agoci: Add a test for ubuntu image using btrfs 331/head
Leandro Lucarella [Thu, 28 Feb 2019 14:40:53 +0000 (11:40 -0300)] 
ci: Add a test for ubuntu image using btrfs

6 years agoDebian: Use btrfs-progs instead of btrfs-tools
Leandro Lucarella [Thu, 28 Feb 2019 14:06:05 +0000 (11:06 -0300)] 
Debian: Use btrfs-progs instead of btrfs-tools

The package was renamed some time ago (around March 2016)[1]. The
package btrfs-progs is available from Stretch[2] but it's also in Jessie
backports[3]; in Ubuntu it was introduced in Bionic[4], so it's probably
more appropriate to use the new name.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=780081
[2] https://packages.debian.org/stretch/btrfs-progs
[3] https://packages.debian.org/jessie-backports/btrfs-progs
[4] https://packages.ubuntu.com/bionic/btrfs-progs

6 years agoMerge pull request #328 from lucasdemarchi/tip-bind
Lucas De Marchi [Wed, 27 Feb 2019 20:53:06 +0000 (12:53 -0800)] 
Merge pull request #328 from lucasdemarchi/tip-bind

Turn root into a mountpoint

6 years agoTurn root into a mountpoint 328/head
Lucas De Marchi [Tue, 19 Feb 2019 21:59:31 +0000 (13:59 -0800)] 
Turn root into a mountpoint

By doing this we can safely rely on mount --recursive doing the right
thing.

6 years agomkosi: take a BSD lock on the workspace directory while running
Lennart Poettering [Fri, 18 Jan 2019 16:14:30 +0000 (17:14 +0100)] 
mkosi: take a BSD lock on the workspace directory while running

This makes use of the new logic here:

https://github.com/systemd/systemd/pull/11482

(And on older systems has no effect, but that shouldn't hurt either)

Fixes: #252
6 years agoMerge pull request #329 from afreof/minor-improvements
Zbigniew Jędrzejewski-Szmek [Mon, 25 Feb 2019 06:48:40 +0000 (07:48 +0100)] 
Merge pull request #329 from afreof/minor-improvements

Minor improvements

6 years agoDo not redefine built-in 'type' 329/head
Adrian Freihofer [Sun, 13 Jan 2019 16:25:18 +0000 (17:25 +0100)] 
Do not redefine built-in 'type'

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in id
Adrian Freihofer [Sun, 13 Jan 2019 16:12:01 +0000 (17:12 +0100)] 
Do not redefine built-in id

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in bytes
Adrian Freihofer [Sun, 13 Jan 2019 16:09:36 +0000 (17:09 +0100)] 
Do not redefine built-in bytes

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoRemove unused variable
Adrian Freihofer [Sun, 13 Jan 2019 16:06:05 +0000 (17:06 +0100)] 
Remove unused variable

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoRemove unused variable
Adrian Freihofer [Sun, 13 Jan 2019 16:05:25 +0000 (17:05 +0100)] 
Remove unused variable

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in sum
Adrian Freihofer [Sun, 13 Jan 2019 15:56:09 +0000 (16:56 +0100)] 
Do not redefine built-in sum

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoCosmetics: intentation
Adrian Freihofer [Sun, 13 Jan 2019 15:53:15 +0000 (16:53 +0100)] 
Cosmetics: intentation

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoCosmetics: unused variable
Adrian Freihofer [Sun, 13 Jan 2019 15:52:45 +0000 (16:52 +0100)] 
Cosmetics: unused variable

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine name 'uuid'
Adrian Freihofer [Sun, 13 Jan 2019 15:48:07 +0000 (16:48 +0100)] 
Do not redefine name 'uuid'

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not override run_build_script function
Adrian Freihofer [Sun, 13 Jan 2019 15:35:31 +0000 (16:35 +0100)] 
Do not override run_build_script function

Does not change anything. It's a renaming of a function parameter
from run_build_script to do_run_build_script to resolve the name
clashing with function run_build_script.

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoCosmetics: Fix intentation
Adrian Freihofer [Sun, 13 Jan 2019 15:26:36 +0000 (16:26 +0100)] 
Cosmetics: Fix intentation

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in dir, type
Adrian Freihofer [Sun, 13 Jan 2019 15:25:57 +0000 (16:25 +0100)] 
Do not redefine built-in dir, type

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in bytes
Adrian Freihofer [Sun, 13 Jan 2019 15:25:14 +0000 (16:25 +0100)] 
Do not redefine built-in bytes

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoMerge pull request #327 from lucasdemarchi/tip-arch
Lucas De Marchi [Wed, 20 Feb 2019 16:23:01 +0000 (08:23 -0800)] 
Merge pull request #327 from lucasdemarchi/tip-arch

Fix archlinux on output type directory

6 years agoarch: kill dirmngr only 327/head
Lucas De Marchi [Tue, 19 Feb 2019 21:54:56 +0000 (13:54 -0800)] 
arch: kill dirmngr only

Using fuser to identify processes using a path doesn't work as I wanted:
it relies on the filesystem mounted there. The outcome is that when
using "-t directory" it will try to kill all processes using that
filesystem rather than processes using files under that path.

Even by bind-mounting root so it's always a mountpoint, it doesn't work.
So instead of killing everything, pinpoint what process pacman/pacstrap
left running and kill only that one.

Right now this means telling dirmngr to exit.

6 years agoarch: trust new signatures while downloading base packages
Lucas De Marchi [Tue, 19 Feb 2019 21:53:06 +0000 (13:53 -0800)] 
arch: trust new signatures while downloading base packages

Otherwise it aborts with:
error: libxml2: signature from "Levente Polyak (anthraxx) <levente@leventepolyak.net>" is unknown trust
:: File /var/tmp/mkosi-__fdh4oy/root/var/cache/pacman/pkg/libxml2-2.9.9-1-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]

6 years agomkosi: add basic support for XBOOTLDR partitions 326/head
Lennart Poettering [Tue, 5 Feb 2019 15:44:06 +0000 (16:44 +0100)] 
mkosi: add basic support for XBOOTLDR partitions

As documentated in the boot loader specification

6 years agomkosi: copy sd-boot EFI binaries to correct directory
Lennart Poettering [Tue, 5 Feb 2019 15:43:45 +0000 (16:43 +0100)] 
mkosi: copy sd-boot EFI binaries to correct directory

6 years agoCreate link at output path for tar format builds
Matt Schreiber [Mon, 28 Jan 2019 01:35:50 +0000 (20:35 -0500)] 
Create link at output path for tar format builds

Previously, mkosi created hardlinks at the specified output path only
for the plain squashfs and "disk" formats

6 years agoMerge pull request #321 from lucasdemarchi/tip-fixes
Lucas De Marchi [Fri, 18 Jan 2019 22:28:47 +0000 (14:28 -0800)] 
Merge pull request #321 from lucasdemarchi/tip-fixes

Fixes

6 years agoFix prepend to environ path 321/head
Lucas De Marchi [Wed, 19 Dec 2018 00:19:02 +0000 (16:19 -0800)] 
Fix prepend to environ path

We need to call this after loading the configuration file, otherwise
anything set there won't work.

6 years agoFix compat option for kernel command line
Lucas De Marchi [Wed, 19 Dec 2018 00:18:12 +0000 (16:18 -0800)] 
Fix compat option for kernel command line

We need to set the dest option, otherwise you will be able to set the
option, but it won't have any effect.

6 years agoRemove plural in arg option
Lucas De Marchi [Mon, 26 Nov 2018 20:14:22 +0000 (12:14 -0800)] 
Remove plural in arg option

Like other lists, make it singular in the option. See --package,
--extra-tree, etc.

6 years agoMerge pull request #315 from jlettman/no-check-valid-until
Lennart Poettering [Fri, 18 Jan 2019 16:19:48 +0000 (17:19 +0100)] 
Merge pull request #315 from jlettman/no-check-valid-until

mkosi: add --no-check-valid-until to debootstrap args to fix #314

6 years agomkosi: remove old --no-check-valid-until argument to debootstrap 315/head
John Lettman [Wed, 9 Jan 2019 02:01:27 +0000 (21:01 -0500)] 
mkosi: remove old --no-check-valid-until argument to debootstrap

6 years agomkosi: generalize debootstrap_knows_merged_user and add check for --no-check-valid...
John Lettman [Wed, 9 Jan 2019 01:59:53 +0000 (20:59 -0500)] 
mkosi: generalize debootstrap_knows_merged_user and add check for --no-check-valid-until to fix #314 on other versions of debootstrap

6 years agomkosi: add --no-check-valid-until to debootstrap args to fix #314
John Lettman [Wed, 9 Jan 2019 01:42:20 +0000 (20:42 -0500)] 
mkosi: add --no-check-valid-until to debootstrap args to fix #314

6 years agoCorrect file extension
Hervé [Thu, 3 Jan 2019 12:32:24 +0000 (13:32 +0100)] 
Correct file extension

6 years agoMerge pull request #312 from poettering/man-page-all
Lucas De Marchi [Tue, 18 Dec 2018 15:52:12 +0000 (07:52 -0800)] 
Merge pull request #312 from poettering/man-page-all

mention plain_squashfs in the summary, and document --all

6 years agoman: document the new --all switches 312/head
Lennart Poettering [Tue, 18 Dec 2018 15:46:59 +0000 (16:46 +0100)] 
man: document the new --all switches

6 years agomkosi: mention that plain_squashfs output format exists now
Lennart Poettering [Tue, 18 Dec 2018 15:46:48 +0000 (16:46 +0100)] 
mkosi: mention that plain_squashfs output format exists now

As requested here:

https://github.com/systemd/mkosi/commit/bb5dad8d4368c32abd008a886999c506448cc3e2#commitcomment-31710412

6 years agoMerge pull request #308 from poettering/man-page
Lennart Poettering [Mon, 17 Dec 2018 12:24:37 +0000 (13:24 +0100)] 
Merge pull request #308 from poettering/man-page

add a man page

6 years agodocs: Replace README.md by a pointer to the man page 308/head
Lennart Poettering [Fri, 14 Dec 2018 20:26:02 +0000 (21:26 +0100)] 
docs: Replace README.md by a pointer to the man page

The man page contains most relevant stuff now, and is a lot more
comprehensive, hence let's drop these explanations from README.md and
point people directly to the man page if they want to know more.

6 years agoman: add a man page
Lennart Poettering [Fri, 14 Dec 2018 20:22:58 +0000 (21:22 +0100)] 
man: add a man page

This adds a pretty comprehensive man page as a MarkDown document.

This does not contain any build system hookup, simply because I couldn't
figure out how to teach Python's setuptools to invoke external programs
as part of the build process.

Currently a manual step is necessary, to convert the man page into an
actual man page:

        # pandoc mkosi.md -s -t man > mkosi.1

The man page is based on the old README.md but substantially extended.

Fixes: #210
6 years agomkosi: move --no-chown to [Output] section in --help text
Lennart Poettering [Fri, 14 Dec 2018 19:59:27 +0000 (20:59 +0100)] 
mkosi: move --no-chown to [Output] section in --help text

6 years agomkosi: move --hostname= to right section
Lennart Poettering [Fri, 14 Dec 2018 19:50:16 +0000 (20:50 +0100)] 
mkosi: move --hostname= to right section

6 years agomkosi: --kernel-commandline= → --kernel-command-line=
Lennart Poettering [Fri, 14 Dec 2018 19:42:46 +0000 (20:42 +0100)] 
mkosi: --kernel-commandline= → --kernel-command-line=

This was just weird, in particular as the mkosi.default option was
aleady called KernelCommandLine=.

Also, let's move this to the right section in the --help text.

6 years agomkosi: be precise with the naming of the squashfs output modes
Lennart Poettering [Fri, 14 Dec 2018 19:14:36 +0000 (20:14 +0100)] 
mkosi: be precise with the naming of the squashfs output modes

6 years agomkosi: pass $WITH_NETWORK= to mkosi.build script
Lennart Poettering [Fri, 14 Dec 2018 19:12:09 +0000 (20:12 +0100)] 
mkosi: pass $WITH_NETWORK= to mkosi.build script

Let's tell the build script whether it is running with networking or
not. I don#t have an immediate usecase for this, but it's more
systematic given how --with-docs and --without-tests is currently
handled and result in environment variables, bus --with-network is not.

A usecase might be to generate the `-nonet` option of `xsltproc` from
this option, but that's purely theoretical.

6 years agomkosi: don't use mkosi.skeleton/mkosi.extra trees when one is specified explicitly
Lennart Poettering [Fri, 14 Dec 2018 19:10:54 +0000 (20:10 +0100)] 
mkosi: don't use mkosi.skeleton/mkosi.extra trees when one is specified explicitly

For our other options the auto-discovery of mkosi.xyz resources is
generally skipped if explicitly configured, do this here, too.

6 years agomkosi: when "gpt_xfs" is used, format /home and /srv with xfs too
Lennart Poettering [Fri, 14 Dec 2018 16:48:53 +0000 (17:48 +0100)] 
mkosi: when "gpt_xfs" is used, format /home and /srv with xfs too

6 years agoadd a TODO document
Lennart Poettering [Fri, 14 Dec 2018 16:40:07 +0000 (17:40 +0100)] 
add a TODO document

6 years agomkosi: always mention XFS before btrfs, as it is more like ext4
Lennart Poettering [Fri, 14 Dec 2018 16:38:22 +0000 (17:38 +0100)] 
mkosi: always mention XFS before btrfs, as it is more like ext4

ext4 and xfs images are pretty much put together the same way, except
for the file system used. btrfs otoh is different as instead of a GPT
partition table any subdivisions are are done as btrfs subvolumes.
Hence, let's reorder the enums a bit, and always handle ext4/xfs
together, and btrfs after that.

6 years agoMerge pull request #306 from keszybz/style-fixes
Lennart Poettering [Mon, 17 Dec 2018 11:51:19 +0000 (12:51 +0100)] 
Merge pull request #306 from keszybz/style-fixes

Two trivial stylistic fixups

6 years agoUse single quotes type instead of escaping 306/head
Zbigniew Jędrzejewski-Szmek [Fri, 14 Dec 2018 12:22:30 +0000 (13:22 +0100)] 
Use single quotes type instead of escaping

That's just more readable and shorter.

6 years agoMerge pull request #309 from poettering/mkosi-all
Lucas De Marchi [Mon, 17 Dec 2018 02:23:52 +0000 (18:23 -0800)] 
Merge pull request #309 from poettering/mkosi-all

add new "mkosi -a" switch for building all images in mkosi.files/

6 years agomkosi: add "-a" switch for building everything in mkosi.files/ 309/head
Lennart Poettering [Sat, 15 Dec 2018 09:44:55 +0000 (10:44 +0100)] 
mkosi: add "-a" switch for building everything in mkosi.files/

Let's add a one-stop solution for building a set of files all in one go.
For that, simply place "mkosi.default"-style files in mkosi.files/ and
use "mkosi -a" to build them all.

Fixes: #176
6 years agomkosi: rename copy() → copy_path() so that we can import copy module without hassles
Lennart Poettering [Sat, 15 Dec 2018 09:44:02 +0000 (10:44 +0100)] 
mkosi: rename copy() → copy_path() so that we can import copy module without hassles

6 years agoMerge pull request #311 from poettering/debian-root-mkdir
Lucas De Marchi [Sat, 15 Dec 2018 16:34:51 +0000 (08:34 -0800)] 
Merge pull request #311 from poettering/debian-root-mkdir

mkosi: create root directory as 0755

6 years agomkosi: create root directory as 0755 311/head
Lennart Poettering [Sat, 15 Dec 2018 10:29:30 +0000 (11:29 +0100)] 
mkosi: create root directory as 0755

Fixes: #253
6 years agotyping: one more type annotation to make mypy happy
Zbigniew Jędrzejewski-Szmek [Fri, 14 Dec 2018 14:22:25 +0000 (15:22 +0100)] 
typing: one more type annotation to make mypy happy

6 years agoMerge pull request #300 from LukeShu:to-upstream/flake8+mypy
Zbigniew Jędrzejewski-Szmek [Fri, 14 Dec 2018 14:17:24 +0000 (15:17 +0100)] 
Merge pull request #300 from LukeShu:to-upstream/flake8+mypy

Merging locally to resolve the few conflicts.

6 years agoFix default value for SourceFileTransfer
Lucas De Marchi [Fri, 7 Dec 2018 17:03:29 +0000 (09:03 -0800)] 
Fix default value for SourceFileTransfer

To retain the same behavior as before folding the options into one.

6 years agomkosi: when a .nspawn file is used, actually use it when running nspawn
Lennart Poettering [Thu, 6 Dec 2018 20:08:23 +0000 (21:08 +0100)] 
mkosi: when a .nspawn file is used, actually use it when running nspawn

Previously, we'd move it in place, but when invoking nspawn on "mkosi
shell" or "mkosi boot" it would be ignored, since that's what nspawn
does unless the file is in /etc or /run.

Let's change this and mark it trusted, after all it's coming from the
same source that just built the image.

This is very useful for cases where you want to default to
"--private-network" when "mkosi boot" is invoked.

6 years agoMerge pull request #303 from michaelweghorn/michaelweghorn/updateHelp
Lucas De Marchi [Wed, 5 Dec 2018 16:20:08 +0000 (08:20 -0800)] 
Merge pull request #303 from michaelweghorn/michaelweghorn/updateHelp

Update help for '--with-docs'

6 years agoUpdate help for '--with-docs' 303/head
Michael Weghorn [Wed, 5 Dec 2018 14:23:41 +0000 (15:23 +0100)] 
Update help for '--with-docs'

The restriction "only Fedora, CentOS and Mageia"
does not (no longer?) apply, since the option is also handled for
other distributions (Debian, Ubuntu, openSUSE).

6 years agoMerge pull request #292 from lucasdemarchi/tip-build-src
Lucas De Marchi [Tue, 4 Dec 2018 18:44:53 +0000 (10:44 -0800)] 
Merge pull request #292 from lucasdemarchi/tip-build-src

Mount build source

6 years agoMerge pull request #299 from michaelweghorn/michaelweghorn/addBiosSupportForDebianAnd...
Lucas De Marchi [Tue, 4 Dec 2018 18:44:04 +0000 (10:44 -0800)] 
Merge pull request #299 from michaelweghorn/michaelweghorn/addBiosSupportForDebianAndUbuntu

Add BIOS support for Debian and Ubuntu

6 years agomypy: Fix issues discovered by mypy 300/head
Luke Shumaker [Sun, 2 Dec 2018 07:46:22 +0000 (02:46 -0500)] 
mypy: Fix issues discovered by mypy

Mostly issues with not handling things that may be None.

6 years agomypy: Fiddle with things to clue mypy in
Luke Shumaker [Sun, 2 Dec 2018 10:05:09 +0000 (05:05 -0500)] 
mypy: Fiddle with things to clue mypy in

But this shouldn't actually change anything; this is just making
stylistic changes to the code to clue mypy in to some stuff.

6 years agomypy: Unify typing of install_distribution() and install_{DISTRIBUTION}()
Luke Shumaker [Sun, 2 Dec 2018 16:51:05 +0000 (11:51 -0500)] 
mypy: Unify typing of install_distribution() and install_{DISTRIBUTION}()

6 years agomypy: invoke_dnf_or_yum(): Match the signature of invoke_dnf() and invoke_yum()
Luke Shumaker [Sun, 2 Dec 2018 16:43:52 +0000 (11:43 -0500)] 
mypy: invoke_dnf_or_yum(): Match the signature of invoke_dnf() and invoke_yum()

6 years agomypy: ListAction: assert that 'values' is a string
Luke Shumaker [Sun, 2 Dec 2018 19:01:00 +0000 (14:01 -0500)] 
mypy: ListAction: assert that 'values' is a string

We must declare 'values' as Union[str, Sequence[Any], None], to keep the
__call__ definition compatible with the parent class.  However, we know
(and rely on) that how we call .add_argument() means that it will always
be a str.  Add an 'assert' to let mypy know what we know.

6 years agomypy: Make arguments Optional[], assert is not None as appropriate
Luke Shumaker [Sun, 2 Dec 2018 18:38:46 +0000 (13:38 -0500)] 
mypy: Make arguments Optional[], assert is not None as appropriate

There are several places where we unconditionally call a function, then
that function checks `args` to decide if it immediately returns, or if it
actually gets to do work.  In the cases where it immediately returns, some
of the arguments can be None, so they should be hinted as Optional[].
However, in the case were it does get to do work, they cannot be None.

So, hint them as Optional[], and the add `assert ARGNAME is not None`
after the check that implies it's not None, to let mypy know what we know.

For instance: args.output_format.is_disk() implies that raw is not None.
So it's safe to make the change:

         if not args.output_format.is_disk():
             return None
    +    assert raw is not None

6 years agomypy: Add missing type annotations
Luke Shumaker [Sun, 2 Dec 2018 07:20:12 +0000 (02:20 -0500)] 
mypy: Add missing type annotations

6 years agomypy: Fix wrong type annotations
Luke Shumaker [Sun, 2 Dec 2018 18:38:03 +0000 (13:38 -0500)] 
mypy: Fix wrong type annotations

Common cases:

 - An `if foo is None` check clearly indicates that foo should be
   Optional[]:
   * prepare_root(): dev
   * prepare_home(): dev
   * prepare_srv(): dev
   * mount_image(): loopdev, home_dev, srv_dev
   * save_cache(): raw, cache_path
   * link_output_nspawn_settings(): path
   * link_output_checksum(): checksum
   * link_output_root_hash(): root_hash_file
   * link_output_signature(): signature
   * link_output_bmap(): bmap
   * process_settings(): key
   * run_build_script(): raw
 - In general, replace IO[str] with TextIO, except that in many cases
   BinaryIO is correct:
   * create_image() returns binary, not text
   * reuse_cache_image() returns binary, not text
   * attach_image_loopback() takes binary, not text
   * make_verity() returns binary, not text
   * calculate_sha256sum():
     . argument: raw is binary, not text
     . argument: tar is binary, not text
   * make_build_dir():
     . return: (raw|squashfs) is binary, not text
     . return: tar is binary, not text
   * run_build_script(): raw is binary, not text
   * remove_artifacts():
     . argument: raw is binary, not text
     . argument: tar is binary, not text
 - In general, replace IO[bytes] with BinaryIO, except that in many cases
   TextIO is correct:
    * calculate_sha256sum() returns text, not binary
    * calculate_bmap() returns text, not binary
 - Avoid ambiguous IO[Any]:
    * hash_file() writes to a TextIO
    * calculate_bmap() reads a BinaryIO

Special cases:

 - run(): Either returns a CompletedProcess, or doesn't return.  Drop
   the Optional[].
 - mount_image(): ???
 - calculate_sha256sum():
   . argument: root_hash_file is a BinaryIO, not a str
   . argument: nspawn_settings is a BinaryIO, not a str
     Wait, BinaryIO, not TextIO!?  Yes, even though the .nspawn file is
     plaintext.  The IO object we have here is a NamedTemporaryFile copy
     of the user's .nspawn file.  We made that copy in binary mode, to
     avoid pointlessly decoding then encoding the text.
 - remove_glob(): When providing a type hint for '*args', you provide the
   hint for the individual 'arg's, not the 'args' list.

6 years agomypy: Change GPTRootTypePair: collections.namedtuple → typing.NamedTuple
Luke Shumaker [Sun, 2 Dec 2018 16:40:31 +0000 (11:40 -0500)] 
mypy: Change GPTRootTypePair: collections.namedtuple → typing.NamedTuple

6 years agomypy: Create new variables instead of reusing names
Luke Shumaker [Sun, 2 Dec 2018 07:45:04 +0000 (02:45 -0500)] 
mypy: Create new variables instead of reusing names

When we have a "new" with a new type, don't reuse the name of another
variable still in scope.

6 years agomypy: Figure out generators and decorators
Luke Shumaker [Sun, 2 Dec 2018 16:19:37 +0000 (11:19 -0500)] 
mypy: Figure out generators and decorators

 - A @contextlib.contextmanager should return a
   typing.Generator[yield, send, return]
 - Because of <https://github.com/python/mypy/issues/1317>, mypy 0.641
   doesn't think complete_step() has the correct type to be a decorator
   that works the way it does.  So, set

       completestep = complete_step

   But do typecasting to make mypy think it's the correct type.  And use
   `@completestep` instead of `@complete_step`.  That's an easy
   search/replace.

6 years agomypy: Fiddle with making it happy with argcomplete
Luke Shumaker [Sun, 2 Dec 2018 06:46:42 +0000 (01:46 -0500)] 
mypy: Fiddle with making it happy with argcomplete

6 years agomypy: Differentiate partition() from optional_partition()
Luke Shumaker [Wed, 7 Nov 2018 19:46:25 +0000 (14:46 -0500)] 
mypy: Differentiate partition() from optional_partition()

6 years agomypy: Introduce manual cast()s around tempfile.NamedTemporaryFile
Luke Shumaker [Sun, 2 Dec 2018 06:41:38 +0000 (01:41 -0500)] 
mypy: Introduce manual cast()s around tempfile.NamedTemporaryFile

Unfortunately, mypy 0.641 isn't yet smart enough to figure out the
appropriate subtype of IO to return from tempfile.NamedTemporaryFile()
based on whether the mode= argument contains "b", so we'll have to cast()
its return value for now.

If this makes a line wider than 119 characters, move the dir= argument to
the next line.

6 years agomypy: Convert type-hint comments to PEP 526 type-hints
Luke Shumaker [Wed, 7 Nov 2018 15:14:38 +0000 (10:14 -0500)] 
mypy: Convert type-hint comments to PEP 526 type-hints

Commit a415718e27b673032a415d35af41f9d3aa1b9a99 bumped the required
Python version to 3.6; which means that we can now use PEP 526
type-hints, instead of doing it in magical `# type:` comments.

6 years agomypy: .gitignore: Add .mypy_cache/
Luke Shumaker [Sun, 2 Dec 2018 10:01:28 +0000 (05:01 -0500)] 
mypy: .gitignore: Add .mypy_cache/

6 years agoflake8: E741: Rename ambiguous variable
Luke Shumaker [Sun, 2 Dec 2018 05:37:03 +0000 (00:37 -0500)] 
flake8: E741: Rename ambiguous variable

PEP 8 says to avoid `l` as a variable name, because it can be hard to
distinguish from `1` and `I`.  This means that flake8 complains about
the variable inside of ListAction.

However, flake8 doesn't trigger E741 for *argument* names.  Even
though flake8 doesn't complain about it, also rename the argument of
line_join_list() from `l` to `ary`.

6 years agoflake8: E722: Ignore complaints about bare 'except'
Luke Shumaker [Sun, 2 Dec 2018 05:36:27 +0000 (00:36 -0500)] 
flake8: E722: Ignore complaints about bare 'except'

6 years agoflake8: F401: Remove unused import
Luke Shumaker [Sun, 2 Dec 2018 05:35:47 +0000 (00:35 -0500)] 
flake8: F401: Remove unused import

6 years agoflake8: E703: Remove pointless semicolons
Luke Shumaker [Sun, 2 Dec 2018 05:12:31 +0000 (00:12 -0500)] 
flake8: E703: Remove pointless semicolons