]> git.ipfire.org Git - thirdparty/mkosi.git/log
thirdparty/mkosi.git
6 years agoTurn root into a mountpoint 328/head
Lucas De Marchi [Tue, 19 Feb 2019 21:59:31 +0000 (13:59 -0800)] 
Turn root into a mountpoint

By doing this we can safely rely on mount --recursive doing the right
thing.

6 years agomkosi: take a BSD lock on the workspace directory while running
Lennart Poettering [Fri, 18 Jan 2019 16:14:30 +0000 (17:14 +0100)] 
mkosi: take a BSD lock on the workspace directory while running

This makes use of the new logic here:

https://github.com/systemd/systemd/pull/11482

(And on older systems has no effect, but that shouldn't hurt either)

Fixes: #252
6 years agoMerge pull request #329 from afreof/minor-improvements
Zbigniew Jędrzejewski-Szmek [Mon, 25 Feb 2019 06:48:40 +0000 (07:48 +0100)] 
Merge pull request #329 from afreof/minor-improvements

Minor improvements

6 years agoDo not redefine built-in 'type' 329/head
Adrian Freihofer [Sun, 13 Jan 2019 16:25:18 +0000 (17:25 +0100)] 
Do not redefine built-in 'type'

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in id
Adrian Freihofer [Sun, 13 Jan 2019 16:12:01 +0000 (17:12 +0100)] 
Do not redefine built-in id

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in bytes
Adrian Freihofer [Sun, 13 Jan 2019 16:09:36 +0000 (17:09 +0100)] 
Do not redefine built-in bytes

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoRemove unused variable
Adrian Freihofer [Sun, 13 Jan 2019 16:06:05 +0000 (17:06 +0100)] 
Remove unused variable

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoRemove unused variable
Adrian Freihofer [Sun, 13 Jan 2019 16:05:25 +0000 (17:05 +0100)] 
Remove unused variable

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in sum
Adrian Freihofer [Sun, 13 Jan 2019 15:56:09 +0000 (16:56 +0100)] 
Do not redefine built-in sum

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoCosmetics: intentation
Adrian Freihofer [Sun, 13 Jan 2019 15:53:15 +0000 (16:53 +0100)] 
Cosmetics: intentation

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoCosmetics: unused variable
Adrian Freihofer [Sun, 13 Jan 2019 15:52:45 +0000 (16:52 +0100)] 
Cosmetics: unused variable

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine name 'uuid'
Adrian Freihofer [Sun, 13 Jan 2019 15:48:07 +0000 (16:48 +0100)] 
Do not redefine name 'uuid'

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not override run_build_script function
Adrian Freihofer [Sun, 13 Jan 2019 15:35:31 +0000 (16:35 +0100)] 
Do not override run_build_script function

Does not change anything. It's a renaming of a function parameter
from run_build_script to do_run_build_script to resolve the name
clashing with function run_build_script.

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoCosmetics: Fix intentation
Adrian Freihofer [Sun, 13 Jan 2019 15:26:36 +0000 (16:26 +0100)] 
Cosmetics: Fix intentation

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in dir, type
Adrian Freihofer [Sun, 13 Jan 2019 15:25:57 +0000 (16:25 +0100)] 
Do not redefine built-in dir, type

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoDo not redefine built-in bytes
Adrian Freihofer [Sun, 13 Jan 2019 15:25:14 +0000 (16:25 +0100)] 
Do not redefine built-in bytes

Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
6 years agoMerge pull request #327 from lucasdemarchi/tip-arch
Lucas De Marchi [Wed, 20 Feb 2019 16:23:01 +0000 (08:23 -0800)] 
Merge pull request #327 from lucasdemarchi/tip-arch

Fix archlinux on output type directory

6 years agoarch: kill dirmngr only 327/head
Lucas De Marchi [Tue, 19 Feb 2019 21:54:56 +0000 (13:54 -0800)] 
arch: kill dirmngr only

Using fuser to identify processes using a path doesn't work as I wanted:
it relies on the filesystem mounted there. The outcome is that when
using "-t directory" it will try to kill all processes using that
filesystem rather than processes using files under that path.

Even by bind-mounting root so it's always a mountpoint, it doesn't work.
So instead of killing everything, pinpoint what process pacman/pacstrap
left running and kill only that one.

Right now this means telling dirmngr to exit.

6 years agoarch: trust new signatures while downloading base packages
Lucas De Marchi [Tue, 19 Feb 2019 21:53:06 +0000 (13:53 -0800)] 
arch: trust new signatures while downloading base packages

Otherwise it aborts with:
error: libxml2: signature from "Levente Polyak (anthraxx) <levente@leventepolyak.net>" is unknown trust
:: File /var/tmp/mkosi-__fdh4oy/root/var/cache/pacman/pkg/libxml2-2.9.9-1-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]

6 years agoCreate link at output path for tar format builds
Matt Schreiber [Mon, 28 Jan 2019 01:35:50 +0000 (20:35 -0500)] 
Create link at output path for tar format builds

Previously, mkosi created hardlinks at the specified output path only
for the plain squashfs and "disk" formats

6 years agoMerge pull request #321 from lucasdemarchi/tip-fixes
Lucas De Marchi [Fri, 18 Jan 2019 22:28:47 +0000 (14:28 -0800)] 
Merge pull request #321 from lucasdemarchi/tip-fixes

Fixes

6 years agoFix prepend to environ path 321/head
Lucas De Marchi [Wed, 19 Dec 2018 00:19:02 +0000 (16:19 -0800)] 
Fix prepend to environ path

We need to call this after loading the configuration file, otherwise
anything set there won't work.

6 years agoFix compat option for kernel command line
Lucas De Marchi [Wed, 19 Dec 2018 00:18:12 +0000 (16:18 -0800)] 
Fix compat option for kernel command line

We need to set the dest option, otherwise you will be able to set the
option, but it won't have any effect.

6 years agoRemove plural in arg option
Lucas De Marchi [Mon, 26 Nov 2018 20:14:22 +0000 (12:14 -0800)] 
Remove plural in arg option

Like other lists, make it singular in the option. See --package,
--extra-tree, etc.

6 years agoMerge pull request #315 from jlettman/no-check-valid-until
Lennart Poettering [Fri, 18 Jan 2019 16:19:48 +0000 (17:19 +0100)] 
Merge pull request #315 from jlettman/no-check-valid-until

mkosi: add --no-check-valid-until to debootstrap args to fix #314

6 years agomkosi: remove old --no-check-valid-until argument to debootstrap 315/head
John Lettman [Wed, 9 Jan 2019 02:01:27 +0000 (21:01 -0500)] 
mkosi: remove old --no-check-valid-until argument to debootstrap

6 years agomkosi: generalize debootstrap_knows_merged_user and add check for --no-check-valid...
John Lettman [Wed, 9 Jan 2019 01:59:53 +0000 (20:59 -0500)] 
mkosi: generalize debootstrap_knows_merged_user and add check for --no-check-valid-until to fix #314 on other versions of debootstrap

6 years agomkosi: add --no-check-valid-until to debootstrap args to fix #314
John Lettman [Wed, 9 Jan 2019 01:42:20 +0000 (20:42 -0500)] 
mkosi: add --no-check-valid-until to debootstrap args to fix #314

6 years agoCorrect file extension
Hervé [Thu, 3 Jan 2019 12:32:24 +0000 (13:32 +0100)] 
Correct file extension

6 years agoMerge pull request #312 from poettering/man-page-all
Lucas De Marchi [Tue, 18 Dec 2018 15:52:12 +0000 (07:52 -0800)] 
Merge pull request #312 from poettering/man-page-all

mention plain_squashfs in the summary, and document --all

6 years agoman: document the new --all switches 312/head
Lennart Poettering [Tue, 18 Dec 2018 15:46:59 +0000 (16:46 +0100)] 
man: document the new --all switches

6 years agomkosi: mention that plain_squashfs output format exists now
Lennart Poettering [Tue, 18 Dec 2018 15:46:48 +0000 (16:46 +0100)] 
mkosi: mention that plain_squashfs output format exists now

As requested here:

https://github.com/systemd/mkosi/commit/bb5dad8d4368c32abd008a886999c506448cc3e2#commitcomment-31710412

6 years agoMerge pull request #308 from poettering/man-page
Lennart Poettering [Mon, 17 Dec 2018 12:24:37 +0000 (13:24 +0100)] 
Merge pull request #308 from poettering/man-page

add a man page

6 years agodocs: Replace README.md by a pointer to the man page 308/head
Lennart Poettering [Fri, 14 Dec 2018 20:26:02 +0000 (21:26 +0100)] 
docs: Replace README.md by a pointer to the man page

The man page contains most relevant stuff now, and is a lot more
comprehensive, hence let's drop these explanations from README.md and
point people directly to the man page if they want to know more.

6 years agoman: add a man page
Lennart Poettering [Fri, 14 Dec 2018 20:22:58 +0000 (21:22 +0100)] 
man: add a man page

This adds a pretty comprehensive man page as a MarkDown document.

This does not contain any build system hookup, simply because I couldn't
figure out how to teach Python's setuptools to invoke external programs
as part of the build process.

Currently a manual step is necessary, to convert the man page into an
actual man page:

        # pandoc mkosi.md -s -t man > mkosi.1

The man page is based on the old README.md but substantially extended.

Fixes: #210
6 years agomkosi: move --no-chown to [Output] section in --help text
Lennart Poettering [Fri, 14 Dec 2018 19:59:27 +0000 (20:59 +0100)] 
mkosi: move --no-chown to [Output] section in --help text

6 years agomkosi: move --hostname= to right section
Lennart Poettering [Fri, 14 Dec 2018 19:50:16 +0000 (20:50 +0100)] 
mkosi: move --hostname= to right section

6 years agomkosi: --kernel-commandline= → --kernel-command-line=
Lennart Poettering [Fri, 14 Dec 2018 19:42:46 +0000 (20:42 +0100)] 
mkosi: --kernel-commandline= → --kernel-command-line=

This was just weird, in particular as the mkosi.default option was
aleady called KernelCommandLine=.

Also, let's move this to the right section in the --help text.

6 years agomkosi: be precise with the naming of the squashfs output modes
Lennart Poettering [Fri, 14 Dec 2018 19:14:36 +0000 (20:14 +0100)] 
mkosi: be precise with the naming of the squashfs output modes

6 years agomkosi: pass $WITH_NETWORK= to mkosi.build script
Lennart Poettering [Fri, 14 Dec 2018 19:12:09 +0000 (20:12 +0100)] 
mkosi: pass $WITH_NETWORK= to mkosi.build script

Let's tell the build script whether it is running with networking or
not. I don#t have an immediate usecase for this, but it's more
systematic given how --with-docs and --without-tests is currently
handled and result in environment variables, bus --with-network is not.

A usecase might be to generate the `-nonet` option of `xsltproc` from
this option, but that's purely theoretical.

6 years agomkosi: don't use mkosi.skeleton/mkosi.extra trees when one is specified explicitly
Lennart Poettering [Fri, 14 Dec 2018 19:10:54 +0000 (20:10 +0100)] 
mkosi: don't use mkosi.skeleton/mkosi.extra trees when one is specified explicitly

For our other options the auto-discovery of mkosi.xyz resources is
generally skipped if explicitly configured, do this here, too.

6 years agomkosi: when "gpt_xfs" is used, format /home and /srv with xfs too
Lennart Poettering [Fri, 14 Dec 2018 16:48:53 +0000 (17:48 +0100)] 
mkosi: when "gpt_xfs" is used, format /home and /srv with xfs too

6 years agoadd a TODO document
Lennart Poettering [Fri, 14 Dec 2018 16:40:07 +0000 (17:40 +0100)] 
add a TODO document

6 years agomkosi: always mention XFS before btrfs, as it is more like ext4
Lennart Poettering [Fri, 14 Dec 2018 16:38:22 +0000 (17:38 +0100)] 
mkosi: always mention XFS before btrfs, as it is more like ext4

ext4 and xfs images are pretty much put together the same way, except
for the file system used. btrfs otoh is different as instead of a GPT
partition table any subdivisions are are done as btrfs subvolumes.
Hence, let's reorder the enums a bit, and always handle ext4/xfs
together, and btrfs after that.

6 years agoMerge pull request #306 from keszybz/style-fixes
Lennart Poettering [Mon, 17 Dec 2018 11:51:19 +0000 (12:51 +0100)] 
Merge pull request #306 from keszybz/style-fixes

Two trivial stylistic fixups

6 years agoUse single quotes type instead of escaping 306/head
Zbigniew Jędrzejewski-Szmek [Fri, 14 Dec 2018 12:22:30 +0000 (13:22 +0100)] 
Use single quotes type instead of escaping

That's just more readable and shorter.

6 years agoMerge pull request #309 from poettering/mkosi-all
Lucas De Marchi [Mon, 17 Dec 2018 02:23:52 +0000 (18:23 -0800)] 
Merge pull request #309 from poettering/mkosi-all

add new "mkosi -a" switch for building all images in mkosi.files/

6 years agomkosi: add "-a" switch for building everything in mkosi.files/ 309/head
Lennart Poettering [Sat, 15 Dec 2018 09:44:55 +0000 (10:44 +0100)] 
mkosi: add "-a" switch for building everything in mkosi.files/

Let's add a one-stop solution for building a set of files all in one go.
For that, simply place "mkosi.default"-style files in mkosi.files/ and
use "mkosi -a" to build them all.

Fixes: #176
6 years agomkosi: rename copy() → copy_path() so that we can import copy module without hassles
Lennart Poettering [Sat, 15 Dec 2018 09:44:02 +0000 (10:44 +0100)] 
mkosi: rename copy() → copy_path() so that we can import copy module without hassles

6 years agoMerge pull request #311 from poettering/debian-root-mkdir
Lucas De Marchi [Sat, 15 Dec 2018 16:34:51 +0000 (08:34 -0800)] 
Merge pull request #311 from poettering/debian-root-mkdir

mkosi: create root directory as 0755

6 years agomkosi: create root directory as 0755 311/head
Lennart Poettering [Sat, 15 Dec 2018 10:29:30 +0000 (11:29 +0100)] 
mkosi: create root directory as 0755

Fixes: #253
6 years agotyping: one more type annotation to make mypy happy
Zbigniew Jędrzejewski-Szmek [Fri, 14 Dec 2018 14:22:25 +0000 (15:22 +0100)] 
typing: one more type annotation to make mypy happy

6 years agoMerge pull request #300 from LukeShu:to-upstream/flake8+mypy
Zbigniew Jędrzejewski-Szmek [Fri, 14 Dec 2018 14:17:24 +0000 (15:17 +0100)] 
Merge pull request #300 from LukeShu:to-upstream/flake8+mypy

Merging locally to resolve the few conflicts.

6 years agoFix default value for SourceFileTransfer
Lucas De Marchi [Fri, 7 Dec 2018 17:03:29 +0000 (09:03 -0800)] 
Fix default value for SourceFileTransfer

To retain the same behavior as before folding the options into one.

6 years agomkosi: when a .nspawn file is used, actually use it when running nspawn
Lennart Poettering [Thu, 6 Dec 2018 20:08:23 +0000 (21:08 +0100)] 
mkosi: when a .nspawn file is used, actually use it when running nspawn

Previously, we'd move it in place, but when invoking nspawn on "mkosi
shell" or "mkosi boot" it would be ignored, since that's what nspawn
does unless the file is in /etc or /run.

Let's change this and mark it trusted, after all it's coming from the
same source that just built the image.

This is very useful for cases where you want to default to
"--private-network" when "mkosi boot" is invoked.

6 years agoMerge pull request #303 from michaelweghorn/michaelweghorn/updateHelp
Lucas De Marchi [Wed, 5 Dec 2018 16:20:08 +0000 (08:20 -0800)] 
Merge pull request #303 from michaelweghorn/michaelweghorn/updateHelp

Update help for '--with-docs'

6 years agoUpdate help for '--with-docs' 303/head
Michael Weghorn [Wed, 5 Dec 2018 14:23:41 +0000 (15:23 +0100)] 
Update help for '--with-docs'

The restriction "only Fedora, CentOS and Mageia"
does not (no longer?) apply, since the option is also handled for
other distributions (Debian, Ubuntu, openSUSE).

6 years agoMerge pull request #292 from lucasdemarchi/tip-build-src
Lucas De Marchi [Tue, 4 Dec 2018 18:44:53 +0000 (10:44 -0800)] 
Merge pull request #292 from lucasdemarchi/tip-build-src

Mount build source

6 years agoMerge pull request #299 from michaelweghorn/michaelweghorn/addBiosSupportForDebianAnd...
Lucas De Marchi [Tue, 4 Dec 2018 18:44:04 +0000 (10:44 -0800)] 
Merge pull request #299 from michaelweghorn/michaelweghorn/addBiosSupportForDebianAndUbuntu

Add BIOS support for Debian and Ubuntu

6 years agomypy: Fix issues discovered by mypy 300/head
Luke Shumaker [Sun, 2 Dec 2018 07:46:22 +0000 (02:46 -0500)] 
mypy: Fix issues discovered by mypy

Mostly issues with not handling things that may be None.

6 years agomypy: Fiddle with things to clue mypy in
Luke Shumaker [Sun, 2 Dec 2018 10:05:09 +0000 (05:05 -0500)] 
mypy: Fiddle with things to clue mypy in

But this shouldn't actually change anything; this is just making
stylistic changes to the code to clue mypy in to some stuff.

6 years agomypy: Unify typing of install_distribution() and install_{DISTRIBUTION}()
Luke Shumaker [Sun, 2 Dec 2018 16:51:05 +0000 (11:51 -0500)] 
mypy: Unify typing of install_distribution() and install_{DISTRIBUTION}()

6 years agomypy: invoke_dnf_or_yum(): Match the signature of invoke_dnf() and invoke_yum()
Luke Shumaker [Sun, 2 Dec 2018 16:43:52 +0000 (11:43 -0500)] 
mypy: invoke_dnf_or_yum(): Match the signature of invoke_dnf() and invoke_yum()

6 years agomypy: ListAction: assert that 'values' is a string
Luke Shumaker [Sun, 2 Dec 2018 19:01:00 +0000 (14:01 -0500)] 
mypy: ListAction: assert that 'values' is a string

We must declare 'values' as Union[str, Sequence[Any], None], to keep the
__call__ definition compatible with the parent class.  However, we know
(and rely on) that how we call .add_argument() means that it will always
be a str.  Add an 'assert' to let mypy know what we know.

6 years agomypy: Make arguments Optional[], assert is not None as appropriate
Luke Shumaker [Sun, 2 Dec 2018 18:38:46 +0000 (13:38 -0500)] 
mypy: Make arguments Optional[], assert is not None as appropriate

There are several places where we unconditionally call a function, then
that function checks `args` to decide if it immediately returns, or if it
actually gets to do work.  In the cases where it immediately returns, some
of the arguments can be None, so they should be hinted as Optional[].
However, in the case were it does get to do work, they cannot be None.

So, hint them as Optional[], and the add `assert ARGNAME is not None`
after the check that implies it's not None, to let mypy know what we know.

For instance: args.output_format.is_disk() implies that raw is not None.
So it's safe to make the change:

         if not args.output_format.is_disk():
             return None
    +    assert raw is not None

6 years agomypy: Add missing type annotations
Luke Shumaker [Sun, 2 Dec 2018 07:20:12 +0000 (02:20 -0500)] 
mypy: Add missing type annotations

6 years agomypy: Fix wrong type annotations
Luke Shumaker [Sun, 2 Dec 2018 18:38:03 +0000 (13:38 -0500)] 
mypy: Fix wrong type annotations

Common cases:

 - An `if foo is None` check clearly indicates that foo should be
   Optional[]:
   * prepare_root(): dev
   * prepare_home(): dev
   * prepare_srv(): dev
   * mount_image(): loopdev, home_dev, srv_dev
   * save_cache(): raw, cache_path
   * link_output_nspawn_settings(): path
   * link_output_checksum(): checksum
   * link_output_root_hash(): root_hash_file
   * link_output_signature(): signature
   * link_output_bmap(): bmap
   * process_settings(): key
   * run_build_script(): raw
 - In general, replace IO[str] with TextIO, except that in many cases
   BinaryIO is correct:
   * create_image() returns binary, not text
   * reuse_cache_image() returns binary, not text
   * attach_image_loopback() takes binary, not text
   * make_verity() returns binary, not text
   * calculate_sha256sum():
     . argument: raw is binary, not text
     . argument: tar is binary, not text
   * make_build_dir():
     . return: (raw|squashfs) is binary, not text
     . return: tar is binary, not text
   * run_build_script(): raw is binary, not text
   * remove_artifacts():
     . argument: raw is binary, not text
     . argument: tar is binary, not text
 - In general, replace IO[bytes] with BinaryIO, except that in many cases
   TextIO is correct:
    * calculate_sha256sum() returns text, not binary
    * calculate_bmap() returns text, not binary
 - Avoid ambiguous IO[Any]:
    * hash_file() writes to a TextIO
    * calculate_bmap() reads a BinaryIO

Special cases:

 - run(): Either returns a CompletedProcess, or doesn't return.  Drop
   the Optional[].
 - mount_image(): ???
 - calculate_sha256sum():
   . argument: root_hash_file is a BinaryIO, not a str
   . argument: nspawn_settings is a BinaryIO, not a str
     Wait, BinaryIO, not TextIO!?  Yes, even though the .nspawn file is
     plaintext.  The IO object we have here is a NamedTemporaryFile copy
     of the user's .nspawn file.  We made that copy in binary mode, to
     avoid pointlessly decoding then encoding the text.
 - remove_glob(): When providing a type hint for '*args', you provide the
   hint for the individual 'arg's, not the 'args' list.

6 years agomypy: Change GPTRootTypePair: collections.namedtuple → typing.NamedTuple
Luke Shumaker [Sun, 2 Dec 2018 16:40:31 +0000 (11:40 -0500)] 
mypy: Change GPTRootTypePair: collections.namedtuple → typing.NamedTuple

6 years agomypy: Create new variables instead of reusing names
Luke Shumaker [Sun, 2 Dec 2018 07:45:04 +0000 (02:45 -0500)] 
mypy: Create new variables instead of reusing names

When we have a "new" with a new type, don't reuse the name of another
variable still in scope.

6 years agomypy: Figure out generators and decorators
Luke Shumaker [Sun, 2 Dec 2018 16:19:37 +0000 (11:19 -0500)] 
mypy: Figure out generators and decorators

 - A @contextlib.contextmanager should return a
   typing.Generator[yield, send, return]
 - Because of <https://github.com/python/mypy/issues/1317>, mypy 0.641
   doesn't think complete_step() has the correct type to be a decorator
   that works the way it does.  So, set

       completestep = complete_step

   But do typecasting to make mypy think it's the correct type.  And use
   `@completestep` instead of `@complete_step`.  That's an easy
   search/replace.

6 years agomypy: Fiddle with making it happy with argcomplete
Luke Shumaker [Sun, 2 Dec 2018 06:46:42 +0000 (01:46 -0500)] 
mypy: Fiddle with making it happy with argcomplete

6 years agomypy: Differentiate partition() from optional_partition()
Luke Shumaker [Wed, 7 Nov 2018 19:46:25 +0000 (14:46 -0500)] 
mypy: Differentiate partition() from optional_partition()

6 years agomypy: Introduce manual cast()s around tempfile.NamedTemporaryFile
Luke Shumaker [Sun, 2 Dec 2018 06:41:38 +0000 (01:41 -0500)] 
mypy: Introduce manual cast()s around tempfile.NamedTemporaryFile

Unfortunately, mypy 0.641 isn't yet smart enough to figure out the
appropriate subtype of IO to return from tempfile.NamedTemporaryFile()
based on whether the mode= argument contains "b", so we'll have to cast()
its return value for now.

If this makes a line wider than 119 characters, move the dir= argument to
the next line.

6 years agomypy: Convert type-hint comments to PEP 526 type-hints
Luke Shumaker [Wed, 7 Nov 2018 15:14:38 +0000 (10:14 -0500)] 
mypy: Convert type-hint comments to PEP 526 type-hints

Commit a415718e27b673032a415d35af41f9d3aa1b9a99 bumped the required
Python version to 3.6; which means that we can now use PEP 526
type-hints, instead of doing it in magical `# type:` comments.

6 years agomypy: .gitignore: Add .mypy_cache/
Luke Shumaker [Sun, 2 Dec 2018 10:01:28 +0000 (05:01 -0500)] 
mypy: .gitignore: Add .mypy_cache/

6 years agoflake8: E741: Rename ambiguous variable
Luke Shumaker [Sun, 2 Dec 2018 05:37:03 +0000 (00:37 -0500)] 
flake8: E741: Rename ambiguous variable

PEP 8 says to avoid `l` as a variable name, because it can be hard to
distinguish from `1` and `I`.  This means that flake8 complains about
the variable inside of ListAction.

However, flake8 doesn't trigger E741 for *argument* names.  Even
though flake8 doesn't complain about it, also rename the argument of
line_join_list() from `l` to `ary`.

6 years agoflake8: E722: Ignore complaints about bare 'except'
Luke Shumaker [Sun, 2 Dec 2018 05:36:27 +0000 (00:36 -0500)] 
flake8: E722: Ignore complaints about bare 'except'

6 years agoflake8: F401: Remove unused import
Luke Shumaker [Sun, 2 Dec 2018 05:35:47 +0000 (00:35 -0500)] 
flake8: F401: Remove unused import

6 years agoflake8: E703: Remove pointless semicolons
Luke Shumaker [Sun, 2 Dec 2018 05:12:31 +0000 (00:12 -0500)] 
flake8: E703: Remove pointless semicolons

6 years agoflake8: E111,E128: Fixup indentation
Luke Shumaker [Sun, 2 Dec 2018 05:06:36 +0000 (00:06 -0500)] 
flake8: E111,E128: Fixup indentation

 - E111: Normal indents should be multiples of 4
 - E128: Fixup visual indents

6 years agoflake8: E201,E202,E203,E221,E222,E251: Remove individual spaces around things
Luke Shumaker [Sun, 2 Dec 2018 05:35:32 +0000 (00:35 -0500)] 
flake8: E201,E202,E203,E221,E222,E251: Remove individual spaces around things

 - E201: Don't put a space after the `[` in a list:
       ary = [ item,
              ^
 - E201: Don't put a space after the `{` in a dict:
       d = { key: val,
            ^
 - E202: Don't put a space before the `]` in a list:
       ary = [item1, item2 ]
                          ^
 - E202: Don't put a space before the `}` in a dict:
       d = [key: val }
                    ^
 - E203: Don't put a space before the `:` in a dict:
       d = {key : val}
               ^
 - E221: Write `# NOQA: E221` if we're lining up the `=` in a list of
   assignments
 - E222: Write `# NOQA: E222` if we're lining numbers on the right
   side of `=` in a list of assignments
 - E251: When defining an argument a default value, and the argument
   isn't type-hinted, don't put spaces around the `=`:
       def fn(argname = default) -> type:
                     ^ ^
   This is in contrast with type-hinted arguments with defaults, which
   DO get spaces around the `=` (E252).
 - E251: Don't put spaces around the `=` when passing a keyword
   argument to a function:
       fn(kw = val)
            ^ ^

6 years agoflake8: E225,E227,E231,E252,E261: Add individual spaces around things
Luke Shumaker [Sun, 2 Dec 2018 05:04:12 +0000 (00:04 -0500)] 
flake8: E225,E227,E231,E252,E261: Add individual spaces around things

 - E225: Put a space on each side of the `+` operator
 - E227: Put a space on each side of the `|` operator
 - E227: Put a space on each side of the `<<` operator
 - E231: Put a space after `,` in a list/set/tuple.
 - E231: Put a space after the `:` when defining a lambda
 - E231: Put a space after the `:` in dict literals
 - E231: When type-hinting an argument, put a space between the `:`
   and the type:
       def fn(argname: type) -> type:
                      ^
 - E252: When type-hinting an argument with a default value, put
   spaces around the `=`:
       def fn(argname: type = default) -> type:
                           ^ ^
   This is in contrast with non-type-hinted arguments with defaults,
   which do NOT get spaces around the `=` (E251).
 - E261: Put two spaces before the `#` of an inline comment

6 years agoflake8: E301: At least 1 blank line between method definitions
Luke Shumaker [Sun, 2 Dec 2018 04:47:34 +0000 (23:47 -0500)] 
flake8: E301: At least 1 blank line between method definitions

6 years agoflake8: E302,E305: 2 blank lines around class/function definitions
Luke Shumaker [Sun, 2 Dec 2018 04:45:06 +0000 (23:45 -0500)] 
flake8: E302,E305: 2 blank lines around class/function definitions

This does not apply to method definitions.

6 years agoflake8: E501: Address lines that are 120 characters or longer
Luke Shumaker [Sun, 2 Dec 2018 04:27:16 +0000 (23:27 -0500)] 
flake8: E501: Address lines that are 120 characters or longer

 - Set the maximum line length to 119, same as systemd.git
 - If the line is a function definition that is long because of many args,
   put one arg per line
 - Don't split user-facing strings over multiple lines; use NOQA to silence
   those complaints
 - In a few cases, re-structure the code to be easier to follow

This doesn't blindly modify the code to flake8's complaints, it looks at
long lines, and sees if we can make them easier to read.

6 years agoflake8-isort: Run `isort *.py`
Luke Shumaker [Sun, 2 Dec 2018 03:50:14 +0000 (22:50 -0500)] 
flake8-isort: Run `isort *.py`

And add `setup.cfg` to configure isort.

6 years agoAvoid multi-line lambdas; use a regular `def` in those cases
Luke Shumaker [Sun, 2 Dec 2018 05:13:58 +0000 (00:13 -0500)] 
Avoid multi-line lambdas; use a regular `def` in those cases

They are hard to read.

6 years agoAdd BIOS support for Debian and Ubuntu 299/head
Michael Weghorn [Sat, 1 Dec 2018 22:36:17 +0000 (23:36 +0100)] 
Add BIOS support for Debian and Ubuntu

The relevant package to support BIOS boot on these platforms
is called 'grub-pc'.

6 years agoMerge pull request #298 from lucasdemarchi/tip-fix-arch
Lucas De Marchi [Fri, 30 Nov 2018 22:05:26 +0000 (14:05 -0800)] 
Merge pull request #298 from lucasdemarchi/tip-fix-arch

Fix non-bootable Arch images

6 years agoFix package manager cleanup
Lucas De Marchi [Thu, 29 Nov 2018 22:45:12 +0000 (14:45 -0800)] 
Fix package manager cleanup

As the cache is still mounted when we try to cleanup the metadata, it
was removing the downloaded packages from mkosi.cache (or whatever path
the user specified).

Tested Fedora only, with multiple invocations of

    touch mkosi.cache

and multiple invocations of

    sudo mkosi -f --default /dev/null  -d fedora -r 29
    find mkosi.cache -name '*.rpm'

6 years agoFix non-bootable Arch images 298/head
Lucas De Marchi [Thu, 29 Nov 2018 23:29:59 +0000 (15:29 -0800)] 
Fix non-bootable Arch images

6 years agoAllow to mount build sources 292/head
Lucas De Marchi [Wed, 21 Nov 2018 09:16:13 +0000 (01:16 -0800)] 
Allow to mount build sources

This allows us to just mount the build sources into the build image in
order to run the script. By doing this the user may avoid problems with
recursive submodules and at the same time speed up execution for large
trees. This means the user will need to take care of possibly cleaning
up the tree if he wants to make sure he has a clean build.

6 years agoFold --use-git-files and --git-files into one option
Lucas De Marchi [Tue, 20 Nov 2018 16:01:25 +0000 (08:01 -0800)] 
Fold --use-git-files and --git-files into one option

Besides being nicer to have a single option, it will allow us to easily
extend with new copy methods. While at it, also add an entry in the
config file.

6 years agoMake none_to_none compatible with other objects
Lucas De Marchi [Wed, 21 Nov 2018 20:21:58 +0000 (12:21 -0800)] 
Make none_to_none compatible with other objects

6 years agoMerge pull request #261 from rantogno/legacy_boot
Lennart Poettering [Wed, 21 Nov 2018 10:45:42 +0000 (11:45 +0100)] 
Merge pull request #261 from rantogno/legacy_boot

Add BootProtocols option

6 years agoset correct defaults for optional args
Thore Bödecker [Tue, 20 Nov 2018 13:19:32 +0000 (14:19 +0100)] 
set correct defaults for optional args

args.with_network and args.with_docs were always initialized to `False`
thus ignoring any setting in a config file (WithNetwork/WithDocs).

This breaks the config parser (load_defaults) because it checks if the
current value of `key` is None before assigning the value from the
config file.

Signed-off-by: Thore Bödecker <me@foxxx0.de>
6 years agoAdd bios support to Archlinux 261/head
Lucas De Marchi [Sat, 17 Nov 2018 00:24:18 +0000 (16:24 -0800)] 
Add bios support to Archlinux

In order to add support to boot on bios mode for Archlinux, the
following changes were made:

    - kernel packages are now installed as a second step: this allows us
      to patch the initrd config file so to disable the autodetection
      hook
    - patching the initrd config is now part of the distro install
      rather than bootloader install

uefi-only is the default and we can generate uefi-only, bios-only and
hybrid images. In the latter case grub is used for bios and systemd-boot
for UEFI.

6 years agoMove grub config generation to grub installation
Lucas De Marchi [Fri, 16 Nov 2018 09:25:11 +0000 (01:25 -0800)] 
Move grub config generation to grub installation

This allows it to be compatible with distros that ship a default configuration
file. Now we patch the file if it exits, or create one otherwise.

6 years agoAdd support to boot on bios
Rafael Antognolli [Tue, 6 Nov 2018 16:30:10 +0000 (08:30 -0800)] 
Add support to boot on bios

This implements boot on systems with legacy bios rather
than UEFI. It works by installing grub-pc and systemd-boot
side by side. The former is responsible for bios boot while
the latter continues to be the bootloader for UEFI.

The resulting image, that is currently working only on Fedora,
is capable of booting on both systems. Generate the image with:

    $ sudo mkosi -d fedora -r 29 -t raw_ext4 -b --boot-protocols uefi bios

Boot with UEFI:

    $ sudo qemu-system-x86_64 -machine accel=kvm -smp 2 -m 1024 \
        -drive if=pflash,format=raw,readonly,file=/usr/share/edk2/ovmf/OVMF_CODE.fd \
        -drive format=raw,file=image.raw

Boot with bios:

    $ sudo qemu-system-x86_64 -machine accel=kvm -smp 2 -m 1024 \
        -drive format=raw,file=image.raw

Lucas: This is a cleanup of the implementation done by Rafael
and allowing to have the hybrid approach rather than mandating one
or the other.

6 years agoAdd option for boot protocols
Lucas De Marchi [Thu, 1 Nov 2018 05:27:46 +0000 (22:27 -0700)] 
Add option for boot protocols

This will allow us to install different boot protocols like
UEFI, BIOS and possibly others.