mkosi: Replace submodules with our own thing

Unfortunately, git submodules break in all sorts of ways:

- Various github workflows (dependabot, github pages) try to do a shallow
clone of git submodules which does not work at all when the git repository
is hosted on pagure (https://pagure.io/pagure/issue/5453,
https://github.com/dependabot/dependabot-core/issues/9391).
- If the git forge hosting the git repository uses SHA256, then it breaks our
usage of it as a submodule as SHA256 repositories cannot be used as submodules
in SHA1 repositories (src.opensuse.org moved to SHA256 which broke our usage of
opensuse's systemd spec as a submodule).
- git submodules completely break usage of git worktrees.
- ...

Let's avoid all these issues by just doing our own home grown implementation of
git submodules. We lose the automatic dependabot updates this way but since dependabot
fails to run more often that not with submodules we don't really lose anything.

diff --git a/.github/dependabot.yml b/.github/dependabot.yml
index 54593c50f7bd04119cc8bc520c66af1ad07da760..3e067c176fd4f9258104336379c8b7b6cf16812a 100644 (file)
--- a/.github/dependabot.yml
+++ b/.github/dependabot.yml
@@ -17,8 +17,3 @@ updates:
     schedule:
       interval: "monthly"
     open-pull-requests-limit: 2
-  - package-ecosystem: "gitsubmodule"
-    directory: "/"
-    schedule:
-      interval: "daily"
-    open-pull-requests-limit: 2

diff --git a/.gitmodules b/.gitmodules
index 7ae6ace6802b28cd8de987f419bb965c6f9321da..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 100644 (file)
--- a/.gitmodules
+++ b/.gitmodules
@@ -1,20 +0,0 @@
-[submodule "pkg/fedora"]
-       path = pkg/fedora
-       url = https://src.fedoraproject.org/rpms/systemd.git
-       branch = rawhide
-[submodule "pkg/opensuse"]
-       path = pkg/opensuse
-       url = https://code.opensuse.org/package/systemd.git
-       branch = master
-[submodule "pkg/debian"]
-       path = pkg/debian
-       url = https://salsa.debian.org/systemd-team/systemd.git
-       branch = debian/master
-[submodule "pkg/centos"]
-       path = pkg/centos
-       url = https://git.centos.org/rpms/systemd.git
-       branch = c9s-sig-hyperscale
-[submodule "pkg/arch"]
-       path = pkg/arch
-       url = https://gitlab.archlinux.org/archlinux/packaging/packages/systemd.git
-       branch = main

diff --git a/docs/HACKING.md b/docs/HACKING.md
index 980a45929eb8653f8b8578c739fadce26ad7d264..51499d7f7986f7bf3a13b5a301f6af4dc4e5f790 100644 (file)
--- a/docs/HACKING.md
+++ b/docs/HACKING.md
@@ -22,15 +22,6 @@ If adding a new source file, consider adding a matching test executable.
 For features at a higher level, tests in `src/test/` are very strongly recommended.
 If that is not possible, integration tests in `test/` are encouraged.
 
-```shell
-$ git config submodule.recurse true
-$ git config fetch.recurseSubmodules on-demand
-$ git config push.recurseSubmodules no
-$ cp .git/hooks/pre-commit.sample .git/hooks/pre-commit
-$ cp tools/git-submodule-update-hook.sh .git/hooks/post-rewrite
-$ cp tools/git-submodule-update-hook.sh .git/hooks/post-checkout
-```
-
 Please always test your work before submitting a PR.
 For many of the components of systemd testing is straightforward as you can simply compile systemd and run the relevant tool from the build directory.
 

diff --git a/mkosi.images/system/mkosi.conf.d/10-arch/mkosi.conf b/mkosi.images/system/mkosi.conf.d/10-arch/mkosi.conf
index 942db4cb967c922cb30cdf18b33cf887082d8f8e..9c7d09d6562336ee0efb79fffde25a02987ab2a1 100644 (file)
--- a/mkosi.images/system/mkosi.conf.d/10-arch/mkosi.conf
+++ b/mkosi.images/system/mkosi.conf.d/10-arch/mkosi.conf
@@ -4,6 +4,11 @@
 Distribution=arch
 
 [Content]
+Environment=
+        GIT_URL=https://gitlab.archlinux.org/archlinux/packaging/packages/systemd.git
+        GIT_BRANCH=main
+        GIT_COMMIT=dc6c099e0785753c1c88b4adcbcbfc209a8d12e3
+
 VolatilePackages=
         systemd
         systemd-libs

diff --git a/mkosi.images/system/mkosi.conf.d/10-centos/mkosi.conf b/mkosi.images/system/mkosi.conf.d/10-centos/mkosi.conf
index 15f888168dc38adc54d5075f4b59183626192331..395a22b56a80b5aad2de187ae4d84b0db8cfdb5a 100644 (file)
--- a/mkosi.images/system/mkosi.conf.d/10-centos/mkosi.conf
+++ b/mkosi.images/system/mkosi.conf.d/10-centos/mkosi.conf
@@ -4,6 +4,11 @@
 Distribution=centos
 
 [Content]
+Environment=
+        GIT_URL=https://git.centos.org/rpms/systemd.git
+        GIT_BRANCH=c9s-sig-hyperscale
+        GIT_COMMIT=8cf2aed0181920611421384f7374720db269d6c7
+
 Packages=
         kernel-modules # For squashfs
         rpmautospec-rpm-macros

diff --git a/mkosi.images/system/mkosi.conf.d/10-debian-ubuntu/mkosi.conf b/mkosi.images/system/mkosi.conf.d/10-debian-ubuntu/mkosi.conf
index 7e870ace870fe9a765a7770250ad18627576c30c..047dbc1dc3d25c61ff3600ccc12a793cd380e6ca 100644 (file)
--- a/mkosi.images/system/mkosi.conf.d/10-debian-ubuntu/mkosi.conf
+++ b/mkosi.images/system/mkosi.conf.d/10-debian-ubuntu/mkosi.conf
@@ -5,6 +5,11 @@ Distribution=|debian
 Distribution=|ubuntu
 
 [Content]
+Environment=
+        GIT_URL=https://salsa.debian.org/systemd-team/systemd.git
+        GIT_BRANCH=debian/master
+        GIT_COMMIT=18201fa98d74172fa1a17242326e3275995cde13
+
 VolatilePackages=
         libnss-myhostname
         libnss-mymachines

diff --git a/mkosi.images/system/mkosi.conf.d/10-fedora/mkosi.conf b/mkosi.images/system/mkosi.conf.d/10-fedora/mkosi.conf
index 3acec1092c39c22c0c514af2998b02c2d05d53d3..7b122e3c699e88d9ae06679e88fafff86f8afc14 100644 (file)
--- a/mkosi.images/system/mkosi.conf.d/10-fedora/mkosi.conf
+++ b/mkosi.images/system/mkosi.conf.d/10-fedora/mkosi.conf
@@ -4,6 +4,11 @@
 Distribution=fedora
 
 [Content]
+Environment=
+        GIT_URL=https://src.fedoraproject.org/rpms/systemd.git
+        GIT_BRANCH=rawhide
+        GIT_COMMIT=74810c5bc4fe7d872e54c253447ffd61bbc8839f
+
 Packages=
         btrfs-progs
         compsize

diff --git a/mkosi.images/system/mkosi.conf.d/10-opensuse/mkosi.conf b/mkosi.images/system/mkosi.conf.d/10-opensuse/mkosi.conf
index b3172bfc1434008d03642ff361b0cea77f206172..90844d4db40a0e581e0d233d681d8abc110e856a 100644 (file)
--- a/mkosi.images/system/mkosi.conf.d/10-opensuse/mkosi.conf
+++ b/mkosi.images/system/mkosi.conf.d/10-opensuse/mkosi.conf
@@ -7,6 +7,11 @@ Distribution=opensuse
 InitrdInclude=initrd/
 
 [Content]
+Environment=
+        GIT_URL=https://src.opensuse.org/rpm/systemd
+        GIT_BRANCH=factory
+        GIT_COMMIT=28c9bef337dd47c88adc1d948cedfdc79c788e2955f17a534bd76da6419722dd
+
 VolatilePackages=
         systemd
         systemd-boot

diff --git a/mkosi.images/system/mkosi.sync b/mkosi.images/system/mkosi.sync
index 3e42a09b2c602ec5635b27078ca78ba8fd7ef353..a4f0ab94ec185fe379837659fddee544bbf2e9b6 100755 (executable)
--- a/mkosi.images/system/mkosi.sync
+++ b/mkosi.images/system/mkosi.sync
@@ -2,8 +2,21 @@
 # SPDX-License-Identifier: LGPL-2.1-or-later
 set -e
 
-if [ -z "$(ls --almost-all "pkg/$DISTRIBUTION")" ] || [ -f "pkg/$DISTRIBUTION/.git" ]; then
-    PKG_SUBDIR="$(realpath "pkg/$DISTRIBUTION" --relative-to "$PWD")"
-    git submodule sync "$PKG_SUBDIR"
-    git submodule update --init "$PKG_SUBDIR"
+if ((NO_SYNC)); then
+    exit 0
 fi
+
+PKG_SUBDIR="$(realpath --canonicalize-missing "pkg/$DISTRIBUTION" --relative-to "$PWD")"
+
+if [[ -d "$PKG_SUBDIR/.git" ]] && [[ "$(git -C "$PKG_SUBDIR" rev-parse HEAD)" == "$GIT_COMMIT" ]]; then
+    exit 0
+fi
+
+if [[ ! -e "$PKG_SUBDIR" ]] || [[ -z "$(ls --almost-all "$PKG_SUBDIR")" ]]; then
+    git clone "$GIT_URL" --branch "$GIT_BRANCH" "$PKG_SUBDIR"
+else
+    git -C "$PKG_SUBDIR" remote set-url origin "$GIT_URL"
+    git -C "$PKG_SUBDIR" fetch origin "$GIT_BRANCH"
+fi
+
+git -C "$PKG_SUBDIR" -c advice.detachedHead=false checkout "$GIT_COMMIT"

diff --git a/pkg/arch b/pkg/arch
deleted file mode 160000 (submodule)
index b578e90..0000000
--- a/pkg/arch
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit b578e90bb68deaae28300300c57b6a99c916b201

diff --git a/pkg/centos b/pkg/centos
deleted file mode 160000 (submodule)
index 61bb5f4..0000000
--- a/pkg/centos
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit 61bb5f47014707a927bc033a21e8dbf685c5d2c5

diff --git a/pkg/debian b/pkg/debian
deleted file mode 160000 (submodule)
index 824c52f..0000000
--- a/pkg/debian
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit 824c52fafc3afda4c8ad3892596e8243c3b45c91

diff --git a/pkg/fedora b/pkg/fedora
deleted file mode 160000 (submodule)
index 8fe1f03..0000000
--- a/pkg/fedora
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit 8fe1f037d21c9d68d96728843f22e5036d769521

diff --git a/pkg/opensuse b/pkg/opensuse
deleted file mode 160000 (submodule)
index c339941..0000000
--- a/pkg/opensuse
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit c3399411ac70a440ca9080d6df391eb897bba4e9

diff --git a/tools/git-setup.sh b/tools/git-setup.sh
index 8cc1bfdfc2a47a6eb85d30b2a5bab40491dcfc5a..b5903b4ca1005f2de9a90105becc63690143f6b1 100755 (executable)
--- a/tools/git-setup.sh
+++ b/tools/git-setup.sh
@@ -4,12 +4,6 @@ set -eu
 
 cd "${MESON_SOURCE_ROOT:?}"
 
-if [ -e .git ]; then
-    git config submodule.recurse true
-    git config fetch.recurseSubmodules on-demand
-    git config push.recurseSubmodules no
-fi
-
 ret=2
 
 if [ -f .git/hooks/pre-commit.sample ] && [ ! -f .git/hooks/pre-commit ]; then
@@ -19,16 +13,4 @@ if [ -f .git/hooks/pre-commit.sample ] && [ ! -f .git/hooks/pre-commit ]; then
     ret=0
 fi
 
-if [ ! -f .git/hooks/post-rewrite ]; then
-    cp -p tools/git-submodule-update-hook.sh .git/hooks/post-rewrite
-    echo 'Activated post-rewrite hook'
-    ret=0
-fi
-
-if [ ! -f .git/hooks/post-checkout ]; then
-    cp -p tools/git-submodule-update-hook.sh .git/hooks/post-checkout
-    echo 'Activated post-checkout hook'
-    ret=0
-fi
-
 exit $ret

diff --git a/tools/git-submodule-update-hook.sh b/tools/git-submodule-update-hook.sh
deleted file mode 100755 (executable)
index 78feb9d..0000000
--- a/tools/git-submodule-update-hook.sh
+++ /dev/null
@@ -1,4 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: LGPL-2.1-or-later
-
-exec git submodule update