rework copy_git_files() to use a git clone

Using a git clone allows to solve some problems:
    - It allows to copy git submodules without having to fallback to
      --use-git-files=no since git-ls-files doesn't recurse submodules
    - It allows build systems that rely on git describe or similar to
      tag the build: this is more important when projects use
      git modules and they want to tag the git revision of each
      submodule.
    - It uses less disk space when using output as directory since
      when passing a local directory git will use hard links

Some additional handling for files that were modified (or added with
git -A) was put in place to allow tests to be performed before
committing.

In addition the call to git ls-files was fixed in the following cases:
    - When mkosi is called with -C option: it was calling ls-files inside
      the caller directory, not inside the src dir.
    - When mkosi is called with --git-files=others and .gitignore
      doesn't contain an entry for '.mkosi-*': it would try to copy the
      temporary directory we created and fail

diff --git a/mkosi b/mkosi
index 76f1f1ff966992d417174ca28fe527cd22e6cc11..cb115046c401e01126f81373012cfafe68a36e0b 100755 (executable)
--- a/mkosi
+++ b/mkosi
@@ -1289,15 +1289,28 @@ def install_extra_trees(args, workspace, for_cache):
             enumerate_and_copy(d, os.path.join(workspace, "root"))
 
 def copy_git_files(src, dest, *, git_files):
-    what_files = ['--exclude-standard', '--cached']
+    subprocess.run(['git', 'clone', '--depth=1', '--recursive', '--shallow-submodules', src, dest],
+                   check=True)
+
+    what_files = ['--exclude-standard', '--modified']
     if git_files == 'others':
-        what_files += ['--others']
-    c = subprocess.run(['git', 'ls-files', '-z'] + what_files,
+        what_files += ['--others', '--exclude=.mkosi-*']
+
+    # everything that's modified from the tree
+    c = subprocess.run(['git', '-C', src, 'ls-files', '-z'] + what_files,
                        stdout=subprocess.PIPE,
                        universal_newlines=False,
                        check=True)
     files = {x.decode("utf-8") for x in c.stdout.rstrip(b'\0').split(b'\0')}
 
+    # everything that's modified and about to be committed
+    c = subprocess.run(['git', '-C', src, 'diff', '--cached', '--name-only', '-z'],
+                       stdout=subprocess.PIPE,
+                       universal_newlines=False,
+                       check=True)
+    files |= {x.decode("utf-8") for x in c.stdout.rstrip(b'\0').split(b'\0')}
+    files.discard('')
+
     del c
 
     for path in files: