From: Christian Couder Date: Thu, 19 Jun 2025 13:36:30 +0000 (+0200) Subject: fast-(import|export): improve on commit signature output format X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=0161880fbfa4f80f5cdb2dda9295eeda49bc12be;p=thirdparty%2Fgit.git fast-(import|export): improve on commit signature output format A recent commit, d9cb0e6ff8 (fast-export, fast-import: add support for signed-commits, 2025-03-10), added support for signed commits to fast-export and fast-import. When a signed commit is processed, fast-export can output either "gpgsig sha1" or "gpgsig sha256" depending on whether the signed commit uses the SHA-1 or SHA-256 Git object format. However, this implementation has a number of limitations: - the output format was not properly described in the documentation, - the output format is not very informative as it doesn't even say if the signature is an OpenPGP, an SSH, or an X509 signature, - the implementation doesn't support having both one signature on the SHA-1 object and one on the SHA-256 object. Let's improve on these limitations by improving fast-export and fast-import so that: - both one signature on the SHA-1 object and one on the SHA-256 object can be exported and imported, - if there is more than one signature on the SHA-1 object or on the SHA-256 object, a warning is emitted, - the output format is "gpgsig ", where is the Git object format as before, and is the signature type ("openpgp", "x509", "ssh" or "unknown", - the output is properly documented. Note that it could be even better to be able to export and import more than one signature on the SHA-1 object and on the SHA-256 object, but other parts of Git don't handle that well for now, so this is left for future improvements. Helped-by: brian m. carlson Helped-by: Elijah Newren Signed-off-by: Christian Couder Signed-off-by: Junio C Hamano --- diff --git a/Documentation/git-fast-export.adoc b/Documentation/git-fast-export.adoc index 43bbb4f63c..64198f2186 100644 --- a/Documentation/git-fast-export.adoc +++ b/Documentation/git-fast-export.adoc @@ -50,6 +50,23 @@ resulting tag will have an invalid signature. is the same as how earlier versions of this command without this option behaved. + +When exported, a signature starts with: ++ +gpgsig ++ +where is the Git object hash so either "sha1" or +"sha256", and is the signature type, so "openpgp", +"x509", "ssh" or "unknown". ++ +For example, an OpenPGP signature on a SHA-1 commit starts with +`gpgsig sha1 openpgp`, while an SSH signature on a SHA-256 commit +starts with `gpgsig sha256 ssh`. ++ +Currently for a given commit, at most one signature for the SHA-1 +object and one signature for the SHA-256 object are exported, each +with their respective identifier. A warning is +emitted for each additional signature found. ++ NOTE: This is highly experimental and the format of the data stream may change in the future without compatibility guarantees. diff --git a/Documentation/git-fast-import.adoc b/Documentation/git-fast-import.adoc index 250d866652..db5e5c8da5 100644 --- a/Documentation/git-fast-import.adoc +++ b/Documentation/git-fast-import.adoc @@ -445,7 +445,7 @@ one). original-oid? ('author' (SP )? SP LT GT SP LF)? 'committer' (SP )? SP LT GT SP LF - ('gpgsig' SP LF data)? + ('gpgsig' SP SP LF data)? ('encoding' SP LF)? data ('from' SP LF)? @@ -518,13 +518,32 @@ their syntax. ^^^^^^^^ The optional `gpgsig` command is used to include a PGP/GPG signature -that signs the commit data. +or other cryptographic signature that signs the commit data. -Here specifies which hashing algorithm is used for this -signature, either `sha1` or `sha256`. +.... + 'gpgsig' SP SP LF + data +.... + +The `gpgsig` command takes two arguments: + +* `` specifies which Git object format this signature + applies to, either `sha1` or `sha256`. + +* `` specifies the type of signature, such as + `openpgp`, `x509`, `ssh`, or `unknown`. + +A commit may have at most one signature for the SHA-1 object format +(stored in the "gpgsig" header) and one for the SHA-256 object format +(stored in the "gpgsig-sha256" header). + +See below for a detailed description of the `data` command which +contains the raw signature data. + +Signatures are not yet checked in the current implementation though. -NOTE: This is highly experimental and the format of the data stream may -change in the future without compatibility guarantees. +NOTE: This is highly experimental and the format of the `gpgsig` +command may change in the future without compatibility guarantees. `encoding` ^^^^^^^^^^ diff --git a/builtin/fast-export.c b/builtin/fast-export.c index fcf6b00d5f..332c036ee4 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -29,6 +29,7 @@ #include "quote.h" #include "remote.h" #include "blob.h" +#include "gpg-interface.h" static const char *const fast_export_usage[] = { N_("git fast-export []"), @@ -652,6 +653,30 @@ static const char *find_commit_multiline_header(const char *msg, return strbuf_detach(&val, NULL); } +static void print_signature(const char *signature, const char *object_hash) +{ + if (!signature) + return; + + printf("gpgsig %s %s\ndata %u\n%s", + object_hash, + get_signature_format(signature), + (unsigned)strlen(signature), + signature); +} + +static void warn_on_extra_sig(const char **pos, struct commit *commit, int is_sha1) +{ + const char *header = is_sha1 ? "gpgsig" : "gpgsig-sha256"; + const char *extra_sig = find_commit_multiline_header(*pos + 1, header, pos); + if (extra_sig) { + const char *hash = is_sha1 ? "SHA-1" : "SHA-256"; + warning("more than one %s signature found on commit %s, using only the first one", + hash, oid_to_hex(&commit->object.oid)); + free((char *)extra_sig); + } +} + static void handle_commit(struct commit *commit, struct rev_info *rev, struct string_list *paths_of_changed_objects) { @@ -660,7 +685,8 @@ static void handle_commit(struct commit *commit, struct rev_info *rev, const char *author, *author_end, *committer, *committer_end; const char *encoding = NULL; size_t encoding_len; - const char *signature_alg = NULL, *signature = NULL; + const char *sig_sha1 = NULL; + const char *sig_sha256 = NULL; const char *message; char *reencoded = NULL; struct commit_list *p; @@ -700,10 +726,28 @@ static void handle_commit(struct commit *commit, struct rev_info *rev, } if (*commit_buffer_cursor == '\n') { - if ((signature = find_commit_multiline_header(commit_buffer_cursor + 1, "gpgsig", &commit_buffer_cursor))) - signature_alg = "sha1"; - else if ((signature = find_commit_multiline_header(commit_buffer_cursor + 1, "gpgsig-sha256", &commit_buffer_cursor))) - signature_alg = "sha256"; + const char *sig_cursor = commit_buffer_cursor; + const char *after_sha1 = commit_buffer_cursor; + const char *after_sha256 = commit_buffer_cursor; + + /* + * Find the first signature for each hash algorithm. + * The searches must start from the same position. + */ + sig_sha1 = find_commit_multiline_header(sig_cursor + 1, + "gpgsig", + &after_sha1); + sig_sha256 = find_commit_multiline_header(sig_cursor + 1, + "gpgsig-sha256", + &after_sha256); + + /* Warn on any additional signatures, as they will be ignored. */ + if (sig_sha1) + warn_on_extra_sig(&after_sha1, commit, 1); + if (sig_sha256) + warn_on_extra_sig(&after_sha256, commit, 0); + + commit_buffer_cursor = (after_sha1 > after_sha256) ? after_sha1 : after_sha256; } message = strstr(commit_buffer_cursor, "\n\n"); @@ -769,7 +813,7 @@ static void handle_commit(struct commit *commit, struct rev_info *rev, printf("%.*s\n%.*s\n", (int)(author_end - author), author, (int)(committer_end - committer), committer); - if (signature) { + if (sig_sha1 || sig_sha256) { switch (signed_commit_mode) { case SIGN_ABORT: die("encountered signed commit %s; use " @@ -780,19 +824,18 @@ static void handle_commit(struct commit *commit, struct rev_info *rev, oid_to_hex(&commit->object.oid)); /* fallthru */ case SIGN_VERBATIM: - printf("gpgsig %s\ndata %u\n%s", - signature_alg, - (unsigned)strlen(signature), - signature); + print_signature(sig_sha1, "sha1"); + print_signature(sig_sha256, "sha256"); break; case SIGN_WARN_STRIP: - warning("stripping signature from commit %s", + warning("stripping signature(s) from commit %s", oid_to_hex(&commit->object.oid)); /* fallthru */ case SIGN_STRIP: break; } - free((char *)signature); + free((char *)sig_sha1); + free((char *)sig_sha256); } if (!reencoded && encoding) printf("encoding %.*s\n", (int)encoding_len, encoding); diff --git a/builtin/fast-import.c b/builtin/fast-import.c index b2839c5f43..48ce8ebb77 100644 --- a/builtin/fast-import.c +++ b/builtin/fast-import.c @@ -29,6 +29,7 @@ #include "commit-reach.h" #include "khash.h" #include "date.h" +#include "gpg-interface.h" #define PACK_ID_BITS 16 #define MAX_PACK_ID ((1<hash_algo is freed */ + char *space = strchr(args, ' '); + + if (!space) + die("Expected gpgsig format: 'gpgsig ', " + "got 'gpgsig %s'", args); + *space++ = '\0'; + + sig->hash_algo = args; + sig->sig_format = space; + + /* Remove any trailing newline from format */ + space = strchr(sig->sig_format, '\n'); + if (space) + *space = '\0'; + + /* Validate hash algorithm */ + if (strcmp(sig->hash_algo, "sha1") && + strcmp(sig->hash_algo, "sha256")) + die("Unknown git hash algorithm in gpgsig: '%s'", sig->hash_algo); + + /* Validate signature format */ + if (!valid_signature_format(sig->sig_format)) + die("Invalid signature format in gpgsig: '%s'", sig->sig_format); + if (!strcmp(sig->sig_format, "unknown")) + warning("'unknown' signature format in gpgsig"); + + /* Read signature data */ + read_next_command(); + parse_data(&sig->data, 0, NULL); +} + +static void add_gpgsig_to_commit(struct strbuf *commit_data, + const char *header, + struct signature_data *sig) +{ + struct string_list siglines = STRING_LIST_INIT_NODUP; + + if (!sig->hash_algo) + return; + + strbuf_addstr(commit_data, header); + string_list_split_in_place(&siglines, sig->data.buf, "\n", -1); + strbuf_add_separated_string_list(commit_data, "\n ", &siglines); + strbuf_addch(commit_data, '\n'); + string_list_clear(&siglines, 1); + strbuf_release(&sig->data); + free(sig->hash_algo); +} + +static void store_signature(struct signature_data *stored_sig, + struct signature_data *new_sig, + const char *hash_type) +{ + if (stored_sig->hash_algo) { + warning("Multiple %s signatures found, ignoring additional signature", + hash_type); + strbuf_release(&new_sig->data); + free(new_sig->hash_algo); + } else { + *stored_sig = *new_sig; + } +} + static void parse_new_commit(const char *arg) { - static struct strbuf sig = STRBUF_INIT; static struct strbuf msg = STRBUF_INIT; - struct string_list siglines = STRING_LIST_INIT_NODUP; + struct signature_data sig_sha1 = { NULL, NULL, STRBUF_INIT }; + struct signature_data sig_sha256 = { NULL, NULL, STRBUF_INIT }; struct branch *b; char *author = NULL; char *committer = NULL; - char *sig_alg = NULL; char *encoding = NULL; struct hash_list *merge_list = NULL; unsigned int merge_count; @@ -2750,13 +2822,23 @@ static void parse_new_commit(const char *arg) } if (!committer) die("Expected committer but didn't get one"); - if (skip_prefix(command_buf.buf, "gpgsig ", &v)) { - sig_alg = xstrdup(v); - read_next_command(); - parse_data(&sig, 0, NULL); + + /* Process signatures (up to 2: one "sha1" and one "sha256") */ + while (skip_prefix(command_buf.buf, "gpgsig ", &v)) { + struct signature_data sig = { NULL, NULL, STRBUF_INIT }; + + parse_one_signature(&sig, v); + + if (!strcmp(sig.hash_algo, "sha1")) + store_signature(&sig_sha1, &sig, "SHA-1"); + else if (!strcmp(sig.hash_algo, "sha256")) + store_signature(&sig_sha256, &sig, "SHA-256"); + else + BUG("parse_one_signature() returned unknown hash algo"); + read_next_command(); - } else - strbuf_setlen(&sig, 0); + } + if (skip_prefix(command_buf.buf, "encoding ", &v)) { encoding = xstrdup(v); read_next_command(); @@ -2830,23 +2912,14 @@ static void parse_new_commit(const char *arg) strbuf_addf(&new_data, "encoding %s\n", encoding); - if (sig_alg) { - if (!strcmp(sig_alg, "sha1")) - strbuf_addstr(&new_data, "gpgsig "); - else if (!strcmp(sig_alg, "sha256")) - strbuf_addstr(&new_data, "gpgsig-sha256 "); - else - die("Expected gpgsig algorithm sha1 or sha256, got %s", sig_alg); - string_list_split_in_place(&siglines, sig.buf, "\n", -1); - strbuf_add_separated_string_list(&new_data, "\n ", &siglines); - strbuf_addch(&new_data, '\n'); - } + + add_gpgsig_to_commit(&new_data, "gpgsig ", &sig_sha1); + add_gpgsig_to_commit(&new_data, "gpgsig-sha256 ", &sig_sha256); + strbuf_addch(&new_data, '\n'); strbuf_addbuf(&new_data, &msg); - string_list_clear(&siglines, 1); free(author); free(committer); - free(sig_alg); free(encoding); if (!store_object(OBJ_COMMIT, &new_data, NULL, &b->oid, next_mark)) diff --git a/gpg-interface.c b/gpg-interface.c index 0896458de5..6f2d87475f 100644 --- a/gpg-interface.c +++ b/gpg-interface.c @@ -144,6 +144,18 @@ static struct gpg_format *get_format_by_sig(const char *sig) return NULL; } +const char *get_signature_format(const char *buf) +{ + struct gpg_format *format = get_format_by_sig(buf); + return format ? format->name : "unknown"; +} + +int valid_signature_format(const char *format) +{ + return (!!get_format_by_name(format) || + !strcmp(format, "unknown")); +} + void signature_check_clear(struct signature_check *sigc) { FREE_AND_NULL(sigc->payload); diff --git a/gpg-interface.h b/gpg-interface.h index e09f12e8d0..60ddf8bbfa 100644 --- a/gpg-interface.h +++ b/gpg-interface.h @@ -47,6 +47,18 @@ struct signature_check { void signature_check_clear(struct signature_check *sigc); +/* + * Return the format of the signature (like "openpgp", "x509", "ssh" + * or "unknown"). + */ +const char *get_signature_format(const char *buf); + +/* + * Is the signature format valid (like "openpgp", "x509", "ssh" or + * "unknown") + */ +int valid_signature_format(const char *format); + /* * Look at a GPG signed tag object. If such a signature exists, store it in * signature and the signed content in payload. Return 1 if a signature was diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 76619765fc..3559a33c72 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -314,7 +314,7 @@ test_expect_success GPG 'signed-commits=abort' ' test_expect_success GPG 'signed-commits=verbatim' ' git fast-export --signed-commits=verbatim --reencode=no commit-signing >output && - grep "^gpgsig sha" output && + test_grep -E "^gpgsig sha(1|256) openpgp" output && grep "encoding ISO-8859-1" output && ( cd new && @@ -328,7 +328,7 @@ test_expect_success GPG 'signed-commits=verbatim' ' test_expect_success GPG 'signed-commits=warn-verbatim' ' git fast-export --signed-commits=warn-verbatim --reencode=no commit-signing >output 2>err && - grep "^gpgsig sha" output && + test_grep -E "^gpgsig sha(1|256) openpgp" output && grep "encoding ISO-8859-1" output && test -s err && ( @@ -369,6 +369,62 @@ test_expect_success GPG 'signed-commits=warn-strip' ' ' +test_expect_success GPGSM 'setup X.509 signed commit' ' + + git checkout -b x509-signing main && + test_config gpg.format x509 && + test_config user.signingkey $GIT_COMMITTER_EMAIL && + echo "X.509 content" >file && + git add file && + git commit -S -m "X.509 signed commit" && + X509_COMMIT=$(git rev-parse HEAD) && + git checkout main + +' + +test_expect_success GPGSM 'round-trip X.509 signed commit' ' + + git fast-export --signed-commits=verbatim x509-signing >output && + test_grep -E "^gpgsig sha(1|256) x509" output && + ( + cd new && + git fast-import && + git cat-file commit refs/heads/x509-signing >actual && + grep "^gpgsig" actual && + IMPORTED=$(git rev-parse refs/heads/x509-signing) && + test $X509_COMMIT = $IMPORTED + ) file && + git add file && + git commit -S -m "SSH signed commit" && + SSH_COMMIT=$(git rev-parse HEAD) && + git checkout main + +' + +test_expect_success GPGSSH 'round-trip SSH signed commit' ' + + git fast-export --signed-commits=verbatim ssh-signing >output && + test_grep -E "^gpgsig sha(1|256) ssh" output && + ( + cd new && + git fast-import && + git cat-file commit refs/heads/ssh-signing >actual && + grep "^gpgsig" actual && + IMPORTED=$(git rev-parse refs/heads/ssh-signing) && + test $SSH_COMMIT = $IMPORTED + )