line-log: free diff queue when processing non-merge commits

When processing a non-merge commit, the line-level log first asks the
tree-diff machinery whether any of the files in the given line ranges
were modified between the current commit and its parent, and if some
of them were, then it loads the contents of those files from both
commits to see whether their line ranges were modified and/or need to
be adjusted.  Alas, it doesn't free() the diff queue holding the
results of that query and the contents of those files once its done.
This can add up to a substantial amount of leaked memory, especially
when the file in question is big and is frequently modified: a user
reported "Out of memory, malloc failed" errors with a 2MB text file
that was modified ~2800 times [1] (I estimate the leak would use up
almost 11GB memory in that case).

Free that diff queue to plug this memory leak.  However, instead of
simply open-coding the necessary three lines, add them as a helper
function to the diff API, because it will be useful elsewhere as well.

[1] https://public-inbox.org/git/CAFOPqVXz2XwzX8vGU7wLuqb2ZuwTuOFAzBLRM_QPk+NJa=eC-g@mail.gmail.com/

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>

diff --git a/diff.c b/diff.c
index 35e46dd9684cd3462899f70ea57bd24946051723..ef94175163d19b467856430e266bca9db42dfcdc 100644 (file)
--- a/diff.c
+++ b/diff.c
@@ -5773,6 +5773,13 @@ void diff_free_filepair(struct diff_filepair *p)
        free(p);
 }
 
+void diff_free_queue(struct diff_queue_struct *q)
+{
+       for (int i = 0; i < q->nr; i++)
+               diff_free_filepair(q->queue[i]);
+       free(q->queue);
+}
+
 const char *diff_aligned_abbrev(const struct object_id *oid, int len)
 {
        int abblen;

diff --git a/diffcore.h b/diffcore.h
index badc2261c201831a620fcc7c29edcc1fd3bdbf1e..9b588a1ee15a3bc93eaf4464f7d1e74265b2f3a2 100644 (file)
--- a/diffcore.h
+++ b/diffcore.h
@@ -162,6 +162,7 @@ struct diff_filepair *diff_queue(struct diff_queue_struct *,
                                 struct diff_filespec *,
                                 struct diff_filespec *);
 void diff_q(struct diff_queue_struct *, struct diff_filepair *);
+void diff_free_queue(struct diff_queue_struct *q);
 
 /* dir_rename_relevance: the reason we want rename information for a dir */
 enum dir_rename_relevance {

diff --git a/line-log.c b/line-log.c
index 51d93310a4dee18ccfb559803583500ec45b27b4..7a74daf2e83e7e77cbcf9d46fc77fc69e41ac204 100644 (file)
--- a/line-log.c
+++ b/line-log.c
@@ -1195,6 +1195,7 @@ static int process_ranges_ordinary_commit(struct rev_info *rev, struct commit *c
        if (parent)
                add_line_range(rev, parent, parent_range);
        free_line_log_data(parent_range);
+       diff_free_queue(&queue);
        return changed;
 }