Completely replace the "fdcache" algorithms in debuginfod, which
manages files extracted from archives. Previous logic was a LRU queue
for files requested by users, and a separate LRU queue for prefetched
files found nearby the ones users requested. The code did not handle
annoying edge cases like infrequently accessed but very costly
extraction of files like fedora kernels' vdso.debug. In addition, the
queue was searched linearly for normal lookups. It was also
unceremoniously dropped at each groom cycle.
New code replaces this with an indexed datastructure for quick
lookups, and extra metadata for use during eviction decisions. Each
entry tracks size and such, but now also tracks how recently and how
many times it was requested, how long it took to originally extract.
The new code combines these quantities in a score, by which eviction
eligibility is ranked. Intuitively, the effect is to prefer to hoard
small / slow-to-access files, and prefer to jettison large / fast /
never accessed ones.
It's a tricky thing to balance. The parameters in this configuration
were tested by timing-accurate replaying a few days' worth of actual
traffic of the main fedora debuginfod server. The peer
debuginfod.stg.fedoraproject.org runs the new code. It shows good
performance, excellent use of the cache storage, and strong preference
to hold onto those vdso.debug files. But who knows, it might need
tweaking later. The new code adds more prometheus metrics to make it
possible to grok the effectiveness of the few remaining
fdcache-related options.
Patch includes doc updates and NEWS. The changes are invisible to the
testsuite (except with respect to the new metrics). Code changes are
focused on all the member functions of class libarchive_fdcache, and
their callers. Unused parameters are removed, with previous command
line options hidden/accepted/ignored. Some other minor error-path
tempfile-gc was fixed in the extraction paths.
projects / thirdparty / elfutils.git / commit
