From: Petr Špaček Date: Fri, 17 Aug 2018 09:17:51 +0000 (+0200) Subject: cache.clear: log when asynchonous clear is finished, document interface X-Git-Tag: v3.0.0~1^2~5 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=8bb5d0c125ea618985cf242ae3e46c52733f39d4;p=thirdparty%2Fknot-resolver.git cache.clear: log when asynchonous clear is finished, document interface --- diff --git a/daemon/README.rst b/daemon/README.rst index 46a8a0785..5e8381d2a 100644 --- a/daemon/README.rst +++ b/daemon/README.rst @@ -864,56 +864,62 @@ daemons or manipulated from other processes, making for example synchronised loa [AAAA] => true } -.. function:: cache.clear([name], [exact_name], [rr_type], [chunk_size], [callback]) +.. function:: cache.clear([name], [exact_name], [rr_type], [chunk_size], [callback], [prev_state]) Purge cache records matching specified criteria. There are two specifics: * To reliably remove **negative** cache entries you need to clear subtree with the whole zone. E.g. to clear negative cache entries for (formerly non-existing) record `www.example.com. A` you need to flush whole subtree starting at zone apex, e.g. `example.com.` [#]_. - * This operation is an asynchonous and might not be yet finished when call to ``cache.clear()`` function returns. Result is indicated return value. You can use custom callback to wait for operation to finish. + * This operation is an asynchonous and might not be yet finished when call to ``cache.clear()`` function returns. Return value indicates if clearing continues asynchronously or not. - :rtype: table - :return: ``count`` field is always present, other fields are optional. - - =========== =========== - Key Description - =========== =========== - count number of items removed from cache by this call - not_apex indicates that cleared subtree is not cached as zone apex; proofs of non-existence were not removed - subtree hint where zone apex lies (this is guess from cache content, might not be accurate) - chunk_limit indicates that more than ``chunk_size`` needs to be cleared, clearing will continue in callback - =========== =========== - - :param string name: if the name isn't provided, whole cache is purged + :param string name: subtree to purge; if the name isn't provided, whole cache is purged (and any other parameters are disregarded). - Otherwise only records in that subtree are removed. :param bool exact_name: if set to ``true``, only records with *the same* name are removed; default: false. :param kres.type rr_type: you may additionally specify the type to remove, - but that is only supported with ``exact_name == true``; default: nil; - :param integer chunk_size: the number of records to remove at one go; default: 100. + but that is only supported with ``exact_name == true``; default: nil. + :param integer chunk_size: the number of records to remove in one round; default: 100. The purpose is not to block the resolver for long. The default ``callback`` repeats the command after one millisecond until all matching data are cleared. - :param function callback: custom code to handle result of the underlying C call. - As the first parameter it gets the return code from :func:`kr_cache_remove_subtree()`, - and the following parameters are copies of those passed to `cache.clear()`. + :param function callback: a custom code to handle result of the underlying C call. + Its parameters are copies of those passed to `cache.clear()` with one additional + parameter ``rettable`` containing table with return value from current call. + ``count`` field contains a return code from :func:`kr_cache_remove_subtree()`. + :param table prev_state: return value from previous run (can be used by callback) + + :rtype: table + :return: ``count`` key is always present. Other keys are optional and their presense indicate special conditions. + + * **count** *(integer)* - number of items removed from cache by this call (can be 0 if no entry matched criteria) + * **not_apex** - cleared subtree is not cached as zone apex; proofs of non-existence were not removed + * **subtree** *(string)* - hint where zone apex lies (this is estimation from cache content and might not be accurate) + * **chunk_limit** - more than ``chunk_size`` items needs to be cleared, clearing will continue asynchonously + Examples: .. code-block:: lua -- Clear whole cache - cache.clear() - -- Clear records at and below 'bad.cz' - cache.clear('bad.cz') - - .. attention:: - - To minimize surprises with partial cache removal, - you may prefer to specify names that have NS/SOA records, - e.g. ``example.com``. Details: validated NSEC and NSEC3 records - (which are used for aggressive non-existence proofs) - will be removed only for zones whose **apex** is at or below the specified name. + > cache.clear() + [count] => 76 + + -- Clear records at and below 'com.' + > cache.clear('com.') + [chunk_limit] => chunk size limit reached; the default callback will continue asynchronously + [not_apex] => to clear proofs of non-existence call cache.clear('com.') + [count] => 100 + [round] => 1 + [subtree] => com. + > worker.sleep(0.1) + [cache] asynchonous cache.clear('com', false) finished + + -- Clear only 'www.example.com.' + > cache.clear('www.example.com.', true) + [round] => 1 + [count] => 1 + [not_apex] => to clear proofs of non-existence call cache.clear('example.com.') + [subtree] => example.com. .. [#] This is a consequence of DNSSEC negative cache which relies on proofs of non-existence on various owner nodes. It is impossible to efficiently flush part of DNS zones signed with NSEC3. diff --git a/daemon/cache.test/clear.test.lua b/daemon/cache.test/clear.test.lua index c790539c7..ff219d619 100644 --- a/daemon/cache.test/clear.test.lua +++ b/daemon/cache.test/clear.test.lua @@ -124,17 +124,19 @@ local function test_callback() local test_exactname = true local test_rrtype = nil local test_maxcount = 1 - local function check_callback(errors, name, exact_name, rr_type, chunk_size, callback) - is(type(errors), 'table', 'callback received table of errors') + local test_prev_state = { works = true } + local function check_callback(name, exact_name, rr_type, chunk_size, callback, prev_state, errors) is(errors.count, 1, 'callback received correct # of removed records') is(test_name, name, 'callback received subtree name') is(test_exactname, exact_name, 'callback received exact_name') is(test_rrtype, rrtype, 'callback received rr_type') is(test_chunksize, chunksize, 'callback received maxcount') is(check_callback, callback, 'callback received reference to itself') + is(type(errors), 'table', 'callback received table of errors') + same(test_prev_state, prev_state, 'callback received previous state') return 666 end - same(cache.clear(test_name, test_exactname, test_rrtype, test_maxcount, check_callback), + same(cache.clear(test_name, test_exactname, test_rrtype, test_maxcount, check_callback, test_prev_state), 666, 'first callback return value is passed to cache.clear() caller') local cnt_before_wait = cache.count() worker.sleep(0.2) @@ -183,6 +185,7 @@ return { test_exact_match_qtype, test_exact_match_qname, test_callback, + import_zone, test_subtree, test_subtree_limit, test_apex, diff --git a/daemon/lua/sandbox.lua b/daemon/lua/sandbox.lua index 13914718c..e7990aafa 100644 --- a/daemon/lua/sandbox.lua +++ b/daemon/lua/sandbox.lua @@ -158,7 +158,7 @@ setmetatable(modules, { }) -cache.clear = function (name, exact_name, rr_type, chunk_size, callback) +cache.clear = function (name, exact_name, rr_type, chunk_size, callback, prev_state) if name == nil then -- keep same output format as for 'standard' clear local total_count = cache.count() if not cache.clear_everything() then @@ -174,7 +174,7 @@ cache.clear = function (name, exact_name, rr_type, chunk_size, callback) then error('cache.clear(): incorrect exact_name passed') end local cach = kres.context().cache; - local errors = {} + local rettable = {} -- Apex warning. If the caller passes a custom callback, -- we assume they are advanced enough not to need the check. -- The point is to avoid repeating the check in each callback iteration. @@ -186,9 +186,9 @@ cache.clear = function (name, exact_name, rr_type, chunk_size, callback) ffi.gc(names[0], ffi.C.free) local apex = kres.dname2str(names[0]) if apex ~= name then - errors.not_apex = 'to clear proofs of non-existence call ' + rettable.not_apex = 'to clear proofs of non-existence call ' .. 'cache.clear(\'' .. tostring(apex) ..'\')' - errors.subtree = apex + rettable.subtree = apex end end @@ -208,31 +208,35 @@ cache.clear = function (name, exact_name, rr_type, chunk_size, callback) then error('cache.clear(): chunk_size has to be a positive integer') end -- Do the C call, and add chunk_size warning. - errors.count = ffi.C.kr_cache_remove_subtree(cach, dname, exact_name, chunk_size) - if errors.count == chunk_size then + rettable.count = ffi.C.kr_cache_remove_subtree(cach, dname, exact_name, chunk_size) + if rettable.count == chunk_size then local msg_extra = '' if callback == nil then msg_extra = '; the default callback will continue asynchronously' end - errors.chunk_limit = 'chunk size limit of ' .. tostring(chunk_size) - .. ' entries reached' .. msg_extra + rettable.chunk_limit = 'chunk size limit reached' .. msg_extra end -- Default callback function: repeat after 1ms if callback == nil then callback = - function (cberrors, cbname, cbexact_name, cbrr_type, cbchunk_size, cbself) - if errors.count < 0 then error(ffi.string(ffi.C.knot_strerror(errors.count))) end - if (errors.count == cbchunk_size) then + function (cbname, cbexact_name, cbrr_type, cbchunk_size, cbself, cbprev_state, cbrettable) + if cbrettable.count < 0 then error(ffi.string(ffi.C.knot_strerror(cbrettable.count))) end + if cbprev_state == nil then cbprev_state = { round = 0 } end + if type(cbprev_state) ~= 'table' + then error('cache.clear() callback: incorrect prev_state passed') end + cbrettable.round = cbprev_state.round + 1 + if (cbrettable.count == cbchunk_size) then event.after(1, function () - cache.clear(cbname, cbexact_name, cbrr_type, cbchunk_size, cbself) + cache.clear(cbname, cbexact_name, cbrr_type, cbchunk_size, cbself, cbrettable) end) - else - log('[cache] asynchonous clear finished: ' .. table_print(cberrors)) + elseif cbrettable.round > 1 then + log('[cache] asynchonous cache.clear(\'' .. cbname .. '\', ' + .. tostring(cbexact_name) .. ') finished') end - return cberrors + return cbrettable end end - return callback(errors, name, exact_name, rr_type, chunk_size, callback) + return callback(name, exact_name, rr_type, chunk_size, callback, prev_state, rettable) end -- Syntactic sugar for cache -- `cache[x] -> cache.get(x)`