dnsdist: Reduce memory usage with fast-changing dynamic backends

The Lua code in `dnsdist-resolver.lua` periodically check whether
the IP addresses associated with configured backend names have changed,
and if they have it does potentially remove backends whose IP addresses
are no longer present, and adds backends for new IP addresses.
Each backend configured in DNSdist uses a fair amount of memory
because of the states we pre-allocate for UDP queries, and unfortunately
it looks like Lua holds to removed backends for quite a while by default,
causing the removed backends to actually co-exist in memory with the new
ones for a long time, leading to a higher than necessary memory consumption.
This is particularly problematic when you realize that the memory allocator
usually hold onto memory even when the program releases it.
So in environments where IP addresses are updated very often, DNSdist can
consume several times the memory it actually needs, which is not nice.
This commit forces Lua to do a garbage collection cycle after refreshing
servers, thus releasing removed servers much more quickly. This is far
from ideal, but I don't have a better idea yet.

diff --git a/dockerdata/dnsdist-resolver.lua b/dockerdata/dnsdist-resolver.lua
index f9bf8d1c3c04b867de443ef7be80b57081568490..16c82fc9d408da921464fab71b6787b45891c484 100644 (file)
--- a/dockerdata/dnsdist-resolver.lua
+++ b/dockerdata/dnsdist-resolver.lua
@@ -145,6 +145,8 @@ function _M.maintenance()
             end
         end
     end
+    collectgarbage()
+    collectgarbage()
 end
 
 return _M