Naoki Kambe [Mon, 6 Aug 2012 05:30:52 +0000 (14:30 +0900)]
[2179] Supported a identifier in isc.cc.data e.g. "xxx/yyy/zzz[i]"
If a target module returns data whose key name is such a identifier type, the
stats module can identify statistics data. The stats module checks the
validation of data even if such a identifier is specified. But when the stats
shows statistics data e.g. via bindctl, this type of identifier is not shown
for now.
Naoki Kambe [Sat, 4 Aug 2012 22:53:29 +0000 (07:53 +0900)]
[2179] Supported differential statistics updates of Stats
Stats updates corresponding statistics data which each module returns. For
that, the method "update_statistics_data()" was updated to merge recursively
old value and new value each other. As for this merging, a new method
"merge_oldnew" was introduced into stats.py. This method is intended to be
internally used so far.
Naoki Kambe [Wed, 8 Aug 2012 12:39:21 +0000 (21:39 +0900)]
[2136] moved invoking update_modules() to outside of the method
When the set command was used for collecting statistics data, the action was
randomly. But in the case that periodical polling is used for collecting
statistics data, update_modules doesn't need to be invoked so many times every
time update_statistics_data is invoked. Invoking update_module() causes an IO
wait for receiving statistics spec from Cfgmgr.
Naoki Kambe [Wed, 8 Aug 2012 12:29:24 +0000 (21:29 +0900)]
[2136] corrected the improper test
Not self.stats.mccs.specification but self.stats.modules['Stats'] should be set
the invalid spec_module into. And it should check whether StatsError is raised
by setting the owner to the self module name in the argument of command_show.
Naoki Kambe [Wed, 8 Aug 2012 12:22:45 +0000 (21:22 +0900)]
[2136] removed the do_polling method from the __init__ method of stats.Stats()
Because do_polling() is done when the show command is invoked, the do_polling
method didn't need to be done when the stats is starting. Related to this
change, the b10-stats_test.py was changed. do_polling() is inserted before the
assertion of statistics_data.
JINMEI Tatuya [Thu, 9 Aug 2012 00:56:16 +0000 (17:56 -0700)]
[2100] (not directly related) cleanup: always destroy old data in setData().
actually the previous implementation seems buggy; it returns the given pionter,
not the old data. Worse, this code path isn't tested. It's a typical example
of YAGNI; if we don't see the need for it, don't do it; if you cannot regist
the temptation of making it smarter, at least you should write tests.
At least ZoneTable doesn't need it, so I chose to clean it up. If and when
we find it necessary, we should implementation from the scratch, and
test-driven.
JINMEI Tatuya [Thu, 9 Aug 2012 00:38:25 +0000 (17:38 -0700)]
[2100] cleanup: removed removeZone().
right now it doesn't work, and when we need to implement it, the internal
will need to be substantially updated anyway, so it doesn't make much sense
to keep the nonworkable version.
JINMEI Tatuya [Thu, 9 Aug 2012 00:15:50 +0000 (17:15 -0700)]
[2100] updated the return type of findZone().
It now simply returns ZoneData as the data associated with the code;
the corresponding client implementation will create a zone finder from it.
also updated some doc.
The foreach functions to manually iterate over the data are used in
tests only, so moved to the tests.
The specifications are used by the tests and encoding and will be used
by the decoding machinery too, probably, so we are making them
accessible and reside in a separate file.
Small trick with just-in-time initialization of a pointer in benchmark
is used instead of copying it and relying on the fact that it can be
copied while it was not yet used.
This is logically part of the next commit. But then, the git diff would
show the old file disappear and the new file appear with new content,
which would be hard to review, so it is split. This commit should
contain no changes in the content (hopefully), so can be skipped during
review.
Naoki Kambe [Wed, 1 Aug 2012 07:19:38 +0000 (16:19 +0900)]
[2136] Updated the "show_processes" to provide address of each module
This information is used for other module e.g. stats to count running instances
of same module. The address information is seen in the third index of the
array.
Naoki Kambe [Wed, 1 Aug 2012 06:47:21 +0000 (15:47 +0900)]
[2136] Updated the 'show' command handler
The stats module decides if polling should be done by the the last time of
polling. If more than one seconds past since the last request to each module,
the stats module requests each module statistics data and then shows the latest
result. Otherwise, the stats module just shows statistics data which it has.
Naoki Kambe [Wed, 1 Aug 2012 06:37:23 +0000 (15:37 +0900)]
[2136] Separated the updating process from the receiving process
In the updating process, it communicates with the cfgmgr module for updating
statistics specification of each module. This fix is for separating such mixed
communication process.
[2136] Revised the test scripts and the config db file
- Fixed the incorrect previous fix.
There should not be the inconsistency on number of mock Auth instances on
both test scripts: b10-stats_test.py and b10-stats-httpd_test.py. In the
previous change, there was the inconsistency in MockBoss class. Actually in
b10-stats-httpd_test.py, the number of auth instances which Cfgmgr answers
was four, but the number of actually invoked auth instances was one.
- Reduce the number of invoked mock Auth instances
In theses test scripts, multiple Auth instances need to be invoked. But if
four instances are actually invoked in these scripts, open files(ulimit -n)
exceeds 256. In the environment where ulimit -n was set to 256, the
b10-stats-httpd_test.py test failed actually. In that case,
b10-stats-httpd_test.py needs open files more than 256. So the number of the
auth instances in mock modules was reduced into 2 in the scripts. This fix is
for the comment at #2136.
The extra MockAuth objects didn't shut down in
BaseModules.shutdown(). But these weren't needed for unittest of stats
httpd. So setting up and shutting up of them were moved to
b10-stats_test.py from test_utils.py.
Changes in b10-stats_test.py:
- added deeply checking statistics values of each of module and
instance
- added stats_server.shutdown() at the end of the test_config method.
Because the running stats_server was never shut down even after
stats_server.run() was invoked.
Changes in test_utils.py:
- Changed the way of invoking multiple auth instances just for efficiency
[2136] renamed 'pid' to 'mid' and change the default value -1 to None on
the second argument of the the 'update_statistics_data' method
Instead of PID(Process Id) used in previous version of stats codes,
'mid' (module id) is used for identifying multiple instances of same
module. 'lname' of CC-session in stats codes is assigned to it.
[2136] removed 'set' command and introduced a 'poll-interval' config item
The description about 'set' command was removed from the manpages, and
A 'poll-interval' config item was introduced into the manpages. Also
the implement, the testcases and specification related to the set
command were removed from source codes and the specfile.
Statistics data transfer method is changed
from peliodic sending via statistics channel
to passive retrieval via config session
- Added "getstats" command to retrieve statistics data via config session.
= type "Auth getstats" command using bindctl shows statistics data.
- Removed StatisticsIntervalTimer
- Removed "sendstats" command
- Changed that each config command can return values.
- returns Element::createlist(), Element::fromJSON("{}") or Statistics
Opinion:
- Each command should return conditions and values using createAnswer()
rather than using throw/catch.
JINMEI Tatuya [Wed, 8 Aug 2012 06:59:44 +0000 (23:59 -0700)]
[2100] simplify: move definitions to the main class instead of pimpl.
this is mostly private class and is not expected to be referenced outside
of this psuedo module ("memory"), so it's raltively safer.
also, the ZoneTable structure will be embedable in a shared memory space,
so introducing the another pointer indirection will make it unnecessarily
complicated.
The previous version was slightly misleading, the reader could
understand it the queries would be just slow as answered from the DB,
instead of not answered at all.