From 56c2654c040062cd6b36de89c75bc9401c8d98a6 Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Fri, 7 Aug 2009 04:55:15 +0000 Subject: [PATCH] Overhauled the how-to-write-a-new-tool chapter. git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10734 --- docs/xml/manual-writing-tools.xml | 356 ++++++++++-------------------- 1 file changed, 119 insertions(+), 237 deletions(-) diff --git a/docs/xml/manual-writing-tools.xml b/docs/xml/manual-writing-tools.xml index 046f4b2e7b..5ee037da14 100644 --- a/docs/xml/manual-writing-tools.xml +++ b/docs/xml/manual-writing-tools.xml @@ -14,14 +14,15 @@ help. Introduction The key idea behind Valgrind's architecture is the division -between its "core" and "tool plug-ins". +between its core and tools. The core provides the common low-level infrastructure to support program instrumentation, including the JIT compiler, low-level memory manager, signal handling and a -scheduler (for pthreads). It also provides certain services that +thread scheduler. It also provides certain services that are useful to some but not all tools, such as support for error -recording and suppression. +recording, and support for replacing heap allocation functions such as +malloc. But the core leaves certain operations undefined, which must be filled by tools. Most notably, tools define how program @@ -34,13 +35,13 @@ occur. But the core takes care of all the hard work. - -Writing a Tool + +Basics How tools work -Tool plug-ins must define various functions for instrumenting programs +Tools must define various functions for instrumenting programs that are called by Valgrind's core. They are then linked against Valgrind's core to define a complete Valgrind tool which will be used when the option is used to select it. @@ -87,19 +88,20 @@ top-level directory valgrind/. - Create empty files - foobar/docs/Makefile.am and - foobar/tests/Makefile.am. + Create an empty file foobar/tests/Makefile.am. Copy none/Makefile.am into foobar/. Edit it by replacing all - occurrences of the string "none" with - "foobar", and all occurrences of - the string "nl_" with - "fb_". + occurrences of the strings + "none", + "nl_" and + "nl-" with + "foobar", + "fb_" and + "fb-" respectively. @@ -107,11 +109,11 @@ top-level directory valgrind/. foobar/, renaming it as fb_main.c. Edit it by changing the details lines in - nl_pre_clo_init() to something appropriate for the + nl_pre_clo_init to something appropriate for the tool. These fields are used in the startup message, except for bug_reports_to which is used if a - tool assertion fails. Also replace the string - "nl_" with + tool assertion fails. Also, replace the string + "nl_" throughout with "fb_" again. @@ -119,13 +121,12 @@ top-level directory valgrind/. Edit Makefile.am, adding the new directory foobar to the TOOLS or - EXP_TOOLSvariables. + EXP_TOOLS variables. Edit configure.in, adding - foobar/Makefile, - foobar/docs/Makefile and + foobar/Makefile and foobar/tests/Makefile to the AC_OUTPUT list. @@ -166,7 +167,7 @@ top-level directory valgrind/. -These steps don't have to be followed exactly - you can choose +These steps don't have to be followed exactly -- you can choose different names for your source files, and use a different for ./configure. @@ -190,8 +191,7 @@ onto the interesting stuff... The names can be different to the above, but these are the usual names. The first one is registered using the macro -VG_DETERMINE_INTERFACE_VERSION (which also -checks that the core/tool interface of the tool matches that of the core). +VG_DETERMINE_INTERFACE_VERSION. The last three are registered using the VG_(basic_tool_funcs) function. @@ -207,25 +207,25 @@ core about them. Initialisation Most of the initialisation should be done in -pre_clo_init(). Only use -post_clo_init() if a tool provides command line +pre_clo_init. Only use +post_clo_init if a tool provides command line options and must do some initialisation after option processing takes place ("clo" stands for "command line options"). First of all, various "details" need to be set for a tool, using -the functions VG_(details_*)(). Some are all +the functions VG_(details_*). Some are all compulsory, some aren't. Some are used when constructing the startup message, detail_bug_reports_to is used -if VG_(tool_panic)() is ever called, or +if VG_(tool_panic) is ever called, or a tool assertion fails. Others have other uses. Second, various "needs" can be set for a tool, using the functions -VG_(needs_*)(). They are mostly booleans, and can +VG_(needs_*). They are mostly booleans, and can be left untouched (they default to False). They determine whether a tool can do various things such as: record, report and suppress errors; process command line options; wrap system calls; -record extra information about heap blocks, etc. +record extra information about heap blocks; etc. For example, if a tool wants the core's help in recording and reporting errors, it must call @@ -233,13 +233,11 @@ reporting errors, it must call eight functions for comparing errors, printing out errors, reading suppressions from a suppressions file, etc. While writing these functions requires some work, it's much less than doing error handling -from scratch because the core is doing most of the work. See the -function VG_(needs_tool_errors) in -include/pub_tool_tooliface.h for full details of -all the needs. +from scratch because the core is doing most of the work. + Third, the tool can indicate which events in core it wants to be -notified about, using the functions VG_(track_*)(). +notified about, using the functions VG_(track_*). These include things such as heap blocks being allocated, the stack pointer changing, a mutex being locked, etc. If a tool wants to know about this, it should provide a pointer to a function, which will be @@ -247,7 +245,7 @@ called when that event happens. For example, if the tool want to be notified when a new heap block is allocated, it should call -VG_(track_new_mem_heap)() with an appropriate +VG_(track_new_mem_heap) with an appropriate function pointer, and the assigned function will be called each time this happens. @@ -262,9 +260,9 @@ can be found in Instrumentation -instrument() is the interesting one. It +instrument is the interesting one. It allows you to instrument VEX IR, which is -Valgrind's RISC-like intermediate language. VEX IR is described fairly well +Valgrind's RISC-like intermediate language. VEX IR is described in the comments of the header file VEX/pub/libvex_ir.h. @@ -308,21 +306,21 @@ are deep reasons for this, trust us). Valgrind provides an implementation of a reasonable subset of the C library, details of which are in pub_tool_libc*.h. -When writing a tool, you shouldn't need to look at any of the code in -Valgrind's core. Although it might be useful sometimes to help understand -something. +When writing a tool, in theory you shouldn't need to look at any of +the code in Valgrind's core, but in practice it might be useful sometimes to +help understand something. The include/pub_tool_basics.h and -VEX/pub/libvex_basictypes.h files file have some basic +VEX/pub/libvex_basictypes.h files have some basic types that are widely used. Ultimately, the tools distributed (Memcheck, Cachegrind, Lackey, etc.) are probably the best documentation of all, for the moment. -Note that the VG_ macro is used +The VG_ macro is used heavily. This just prepends a longer string in front of names to avoid potential namespace clashes. It is defined in -include/pub_tool_basics_asm.h. +include/pub_tool_basics.h. There are some assorted notes about various aspects of the implementation in docs/internals/. Much of it @@ -331,15 +329,21 @@ isn't that relevant to tool-writers, however. - -Words of Advice + -Writing and debugging tools is not trivial. Here are some -suggestions for solving common problems. - -Segmentation Faults + +Advanced Topics + +Once a tool becomes more complicated, there are some extra +things you may want/need to do. + + +Debugging Tips + +Writing and debugging tools is not trivial. Here are some +suggestions for solving common problems. If you are getting segmentation faults in C functions used by your tool, the usual GDB command: @@ -348,75 +352,15 @@ tool, the usual GDB command: gdb core]]> usually gives the location of the segmentation fault. - - - - -Debugging C functions - -If you want to debug C functions used by your tool, you can -achieve this by following these steps: - - - Set VALGRIND_LAUNCHER to - /bin/valgrind]]>: - - export VALGRIND_LAUNCHER=/usr/local/bin/valgrind - - - - Then run /lib/valgrind//:]]> - - gdb /usr/local/lib/valgrind/ppc32-linux/lackey - - - - Do handle SIGSEGV SIGILL nostop - noprint in GDB to prevent GDB from stopping on a - SIGSEGV or SIGILL: - - (gdb) handle SIGILL SIGSEGV nostop noprint - - - - Set any breakpoints you want and proceed as normal for GDB: - - (gdb) b vgPlain_do_exec - The macro VG_(FUNC) is expanded to vgPlain_FUNC, so If you - want to set a breakpoint VG_(do_exec), you could do like this in - GDB. - - - - Run the tool with required options: - - (gdb) run `pwd` - - - - -GDB may be able to give you useful information. Note that by -default most of the system is built with -, and you'll need to get rid of -this to extract useful tracebacks from GDB. - - - - - -IR Instrumentation Problems +If you want to debug C functions used by your tool, there are +instructions on how to do so in the file +README_DEVELOPERS. If you are having problems with your VEX IR instrumentation, it's likely that GDB won't be able to help at all. In this case, Valgrind's option is invaluable for observing the results of instrumentation. - - - - -Miscellaneous - If you just want to know whether a program point has been reached, using the OINK macro (in include/pub_tool_libcprint.h) can be easier than @@ -426,26 +370,14 @@ using GDB. valgrind --help-debug for the list). - - - - - - - -Advanced Topics - -Once a tool becomes more complicated, there are some extra -things you may want/need to do. - Suppressions If your tool reports errors and you want to suppress some common ones, you can add suppressions to the suppression files. The relevant -files are valgrind/*.supp; the final suppression +files are *.supp; the final suppression file is aggregated from these files by combining the relevant .supp files depending on the versions of linux, X and glibc on a system. @@ -454,7 +386,7 @@ and glibc on a system. tool_name:suppression_name. The tool_name here is the name you specify for the tool during initialisation with -VG_(details_name)(). +VG_(details_name). @@ -462,101 +394,60 @@ for the tool during initialisation with Documentation -As of version 3.0.0, Valgrind documentation has been converted to -XML. Why? See The XML FAQ. - - - - -The XML Toolchain - If you are feeling conscientious and want to write some -documentation for your tool, please use XML. The Valgrind -Docs use the following toolchain and versions: - - - xmllint: using libxml version 20607 - xsltproc: using libxml 20607, libxslt 10102 and libexslt 802 - pdfxmltex: pdfTeX (Web2C 7.4.5) 3.14159-1.10b - pdftops: version 3.00 - DocBook: version 4.2 - - -Latency: you should note that latency is -a big problem: DocBook is constantly being updated, but the tools -tend to lag behind somewhat. It is important that the versions -get on with each other, so if you decide to upgrade something, -then you need to ascertain whether things still work nicely - -this *cannot* be assumed. - -Stylesheets: The Valgrind docs use -various custom stylesheet layers, all of which are in -valgrind/docs/lib/. You -shouldn't need to modify these in any way. - -Catalogs: Catalogs provide a mapping from -generic addresses to specific local directories on a given machine. -Most recent Linux distributions have adopted a common place for storing -catalogs (/etc/xml/). Assuming that you have the -various tools listed above installed, you probably won't need to modify -your catalogs. But if you do, then just add another -group to this file, reflecting your -local installation. - - - - - -Writing the Documentation - -Follow these steps (using foobar -as the example tool name again): +documentation for your tool, please use XML as the rest of Valgrind does. +The file docs/README has more details on getting +the XML toolchain to work; this can be difficult, unfortunately. + +To write the documentation, follow these steps (using +foobar as the example tool name +again): The docs go in - valgrind/foobar/docs/, which you will + foobar/docs/, which you will have created when you started writing the tool. - - Write foobar/docs/Makefile.am. Use - memcheck/docs/Makefile.am as an - example. - - Copy the XML documentation file for the tool Nulgrind from - valgrind/none/docs/nl-manual.xml to + none/docs/nl-manual.xml to foobar/docs/, and rename it to foobar/docs/fb-manual.xml. - Note: there is a *really stupid* tetex bug - with underscores in filenames, so don't use '_'. + Note: there is a tetex bug + involving underscores in filenames, so don't use '_'. Write the documentation. There are some helpful bits and - pieces on using xml markup in - valgrind/docs/xml/xml_help.txt. + pieces on using XML markup in + docs/xml/xml_help.txt. Include it in the User Manual by adding the relevant entry to - valgrind/docs/xml/manual.xml. Copy and edit an + docs/xml/manual.xml. Copy and edit an existing entry. + + Include it in the man page by adding the relevant entry to + docs/xml/valgrind-manpage.xml. Copy and + edit an existing entry. + + Validate foobar/docs/fb-manual.xml using - the following command from within valgrind/docs/: + the following command from within docs/: - You will probably get errors that look like this: + You may get errors that look like this: Ignore (only) these -- they're not important. - Because the xml toolchain is fragile, it is important to ensure + Because the XML toolchain is fragile, it is important to ensure that fb-manual.xml won't break the documentation - set build. Note that just because an xml file happily transforms to + set build. Note that just because an XML file happily transforms to html does not necessarily mean the same holds true for pdf/ps. @@ -575,30 +466,31 @@ attribute base of element chapter You can (re-)generate the HTML docs while you are writing fb-manual.xml to help you see how it's looking. The generated files end up in - valgrind/docs/html/. Use the following - command, within valgrind/docs/: + docs/html/. Use the following + command, within docs/: - When you have finished, also generate pdf and ps output to - check all is well, from within valgrind/docs/: + When you have finished, try to generate PDF and PostScript output to + check all is well, from within docs/: Check the output .pdf and .ps files in - valgrind/docs/print/. + docs/print/. + + Note that the toolchain is even more fragile for the print docs, + so don't feel too bad if you can't get it working. - - @@ -650,12 +542,14 @@ write regression tests for your tool: Profiling -To profile a tool, use Cachegrind on it. Read README_DEVELOPERS for -details on running Valgrind under Valgrind. +Lots of profiling tools have trouble running Valgrind. For example, +trying to use gprof is hopeless. -Alternatively, you can use OProfile. In most cases, it is better than -Cachegrind because it's much faster, and gives real times, as opposed to -instruction and cache hit/miss counts. +Probably the best way to profile a tool is with OProfile on Linux. + +You can also use Cachegrind on it. Read +README_DEVELOPERS for details on running Valgrind under +Valgrind; it's a bit fragile but can usually be made to work. @@ -665,40 +559,35 @@ instruction and cache hit/miss counts. Other Makefile Hackery If you add any directories under -valgrind/foobar/, you will need to add +foobar/, you will need to add an appropriate Makefile.am to it, and add a corresponding entry to the AC_OUTPUT -list in valgrind/configure.in. +list in configure.in. If you add any scripts to your tool (see Cachegrind for an example) you need to add them to the bin_SCRIPTS variable in -valgrind/foobar/Makefile.am. +foobar/Makefile.am and possible also to the +AC_OUTPUT list in +configure.in. -Core/tool Interface Versions - -In order to allow for the core/tool interface to evolve over time, -Valgrind uses a basic interface versioning system. All a tool has to do -is use the -VG_DETERMINE_INTERFACE_VERSION macro -exactly once in its code. If not, a link error will occur when the tool -is built. - -The interface version number is changed when binary incompatible -changes are made to the interface. If the core and tool has the same major -version number X they should work together. If X doesn't match, Valgrind -will abort execution with an explanation of the problem. - -This approach was chosen so that if the interface changes in the -future, old tools won't work and the reason will be clearly explained, -instead of possibly crashing mysteriously. We have attempted to -minimise the potential for binary incompatible changes by means such as -minimising the use of naked structs in the interface. +The Core/tool Interface + +The core/tool interface evolves over time, but it's pretty stable. +We deliberately do not provide backward compatibility with old interfaces, +because it is too difficult and too restrictive. We view this as a good +thing -- if we had to be backward compatible with earlier versions, many +improvements now in the system could not have been added. + +Because tools are statically linked with the core, if a tool compiles +successfully then it should be compatible with the core. We would not +deliberately violate this property by, for example, changing the behaviour +of a core function without changing its prototype. @@ -709,16 +598,9 @@ minimising the use of naked structs in the interface. Final Words -The core/tool interface is not fixed. It's pretty stable these days, -but it does change. We deliberately do not provide backward compatibility -with old interfaces, because it is too difficult and too restrictive. -The interface checking should catch any incompatibilities. We view this as -a good thing -- if we had to be backward compatible with earlier versions, -many improvements now in the system could not have been added. - - - -Happy programming. +Writing a new Valgrind tool is not easy, but the tools you can write +with Valgrind are among the most powerful programming tools there are. +Happy programming! -- 2.47.3