From c3617f305ae2ee928ce94ab57022b6e87073382b Mon Sep 17 00:00:00 2001 From: Julian Seward Date: Wed, 11 May 2011 15:26:06 +0000 Subject: [PATCH] Update the manual following removal of heap-check functionality. git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11743 --- exp-ptrcheck/docs/pc-manual.xml | 366 ++++++++------------------------ 1 file changed, 86 insertions(+), 280 deletions(-) diff --git a/exp-ptrcheck/docs/pc-manual.xml b/exp-ptrcheck/docs/pc-manual.xml index 01c549e069..8632199724 100644 --- a/exp-ptrcheck/docs/pc-manual.xml +++ b/exp-ptrcheck/docs/pc-manual.xml @@ -4,141 +4,49 @@ [ %vg-entities; ]> - - Ptrcheck: an experimental heap, stack and global array overrun detector + + SGCheck: an experimental stack and global array overrun detector To use this tool, you must specify - on the Valgrind + on the Valgrind command line. - + Overview -Ptrcheck is a tool for finding overruns of heap, stack -and global arrays. Its functionality overlaps somewhat with -Memcheck's, but it is able to catch invalid accesses in a number of -cases that Memcheck would miss. A detailed comparison against -Memcheck is presented below. - -Ptrcheck is composed of two almost completely independent tools -that have been glued together. One part, -in h_main.[ch], checks accesses -through heap-derived pointers. The other part, in -sg_main.[ch], checks accesses to -stack and global arrays. The remaining -files pc_{common,main}.[ch], provide -common error-management and coordination functions, so as to make it -appear as a single tool. - -The heap-check part is an extensively-hacked (largely rewritten) -version of the experimental "Annelid" tool developed and described by -Nicholas Nethercote and Jeremy Fitzhardinge. The stack- and global- -check part uses a heuristic approach derived from an observation about -the likely forms of stack and global array accesses, and, as far as is -known, is entirely novel. - - - - - - - -Ptrcheck Command-line Options - -Ptrcheck-specific command-line options are: - - - - - - - - - - By default, Ptrcheck checks for overruns of stack, global - and heap arrays. - With --enable-sg-checks=no, the stack and - global array checks are omitted, and only heap checking is - performed. This can be useful because the stack and global - checks are quite expensive, so omitting them speeds Ptrcheck up - a lot. - - - - - - - - - - This option has the same meaning as it does for - Memcheck. - Controls how Ptrcheck handles word-sized, word-aligned - loads which partially overlap the end of heap blocks -- that is, - some of the bytes in the word are validly addressable, but - others are not. When yes, such loads do not - produce an address error. When no (the - default), loads from partially invalid addresses are treated the - same as loads from completely invalid addresses: an illegal heap - access error is issued. - - Note that code that behaves in this way is in violation of - the the ISO C/C++ standards, and should be considered broken. If - at all possible, such code should be fixed. This option should be - used only as a last resort. - - - - - +SGCheck is a tool for finding overruns of stack and global +arrays. It works by using a heuristic approach derived from an +observation about the likely forms of stack and global array accesses. + - -How Ptrcheck Works: Heap Checks + +SGCheck Command-line Options -Ptrcheck can check for invalid uses of heap pointers, including -out of range accesses and accesses to freed memory. The mechanism is -however completely different from Memcheck's, and the checking is more -powerful. +There are no SGCheck-specific command-line options at present. + - -How Ptrcheck Works: Stack and Global Checks + +How SGCheck Works When a source file is compiled with , the compiler attaches DWARF3 @@ -150,7 +58,7 @@ simple, if the compiler could also tell us which array (if any) each memory referencing instruction was supposed to access. Unfortunately the DWARF3 debugging format does not provide a way to represent such information, so we have to resort to a heuristic technique to -approximate the same information. The key observation is that +approximate it. The key observation is that if a memory referencing instruction accesses inside a stack or global array once, then it is highly likely to always access that @@ -173,7 +81,7 @@ we will (correctly) assume that that instruction is intended always to access a[]. Then, on the 11th iteration, it accesses somewhere else, possibly a different local, possibly an un-accounted for area of the stack (eg, spill slot), so -Ptrcheck reports an error. +SGCheck reports an error. There is an important caveat. @@ -184,86 +92,31 @@ copying loop only ever access one particular stack or global variable, we will be flooded with errors resulting from calls to memcpy. -To avoid this problem, Ptrcheck instantiates fresh likely-target +To avoid this problem, SGCheck instantiates fresh likely-target records for each entry to a function, and discards them on exit. This -allows detection of cases where (e.g.) memcpy overflows -its source or destination buffers for any specific call, but does not carry -any restriction from one call to the next. Indeed, multiple threads may be -multiple simultaneous calls to (e.g.) memcpy without -mutual interference. +allows detection of cases where (e.g.) memcpy +overflows its source or destination buffers for any specific call, but +does not carry any restriction from one call to the next. Indeed, +multiple threads may make multiple simultaneous calls to +(e.g.) memcpy without mutual interference. - Comparison with Memcheck -Memcheck does not do any access checks for stack or global arrays, so -the presence of those in Ptrcheck is a straight win. (But see -"Limitations" below). - -Memcheck and Ptrcheck use different approaches for checking heap -accesses. Memcheck maintains bitmaps telling it which areas of memory -are accessible and which are not. If a memory access falls in an -unaccessible area, it reports an error. By marking the 16 bytes -before and after an allocated block unaccessible, Memcheck is able to -detect small over- and underruns of the block. Similarly, by marking -freed memory as unaccessible, Memcheck can detect all accesses to -freed memory. - -Memcheck's approach is simple. But it's also weak. It can't -catch block overruns beyond 16 bytes. And, more generally, because it -focusses only on the question "is the target address accessible", it -fails to detect invalid accesses which just happen to fall within some -other valid area. This is not improbable, especially in crowded areas -of the process' address space. - -Ptrcheck's approach is to keep track of pointers derived from -heap blocks. It tracks pointers which are derived directly from calls -to malloc et al, but also ones derived indirectly, by -adding or subtracting offsets from the directly-derived pointers. When a -pointer is finally used to access memory, Ptrcheck compares the access -address with that of the block it was originally derived from, and -reports an error if the access address is not within the block -bounds. - -Consequently Ptrcheck can detect any out of bounds access -through a heap-derived pointer, no matter how far from the original -block it is. - -A second advantage is that Ptrcheck is better at detecting -accesses to blocks freed very far in the past. Memcheck can detect -these too, but only for blocks freed relatively recently. To detect -accesses to a freed block, Memcheck must make it inaccessible, hence -requiring a space overhead proportional to the size of the block. If -the blocks are large, Memcheck will have to make them available for -re-allocation relatively quickly, thereby losing the ability to detect -invalid accesses to them. - -By contrast, Ptrcheck has a constant per-block space requirement -of four machine words, for detection of accesses to freed blocks. A -freed block can be reallocated immediately, yet Ptrcheck can still -detect all invalid accesses through any pointers derived from the old -allocation, providing only that the four-word descriptor for the old -allocation is stored. For example, on a 64-bit machine, to detect -accesses in any of the most recently freed 10 million blocks, Ptrcheck -will require only 320MB of extra storage. Achieving the same level of -detection with Memcheck is close to impossible and would likely -involve several gigabytes of extra storage. - -Having said all that, remember that Memcheck performs uninitialised -value checking, invalid and mismatched free checking, overlap checking, and -leak checking, none of which Ptrcheck do. Memcheck has also benefitted from -years of refinement, tuning, and experience with production-level usage, and -so is much faster than Ptrcheck as it currently stands. - +SGCheck and Memcheck are complementary: their capabilities do +not overlap. Memcheck performs bounds checks and use-after-free +checks for heap arrays. It also finds uses of uninitialised values +created by heap or stack allocations. But it does not perform bounds +checking for stack or global arrays. -Consequently we recommend you first make your programs run Memcheck -clean. Once that's done, try Ptrcheck to see if you can shake out any -further heap, global or stack errors. +SGCheck, on the other hand, does do bounds checking for stack or +global arrays, but it doesn't do anything else. @@ -271,7 +124,7 @@ further heap, global or stack errors. - Limitations @@ -283,46 +136,23 @@ of. - Heap checks: Ptrcheck can occasionally lose track of, or - become confused about, which heap block a given pointer has been - derived from. This can cause it to falsely report errors, or to - miss some errors. This is not believed to be a serious - problem. - - - - Heap checks: Ptrcheck only tracks pointers that are stored - properly aligned in memory. If a pointer is stored at a misaligned - address, and then later read again, Ptrcheck will lose track of - what it points at. Similar problem if a pointer is split into - pieces and later reconsitituted. - - - - Heap checks: Ptrcheck needs to "understand" which system - calls return pointers and which don't. Many, but not all system - calls are handled. If an unhandled one is encountered, Ptrcheck will - abort. Fortunately, adding support for a new syscall is very - easy. - - - - Stack checks: It follows from the description above () that the first access by a - memory referencing instruction to a stack or global array creates an - association between that instruction and the array, which is checked on - subsequent accesses by that instruction, until the containing function - exits. Hence, the first access by an instruction to an array (in any - given function instantiation) is not checked for overrun, since Ptrcheck + False negatives (missed errors): it follows from the + description above () + that the first access by a memory referencing instruction to a + stack or global array creates an association between that + instruction and the array, which is checked on subsequent accesses + by that instruction, until the containing function exits. Hence, + the first access by an instruction to an array (in any given + function instantiation) is not checked for overrun, since SGCheck uses that as the "example" of how subsequent accesses should behave. - Stack checks: Similarly, and more serious, it is clearly - possible to write legitimate pieces of code which break the basic - assumption upon which the stack/global checking rests. For - example: + False positives (false errors): similarly, and more serious, + it is clearly possible to write legitimate pieces of code which + break the basic assumption upon which the checking algorithm + depends. For example: - Performance: the stack/global checks require reading all of + Performance: SGCheck has to read all of the DWARF3 type and variable information on the executable and its shared objects. This is computationally expensive and makes startup quite slow. You can expect debuginfo reading time to be in the region of a minute for an OpenOffice sized application, on a 2.4 GHz Core 2 machine. Reading this information also requires a - lot of memory. To make it viable, Ptrcheck goes to considerable + lot of memory. To make it viable, SGCheck goes to considerable trouble to compress the in-memory representation of the DWARF3 data, which is why the process of reading it appears slow. - Performance: Ptrcheck runs slower than Memcheck. This is + Performance: SGCheck runs slower than Memcheck. This is partly due to a lack of tuning, but partly due to algorithmic - difficulties. The heap-check side is potentially quite fast. The + difficulties. The stack and global checks can sometimes require a number of range - checks per memory access, and these are difficult to short-circuit - (despite considerable efforts having been made). + checks per memory access, and these are difficult to short-circuit, + despite considerable efforts having been made. A + redesign and reimplementation could potentially make it much faster. - Coverage: the heap checking is relatively robust, requiring - only that Ptrcheck can see calls to malloc et al. - In that sense it has debug-info requirements comparable with Memcheck, - and is able to heap-check programs even with no debugging information - attached. - - Stack/global checking is much more fragile. If a shared - object does not have debug information attached, then Ptrcheck will + Coverage: Stack and global checking is fragile. If a shared + object does not have debug information attached, then SGCheck will not be able to determine the bounds of any stack or global arrays defined within that shared object, and so will not be able to check accesses to them. This is true even when those arrays are accessed from some other shared object which was compiled with debug info. - At the moment Ptrcheck accepts objects lacking debuginfo - without comment. This is dangerous as it causes Ptrcheck to + At the moment SGCheck accepts objects lacking debuginfo + without comment. This is dangerous as it causes SGCheck to silently skip stack and global checking for such objects. It would be better to print a warning in such circumstances. - Coverage: Ptrcheck checks that the areas read or written by - system calls do not overrun heap blocks. But it doesn't currently - check them for overruns stack and global arrays. This would be - easy to add. + Coverage: SGCheck does not check whether the the areas read + or written by system calls do overrun stack or global arrays. This + would be easy to add. - Platforms: the stack/global checks won't work properly on any - PowerPC platforms, only on x86 and amd64 targets. That's because - the stack and global checking requires tracking function calls and - exits reliably, and there's no obvious way to do it with the PPC - ABIs. (In comparison, with the x86 and amd64 ABIs this is relatively - straightforward.) + Platforms: the stack/global checks won't work properly on + PowerPC, ARM or S390X platforms, only on X86 and AMD64 targets. + That's because the stack and global checking requires tracking + function calls and exits reliably, and there's no obvious way to do + it on ABIs that use a link register for function returns. + Robustness: related to the previous point. Function - call/exit tracking for x86/amd64 is believed to work properly even - in the presence of longjmps within the same stack (although this - has not been tested). However, code which switches stacks is + call/exit tracking for X86 and AMD64 is believed to work properly + even in the presence of longjmps within the same stack (although + this has not been tested). However, code which switches stacks is likely to cause breakage/chaos. @@ -422,7 +246,7 @@ of. - Still To Do: User-visible Functionality @@ -438,6 +262,13 @@ of. found, or read. + + Add some heuristic filtering that removes obvious false + positives. This would be easy to do. For example, an access + transition from a heap to a stack object almost certainly isn't a + bug and so should not be reported to the user. + + @@ -445,7 +276,7 @@ of. - Still To Do: Implementation Tidying @@ -456,9 +287,12 @@ in real use. - h_main.c: make N_FREED_SEGS command-line configurable. + sg_main.c: Redesign and reimplement the basic checking + algorithm. It could be done much faster than it is -- the current + implementation isn't very good. + - + sg_main.c: Improve the performance of the stack / global checks by doing some up-front filtering to ignore references in @@ -467,34 +301,6 @@ in real use. space layout. - - h_main.c: get rid of the last_seg_added hack; add suitable - plumbing to the core/tool interface to do this cleanly. - - - - h_main.c: move vast amounts of arch-dependent ugliness - (get_IntRegInfo et al) to its own source file, a la - mc_machine.c. - - - - h_main.c: make the lossage-check stuff work again, as a way - of doing quality assurance on the implementation. - - - - h_main.c: schemeEw_Atom: don't generate a call to - nonptr_or_unknown, this is really stupid, since it could be done at - translation time instead. - - - - CRITICAL: h_main.c: h_instrument (main instrumentation fn): - generate shadows for word-sized temps defined in the block's - preamble. (Why does this work at all, as it stands?) - - sg_main.c: fix compute_II_hash to make it a bit more sensible for ppc32/64 targets (except that sg_ doesn't work on ppc32/64 -- 2.47.2