From: Nicholas Nethercote Date: Fri, 7 Aug 2009 02:18:00 +0000 (+0000) Subject: Overhaul Helgrind's manual chapter. X-Git-Tag: svn/VALGRIND_3_5_0~119 X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=789339fe6f5823b7f11ede61e2b9224e2ea9f5f3;p=thirdparty%2Fvalgrind.git Overhaul Helgrind's manual chapter. git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10731 --- diff --git a/helgrind/docs/hg-manual.xml b/helgrind/docs/hg-manual.xml index 2b2389899b..458ba6b64d 100644 --- a/helgrind/docs/hg-manual.xml +++ b/helgrind/docs/hg-manual.xml @@ -41,9 +41,6 @@ in detail in the next three sections: Data races -- accessing memory without adequate locking or synchronisation. - Note that race detection in versions 3.4.0 and later uses a - different algorithm than in 3.3.x. Hence, if you have been using - Helgrind in 3.3.x, you may want to re-read this section. @@ -106,7 +103,7 @@ are: error code that must be handled when a thread exits whilst still holding locked locks - calling pthread_cond_wait + calling pthread_cond_wait with a not-locked mutex, an invalid mutex, or one locked by a different thread @@ -119,7 +116,7 @@ are: waiting waiting on an uninitialised pthread barrier - for all of the pthread_ functions that Helgrind + for all of the pthreads functions that Helgrind intercepts, an error is reported, along with a stack trace, if the system threading library routine returns an error code, even if Helgrind itself detected no @@ -288,10 +285,10 @@ int main ( void ) { ]]> The problem is there is nothing to -stop var being updated simultaneously +stop var being updated simultaneously by both threads. A correct program would -protect var with a lock of type -pthread_mutex_t, which is acquired +protect var with a lock of type +pthread_mutex_t, which is acquired before each access and released afterwards. Helgrind's output for this program is: @@ -374,8 +371,8 @@ the basic functionality provided by the threading library (POSIX Pthreads): thread creation, thread joining, locks, condition variables, semaphores and barriers. -The effect of using these functions is to impose on a threaded -program, constraints upon the order in which memory accesses can +The effect of using these functions is to impose +constraints upon the order in which memory accesses can happen. This implied ordering is generally known as the "happens-before relation". Once you understand the happens-before relation, it is easy to see how Helgrind finds races in your code. @@ -465,7 +462,7 @@ is accessed by two different threads, Helgrind checks to see if the two accesses are ordered by the happens-before relation. If so, that's fine; if not, it reports a race. -It is important to understand the the happens-before relation +It is important to understand that the happens-before relation creates only a partial ordering, not a total ordering. An example of a total ordering is comparison of numbers: for any two numbers x and @@ -535,9 +532,9 @@ primitives are as follows: of the child. Similarly, when an exiting thread is reaped via a - call to pthread_join, once the call returns, the reaping thread - acquires a happens-after dependency relative to all memory accesses - made by the exiting thread. + call to pthread_join, once the call returns, the + reaping thread acquires a happens-after dependency relative to all memory + accesses made by the exiting thread. @@ -559,9 +556,9 @@ to the other, then it reports a race. Two accesses are considered to be ordered by the happens-before dependency even through arbitrarily long chains of synchronisation events. For example, if T1 accesses some location - L, and then pthread_cond_signals T2, which later - pthread_cond_signals T3, which then accesses L, then a suitable - happens-before dependency exists between the first and second + L, and then pthread_cond_signals T2, which later + pthread_cond_signals T3, which then accesses L, then + a suitable happens-before dependency exists between the first and second accesses, even though it involves two different inter-thread synchronisation events. @@ -708,11 +705,11 @@ of false data-race errors. use the POSIX threading primitives. Helgrind needs to be able to see all events pertaining to thread creation, exit, locking and other synchronisation events. To do so it intercepts many POSIX - pthread_ functions. + pthreads functions. Do not roll your own threading primitives (mutexes, etc) - from combinations of the Linux futex syscall, atomic counters and - wotnot. These throw Helgrind's internal what's-going-on models + from combinations of the Linux futex syscall, atomic counters, etc. + These throw Helgrind's internal what's-going-on models way off course and will give bogus results. Also, do not reimplement existing POSIX abstractions using @@ -742,11 +739,11 @@ of false data-race errors. Qt 4 and/or KDE4 applications. Runtime support library for GNU OpenMP (part of - GCC), at least GCC versions 4.2 and 4.3. The GNU OpenMP runtime - library (libgomp.so) constructs its own synchronisation - primitives using combinations of atomic memory instructions and - the futex syscall, which causes total chaos since in Helgrind - since it cannot "see" those. + GCC), at least for GCC versions 4.2 and 4.3. The GNU OpenMP runtime + library (libgomp.so) constructs its own + synchronisation primitives using combinations of atomic memory + instructions and the futex syscall, which causes total chaos since in + Helgrind since it cannot "see" those. Fortunately, this can be solved using a configuration-time flag (for GCC). Rebuild GCC from source, and configure using --disable-linux-futex. @@ -761,43 +758,47 @@ of false data-race errors. Avoid memory recycling. If you can't avoid it, you must use - tell Helgrind what is going on via the VALGRIND_HG_CLEAN_MEMORY - client request - (in helgrind.h). - - Helgrind is aware of standard memory allocation and - deallocation that occurs via malloc/free/new/delete and from entry - and exit of stack frames. In particular, when memory is - deallocated via free, delete, or function exit, Helgrind considers - that memory clean, so when it is eventually reallocated, its - history is irrelevant. + tell Helgrind what is going on via the + VALGRIND_HG_CLEAN_MEMORY client request (in + helgrind.h). + + Helgrind is aware of standard heap memory allocation and + deallocation that occurs via + malloc/free/new/delete + and from entry and exit of stack frames. In particular, when memory is + deallocated via free, delete, + or function exit, Helgrind considers that memory clean, so when it is + eventually reallocated, its history is irrelevant. However, it is common practice to implement memory recycling schemes. In these, memory to be freed is not handed to - malloc/delete, but instead put into a pool of free buffers to be - handed out again as required. The problem is that Helgrind has no + free/delete, but instead put + into a pool of free buffers to be handed out again as required. The + problem is that Helgrind has no way to know that such memory is logically no longer in use, and its history is irrelevant. Hence you must make that explicit, - using the VALGRIND_HG_CLEAN_MEMORY client request to specify the - relevant address ranges. It's easiest to put these requests into - the pool manager code, and use them either when memory is returned - to the pool, or is allocated from it. + using the VALGRIND_HG_CLEAN_MEMORY client request + to specify the relevant address ranges. It's easiest to put these + requests into the pool manager code, and use them either when memory is + returned to the pool, or is allocated from it. Avoid POSIX condition variables. If you can, use POSIX - semaphores (sem_t, sem_post, sem_wait) to do inter-thread event - signalling. Semaphores with an initial value of zero are - particularly useful for this. + semaphores (sem_t, sem_post, + sem_wait) to do inter-thread event signalling. + Semaphores with an initial value of zero are particularly useful for + this. Helgrind only partially correctly handles POSIX condition variables. This is because Helgrind can see inter-thread - dependencies between a pthread_cond_wait call and a - pthread_cond_signal/broadcast call only if the waiting thread - actually gets to the rendezvous first (so that it actually calls - pthread_cond_wait). It can't see dependencies between the threads - if the signaller arrives first. In the latter case, POSIX - guidelines imply that the associated boolean condition still + dependencies between a pthread_cond_wait call and a + pthread_cond_signal/pthread_cond_broadcast + call only if the waiting thread actually gets to the rendezvous first + (so that it actually calls + pthread_cond_wait). It can't see dependencies + between the threads if the signaller arrives first. In the latter case, + POSIX guidelines imply that the associated boolean condition still provides an inter-thread synchronisation event, but one which is invisible to Helgrind. @@ -859,16 +860,18 @@ unlock(mx) unlock(mx) - Round up all finished threads using pthread_join. Avoid + Round up all finished threads using + pthread_join. Avoid detaching threads: don't create threads in the detached state, and - don't call pthread_detach on existing threads. - - Using pthread_join to round up finished threads provides a - clear synchronisation point that both Helgrind and programmers can - see. If you don't call pthread_join on a thread, Helgrind has no - way to know when it finishes, relative to any significant - synchronisation points for other threads in the program. So it - assumes that the thread lingers indefinitely and can potentially + don't call pthread_detach on existing threads. + + Using pthread_join to round up finished + threads provides a clear synchronisation point that both Helgrind and + programmers can see. If you don't call + pthread_join on a thread, Helgrind has no way to + know when it finishes, relative to any + significant synchronisation points for other threads in the program. So + it assumes that the thread lingers indefinitely and can potentially interfere indefinitely with the memory state of the program. It has every right to assume that -- after all, it might really be the case that, for scheduling reasons, the exiting thread did run @@ -899,11 +902,12 @@ unlock(mx) unlock(mx) - POSIX requires that implementations of standard I/O (printf, - fprintf, fwrite, fread, etc) are thread safe. Unfortunately GNU - libc implements this by using internal locking primitives that - Helgrind is unable to intercept. Consequently Helgrind generates - many false race reports when you use these functions. + POSIX requires that implementations of standard I/O + (printf, fprintf, + fwrite, fread, etc) are thread + safe. Unfortunately GNU libc implements this by using internal locking + primitives that Helgrind is unable to intercept. Consequently Helgrind + generates many false race reports when you use these functions. Helgrind attempts to hide these errors using the standard Valgrind error-suppression mechanism. So, at least for simple @@ -923,7 +927,8 @@ unlock(mx) unlock(mx) where libpthread.so or ld.so is the object associated with the innermost stack frame, please file a bug report at - http://www.valgrind.org. + &vg-url;. + @@ -956,27 +961,36 @@ unlock(mx) unlock(mx) - + --history-level=none|partial|full [full] + full: show both stack traces for a data race (can be very slow) + approx: full trace for one thread, approx for the other (faster) + none: only show trace for one thread in a race (fastest) + + + + - + - When enabled (the default), Helgrind collects enough - information about "old" accesses that it can produce two stack - traces in a race report -- both the stack trace for the + When set to (the default), Helgrind + collects enough information about "old" accesses that it can produce + two stack traces in a race report -- both the stack trace for the current access, and the trace for the older, conflicting access. Collecting such information is expensive in both speed and - memory. This flag disables collection of such information. - Helgrind will run significantly faster and use less memory, - but without the conflicting access stacks, it will be very - much more difficult to track down the root causes of - races. However, this option may be useful in situations where - you just want to check for the presence or absence of races, - for example, when doing regression testing of a previously - race-free program. + memory. However, without it, it is very much more difficult to + track down the root causes of races. Nonetheless, you may not need + it in situations where you just want to check for the presence or + absence of races, for example, when doing regression testing of a + previously race-free program. + Setting this option to means that + Helgrind will show a full trace for one thread, and an approximation + for the other, and run faster. Setting it to + means that Helgrind will show a full trace for one thread, and + nothing for the other, and run faster again. @@ -1010,42 +1024,46 @@ unlock(mx) unlock(mx) + + + + +Helgrind Client Requests + +The following client requests are defined in +helgrind.h. See that file for exact details of their +arguments. + + + + + VALGRIND_HG_CLEAN_MEMORY, + This makes Helgrind forget everything it knows about a specified memory + range. This is particularly useful for memory allocators that wish to + recycle memory. + + + + + + + + A To-Do List for Helgrind @@ -1088,9 +1131,6 @@ some time. cycle, rather than only doing for size-2 cycles as at present. - Document the VALGRIND_HG_CLEAN_MEMORY client - request. - The conflicting access mechanism sometimes mysteriously fails to show the conflicting access' stack, even when provided with unbounded storage for conflicting access info. @@ -1104,8 +1144,8 @@ some time. Don't update the lock-order graph, and don't check - for errors, when a "try"-style lock operation happens (eg - pthread_mutex_trylock). Such calls do not add any real + for errors, when a "try"-style lock operation happens (e.g. + pthread_mutex_trylock). Such calls do not add any real restrictions to the locking order, since they can always fail to acquire the lock, resulting in the caller going off and doing Plan B (presumably it will have a Plan B). Doing such checks could diff --git a/helgrind/hg_main.c b/helgrind/hg_main.c index 6798065fd3..7758a344df 100644 --- a/helgrind/hg_main.c +++ b/helgrind/hg_main.c @@ -4210,7 +4210,7 @@ static void hg_print_usage ( void ) { VG_(printf)( " --track-lockorders=no|yes show lock ordering errors? [yes]\n" -" --history-level=none|partial|full [full]\n" +" --history-level=none|approx|full [full]\n" " full: show both stack traces for a data race (can be very slow)\n" " approx: full trace for one thread, approx for the other (faster)\n" " none: only show trace for one thread in a race (fastest)\n"