2 * This file is part of PowerDNS or dnsdist.
3 * Copyright -- PowerDNS.COM B.V. and its contributors
5 * This program is free software; you can redistribute it and/or modify
6 * it under the terms of version 2 of the GNU General Public License as
7 * published by the Free Software Foundation.
9 * In addition, for the avoidance of any doubt, permission is granted to
10 * link this program with OpenSSL and to (re)distribute the binaries
11 * produced as the result of such linking.
13 * This program is distributed in the hope that it will be useful,
14 * but WITHOUT ANY WARRANTY; without even the implied warranty of
15 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 * GNU General Public License for more details.
18 * You should have received a copy of the GNU General Public License
19 * along with this program; if not, write to the Free Software
20 * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
30 #ifdef PDNS_USE_VALGRIND
31 #include <valgrind/valgrind.h>
32 #endif /* PDNS_USE_VALGRIND */
35 Simple system for implementing cooperative multitasking of functions, with
36 support for waiting on events which can return values.
38 \section copyright Copyright and License
39 MTasker is (c) 2002 - 2009 by bert hubert. It is licensed to you under the terms of the GPL version 2.
41 \section overview High level overview
42 MTasker is designed to support very simple cooperative multitasking to facilitate writing
43 code that would ordinarily require a statemachine, for which the author does not consider
46 This class does not perform any magic it only makes calls to makecontext() and swapcontext().
47 Getting the details right however is complicated and MTasker does that for you.
49 If preemptive multitasking or more advanced concepts such as semaphores, locks or mutexes
50 are required, the use of POSIX threads is advised.
52 MTasker is designed to offer the performance of statemachines while maintaining simple thread semantics. It is not
53 a replacement for a full threading system.
55 \section compatibility Compatibility
56 MTasker is only guaranteed to work on Linux with glibc 2.2.5 and higher. It does not work on FreeBSD and notably,
57 not on Red Hat 6.0. It may work on Solaris, please test.
59 \section concepts Concepts
61 There are two important concepts, the 'kernel' and the 'thread'. Each thread starts out as a function,
62 which is passed to MTasker::makeThread(), together with a possible argument.
64 This function is now free to do whatever it wants, but realise that MTasker implements cooperative
65 multitasking, which means that the coder has the responsibility of not taking the CPU overly long.
66 Other threads can only get the CPU if MTasker::yield() is called or if a thread sleeps to wait for an event,
67 using the MTasker::waitEvent() method.
69 \section kernel The Kernel
70 The Kernel consists of functions that do housekeeping, but also of code that the client coder
71 can call to report events. A minimal kernel loop looks like this:
75 if(MT.noProcesses()) // exit if no processes are left
80 The kernel typically starts from the main() function of your program. New threads are also
81 created from the kernel. This can also happen before entering the main loop. To start a thread,
82 the method MTasker::makeThread is provided.
84 \section events Events
85 By default, Events are recognized by an int and their value is also an int.
86 This can be overridden by specifying the EventKey and EventVal template parameters.
88 An event can be a keypress, but also a UDP packet, or a bit of data from a TCP socket. The
89 sample code provided works with keypresses, but that is just a not very useful example.
91 A thread can also wait for an event only for a limited time, and receive a timeout of that
92 event did not occur within the specified timeframe.
94 \section example A simple menu system
98 void menuHandler(void *p)
101 cout<<"Key handler for key "<<num<<" launched"<<endl;
104 cout<<"Key "<<num<<" was pressed!"<<endl;
112 for(int i=0;i<10;++i)
113 MT.makeThread(menuHandler,(void *)i);
116 while(MT.schedule()); // do everything we can do
117 if(MT.noProcesses()) // exit if no processes are left
120 if(!fgets(line,sizeof(line),stdin))
123 MT.sendEvent(*line-'0');
128 \section example2 Canonical multitasking example
129 This implements the canonical multitasking example, alternately printing an 'A' and a 'B'. The Linux kernel
130 started this way too.
132 void printer(void *p)
144 MT.makeThread(printer,(void*)'a');
145 MT.makeThread(printer,(void*)'b');
148 while(MT.schedule()); // do everything we can do
149 if(MT.noProcesses()) // exit if no processes are left
157 //! puts a thread to sleep waiting until a specified event arrives
158 /** Threads can call waitEvent to register that they are waiting on an event with a certain key.
159 If so desired, the event can carry data which is returned in val in case that is non-zero.
161 Furthermore, a timeout can be specified in seconds.
163 Only one thread can be waiting on a key, results of trying to have more threads
164 waiting on the same key are undefined.
166 \param key Event key to wait for. Needs to match up to a key reported to sendEvent
167 \param val If non-zero, the value of the event will be stored in *val
168 \param timeout If non-zero, number of seconds to wait for an event.
170 \return returns -1 in case of error, 0 in case of timeout, 1 in case of an answer
173 template<class EventKey
, class EventVal
>int MTasker
<EventKey
,EventVal
>::waitEvent(EventKey
&key
, EventVal
*val
, unsigned int timeoutMsec
, struct timeval
* now
)
175 if(d_waiters
.count(key
)) { // there was already an exact same waiter
180 w
.context
=std::make_shared
<pdns_ucontext_t
>();
181 w
.ttd
.tv_sec
= 0; w
.ttd
.tv_usec
= 0;
183 struct timeval increment
;
184 increment
.tv_sec
= timeoutMsec
/ 1000;
185 increment
.tv_usec
= 1000 * (timeoutMsec
% 1000);
187 w
.ttd
= increment
+ *now
;
189 struct timeval realnow
;
190 gettimeofday(&realnow
, 0);
191 w
.ttd
= increment
+ realnow
;
200 unsigned int diff
=d_threads
[d_tid
].dt
.ndiff()/1000;
201 d_threads
[d_tid
].totTime
+=diff
;
203 notifyStackSwitchToKernel();
204 pdns_swapcontext(*d_waiters
.find(key
)->context
,d_kernel
); // 'A' will return here when 'key' has arrived, hands over control to kernel first
205 notifyStackSwitchDone();
207 d_threads
[d_tid
].dt
.start();
209 if(val
&& d_waitstatus
==Answer
)
212 if((char*)&w
< d_threads
[d_tid
].highestStackSeen
) {
213 d_threads
[d_tid
].highestStackSeen
= (char*)&w
;
219 //! yields control to the kernel or other threads
220 /** Hands over control to the kernel, allowing other processes to run, or events to arrive */
222 template<class Key
, class Val
>void MTasker
<Key
,Val
>::yield()
224 d_runQueue
.push(d_tid
);
225 notifyStackSwitchToKernel();
226 pdns_swapcontext(*d_threads
[d_tid
].context
,d_kernel
); // give control to the kernel
227 notifyStackSwitchDone();
230 //! reports that an event took place for which threads may be waiting
231 /** From the kernel loop, sendEvent can be called to report that something occurred for which there may be waiters.
232 \param key Key of the event for which threads may be waiting
233 \param val If non-zero, pointer to the content of the event
234 \return Returns -1 in case of error, 0 if there were no waiters, 1 if a thread was woken up.
236 WARNING: when passing val as zero, d_waitval is undefined, and hence waitEvent will return undefined!
238 template<class EventKey
, class EventVal
>int MTasker
<EventKey
,EventVal
>::sendEvent(const EventKey
& key
, const EventVal
* val
)
240 typename
waiters_t::iterator waiter
=d_waiters
.find(key
);
242 if(waiter
== d_waiters
.end()) {
243 // cout<<"Event sent nobody was waiting for!"<<endl;
251 d_tid
=waiter
->tid
; // set tid
252 d_eventkey
=waiter
->key
; // pass waitEvent the exact key it was woken for
253 auto userspace
=std::move(waiter
->context
);
254 d_waiters
.erase(waiter
); // removes the waitpoint
255 notifyStackSwitch(d_threads
[d_tid
].startOfStack
, d_stacksize
);
256 pdns_swapcontext(d_kernel
,*userspace
); // swaps back to the above point 'A'
257 notifyStackSwitchDone();
261 //! launches a new thread
262 /** The kernel can call this to make a new thread, which starts at the function start and gets passed the val void pointer.
263 \param start Pointer to the function which will form the start of the thread
264 \param val A void pointer that can be used to pass data to the thread
266 template<class Key
, class Val
>void MTasker
<Key
,Val
>::makeThread(tfunc_t
*start
, void* val
)
268 auto uc
=std::make_shared
<pdns_ucontext_t
>();
270 uc
->uc_link
= &d_kernel
; // come back to kernel after dying
271 uc
->uc_stack
.resize (d_stacksize
+1);
272 #ifdef PDNS_USE_VALGRIND
273 uc
->valgrind_id
= VALGRIND_STACK_REGISTER(&uc
->uc_stack
[0],
274 &uc
->uc_stack
[uc
->uc_stack
.size()-1]);
275 #endif /* PDNS_USE_VALGRIND */
278 auto& thread
= d_threads
[d_maxtid
];
280 thread
.start
= [start
, val
, mt
]() {
282 mt
->d_threads
[mt
->d_tid
].startOfStack
= mt
->d_threads
[mt
->d_tid
].highestStackSeen
= &dummy
;
283 auto const tid
= mt
->d_tid
;
285 mt
->d_zombiesQueue
.push(tid
);
287 pdns_makecontext (*uc
, thread
.start
);
289 thread
.context
= std::move(uc
);
290 d_runQueue
.push(d_maxtid
++); // will run at next schedule invocation
294 //! needs to be called periodically so threads can run and housekeeping can be performed
295 /** The kernel should call this function every once in a while. It makes sense
296 to call this function if you:
299 - want to have threads running waitEvent() to get a timeout if enough time passed
301 \return Returns if there is more work scheduled and recalling schedule now would be useful
304 template<class Key
, class Val
>bool MTasker
<Key
,Val
>::schedule(struct timeval
* now
)
306 if(!d_runQueue
.empty()) {
307 d_tid
=d_runQueue
.front();
309 d_threads
[d_tid
].dt
.start();
311 notifyStackSwitch(d_threads
[d_tid
].startOfStack
, d_stacksize
);
312 pdns_swapcontext(d_kernel
, *d_threads
[d_tid
].context
);
313 notifyStackSwitchDone();
318 if(!d_zombiesQueue
.empty()) {
319 d_threads
.erase(d_zombiesQueue
.front());
321 d_zombiesQueue
.pop();
324 if(!d_waiters
.empty()) {
327 gettimeofday(&rnow
, 0);
331 typedef typename
waiters_t::template index
<KeyTag
>::type waiters_by_ttd_index_t
;
332 // waiters_by_ttd_index_t& ttdindex=d_waiters.template get<KeyTag>();
333 waiters_by_ttd_index_t
& ttdindex
=boost::multi_index::get
<KeyTag
>(d_waiters
);
335 for(typename
waiters_by_ttd_index_t::iterator i
=ttdindex
.begin(); i
!= ttdindex
.end(); ) {
336 if(i
->ttd
.tv_sec
&& i
->ttd
< rnow
) {
337 d_waitstatus
=TimeOut
;
338 d_eventkey
=i
->key
; // pass waitEvent the exact key it was woken for
339 auto uc
= i
->context
;
341 ttdindex
.erase(i
++); // removes the waitpoint
343 notifyStackSwitch(d_threads
[d_tid
].startOfStack
, d_stacksize
);
344 pdns_swapcontext(d_kernel
, *uc
); // swaps back to the above point 'A'
345 notifyStackSwitchDone();
347 else if(i
->ttd
.tv_sec
)
356 //! returns true if there are no processes
357 /** Call this to check if no processes are running anymore
358 \return true if no processes are left
360 template<class Key
, class Val
>bool MTasker
<Key
,Val
>::noProcesses() const
362 return d_threadsCount
== 0;
365 //! returns the number of processes running
366 /** Call this to perhaps limit activities if too many threads are running
367 \return number of processes running
369 template<class Key
, class Val
>unsigned int MTasker
<Key
,Val
>::numProcesses() const
371 return d_threadsCount
;
374 //! gives access to the list of Events threads are waiting for
375 /** The kernel can call this to get a list of Events threads are waiting for. This is very useful
376 to setup 'select' or 'poll' or 'aio' events needed to satisfy these requests.
377 getEvents clears the events parameter before filling it.
379 \param events Vector which is to be filled with keys threads are waiting for
381 template<class Key
, class Val
>void MTasker
<Key
,Val
>::getEvents(std::vector
<Key
>& events
)
384 for(typename
waiters_t::const_iterator i
=d_waiters
.begin();i
!=d_waiters
.end();++i
) {
385 events
.push_back(i
->first
);
389 //! Returns the current Thread ID (tid)
390 /** Processes can call this to get a numerical representation of their current thread ID.
391 This can be useful for logging purposes.
393 template<class Key
, class Val
>int MTasker
<Key
,Val
>::getTid() const
398 //! Returns the maximum stack usage so far of this MThread
399 template<class Key
, class Val
>unsigned int MTasker
<Key
,Val
>::getMaxStackUsage()
401 return d_threads
[d_tid
].startOfStack
- d_threads
[d_tid
].highestStackSeen
;
404 //! Returns the maximum stack usage so far of this MThread
405 template<class Key
, class Val
>unsigned int MTasker
<Key
,Val
>::getUsec()
408 return d_threads
[d_tid
].totTime
+ d_threads
[d_tid
].dt
.ndiff()/1000;