--- /dev/null
+Last Updated: Fri 30 Dec, 2010
+
+== Background ==
+
+FreeTDM is a threaded library. As such, locking considerations must be taken when using it and when writing code inside the library.
+
+At the moment locks are not exposed to users. This means API users cannot acquire a lock on a channel or span structure. There is no
+need for users to lock channels or spans since all their interactions with those structures should be done thru the FreeTDM API which
+can (and in most cases must) internally lock on their behalf.
+
+Internally, locking can be done either by the core or the signaling modules. To better understand the locking considerations we must
+understand first the threading model of FreeTDM.
+
+== Threading Model ==
+
+At startup, when the user calls ftdm_global_init(), just one timing thread is created to dispatch internal timers. If you write
+a signaling module or any other code using the scheduling API, you can choose to run your schedule in this timing thread or in
+a thread of your choice. This is the only thread launched at initialization.
+
+If the application decides to use ftdm_global_configuration(), which reads freetdm.conf to create the spans and channels data
+structures, then possibly another thread will be launched for CPU usage monitoring (only if enabled in the configuration cpu_monitor=yes
+This thread sole purpose is to check the CPU and raise an alarm if reaches a configurable threshold, the alarm then is checked to avoid
+placing or receiving further calls.
+
+At this point FreeTDM has initialized and configured its channels input output configuration.
+
+The user is then supposed to configure the signaling via ftdm_configure_span_signaling() and then start the signaling work
+using ftdm_span_start(). This will typically launch at least 1 thread per span. Some signaling modules (actually just the analog one)
+launches another thread per channel when receiving a call. The rest of the signaling modules currently launch only one thread per
+span and the signaling for all channels within the span is handled in that thread. We call that thread 'the signaling thread'.
+
+At this point the user can start placing calls using the FreeTDM call API ftdm_channel_call_place(). Any of the possible threads in
+which the user calls the FreeTDM API is called 'the user thread', depending on the application thread model (the application using FreeTDM)
+this user thread may be different each time or the same all the time, we cannot make any assumptions. In the case of FreeSWITCH, the most
+common user of FreeTDM, the user thread is most of the cases a thread for each new call leg.
+
+At this point we have identified 4 types of threads.
+
+1. The timing thread (the core thread that triggers timers).
+ Its responsibility is simply check for timers that were scheduled and trigger them when the time comes. This means that if you decide
+ to use the scheduling API in freerun mode (where you use the core timer thread) you callbacks will be executed in this global thread
+ and you MUST not block at all since there might be other events waiting.
+
+2. The CPU thread (we don't really care about this one as it does not interact with channels or spans).
+
+3. The signaling thread.
+ There is one thread of this per span. This thread takes care of reading signaling specific messages from the network (ISDN network, etc) and
+ changing the channel call state accordingly and processing state changes caused by user API calls (like ftdm_channel_call_hangup for example).
+
+4. The user thread.
+ This is a thread in which the user decides to execute FreeTDM APIs, in some cases it might even be the same than the signaling thread (because
+ most SIGEVENT notifications are delivered by the signaling thread, however we are advicing users to not use FreeTDM unsafe APIs from the
+ thread where they receive SIGEVENT notifications as some APIs may block for a few milliseconds, effectively blocking the whole signaling thread
+ that is servicing a span.
+
+== Application Locking ==
+
+Users of the FreeTDM API will typically have locking of their own for their own application-specific data structures (in the case of FreeSWITCH, the
+session lock for example). Other application-specific locks may be involved.
+
+== DeadLocks ==
+
+As soon as we think of application locks, and we mix them with the FreeTDM internal locks, the possibility of deadlocks arise.
+
+A typical deadlock scenario when 2 locks are involved is:
+
+- User Thread - - Signaling Thread -
+1. Application locks applock. 1. A network message is received for a channel.
+
+2. Aplication invokes a FreeTDM call API (ie: ftdm_channel_call_hangup()). 2. The involved channel is locked.
+
+3. The FreeTDM API attempts to acquire the channel lock and stalls because 3. The message processing results in a notification
+ the signaling thread just acquired it. to be delivered to the user via the callback function
+ provided for that purpose. The callback is then called.
+
+4. The thread is now deadlocked because the signaling thread will never 4. The application callback attempts to acquire its application
+ release the channel lock. lock but deadlocks because the user thread already has it.
+
+To avoid this signaling modules should not deliver signals to the user while holding the channel lock. An easy way to avoid this is
+to not deliver signals while processing a state change, but rather defer them until the channel lock is released. Most new signaling modules
+accomplish this by setting the span flag FTDM_SPAN_USE_SIGNALS_QUEUE, this flag tells the core to enqueue signals (ie FTDM_SIGEVENT_START)
+when ftdm_span_send_signal() is called and not deliver them until ftdm_span_trigger_signals() is called, which is done by the signaling module
+in its signaling thread when no channel lock is being held.
+
+== State changes while locking ==
+
+Only 2 types of threads should be performing state changes.
+
+User threads.
+The user thread is a random thread that was crated by the API user. We do not know what threading model users of FreeTDM will follow
+and therefore cannot make assumptions about it. The user should be free to call FreeTDM APIs from any thread, except threads that
+are under our control, like the signaling threads. Although it may work in most situations, is discouraged for users to try
+to use FreeTDM APIs from the signaling thread, that is, the thread where the signaling callback provided during configuration
+is called (the callback where FTDM_SIGEVENT_XXX signals are delivered).
+
+A user thread may request state changes implicitly through calls to FreeTDM API's. The idea of state changes is internal to freetdm
+and should not be exposed to users of the API (except for debugging purposes, like the ftdm_channel_get_state, ftdm_channel_get_state_str etc)
+
+This is an example of the API's that implicitly request a state change.
+
+ftdm_channel_call_answer()
+
+Signaling modules should guarantee that upon releasing a lock on a channel, any state changes will be already processed and
+not deferred to other threads, otherwise that leads to a situation where a state change requested by the signaling module is pending
+to be serviced by another signaling module thread but a user thread wins the channel lock and attempts to perform a state change which will
+fail because another state change is pending (and user threads are not meant to process signaling states).
+
+ONLY one signaling thread per channel should try to perform state changes and processing of the states,
+otherwise complexity arises and is not worth it!
+
+At some point before we stablished this policies we could have 3 different threads doing state changes.
+
+1. A user random thread could implcitly try to change the state in response to a call API.
+2. The ftmod signaling thread could try to change the state in response to other state changes.
+3. The lower level signaling stack threads could try to change the state in response to stack events.
+
+As a result, lower level signaling stack thread could set a state and then let the signaling thread to
+process it, but when unlocking the channel, the user thread may win the lock over the signaling thread and
+may try to set a state change of its own and fail (due to the unprocessed state change)!
+
+The rule is, the signaling module should never unlock a channel with states pending to process this way the user,
+when acquiring a channel lock (inside ftdm_channel_call_answer for example) it will always find a consistent state
+for the channel and not in the middle of state processing.
+
+
return stream.data;
}
-FT_DECLARE(ftdm_status_t) ftdm_channel_advance_states(ftdm_channel_t *fchan, ftdm_channel_state_processor_t state_processor)
+FT_DECLARE(ftdm_status_t) ftdm_channel_advance_states(ftdm_channel_t *fchan)
{
ftdm_channel_state_t state;
- ftdm_channel_lock(fchan);
while (fchan->state_status == FTDM_STATE_STATUS_NEW) {
state = fchan->state;
ftdm_log_chan(fchan, FTDM_LOG_DEBUG, "Executing state processor for %s\n", ftdm_channel_state2str(fchan->state));
- state_processor(fchan);
+ fchan->span->state_processor(fchan);
if (state == fchan->state && fchan->state_status == FTDM_STATE_STATUS_NEW) {
/* if the state did not change and is still NEW, the state status must go to PROCESSED
* otherwise we don't touch it since is a new state and the old state was
fchan->state_status = FTDM_STATE_STATUS_PROCESSED;
}
}
- ftdm_channel_unlock(fchan);
+
return FTDM_SUCCESS;
}
ftdm_sched_cancel_timer(r2data->sched, r2call->protocol_error_recovery_timer);
ftdm_log_chan_msg(ftdmchan, FTDM_LOG_DEBUG, "Cancelled protocol error recovery timer\n");
ftdm_set_state(ftdmchan, FTDM_CHANNEL_STATE_DOWN);
- ftdm_channel_advance_states(ftdmchan, ftdm_r2_state_advance);
+ ftdm_channel_advance_states(ftdmchan);
}
}
ftdm_set_state(ftdmchan, FTDM_CHANNEL_STATE_DOWN);
/* in some circumstances openr2 can call on_call_init right after this, so let's advance the state right here */
- ftdm_channel_advance_states(ftdmchan, ftdm_r2_state_advance);
+ ftdm_channel_advance_states(ftdmchan);
}
static void ftdm_r2_on_call_read(openr2_chan_t *r2chan, const unsigned char *buf, int buflen)
goto done;
}
ftdm_set_state(ftdmchan, FTDM_CHANNEL_STATE_DOWN);
- ftdm_channel_advance_states(ftdmchan, ftdm_r2_state_advance);
+ ftdm_channel_advance_states(ftdmchan);
done:
ftdm_channel_unlock(ftdmchan);
}
span->set_channel_sig_status = ftdm_r2_set_channel_sig_status;
span->state_map = &r2_state_map;
+ span->state_processor = ftdm_r2_state_advance;
/* use signals queue */
ftdm_set_flag(span, FTDM_SPAN_USE_SIGNALS_QUEUE);
ftdm_clear_flag(ftdmchan, FTDM_CHANNEL_RX_DISABLED);
ftdm_clear_flag(ftdmchan, FTDM_CHANNEL_TX_DISABLED);
- ftdm_channel_advance_states(ftdmchan, ftdm_r2_state_advance);
+ ftdm_channel_advance_states(ftdmchan);
r2chan = call->r2chan;
openr2_chan_process_signaling(r2chan);
- ftdm_channel_advance_states(ftdmchan, ftdm_r2_state_advance);
+ ftdm_channel_advance_states(ftdmchan);
if (!call->accepted) {
/* if the call is not accepted we do not want users reading */
static ftdm_status_t ftdm_sangoma_isdn_start(ftdm_span_t *span);
ftdm_channel_t* ftdm_sangoma_isdn_process_event_states(ftdm_span_t *span, sngisdn_event_data_t *sngisdn_event);
-static void ftdm_sangoma_isdn_advance_chan_states(ftdm_channel_t *ftdmchan);
static void ftdm_sangoma_isdn_poll_events(ftdm_span_t *span);
static void ftdm_sangoma_isdn_process_phy_events(ftdm_span_t *span, ftdm_oob_event_t event);
-static void ftdm_sangoma_isdn_process_state_change(ftdm_channel_t *ftdmchan);
+static ftdm_status_t ftdm_sangoma_isdn_process_state_change(ftdm_channel_t *ftdmchan);
static void ftdm_sangoma_isdn_process_stack_event (ftdm_span_t *span, sngisdn_event_data_t *sngisdn_event);
static void ftdm_sangoma_isdn_wakeup_phy(ftdm_channel_t *dchan);
static void ftdm_sangoma_isdn_dchan_set_queue_size(ftdm_channel_t *ftdmchan);
}
};
-static __inline__ void ftdm_sangoma_isdn_advance_chan_states(ftdm_channel_t *ftdmchan)
-{
- while (ftdm_test_flag(ftdmchan, FTDM_CHANNEL_STATE_CHANGE)) {
- ftdm_sangoma_isdn_process_state_change(ftdmchan);
- }
-}
-
static void ftdm_sangoma_isdn_process_phy_events(ftdm_span_t *span, ftdm_oob_event_t event)
{
sngisdn_span_data_t *signal_data = (sngisdn_span_data_t*) span->signal_data;
while ((ftdmchan = ftdm_queue_dequeue(span->pendingchans))) {
/* double check that this channel has a state change pending */
ftdm_channel_lock(ftdmchan);
- ftdm_sangoma_isdn_advance_chan_states(ftdmchan);
+ ftdm_channel_advance_states(ftdmchan);
ftdm_channel_unlock(ftdmchan);
}
/* twiddle */
break;
case FTDM_FAIL:
- ftdm_log(FTDM_LOG_ERROR,"%s:ftdm_interrupt_wait returned error!\n", span->name);
+ ftdm_log(FTDM_LOG_ERROR, "%s: ftdm_interrupt_wait returned error!\n", span->name);
break;
default:
- ftdm_log(FTDM_LOG_ERROR,"%s:ftdm_interrupt_wait returned with unknown code\n", span->name);
+ ftdm_log(FTDM_LOG_ERROR, "%s: ftdm_interrupt_wait returned with unknown code\n", span->name);
break;
}
break;
}
ftdm_channel_lock(ftdmchan);
- ftdm_sangoma_isdn_advance_chan_states(ftdmchan);
+ ftdm_channel_advance_states(ftdmchan);
return ftdmchan;
}
sngisdn_process_rst_ind(sngisdn_event);
break;
}
- if(ftdmchan != NULL) {
- ftdm_sangoma_isdn_advance_chan_states(ftdmchan);
+ if (ftdmchan != NULL) {
+ ftdm_channel_advance_states(ftdmchan);
ftdm_channel_unlock(ftdmchan);
}
}
-static void ftdm_sangoma_isdn_process_state_change(ftdm_channel_t *ftdmchan)
+/* this function is called with the channel already locked by the core */
+static ftdm_status_t ftdm_sangoma_isdn_process_state_change(ftdm_channel_t *ftdmchan)
{
ftdm_sigmsg_t sigev;
ftdm_channel_state_t initial_state;
sigev.span_id = ftdmchan->span_id;
sigev.channel = ftdmchan;
- /*first lock the channel*/
- ftdm_channel_lock(ftdmchan);
- /*clear the state change flag...since we might be setting a new state*/
- ftdm_clear_flag(ftdmchan, FTDM_CHANNEL_STATE_CHANGE);
+ /* Acknowledge the state change */
+ ftdm_channel_complete_state(ftdmchan);
+
#ifdef FTDM_DEBUG_CHAN_MEMORY
if (ftdmchan->state == FTDM_CHANNEL_STATE_DIALING) {
- ftdm_assert(mprotect(ftdmchan, sizeof(*ftdmchan), PROT_READ)==0, "Failed to mprotect");
+ ftdm_assert(mprotect(ftdmchan, sizeof(*ftdmchan), PROT_READ) == 0, "Failed to mprotect");
}
#endif
}
#ifdef FTDM_DEBUG_CHAN_MEMORY
if (ftdmchan->state == FTDM_CHANNEL_STATE_DIALING) {
- ftdm_assert(mprotect(ftdmchan, sizeof(*ftdmchan), PROT_READ|PROT_WRITE)==0, "Failed to mprotect");
+ ftdm_assert(mprotect(ftdmchan, sizeof(*ftdmchan), PROT_READ|PROT_WRITE) == 0, "Failed to mprotect");
}
#endif
- ftdm_channel_unlock(ftdmchan);
- return;
+ return FTDM_SUCCESS;
}
static FIO_CHANNEL_SEND_MSG_FUNCTION(ftdm_sangoma_isdn_send_msg)
span->get_span_sig_status = ftdm_sangoma_isdn_get_span_sig_status;
span->set_span_sig_status = ftdm_sangoma_isdn_set_span_sig_status;
span->state_map = &sangoma_isdn_state_map;
+ span->state_processor = ftdm_sangoma_isdn_process_state_change;
ftdm_set_flag(span, FTDM_SPAN_USE_CHAN_QUEUE);
ftdm_set_flag(span, FTDM_SPAN_USE_SIGNALS_QUEUE);
ftdm_set_flag(span, FTDM_SPAN_USE_PROCEED_STATE);
"PROGRESS_MEDIA", "ALARM_TRAP", "ALARM_CLEAR", \
"COLLECTED_DIGIT", "ADD_CALL", "RESTART", "SIGSTATUS_CHANGED", "COLLISION", "FACILITY", \
"TRACE", "TRACE_RAW", "INDICATION_COMPLETED", "INVALID"
-
/*! \brief Move from string to ftdm_signal_event_t and viceversa */
FTDM_STR2ENUM_P(ftdm_str2ftdm_signal_event, ftdm_signal_event2str, ftdm_signal_event_t)
typedef ftdm_status_t (*fio_span_poll_event_t) FIO_SPAN_POLL_EVENT_ARGS ;
typedef ftdm_status_t (*fio_span_next_event_t) FIO_SPAN_NEXT_EVENT_ARGS ;
typedef ftdm_status_t (*fio_channel_next_event_t) FIO_CHANNEL_NEXT_EVENT_ARGS ;
+
+/*! \brief Callback for signal delivery (FTDM_SIGEVENT_START and friends)
+ * \note This callback is provided by the user during ftdm_configure_span_signaling
+ *
+ * \note You must NOT do any blocking during this callback since this function is
+ * most likely called in an internal signaling thread that can potentially be
+ * shared for all the channels in a span and blocking will delay processing
+ * (sometimes even audio processing) for other channels
+ *
+ * \note Although some simple FreeTDM APIs can work (ie: ftdm_span_get_id etc), the
+ * use of any FreeTDM call API (ie ftdm_channel_call_answer) is discouraged
+ */
typedef ftdm_status_t (*fio_signal_cb_t) FIO_SIGNAL_CB_ARGS ;
+
typedef ftdm_status_t (*fio_event_cb_t) FIO_EVENT_CB_ARGS ;
typedef ftdm_status_t (*fio_configure_span_t) FIO_CONFIGURE_SPAN_ARGS ;
typedef ftdm_status_t (*fio_configure_t) FIO_CONFIGURE_ARGS ;
ftdm_span_stop_t stop;
ftdm_channel_sig_read_t sig_read;
ftdm_channel_sig_write_t sig_write;
- /* Private I/O data per span. Do not touch unless you are an I/O module */
- void *io_data;
+ ftdm_channel_state_processor_t state_processor; /*!< This guy is called whenever state processing is required */
+ void *io_data; /*!< Private I/O data per span. Do not touch unless you are an I/O module */
char *type;
char *dtmf_hangup;
size_t dtmf_hangup_len;
ftdm_state_map_t *state_map;
ftdm_caller_data_t default_caller_data;
- ftdm_queue_t *pendingchans;
- ftdm_queue_t *pendingsignals;
+ ftdm_queue_t *pendingchans; /*!< Channels pending of state processing */
+ ftdm_queue_t *pendingsignals; /*!< Signals pending from being delivered to the user */
struct ftdm_span *next;
};
const char *file;
const char *func;
int line;
- ftdm_channel_state_t state;
- ftdm_channel_state_t last_state;
- ftdm_time_t time;
- ftdm_time_t end_time;
+ ftdm_channel_state_t state; /*!< Current state (processed or not) */
+ ftdm_channel_state_t last_state; /*!< Previous state */
+ ftdm_time_t time; /*!< Time the state was set */
+ ftdm_time_t end_time; /*!< Time the state processing was completed */
} ftdm_state_history_entry_t;
typedef ftdm_status_t (*ftdm_channel_state_processor_t)(ftdm_channel_t *fchan);
-FT_DECLARE(ftdm_status_t) ftdm_channel_advance_states(ftdm_channel_t *fchan, ftdm_channel_state_processor_t processor);
+/*!
+ * \brief Process channel states by invoking the channel state processing routine
+ * it will keep calling the processing routine while the state status
+ * is FTDM_STATE_STATUS_NEW, it will not do anything otherwise
+ */
+FT_DECLARE(ftdm_status_t) ftdm_channel_advance_states(ftdm_channel_t *fchan);
+
FT_DECLARE(ftdm_status_t) _ftdm_channel_complete_state(const char *file, const char *function, int line, ftdm_channel_t *fchan);
#define ftdm_channel_complete_state(obj) _ftdm_channel_complete_state(__FILE__, __FUNCTION__, __LINE__, obj)
FT_DECLARE(int) ftdm_check_state_all(ftdm_span_t *span, ftdm_channel_state_t state);
};
typedef struct ftdm_state_map ftdm_state_map_t;
+/*!\brief Set the state for a channel (the channel must be locked when calling this function)
+ * \note Signaling modules should use ftdm_set_state macro instead
+ * \note If this function is called with the wait parameter set to a non-zero value, the recursivity
+ * of the channel lock must be == 1 because the channel will be unlocked/locked when waiting */
FT_DECLARE(ftdm_status_t) ftdm_channel_set_state(const char *file, const char *func, int line,
ftdm_channel_t *ftdmchan, ftdm_channel_state_t state, int wait);
-/*!\brief Set the state of a channel immediately and implicitly complete the previous state */
+/*!\brief Set the state of a channel immediately and implicitly complete the previous state if needed
+ * \note FTDM_SIGEVENT_INDICATION_COMPLETED will be sent if the state change
+ * is associated to some indication (ie FTDM_CHANNEL_INDICATE_PROCEED)
+ * \note The channel must be locked when calling this function
+ * */
FT_DECLARE(ftdm_status_t) _ftdm_set_state(const char *file, const char *func, int line,
ftdm_channel_t *fchan, ftdm_channel_state_t state);
#define ftdm_set_state(obj, s) _ftdm_set_state(__FILE__, __FUNCTION__, __LINE__, obj, s); \
/*!\brief This macro is deprecated, signaling modules should always lock the channel themselves anyways since they must
* process first the user pending state changes then set a new state before releasing the lock
+ * this macro is here for backwards compatibility, DO NOT USE IT in new code since it is *always* wrong to set
+ * a state in a signaling module without checking and processing the current state first (and for that you must lock the channel)
*/
#define ftdm_set_state_locked(obj, s) \
do { \
projects / thirdparty / freeswitch.git / commitdiff
