Sean Bright [Thu, 16 Feb 2017 14:38:06 +0000 (09:38 -0500)]
realtime: Fix LIKE escaping in SQL backends
The realtime framework allows for components to look up values using a
LIKE clause with similar syntax to SQL's. pbx_realtime uses this
functionality to search for pattern matching extensions that start with
an underscore (_).
When passing an underscore to SQL's LIKE clause, it will be interpreted
as a wildcard matching a single character and therefore needs to be
escaped. It is (for better or for worse) the responsibility of the
component that is querying realtime to escape it with a backslash before
passing it in. Some RDBMs support escape characters by default, but the
SQL92 standard explicitly says that there are no escape characters
unless they are specified with an ESCAPE clause, e.g.
SELECT * FROM table WHERE column LIKE '\_%' ESCAPE '\'
This patch instructs 3 backends - res_config_mysql, res_config_pgsql,
and res_config_sqlite3 - to use the ESCAPE clause where appropriate.
Looking through documentation and source tarballs, I was able to
determine that the ESCAPE clause is supported in:
MySQL 5.0.15 (released 2005-10-22 - earliest version available from
archives)
PostgreSQL 7.1 (released 2001-04-13)
SQLite 3.1.0 (released 2005-01-21)
The versions of the relevant libraries that we depend on to access MySQL
and PostgreSQL will not work on versions that old, and I've added an
explicit check in res_config_sqlite3 to only use the ESCAPE clause when
we have a sufficiently new version of SQLite3.
res_config_odbc already handles the escape characters appropriately, so
no changes were required there.
chan_unistim: fix char type to have consistent behavior on ARM
There is difference exists in behaviour of char type on x86 and ARM.
On x86 by default char variable type means signed char, but in ARM
unsigned char used. This make binary calculations and negative values
works wrong on ARM.
This patch change type of char variables used for store negative
values and binary calculations to signed char.
This patch fixes 2 original issues and more that those 2 exposed.
* When we send a NOTIFY, and the client either doesn't respond or
responds with a non OK, pjproject only calls our
pubsub_on_evsub_state callback, no others. Since
pubsub_on_evsub_state (which does the sub_tree cleanup) does not
expect to be called back without the other callbacks being called
first, it just returns leaving the sub_tree orphaned. Now
pubsub_on_evsub_state checks the event for PJSIP_EVENT_TSX_STATE
which is what pjproject will set to tell us that it was the
transaction that timed out or failed and not the subscription
itself timing our or being terminated by the client. If is
TSX_STATE, pubsub_on_evsub_state now does the proper cleanup
regardless of the state of the subscription.
* When a client renews a subscription, we don't update the
persisted subscription with the new expires timestamp. This causes
subscription_persistence_recreate to prune the subscription if/when
asterisk restarts. Now, pubsub_on_rx_refresh calls
subscription_persistence_update to apply the new expires timestamp.
This exposed other issues however...
* When creating a dialog from rdata (which sub_persistence_recreate
does from the packet buffer) there must NOT be a tag on the To
header (which there will be when a client refreshes a
subscription). If there is one, pjsip_dlg_create_uas will fail.
To address this, subscription_persistence_update now accepts a flag
that indicates that the original packet buffer must not be updated.
New subscribes don't set the flag and renews do. This makes sure
that when the rdata is recreated on asterisk startup, it's done
from the original subscribe packet which won't have the tag on To.
* When creating a dialog from rdata, we were setting the dialog's
remote (SUBSCRIBE) cseq to be the same as the local (NOTIFY) cseq.
When the client tried to resubscribe after a restart with the
correct cseq, we'd reject the request with an Invalid CSeq error.
* The acts of creating a dialog and evsub by themselves when
recreating a subscription does NOT restart pjproject's subscription
timer. The result was that even if we did correctly recreate the
subscription, we never removed it if the client happened to go away
or send a non-OK response to a NOTIFY. However, there is no
pjproject function exposed to just set the timer on an evsub that
wasn't created by an incoming subscribe request. To address this,
we create our own timer using ast_sip_schedule_task. This timer is
used only for re-establishing subscriptions after a restart.
An earlier approach was to add support for setting pjproject's
timer (via a pjproject patch) and while that patch is still included
here, we don't use that call at the moment.
While addressing these issues, additional debugging was added and
some existing messages made more useful. A few formatting changes
were also made to 'pjsip show scheduled tasks' to make displaying
the subscription timers a little more friendly.
Sean Bright [Wed, 15 Feb 2017 17:03:00 +0000 (12:03 -0500)]
res_rtp_asterisk: Use PJ_ICE_MAX_CAND instead of hard-coding 16
pjsip limits the total number of ICE candidates to PJ_ICE_MAX_CAND,
which is a compile-time constant. Instead of hard-coding 16 when we
enumerate local interfaces, use PJ_ICE_MAX_CAND so that we can
potentially collect more interfaces if the compile time options are
changed.
Tzafrir Cohen [Sat, 21 Jan 2017 05:59:15 +0000 (07:59 +0200)]
tcptls: use TLS_client_method with OpenSSL 1.1
OpenSSL 1.1 introduced TLS_client_method() and deprecated the previous
version-specific methods (such as TLSv1_client_method(). Other than
being simpler to use and more correct (gain support for TLS newer that
TLS1, in our case), the older ones produce a deprecation warning that
fails the build in dev-mode.
rrittgarn [Wed, 25 Jan 2017 22:25:21 +0000 (16:25 -0600)]
app_voicemail: VoiceMailPlayMsg did not play database stored messages
When attempting to use VoiceMailPlayMsg with a realtime data backend
the message is located, but never retrieved. This patch adds the
required RETRIEVE and DISPOSE calls that will fetch the message from
the database (and IMAP storage as well for that matter).
Sean Bright [Tue, 14 Feb 2017 14:12:31 +0000 (09:12 -0500)]
app_record: Add option to prevent silence from being truncated
When using Record() with the silence detection feature, the stream is
written out to the given file. However, if only 'silence' is detected,
this file is then truncated to the first second of the recording.
This patch adds the 'u' option to Record() to override that behavior.
ASTERISK-18286 #close
Reported by: var
Patches:
app_record-1.8.7.1.diff (license #6184) patch uploaded by var
Sean Bright [Sat, 11 Feb 2017 15:57:03 +0000 (10:57 -0500)]
cli: Fix various CLI documentation and completion issues
* app_minivm: Use built-in completion facilities to complete optional
arguments.
* app_voicemail: Use built-in completion facilities to complete
optional arguments.
* app_confbridge: Add missing colons after 'Usage' text.
* chan_alsa: Use built-in completion facilities to complete optional
arguments.
* chan_sip: Use built-in completion facilities to complete optional
arguments. Add completions for 'load' for 'sip show user', 'sip show
peer', and 'sip qualify peer.'
* chan_skinny: Correct and extend completions for 'skinny reset' and
'skinny show line.'
* func_odbc: Correct completions for 'odbc read' and 'odbc write'
* main/asterisk: Correct and extend completions for 'core show file
version.'
* main/astmm: Use built-in completion facilities to complete arguments
for 'memory' commands.
* main/bridge: Correct completions for 'bridge kick.'
* main/ccss: Use built-in completion facilities to complete arguments
for 'cc cancel' command.
* main/cli: Add 'all' completion for 'channel request hangup.' Correct
completions for 'core set debug channel.' Correct completions for 'core
show calls.'
* main/pbx_app: Remove redundant completions for 'core show
applications.'
* main/pbx_hangup_handler: Remove unused completions for 'core show
hanguphandlers all.'
* res_sorcery_memory_cache: Add completion for 'reload' argument of
'sorcery memory cache stale' and properly implement.
Norbert Varga [Fri, 13 Jan 2017 17:21:36 +0000 (18:21 +0100)]
chan_pjsip: Multidomain endpoint finding on call
When PJSIP tries to call an endpoint with a domain (e.g. 1000@test.com),
the user part is stripped down as it would be a trunk with a specified user,
and only the host part is called as a PJSIP endpoint and can't be found.
This is not correct in the case of a multidomain SIP account, so the stripping
after the @ sign is done only if the whole endpoint (in multidomain case
1000@test.com) can't be found.
Joshua Colp [Mon, 13 Feb 2017 11:05:51 +0000 (11:05 +0000)]
channel: Protect flags in ast_waitfor_nandfds operation.
The ast_waitfor_nandfds operation will manipulate the flags
of channels passed in. This was previously done without
the channel lock being held. This could result in incorrect
values existing for the flags if another thread manipulated
the flags at the same time.
This change locks the channel during flag manipulation.
Sean Bright [Thu, 9 Feb 2017 16:01:22 +0000 (11:01 -0500)]
manager: Restore Originate failure behavior from Asterisk 11
In Asterisk 11, if the 'Originate' AMI command failed to connect the provided
Channel while in extension mode, a 'failed' extension would be looked up and
run. This was, I believe, unintentionally removed in 51b6c49. This patch
restores that behavior.
This also adds an enum for the various 'synchronous' modes in an attempt to
make them meaningful.
Richard Mudgett [Wed, 8 Feb 2017 20:27:18 +0000 (14:27 -0600)]
core: Cleanup some channel snapshot staging anomalies.
We shouldn't unlock the channel after starting a snapshot staging because
another thread may interfere and do its own snapshot staging.
* app_dial.c:dial_exec_full() made hold the channel lock while setting up
the outgoing channel staging. Made hold the channel lock after the called
party answers while updating the caller channel staging.
* chan_sip.c:sip_new() completed the channel staging on off-nominal exit.
Also we need to use ast_hangup() instead of ast_channel_unref() at that
location.
* channel.c:__ast_channel_alloc_ap() added a comment about not needing to
complete the channel snapshot staging on off-nominal exit paths.
* rtp_engine.c:ast_rtp_instance_set_stats_vars() made hold the channel
locks while staging the channels for the stats channel variables.
The change in question was intended to prevent the need to reload in
order to update qualifies on contacts when an AOR changes. However, this
ended up causing a deadlock instead.
Joshua Colp [Tue, 7 Feb 2017 18:01:03 +0000 (18:01 +0000)]
srv: Fix crash when ast_srv_lookup is used and 0 records are returned.
When performing an SRV lookup using the ast_srv_lookup function it
did not properly handle the situation where 0 records are returned.
If this happened it would wrongly assume that at least one record
was present.
This change fixes the code so it will exit early if an error occurs
or if 0 records are returned.
ASTERISK-26772
patches:
srv_lookup.patch submitted by nappsoft (license 6822)
Joshua Colp [Mon, 6 Feb 2017 17:40:45 +0000 (17:40 +0000)]
res_stasis_device_state: Protect the adding/removing of subscriptions.
The adding and removing of device state subscriptions did not protect
fully against simultaneous manipulation. In particular the subscribe
case allowed a small window where two subscriptions could be added for
the same device state instead of just one.
This change makes the code hold the subscriptions lock for the entirety
of each operation to ensure that two are not occurring at the same time.
Mark Michelson [Wed, 1 Feb 2017 19:54:50 +0000 (13:54 -0600)]
Update qualifies when AOR configuration changes.
Prior to this change, qualifies would only update in the following
cases:
* A reload of res_pjsip.so was issued.
* A dynamic contact was re-registered after its AOR's qualify_frequency
had been changed
This does not work well if you are using realtime for your AORs. You can
update your database to have a new qualify_frequency, but the permanent
contacts on that AOR will not have their qualifies updated. And the
dynamic contacts on that AOR will not have their qualifies updated until
the next registration, which could be a long time.
This change seeks to fix this problem by making it so that whenever AOR
configuration is applied, the contacts pertaining to that AOR have their
qualifies updated.
Additions from this patch:
* AOR sorcery objects now have an apply handler that calls into a newly
added function in the OPTIONS code. This causes all contacts
associated with that AOR to re-schedule qualifies.
* When it is time to qualify a contact, the OPTIONS code checks to see
if the AOR can still be retrieved. If not, then qualification is
canceled on the contact.
Alterations from this patch:
* The registrar code no longer updates contact's qualify_frequence and
qualify_timeout. There is no point to this since those values already
get updated when the AOR changes.
* Reloading res_pjsip.so no longer calls the OPTIONS initialization
function. Reloading res_pjsip.so results in re-loading AORs, which
results in re-scheduling qualifies.
Richard Mudgett [Wed, 1 Feb 2017 00:28:15 +0000 (18:28 -0600)]
channel.c: Fix unbalanced read queue deadlocking local channels.
Using the timerfd timing module can cause channel freezing, lingering, or
deadlock issues. The problem is because this is the only timing module
that uses an associated alert-pipe. When the alert-pipe becomes
unbalanced with respect to the number of frames in the read queue bad
things can happen. If the alert-pipe has fewer alerts queued than the
read queue then nothing might wake up the thread to handle received frames
from the channel driver. For local channels this is the only way to wake
up the thread to handle received frames. Being unbalanced in the other
direction is less of an issue as it will cause unnecessary reads into the
channel driver.
ASTERISK-26716 is an example of this deadlock which was indirectly fixed
by the change that found the need for this patch.
* In channel.c:__ast_queue_frame(): Adding frame lists to the read queue
did not add the same number of alerts to the alert-pipe. Correspondingly,
when there is an exceptionally long queue event, any removed frames did
not also remove the corresponding number of alerts from the alert-pipe.
Richard Mudgett [Tue, 31 Jan 2017 22:38:49 +0000 (16:38 -0600)]
res_agi: Prevent an AGI from eating frames it should not. (Re-do)
A dialplan intercept routine is equivalent to an interrupt routine. As
such, the routine must be done quickly and you do not have access to the
media stream. These restrictions are necessary because the media stream
is the responsibility of some other code and interfering with or delaying
that processing is bad. A possible future dialplan processing
architecture change may allow the interception routine to run in a
different thread from the main thread handling the media and remove the
execution time restriction.
* Made res_agi.c:run_agi() running an AGI in an interception routine run
in DeadAGI mode. No touchy channel frames.
Richard Mudgett [Tue, 31 Jan 2017 22:32:18 +0000 (16:32 -0600)]
Frame deferral: Revert API refactoring.
There are several issues with deferring frames that are caused by the
refactoring.
1) The code deferring frames mishandles adding a deferred frame to the
deferred queue. As a result the deferred queue can only be one frame
long.
2) Deferrable frames can come directly from the channel driver as well as
the read queue. These frames need to be added to the deferred queue.
3) Whoever is deferring frames is really only doing the __ast_read() to
collect deferred frames and doesn't care about the returned frames except
to detect a hangup event. When frame deferral is completed we must make
the normal frame processing see the hangup as a frame anyway. As such,
there is no need to have varying hangup frame deferral methods. We also
need to be aware of the AST_SOFTHANGUP_ASYNCGOTO hangup that isn't real.
That fake hangup is to cause the PBX thread to break out of loops to go
execute a new dialplan location.
4) To properly deal with deferrable frames from the channel driver as
pointed out by (2) above, means that it is possible to process a dialplan
interception routine while frames are deferred because of the
AST_CONTROL_READ_ACTION control frame. Deferring frames is not
implemented as a re-entrant operation so you could have the unsupported
case of two sections of code thinking they have control of the media
stream.
A worse problem is because of the bad implementation of the AMI PlayDTMF
action. It can cause two threads to be deferring frames on the same
channel at the same time. (ASTERISK_25940)
* Rather than fix all these problems simply revert the API refactoring as
there is going to be only autoservice and safe_sleep deferring frames
anyway.
Joshua Colp [Tue, 31 Jan 2017 17:17:50 +0000 (17:17 +0000)]
res_pjsip: Handle invocation of callback on outgoing request when error occurs.
There are some error cases in PJSIP when sending a request that will
result in the callback for the request being invoked. The code did not
handle this case and assumed on every error case that the callback was not
invoked.
The code has been changed to check whether the callback has been invoked
and if so to absorb the error and treat it as a success.
Sean Bright [Mon, 30 Jan 2017 15:02:14 +0000 (10:02 -0500)]
res_rtp_asterisk: Swap byte-order when sending signed linear
Before Asterisk 13, signed linear was converted into network byte order by a
smoother before being sent over the network. We restore this behavior by
forcing the creation of a smoother when slinear is in use and setting the
appropriate flags so that the byte order conversion is always done.
George Joseph [Wed, 25 Jan 2017 12:50:43 +0000 (05:50 -0700)]
debug_utilities: Add ast_logescalator
The escalator works by creating a set of startup commands in cli.conf
that set up logger channels and issue the debug commands for the
subsystems specified. If asterisk is running when it is executed,
the same commands will be issued to the running instance. The original
cli.conf is saved before any changes are made and can be restored by
executing '$prog --reset'.
The log output will be stored in...
$astlogdir/message.$uniqueid
$astlogdir/debug.$uniqueid
$astlogdir/dtmf.$uniqueid
$astlogdir/fax.$uniqueid
$astlogdir/security.$uniqueid
$astlogdir/pjsip_history.$uniqueid
$astlogdir/sip_history.$uniqueid
Some minor tweaks were made to chan_sip, and res_pjsip_history
so their history output could be send to a log channel as packets
are captured.
A minor tweak was also made to manager so events are output to verbose
when "manager set debug on" is issued.
kkm [Wed, 25 Jan 2017 01:51:07 +0000 (17:51 -0800)]
make_build_h: handle backslashes in external strings
LikewiseOpen creates user names with a backslash in them. A gentle
massage with sed(1) allows such strings to be inserted into build.h
properly quoted. I am also adding the same for host name and other
strings used in the script that are more or less user-controlled.
kkm [Wed, 18 Jan 2017 02:46:43 +0000 (18:46 -0800)]
ast_careful_fwrite to support EPIPE gracefully
When a reading end of the network socket is closed by an AMI manager,
the EPIPE is signaled when writing to our end, resulting in the
spurious log error message
ast_careful_fwrite: fwrite() returned error: Broken pipe
Previously EPIPE was handled in ast_carefulwrite() a few lines above,
but not in this function.
kkm [Wed, 25 Jan 2017 04:31:38 +0000 (20:31 -0800)]
app_queue: Fix queues randomly disappearing on reload
With 500+ queues and a reload every minute, a random queue disappears
upon reload. The cause is mususe of the 'dead' flag. Namely, all queues
were marked dead up front, and then "resurrected" by dropping this flag
for those found in the configuration. But a queue marked dead can be
removed also when control leaves the app entry point on a PBX thread.
With this change, the queue is marked only not found, and at the end of
reload only the queues that are still not found are actually marked as
dead, so the dead flag is never reset, and set only on positively dead
queues.
Mark Michelson [Wed, 25 Jan 2017 21:26:53 +0000 (15:26 -0600)]
Add reload options to CLI/AMI stale object commands.
Marking an object as stale in a memory cache is supposed to prime the
cache so that the next time the item is retrieved, the stale item is
deleted from the cache and a background task is run to re-populate the
cache with a fresh version of the object.
The problem is, there are some object types out there for which there is
no natural reason that they would be retrieved from the backend with any
regularity. Outbound PJSIP registrations are a good example of this. At
startup, they are read, and an object-specific state is created that
refers to the initially-retrieved object for all time.
Adding the "reload" option to the CLI/AMI commands gives the cache the
opportunity to manually re-retrieve the object from the backend, both
storing the new object in the cache and applying the new object's
configuration to the module that uses that object.
Joshua Colp [Tue, 24 Jan 2017 21:39:39 +0000 (21:39 +0000)]
res_pjsip_endpoint_identifier_ip: Ensure error defaults to 0.
When configuring a match using a netmask the error variable was
not defaulting to 0. For some people this would cause the code
to think an error occurred when adding the match when in reality
it added perfectly fine.
Richard Mudgett [Tue, 10 Jan 2017 19:11:20 +0000 (13:11 -0600)]
main/app.c: Memory corruption from early format destruction.
* make_silence() created a malloced silence slin frame without adding a
slin format ref. When the frame is destroyed it will unref the slin
format that never had a ref added. Memory corruption is expected to
follow.
* Simplified and fixed counting the number of samples in a frame list for
make_silence().
* Eliminated an unnecessary RAII_VAR associated with the make_silence()
frame.
George Joseph [Thu, 19 Jan 2017 15:05:36 +0000 (08:05 -0700)]
ari: Implement 'debug all' and request/response logging
The 'ari set debug' command has been enhanced to accept 'all' as an
application name. This allows dumping of all apps even if an app
hasn't registered yet. To accomplish this, a new global_debug global
variable was added to res/stasis/app.c and new APIs were added to
set and query the value.
'ari set debug' now displays requests and responses as well as events.
This required refactoring the existing debug code.
* The implementation for 'ari set debug' was moved from stasis/cli.{c,h}
to ari/cli.{c,h}, and stasis/cli.{c,h} were deleted.
* In order to print the body of incoming requests even if a request
failed, the consumption of the body was moved from the ari stubs
to ast_ari_callback in res_ari.c and the moustache templates were
then regenerated. The body is now passed to ast_ari_invoke and then
on to the handlers. This results in code savings since that template
was inserted multiple times into all the stubs.
An additional change was made to the ao2_str_container implementation
to add partial key searching and a sort function. The existing cli
code assumed it was already there when it wasn't so the tab completion
was never working.
Richard Mudgett [Sat, 21 Jan 2017 03:13:34 +0000 (21:13 -0600)]
PJPROJECT logging: Fix detection of max supported log level.
The mechanism used for detecting the maximum log level compiled into the
linked pjproject did not work. The API call simply stores the requested
level into an integer and does no range checking. Asterisk was assuming
that there was range checking and limited the new value to the allowable
range. To get the actual maximum log level compiled into the linked
pjproject we need to get and save off the initial set log level from
pjproject. This is the maximum log level supported.
* Get and save off the initial log level setting before altering it to the
desired level on startup. This has to be done by a macro rather than
calling a core function to avoid incorrectly linking pjproject.
* Split the initial log level warning messages to warn if the linked
pjproject cannot support the requested startup level and if it is too low
to get the pjproject buildopts for "pjproject show buildopts".
* Adjust the CLI "pjproject set log level" to check the saved max log
level and to generate normal output messages instead of a warning message.
projects / thirdparty / asterisk.git / log
