url_rewrite_extras and store_id_extras patch fixes
Fixes to patch "Add url_rewrite_extras and store_id_extras for redirector and store_id helpers",r13308:
- Fix cf_gen.cc:gen_conf(..) function to not escape quotes before write to
conf file
- The Format::Format name is used to inform the user about parsing problems.
Fix the names of related objects for the new redirecor_extras and
store_id_extras directives.
- cf.data.pre: The NAME tag take as argument only the name of directive. Fix
the new redirecor_extras and store_id_extras related tags.
Add url_rewrite_extras and store_id_extras for redirector and store_id helpers
The url_rewrite_extras/store_id_extras is a "quoted string" with logformat
%macro support. It is used to modify the request line for redirector and
storeId helpers.
The url rewrite and store_id helpers request format now is:
url [<SP> extras]
and the default value for extras is:
"%>a/%>A %un %>rm myip=%la myport=%lp"
Example usage:
url_rewrite_extras "Note1=%{Note1}note Note2=%{Note2}note"
Alex Rousskov [Sat, 8 Mar 2014 17:28:23 +0000 (10:28 -0700)]
Avoid assertions on Range requests that trigger Squid-generated errors.
Added HttpRequest::ignoreRange() to encapsulate range ignoring logic.
Currently the new method only contains the code common among all callers. More
work is needed to check whether further caller homogenization is possible.
Documented that ClientSocketContext::getNextRangeOffset() may sometimes be
called before it is ready to do its job.
Amos Jeffries [Fri, 7 Mar 2014 11:18:03 +0000 (04:18 -0700)]
Protect MemBlob::append() against raw-space writes
There is no guarantee that the 'unused' area of MemBlob is actually
unused. For example if a read buffer was being filled into the
rawSpace() of a SBuf or MemBlob it will overlap with this empty area
until a read call updates the related size state in MemBlob/SBuf.
For these cases we must use memmove() which guarantees no buffer
corruption will take place on memory overlaps.
Amos Jeffries [Thu, 6 Mar 2014 03:55:41 +0000 (20:55 -0700)]
Copyright: Relicense helpers by Treehouse Networks Ltd.
Update the license on helper code designed and authored by myself using
the BSD 2-clause license. This makes the example helper code and license
more legally acceptible for use as a basis of proprietary helpers while
remaining compatible with GPL for distribution with Squid.
Amos Jeffries [Wed, 5 Mar 2014 12:08:54 +0000 (01:08 +1300)]
Better fix for CMSG definitions
It turns out autoconf versions are not consistent with $ symbol escaping
which can cause incorrect definitions. Revert to AC_CHECK_TYPE instead.
Its a bit more verbose in configure.ac but works more often than not.
Amos Jeffries [Wed, 5 Mar 2014 06:32:34 +0000 (19:32 +1300)]
Portability: define CMSG related structures individually
Some OS provide the CMSG related definitions and others only partially
define them. Sometimes (Windows particularly) this varies between build
environments.
Checking for each symbol separately and providing only those needed
avoids problems we have been having with missing or redefined symbols
on Windows and elsewhere.
Amos Jeffries [Tue, 4 Mar 2014 10:33:08 +0000 (23:33 +1300)]
Fix helper ID number assignment
Since helpers are now dynamically started the old method of allocating
an ID number based on the current start sequence can result in many
helpers being assigned overlapping ID numbers.
Use InstanceID template instead to assure a unique incremental ID is
assigned to each helper no matter when it is started.
Alex Rousskov [Fri, 21 Feb 2014 16:14:05 +0000 (09:14 -0700)]
Migrated RegisteredRunners to a multi-action interface.
Old generic two-action RegisteredRunners were good for handling paired
create/destroy events, but not all main.cc events fit that model well. In
fact, even the old runners implemented the destruction action for one event
only (rrAfterConfig); all other runners implemented a single action.
The adjusted API better supports runners that are interested in any number
of the supported events. It also allows a single runner object to handle
multiple events, which simplifies current code and may help with better
[re]configuration handling in the future.
Added startShutdown() and finishShutdown() events. The former will be needed
for authentication module shutdown and more polished shutdown initiation code
in general (patch pending). The latter is needed for final cleanup code that
previously ran as the destruction action for rrAfterConfig. finishShutdown()
also destroys all runners.
Note that the master process in SMP mode does not run startShutdown because
that process lacks the main loop and startShutdown() promises at least one
main loop iteration (to help with clean connections closures, for example).
Added syncConfig() event that will be needed for the standby pool
implementation (patch pending) and future code that reacts to Squid
configuration changes caused by reconfiguration.
"after config" event is now called "use config" to better match verb+noun or
action+object naming scheme.
Amos Jeffries [Fri, 21 Feb 2014 10:46:19 +0000 (03:46 -0700)]
Cleanup: un-wrap C++ header includes
Coding guideline is now that standard C++ headers are not to be
wrapped in HAVE_ macros.
* Remove HAVE_ macros for currently wrapped C++ headers.
Includes removing autoconf checks.
* Replace C includes with C++ includes where possible
Also, <cstdio> / <stdio.h> has issues on 64-bit systems and a
portable fixed version is provided by libcompat via squid.h
It should not be included anywhere in the Squid sources.
Amos Jeffries [Thu, 20 Feb 2014 13:03:07 +0000 (06:03 -0700)]
squidclient: --ping mode module support
Module support:
Update squidclient support modules with different logics
and configuration option sets as a basis for multiple
protocol support.
A mechanism is added to allow each module to have its own
command line option set. Any option unknown to the current
module handler drops back to the main loop for processing.
--ping mode module:
Break the existing code "ping mode" operations and command
line processing out from the main squidclient.cc into Ping.*
Ping-specific short command line options are now only parsed
after a mode flag (--ping) is presented. This frees up the
-g and -I options for use by other non-ping modules in future.
Also, shuffle squidclient code into its own directory
tools/squidclient/ to keep the tool code files clearly
identifiable now that they are multiplying.
1) The dynamic_cert_mem_cache_size does not change on reconfigure
2) When dynamic_cert_mem_cache_size of http_port set to 0 then:
a) The dynamic certs cache is grow unlimited.
This patch just disables certificates caching when this option set to 0.
b) Huge amount of memory appeared as free cache memory in "Cached ssl
certificates statistic" page of cache manager.
This problem caused because of a signed to unsigned int conversion.
Amos Jeffries [Sun, 16 Feb 2014 05:15:45 +0000 (22:15 -0700)]
squidclient: support verbosity levels
This makes the -v option repeatable. By default no debug is displayed.
Each time -v is repeated the level of debug message verbosity is raised.
Three levels of verbosity are currently defined:
0 - no output except ERROR messages.
1 - display HTTP request sent
2 - display actions taken connecting to server
Amos Jeffries [Thu, 13 Feb 2014 07:02:35 +0000 (20:02 +1300)]
Bug 4001: remove use of strsep()
The strsep() function is not defined by POSIX. Additionally
auto-tools has been having some obscure issues detecting
or linking the provided implementation into libcompat on
Windows and Solaris respectively. Which are the two known
OS requiring it.
Investigation of its use in Squid revealed that it can be
replaced with strcspan() which is both portable and more
efficient since it also removes the need for several
strdup()/free() operations used to protect Squid from
strsep() memory fiddling.
Bug 3969: user credentials cache lookup for Digest authentication broken
Changes to the username credentials cache were made in Basic auth but
the matching changes were not duplicated to Digest auth. Since the
lookup is identical move it to generic Auth::Config.
Amos Jeffries [Sat, 8 Feb 2014 12:33:31 +0000 (05:33 -0700)]
Move compat/unsafe.h protections from libcompat to source maintenance
It is sufficient to run a code scan from source-maintenance.sh for the
unsafe functions being used in Squid-specific code instead of
hard-coding compiler breakage on users.
This also "fixes" reporting of errors when cstdio pulls in use of the
unsafe functions by stdlib.
Alex Rousskov [Wed, 5 Feb 2014 18:04:47 +0000 (19:04 +0100)]
Fix keepalive handling for non-ranged requests.
Internal keepalive flag was ignored by a mismatched interface between ClientSocketContext::socketState
and writeComplete in the case of non-ranged requests.
Amos Jeffries [Sun, 2 Feb 2014 01:24:53 +0000 (18:24 -0700)]
Fix peerSelectDnsResults() IP address cycling
The local ip variable is the index of the IP address to be used.
Loop counter n is only used to prevent cycling indefinitely and should
not be used to access the array indexes.
Patch written by 'dim [1]' contributor to FreeBSD and imported to Squid
under FreeBSD license. see
http://svnweb.freebsd.org/ports/head/www/squid33/files/patch-include__Array.h
Amos Jeffries [Fri, 24 Jan 2014 01:57:15 +0000 (18:57 -0700)]
Enable -n command line option for non-Windows Squid builds
This command line option is used on Windows to name the service instance
of Squid being run and/or managed. At this point it still only has
useful effect on Windows, but can now be used by components on other
systems as well.
Show the running instance service name in cacehmgr and -v output.
Also remove _WIN_SQUID_DEFAULT_SERVICE_NAME macro which duplicated the
APP_SHORTNAME macro. This changes the Windows service name from Squid to
squid (lower case) on future Squid-3 for Windows.
When running Squid in SMP mode, the 'client_list' command cannot be used as the
coordinator doesn't call clientdbInit(), and thus doesn't have the client_list
action registered.
This patch uses RegisteredRunner to initialize clientdb and register the
'client_list' command
Doug Dixon [Mon, 20 Jan 2014 10:43:03 +0000 (23:43 +1300)]
Bug 1202 part 1: documentation and debugs for refresh_pattern algorithms
This contains only the code documentation and debugs changes as relevant
to the current code. Some which are only relevant to the new code from
that bug patch have been omitted.
It omits all logic changes and function renaming changes.