This patch:
- converts type of the Token::[width|precision] members from "unsigned int" to "int"
- renames the Token::[width|precision] members to Token::[widthMin/widthMax]
- removes unneeded typecastings
Alex Rousskov [Wed, 12 Oct 2011 20:56:41 +0000 (14:56 -0600)]
Allow non-shared memory caching when there are no cache_dirs.
Before this change, we destroyed unused/idle StoreEntries if nobody was voting
to keep them in store_table. That blocked non-shared memory cache from getting
new entries if there were no cache_dirs to vote for them, which is wrong. The
new code keeps unused/idle StoreEntries in store_table if nobody objects.
Alex Rousskov [Mon, 10 Oct 2011 14:39:00 +0000 (08:39 -0600)]
Fixed typos in the host_verify_strict description.
Frankly, the description is likely to still make little sense to
uninitiated because we do not explain what is "Host vs IP validation"
and what the "additional strict validation comparisons" are. There was
an attempt to explain the latter, but I think it failed. Perhaps there
are more typos that hide the intended meaning?
Amos Jeffries [Sun, 9 Oct 2011 05:44:22 +0000 (23:44 -0600)]
Add directive dns_v4_first to make IPv4 connections before IPv6 is tried.
Default off, to prefer the faster protocol.
The use-case for this is networks which are IPv6-enabled but stuck
behind slow tunnels and whose upstream is not supporting full transit
services over IP.
Properly parse HTTP list headers with embedded 8-bit characters
MSIE and maybe other browsers sometimes sends 8-bit high characters
in HTTP headers (and URLs). This was mistakenly read as CTL characters
on platforms with signed char type (i.e. x86 etc).
One visible effect of this was that HTTP Digest authentication failed
in MSIE when following a link with embedded 8-bit or UTF-8 characters.
Currently we are locking every file going to be accessed by CertificateDB code
even if it is not realy needed, because of a more general lock.
This patch:
- Replace the old FileLocker class with the pair Lock/Locker classes
- Remove most of the locks in CertificateDB with only two locks one
for main database locking and one lock for the file contain the
current serial number.
Bug 3349: Bad support for adaptation service URIs with '='
Currently using URIs which include "=" is not supported by
ecap_service/icap_service configuration parameters. Also the squid.conf
documentation about ecap_service is outdated.
This patch
- Fixes the [e|i]cap_service line parser to allow URIs with "="
- Changes the [e|i]cap_service configuration parameter to use the following syntax:
ecap_service id vectoring_point uri [name=value ...]
- Check for duplicated options
- Fixes the related documentation
- Also the older [e|i]cap_service syntax forms are supported:
ecap_service id vectoring_point [1|0] uri
ecap_service id vectoring_point [name=value ...] uri
- The "uri" options is not documented but supported.
Alex Rousskov [Thu, 6 Oct 2011 16:41:46 +0000 (10:41 -0600)]
Polished SMP caching code, primarily to stay out of the way in non-SMP mode.
Do not start useless diskers. Do not assume Rock cache_dirs are present.
Do not require IpcIo DiskIO module to build Rock store.
Check IPC I/O pages limits in Rock store only when using a disker.
Warn about Rock cache_dir disk space waste.
Warn if shared memory cache is enabled in non-SMP mode.
Fake shared memory segments if needed (e.g., we are using Rock cache_dirs with
no POSIX shared memory support) and possible (e.g., no SMP).
Alex Rousskov [Thu, 6 Oct 2011 16:38:03 +0000 (10:38 -0600)]
Added max-swap-rate=swaps/sec option to Rock cache_dir.
The option limits the rate of Rock disk access to smooth out OS disk commit
activity and to avoid blocking Rock diskers (or even other processes) on I/O.
Should be used when swap demand exceeds disk performance limits but the
underlying file system does not slow down incoming I/Os until the situation
gets out of control.
Bug 3190: Large HTTP POST stuck after early ICAP 400 error response
When an ICAP REQMOD service responds with an error to
(or the REQMOD transaction aborts while processing) a large HTTP
request, the HTTP request may get stuck because the request body
buffer gets full and nobody consumes the no-longer-needed content.
The ICAP code quits but leaves the body buffer intact in case the
client-side code wants to bypass the error. After that, nobody consumes
the request body because the buggy client side does not inform the body
pipe that there will be no other consumers, which would have triggered
a noteBodyConsumerAborted() callback and enable auto-consumption or closed
the client connection.
Remove entryLimitAllowed() TODO from RockSwapDirRr::run().
RockSwapDirRr::run() calls sd->entryLimitAllowed() when sd does not
have a map yet, which causes entryLimitAllowed() to use zero for the
lower limit. But that OK so we can remove the TODO.
In some configurations Rock cache_dir may hit the maximum entry limit.
In that case a significant disk space may be wasted. Before recent
SMP-related changes, there was code to warn the user about such
situations. The patch resurrects that code, adjusting it as needed to
match the current realities.
Check IPC I/O pages limits in Rock store only when using a disker.
Shared memory pages are used only when Rock is running with diskers.
Otherwise, Blocking IO is used and there is no need to check for IPC
I/O pages limits.
Before the change, a disker process was started for each configured
cache_dir. But not all cache_dirs need a disker: only Rock store
requires one when running in SMP mode. There was some existing code
to avoid starting useless diskers (see SwapDir::needsDiskStrand and
Config.cacheSwap.n_strands) but it was not complete. The patch fixes
this issue.
Add base class for registered runners that work with shared memory.
Shared memory runners do two main things:
* create shared memory segments
* open shared memory segments created earlier
Each runner has to decide whether it needs to create and/or open
shared memory segments. Common runners share (some) logic in making
this decisions. The purpose of this patch is to implement this logic
in a single place to allow easy reuse and avoid duplication.
"Fake" shared memory segments are enabled #if a system does not
support POSIX shared memory. Such segments use regular new to
allocate memory, it is not shared among multiple processes. The
purpose of the change is to allow code that uses Ipc::Mem::Segment to
run in non-SMP mode (that is when only a single process, except for
master, is running) on systems without POSIX shared memory support.
Note: running SHM-using features in SMP mode still requires POSIX
shared memory support, the patch does not change that.
Alex Rousskov [Mon, 3 Oct 2011 20:19:24 +0000 (14:19 -0600)]
Use atomic int for millisecond-based Balance instead of atomic int64_t.
Atomic int64_t may cause "undefined reference to __sync_fetch_and_add_8"
linking errors, due to GCC bugs. Those errors can be fixed by building with
CXXFLAGS='-march=i586' but there is no reason to introduce that complexity
as 2^31 milliseconds is 35791 hours which is still plenty for our purposes.
While the server writes the response to Store, the client side may
synchronously abort the entry. This happens, for example, when the
server receives a 304 response and handleIMSReply calls
sendClientOldEntry, which calls storeUnregister with our entry,
resulting in CheckQuickAbort.
Once server store write returns, if the server is done, it calls
FwdState::completed(). At that time, the server does not know that (and
should not care whether) the entry was aborted. Thus, we need to handle
aborted entries in FwdState::completed.
Optimized HttpHdrCc::packInto and clarified its behaviour in comments
Negative values to Cache-Control directives are now explicitly discarded
Clarified documentation
Explicitly check that integer-carrying Cache-control directives are positive.
Rename MAX_STALE_ALWAYS to MAX_STALE_ANY and changed its representation to very large positive integer.
Implemented internal HttpHdrCc::setValue call.
Removed unnecessarily explicitly-masked StringArea default constructor.
avoided temporary in HttpHdrCc name-to-id lookup
Started implementing have/set/get/clear combo for all members of HttpHdrCc
Fixed zero-pointer-dereference in http.cc
Remove SwapDir::reconfigure() arguments since they are not used.
Before the change, SwapDir::reconfigure() took index and path
arguments, but none of them was actually used: neither index nor path
can be changed during reconfigure. And both index and path are
available as SwapDir members so there is no reason to have these
arguments.
Ignore and warn about attempts to reconfigure static Rock store options.
Some Rock store options cannot be changed dynamically: path, size, and
max-size. Before the change, there were no checks during reconfigure
to prevent changing these options. This may lead to Rock cache
corruption and other bugs. The patch adds necessary checks to Rock
store code. If user tries to change an option that cannot be updated
dynamically, a warning is reported and the value is left unchanged.
Bug fix: "(ssl_crtd): Cannot add certificate to db" when updating expired cert
When ssl_crtd helper needs to add a fresh certificate to the database but
finds an expired certificate already stored, ssl_crtd deletes the expired
certificate file from disk before adding the fresh one. However, the addition
still fails because the expired certificate was not removed from database
indexes.
This fix:
- Adds code to update database indexes upon deletion of a row.
- Polishes certificates deletion code to avoid duplication.
TODO: Report failure details to Squid and make certificate-specific failures
not fatal for the ssl_crtd helper.
Alex Rousskov [Wed, 21 Sep 2011 18:05:55 +0000 (12:05 -0600)]
Temporary fix: Avoid killing Coordinator with unregistered cache mgr actions
that cause isOpen() assertions.
If a worker forwards a cache manager request to Coordinator and Coordinator
does not have that action registered, CacheManager::createRequestedAction()
throws (as it should) and Mgr::Request cleanup asserts when its half-baked
connection tries to close a not-yet-imported socket descriptor.
This workaround catches the exception, reports it, and manually closes the
socket descriptor. It also prevents an ACK response from being sent to the
worker, which triggers a worker timeout.
Mid-term TODO: Coordinator should register all actions that are known to kids.
Should Coordinator respond with an error instead of relying on a timeout?
Long-term TODO: Consider an API where cache manager responses can be
aggregated and formatted by Coordinator without knowing action-specific
details. After all, there are not so many types of action information (size,
count, rate, etc.) and most actions have simple reporting formats. Currently,
it is awkward to guarantee that Coordinator and all workers know all actions,
especially when some actions may be specific to non-worker kids such as
Coordinator and diskers.
Alex Rousskov [Thu, 15 Sep 2011 18:16:59 +0000 (12:16 -0600)]
Do not let cache manager requests kill SMP Squid using isOpen() assertion.
As the comment above the close call implies, we have not imported the foreign
socket descriptor into our fd_table yet. We must use raw close(2), just like
the corresponding Mgr::Request::Request(msg) code that allocates request.conn,
uses raw assignment to give that half-baked connection a descriptor.
TODO: This direct manipulation of Connection::fd is ugly, and this half-baked
connection will most likely cause more [hidden] problems down the road. For
example, Mgr::Request destructor will assert in a similar way if the request
object is destroyed before Action::respond() is called.
Alex Rousskov [Thu, 15 Sep 2011 17:51:23 +0000 (11:51 -0600)]
Added max-swap-rate=swaps/sec option to Rock cache_dir.
The option limits disk access to smooth out OS disk commit activity and to
avoid blocking Rock diskers (or even other processes) on I/O. Should be used
when swap demand exceeds disk performance limits but the underlying file
system does not slow down incoming I/Os until the situation gets out of
control.
TODO: Account for the I/O rate limit when estimating whether a future I/O
will complete in time (for swap-timeout).
TODO: Consider allowing the next swap in (i.e., read) through regardless of
the limit because, unlike writes, reads do not usually accumulate in OS
buffers.
projects / thirdparty / squid.git / log
