Bug #1819 fix, part 1: simplify ICAPInitiator-ICAPXaction link.
Part 1 changes should have no effect on Squid functionality.
To retry failing persistent ICAP connections we may need to
start more than one ICAP transaction for a given HTTP message.
This needs to be done somewhere between ICAPInitiator and
ICAPXaction so that neither is affected much. The design will
be similar to HTTP FwdState.
This change reduces ICAPInitiator dependency on ICAPXaction so
that it is easier to insert transaction retrying logic/code
between them. It should have no effect on Squid functionality.
HTTP client and server (both ICAPInitiators) used to keep a
Pointer to ICAP transaction and extracted adapted headers from
the finished transaction using that Pointer (which forced
ICAPModXact to use a self-Pointer). ICAP service
representative used custom callbacks to receive new options
from ICAPOptXact.
Now, all ICAP initiators use the same asynchronous
message-based API to communicate with ICAP transactions they
initiate. On the core side, the API is defined by
ICAPInitiator, as before. On the ICAP side, the API is defined
by the newly added ICAPInitiate class. The latter will also be
used as a base for ICAPLauncher, the future ICAP transaction
retrying class.
Bug #1819 fix, part 2: retry the ICAP transaction when its
pconn fails. Part 2 changes should enable Squid to handle race
conditions of ICAP pconns.
HTTP client, server, and ICAP server representative (all
ICAPInitiators) used to start ICAP transactions. If a
transaction fails due to a persistent connection race
condition, the initiator would either terminate the HTTP
transaction or bypass the ICAP failure, violating the protocol.
Now, instead of starting an ICAP transaction directly, the
above initiators start ICAP Launcher. The Launcher starts the
ICAP transaction and retries it (i.e., starts another
transaction with the same set of parameters) if needed. The
Launcher is both ICAP initiator (because it starts ICAP
transactions) and ICAP initiate (because it is started by other
ICAP initiators). The launcher proxies ICAP communication
(usually one or two messages each way) and makes retries
transparent to initiators.
All ICAP transactions corresponding to the same adaptation of
an HTTP message are sometimes referred to as a single ICAP
query.
An ICAP transaction can be retried until it read data from the
ICAP server or consumed virgin body. This means that a ICAP
transaction may go as far as writing the entire ICAP Preview
before aborting in a retriable state. When retrying, persistent
connections are not used. If the first retry fails, the query
fails. Thus, a query cannot contain more than two transactions.
Two new things were added to support ICAPLauncher:
First, AsyncJob, is a class that handles the basic logic of
maintaining an asynchronous job or task such as an ICAP
transaction or a launcher. All core AsyncJob functionality was
moved from ICAPXaction and ICAPModXact classes. We could not
use the two classes for the launcher because ICAPLauncher is
not a transaction. While currently specific to ICAP, this
object may eventually be moved to Squid core and serve as a
base for other asynchronous jobs or logical threads.
Second, a temporary hack was added to support cbdata operations
by two parents of a single CBDATA class (ICAPLauncher is both
ICAP Initiator and Initiate). The second parent (ICAPInitiator)
defines toCbdata() method that ICAPLauncher overwrites to
provide a usable cbdata pointer to code that only sees an
ICAPInitiator pointer. The ICAPInitiator pointer is unusable
for cbdata when it comes from the middle of the ICAPLauncher
object. Also, the initiator pointer is stored along with its
cbdata so that the latter can be accessed (and unlocked) even
if the initiator object is already deleted. These hacks will
disappear once cbdata becomes object-aware.
Bug #1819 fix, part 3: do not reuse a pconn when we may fail to
retry it. Part 3 changes ensure that the HTTP message body is
backed up when reusing a persistent connection (until we know
there are no race conditions). They also disable pconns when we
cannot backup enough.
Decide on preview before connecting to the ICAP server because
if we are doing preview, we can retry a failed pconn even if we
cannot backup the entire body. Renamed shouldPreview() to
decideOnPreview() and made the call less dependent on the
context.
Disable transaction retries if we are not using a persistent
connection. ICAP does not talk about retrying failures other
than pconn race conditions.
It might be OK to retry other failures, but I have not done the
required analysis and am not going to do that in the
foreseeable future. For example, other failures may need other
conditions for disabling retries (current code should disable
retries when the first response byte is read).
--- /dev/null
+/*
+ * DEBUG: section 93 ICAP (RFC 3507) Client
+ */
+
+#include "squid.h"
+#include "cbdata.h"
+#include "TextException.h"
+#include "AsyncJob.h"
+
+
+AsyncJob *AsyncJob::AsyncStart(AsyncJob *job) {
+ assert(job);
+ job = cbdataReference(job); // unlocked when done() in callEnd()
+ AsyncCall(93,5, job, AsyncJob::noteStart);
+ return job;
+}
+
+AsyncJob::AsyncJob(const char *aTypeName): typeName(aTypeName), inCall(NULL)
+{
+}
+
+AsyncJob::~AsyncJob()
+{
+}
+
+void AsyncJob::noteStart()
+{
+ AsyncCallEnter(noteStart);
+
+ start();
+
+ AsyncCallExit();
+}
+
+void AsyncJob::start()
+{
+ Must(cbdataReferenceValid(this)); // locked in AsyncStart
+}
+
+void AsyncJob::mustStop(const char *aReason)
+{
+ Must(inCall); // otherwise nobody will delete us if we are done()
+ Must(aReason);
+ if (!stopReason) {
+ stopReason = aReason;
+ debugs(93, 5, typeName << " will stop, reason: " << stopReason);
+ } else {
+ debugs(93, 5, typeName << " will stop, another reason: " << aReason);
+ }
+}
+
+bool AsyncJob::done() const
+{
+ // stopReason, set in mustStop(), overwrites all other conditions
+ return stopReason != NULL || doneAll();
+}
+
+bool AsyncJob::doneAll() const
+{
+ return true; // so that it is safe for kids to use
+}
+
+bool AsyncJob::callStart(const char *method)
+{
+ debugs(93, 5, typeName << "::" << method << " called" << status());
+
+ if (inCall) {
+ // this may happen when we have bugs or when arguably buggy
+ // comm interface calls us while we are closing the connection
+ debugs(93, 5, HERE << typeName << "::" << inCall <<
+ " is in progress; " << typeName << "::" << method <<
+ " cancels reentry.");
+ return false;
+ }
+
+ inCall = method;
+ return true;
+}
+
+void AsyncJob::callException(const TextException &e)
+{
+ debugs(93, 2, typeName << "::" << inCall << " caught an exception: " <<
+ e.message << ' ' << status());
+
+ mustStop("exception");
+}
+
+void AsyncJob::callEnd()
+{
+ if (done()) {
+ debugs(93, 5, typeName << "::" << inCall << " ends job " <<
+ status());
+
+ const char *inCallSaved = inCall;
+ const char *typeNameSaved = typeName;
+ void *thisSaved = this;
+
+ swanSong();
+
+ void *cbdata = this;
+ delete this; // this is the only place where the object is deleted
+ cbdataReferenceDone(cbdata); // locked by AsyncStart
+
+ // careful: this object does not exist any more
+ debugs(93, 6, HERE << typeNameSaved << "::" << inCallSaved <<
+ " ended " << thisSaved);
+ return;
+ }
+
+ debugs(93, 6, typeName << "::" << inCall << " ended" << status());
+ inCall = NULL;
+}
+
--- /dev/null
+
+/*
+ * $Id: AsyncJob.h,v 1.1 2007/05/08 16:32:11 rousskov Exp $
+ *
+ *
+ * SQUID Web Proxy Cache http://www.squid-cache.org/
+ * ----------------------------------------------------------
+ *
+ * Squid is the result of efforts by numerous individuals from
+ * the Internet community; see the CONTRIBUTORS file for full
+ * details. Many organizations have provided support for Squid's
+ * development; see the SPONSORS file for full details. Squid is
+ * Copyrighted (C) 2001 by the Regents of the University of
+ * California; see the COPYRIGHT file for full details. Squid
+ * incorporates software developed and/or copyrighted by other
+ * sources; see the CREDITS file for full details.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA.
+ *
+ */
+
+#ifndef SQUID_ASYNC_JOB_H
+#define SQUID_ASYNC_JOB_H
+
+#include "AsyncCall.h"
+
+/*
+ * AsyncJob is an API and a base for a class that implements a stand-alone
+ * "job", "task", or "logical processing thread" which receives asynchronous
+ * calls.
+ *
+ * Implementations should wrap each method receiving an asynchronous call in
+ * a pair of macros: AsyncCallEnter and AsyncCallExit. These macros:
+ * - provide call debugging
+ * - trap exceptions and terminate the task if an exception occurs
+ * - ensure that only one asynchronous call is active per object
+ * Most of the work is done by AsyncJob class methods. Macros just provide
+ * an enter/try/catch/exit framework.
+ *
+ * Eventually, the macros can and perhaps should be replaced with call/event
+ * processing code so that individual job classes do not have to wrap all
+ * asynchronous calls.
+ */
+
+class TextException;
+
+class AsyncJob
+{
+
+public:
+ static AsyncJob *AsyncStart(AsyncJob *job); // use this to start jobs
+
+ AsyncJob(const char *aTypeName);
+ virtual ~AsyncJob();
+
+ void noteStart(); // calls virtual start
+ AsyncCallWrapper(93,3, AsyncJob, noteStart);
+
+protected:
+ void mustStop(const char *aReason); // force done() for a reason
+
+ bool done() const; // the job is destroyed in callEnd() when done()
+
+ virtual void start() = 0;
+ virtual bool doneAll() const = 0; // return true when done
+ virtual void swanSong() = 0; // perform internal cleanup
+ virtual const char *status() const = 0; // for debugging
+
+ // asynchronous call maintenance
+ bool callStart(const char *methodName);
+ void callException(const TextException &e);
+ virtual void callEnd();
+
+ const char *stopReason; // reason for forcing done() to be true
+ const char *typeName; // kid (leaf) class name, for debugging
+ const char *inCall; // name of the asynchronous call being executed, if any
+};
+
+
+// call guards for all "asynchronous" note*() methods
+// TODO: Move to core.
+
+// asynchronous call entry:
+// - open the try clause;
+// - call callStart().
+#define AsyncCallEnter(method) \
+ try { \
+ if (!callStart(#method)) \
+ return;
+
+// asynchronous call exit:
+// - close the try clause;
+// - catch exceptions;
+// - let callEnd() handle transaction termination conditions
+#define AsyncCallExit() \
+ } \
+ catch (const TextException &e) { \
+ callException(e); \
+ } \
+ callEnd();
+
+
+#endif /* SQUID_ASYNC_JOB_H */
--- /dev/null
+/*
+ * DEBUG: section 93 ICAP (RFC 3507) Client
+ */
+
+#include "squid.h"
+#include "HttpMsg.h"
+#include "ICAPInitiator.h"
+#include "ICAPInitiate.h"
+
+/* The call objects below are not cbdata-protected or refcounted because
+ * nobody holds a pointer to them except for the event queue.
+ *
+ * The calls do check the Initiator pointer to see if that is still valid.
+ *
+ * TODO: convert this to a generic AsyncCall1 class
+ * TODO: mempool kids of this class.
+ */
+
+/* Event data and callback wrapper to call noteIcapAnswer with
+ * the answer message as a parameter.
+ */
+class ICAPAnswerCall {
+public:
+ // use this function to make an asynchronous call
+ static void Schedule(const ICAPInitiatorHolder &anInitiator, HttpMsg *aMessage);
+
+ static void Wrapper(void *data);
+
+protected:
+ ICAPAnswerCall(const ICAPInitiatorHolder &anInitiator, HttpMsg *aMessage);
+ ~ICAPAnswerCall();
+
+ void schedule();
+ void call();
+
+ ICAPInitiatorHolder theInitiator;
+ HttpMsg *theMessage;
+};
+
+
+/* Event data and callback wrapper to call noteIcapQueryAbort with
+ * the termination status as a parameter.
+ *
+ * XXX: This class is a clone of ICAPAnswerCall.
+ */
+class ICAPQueryAbortCall {
+public:
+ // use this function to make an asynchronous call
+ static void Schedule(const ICAPInitiatorHolder &anInitiator, bool beFinal);
+
+ static void Wrapper(void *data);
+
+protected:
+ ICAPQueryAbortCall(const ICAPInitiatorHolder &anInitiator, bool beFinal);
+
+ void schedule();
+ void call();
+
+ ICAPInitiatorHolder theInitiator;
+ bool isFinal;
+};
+
+
+/* ICAPInitiate */
+
+ICAPInitiate::ICAPInitiate(const char *aTypeName,
+ ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService):
+ AsyncJob(aTypeName), theInitiator(anInitiator), theService(aService)
+{
+ assert(theService != NULL);
+ assert(theInitiator);
+}
+
+ICAPInitiate::~ICAPInitiate()
+{
+ assert(!theInitiator);
+}
+
+// internal cleanup
+void ICAPInitiate::swanSong()
+{
+ debugs(93, 5, HERE << "swan sings" << status());
+
+ if (theInitiator) {
+ debugs(93, 3, HERE << "fatal failure; sending abort notification");
+ tellQueryAborted(true); // final by default
+ }
+
+ debugs(93, 5, HERE << "swan sang" << status());
+}
+
+void ICAPInitiate::clearInitiator()
+{
+ if (theInitiator)
+ theInitiator.clear();
+}
+
+void ICAPInitiate::sendAnswer(HttpMsg *msg)
+{
+ ICAPAnswerCall::Schedule(theInitiator, msg);
+ clearInitiator();
+}
+
+void ICAPInitiate::tellQueryAborted(bool final)
+{
+ ICAPQueryAbortCall::Schedule(theInitiator, final);
+ clearInitiator();
+}
+
+ICAPServiceRep &ICAPInitiate::service()
+{
+ assert(theService != NULL);
+ return *theService;
+}
+
+const char *ICAPInitiate::status() const {
+ return ""; // for now
+}
+
+
+/* ICAPInitiatorHolder */
+
+ICAPInitiatorHolder::ICAPInitiatorHolder(ICAPInitiator *anInitiator):
+ ptr(0), cbdata(0)
+{
+ if (anInitiator) {
+ cbdata = cbdataReference(anInitiator->toCbdata());
+ ptr = anInitiator;
+ }
+}
+
+ICAPInitiatorHolder::ICAPInitiatorHolder(const ICAPInitiatorHolder &anInitiator):
+ ptr(0), cbdata(0)
+{
+ if (anInitiator != NULL && cbdataReferenceValid(anInitiator.cbdata)) {
+ cbdata = cbdataReference(anInitiator.cbdata);
+ ptr = anInitiator.ptr;
+ }
+}
+
+ICAPInitiatorHolder::~ICAPInitiatorHolder()
+{
+ clear();
+}
+
+void ICAPInitiatorHolder::clear() {
+ if (ptr) {
+ ptr = NULL;
+ cbdataReferenceDone(cbdata);
+ }
+}
+
+// should not be used
+ICAPInitiatorHolder &ICAPInitiatorHolder::operator =(const ICAPInitiatorHolder &anInitiator)
+{
+ assert(false);
+ return *this;
+}
+
+/* ICAPAnswerCall */
+
+ICAPAnswerCall::ICAPAnswerCall(const ICAPInitiatorHolder &anInitiator, HttpMsg *aMessage):
+ theInitiator(anInitiator), theMessage(0)
+{
+ if (theInitiator) {
+ assert(aMessage);
+ theMessage = HTTPMSGLOCK(aMessage);
+ }
+}
+
+void ICAPAnswerCall::schedule()
+{
+ if (theInitiator) {
+ debugs(93,3, __FILE__ << "(" << __LINE__ << ") will call " <<
+ theInitiator << "->ICAPInitiator::noteIcapAnswer(" <<
+ theMessage << ")");
+ eventAdd("ICAPInitiator::noteIcapAnswer",
+ &ICAPAnswerCall::Wrapper, this, 0.0, 0, false);
+ } else {
+ debugs(93,3, __FILE__ << "(" << __LINE__ << ") will not call " <<
+ theInitiator << "->ICAPInitiator::noteIcapAnswer(" <<
+ theMessage << ")");
+ }
+}
+
+ICAPAnswerCall::~ICAPAnswerCall()
+{
+ if (theInitiator)
+ HTTPMSGUNLOCK(theMessage);
+}
+
+void ICAPAnswerCall::Wrapper(void *data)
+{
+ assert(data);
+ ICAPAnswerCall *c = static_cast<ICAPAnswerCall*>(data);
+ c->call();
+ delete c;
+}
+
+void ICAPAnswerCall::call() {
+ assert(theInitiator);
+ if (cbdataReferenceValid(theInitiator.cbdata)) {
+ debugs(93, 3, "entering " <<
+ theInitiator << "->ICAPInitiator::noteIcapAnswer(" <<
+ theMessage << ")");
+ theInitiator.ptr->noteIcapAnswer(theMessage);
+ debugs(93, 3, "exiting " <<
+ theInitiator << "->ICAPInitiator::noteIcapAnswer(" <<
+ theMessage << ")");
+ } else {
+ debugs(93, 3, "ignoring " <<
+ theInitiator << "->ICAPInitiator::noteIcapAnswer(" <<
+ theMessage << ")");
+ }
+}
+
+void ICAPAnswerCall::Schedule(const ICAPInitiatorHolder &anInitiator, HttpMsg *aMessage)
+{
+ ICAPAnswerCall *call = new ICAPAnswerCall(anInitiator, aMessage);
+ call->schedule();
+ // The call object is deleted in ICAPAnswerCall::Wrapper
+}
+
+
+/* ICAPQueryAbortCall */
+
+ICAPQueryAbortCall::ICAPQueryAbortCall(const ICAPInitiatorHolder &anInitiator, bool beFinal):
+ theInitiator(anInitiator), isFinal(beFinal)
+{
+}
+
+void ICAPQueryAbortCall::schedule()
+{
+ if (theInitiator) {
+ debugs(93,3, __FILE__ << "(" << __LINE__ << ") will call " <<
+ theInitiator << "->ICAPInitiator::noteIcapQueryAbort(" <<
+ isFinal << ")");
+ eventAdd("ICAPInitiator::noteIcapQueryAbort",
+ &ICAPQueryAbortCall::Wrapper, this, 0.0, 0, false);
+ } else {
+ debugs(93,3, __FILE__ << "(" << __LINE__ << ") will not call " <<
+ theInitiator << "->ICAPInitiator::noteIcapQueryAbort(" <<
+ isFinal << ")");
+ }
+}
+
+void ICAPQueryAbortCall::Wrapper(void *data)
+{
+ assert(data);
+ ICAPQueryAbortCall *c = static_cast<ICAPQueryAbortCall*>(data);
+ c->call();
+ delete c;
+}
+
+void ICAPQueryAbortCall::call() {
+ assert(theInitiator);
+ if (cbdataReferenceValid(theInitiator.cbdata)) {
+ debugs(93, 3, "entering " <<
+ theInitiator << "->ICAPInitiator::noteIcapQueryAbort(" <<
+ isFinal << ")");
+ theInitiator.ptr->noteIcapQueryAbort(isFinal);
+ debugs(93, 3, "exiting " <<
+ theInitiator << "->ICAPInitiator::noteIcapQueryAbort(" <<
+ isFinal << ")");
+ } else {
+ debugs(93, 3, "ignoring " <<
+ theInitiator << "->ICAPInitiator::noteIcapQueryAbort(" <<
+ isFinal << ")");
+ }
+}
+
+void ICAPQueryAbortCall::Schedule(const ICAPInitiatorHolder &anInitiator, bool beFinal)
+{
+ ICAPQueryAbortCall *call = new ICAPQueryAbortCall(anInitiator, beFinal);
+ call->schedule();
+ // The call object is deleted in ICAPQueryAbortCall::Wrapper
+}
--- /dev/null
+
+/*
+ * $Id: ICAPInitiate.h,v 1.1 2007/05/08 16:32:11 rousskov Exp $
+ *
+ *
+ * SQUID Web Proxy Cache http://www.squid-cache.org/
+ * ----------------------------------------------------------
+ *
+ * Squid is the result of efforts by numerous individuals from
+ * the Internet community; see the CONTRIBUTORS file for full
+ * details. Many organizations have provided support for Squid's
+ * development; see the SPONSORS file for full details. Squid is
+ * Copyrighted (C) 2001 by the Regents of the University of
+ * California; see the COPYRIGHT file for full details. Squid
+ * incorporates software developed and/or copyrighted by other
+ * sources; see the CREDITS file for full details.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA.
+ *
+ */
+
+#ifndef SQUID_ICAPINITIATE_H
+#define SQUID_ICAPINITIATE_H
+
+#include "comm.h"
+#include "MemBuf.h"
+#include "ICAPServiceRep.h"
+#include "AsyncJob.h"
+
+class HttpMsg;
+class ICAPInitiator;
+
+/* Initiator holder associtates an initiator with its cbdata. It is used as
+ * a temporary hack to make cbdata work with multiple inheritance */
+class ICAPInitiatorHolder {
+public:
+ ICAPInitiatorHolder(ICAPInitiator *anInitiator);
+ ICAPInitiatorHolder(const ICAPInitiatorHolder &anInitiator);
+ ~ICAPInitiatorHolder();
+
+
+ void clear();
+
+ // to make comparison with NULL possible
+ operator void*() { return ptr; }
+ bool operator == (void *) const { return ptr == NULL; }
+ bool operator != (void *) const { return ptr != NULL; }
+ bool operator !() const { return !ptr; }
+
+ ICAPInitiator *ptr;
+ void *cbdata;
+
+private:
+ ICAPInitiatorHolder &operator =(const ICAPInitiatorHolder &anInitiator);
+};
+
+/*
+ * The ICAP Initiate is a common base for ICAP queries or transactions
+ * initiated by an ICAPInitiator. This interface exists to allow an ICAP
+ * initiator to signal its "initiatees" that it is aborting and no longer
+ * expecting an answer. The class is also handy for implementing common
+ * initiate actions such as maintaining and notifying the initiator.
+ *
+ * ICAPInitiate implementations must cbdata-protect themselves.
+ *
+ * This class could have been named ICAPInitiatee.
+ */
+class ICAPInitiate: public AsyncJob
+{
+
+public:
+ ICAPInitiate(const char *aTypeName, ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService);
+ virtual ~ICAPInitiate();
+
+ // communication with the initiator
+ virtual void noteInitiatorAborted() = 0;
+ AsyncCallWrapper(93,3, ICAPInitiate, noteInitiatorAborted)
+
+protected:
+ ICAPServiceRep &service();
+
+ void sendAnswer(HttpMsg *msg); // send to the initiator
+ void tellQueryAborted(bool final); // tell initiator
+ void clearInitiator(); // used by noteInitiatorAborted; TODO: make private
+
+ virtual void swanSong(); // internal cleanup
+
+ virtual const char *status() const; // for debugging
+
+ ICAPInitiatorHolder theInitiator;
+ ICAPServiceRep::Pointer theService;
+};
+
+#endif /* SQUID_ICAPINITIATE_H */
-// XXX: remove me!
+/*
+ * DEBUG: section 93 ICAP (RFC 3507) Client
+ */
+
+#include "squid.h"
+#include "ICAPXaction.h"
+#include "ICAPInitiator.h"
+
+ICAPInitiate *ICAPInitiator::initiateIcap(ICAPInitiate *x) {
+ x = cbdataReference(x);
+ return dynamic_cast<ICAPInitiate*>(ICAPInitiate::AsyncStart(x));
+}
+
+void ICAPInitiator::clearIcap(ICAPInitiate *&x) {
+ assert(x);
+ cbdataReferenceDone(x);
+}
+
+void ICAPInitiator::announceInitiatorAbort(ICAPInitiate *&x)
+{
+ if (x) {
+ AsyncCall(93,5, x, ICAPInitiate::noteInitiatorAborted);
+ clearIcap(x);
+ }
+}
/*
- * $Id: ICAPInitiator.h,v 1.1 2007/04/06 04:50:07 rousskov Exp $
+ * $Id: ICAPInitiator.h,v 1.2 2007/05/08 16:32:11 rousskov Exp $
*
*
* SQUID Web Proxy Cache http://www.squid-cache.org/
#ifndef SQUID_ICAPINITIATOR_H
#define SQUID_ICAPINITIATOR_H
-#include "AsyncCall.h"
-
/*
* The ICAP Initiator is an ICAP vectoring point that initates ICAP
* transactions. This interface exists to allow ICAP transactions to
- * signal their initiators that they are finished or aborted.
+ * signal their initiators that they have the answer from the ICAP server
+ * or that the ICAP query has aborted and there will be no answer. It
+ * is also handy for implementing common initiator actions such as starting
+ * or aborting an ICAP transaction.
*/
-class ICAPXaction;
+class HttpMsg;
+class ICAPInitiate;
class ICAPInitiator
{
public:
virtual ~ICAPInitiator() {}
- virtual void noteIcapHeadersAdapted() = 0;
- virtual void noteIcapHeadersAborted() = 0;
+ // called when ICAP response headers are successfully interpreted
+ virtual void noteIcapAnswer(HttpMsg *message) = 0;
+
+ // called when valid ICAP response headers are no longer expected
+ // the final parameter is set to disable bypass or retries
+ virtual void noteIcapQueryAbort(bool final) = 0;
+
+ // a temporary cbdata-for-multiple inheritance hack, see ICAPInitiator.cc
+ virtual void *toCbdata() { return this; }
+
+protected:
+ ICAPInitiate *initiateIcap(ICAPInitiate *x); // locks and returns x
+
+ // done with x (and not calling announceInitiatorAbort)
+ void clearIcap(ICAPInitiate *&x); // unlocks x
- AsyncCallWrapper(93,4, ICAPInitiator, noteIcapHeadersAdapted);
- AsyncCallWrapper(93,3, ICAPInitiator, noteIcapHeadersAborted);
+ // inform the transaction about abnormal termination and clear it
+ void announceInitiatorAbort(ICAPInitiate *&x); // unlocks x
};
#endif /* SQUID_ICAPINITIATOR_H */
--- /dev/null
+/*
+ * DEBUG: section 93 ICAP (RFC 3507) Client
+ */
+
+#include "squid.h"
+#include "TextException.h"
+#include "HttpMsg.h"
+#include "ICAPLauncher.h"
+#include "ICAPXaction.h"
+
+
+ICAPLauncher::ICAPLauncher(const char *aTypeName, ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService):
+ ICAPInitiate(aTypeName, anInitiator, aService),
+ theXaction(0), theLaunches(0)
+{
+}
+
+ICAPLauncher::~ICAPLauncher()
+{
+ assert(!theXaction);
+}
+
+void ICAPLauncher::start()
+{
+ ICAPInitiate::start();
+
+ Must(theInitiator);
+ launchXaction(false);
+}
+
+void ICAPLauncher::launchXaction(bool final)
+{
+ Must(!theXaction);
+ ++theLaunches;
+ debugs(93,4, HERE << "launching xaction #" << theLaunches);
+ ICAPXaction *x = createXaction();
+ if (final)
+ x->disableRetries();
+ theXaction = initiateIcap(x);
+ Must(theXaction);
+}
+
+void ICAPLauncher::noteIcapAnswer(HttpMsg *message)
+{
+ AsyncCallEnter(noteIcapAnswer);
+
+ sendAnswer(message);
+ clearIcap(theXaction);
+ Must(done());
+
+ AsyncCallExit();
+}
+
+void ICAPLauncher::noteInitiatorAborted()
+{
+ AsyncCallEnter(noteInitiatorAborted);
+
+ announceInitiatorAbort(theXaction); // propogate to the transaction
+ clearInitiator();
+ Must(done()); // should be nothing else to do
+
+ AsyncCallExit();
+}
+
+void ICAPLauncher::noteIcapQueryAbort(bool final)
+{
+ AsyncCallEnter(noteQueryAbort);
+
+ clearIcap(theXaction);
+
+ // TODO: add more checks from FwdState::checkRetry()?
+ if (!final && theLaunches < 2 && !shutting_down) {
+ launchXaction(true);
+ } else {
+ debugs(93,3, HERE << "cannot retry the failed ICAP xaction; tries: " <<
+ theLaunches << "; final: " << final);
+ Must(done()); // swanSong will notify the initiator
+ }
+
+ AsyncCallExit();
+}
+
+bool ICAPLauncher::doneAll() const {
+ return (!theInitiator || !theXaction) && ICAPInitiate::doneAll();
+}
+
+void ICAPLauncher::swanSong()
+{
+ if (theInitiator)
+ tellQueryAborted(!service().bypass);
+
+ if (theXaction)
+ clearIcap(theXaction);
+
+ ICAPInitiate::swanSong();
+}
--- /dev/null
+
+/*
+ * $Id: ICAPLauncher.h,v 1.1 2007/05/08 16:32:11 rousskov Exp $
+ *
+ *
+ * SQUID Web Proxy Cache http://www.squid-cache.org/
+ * ----------------------------------------------------------
+ *
+ * Squid is the result of efforts by numerous individuals from
+ * the Internet community; see the CONTRIBUTORS file for full
+ * details. Many organizations have provided support for Squid's
+ * development; see the SPONSORS file for full details. Squid is
+ * Copyrighted (C) 2001 by the Regents of the University of
+ * California; see the COPYRIGHT file for full details. Squid
+ * incorporates software developed and/or copyrighted by other
+ * sources; see the CREDITS file for full details.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA.
+ *
+ */
+
+#ifndef SQUID_ICAPLAUNCHER_H
+#define SQUID_ICAPLAUNCHER_H
+
+#include "ICAP/ICAPInitiator.h"
+#include "ICAP/ICAPInitiate.h"
+
+/*
+ * The ICAP Launcher starts an ICAP transaction. If the transaction fails
+ * due to what looks like a persistent connection race condition, the launcher
+ * starts a new ICAP transaction using a freshly opened connection.
+ *
+ * ICAPLauncher and one or more ICAP transactions initiated by it form an
+ * ICAP "query".
+ *
+ * An ICAP Initiator deals with the ICAP Launcher and not an individual ICAP
+ * transaction because the latter may disappear and be replaced by another
+ * transaction.
+ *
+ * Specific ICAP launchers implement the createXaction() method to create
+ * REQMOD, RESPMOD, or OPTIONS transaction from initiator-supplied data.
+ *
+ * TODO: This class might be the right place to initiate ICAP ACL checks or
+ * implement more sophisticated ICAP transaction handling like chaining of
+ * ICAP transactions.
+ */
+
+class ICAPXaction;
+
+// Note: ICAPInitiate must be the first parent for cbdata to work. We use
+// a temporary ICAPInitaitorHolder/toCbdata hacks and do not call cbdata
+// operations on the initiator directly.
+class ICAPLauncher: public ICAPInitiate, public ICAPInitiator
+{
+public:
+ ICAPLauncher(const char *aTypeName, ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService);
+ virtual ~ICAPLauncher();
+
+ // ICAPInitiate: asynchronous communication with the initiator
+ void noteInitiatorAborted();
+
+ // ICAPInitiator: asynchronous communication with the current transaction
+ virtual void noteIcapAnswer(HttpMsg *message);
+ virtual void noteIcapQueryAbort(bool final);
+
+ // a temporary cbdata-for-multiple inheritance hack, see ICAPInitiator.cc
+ virtual void *toCbdata() { return this; }
+
+protected:
+ // ICAPInitiate API implementation
+ virtual void start();
+ virtual bool doneAll() const;
+ virtual void swanSong();
+
+ // creates the right ICAP transaction using stored configuration params
+ virtual ICAPXaction *createXaction() = 0;
+
+ void launchXaction(bool final);
+
+ ICAPInitiate *theXaction; // current ICAP transaction
+ int theLaunches; // the number of transaction launches
+};
+
+#endif /* SQUID_ICAPLAUNCHER_H */
#include "HttpReply.h"
#include "ICAPServiceRep.h"
#include "ICAPInitiator.h"
+#include "ICAPLauncher.h"
#include "ICAPModXact.h"
#include "ICAPClient.h"
#include "ChunkedCodingParser.h"
// TODO: replace gotEncapsulated() with something faster; we call it often
CBDATA_CLASS_INIT(ICAPModXact);
+CBDATA_CLASS_INIT(ICAPModXactLauncher);
static const size_t TheBackupLimit = BodyPipe::MaxCapacity;
ICAPModXact::ICAPModXact(ICAPInitiator *anInitiator, HttpMsg *virginHeader,
HttpRequest *virginCause, ICAPServiceRep::Pointer &aService):
- ICAPXaction("ICAPModXact"),
- initiator(cbdataReference(anInitiator)),
+ ICAPXaction("ICAPModXact", anInitiator, aService),
icapReply(NULL),
virginConsumed(0),
bodyParser(NULL)
{
assert(virginHeader);
- service(aService);
-
virgin.setHeader(virginHeader); // sets virgin.body_pipe if needed
virgin.setCause(virginCause); // may be NULL
// initiator wants us to start
void ICAPModXact::start()
{
- ICAPXaction_Enter(start);
-
ICAPXaction::start();
estimateVirginBody(); // before virgin disappears!
else
waitForService();
- // XXX: but this has to be here to catch other errors. Thus, if
- // commConnectStart in startWriting fails, we may get here
+ // XXX: If commConnectStart in startWriting fails, we may get here
//_after_ the object got destroyed. Somebody please fix commConnectStart!
- // XXX: Is the above comment still valid?
- ICAPXaction_Exit();
+ // TODO: Does re-entrance protection in callStart() solve the above?
}
static
Must(state.serviceWaiting);
state.serviceWaiting = false;
- Must(service().up());
-
- startWriting();
+ if (service().up()) {
+ startWriting();
+ } else {
+ disableRetries();
+ mustStop("ICAP service unusable");
+ }
ICAPXaction_Exit();
}
void ICAPModXact::startWriting()
{
state.writing = State::writingConnect;
+
+ decideOnPreview(); // must be decided before we decideOnRetries
+ decideOnRetries();
+
openConnection();
// put nothing here as openConnection calls commConnectStart
// and that may call us back without waiting for the next select loop
void ICAPModXact::virginConsume()
{
if (!virgin.body_pipe)
- return;
+ return; // nothing to consume
+
+ if (isRetriable)
+ return; // do not consume if we may have to retry later
BodyPipe &bp = *virgin.body_pipe;
const size_t have = static_cast<size_t>(bp.buf().contentSize());
" virgin body bytes");
bp.consume(size);
virginConsumed += size;
+ Must(!isRetriable); // or we should not be consuming
}
}
return;
}
- AsyncCall(93,5, initiator, ICAPInitiator::noteIcapHeadersAdapted);
- cbdataReferenceDone(initiator);
+ sendAnswer(adapted.header);
if (state.sending == State::sendingVirgin)
echoMore();
ICAPXaction_Exit();
}
-// initiator aborted
-void ICAPModXact::noteInitiatorAborted()
-{
- ICAPXaction_Enter(noteInitiatorAborted);
-
- if (initiator) {
- cbdataReferenceDone(initiator);
- mustStop("initiator aborted");
- }
-
- ICAPXaction_Exit();
-}
-
// adapted body consumer wants more adapted data and
// possibly freed some buffer space
void ICAPModXact::noteMoreBodySpaceAvailable(BodyPipe &)
{
debugs(93, 5, HERE << "swan sings" << status());
- if (initiator) {
- debugs(93, 2, HERE << "swan sings for " << stopReason << status());
- AsyncCall(93,5, initiator, ICAPInitiator::noteIcapHeadersAborted);
- cbdataReferenceDone(initiator);
- }
-
stopWriting(false);
- stopBackup();
+ stopSending(false);
if (icapReply) {
delete icapReply;
icapReply = NULL;
}
- stopSending(false);
-
ICAPXaction::swanSong();
}
buf.append(ICAP::crlf, 2); // terminate Encapsulated line
- if (shouldPreview(urlPath)) {
+ if (preview.enabled()) {
buf.Printf("Preview: %d\r\n", (int)preview.ad());
if (virginBody.expected()) // there is a body to preview
virginBodySending.plan();
}
// decides whether to offer a preview and calculates its size
-bool ICAPModXact::shouldPreview(const String &urlPath)
+void ICAPModXact::decideOnPreview()
{
if (!TheICAPConfig.preview_enable) {
debugs(93, 5, HERE << "preview disabled by squid.conf");
- return false;
+ return;
}
+ const HttpRequest *request = virgin.cause ?
+ virgin.cause :
+ dynamic_cast<const HttpRequest*>(virgin.header);
+ const String urlPath = request ? request->urlpath : String();
size_t wantedSize;
-
if (!service().wantsPreview(urlPath, wantedSize)) {
debugs(93, 5, "ICAPModXact should not offer preview for " << urlPath);
- return false;
+ return;
}
+ // we decided to do preview, now compute its size
+
Must(wantedSize >= 0);
// cannot preview more than we can backup
preview.enable(ad);
Must(preview.enabled());
-
- return true;
}
// decides whether to allow 204 responses
if (!service().allows204())
return false;
+ return canBackupEverything();
+}
+
+// used by shouldAllow204 and decideOnRetries
+bool ICAPModXact::canBackupEverything() const
+{
if (!virginBody.expected())
- return true; // no body means no problems with supporting 204s.
+ return true; // no body means no problems with backup
- // if there is a body, make sure we can backup it all
+ // if there is a body, check whether we can backup it all
if (!virginBody.knownSize())
return false;
return virginBody.size() < TheBackupLimit;
}
+// Decide whether this transaction can be retried if pconn fails
+// Must be called after decideOnPreview and before openConnection()
+void ICAPModXact::decideOnRetries()
+{
+ if (!isRetriable)
+ return; // no, already decided
+
+ if (preview.enabled())
+ return; // yes, because preview provides enough guarantees
+
+ if (canBackupEverything())
+ return; // yes, because we can back everything up
+
+ disableRetries(); // no, because we cannot back everything up
+}
+
// Normally, the body-writing code handles preview body. It can deal with
// bodies of unexpected size, including those that turn out to be empty.
// However, that code assumes that the body was expected and body control
return true;
}
+
+
+/* ICAPModXactLauncher */
+
+ICAPModXactLauncher::ICAPModXactLauncher(ICAPInitiator *anInitiator, HttpMsg *virginHeader, HttpRequest *virginCause, ICAPServiceRep::Pointer &aService):
+ ICAPLauncher("ICAPModXactLauncher", anInitiator, aService)
+{
+ virgin.setHeader(virginHeader);
+ virgin.setCause(virginCause);
+}
+
+ICAPXaction *ICAPModXactLauncher::createXaction()
+{
+ return new ICAPModXact(this, virgin.header, virgin.cause, theService);
+}
/*
- * $Id: ICAPModXact.h,v 1.7 2007/04/06 04:50:07 rousskov Exp $
+ * $Id: ICAPModXact.h,v 1.8 2007/05/08 16:32:11 rousskov Exp $
*
*
* SQUID Web Proxy Cache http://www.squid-cache.org/
#ifndef SQUID_ICAPMODXACT_H
#define SQUID_ICAPMODXACT_H
+#include "BodyPipe.h"
#include "ICAPXaction.h"
#include "ICAPInOut.h"
-#include "BodyPipe.h"
+#include "ICAPLauncher.h"
/*
* ICAPModXact implements ICAP REQMOD and RESPMOD transaction using
class ICAPModXact: public ICAPXaction, public BodyProducer, public BodyConsumer
{
-public:
- typedef RefCount<ICAPModXact> Pointer;
-
public:
ICAPModXact(ICAPInitiator *anInitiator, HttpMsg *virginHeader, HttpRequest *virginCause, ICAPServiceRep::Pointer &s);
- // communication with the initiator
- void noteInitiatorAborted();
- AsyncCallWrapper(93,3, ICAPModXact, noteInitiatorAborted)
-
// BodyProducer methods
virtual void noteMoreBodySpaceAvailable(BodyPipe &);
virtual void noteBodyConsumerAborted(BodyPipe &);
void virginConsume();
void finishNullOrEmptyBodyPreview(MemBuf &buf);
- bool shouldPreview(const String &urlPath);
+ void decideOnPreview();
+ void decideOnRetries();
bool shouldAllow204();
+ bool canBackupEverything() const;
+
void prepBackup(size_t expectedSize);
void backup(const MemBuf &buf);
bool gotEncapsulated(const char *section) const;
void checkConsuming();
- ICAPInitiator *initiator;
HttpReply *icapReply;
CBDATA_CLASS2(ICAPModXact);
};
-// destroys the transaction; implemented in ICAPClient.cc (ick?)
-extern void ICAPNoteXactionDone(ICAPModXact::Pointer x);
+// An ICAPLauncher that stores ICAPModXact construction info and
+// creates ICAPModXact when needed
+class ICAPModXactLauncher: public ICAPLauncher
+{
+public:
+ ICAPModXactLauncher(ICAPInitiator *anInitiator, HttpMsg *virginHeader, HttpRequest *virginCause, ICAPServiceRep::Pointer &s);
+
+protected:
+ virtual ICAPXaction *createXaction();
+
+ ICAPInOut virgin;
+
+private:
+ CBDATA_CLASS2(ICAPModXactLauncher);
+};
#endif /* SQUID_ICAPMOD_XACT_H */
#include "TextException.h"
CBDATA_CLASS_INIT(ICAPOptXact);
+CBDATA_CLASS_INIT(ICAPOptXactLauncher);
-ICAPOptXact::ICAPOptXact(ICAPServiceRep::Pointer &aService, Callback *aCbAddr, void *aCbData):
- ICAPXaction("ICAPOptXact"),
- cbAddr(aCbAddr), cbData(cbdataReference(aCbData))
-{
- Must(aCbAddr && aCbData);
- service(aService);
-}
-ICAPOptXact::~ICAPOptXact()
+ICAPOptXact::ICAPOptXact(ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService):
+ ICAPXaction("ICAPOptXact", anInitiator, aService)
{
- if (cbAddr) {
- debugs(93, 1, HERE << "BUG: exiting without sending options");
- cbdataReferenceDone(cbData);
- }
}
void ICAPOptXact::start()
{
- ICAPXaction_Enter(start);
-
ICAPXaction::start();
- Must(self != NULL); // set by AsyncStart;
-
openConnection();
-
- ICAPXaction_Exit();
}
void ICAPOptXact::handleCommConnected()
// comm module read a portion of the ICAP response for us
void ICAPOptXact::handleCommRead(size_t)
{
- if (ICAPOptions *options = parseResponse()) {
- sendOptions(options);
+ if (HttpMsg *r = parseResponse()) {
+ sendAnswer(r);
Must(done()); // there should be nothing else to do
return;
}
scheduleRead();
}
-ICAPOptions *ICAPOptXact::parseResponse()
+HttpMsg *ICAPOptXact::parseResponse()
{
debugs(93, 5, HERE << "have " << readBuf.contentSize() << " bytes to parse" <<
status());
if (httpHeaderHasConnDir(&r->header, "close"))
reuseConnection = false;
- ICAPOptions *options = new ICAPOptions;
- options->configure(r);
-
- delete r;
-
- return options;
-}
-
-void ICAPOptXact::swanSong() {
- if (cbAddr) {
- debugs(93, 3, HERE << "probably failed; sending NULL options");
- sendOptions(0);
- }
- ICAPXaction::swanSong();
+ return r;
}
-void ICAPOptXact::sendOptions(ICAPOptions *options) {
- debugs(93, 3, HERE << "sending options " << options << " to " << cbData <<
- " at " << (void*)cbAddr << status());
+/* ICAPOptXactLauncher */
- Must(cbAddr);
- Callback *addr = cbAddr;
- cbAddr = NULL; // in case the callback calls us or throws
+ICAPOptXactLauncher::ICAPOptXactLauncher(ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService):
+ ICAPLauncher("ICAPOptXactLauncher", anInitiator, aService)
+{
+}
- void *data = NULL;
- if (cbdataReferenceValidDone(cbData, &data))
- (*addr)(options, data); // callee takes ownership of the options
- else
- debugs(93, 2, HERE << "sending options " << options << " to " <<
- data << " failed" << status());
+ICAPXaction *ICAPOptXactLauncher::createXaction()
+{
+ return new ICAPOptXact(this, theService);
}
/*
- * $Id: ICAPOptXact.h,v 1.5 2007/04/06 04:50:07 rousskov Exp $
+ * $Id: ICAPOptXact.h,v 1.6 2007/05/08 16:32:11 rousskov Exp $
*
*
* SQUID Web Proxy Cache http://www.squid-cache.org/
#define SQUID_ICAPOPTXACT_H
#include "ICAPXaction.h"
+#include "ICAPLauncher.h"
class ICAPOptions;
+
/* ICAPOptXact sends an ICAP OPTIONS request to the ICAP service,
- * converts the response into ICAPOptions object, and notifies
- * the caller via the callback. NULL options objects means the
- * ICAP service could not be contacted or did not return any response */
+ * parses the ICAP response, and sends it to the initiator. A NULL response
+ * means the ICAP service could not be contacted or did not return any
+ * valid response. */
class ICAPOptXact: public ICAPXaction
{
public:
- typedef void Callback(ICAPOptions *newOptions, void *callerData);
-
- ICAPOptXact(ICAPServiceRep::Pointer &aService, Callback *aCb, void *aCbData);
- virtual ~ICAPOptXact();
+ ICAPOptXact(ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService);
protected:
virtual void start();
virtual void handleCommConnected();
virtual void handleCommWrote(size_t size);
virtual void handleCommRead(size_t size);
- virtual void swanSong();
void makeRequest(MemBuf &buf);
- ICAPOptions *parseResponse();
- void sendOptions(ICAPOptions *options);
+ HttpMsg *parseResponse();
void startReading();
private:
- Callback *cbAddr; // callback to call with newly fetched options
- void *cbData; // callback data
-
CBDATA_CLASS2(ICAPOptXact);
};
-// TODO: replace the callback API with a class-base interface?
+// An ICAPLauncher that stores ICAPOptXact construction info and
+// creates ICAPOptXact when needed
+class ICAPOptXactLauncher: public ICAPLauncher
+{
+public:
+ ICAPOptXactLauncher(ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService);
+
+protected:
+ virtual ICAPXaction *createXaction();
+
+private:
+ CBDATA_CLASS2(ICAPOptXactLauncher);
+};
#endif /* SQUID_ICAPOPTXACT_H */
#include "squid.h"
#include "TextException.h"
+#include "HttpReply.h"
#include "ICAPServiceRep.h"
#include "ICAPOptions.h"
#include "ICAPOptXact.h"
wasAnnouncedUp = !wasAnnouncedUp;
}
-static
-void ICAPServiceRep_noteNewOptions(ICAPOptions *newOptions, void *data)
+// we are receiving ICAP OPTIONS response headers here or NULL on failures
+void ICAPServiceRep::noteIcapAnswer(HttpMsg *msg)
{
- ICAPServiceRep *service = static_cast<ICAPServiceRep*>(data);
- Must(service);
- service->noteNewOptions(newOptions);
-}
+ Must(waiting);
+ waiting = false;
-void ICAPServiceRep::noteNewOptions(ICAPOptions *newOptions)
-{
- // newOptions may be NULL
+ Must(msg);
+
+ debugs(93,5, "ICAPService is interpreting new options " << status());
+ ICAPOptions *newOptions = NULL;
+ if (HttpReply *r = dynamic_cast<HttpReply*>(msg)) {
+ newOptions = new ICAPOptions;
+ newOptions->configure(r);
+ } else {
+ debugs(93,1, "ICAPService got wrong options message " << status());
+ }
+
+ handleNewOptions(newOptions);
+}
+
+void ICAPServiceRep::noteIcapQueryAbort(bool) {
Must(waiting);
waiting = false;
+ debugs(93,3, "ICAPService failed to fetch options " << status());
+ handleNewOptions(0);
+}
+
+void ICAPServiceRep::handleNewOptions(ICAPOptions *newOptions)
+{
+ // new options may be NULL
changeOptions(newOptions);
debugs(93,3, "ICAPService got new options and is now " << status());
debugs(93,6, "ICAPService will get new options " << status());
waiting = true;
- ICAPOptXact::AsyncStart(
- new ICAPOptXact(self, &ICAPServiceRep_noteNewOptions, this));
+ initiateIcap(new ICAPOptXactLauncher(this, self));
// TODO: timeout in case ICAPOptXact never calls us back?
+ // Such a timeout should probably be a generic AsyncStart feature.
}
void ICAPServiceRep::scheduleUpdate()
/*
- * $Id: ICAPServiceRep.h,v 1.6 2007/04/06 04:50:08 rousskov Exp $
+ * $Id: ICAPServiceRep.h,v 1.7 2007/05/08 16:32:12 rousskov Exp $
*
*
* SQUID Web Proxy Cache http://www.squid-cache.org/
#define SQUID_ICAPSERVICEREP_H
#include "cbdata.h"
+#include "ICAPInitiator.h"
#include "ICAPElements.h"
class ICAPOptions;
-
class ICAPOptXact;
/* The ICAP service representative maintains information about a single ICAP
*/
-class ICAPServiceRep : public RefCountable
+class ICAPServiceRep : public RefCountable, public ICAPInitiator
{
public:
public: // treat these as private, they are for callbacks only
void noteTimeToUpdate();
void noteTimeToNotify();
- void noteNewOptions(ICAPOptions *newOptions);
+
+ // receive either an ICAP OPTIONS response header or an abort message
+ virtual void noteIcapAnswer(HttpMsg *msg);
+ virtual void noteIcapQueryAbort(bool);
private:
// stores Prepare() callback info
void scheduleNotification();
void startGettingOptions();
+ void handleNewOptions(ICAPOptions *newOptions);
void changeOptions(ICAPOptions *newOptions);
void checkOptions();
ICAPXaction_fromData(data).noteCommRead(status, size);
}
-ICAPXaction *ICAPXaction::AsyncStart(ICAPXaction *x) {
- assert(x != NULL);
- x->self = x; // yes, this works with the current RefCount cimplementation
- AsyncCall(93,5, x, ICAPXaction::start);
- return x;
-}
-
-ICAPXaction::ICAPXaction(const char *aTypeName):
+ICAPXaction::ICAPXaction(const char *aTypeName, ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService):
+ ICAPInitiate(aTypeName, anInitiator, aService),
id(++TheLastId),
connection(-1),
commBuf(NULL), commBufSize(0),
commEof(false),
reuseConnection(true),
- connector(NULL), reader(NULL), writer(NULL), closer(NULL),
- typeName(aTypeName),
- theService(NULL),
- inCall(NULL)
+ isRetriable(true),
+ connector(NULL), reader(NULL), writer(NULL), closer(NULL)
{
debugs(93,3, typeName << " constructed, this=" << this <<
" [icapx" << id << ']'); // we should not call virtual status() here
" [icapx" << id << ']'); // we should not call virtual status() here
}
+void ICAPXaction::disableRetries() {
+ debugs(93,5, typeName << (isRetriable ? "becomes" : "remains") <<
+ " final" << status());
+ isRetriable = false;
+}
+
void ICAPXaction::start()
{
- debugs(93,3, HERE << typeName << " starts" << status());
-
- Must(self != NULL); // set by AsyncStart;
+ ICAPInitiate::start();
readBuf.init(SQUID_TCP_SO_RCVBUF, SQUID_TCP_SO_RCVBUF);
commBuf = (char*)memAllocBuf(SQUID_TCP_SO_RCVBUF, &commBufSize);
// TODO: obey service-specific, OPTIONS-reported connection limit
void ICAPXaction::openConnection()
{
+ Must(connection < 0);
+
const ICAPServiceRep &s = service();
- // TODO: check whether NULL domain is appropriate here
- connection = icapPconnPool->pop(s.host.buf(), s.port, NULL, NULL);
- if (connection >= 0) {
- debugs(93,3, HERE << "reused pconn FD " << connection);
- connector = &ICAPXaction_noteCommConnected; // make doneAll() false
- eventAdd("ICAPXaction::reusedConnection",
+ // if we cannot retry, we must not reuse pconns because of race conditions
+ if (isRetriable) {
+ // TODO: check whether NULL domain is appropriate here
+ connection = icapPconnPool->pop(s.host.buf(), s.port, NULL, NULL);
+
+ if (connection >= 0) {
+ debugs(93,3, HERE << "reused pconn FD " << connection);
+ connector = &ICAPXaction_noteCommConnected; // make doneAll() false
+ eventAdd("ICAPXaction::reusedConnection",
reusedConnection,
this,
0.0,
0,
true);
- return;
+ return;
+ }
+
+ disableRetries(); // we only retry pconn failures
}
- if (connection < 0) {
- connection = comm_open(SOCK_STREAM, 0, getOutgoingAddr(NULL), 0,
- COMM_NONBLOCKING, s.uri.buf());
+ connection = comm_open(SOCK_STREAM, 0, getOutgoingAddr(NULL), 0,
+ COMM_NONBLOCKING, s.uri.buf());
- if (connection < 0)
- dieOnConnectionFailure(); // throws
- }
+ if (connection < 0)
+ dieOnConnectionFailure(); // throws
debugs(93,3, typeName << " opens connection to " << s.host.buf() << ":" << s.port);
debugs(93,3, HERE << "pushing pconn" << status());
commSetTimeout(connection, -1, NULL, NULL);
icapPconnPool->push(connection, theService->host.buf(), theService->port, NULL, NULL);
+ disableRetries();
} else {
debugs(93,3, HERE << "closing pconn" << status());
// comm_close will clear timeout
mustStop("ICAP service connection externally closed");
}
-bool ICAPXaction::done() const
+void ICAPXaction::callEnd()
{
- // stopReason, set in mustStop(), overwrites all other conditions
- return stopReason != NULL || doneAll();
+ if (doneWithIo()) {
+ debugs(93, 5, HERE << typeName << " done with I/O" << status());
+ closeConnection();
+ }
+ ICAPInitiate::callEnd(); // may destroy us
}
bool ICAPXaction::doneAll() const
{
- return !connector && !reader && !writer;
+ return !connector && !reader && !writer && ICAPInitiate::doneAll();
}
void ICAPXaction::updateTimeout() {
* here instead of reading directly into readBuf.buf.
*/
- if (sz > 0)
+ if (sz > 0) {
readBuf.append(commBuf, sz);
- else
+ disableRetries(); // because pconn did not fail
+ } else {
+ reuseConnection = false;
commEof = true;
+ }
handleCommRead(sz);
doneReading() && doneWriting();
}
-void ICAPXaction::mustStop(const char *aReason)
+// initiator aborted
+void ICAPXaction::noteInitiatorAborted()
{
- Must(inCall); // otherwise nobody will delete us if we are done()
- Must(aReason);
- if (!stopReason) {
- stopReason = aReason;
- debugs(93, 5, typeName << " will stop, reason: " << stopReason);
- } else {
- debugs(93, 5, typeName << " will stop, another reason: " << aReason);
+ ICAPXaction_Enter(noteInitiatorAborted);
+
+ if (theInitiator) {
+ clearInitiator();
+ mustStop("initiator aborted");
}
+
+ ICAPXaction_Exit();
}
// This 'last chance' method is called before a 'done' transaction is deleted.
if (commBuf)
memFreeBuf(commBufSize, commBuf);
- debugs(93, 5, HERE << "swan sang" << status());
-}
-
-void ICAPXaction::service(ICAPServiceRep::Pointer &aService)
-{
- Must(!theService);
- Must(aService != NULL);
- theService = aService;
-}
-
-ICAPServiceRep &ICAPXaction::service()
-{
- Must(theService != NULL);
- return *theService;
-}
-
-bool ICAPXaction::callStart(const char *method)
-{
- debugs(93, 5, typeName << "::" << method << " called" << status());
-
- if (inCall) {
- // this may happen when we have bugs or when arguably buggy
- // comm interface calls us while we are closing the connection
- debugs(93, 5, HERE << typeName << "::" << inCall <<
- " is in progress; " << typeName << "::" << method <<
- " cancels reentry.");
- return false;
- }
-
- if (!self) {
- // this may happen when swanSong() has not properly cleaned up.
- debugs(93, 5, HERE << typeName << "::" << method <<
- " is not admitted to a finished transaction " << this);
- return false;
- }
-
- inCall = method;
- return true;
-}
-
-void ICAPXaction::callException(const TextException &e)
-{
- debugs(93, 2, typeName << "::" << inCall << " caught an exception: " <<
- e.message << ' ' << status());
-
- reuseConnection = false; // be conservative
- mustStop("exception");
-}
-
-void ICAPXaction::callEnd()
-{
- if (done()) {
- debugs(93, 5, typeName << "::" << inCall << " ends xaction " <<
- status());
- swanSong();
- const char *inCallSaved = inCall;
- const char *typeNameSaved = typeName;
- inCall = NULL;
- self = NULL; // will delete us, now or eventually
- debugs(93, 6, HERE << typeNameSaved << "::" << inCallSaved <<
- " ended " << this);
- return;
- } else
- if (doneWithIo()) {
- debugs(93, 5, HERE << typeName << " done with I/O" << status());
- closeConnection();
- }
+ if (theInitiator)
+ tellQueryAborted(!isRetriable);
- debugs(93, 6, typeName << "::" << inCall << " ended" << status());
- inCall = NULL;
+ ICAPInitiate::swanSong();
}
// returns a temporary string depicting transaction status, for debugging
/*
- * $Id: ICAPXaction.h,v 1.10 2007/04/06 04:50:08 rousskov Exp $
+ * $Id: ICAPXaction.h,v 1.11 2007/05/08 16:32:12 rousskov Exp $
*
*
* SQUID Web Proxy Cache http://www.squid-cache.org/
#include "comm.h"
#include "MemBuf.h"
#include "ICAPServiceRep.h"
-#include "AsyncCall.h"
+#include "ICAPInitiate.h"
class HttpMsg;
-class TextException;
/*
* The ICAP Xaction implements common tasks for ICAP OPTIONS, REQMOD, and
- * RESPMOD transactions.
- *
- * All ICAP transactions are refcounted and hold a pointer to self.
- * Both are necessary because a user need to access transaction data
- * after the transaction has finished, while a transaction may need to
- * finish after all its explicit users are gone. For safety and simplicity,
- * the code assumes that both cases can happen to any ICAP transaction.
+ * RESPMOD transactions. It is started by an ICAPInitiator. It terminates
+ * on its own, when done. Transactions communicate with Initiator using
+ * asynchronous messages because a transaction or Initiator may be gone at
+ * any time.
*/
// Note: ICAPXaction must be the first parent for object-unaware cbdata to work
-class ICAPXaction: public RefCountable
+class ICAPXaction: public ICAPInitiate
{
public:
- typedef RefCount<ICAPXaction> Pointer;
-
- // Use this to start ICAP transactions because they need a pointer
- // to self and because the start routine may result in failures/callbacks.
- static ICAPXaction *AsyncStart(ICAPXaction *x);
-
-public:
- ICAPXaction(const char *aTypeName);
+ ICAPXaction(const char *aTypeName, ICAPInitiator *anInitiator, ICAPServiceRep::Pointer &aService);
virtual ~ICAPXaction();
+ void disableRetries();
+
// comm handler wrappers, treat as private
void noteCommConnected(comm_err_t status);
void noteCommWrote(comm_err_t status, size_t sz);
void noteCommTimedout();
void noteCommClosed();
- // start handler, treat as protected and call it from the kids
- virtual void start() = 0;
- AsyncCallWrapper(93,3, ICAPXaction, start);
-
protected:
- // Set or get service pointer; ICAPXaction cbdata-locks it.
- void service(ICAPServiceRep::Pointer &aService);
- ICAPServiceRep &service();
+ virtual void start();
+ virtual void noteInitiatorAborted(); // TODO: move to ICAPInitiate
// comm hanndlers; called by comm handler wrappers
virtual void handleCommConnected() = 0;
bool parseHttpMsg(HttpMsg *msg); // true=success; false=needMore; throw=err
bool mayReadMore() const;
+
virtual bool doneReading() const;
virtual bool doneWriting() const;
bool doneWithIo() const;
-
- bool done() const;
virtual bool doneAll() const;
- void mustStop(const char *reason);
// called just before the 'done' transaction is deleted
virtual void swanSong();
// returns a temporary string depicting transaction status, for debugging
- const char *status() const;
+ virtual const char *status() const;
virtual void fillPendingStatus(MemBuf &buf) const;
virtual void fillDoneStatus(MemBuf &buf) const;
// useful for debugging
virtual bool fillVirginHttpHeader(MemBuf&) const;
+ // custom end-of-call checks
+ virtual void callEnd();
+
protected:
- Pointer self; // see comments in the class description above
const int id; // transaction ID for debugging, unique across ICAP xactions
int connection; // FD of the ICAP server connection
size_t commBufSize;
bool commEof;
bool reuseConnection;
+ bool isRetriable;
const char *stopReason;
- // asynchronous call maintenance
- bool callStart(const char *method);
- void callException(const TextException &e);
- void callEnd();
-
// active (pending) comm callbacks for the ICAP server connection
CNCB *connector;
IOCB *reader;
private:
static int TheLastId;
- ICAPServiceRep::Pointer theService;
const char *inCall; // name of the asynchronous call being executed, if any
};
// call guards for all "asynchronous" note*() methods
-
-// asynchronous call entry:
-// - open the try clause;
-// - call callStart().
-#define ICAPXaction_Enter(method) \
- try { \
- if (!callStart(#method)) \
- return;
-
-// asynchronous call exit:
-// - close the try clause;
-// - catch exceptions;
-// - let callEnd() handle transaction termination conditions
-#define ICAPXaction_Exit() \
- } \
- catch (const TextException &e) { \
- callException(e); \
- } \
- callEnd();
-
+// If we move ICAPXaction_* macros to core, they can use these generic names:
+#define ICAPXaction_Enter(method) AsyncCallEnter(method)
+#define ICAPXaction_Exit() AsyncCallExit()
#endif /* SQUID_ICAPXACTION_H */
#
# Makefile for the Squid Object Cache server
#
-# $Id: Makefile.am,v 1.179 2007/04/24 15:04:22 hno Exp $
+# $Id: Makefile.am,v 1.180 2007/05/08 16:32:11 rousskov Exp $
#
# Uncomment and customize the following to suit your needs:
#
@ICAP_LIBS@
ICAP_libicap_a_SOURCES = \
+ ICAP/AsyncJob.cc \
+ ICAP/AsyncJob.h \
ICAP/ChunkedCodingParser.cc \
ICAP/ChunkedCodingParser.h \
ICAP/ICAPClient.cc \
ICAP/ICAPClient.h \
ICAP/ICAPInitiator.cc \
ICAP/ICAPInitiator.h \
+ ICAP/ICAPInitiate.cc \
+ ICAP/ICAPInitiate.h \
ICAP/ICAPInOut.h \
+ ICAP/ICAPLauncher.cc \
+ ICAP/ICAPLauncher.h \
ICAP/ICAPConfig.cc \
ICAP/ICAPConfig.h \
ICAP/ICAPElements.cc \
projects / thirdparty / squid.git / commitdiff
