From: wessels <> Date: Wed, 11 Feb 1998 06:49:49 +0000 (+0000) Subject: update X-Git-Tag: SQUID_3_0_PRE1~4117 X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=1e6dfd6d361a121d6371bf9b4f7a53b2c9add6aa;p=thirdparty%2Fsquid.git update --- diff --git a/doc/Programming-Guide/prog-guide.sgml b/doc/Programming-Guide/prog-guide.sgml index fa8d151c3a..e1a2be4101 100644 --- a/doc/Programming-Guide/prog-guide.sgml +++ b/doc/Programming-Guide/prog-guide.sgml @@ -1,7 +1,7 @@
Squid v1.2 Programmers Guide -Duane Wessels, Kostas Anagnostakis, Alex Rousskov, Squid Developers +Duane Wessels, Squid Developers Squid is a WWW Cache application developed by the National Laboratory @@ -12,6 +12,8 @@ source code and is intended to be used by others who wish to customize or improve it. + + Introduction @@ -57,15 +59,17 @@ Function names and file names will be written in a courier font, such as The Big Picture +The Big Picture

Squid consists of the following major components -Client Side +Client Side + +

+ - These routines exist in Server Side +Server Side

- These routines exist in various source files, namely + These routines are responsible for forwarding cache misses to other servers, depending on the protocol. Cache misses may be forwarded to either origin servers, or other proxy caches. @@ -102,7 +109,20 @@ Squid consists of the following major components between the two is that the SSL module sends a special ``connection established'' message. -Storage Manager +Storage Manager + +

+ The Storage Manager is the glue between client and server sides. @@ -136,56 +156,59 @@ Squid consists of the following major components PUT request, this process works in reverse. Server-side functions are notified when additional data is read from the client. -

- The storage manager code resides in a number of files: - Peer Selection -Peer Selection +

+ - The functions in Access Control +Access Control + +

+ - The functions in Network Communication +Network Communication

- The file + These are the routines for communicating over TCP and UDP network sockets. Here is where sockets are opened, closed, read, and written. In addition, note that the heart of Squid (File/Disk I/O +File/Disk I/O + +

+ - Routines for reading and writing disk files (and FIFOs) are - implemented in Other Components - Neighbors +

+ Maintains the list of neighbor caches. Sends and receives ICP messages to neighbors. Decides which neighbors to @@ -205,6 +230,10 @@ Squid consists of the following major components IP/FQDN Cache +

+ A cache of name-to-address and address-to-name lookups. These are hash tables keyed on the names and addresses. @@ -214,6 +243,10 @@ Squid consists of the following major components Cache Manager +

+ This provides access to certain information needed by the cache administrator. A companion program, Network Measurement Database +

+ In a number of situation, Squid finds it useful to know the estimated network round-trip time (RTT) between itself and @@ -248,6 +285,10 @@ Squid consists of the following major components Redirectors +

+ Squid has the ability to rewrite requests from clients. After checking the access controls, but before checking for cache hits, @@ -259,6 +300,10 @@ Squid consists of the following major components Autonomous System Numbers +

+ Squid supports Autonomous System (AS) numbers as another access control element. The routines in Asynchronous I/O Operations +

+ These routines in Configuation File Parsing +

+ The primary configuration file specification is in the file Callback Data Database +

+ Squid's extensive use of callback functions makes it very susceptible to memory access errors. Care must be taken @@ -294,6 +354,10 @@ Squid consists of the following major components Debugging +

+ Squid includes extensive debugging statements to assist in tracking down bugs and strange behaviour. Every debug statement @@ -311,6 +375,10 @@ Squid consists of the following major components Error Generation +

+ The routines in Event Queue +

+ The routines in Filedescriptor Managment +

+ + Here we track the number of filedescriptors in use, and the + number of bytes which has been read from or written to each + file descriptor. + + Hashtable Support +

+ + These routines implement generic hash tables. A hash table + is created with a function for hashing the key values, and a + function for comparing the key values. + HTTP Anonymization +

+ + These routines support anonymizing of HTTP requests leaving + the cache. Either specific request headers will be removed + (the ``standard'' mode), or only specific request headers + will be allowed (the ``paranoid'' mode). + Internet Cache Protocol +

+ + Here we implement the Internet Cache Protocol. This + protocol is documented in the RFC 2186 and RFC 2187. + The bulk of code is in the Ident Lookups +

+ + These routines support RFC 931 ``Ident'' lookups. An ident + server running on a host will report the user name associated + with a connected TCP socket. Some sites use this facility for + access control and logging purposes. + Memory Management +

+ + These routines allocate and manage pools of memory for + frequently-used data structures. When the Multicast Support +

+ + Currently, multicast is only used for ICP queries. The + routines in this file implement joining a UDP + socket to a multicast group (or groups), and setting + the multicast TTL value on outgoing packets. + Persistent Server Connections +

+ + These routines manage idle, persistent HTTP connections + to origin servers and neighbor caches. Idle sockets + are indexed in a hash table by their socket address + (IP address and port number). Up to 10 idle sockets + will be kept for each socket address, but only for + 15 seconds. After 15 seconds, idle socket connections + are closed. + Refresh Rules -Request Redirection + +

+ + These routines decide wether a cached object is stale or fresh, + based on the SNMP Support +

+ + These routines implement SNMP for Squid. At the present time, + we have made almost all of the cachemgr information avaialble + via SNMP. + URN Support +

+ +We are experimenting with URN support in Squid version 1.2. Note, +we're not talking full-blown generic URN's here. This is primarily +targeted towards using URN's as an smart way of handling lists of +mirror sites. For more details, please see +. + External Programs dnsserver +

+ Because the standard pinger +

+ + Although it would be possible for Squid to send and recieve + ICMP messages directly, we use an external process for + two important reasons: + + Because squid handles many filedescriptors simultaneously, + we get much more accruate RTT measurements when ICMP is + handled by a separate process. + Superuser priveleges are required to send and receive + ICMP. Rather than require Squid to be started as root, + we prefer to have the smaller and simpler + unlinkd +

+ + The redirector -Function Sequence of a Typical Request +

+ - - asciiHandleConn - clientReadRequest - parseHttpRequest - clientAccessCheck -- clientAccessCheckDone - redirectStart -- clientRedirectDone - icpProcessRequest - icpProcessMISS - protoDispatch -- protoDispatchDNSHandle - protoStart - proxyhttpStart - httpConnect -- httpConnectDone - httpSendRequest -- httpSendComplete - - while (data arriving from server) { - httpReadReply - storeAppend - InvokeHandlers - icpHandleStore - icpSendMoreData - } - - storeComplete - comm_close(server socket) - httpStateFree - comm_close(client_socket) - icpStateFree - + A redirector process reads URLs on stdin and writes (possibly + changed) URLs on stdout. It is implemented as an external + process to maximize flexibility. +Sequence of a Typical Request + +

+ + +A client connetion is accepted by the +The access controls are checked. The client-side builds an +ACL state data structure and registers a callback function +for notification when access control checking is completed. + + +After the access controls have been verified, the client-side looks for +the requested object in the cache. If is a cache hit, then the +client-side registers its interest in the +The request-forwarding process begins with +When the ICP replies (if any) have been processed, we end up +at +The HTTP module first opens a connection to the origin server +or cache peer. If there is no idle persistent socket available, +a new connection request is given to the Network Communication +module with a callback function. The +When a TCP connection has been established, HTTP builds a request +buffer and submits it for writing on the socket. It then registers +a read handler to receive and process the HTTP reply. + + +As the reply is initially received, the HTTP reply headers are +parsed and placed into a reply data structure. As reply data +is read, it is appended to the +As the client-side is notified of new data, it copies the data +from the StoreEntry and submits it for writing on the client socket. + + +As data is appended to the +When the HTTP module finishes reading the reply from the upstream +server, it marks the +When the client-side has written all of the object data, it unregisters +itself from the The Main Loop: -Data Structures -Main Config + +Processing Client Requests Storage Manager - -IP Cache + +Forwarding Selection + + +IP Cache and FQDN Cache Introduction @@ -561,6 +838,9 @@ according to the ICP + +Network Measurement Database + Error Pages @@ -622,8 +902,12 @@ according to the - In this case, Cache Manager