From: antiago Garcia Mantinan Date: Sat, 1 Nov 2014 00:43:23 +0000 (-0700) Subject: Docs: convert purge tool README to man.1 page X-Git-Tag: merge-candidate-3-v1~516 X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=f655a258526c8a02a9a91ebcde3f75ffd4fe6184;p=thirdparty%2Fsquid.git Docs: convert purge tool README to man.1 page Initial conversion performed by Santiago Garcia Mantinan of Debian. With po4a adjustments by Amos Jeffries. --- diff --git a/tools/purge/Makefile.am b/tools/purge/Makefile.am index 5b87ebc4e2..769e66e060 100644 --- a/tools/purge/Makefile.am +++ b/tools/purge/Makefile.am @@ -9,6 +9,7 @@ include $(top_srcdir)/src/Common.am if !ENABLE_WIN32SPECIFIC bin_PROGRAMS = purge +man_MANS = purge.1 endif purge_SOURCES = \ @@ -30,4 +31,4 @@ purge_LDADD = \ $(COMPAT_LIB) \ $(XTRA_LIBS) -EXTRA_DIST = README +EXTRA_DIST = purge.1 diff --git a/tools/purge/README b/tools/purge/README deleted file mode 100644 index 251a4d1c29..0000000000 --- a/tools/purge/README +++ /dev/null @@ -1,283 +0,0 @@ -purge -===== - -The purge tool is a kind of magnifying glass into your squid-2 cache. You -can use purge to have a look at what URLs are stored in which file within -your cache. The purge tool can also be used to release objects which URLs -match user specified regular expressions. A more troublesome feature is the -ability to remove files squid does not seem to know about any longer. - - USE AT YOUR OWN RISK! NO GUARANTEES, WHATSOEVER! DON'T BLAME US! - YOU HAVE BEEN WARNED! - - - -compilation -=========== - -Purge has been successfully compiled under the following OSes: - - SYSTEM g++ native - ------ --- ------ - Solaris 2.7 yes CC - IRIX 6.5 yes CC -n32 - Linux 2.0.36 yes (g++ IS native) - FreeBSD 4.x yes gmake port must be installed - (g++ IS supported) - -The recent move of the Linux community towards glibc2 may cause some -troubles, though. The compilation requires GNU make, no other make will work -correctly. The source distribution contains all files checked into the -revision control repository. Therefore, you will need to install GNU RCS -first (which in turn needs the GNU diffutils). - -The repository also contains the prototypical Perl implementation. The user -interface in the C++ implementation changed a little when compared to the -Perl one. You will have to state at least one regular expression for purge -to start working. Also, printing the complete cache URLs, you will need to -specify the "-e ." regular expression. - -In order to compile the purge tool, untar the source distribution and -change into the purge directory. With RCS and GNU make installed, just say -"make". GNU make will automagically retrieve all necessary files from the -repository and create the binary. - -Systems not stated above will need to retrieve the makefile (use "co -l -Makefile" for this) and add their own platform specific definitions to -section [2] in the makefile. - - - -squid preparation -================= - -In order to use purge for real PURGEs, you will have to enable this feature -in squid. By default, PURGE is disabled. You should watch closely for whom -you enable the PURGE ability, otherwise total stranger just might wipe your -cache content. The following lines will need to be added to your squid.conf -(you may want to add further networks to the src_local ACL): - - acl purge method PURGE - acl src_local src 127.0.0.0/8 - http_access allow purge src_local - http_access deny purge - -Reconfigure or restart (preferred) your squid after changing the -configuration file. - - - -modes of operation -================== - -Usage: purge [-a] [-c cf] [-d l] [-(f|F) fn | -(e|E) re] [-p h[:p]] - [-P #] [-s] [-v] [-C dir [-H]] [-n] - - -a display a little rotating thingy to indicate that I am alive (tty only). - -c c squid.conf location, default "/usr/local/squid/etc/squid.conf". - -C dir base directory for content extraction (copy-out mode). - -d l debug level, an OR of different debug options. - -e re single regular expression per -e instance (use quotes!). - -E re single case sensitive regular expression like -e. - -f fn name of textfile containing one regular expression per line. - -F fn name of textfile like -f containing case sensitive REs. - -H prepend HTTP reply header to destination files in copy-out mode. - -n do not fork() when using more than one cache_dir. - -p h:p cache runs on host h and optional port p, default is localhost:3128. - -P # if 0, just print matches; otherwise OR the following purge modes: - 0x01 really send PURGE to the cache. - 0x02 remove all caches files reported as 404 (not found). - 0x04 remove all weird (inaccessible or too small) cache files. - 0 and 1 are recommended - slow rebuild your cache with other modes. - -s show all options after option parsing, but before really starting. - -v show more information about the file, e.g. MD5, timestamps and flags. - ---- &< snip, snip --- - --a is a kind of "i am alive" flag. It can only be activated, if - your stdout is a tty. If active, it will display a little - rotating line to indicate that there is actually something - happening. You should not use this switch, if you capture - your stdout in a file, or if your expression list produces - many matches. The -a flag is also incompatible with the - (default) multi cache_dir mode. - - default: off - See also: -n - --c cd CHANGED! - this option lets you specify the location of the squid.conf file. - Purge now understands about more than one cache_dir, and does so - by parsing Squid's configuration file. It knows about both ways - of Squid-2 cache_dir specifications, and will automatically try - to use the correct one. - - default: /usr/local/squid/etc/squid.conf - --C cd if you want to rescue files from your cache, you need to specify - the directory into which the files will be copied. Please note - that purge will try to establish the original server's directory - structure. This switch also activates copy-out mode. Please do - not use copy-out mode with any purge mode (-P) other than 0. - - For instance, if you specified "-C /tmp", Purge will try to - recreate /tmp/www.server.1/url/path/file, and so forth. - - default: off - See also: -H, -P - --d l lets you specify a debug level. Differents bits are reserved for - different output. - - default: 0 - --e re the "-e" options let you specify one regular expression at the --E re commandline. This is useful, if there is only a handful you - want to check. Please remember to escape the shell metachars - used in your regular expression. The use of single quotes - around your expression is recommended. The capital letter - version works case sensitive, the lower caps version does not. - - default: (no default) - --f fn if you have more than a handful of expression, or want to check --F fn the same set at regular intervals, the file option might be more - useful to you. Each line in the text file will be regarded as - one regular expression. Again, the capital letter version works - case sensitive, the lower caps version does not. - - default: (no default) - --H if in copy-out mode (see: -C), you can specify to keep the - HTTP Header in the recreated file. - - default: off - See also: -C - --n by specifying the "-n" switch, you will tell Purge to process - one cache_dir after another, instead of doing things in parallel. - If you have more than one cache_dir in your configuration, - Purge will fork off a worker process for each cache_dir to - do the checks for optimum speed - assuming a decently designed - cache. Since parallel execution will put quite some load on the - system and its controllers, it is sometimes preferred to use - less resources, though it will take longer. - - default: parallel mode for more than one cache_dir - --p h[:p] Some cache admins (i.e. me) use a different port than 3128. The - purge tool will need to connect to your cache in order to send - the PURGE request (see -P). This option lets you specify the - host and port to connect to. The port is optional. The port - can be a name (check your /etc/services) or number. It is - separated from the host name portion by a single colon, no - spaces allowed. - - default: localhost:3128 - --P # If you want to do more than just print your cache content, you - will need to specify this option. Each bit is reserved for a - different action. Only the use of the LSB is recommended, the - rest should be considered experimental. - - no bit set: just print - bit#0 set: send PURGE for matches - bit#1 set: unlink object file for 404 not found PURGEs - bit#2 set: unlink weird object files - - If you use a value other than 0 or 1, you will need to slow - rebuild your cache content. A warning message will remind you - of that. If you use bit#1, all unsuccessful PURGEs will result - in the object file in your cache directory to be removed, because - squid does not seem to know about it any longer. Beware that the - asyncio might try to remove it after the purge tool, and thus - complains bitterly. Bit#1 only makes sense, if Bit#0 is also - set, otherwise it has no effect (since the HTTP status 404 is - never returned). - - Bit#2 is reserved for strange files which do not even contain - a URL. Beware that these files may indicate a new object squid - currently intends to swap onto disk. If the file suddenly went - away, or is removed when squid tries to fetch the object, it - will complain bitterly. You must slow rebuild your cache, if - you use this option. - - It is recommended that if you dare to use bit#1 or bit#2, you - should only grant the purge tool access to your squid, e.g. - move the HTTP and ICP listening port of squid to a different - non-standard location during the purge. - - default: 0 (just print) - --s If you specify this switch, all commandline parameters will be - shown after they were parsed. - - default: off - --v be verbose in the things reported about the file. See the output - section below. - - -output -====== - -In regular mode, the output of purge consists of four columns. If the -URL contains not encoded whitespaces, it may look as if there are more -columns, but the last one is the URI. - - # name meaning - - ------ ----------------------------------------------------------- - 1 file name of cache file eximed which matches the re. - 2 status return result of purge request, " 0" in print mode. - 3 size object size including stored headers, not file size. - 4 uri perceived uri - -Example for non-verbose output in print-mode: - -/cache3/00/00/0000004A 0 5682 http://graphics.userfriendly.org/images/slovenia.gif - -In verbose mode, additional columns are inserted before the uri. Time -stamps are reported using hexadecimal notation, and Squid's standard -for reporting "no such timestamp" == -1, and "unparsable timestamp" == -2. - - # name meaning - - ------ ----------------------------------------------------------- - 1 file name of cache file eximed which matches the re. - 2 status return result of purge request, " 0" in print mode "-P 0". - 3 size object size including stored headers, not file size. - 4 md5 MD5 of URI from file, or "(no_md5_data_available)" string. - 5 ts UTC of Value of Date: header in hex notation - 6 lr UTC of last time the object was referenced - 7 ex UTC of Expires: header - 8 lr UTC of Last-Modified: header - 9 flags Value of objects flags field in hex, see: Programmers Guide -10 refcnt number of times the object was referenced. -11 uri STORE_META_URL uri or "strange_file" - -Example for verbose output in print-mode: - -/cache1/00/00/000000B7 0 406 7CFCB1D319F158ADC9CFD991BB8F6DCE 397d449b 39bf677b ffffffff 3820abfc 0460 1 http://www.netscape.com/images/nc_vera_tile.gif - - -limitations -=========== - -o Purge does not slow rebuild the cache for you. - -o It is still relatively slow, especially if your machine is low on memory -and/or unable to hold all OS directory cache entries in main memory. - -o should never be used on "busy" caches with purge modes higher than 1. - - -TODO -==== - -1) use the stat() result on weird files to have a look at their ctime and - mtime. If they are younger than, lets say 30 seconds, they were just - created by squid and should not be removed. - -2) Add a query before purging objects or removing files, and add another - option to remove nagging for the experienced user. - -3) The reported object size may be off by one. diff --git a/tools/purge/purge.1 b/tools/purge/purge.1 new file mode 100644 index 0000000000..11cb99544c --- /dev/null +++ b/tools/purge/purge.1 @@ -0,0 +1,289 @@ +.if !'po4a'hide' .TH purge 8 "October 12, 2014" "" "" +. +.SH NAME +purge \- magnifying glass into your squid cache +. +.SH SYNOPSIS +.if !'po4a'hide' .B purge +.if !'po4a'hide' .B " [\-a] [\-c cf] [\-d l] [\-(f|F) fn | \-(e|E) re] [\-p h[:p]] [\-P #] [\-s] [\-v] [\-C dir [\-H]] [\-n]" +. +.SH DESCRIPTION +.PP +.B purge +is used to have a look at what URLs are stored in which file within +your cache. The +.B purge +tool can also be used to release objects which URLs +match user specified regular expressions. A more troublesome feature is the +ability to remove files +.B squid +does not seem to know about any longer. +.PP +This is a tool for expert usage only, use it under your own responsability. +. +.SH OPTIONS +.if !'po4a'hide' .TP 12 +.if !'po4a'hide' .B \-a +a kind of "i am alive" flag. It can only be activated, if +your stdout is a tty. If active, it will display a little +rotating line to indicate that there is actually something +happening. You should not use this switch if you capture +your stdout in a file or if your expression list produces +many matches. The \-a flag is also incompatible with the +(default) multi cache_dir mode. +.br +default: off See also: \-n +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-c cd +this option lets you specify the location of the squid.conf file. +Purge understands about more than one cache_dir, and does so +by parsing squid.conf. It knows about both ways of Squid\-2 cache_dir +specifications, and will automatically try to use the correct one. +.br +default: /usr/local/squid/etc/squid.conf +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-C cf +if you want to rescue files from your cache, you need to specify +the directory into which the files will be copied. Please note +that purge will try to establish the original server directory +structure. This switch also activates copy\-out mode. Please do +not use copy\-out mode with any purge mode (\-P) other than 0. +.br +For instance, if you specified "\-C /tmp", purge will try to +recreate /tmp/www.server.1/url/path/file, and so forth. +.br +default: off See also: \-H, \-P +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-d l +lets you specify a debug level. Differents bits are reserved for +different output. +.br +default: 0 +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-e|\-E re +Specify one regular expression to be searched for in the cache. +This is useful if there is only a handful of objects you +want to check. Please remember to escape the shell meta characters +used in your regular expression. The use of single quotes +around your expression is recommended. The capital letter +version works case sensitive, the lower caps version does not. +.br +default: (no default) +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-f|\-F fn +if you have more than a handful of expressions, or want to check +the same set at regular intervals, the file option might be more +useful to you. Each line in the text file will be regarded as +one regular expression. Again, the capital letter version works +case sensitive, the lower caps version does not. +.br +default: (no default) +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-H +if in copy\-out mode (see: \-C), you can specify to keep the +HTTP Header in the recreated file. +.br +default: off See also: \-C +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-n +tell purge to process one cache_dir after another, +instead of doing things in parallel. +If you have more than one cache_dir in your configuration +purge will fork off a worker process for each cache_dir to +do the checks for optimum speed, assuming a decently designed +cache. Since parallel execution will put quite some load on the +system and its controllers, it is sometimes preferred to use +less resources, though it will take longer. +.br +default: parallel mode for more than one cache_dir +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-p h[:p] +Some cache admins use a different port than 3128. The +purge tool will need to connect to your cache in order to send +the PURGE request (see \-P). This option lets you specify the +host and port to connect to. The port is optional. The port +can be a name (check your /etc/services) or number. It is +separated from the host name portion by a single colon, no +spaces allowed. +.br +default: localhost:3128 +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-P # +If you want to do more than just print your cache content, you +will need to specify this option. Each bit is reserved for a +different action. Only the use of the LSB is recommended, the +rest should be considered experimental. +.PP +.RS +.B no bit set: just print +.br +.B bit#0 set: send PURGE for matches +.br +.B bit#1 set: unlink object file for 404 not found PURGEs +.br +.B bit#2 set: unlink weird object files +.RE +.PP +If you use a value other than 0 or 1, you will need to slow +rebuild your cache content. A warning message will remind you +of that. If you use bit#1, all unsuccessful PURGEs will result +in the object file in your cache directory to be removed, because +squid does not seem to know about it any longer. Beware that the +asyncio might try to remove it after the purge tool, and thus +complains bitterly. Bit#1 only makes sense, if Bit#0 is also +set, otherwise it has no effect (since the HTTP status 404 is +never returned). +.PP +Bit#2 is reserved for strange files which do not even contain +a URL. Beware that these files may indicate a new object squid +currently intends to swap onto disk. If the file suddenly went +away, or is removed when squid tries to fetch the object, it +will complain bitterly. You must slow rebuild your cache, if +you use this option. +.PP +It is recommended that if you dare to use bit#1 or bit#2, you +should only grant the purge tool access to your squid, e.g. +move the HTTP and ICP listening port of squid to a different +non\-standard location during the purge. +.br +default: 0 (just print) +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-s +If you specify this switch, all commandline parameters will be +shown after they were parsed. +.br +default: off +. +.if !'po4a'hide' .TP +.if !'po4a'hide' .B \-v +be verbose in the things reported about the file. See the output +section below. +. +.SH CONFIGURATION +.PP +In order to use +.B purge +to affect a running proxy with PURGE method, you will have to enable this feature +in squid.conf. By default, PURGE is disabled. You should watch closely for whom +you enable the PURGE ability, otherwise a total stranger just might wipe your +cache content. Lines similar to the following will need to be added to your squid.conf: +.if !'po4a'hide' .PP +.if !'po4a'hide' .RS +.if !'po4a'hide' .P +.if !'po4a'hide' .B acl purge method PURGE +.if !'po4a'hide' .br +.if !'po4a'hide' .B http_access allow localhost purge +.if !'po4a'hide' .br +.if !'po4a'hide' .B http_access deny purge +.if !'po4a'hide' .RE +.PP +Reconfigure or restart (preferred) your squid after changing the +configuration file. +. +.SH OUTPUT +.PP +In regular mode, the output of purge consists of four columns. If the +URL contains not encoded whitespaces, it may look as if there are more +columns, but the last one is the URI. +. + # name meaning + = ====== =========================================================== + 1 file name of cache file eximed which matches the regular expression. + 2 status return result of purge request, " 0" in print mode. + 3 size object size including stored headers, not file size. + 4 uri perceived uri + +Example for non\-verbose output in print\-mode: + +/cache3/00/00/0000004A 0 5682 http://graphics.userfriendly.org/images/slovenia.gif +. +.PP +In verbose mode, additional columns are inserted before the uri. Time +stamps are reported using hexadecimal notation, and Squid's standard +for reporting "no such timestamp" == \-1, and "unparsable timestamp" == \-2. +. + # name meaning + = ====== =========================================================== + 1 file name of cache file eximed which matches the re. + 2 status return result of purge request, " 0" in print mode "\-P 0". + 3 size object size including stored headers, not file size. + 4 md5 MD5 of URI from file, or "(no_md5_data_available)" string. + 5 ts UTC of Value of Date: header in hex notation + 6 lr UTC of last time the object was referenced + 7 ex UTC of Expires: header + 8 lr UTC of Last\-Modified: header + 9 flags Value of objects flags field in hex, see: Programmers Guide +10 refcnt number of times the object was referenced. +11 uri STORE_META_URL uri or "strange_file" +. +Example for verbose output in print\-mode: +. +/cache1/00/00/000000B7 0 406 7CFCB1D319F158ADC9CFD991BB8F6DCE 397d449b 39bf677b ffffffff 3820abfc 0460 1 http://www.netscape.com/images/nc_vera_tile.gif +. +.SH KNOWN ISSUES +Purge does not slow rebuild the cache for you. +.PP +It is still relatively slow, especially if your machine is low on memory +and/or unable to hold all OS directory cache entries in main memory. +.PP +Should never be used on "busy" caches with purge modes higher than 1. +. +.SH TODO +.PP +1) use the stat() result on weird files to have a look at their ctime and +mtime. If they are younger than, lets say 30 seconds, they were just +created by +.B squid +and should not be removed. +.PP +2) Add a query before purging objects or removing files, and add another +option to remove nagging for the experienced user. +.PP +3) The reported object size may be off by one. +. +.SH AUTHOR +This program and manual was written by +.if !'po4a'hide' .B Santiago Garcia Mantinan +.if !'po4a'hide' .I Amos Jeffries +.PP +Based on original squidpurge README. +. +..SH COPYRIGHT +.PP + * Copyright (C) 1996-2014 The Squid Software Foundation and contributors + * + * Squid software is distributed under GPLv2+ license and includes + * contributions from numerous individuals and organizations. + * Please see the COPYING and CONTRIBUTORS files for details. +. +.SH QUESTIONS +Questions on the usage of this program can be sent to the +.I Squid Users mailing list +.if !'po4a'hide' +. +.SH REPORTING BUGS +See http://wiki.squid-cache.org/SquidFaq/BugReporting for details of what you need to include with your bug report. +.PP +Report bugs or bug fixes using http://bugs.squid-cache.org/ +.PP +Report serious security bugs to +.I Squid Bugs +.PP +Report ideas for new improvements to the +.I Squid Developers mailing list +.if !'po4a'hide' +. +.SH SEE ALSO +.if !'po4a'hide' .BR squid "(8), " +.if !'po4a'hide' .BR squidclient "(1)" +.if !'po4a'hide' .BR cachemgr.cgi "(8)"