wessels [Tue, 10 Feb 1998 07:55:03 +0000 (07:55 +0000)]
Changes to accomodate disk write failures:
0) The problem: file_close() doesn't always close the file immediately;
i.e. when there are pending buffers to write. However, after
calling file_close, we were also calling storeUnlockObject and
the lock count could become zero then. Bad things would happen
if we got a callback from a file_write after the lock count went
to zero. We need a way to prevent the file_write callback from
happening when we "close" the swapout FD.
1) the file_write() callback data must now be in the callback
database (cbdata). We can't add StoreEntry's to cbdata for a couple
of reasons. So now we use the swapout_ctrl_t structure for the
callback data. Previously this was used only for the file_open
process, but now we keep it around for the entire swapout duration.
A new element MemObject->swapout.ctrl points to this data strucutre
so we can re-access it from storeCheckSwapOut.
2) Changed the way write errors are handled by diskHandleWrite.
If there is no callback function, now we exit with a fatal
message under the assumption that the file in question is a log
file or IPC pipe. Otherwise, we flush all the pending write
buffers (so we don't see multiple repeated write errors from
the same descriptor) and let the upper layer decide how to handle
the failure.
3) Fixes to storeDirWriteCleanLogs. A write failure was leaving some
empty swap.state files, even though it tells us that its "not
replacing the file." Don't flush/rename logs which we have
prematurely closed due to write failures, indiciated by
fd[dirn] == -1. Close these files LAST, not before renaming.
wessels [Sat, 7 Feb 1998 00:47:07 +0000 (00:47 +0000)]
Fixed store_rebuild handling of SWAP_LOG_DEL. Don't want to call
storeReleaseRequest() directly, because it won't free up the fileno
and other things. Calling storeRelease will just call storeReleaseRequest.
Also don't write the ENTRY_SPECIAL entries to the swap.state log.
wessels [Thu, 5 Feb 1998 06:41:00 +0000 (06:41 +0000)]
- Take Stewarts MONOTONIC_STORE patch, remove the #else code
- create storeUnlinkFileno() to consolidate some USE_ASYNC_IO #ifdefs
- added storeKeyInit()
- added storeKeyNull()
- Fixed storeRebuildFromDirectory() FUM bug.
- Removed unused fileno stack stuff. replaced by MONOTONIC_STORE changes
- Fixed storeClientReadBody() and storeClientReadHeaders() to call
httpParseReplyHeaders()
- Modified storeDirOpenTmpSwapLog() to return whether or not the
existing swap.state file has any entries, so we know to use
storeRebuildFromDirectory.
wessels [Wed, 4 Feb 1998 07:18:00 +0000 (07:18 +0000)]
- Changed swap.state to be binary
- Added Stew's transaction logging to storeDirSwapLog().
- Munged store_rebuild.c to support rebuilding one cache_dir from
swap.state, and another from walking the directory.
wessels [Tue, 3 Feb 1998 14:35:25 +0000 (14:35 +0000)]
Started working on swap headers and meta data. I changed the Get/Add
routines to something which Packs/Unpacks the whole buffer to/from a
linked list of TLV structures. The hard part now is that the swapin
and swapout routines most certainly need to be modified to have
special read/write meta header code.
wessels [Tue, 3 Feb 1998 04:14:49 +0000 (04:14 +0000)]
Stewart Forster Jumbo Patch
- Completely rewritten aiops.c that creates and manages a pool of
threads so thread creation overhead is eliminated
- Lots of extra debugging
- Lots of mods to store.c to detect and cancel outstanding ASYNC ops.
Code is not proven exhaustive and there are definately still cases
to be found where outstanding disk ops aren't cancelled properly.
- Changes to call interface to a few routines to support disk op
`tagging', so operations can be cleanly cancelled on store_abort()s.
- Implementation of swap.state files as transaction logs. Removed
objects are now noted with a negative object size. This allows
reliatively clean rebuilds from non-clean shutdowns.
- Now that the swap.state files are transaction logs, there's now no
need to validate by stat()ing. All the validation procedure does
is now just set the valid bit AFTER all the swap.state files have
been read, because by that time, only valid objects can be left.
Object still need to be marked invalid when reading the swap.state
file because there's no guarantee the file has been retaken or
deleted.
- An fstat() call is now added after every storeSwapInFileOpened() so
object sizes can be checked. Added code to storeRelease() the
object if the sizes don't match.
- #defining USE_ASYNC_IO now uses the async unlink() rather than
unlinkd()
- #defining MONOTONIC_STORE will support the creation of disk objects
clustered into directories. This GREATLY improves disk performance
(factor of 3) over old `write-over-old-object' method. If using the
MONOTONIC_STORE, the {get/put}_unusedFileno stack stuff is disabled.
This is actually a good thing and greatly reduces the risk of serving
up bad objects.
- Fixed unlink() in storeWriteCleanLogs to be real unlink() rather
than ASYNC/unlinkd unlinks. swap.state.new files were being removed
just after they were created due to delayed unlinks.
- Disabled various assertions and made these into debug warning
messages to make the code more stable until the bugs can be tracked
down.
- Added most of Michael O'Reilly's patches which included many bug
fixes. Ask him for full details.
- Moved aio_check_callbacks in comm_{poll|select}(). It was called
after the fdset had been built which was wrong because the callbacks
were changing the state of the read/write handlers prior to the
poll/select() calls.
wessels [Tue, 3 Feb 1998 02:49:53 +0000 (02:49 +0000)]
From: Henrik Nordstrom <hno@hem.passagen.se>
Squid-1.2.beta11: Support for NLST listings
Format NLST listings, and inform the user that the NLST
(plain) format is available when we find a LIST listing
that we don't understand.
Squid-1.2.beta11: SIZE on binary FTP transfers
Use SIZE on Binary transfers, and not ASCII. The condition
was inversed, making squid use SIZE on ASCII transfers.
Squid-1.2.beta11: Enable virtual and Host: based acceleration in
order to be able to use Squid as a transparent proxy without
breaking either virtual servers or clients not sending Host:
header the order of the virtual and Host: based acceleration
needs to be swapped, giving Host: a higher precendence than
virtual host.
Squid-1.2.beta11: Use memmove/bcopy as detected by configure
Some systems does not have memmove, but have the older bcopy
implementation.
wessels [Tue, 3 Feb 1998 02:39:21 +0000 (02:39 +0000)]
From: Masashi Fujita <objectx@bandit.co.jp>
1) --enable-acltree message displayed while running configure
seems to be reveresed (typo?).
2) --enable-dlmalloc seems to be of no effect.
(src/Makefile doesn't refer @LIB_MALLOC@.)
3) Could not build squid outside of source tree.
wessels [Sat, 31 Jan 1998 12:31:52 +0000 (12:31 +0000)]
- Combined various interprocess communication setup functions
into ipcCreate().
- Removed some leftover ICP_HIT_OBJ things.
- Removed cacheinfo and proto_count() and friends; these are to
be replaced in functionality by StatCounters and 5/60 minute
average views via cachemgr.
Changes to squid-1.2.beta11 (Jan 6, 1998):