From: Cliff Woolley Date: Mon, 4 Jun 2001 15:29:53 +0000 (+0000) Subject: Further explain the custom bucket types idea for mod_file_cache X-Git-Tag: 2.0.19~145 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=3b3bdfbbdcaa35d105ee23d3beef9a279a346a3f;p=thirdparty%2Fapache%2Fhttpd.git Further explain the custom bucket types idea for mod_file_cache git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@89267 13f79535-47bb-0310-9956-ffa450edef68 --- diff --git a/STATUS b/STATUS index 10e0f21394c..182080f85c7 100644 --- a/STATUS +++ b/STATUS @@ -1,5 +1,5 @@ APACHE 2.0 STATUS: -*-text-*- -Last modified at [$Date: 2001/06/04 14:20:20 $] +Last modified at [$Date: 2001/06/04 15:29:53 $] Release: @@ -93,44 +93,39 @@ RELEASE SHOWSTOPPERS: for use on a sendfile call (because the filepointer is not used, in sendfile) it is NOT okay for threads to share a file handle for reads or writes (because the filepointer is used in reads - and writes). - - You can share a file handle across threads in Windows if you - open the file for OVERLAPPED io. A file opened for overlapped - io does not have a file pointer; each thread must manage - tracking its location in the file explicity. This is a cool - feature IMHO. APR would need to be expanded to handle - reading and writing to files opened for overlapped io - (specifically to manage a per-thread file pointer and handle - async io events internal to APR). This doesn't fix the problem - under *ix though. + and writes). This may also be broken on a few quirky platforms + whose sendfile() actually uses the file pointer. Jeff was looking + into that. Potential Solutions: 1. We either need to ensure that a cached file handle is only - used on a sendfile call (Bill prefers this solution. I think.). - 2. Cliff Wooley has some ideas about special purpose buckets - that would recognise when an apr_file_t is cached (thus shared) - and do the right thing (locking, whatever) to ensure that the - filepointer does not get trashed. - 3.? - - * There is a big leak of MMAPs that occurs in modules such as - mod_file_cache that needs to be taken care of. Several - potential solutions were tossed about on new-httpd and apr-dev - in late April/early May. Perhaps the cleanest proposed - approaches were the following: - - dup the cached apr_file_t into the request pool on each - request so that the MMAP is created in the request pool - - just cache the FD, not the whole apr_file_t. Build the apr_file_t - and its apr_bucket_file in the request pool. - - add an extra parameter to apr_bucket_file_create() which is the - pool that an MMAP (if any) for that file should be created in - [Bill says that this memory leak has been fixed in - mod_file_cache. We now create a new apr_file_t in the request pool that - we send on the ap_send_fd in mod_file_cache. There are other significant - problems (see above)] - - + used on a sendfile call (Bill prefers this solution. I think.). + 2. Cliff Woolley has some ideas about a custom bucket type + that would recognise when an apr_file_t is cached (thus shared + across threads) and do the right thing to ensure that the + filepointer does not get trashed. The idea here is that the + "cacheable" file bucket would never allow its file handle to be + read by any method OTHER than sendfile. Attempts to do otherwise + would result in the file being re-opened for sync io into the + request pool, and the new file handle would get put into a regular + file bucket which would then be read. This method thus incorporates + the spirit of solution #1 while having a fall-back protection + scheme. Note that mod_file_cache can still attempt to + be discriminatory and DECLINE requests that it believes will + result in an apr_bucket_read() call; this solution just allows + it to not worry TOO much about missing some cases. Note that + no locking is involved, and in the worst case (ie, when the cached + file bucket is read) we are no worse off performance-wise than + if we'd just DECLINED in the first place. Patch forthcoming. + 3. You can share a file handle across threads in Windows if you + open the file for OVERLAPPED io. A file opened for overlapped + io does not have a file pointer; each thread must manage + tracking its location in the file explicity. This is a cool + feature IMHO. APR would need to be expanded to handle + reading and writing to files opened for overlapped io + (specifically to manage a per-thread file pointer and handle + async io events internal to APR). This doesn't fix the problem + under *ix though. * There is a bug in how we sort some hooks, at least the pre-config hook. The first time we call the hooks, they are in the correct