From: robertc <> Date: Mon, 20 Dec 2004 23:30:12 +0000 (+0000) Subject: Merge in second phase Disk IO refactoring, separating out the store and the io logic. X-Git-Tag: SQUID_3_0_PRE4~978 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=b9ae18aa6c61e83f0035239058127e2b4d0615b2;p=thirdparty%2Fsquid.git Merge in second phase Disk IO refactoring, separating out the store and the io logic. Patches applied: * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-15 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-286 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-16 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-287 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-17 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-288 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-18 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-289 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-19 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-290 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-20 Sync to HEAD. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-21 More OOification. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-22 Extraction of Disk IO modules. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-23 Make diskd use the IO separation facility. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-24 More diskd layering work. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-25 Rename SwapDirOption to ConfigOption. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-26 More diskd works. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-27 Move diskd's IO logic to DiskIO/DiskDaemon * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-28 Finish AUFS logic separation. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-29 Add AIO Disk IO Module. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-30 Fix double-free in coss. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-31 AIO as a full IO Module. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-32 Make IO usage flexible. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-33 Make coss work with diskd. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-34 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-356 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-35 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-357 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-36 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-358 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-37 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-359 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-38 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-360 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-39 Add missed files. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-40 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0--patch-361 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-41 AIO stability work. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-42 Coss tidyup. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-43 Merge of robertc@squid-cache.org--squid/squid--HEAD--3.0 * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-44 Bugfix: coss aliasing error. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-45 Merge http_version fix. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-46 CossLog tweaks. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-47 Remove all AIO dependencies from coss. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-48 Merge from HEAD. * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-49 merge from HEAD * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-50 sync with MAIN * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-51 merge in bugfix for auth double deletes in digest * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-52 update from HEAD * robertc@squid-cache.org--squid/squid--disk-io--3.0--patch-53 sync with HEAD --- diff --git a/Makefile.in b/Makefile.in index ce4489be4c..5a0ed77cd9 100644 --- a/Makefile.in +++ b/Makefile.in @@ -15,7 +15,7 @@ @SET_MAKE@ # -# $Id: Makefile.in,v 1.60 2004/12/20 14:52:07 robertc Exp $ +# $Id: Makefile.in,v 1.61 2004/12/20 16:30:12 robertc Exp $ # srcdir = @srcdir@ top_srcdir = @top_srcdir@ @@ -108,6 +108,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -195,9 +198,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/bootstrap.sh b/bootstrap.sh index 990eddec92..95d6d1527c 100755 --- a/bootstrap.sh +++ b/bootstrap.sh @@ -81,7 +81,7 @@ do perl -i.bak -p -e 's/m4_patsubst/m4_bpatsubst/g; s/m4_regexp/m4_bregexp/g;' aclocal.m4 fi bootstrap autoheader$acver - bootstrap libtoolize --automake + bootstrap libtoolize --force --automake bootstrap automake$amver --foreign --add-missing bootstrap autoconf$acver fi ); then diff --git a/configure b/configure index 441ea3bff9..1f011b36c0 100755 --- a/configure +++ b/configure @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 1.363 . +# From configure.in Revision: 1.364 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.59. # @@ -467,7 +467,7 @@ ac_includes_default="\ ac_subdirs_all="$ac_subdirs_all lib/libTrie" ac_subdirs_all="$ac_subdirs_all lib/cppunit-1.10.0" -ac_subst_vars='SHELL PATH_SEPARATOR PACKAGE_NAME PACKAGE_TARNAME PACKAGE_VERSION PACKAGE_STRING PACKAGE_BUGREPORT exec_prefix prefix program_transform_name bindir sbindir libexecdir datadir sysconfdir sharedstatedir localstatedir libdir includedir oldincludedir infodir mandir build_alias host_alias target_alias DEFS ECHO_C ECHO_N ECHO_T LIBS INSTALL_PROGRAM INSTALL_SCRIPT INSTALL_DATA CYGPATH_W PACKAGE VERSION ACLOCAL AUTOCONF AUTOMAKE AUTOHEADER MAKEINFO install_sh STRIP ac_ct_STRIP INSTALL_STRIP_PROGRAM mkdir_p AWK SET_MAKE am__leading_dot AMTAR am__tar am__untar MAINTAINER_MODE_TRUE MAINTAINER_MODE_FALSE MAINT CC CFLAGS LDFLAGS CPPFLAGS ac_ct_CC EXEEXT OBJEXT DEPDIR am__include am__quote AMDEP_TRUE AMDEP_FALSE AMDEPBACKSLASH CCDEPMODE am__fastdepCC_TRUE am__fastdepCC_FALSE CXX CXXFLAGS ac_ct_CXX CXXDEPMODE am__fastdepCXX_TRUE am__fastdepCXX_FALSE build build_cpu build_vendor build_os host host_cpu host_vendor host_os EGREP LN_S ECHO AR ac_ct_AR RANLIB ac_ct_RANLIB CPP CXXCPP F77 FFLAGS ac_ct_F77 LIBTOOL LIBADD_DL CGIEXT ENABLE_WIN32SPECIFIC_TRUE ENABLE_WIN32SPECIFIC_FALSE SQUID_CFLAGS SQUID_CXXFLAGS LIBDLMALLOC LIB_MALLOC STORE_OBJS STORE_LIBS STORE_PROGRAMS STORE_MODULE_SUBDIRS STORE_LINKOBJS REPL_POLICIES REPL_OBJS REPL_LIBS ENABLE_PINGER_TRUE ENABLE_PINGER_FALSE USE_DELAY_POOLS_TRUE USE_DELAY_POOLS_FALSE USE_ESI_TRUE USE_ESI_FALSE USE_SNMP_TRUE USE_SNMP_FALSE SNMPLIB makesnmplib ENABLE_ARP_ACL_TRUE ENABLE_ARP_ACL_FALSE ENABLE_HTCP_TRUE ENABLE_HTCP_FALSE ENABLE_SSL_TRUE ENABLE_SSL_FALSE NEED_OWN_MD5_TRUE NEED_OWN_MD5_FALSE SSLLIB ERR_DEFAULT_LANGUAGE ERR_LANGUAGES USE_EPOLL_TRUE USE_EPOLL_FALSE EPOLL_LIBS MAKE_LEAKFINDER_TRUE MAKE_LEAKFINDER_FALSE ENABLE_IDENT_TRUE ENABLE_IDENT_FALSE USE_DNSSERVER_TRUE USE_DNSSERVER_FALSE OPT_DEFAULT_HOSTS AUTH_MODULES AUTH_LIBS AUTH_LINKOBJS AUTH_OBJS BASIC_AUTH_HELPERS NTLM_AUTH_HELPERS DIGEST_AUTH_HELPERS EXTERNAL_ACL_HELPERS SAMBASOURCES LIBSASL ENABLE_UNLINKD_TRUE ENABLE_UNLINKD_FALSE ENABLE_XPROF_STATS_TRUE ENABLE_XPROF_STATS_FALSE SQUID_CPPUNIT_LA SQUID_CPPUNIT_INC SQUID_CPPUNIT_DIR SH FALSE TRUE RM MV MKDIR LN PERL AR_R ALLOCA CRYPTLIB NEED_OWN_SNPRINTF_TRUE NEED_OWN_SNPRINTF_FALSE REGEXLIB LIBREGEX LIBOBJS XTRA_OBJS XTRA_LIBS subdirs LTLIBOBJS' +ac_subst_vars='SHELL PATH_SEPARATOR PACKAGE_NAME PACKAGE_TARNAME PACKAGE_VERSION PACKAGE_STRING PACKAGE_BUGREPORT exec_prefix prefix program_transform_name bindir sbindir libexecdir datadir sysconfdir sharedstatedir localstatedir libdir includedir oldincludedir infodir mandir build_alias host_alias target_alias DEFS ECHO_C ECHO_N ECHO_T LIBS INSTALL_PROGRAM INSTALL_SCRIPT INSTALL_DATA CYGPATH_W PACKAGE VERSION ACLOCAL AUTOCONF AUTOMAKE AUTOHEADER MAKEINFO install_sh STRIP ac_ct_STRIP INSTALL_STRIP_PROGRAM mkdir_p AWK SET_MAKE am__leading_dot AMTAR am__tar am__untar MAINTAINER_MODE_TRUE MAINTAINER_MODE_FALSE MAINT CC CFLAGS LDFLAGS CPPFLAGS ac_ct_CC EXEEXT OBJEXT DEPDIR am__include am__quote AMDEP_TRUE AMDEP_FALSE AMDEPBACKSLASH CCDEPMODE am__fastdepCC_TRUE am__fastdepCC_FALSE CXX CXXFLAGS ac_ct_CXX CXXDEPMODE am__fastdepCXX_TRUE am__fastdepCXX_FALSE build build_cpu build_vendor build_os host host_cpu host_vendor host_os EGREP LN_S ECHO AR ac_ct_AR RANLIB ac_ct_RANLIB CPP CXXCPP F77 FFLAGS ac_ct_F77 LIBTOOL LIBADD_DL CGIEXT ENABLE_WIN32SPECIFIC_TRUE ENABLE_WIN32SPECIFIC_FALSE SQUID_CFLAGS SQUID_CXXFLAGS LIBDLMALLOC LIB_MALLOC STORE_OBJS STORE_LIBS STORE_LINKOBJS DISK_LIBS DISK_PROGRAMS DISK_LINKOBJS REPL_POLICIES REPL_OBJS REPL_LIBS ENABLE_PINGER_TRUE ENABLE_PINGER_FALSE USE_DELAY_POOLS_TRUE USE_DELAY_POOLS_FALSE USE_ESI_TRUE USE_ESI_FALSE USE_SNMP_TRUE USE_SNMP_FALSE SNMPLIB makesnmplib ENABLE_ARP_ACL_TRUE ENABLE_ARP_ACL_FALSE ENABLE_HTCP_TRUE ENABLE_HTCP_FALSE ENABLE_SSL_TRUE ENABLE_SSL_FALSE NEED_OWN_MD5_TRUE NEED_OWN_MD5_FALSE SSLLIB ERR_DEFAULT_LANGUAGE ERR_LANGUAGES USE_EPOLL_TRUE USE_EPOLL_FALSE EPOLL_LIBS MAKE_LEAKFINDER_TRUE MAKE_LEAKFINDER_FALSE ENABLE_IDENT_TRUE ENABLE_IDENT_FALSE USE_DNSSERVER_TRUE USE_DNSSERVER_FALSE OPT_DEFAULT_HOSTS AUTH_MODULES AUTH_LIBS AUTH_LINKOBJS AUTH_OBJS BASIC_AUTH_HELPERS NTLM_AUTH_HELPERS DIGEST_AUTH_HELPERS EXTERNAL_ACL_HELPERS SAMBASOURCES LIBSASL ENABLE_UNLINKD_TRUE ENABLE_UNLINKD_FALSE ENABLE_XPROF_STATS_TRUE ENABLE_XPROF_STATS_FALSE SQUID_CPPUNIT_LA SQUID_CPPUNIT_INC SQUID_CPPUNIT_DIR SH FALSE TRUE RM MV MKDIR LN PERL AR_R ALLOCA CRYPTLIB NEED_OWN_SNPRINTF_TRUE NEED_OWN_SNPRINTF_FALSE REGEXLIB LIBREGEX LIBOBJS XTRA_OBJS XTRA_LIBS subdirs LTLIBOBJS' ac_subst_files='' # Initialize some variables set by options. @@ -1070,6 +1070,12 @@ Optional Features: See src/fs for a list of available modules, or Programmers Guide section for details on how to build your custom store module + --enable-disk-io=\"list of modules\" + Build support for the list of disk I/O modules. + The default is only to build the "Blocking" module. + See src/DiskIO for a list of available modules, or + Programmers Guide section + for details on how to build your custom disk module --enable-removal-policies=\"list of policies\" Build support for the list of removal policies. The default is only to build the "lru" module. @@ -4436,7 +4442,7 @@ ia64-*-hpux*) ;; *-*-irix6*) # Find out which ABI we are using. - echo '#line 4439 "configure"' > conftest.$ac_ext + echo '#line 4445 "configure"' > conftest.$ac_ext if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 (eval $ac_compile) 2>&5 ac_status=$? @@ -5540,7 +5546,7 @@ fi # Provide some information about the compiler. -echo "$as_me:5543:" \ +echo "$as_me:5549:" \ "checking for Fortran 77 compiler version" >&5 ac_compiler=`set X $ac_compile; echo $2` { (eval echo "$as_me:$LINENO: \"$ac_compiler --version &5\"") >&5 @@ -6572,11 +6578,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:6575: $lt_compile\"" >&5) + (eval echo "\"\$as_me:6581: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:6579: \$? = $ac_status" >&5 + echo "$as_me:6585: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -6805,11 +6811,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:6808: $lt_compile\"" >&5) + (eval echo "\"\$as_me:6814: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:6812: \$? = $ac_status" >&5 + echo "$as_me:6818: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -6865,11 +6871,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:6868: $lt_compile\"" >&5) + (eval echo "\"\$as_me:6874: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:6872: \$? = $ac_status" >&5 + echo "$as_me:6878: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -9053,7 +9059,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext < conftest.$ac_ext <&5) + (eval echo "\"\$as_me:11337: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:11335: \$? = $ac_status" >&5 + echo "$as_me:11341: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -11388,11 +11394,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:11391: $lt_compile\"" >&5) + (eval echo "\"\$as_me:11397: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:11395: \$? = $ac_status" >&5 + echo "$as_me:11401: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -12757,7 +12763,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext < conftest.$ac_ext <&5) + (eval echo "\"\$as_me:13691: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:13689: \$? = $ac_status" >&5 + echo "$as_me:13695: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -13742,11 +13748,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:13745: $lt_compile\"" >&5) + (eval echo "\"\$as_me:13751: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:13749: \$? = $ac_status" >&5 + echo "$as_me:13755: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -15787,11 +15793,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:15790: $lt_compile\"" >&5) + (eval echo "\"\$as_me:15796: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:15794: \$? = $ac_status" >&5 + echo "$as_me:15800: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -16020,11 +16026,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:16023: $lt_compile\"" >&5) + (eval echo "\"\$as_me:16029: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:16027: \$? = $ac_status" >&5 + echo "$as_me:16033: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -16080,11 +16086,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:16083: $lt_compile\"" >&5) + (eval echo "\"\$as_me:16089: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:16087: \$? = $ac_status" >&5 + echo "$as_me:16093: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -18268,7 +18274,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext < conftest.$ac_ext < + for details on how to build your custom disk module], +[ case $enableval in + yes) + for dir in $srcdir/src/DiskIO/*; do + module="`basename $dir`" + if test -d "$dir" && test "$module" != CVS; then + DISK_MODULES="$DISK_MODULES $module" + fi + done + ;; + no) + ;; + *) + DISK_MODULES="`echo $enableval| sed -e 's/,/ /g;s/ */ /g'`" + ;; + esac +], +[ if test -z "$DISK_MODULES"; then + DISK_MODULES="Blocking" + fi +]) +if test -n "$DISK_MODULES"; then + for module in $DISK_MODULES; do + if test -d $srcdir/src/DiskIO/$module; then + : + else + echo "ERROR: disk-io $module does not exists" + exit 1 + fi + done + DISK_LIBS="lib`echo $DISK_MODULES|sed -e 's% %.a lib%g'`.a" + DISK_LINKOBJS= + for module in $DISK_MODULES; do + DISK_LINKOBJS="$DISK_LINKOBJS DiskIO/${module}/${module}DiskIOModule.o" + done +fi +for fs in $DISK_MODULES none; do + case "$fs" in + DiskDaemon) + DISK_PROGRAMS="$DISK_PROGRAMS DiskIO/DiskDaemon/diskd" + FOUND_DISKDAEMON="true" + ;; + DiskThreads) + FOUND_DISKTHREADS="true" + ;; + AIO) + FOUND_AIO="true" + ;; + Blocking) + FOUND_BLOCKING="true" + esac +done + +if test -z "$FOUND_BLOCKING" && test -n "$NEED_BLOCKING"; then + echo "adding Blocking, as it is used by an active, legacy Store Module" + DISK_LIBS="$DISK_LIBS libBlocking.a" + DISK_MODULES="$DISK_MODULES Blocking" + DISK_LINKOBJS="$DISK_LINKOBJS DiskIO/Blocking/BlockingDiskIOModule.o" +fi + +if test -z "$FOUND_DISKDAEMON" && test -n "$NEED_DISKDAEMON"; then + echo "adding DiskDaemon, as it is used by an active, legacy Store Module" + DISK_LIBS="$DISK_LIBS libDiskDaemon.a" + DISK_MODULES="$DISK_MODULES DiskDaemon" + DISK_PROGRAMS="$DISK_PROGRAMS DiskIO/DiskDaemon/diskd" + DISK_LINKOBJS="$DISK_LINKOBJS DiskIO/DiskDaemon/DiskDaemonDiskIOModule.o" +fi + +if test -z "$FOUND_DISKTHREADS" && test -n "$NEED_DISKTHREADS"; then + echo "adding DiskThreads, as it is used by an active, legacy Store Module" + DISK_LIBS="$DISK_LIBS libDiskThreads.a" + DISK_MODULES="$DISK_MODULES DiskThreads" + DISK_LINKOBJS="$DISK_LINKOBJS DiskIO/DiskThreads/DiskThreadsDiskIOModule.o" +fi +if test -z "$FOUND_AIO" && test -n "$NEED_AIO"; then + echo "adding AIO, as it is used by an active, legacy Store Module" + DISK_LIBS="$DISK_LIBS libAIO.a" + DISK_MODULES="$DISK_MODULES AIO" + DISK_LINKOBJS="$DISK_LINKOBJS DiskIO/AIO/AIODiskIOModule.o" +fi +echo "IO Modules built: $DISK_MODULES" +dnl we know what is being built. now add dependencies. +for fs in $DISK_MODULES none; do + case "$fs" in + DiskThreads) + if test -z "$with_pthreads"; then + echo "DiskThreads IO Module used, pthreads support automatically enabled" + with_pthreads=yes + fi + ;; + AIO) + if test -z "$with_aio"; then + echo "Aio IO Module used, aio support automatically enabled" + with_aio=yes + fi + ;; + esac +done + +AC_SUBST(DISK_LIBS) +AC_SUBST(DISK_PROGRAMS) +AC_SUBST(DISK_LINKOBJS) + AC_ARG_ENABLE(removal-policies, [ --enable-removal-policies=\"list of policies\" Build support for the list of removal policies. diff --git a/contrib/Makefile.in b/contrib/Makefile.in index be7574a8e1..76b5703ee4 100644 --- a/contrib/Makefile.in +++ b/contrib/Makefile.in @@ -84,6 +84,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -171,9 +174,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/doc/Makefile.in b/doc/Makefile.in index 8f9650cfa4..4b5e562f02 100644 --- a/doc/Makefile.in +++ b/doc/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.39 2004/12/20 14:52:10 robertc Exp $ +# $Id: Makefile.in,v 1.40 2004/12/20 16:30:13 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -91,6 +91,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -178,9 +181,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/doc/release-notes/release-3.0.sgml b/doc/release-notes/release-3.0.sgml index a57f5b4afb..6f6e6278e2 100644 --- a/doc/release-notes/release-3.0.sgml +++ b/doc/release-notes/release-3.0.sgml @@ -2,7 +2,7 @@
Squid 3.0 release notes Squid Developers -$Id: release-3.0.sgml,v 1.18 2004/12/20 14:52:11 robertc Exp $ +$Id: release-3.0.sgml,v 1.19 2004/12/20 16:30:15 robertc Exp $ This document contains the release notes for version 3.0 of Squid. diff --git a/errors/Makefile.in b/errors/Makefile.in index e61f280678..eb9b1e2f28 100644 --- a/errors/Makefile.in +++ b/errors/Makefile.in @@ -15,7 +15,7 @@ @SET_MAKE@ # -# $Id: Makefile.in,v 1.56 2004/12/20 14:52:11 robertc Exp $ +# $Id: Makefile.in,v 1.57 2004/12/20 16:30:15 robertc Exp $ # srcdir = @srcdir@ top_srcdir = @top_srcdir@ @@ -83,6 +83,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -170,9 +173,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/Makefile.in b/helpers/Makefile.in index 8ca4eaa56d..6ff44ea394 100644 --- a/helpers/Makefile.in +++ b/helpers/Makefile.in @@ -88,6 +88,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -175,9 +178,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/LDAP/Makefile.in b/helpers/basic_auth/LDAP/Makefile.in index 2ed7cc7965..c81832beaf 100644 --- a/helpers/basic_auth/LDAP/Makefile.in +++ b/helpers/basic_auth/LDAP/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid LDAP authentication helper # -# $Id: Makefile.in,v 1.48 2004/12/20 14:52:12 robertc Exp $ +# $Id: Makefile.in,v 1.49 2004/12/20 16:30:15 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -115,6 +115,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -202,9 +205,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/MSNT/Makefile.in b/helpers/basic_auth/MSNT/Makefile.in index c962f0600b..df79f60cf8 100644 --- a/helpers/basic_auth/MSNT/Makefile.in +++ b/helpers/basic_auth/MSNT/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.51 2004/12/20 14:52:12 robertc Exp $ +# $Id: Makefile.in,v 1.52 2004/12/20 16:30:15 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -125,6 +125,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -212,9 +215,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/Makefile.in b/helpers/basic_auth/Makefile.in index f1e2f60d8c..19da868d46 100644 --- a/helpers/basic_auth/Makefile.in +++ b/helpers/basic_auth/Makefile.in @@ -16,7 +16,7 @@ # Makefile for storage modules in the Squid Object Cache server # -# $Id: Makefile.in,v 1.46 2004/12/20 14:52:12 robertc Exp $ +# $Id: Makefile.in,v 1.47 2004/12/20 16:30:15 robertc Exp $ # srcdir = @srcdir@ top_srcdir = @top_srcdir@ @@ -92,6 +92,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -179,9 +182,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/NCSA/Makefile.in b/helpers/basic_auth/NCSA/Makefile.in index db82732135..e60a5e0b12 100644 --- a/helpers/basic_auth/NCSA/Makefile.in +++ b/helpers/basic_auth/NCSA/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.48 2004/12/20 14:52:14 robertc Exp $ +# $Id: Makefile.in,v 1.49 2004/12/20 16:30:15 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -113,6 +113,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -200,9 +203,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/PAM/Makefile.in b/helpers/basic_auth/PAM/Makefile.in index 778fb633b7..c6399b4130 100644 --- a/helpers/basic_auth/PAM/Makefile.in +++ b/helpers/basic_auth/PAM/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid PAM authentication helper # -# $Id: Makefile.in,v 1.49 2004/12/20 14:52:14 robertc Exp $ +# $Id: Makefile.in,v 1.50 2004/12/20 16:30:15 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -115,6 +115,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -202,9 +205,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/SASL/Makefile.am b/helpers/basic_auth/SASL/Makefile.am index c66ef7acc4..522c9e8177 100644 --- a/helpers/basic_auth/SASL/Makefile.am +++ b/helpers/basic_auth/SASL/Makefile.am @@ -1,7 +1,7 @@ # # Makefile for the Squid SASL authentication helper # -# $Id: Makefile.am,v 1.6 2004/12/20 14:52:14 robertc Exp $ +# $Id: Makefile.am,v 1.7 2004/12/20 16:30:15 robertc Exp $ # # Uncomment and customize the following to suit your needs: # diff --git a/helpers/basic_auth/SASL/Makefile.in b/helpers/basic_auth/SASL/Makefile.in index a3e42ace97..5cbecbf148 100644 --- a/helpers/basic_auth/SASL/Makefile.in +++ b/helpers/basic_auth/SASL/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid SASL authentication helper # -# $Id: Makefile.in,v 1.45 2004/12/20 14:52:14 robertc Exp $ +# $Id: Makefile.in,v 1.46 2004/12/20 16:30:15 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -112,6 +112,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -199,9 +202,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/SASL/sasl_auth.c b/helpers/basic_auth/SASL/sasl_auth.c index 15d6b30d7c..817efdec4e 100644 --- a/helpers/basic_auth/SASL/sasl_auth.c +++ b/helpers/basic_auth/SASL/sasl_auth.c @@ -1,5 +1,5 @@ /* - * $Id: sasl_auth.c,v 1.7 2004/12/20 14:52:14 robertc Exp $ + * $Id: sasl_auth.c,v 1.8 2004/12/20 16:30:16 robertc Exp $ * * SASL authenticator module for Squid. * Copyright (C) 2002 Ian Castle diff --git a/helpers/basic_auth/SMB/Makefile.in b/helpers/basic_auth/SMB/Makefile.in index 410e189e4f..8f17101cc6 100644 --- a/helpers/basic_auth/SMB/Makefile.in +++ b/helpers/basic_auth/SMB/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.47 2004/12/20 14:52:15 robertc Exp $ +# $Id: Makefile.in,v 1.48 2004/12/20 16:30:16 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -121,6 +121,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -208,9 +211,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/YP/Makefile.in b/helpers/basic_auth/YP/Makefile.in index 256099bfb2..addfd1a634 100644 --- a/helpers/basic_auth/YP/Makefile.in +++ b/helpers/basic_auth/YP/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.48 2004/12/20 14:52:15 robertc Exp $ +# $Id: Makefile.in,v 1.49 2004/12/20 16:30:17 robertc Exp $ # # @@ -111,6 +111,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -198,9 +201,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/getpwnam/Makefile.in b/helpers/basic_auth/getpwnam/Makefile.in index 9f15610c66..2d136d805f 100644 --- a/helpers/basic_auth/getpwnam/Makefile.in +++ b/helpers/basic_auth/getpwnam/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.46 2004/12/20 14:52:16 robertc Exp $ +# $Id: Makefile.in,v 1.47 2004/12/20 16:30:18 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -113,6 +113,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -200,9 +203,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/multi-domain-NTLM/Makefile.in b/helpers/basic_auth/multi-domain-NTLM/Makefile.in index 10080eb868..4c59500ad4 100644 --- a/helpers/basic_auth/multi-domain-NTLM/Makefile.in +++ b/helpers/basic_auth/multi-domain-NTLM/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.44 2004/12/20 14:52:16 robertc Exp $ +# $Id: Makefile.in,v 1.45 2004/12/20 16:30:19 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -91,6 +91,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -178,9 +181,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/basic_auth/winbind/Makefile.in b/helpers/basic_auth/winbind/Makefile.in index b4a649bf33..eddd0c6b22 100644 --- a/helpers/basic_auth/winbind/Makefile.in +++ b/helpers/basic_auth/winbind/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.38 2004/12/20 14:52:17 robertc Exp $ +# $Id: Makefile.in,v 1.39 2004/12/20 16:30:19 robertc Exp $ # SOURCES = $(wb_auth_SOURCES) @@ -110,6 +110,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -197,9 +200,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/digest_auth/Makefile.in b/helpers/digest_auth/Makefile.in index 46cfabd16c..9a49170b46 100644 --- a/helpers/digest_auth/Makefile.in +++ b/helpers/digest_auth/Makefile.in @@ -16,7 +16,7 @@ # Makefile for digest auth helpers in the Squid Object Cache server # -# $Id: Makefile.in,v 1.43 2004/12/20 14:52:17 robertc Exp $ +# $Id: Makefile.in,v 1.44 2004/12/20 16:30:22 robertc Exp $ # srcdir = @srcdir@ top_srcdir = @top_srcdir@ @@ -92,6 +92,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -179,9 +182,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/digest_auth/password/Makefile.in b/helpers/digest_auth/password/Makefile.in index 2725833715..c4a6f8e4d2 100644 --- a/helpers/digest_auth/password/Makefile.in +++ b/helpers/digest_auth/password/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.46 2004/12/20 14:52:17 robertc Exp $ +# $Id: Makefile.in,v 1.47 2004/12/20 16:30:23 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -114,6 +114,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -201,9 +204,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/external_acl/Makefile.in b/helpers/external_acl/Makefile.in index d6c9a74d84..17842cdb9a 100644 --- a/helpers/external_acl/Makefile.in +++ b/helpers/external_acl/Makefile.in @@ -16,7 +16,7 @@ # Makefile for storage modules in the Squid Object Cache server # -# $Id: Makefile.in,v 1.33 2004/12/20 14:52:18 robertc Exp $ +# $Id: Makefile.in,v 1.34 2004/12/20 16:30:23 robertc Exp $ # srcdir = @srcdir@ top_srcdir = @top_srcdir@ @@ -92,6 +92,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -179,9 +182,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/external_acl/ip_user/Makefile.in b/helpers/external_acl/ip_user/Makefile.in index 38c616aa24..6ba165e63f 100644 --- a/helpers/external_acl/ip_user/Makefile.in +++ b/helpers/external_acl/ip_user/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the ip_user external_acl helper by Rodrigo Campos # -# $Id: Makefile.in,v 1.34 2004/12/20 14:52:19 robertc Exp $ +# $Id: Makefile.in,v 1.35 2004/12/20 16:30:23 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -112,6 +112,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -199,9 +202,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/external_acl/ldap_group/Makefile.in b/helpers/external_acl/ldap_group/Makefile.in index fd8527c8bc..64311f02b2 100644 --- a/helpers/external_acl/ldap_group/Makefile.in +++ b/helpers/external_acl/ldap_group/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid LDAP authentication helper # -# $Id: Makefile.in,v 1.36 2004/12/20 14:52:19 robertc Exp $ +# $Id: Makefile.in,v 1.37 2004/12/20 16:30:23 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -115,6 +115,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -202,9 +205,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/external_acl/unix_group/Makefile.in b/helpers/external_acl/unix_group/Makefile.in index 668d7bbdac..46ff5953bf 100644 --- a/helpers/external_acl/unix_group/Makefile.in +++ b/helpers/external_acl/unix_group/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid LDAP authentication helper # -# $Id: Makefile.in,v 1.34 2004/12/20 14:52:19 robertc Exp $ +# $Id: Makefile.in,v 1.35 2004/12/20 16:30:23 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -114,6 +114,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -201,9 +204,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/external_acl/wbinfo_group/Makefile.in b/helpers/external_acl/wbinfo_group/Makefile.in index f146179d2d..6046013657 100644 --- a/helpers/external_acl/wbinfo_group/Makefile.in +++ b/helpers/external_acl/wbinfo_group/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid LDAP authentication helper # -# $Id: Makefile.in,v 1.32 2004/12/20 14:52:19 robertc Exp $ +# $Id: Makefile.in,v 1.33 2004/12/20 16:30:23 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -91,6 +91,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -178,9 +181,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/external_acl/winbind_group/Makefile.in b/helpers/external_acl/winbind_group/Makefile.in index 2e63d488bb..9b15236631 100644 --- a/helpers/external_acl/winbind_group/Makefile.in +++ b/helpers/external_acl/winbind_group/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the wb_group external_acl helper # -# $Id: Makefile.in,v 1.35 2004/12/20 14:52:19 robertc Exp $ +# $Id: Makefile.in,v 1.36 2004/12/20 16:30:25 robertc Exp $ # SOURCES = $(wb_group_SOURCES) @@ -109,6 +109,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -196,9 +199,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/ntlm_auth/Makefile.in b/helpers/ntlm_auth/Makefile.in index 4117480c92..9ad555b196 100644 --- a/helpers/ntlm_auth/Makefile.in +++ b/helpers/ntlm_auth/Makefile.in @@ -16,7 +16,7 @@ # Makefile for storage modules in the Squid Object Cache server # -# $Id: Makefile.in,v 1.46 2004/12/20 14:52:20 robertc Exp $ +# $Id: Makefile.in,v 1.47 2004/12/20 16:30:25 robertc Exp $ # srcdir = @srcdir@ top_srcdir = @top_srcdir@ @@ -92,6 +92,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -179,9 +182,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/ntlm_auth/SMB/Makefile.in b/helpers/ntlm_auth/SMB/Makefile.in index 0a1d3de9a5..c07bc0c2ab 100644 --- a/helpers/ntlm_auth/SMB/Makefile.in +++ b/helpers/ntlm_auth/SMB/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.46 2004/12/20 14:52:20 robertc Exp $ +# $Id: Makefile.in,v 1.47 2004/12/20 16:30:25 robertc Exp $ # SOURCES = $(ntlm_auth_SOURCES) @@ -118,6 +118,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -205,9 +208,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/ntlm_auth/SMB/smbval/Makefile.in b/helpers/ntlm_auth/SMB/smbval/Makefile.in index da79c26d4e..cef34fe8f8 100644 --- a/helpers/ntlm_auth/SMB/smbval/Makefile.in +++ b/helpers/ntlm_auth/SMB/smbval/Makefile.in @@ -106,6 +106,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -193,9 +196,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/ntlm_auth/fakeauth/Makefile.in b/helpers/ntlm_auth/fakeauth/Makefile.in index 0be5fa6f53..2ba5d2bdac 100644 --- a/helpers/ntlm_auth/fakeauth/Makefile.in +++ b/helpers/ntlm_auth/fakeauth/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.45 2004/12/20 14:52:21 robertc Exp $ +# $Id: Makefile.in,v 1.46 2004/12/20 16:30:25 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -113,6 +113,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -200,9 +203,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/ntlm_auth/no_check/Makefile.in b/helpers/ntlm_auth/no_check/Makefile.in index 01de6da8bc..564909d3ee 100644 --- a/helpers/ntlm_auth/no_check/Makefile.in +++ b/helpers/ntlm_auth/no_check/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.47 2004/12/20 14:52:21 robertc Exp $ +# $Id: Makefile.in,v 1.48 2004/12/20 16:30:27 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -91,6 +91,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -178,9 +181,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/helpers/ntlm_auth/winbind/Makefile.in b/helpers/ntlm_auth/winbind/Makefile.in index 526ba45491..e937251003 100644 --- a/helpers/ntlm_auth/winbind/Makefile.in +++ b/helpers/ntlm_auth/winbind/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.38 2004/12/20 14:52:22 robertc Exp $ +# $Id: Makefile.in,v 1.39 2004/12/20 16:30:29 robertc Exp $ # SOURCES = $(wb_ntlmauth_SOURCES) @@ -110,6 +110,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -197,9 +200,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/icons/Makefile.in b/icons/Makefile.in index 669cb23d97..7b875cfe1e 100644 --- a/icons/Makefile.in +++ b/icons/Makefile.in @@ -14,7 +14,7 @@ @SET_MAKE@ -# $Id: Makefile.in,v 1.61 2004/12/20 14:52:22 robertc Exp $ +# $Id: Makefile.in,v 1.62 2004/12/20 16:30:29 robertc Exp $ # srcdir = @srcdir@ @@ -92,6 +92,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -179,9 +182,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/include/Array.h b/include/Array.h index a07884f098..9b48e581d9 100644 --- a/include/Array.h +++ b/include/Array.h @@ -1,5 +1,5 @@ /* - * $Id: Array.h,v 1.18 2004/12/20 14:52:22 robertc Exp $ + * $Id: Array.h,v 1.19 2004/12/20 16:30:29 robertc Exp $ * * AUTHOR: Alex Rousskov * diff --git a/include/Stack.h b/include/Stack.h index 50162b9ea8..a4c8b9baf9 100644 --- a/include/Stack.h +++ b/include/Stack.h @@ -1,5 +1,5 @@ /* - * $Id: Stack.h,v 1.17 2004/12/20 14:52:22 robertc Exp $ + * $Id: Stack.h,v 1.18 2004/12/20 16:30:29 robertc Exp $ * * AUTHOR: Alex Rousskov * diff --git a/include/config.h b/include/config.h index c7c7445373..f1264cbb26 100644 --- a/include/config.h +++ b/include/config.h @@ -1,5 +1,5 @@ /* - * $Id: config.h,v 1.12 2004/12/20 14:52:23 robertc Exp $ + * $Id: config.h,v 1.13 2004/12/20 16:30:29 robertc Exp $ * * AUTHOR: Duane Wessels * diff --git a/lib/GNUregex.c b/lib/GNUregex.c index b803fa55aa..e1f170ddeb 100644 --- a/lib/GNUregex.c +++ b/lib/GNUregex.c @@ -1,5 +1,5 @@ /* - * $Id: GNUregex.c,v 1.18 2004/12/20 14:52:23 robertc Exp $ + * $Id: GNUregex.c,v 1.19 2004/12/20 16:30:30 robertc Exp $ */ /* Extended regular expression matching and search library, diff --git a/lib/Makefile.am b/lib/Makefile.am index c83e05db03..1a15749e11 100644 --- a/lib/Makefile.am +++ b/lib/Makefile.am @@ -1,6 +1,6 @@ ## Process this file with automake to produce Makefile.in # -# $Id: Makefile.am,v 1.15 2004/12/20 14:52:24 robertc Exp $ +# $Id: Makefile.am,v 1.16 2004/12/20 16:30:32 robertc Exp $ # DIST_SUBDIRS = libTrie cppunit-1.10.0 diff --git a/lib/Makefile.in b/lib/Makefile.in index 9f057a77aa..8bc1303bf3 100644 --- a/lib/Makefile.in +++ b/lib/Makefile.in @@ -15,7 +15,7 @@ @SET_MAKE@ # -# $Id: Makefile.in,v 1.98 2004/12/20 14:52:24 robertc Exp $ +# $Id: Makefile.in,v 1.99 2004/12/20 16:30:32 robertc Exp $ # SOURCES = $(libdlmalloc_a_SOURCES) $(libmiscutil_a_SOURCES) $(EXTRA_libmiscutil_a_SOURCES) $(libntlmauth_a_SOURCES) $(libregex_a_SOURCES) $(tests_testAll_SOURCES) @@ -159,6 +159,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -246,9 +249,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/scripts/Makefile.in b/scripts/Makefile.in index 7db8290123..71b1f002a7 100644 --- a/scripts/Makefile.in +++ b/scripts/Makefile.in @@ -89,6 +89,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -176,9 +179,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/snmplib/Makefile.in b/snmplib/Makefile.in index fcaf46c808..dd8e00c886 100644 --- a/snmplib/Makefile.in +++ b/snmplib/Makefile.in @@ -104,6 +104,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = -DSQUID_SNMP=1 DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -191,9 +194,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/src/ACLARP.cc b/src/ACLARP.cc index 57bdf4654c..1f582648f7 100644 --- a/src/ACLARP.cc +++ b/src/ACLARP.cc @@ -1,5 +1,5 @@ /* - * $Id: ACLARP.cc,v 1.8 2004/12/20 14:52:24 robertc Exp $ + * $Id: ACLARP.cc,v 1.9 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 28 Access Control * AUTHOR: Duane Wessels diff --git a/src/ACLBrowser.cc b/src/ACLBrowser.cc index e039818fbc..d682bc7414 100644 --- a/src/ACLBrowser.cc +++ b/src/ACLBrowser.cc @@ -1,5 +1,5 @@ /* - * $Id: ACLBrowser.cc,v 1.4 2004/12/20 14:52:24 robertc Exp $ + * $Id: ACLBrowser.cc,v 1.5 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 28 Access Control * AUTHOR: Duane Wessels diff --git a/src/ACLChecklist.cc b/src/ACLChecklist.cc index bf9a66a696..4bc36026cf 100644 --- a/src/ACLChecklist.cc +++ b/src/ACLChecklist.cc @@ -1,5 +1,5 @@ /* - * $Id: ACLChecklist.cc,v 1.20 2004/12/20 14:52:24 robertc Exp $ + * $Id: ACLChecklist.cc,v 1.21 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 28 Access Control * AUTHOR: Duane Wessels diff --git a/src/ACLDomainData.cc b/src/ACLDomainData.cc index fbc8d0e964..09c49e18e8 100644 --- a/src/ACLDomainData.cc +++ b/src/ACLDomainData.cc @@ -1,5 +1,5 @@ /* - * $Id: ACLDomainData.cc,v 1.8 2004/12/20 14:52:24 robertc Exp $ + * $Id: ACLDomainData.cc,v 1.9 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 28 Access Control * AUTHOR: Duane Wessels diff --git a/src/ACLReplyHeaderStrategy.h b/src/ACLReplyHeaderStrategy.h index f3b71fdfec..82e0dadbac 100644 --- a/src/ACLReplyHeaderStrategy.h +++ b/src/ACLReplyHeaderStrategy.h @@ -1,6 +1,6 @@ /* - * $Id: ACLReplyHeaderStrategy.h,v 1.7 2004/12/20 14:52:24 robertc Exp $ + * $Id: ACLReplyHeaderStrategy.h,v 1.8 2004/12/20 16:30:32 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ diff --git a/src/ACLReplyMIMEType.h b/src/ACLReplyMIMEType.h index 15b79927c7..e731aec122 100644 --- a/src/ACLReplyMIMEType.h +++ b/src/ACLReplyMIMEType.h @@ -1,6 +1,6 @@ /* - * $Id: ACLReplyMIMEType.h,v 1.5 2004/12/20 14:52:24 robertc Exp $ + * $Id: ACLReplyMIMEType.h,v 1.6 2004/12/20 16:30:32 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ diff --git a/src/ACLRequestHeaderStrategy.h b/src/ACLRequestHeaderStrategy.h index ca8744852b..f5115ca98b 100644 --- a/src/ACLRequestHeaderStrategy.h +++ b/src/ACLRequestHeaderStrategy.h @@ -1,6 +1,6 @@ /* - * $Id: ACLRequestHeaderStrategy.h,v 1.9 2004/12/20 14:52:25 robertc Exp $ + * $Id: ACLRequestHeaderStrategy.h,v 1.10 2004/12/20 16:30:32 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ diff --git a/src/ACLTimeData.cc b/src/ACLTimeData.cc index 9ed427072d..80c96e075d 100644 --- a/src/ACLTimeData.cc +++ b/src/ACLTimeData.cc @@ -1,5 +1,5 @@ /* - * $Id: ACLTimeData.cc,v 1.7 2004/12/20 14:52:25 robertc Exp $ + * $Id: ACLTimeData.cc,v 1.8 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 28 Access Control * AUTHOR: Duane Wessels diff --git a/src/ConfigOption.cc b/src/ConfigOption.cc new file mode 100644 index 0000000000..d9578eb3ab --- /dev/null +++ b/src/ConfigOption.cc @@ -0,0 +1,68 @@ + +/* + * $Id: ConfigOption.cc,v 1.1 2004/12/20 16:30:32 robertc Exp $ + * + * DEBUG: section ?? Swap Dir base object + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + */ + +#include "squid.h" +#include "ConfigOption.h" + +ConfigOptionVector::~ConfigOptionVector() +{ + while (options.size()) { + delete options.back(); + options.pop_back(); + } +} + +bool +ConfigOptionVector::parse(char const *option, const char *value, int reconfiguring) +{ + Vector::iterator i = options.begin(); + + while (i != options.end()) { + if ((*i)->parse(option,value, reconfiguring)) + return true; + + ++i; + } + + return false; +} + +void +ConfigOptionVector::dump(StoreEntry * e) const +{ + for(Vector::const_iterator i = options.begin(); + i != options.end(); ++i) + (*i)->dump(e); +} diff --git a/src/ConfigOption.h b/src/ConfigOption.h new file mode 100644 index 0000000000..60efb1e344 --- /dev/null +++ b/src/ConfigOption.h @@ -0,0 +1,88 @@ +/* + * $Id: ConfigOption.h,v 1.1 2004/12/20 16:30:32 robertc Exp $ + * + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + */ + +#ifndef SQUID_CONFIGOPTION_H +#define SQUID_CONFIGOPTION_H + +#include "Array.h" + +/* cache option parsers */ + +class ConfigOption +{ + +public: + virtual ~ConfigOption() {} + + virtual bool parse(char const *option, const char *value, int reconfiguring) = 0; + virtual void dump (StoreEntry * e) const = 0; +}; + +class ConfigOptionVector : public ConfigOption +{ + +public: + virtual ~ConfigOptionVector(); + virtual bool parse(char const *option, const char *value, int reconfiguring); + virtual void dump(StoreEntry * e) const; + Vectoroptions; +}; + +template + +class ConfigOptionAdapter : public ConfigOption +{ + +public: + ConfigOptionAdapter (C& theObject, bool (C::*parseFP)(char const *option, const char *value, int reconfiguring), void (C::*dumpFP) (StoreEntry * e) const) : object(theObject), parser (parseFP), dumper(dumpFP) {} + + bool parse(char const *option, const char *value, int reconfiguring) + { + if (parser) + return (object.*parser)(option, value, reconfiguring); + + return false; + } + + void dump (StoreEntry * e) const + { + if (dumper) + (object.*dumper) (e); + } + +private: + C &object; + bool (C::*parser) (char const *option, const char *value, int reconfiguring) ; + void (C::*dumper)(StoreEntry * e) const; +}; + +#endif /* SQUID_CONFIGOPTION_H */ diff --git a/src/DiskIO/AIO/AIODiskFile.cc b/src/DiskIO/AIO/AIODiskFile.cc new file mode 100644 index 0000000000..68cb3ce7d0 --- /dev/null +++ b/src/DiskIO/AIO/AIODiskFile.cc @@ -0,0 +1,263 @@ + +/* + * $Id: AIODiskFile.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ +/* + * Author: Adrian Chadd + * + * These routines are simple plugin replacements for the file_* routines + * in disk.c . They back-end into the POSIX AIO routines to provide + * a nice and simple async IO framework for COSS. + * + * AIO is suitable for COSS - the only sync operations that the standard + * supports are read/write, and since COSS works on a single file + * per storedir it should work just fine. + */ + +#include "squid.h" +#include "AIODiskFile.h" +#include "AIODiskIOStrategy.h" +#include "DiskIO/IORequestor.h" +#include "DiskIO/ReadRequest.h" +#include "DiskIO/WriteRequest.h" + +CBDATA_CLASS_INIT(AIODiskFile); +void * +AIODiskFile::operator new (size_t) +{ + CBDATA_INIT_TYPE(AIODiskFile); + return cbdataAlloc(AIODiskFile); +} + +void +AIODiskFile::operator delete (void *address) +{ + cbdataFree(address); +} + +AIODiskFile::AIODiskFile (char const *aPath, AIODiskIOStrategy *aStrategy) : fd(-1), closed(true), error_(false) +{ + assert (aPath); + path = aPath; + strategy = aStrategy; + debug (79,3)("AIODiskFile::AIODiskFile: %s\n", aPath); +} + +AIODiskFile::~AIODiskFile() +{} + +void +AIODiskFile::error(bool const &aBool) +{ + error_ = aBool; +} + +void +AIODiskFile::open (int flags, mode_t mode, IORequestor::Pointer callback) +{ + /* Simulate async calls */ + fd = file_open(path.buf() , flags); + ioRequestor = callback; + + if (fd < 0) { + debug(79, 3) ("BlockingFile::open: got failure (%d)\n", errno); + error(true); + } else { + closed = false; + store_open_disk_fd++; + debug(79, 3) ("BlockingFile::open: opened FD %d\n", fd); + } + + callback->ioCompletedNotification(); +} + +void +AIODiskFile::create (int flags, mode_t mode, RefCount callback) +{ + /* We use the same logic path for open */ + open(flags, mode, callback); +} + +void +AIODiskFile::read(ReadRequest *request) +{ + int slot; + async_queue_entry_t *qe; + + assert(strategy->aq.aq_state == AQ_STATE_SETUP); + + /* Find a free slot */ + slot = strategy->findSlot(); + + if (slot < 0) { + /* No free slot? Callback error, and return */ + fatal("Aiee! out of aiocb slots! - FIXME and wrap file_read\n"); + debug(79, 1) ("WARNING: out of aiocb slots!\n"); + /* fall back to blocking method */ + // file_read(fd, request->buf, request->len, request->offset, callback, data); + return; + } + + /* Mark slot as ours */ + qe = &strategy->aq.aq_queue[slot]; + + qe->aq_e_state = AQ_ENTRY_USED; + + qe->aq_e_callback_data = cbdataReference(request); + + qe->theFile = cbdataReference(this); + + qe->aq_e_type = AQ_ENTRY_READ; + + qe->aq_e_free = NULL; + + qe->aq_e_buf = request->buf; + + qe->aq_e_fd = getFD(); + + qe->aq_e_aiocb.aio_fildes = getFD(); + + qe->aq_e_aiocb.aio_nbytes = request->len; + + qe->aq_e_aiocb.aio_offset = request->offset; + + qe->aq_e_aiocb.aio_buf = request->buf; + + /* Account */ + strategy->aq.aq_numpending++; + + /* Initiate aio */ + if (aio_read(&qe->aq_e_aiocb) < 0) { + fatalf("Aiee! aio_read() returned error (%d) FIXME and wrap file_read !\n", errno); + debug(79, 1) ("WARNING: aio_read() returned error: %s\n", xstrerror()); + /* fall back to blocking method */ + // file_read(fd, request->buf, request->len, request->offset, callback, data); + } + +} + +void +AIODiskFile::write(WriteRequest *request) +{ + int slot; + async_queue_entry_t *qe; + + assert(strategy->aq.aq_state == AQ_STATE_SETUP); + + /* Find a free slot */ + slot = strategy->findSlot(); + + if (slot < 0) { + /* No free slot? Callback error, and return */ + fatal("Aiee! out of aiocb slots FIXME and wrap file_write !\n"); + debug(79, 1) ("WARNING: out of aiocb slots!\n"); + /* fall back to blocking method */ + // file_write(fd, offset, buf, len, callback, data, freefunc); + return; + } + + /* Mark slot as ours */ + qe = &strategy->aq.aq_queue[slot]; + + qe->aq_e_state = AQ_ENTRY_USED; + + qe->aq_e_callback_data = cbdataReference(request); + + qe->theFile = cbdataReference(this); + + qe->aq_e_type = AQ_ENTRY_WRITE; + + qe->aq_e_free = request->free_func; + + qe->aq_e_buf = (void *)request->buf; + + qe->aq_e_fd = fd; + + qe->aq_e_aiocb.aio_fildes = fd; + + qe->aq_e_aiocb.aio_nbytes = request->len; + + qe->aq_e_aiocb.aio_offset = request->offset; + + qe->aq_e_aiocb.aio_buf = (void *)request->buf; + + /* Account */ + ++strategy->aq.aq_numpending; + + /* Initiate aio */ + if (aio_write(&qe->aq_e_aiocb) < 0) { + fatalf("Aiee! aio_read() returned error (%d) FIXME and wrap file_write !\n", errno); + debug(79, 1) ("WARNING: aio_write() returned error: %s\n", xstrerror()); + /* fall back to blocking method */ + // file_write(fd, offset, buf, len, callback, data, freefunc); + } +} + +void +AIODiskFile::close () +{ + assert (!closed); + file_close(fd); + fd = -1; + closed = true; + assert (ioRequestor.getRaw()); + ioRequestor->closeCompleted(); +} + +bool +AIODiskFile::canRead() const +{ + return true; +} + +bool +AIODiskFile::canWrite() const +{ + return true; +} + +int +AIODiskFile::getFD() const +{ + return fd; +} + +bool +AIODiskFile::error() const +{ + return error_; +} + +bool +AIODiskFile::ioInProgress() const +{ + return false; +} diff --git a/src/DiskIO/AIO/AIODiskFile.h b/src/DiskIO/AIO/AIODiskFile.h new file mode 100644 index 0000000000..b20c5516e2 --- /dev/null +++ b/src/DiskIO/AIO/AIODiskFile.h @@ -0,0 +1,79 @@ + +/* + * $Id: AIODiskFile.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_AIODISKFILE_H +#define SQUID_AIODISKFILE_H + +#include "DiskIO/DiskFile.h" +#include "async_io.h" + +class AIODiskIOStrategy; + +class AIODiskFile : public DiskFile +{ + +public: + + friend class AIODiskIOStrategy; + void * operator new (size_t); + void operator delete (void *); + AIODiskFile (char const *path, AIODiskIOStrategy *); + ~AIODiskFile(); + virtual void open (int, mode_t, RefCount); + virtual void create (int, mode_t, RefCount); + virtual void read(ReadRequest *); + virtual void write(WriteRequest *); + virtual void close (); + virtual bool canRead() const; + virtual bool canWrite() const; + + /* During migration only */ + virtual int getFD() const; + + virtual bool error() const; + + /* Inform callers if there is IO in progress */ + virtual bool ioInProgress() const; + +private: + CBDATA_CLASS(AIODiskFile); + void error(bool const &); + int fd; + String path; + AIODiskIOStrategy *strategy; + RefCount ioRequestor; + bool closed; + bool error_; +}; + +#endif /* SQUID_AIODISKFILE_H */ diff --git a/src/DiskIO/AIO/AIODiskIOModule.cc b/src/DiskIO/AIO/AIODiskIOModule.cc new file mode 100644 index 0000000000..dcb6474192 --- /dev/null +++ b/src/DiskIO/AIO/AIODiskIOModule.cc @@ -0,0 +1,70 @@ + +/* + * $Id: AIODiskIOModule.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "squid.h" +#include "AIODiskIOModule.h" +#include "AIODiskIOStrategy.h" +#include "Store.h" + +AIODiskIOModule::AIODiskIOModule() +{ + ModuleAdd(*this); +} + +AIODiskIOModule & +AIODiskIOModule::GetInstance() +{ + return Instance; +} + +void +AIODiskIOModule::init() +{} + +void +AIODiskIOModule::shutdown() +{} + +DiskIOStrategy * +AIODiskIOModule::createStrategy() +{ + return new AIODiskIOStrategy(); +} + +AIODiskIOModule AIODiskIOModule::Instance; + +char const * +AIODiskIOModule::type () const +{ + return "AIO"; +} diff --git a/src/DiskIO/AIO/AIODiskIOModule.h b/src/DiskIO/AIO/AIODiskIOModule.h new file mode 100644 index 0000000000..4e0a8de1b0 --- /dev/null +++ b/src/DiskIO/AIO/AIODiskIOModule.h @@ -0,0 +1,54 @@ + +/* + * $Id: AIODiskIOModule.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_AIODISKIOMODULE_H +#define SQUID_AIODISKIOMODULE_H + +#include "DiskIO/DiskIOModule.h" + +class AIODiskIOModule : public DiskIOModule +{ + +public: + static AIODiskIOModule &GetInstance(); + AIODiskIOModule(); + virtual void init(); + virtual void shutdown(); + virtual char const *type () const; + virtual DiskIOStrategy* createStrategy(); + +private: + static AIODiskIOModule Instance; +}; + +#endif /* SQUID_AIODISKIOMODULE_H */ diff --git a/src/DiskIO/AIO/AIODiskIOStrategy.cc b/src/DiskIO/AIO/AIODiskIOStrategy.cc new file mode 100644 index 0000000000..8fd4c38f7d --- /dev/null +++ b/src/DiskIO/AIO/AIODiskIOStrategy.cc @@ -0,0 +1,229 @@ + +/* + * $Id: AIODiskIOStrategy.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ +/* + * Author: Adrian Chadd + * + * These routines are simple plugin replacements for the file_* routines + * in disk.c . They back-end into the POSIX AIO routines to provide + * a nice and simple async IO framework for COSS. + * + * AIO is suitable for COSS - the only sync operations that the standard + * supports are read/write, and since COSS works on a single file + * per storedir it should work just fine. + */ + +#include "squid.h" +#include "AIODiskIOStrategy.h" +#include "AIODiskFile.h" +#include "DiskIO/IORequestor.h" +#include "DiskIO/ReadRequest.h" +#include "DiskIO/WriteRequest.h" + +AIODiskIOStrategy::AIODiskIOStrategy() +{ + aq.aq_numpending = 0; +} + +AIODiskIOStrategy::~AIODiskIOStrategy() +{ + assert(aq.aq_state == AQ_STATE_SETUP || + aq.aq_numpending == 0); + + sync(); + aq.aq_state = AQ_STATE_NONE; +} + +bool +AIODiskIOStrategy::shedLoad() +{ + return false; +} + +int +AIODiskIOStrategy::load() +{ + return aq.aq_numpending * 1000 / MAX_ASYNCOP; +} + +RefCount +AIODiskIOStrategy::newFile (char const *path) +{ + if (shedLoad()) { + return NULL; + } + + return new AIODiskFile (path, this); +} + +void +AIODiskIOStrategy::sync() +{ + assert(aq.aq_state == AQ_STATE_SETUP); + + /* + * Keep calling callback to complete ops until the queue is empty + * We can't quit when callback returns 0 - some calls may not + * return any completed pending events, but they're still pending! + */ + + while (aq.aq_numpending) + callback(); +} + +void +AIODiskIOStrategy::unlinkFile (char const *) +{} + +/* + * Note: we grab the state and free the state before calling the callback + * because this allows us to cut down the amount of time it'll take + * to find a free slot (since if we call the callback first, we're going + * to probably be allocated the slot _after_ this one..) + * + * I'll make it much more optimal later. + */ +int +AIODiskIOStrategy::callback() +{ + return 0; + int i; + int completed = 0; + int retval, reterr; + FREE *freefunc; + void *cbdata; + int callback_valid; + void *buf; + int fd; + async_queue_entry_t *aqe; + async_queue_entry_type_t type; + + assert(aq.aq_state == AQ_STATE_SETUP); + + /* Loop through all slots */ + + for (i = 0; i < MAX_ASYNCOP; i++) { + if (aq.aq_queue[i].aq_e_state == AQ_ENTRY_USED) { + aqe = &aq.aq_queue[i]; + /* Active, get status */ + reterr = aio_error(&aqe->aq_e_aiocb); + + if (reterr < 0) { + fatal("aio_error returned an error!\n"); + } + + if (reterr != EINPROGRESS) { + /* Get the return code */ + retval = aio_return(&aqe->aq_e_aiocb); + + /* Get the callback parameters */ + freefunc = aqe->aq_e_free; + buf = aqe->aq_e_buf; + fd = aqe->aq_e_fd; + type = aqe->aq_e_type; + callback_valid = cbdataReferenceValidDone(aqe->aq_e_callback_data, &cbdata); + AIODiskFile * theFile = NULL; + void *theFileVoid = NULL; + bool fileOk = cbdataReferenceValidDone(aqe->theFile, &theFileVoid); + + if (fileOk) { + theFile = static_cast(theFileVoid); + } + + /* Free slot */ + bzero(aqe, sizeof(async_queue_entry_t)); + + aqe->aq_e_state = AQ_ENTRY_FREE; + + --aq.aq_numpending; + + /* Callback */ + + if (callback_valid) { + assert (fileOk); + + if (type == AQ_ENTRY_READ) + theFile->ioRequestor->readCompleted((const char *)buf, retval, reterr, static_cast(cbdata)); + + if (type == AQ_ENTRY_WRITE) + theFile->ioRequestor->writeCompleted(reterr,retval, static_cast(cbdata)); + } + + if (type == AQ_ENTRY_WRITE && freefunc) + freefunc(buf); + } + } + } + + return completed; +} + +void +AIODiskIOStrategy::init() +{ + /* Make sure the queue isn't setup */ + assert(aq.aq_state == AQ_STATE_NONE); + + /* Loop through, blanking the queue entries */ + + /* Done */ + aq.aq_state = AQ_STATE_SETUP; +} + +void +AIODiskIOStrategy::statfs(StoreEntry & sentry)const + {} + +ConfigOption * +AIODiskIOStrategy::getOptionTree() const +{ + return NULL; +} + +/* + * find a free aio slot. + * Return the index, or -1 if we can't find one. + */ +int +AIODiskIOStrategy::findSlot() +{ + /* Later we should use something a little more .. efficient :) */ + + for (int i = 0; i < MAX_ASYNCOP; i++) { + if (aq.aq_queue[i].aq_e_state == AQ_ENTRY_FREE) + /* Found! */ + return i; + } + + /* found nothing */ + return -1; +} diff --git a/src/DiskIO/AIO/AIODiskIOStrategy.h b/src/DiskIO/AIO/AIODiskIOStrategy.h new file mode 100644 index 0000000000..7546148662 --- /dev/null +++ b/src/DiskIO/AIO/AIODiskIOStrategy.h @@ -0,0 +1,75 @@ + +/* + * $Id: AIODiskIOStrategy.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_AIODISKIOSTRATEGY_H +#define SQUID_AIODISKIOSTRATEGY_H + +#include "DiskIO/DiskIOStrategy.h" +#include "async_io.h" + +class AIODiskIOStrategy : public DiskIOStrategy +{ + +public: + AIODiskIOStrategy(); + virtual ~AIODiskIOStrategy(); + + virtual bool shedLoad(); + /* What is the current load? 999 = 99.9% */ + virtual int load(); + /* Return a handle for performing IO operations */ + virtual RefCount newFile (char const *path); + /* flush all IO operations */ + virtual void sync(); + /* unlink a file by path */ + virtual void unlinkFile (char const *); + + /* perform any pending callbacks */ + virtual int callback(); + + /* Init per-instance logic */ + virtual void init(); + + /* cachemgr output on the IO instance stats */ + virtual void statfs(StoreEntry & sentry)const; + /* module specific options */ + virtual ConfigOption *getOptionTree() const; + /* a file descriptor */ + int fd; + /* queue of requests */ + async_queue_t aq; + + int findSlot(); +}; + +#endif /* SQUID_AIODISKIOSTRATEGY_H */ diff --git a/src/fs/coss/async_io.h b/src/DiskIO/AIO/async_io.h similarity index 61% rename from src/fs/coss/async_io.h rename to src/DiskIO/AIO/async_io.h index fcc67827ff..f2ea778d0d 100644 --- a/src/fs/coss/async_io.h +++ b/src/DiskIO/AIO/async_io.h @@ -27,16 +27,15 @@ typedef struct _async_queue async_queue_t; /* An async queue entry */ +class AIODiskFile; + struct _async_queue_entry { async_queue_entry_state_t aq_e_state; async_queue_entry_type_t aq_e_type; struct aiocb aq_e_aiocb; - union { - DRCB *read; - DWCB *write; - } aq_e_callback; + AIODiskFile *theFile; void *aq_e_callback_data; FREE *aq_e_free; int aq_e_fd; @@ -52,15 +51,4 @@ struct _async_queue int aq_numpending; /* Num of pending ops */ }; - -/* Functions */ -extern void a_file_read(async_queue_t * q, int fd, void *buf, int req_len, - off_t offset, DRCB * callback, void *data); -extern void a_file_write(async_queue_t * q, int fd, off_t offset, void *buf, - int len, DWCB * callback, void *data, FREE * freefunc); -extern int a_file_callback(async_queue_t * q); -extern void a_file_setupqueue(async_queue_t * q); -extern void a_file_syncqueue(async_queue_t * q); -extern void a_file_closequeue(async_queue_t * q); - #endif diff --git a/src/DiskIO/Blocking/BlockingDiskIOModule.cc b/src/DiskIO/Blocking/BlockingDiskIOModule.cc new file mode 100644 index 0000000000..42e5df2cc0 --- /dev/null +++ b/src/DiskIO/Blocking/BlockingDiskIOModule.cc @@ -0,0 +1,70 @@ + +/* + * $Id: BlockingDiskIOModule.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "squid.h" +#include "BlockingDiskIOModule.h" +#include "BlockingIOStrategy.h" + +BlockingDiskIOModule::BlockingDiskIOModule() +{ + ModuleAdd(*this); +} + +BlockingDiskIOModule & +BlockingDiskIOModule::GetInstance() +{ + return Instance; +} + +void +BlockingDiskIOModule::init() +{} + +void +BlockingDiskIOModule::shutdown() +{} + + +DiskIOStrategy* +BlockingDiskIOModule::createStrategy() +{ + return new BlockingIOStrategy(); +} + +BlockingDiskIOModule BlockingDiskIOModule::Instance; + +char const * +BlockingDiskIOModule::type () const +{ + return "Blocking"; +} diff --git a/src/DiskIO/Blocking/BlockingDiskIOModule.h b/src/DiskIO/Blocking/BlockingDiskIOModule.h new file mode 100644 index 0000000000..e206122e8e --- /dev/null +++ b/src/DiskIO/Blocking/BlockingDiskIOModule.h @@ -0,0 +1,56 @@ + +/* + * $Id: BlockingDiskIOModule.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_BLOCKINGDISKIOMODULE_H +#define SQUID_BLOCKINGDISKIOMODULE_H + +#include "DiskIO/DiskIOModule.h" + +class BlockingDiskIOModule : public DiskIOModule +{ + +public: + static BlockingDiskIOModule &GetInstance(); + BlockingDiskIOModule(); + virtual void init(); + virtual void shutdown(); + virtual char const *type () const; + virtual DiskIOStrategy* createStrategy(); + +private: + static BlockingDiskIOModule Instance; +}; + + + +#endif /* SQUID_BLOCKINGDISKIOMODULE_H */ diff --git a/src/DiskIO/Blocking/BlockingFile.cc b/src/DiskIO/Blocking/BlockingFile.cc new file mode 100644 index 0000000000..49dc2525b8 --- /dev/null +++ b/src/DiskIO/Blocking/BlockingFile.cc @@ -0,0 +1,233 @@ + +/* + * $Id: BlockingFile.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 47 Store Directory Routines + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "BlockingFile.h" +#include "DiskIO/IORequestor.h" +#include "DiskIO/ReadRequest.h" +#include "DiskIO/WriteRequest.h" + +CBDATA_CLASS_INIT(BlockingFile); +void * +BlockingFile::operator new (size_t) +{ + CBDATA_INIT_TYPE(BlockingFile); + BlockingFile *result = cbdataAlloc(BlockingFile); + /* Mark result as being owned - we want the refcounter to do the delete + * call */ + cbdataReference(result); + return result; +} + +void +BlockingFile::operator delete (void *address) +{ + BlockingFile *t = static_cast(address); + cbdataFree(address); + /* And allow the memory to be freed */ + cbdataReferenceDone (t); +} + +BlockingFile::BlockingFile (char const *aPath) : fd (-1), closed (true), error_(false) +{ + assert (aPath); + debug (79,3)("BlockingFile::BlockingFile: %s\n", aPath); + path_ = xstrdup (aPath); +} + +BlockingFile::~BlockingFile() +{ + safe_free (path_); + doClose(); +} + +void +BlockingFile::open (int flags, mode_t mode, IORequestor::Pointer callback) +{ + /* Simulate async calls */ + fd = file_open(path_ , flags); + ioRequestor = callback; + + if (fd < 0) { + debug(79, 3) ("BlockingFile::open: got failure (%d)\n", errno); + error(true); + } else { + closed = false; + store_open_disk_fd++; + debug(79, 3) ("BlockingFile::open: opened FD %d\n", fd); + } + + callback->ioCompletedNotification(); +} + +void +BlockingFile::create (int flags, mode_t mode, IORequestor::Pointer callback) +{ + /* We use the same logic path for open */ + open(flags, mode, callback); +} + + +void BlockingFile::doClose() +{ + if (fd > -1) { + closed = true; + file_close(fd); + store_open_disk_fd--; + fd = -1; + } +} + +void +BlockingFile::close () +{ + debug (79,3)("BlockingFile::close: %p closing for %p\n", this, ioRequestor.getRaw()); + doClose(); + assert (ioRequestor.getRaw()); + ioRequestor->closeCompleted(); +} + +bool +BlockingFile::canRead() const +{ + return fd > -1; +} + +bool +BlockingFile::error() const +{ + if ((fd < 0 && !closed) || error_) + return true; + + return false; +} + +void BlockingFile::error(bool const &aBool) +{ + error_ = aBool; +} + +void +BlockingFile::read(ReadRequest *aRequest) +{ + assert (fd > -1); + assert (ioRequestor.getRaw()); + readRequest = aRequest; + file_read(fd, aRequest->buf, aRequest->len, aRequest->offset, ReadDone, this); +} + +void +BlockingFile::ReadDone(int fd, const char *buf, int len, int errflag, void *my_data) +{ + BlockingFile *myFile = static_cast(my_data); + assert (myFile); + myFile->readDone (fd, buf, len, errflag); +} + +void +BlockingFile::write(WriteRequest *aRequest) +{ + debug(79, 3) ("storeUfsWrite: FD %d\n",fd); + writeRequest = aRequest; + file_write(fd, + aRequest->offset, + (char *)aRequest->buf, + aRequest->len, + WriteDone, + this, + aRequest->free_func); +} + +bool +BlockingFile::ioInProgress()const +{ + /* IO is never pending with UFS */ + return false; +} + +/* === STATIC =========================================================== */ + +void +BlockingFile::readDone(int rvfd, const char *buf, int len, int errflag) +{ + debug (79,3)("BlockingFile::readDone: FD %d\n",rvfd); + assert (fd == rvfd); + + ssize_t rlen; + + if (errflag) { + debug(79, 3) ("BlockingFile::readDone: got failure (%d)\n", errflag); + rlen = -1; + } else { + rlen = (ssize_t) len; + } + + if (errflag == DISK_EOF) + errflag = DISK_OK; /* EOF is signalled by len == 0, not errors... */ + + ReadRequest::Pointer result = readRequest; + + readRequest = NULL; + + ioRequestor->readCompleted(buf, rlen, errflag, result); +} + +void +BlockingFile::WriteDone (int fd, int errflag, size_t len, void *me) +{ + BlockingFile *aFile = static_cast(me); + aFile->writeDone (fd, errflag, len); +} + +void +BlockingFile::writeDone(int rvfd, int errflag, size_t len) +{ + assert (rvfd == fd); + debug(79, 3) ("storeUfsWriteDone: FD %d, len %ld\n", + fd, (long int) len); + + WriteRequest::Pointer result = writeRequest; + writeRequest = NULL; + + if (errflag) { + debug(79, 0) ("storeUfsWriteDone: got failure (%d)\n", errflag); + doClose(); + ioRequestor->writeCompleted (DISK_ERROR,0, result); + return; + } + + ioRequestor->writeCompleted(DISK_OK, len, result); +} + diff --git a/src/DiskIO/Blocking/BlockingFile.h b/src/DiskIO/Blocking/BlockingFile.h new file mode 100644 index 0000000000..f26580f101 --- /dev/null +++ b/src/DiskIO/Blocking/BlockingFile.h @@ -0,0 +1,77 @@ + +/* + * $Id: BlockingFile.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 47 Store Directory Routines + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_BLOCKINGFILE_H +#define SQUID_BLOCKINGFILE_H +#include "DiskIO/DiskFile.h" + +class BlockingFile : public DiskFile +{ + +public: + void *operator new(size_t); + void operator delete(void *); + BlockingFile (char const *path); + ~BlockingFile(); + virtual void open (int, mode_t, RefCount); + virtual void create (int, mode_t, RefCount); + virtual void read(ReadRequest *); + virtual void write(WriteRequest *); + virtual void close (); + virtual bool error() const; + virtual int getFD() const { return fd;} + + virtual bool canRead() const; + virtual bool ioInProgress()const; + +private: + static DRCB ReadDone; + static DWCB WriteDone; + CBDATA_CLASS(BlockingFile); + int fd; + bool closed; + void error (bool const &); + bool error_; + char const *path_; + RefCount ioRequestor; + RefCount readRequest; + RefCount writeRequest; + void doClose(); + void readDone(int fd, const char *buf, int len, int errflag); + void writeDone(int fd, int errflag, size_t len); +}; + +#endif /* SQUID_BLOCKINGFILE_H */ diff --git a/src/DiskIO/Blocking/BlockingIOStrategy.cc b/src/DiskIO/Blocking/BlockingIOStrategy.cc new file mode 100644 index 0000000000..ae0f7fec90 --- /dev/null +++ b/src/DiskIO/Blocking/BlockingIOStrategy.cc @@ -0,0 +1,70 @@ + +/* + * $Id: BlockingIOStrategy.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 47 Store Directory Routines + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "BlockingIOStrategy.h" +#include "BlockingFile.h" +bool +BlockingIOStrategy::shedLoad() +{ + return false; +} + +int +BlockingIOStrategy::load() +{ + /* Return 999 (99.9%) constant load */ + return 999; +} + +DiskFile::Pointer +BlockingIOStrategy::newFile (char const *path) +{ + return new BlockingFile (path); +} + +void +BlockingIOStrategy::unlinkFile(char const *path) +{ +#if USE_UNLINKD + unlinkdUnlink(path); +#elif USE_TRUNCATE + + truncate(path, 0); +#else + + ::unlink(path); +#endif +} diff --git a/src/DiskIO/Blocking/BlockingIOStrategy.h b/src/DiskIO/Blocking/BlockingIOStrategy.h new file mode 100644 index 0000000000..c417dcf7ab --- /dev/null +++ b/src/DiskIO/Blocking/BlockingIOStrategy.h @@ -0,0 +1,51 @@ + +/* + * $Id: BlockingIOStrategy.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 47 Store Directory Routines + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_BLOCKINGIOSTRATEGY_H +#define SQUID_BLOCKINGIOSTRATEGY_H +#include "DiskIO/DiskIOStrategy.h" + +class BlockingIOStrategy : public DiskIOStrategy +{ + +public: + virtual bool shedLoad(); + virtual int load(); + virtual RefCount newFile(char const *path); + virtual void unlinkFile (char const *); +}; + +#endif /* SQUID_BLOCKINGIOSTRATEGY_H */ diff --git a/src/DiskIO/Blocking/DiskIOBlocking.cc b/src/DiskIO/Blocking/DiskIOBlocking.cc new file mode 100644 index 0000000000..0da58c37af --- /dev/null +++ b/src/DiskIO/Blocking/DiskIOBlocking.cc @@ -0,0 +1,32 @@ + +/* + * $Id: DiskIOBlocking.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003 Robert Collins + */ diff --git a/src/DiskIO/DiskDaemon/DiskDaemonDiskIOModule.cc b/src/DiskIO/DiskDaemon/DiskDaemonDiskIOModule.cc new file mode 100644 index 0000000000..0d8e38f882 --- /dev/null +++ b/src/DiskIO/DiskDaemon/DiskDaemonDiskIOModule.cc @@ -0,0 +1,110 @@ + +/* + * $Id: DiskDaemonDiskIOModule.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "squid.h" +#include "DiskDaemonDiskIOModule.h" +#include "DiskdIOStrategy.h" +#include "Store.h" + +DiskDaemonDiskIOModule::DiskDaemonDiskIOModule() : initialised(false) +{ + ModuleAdd(*this); +} + +DiskDaemonDiskIOModule & +DiskDaemonDiskIOModule::GetInstance() +{ + return Instance; +} + +void +DiskDaemonDiskIOModule::init() +{ + /* We may be reused - for instance in coss - eventually. + * When we do, we either need per-using-module stats ( + * no singleton pattern), or we need to refcount the + * initialisation level and handle multiple clients. + * RBC - 20030718. + */ + assert(!initialised); + memset(&diskd_stats, '\0', sizeof(diskd_stats)); + cachemgrRegister("diskd", "DISKD Stats", Stats, 0, 1); + + debug(47, 1) ("diskd started\n"); + initialised = true; +} + +void +DiskDaemonDiskIOModule::shutdown() +{ + initialised = false; +} + +DiskIOStrategy * +DiskDaemonDiskIOModule::createStrategy() +{ + return new DiskdIOStrategy(); +} + +DiskDaemonDiskIOModule DiskDaemonDiskIOModule::Instance; + +void +DiskDaemonDiskIOModule::Stats(StoreEntry * sentry) +{ + storeAppendPrintf(sentry, "sent_count: %d\n", diskd_stats.sent_count); + storeAppendPrintf(sentry, "recv_count: %d\n", diskd_stats.recv_count); + storeAppendPrintf(sentry, "max_away: %d\n", diskd_stats.max_away); + storeAppendPrintf(sentry, "max_shmuse: %d\n", diskd_stats.max_shmuse); + storeAppendPrintf(sentry, "open_fail_queue_len: %d\n", diskd_stats.open_fail_queue_len); + storeAppendPrintf(sentry, "block_queue_len: %d\n", diskd_stats.block_queue_len); + diskd_stats.max_away = diskd_stats.max_shmuse = 0; + storeAppendPrintf(sentry, "\n OPS SUCCESS FAIL\n"); + storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", + "open", diskd_stats.open.ops, diskd_stats.open.success, diskd_stats.open.fail); + storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", + "create", diskd_stats.create.ops, diskd_stats.create.success, diskd_stats.create.fail); + storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", + "close", diskd_stats.close.ops, diskd_stats.close.success, diskd_stats.close.fail); + storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", + "unlink", diskd_stats.unlink.ops, diskd_stats.unlink.success, diskd_stats.unlink.fail); + storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", + "read", diskd_stats.read.ops, diskd_stats.read.success, diskd_stats.read.fail); + storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", + "write", diskd_stats.write.ops, diskd_stats.write.success, diskd_stats.write.fail); +} + +char const * +DiskDaemonDiskIOModule::type () const +{ + return "DiskDaemon"; +} diff --git a/src/DiskIO/DiskDaemon/DiskDaemonDiskIOModule.h b/src/DiskIO/DiskDaemon/DiskDaemonDiskIOModule.h new file mode 100644 index 0000000000..dc9b1e3605 --- /dev/null +++ b/src/DiskIO/DiskDaemon/DiskDaemonDiskIOModule.h @@ -0,0 +1,56 @@ + +/* + * $Id: DiskDaemonDiskIOModule.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_DISKDAEMONDISKIOMODULE_H +#define SQUID_DISKDAEMONDISKIOMODULE_H + +#include "DiskIO/DiskIOModule.h" + +class DiskDaemonDiskIOModule : public DiskIOModule +{ + +public: + static DiskDaemonDiskIOModule &GetInstance(); + DiskDaemonDiskIOModule(); + virtual void init(); + virtual void shutdown(); + virtual char const *type () const; + virtual DiskIOStrategy* createStrategy(); + +private: + static void Stats(StoreEntry * sentry); + static DiskDaemonDiskIOModule Instance; + bool initialised; +}; + +#endif /* SQUID_DISKDAEMONDISKIOMODULE_H */ diff --git a/src/DiskIO/DiskDaemon/DiskdFile.cc b/src/DiskIO/DiskDaemon/DiskdFile.cc new file mode 100644 index 0000000000..96cb41d5c9 --- /dev/null +++ b/src/DiskIO/DiskDaemon/DiskdFile.cc @@ -0,0 +1,432 @@ + +/* + * $Id: DiskdFile.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 79 Squid-side DISKD I/O functions. + * AUTHOR: Duane Wessels + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * CopyRight (c) 2003, Robert Collins + */ + +#include "squid.h" + +#include +#include +#include + +#include "DiskdFile.h" +#include "ConfigOption.h" +#include "diomsg.h" + +#include "DiskdIOStrategy.h" +#include "DiskIO/IORequestor.h" +#include "DiskIO/ReadRequest.h" +#include "DiskIO/WriteRequest.h" +CBDATA_CLASS_INIT(DiskdFile); + +void * +DiskdFile::operator new (size_t) +{ + CBDATA_INIT_TYPE(DiskdFile); + DiskdFile *result = cbdataAlloc(DiskdFile); + /* Mark result as being owned - we want the refcounter to do the delete + * call */ + cbdataReference(result); + debug (79,3)("diskdFile with base %p allocating\n", result); + return result; +} + +void +DiskdFile::operator delete (void *address) +{ + debug (79,3)("diskdFile with base %p deleting\n",address); + DiskdFile *t = static_cast(address); + cbdataFree(address); + /* And allow the memory to be freed */ + cbdataReferenceDone (t); +} + +DiskdFile::DiskdFile (char const *aPath, DiskdIOStrategy *anIO) : errorOccured (false), IO(anIO), + inProgressIOs (0) +{ + assert (aPath); + debug (79,3)("DiskdFile::DiskdFile: %s\n", aPath); + path_ = xstrdup (aPath); + id = diskd_stats.sio_id++; +} + +DiskdFile::~DiskdFile() +{ + assert (inProgressIOs == 0); + safe_free (path_); +} + +void +DiskdFile::open (int flags, mode_t aMode, IORequestor::Pointer callback) +{ + debug (79,3)("DiskdFile::open: %p opening for %p\n", this, callback.getRaw()); + assert (ioRequestor.getRaw() == NULL); + ioRequestor = callback; + assert (callback.getRaw()); + mode = flags; + off_t shm_offset; + char *buf = (char *)IO->shm.get(&shm_offset); + xstrncpy(buf, path_, SHMBUF_BLKSZ); + ioAway(); + int x = IO->send(_MQD_OPEN, + id, + this, + strlen(buf) + 1, + mode, + shm_offset, + NULL); + + if (x < 0) { + ioCompleted(); + errorOccured = true; + // IO->shm.put (shm_offset); + ioRequestor->ioCompletedNotification(); + ioRequestor = NULL; + } + + diskd_stats.open.ops++; +} + +void +DiskdFile::create (int flags, mode_t aMode, IORequestor::Pointer callback) +{ + debug (79,3)("DiskdFile::create: %p creating for %p\n", this, callback.getRaw()); + assert (ioRequestor.getRaw() == NULL); + ioRequestor = callback; + assert (callback.getRaw()); + mode = flags; + off_t shm_offset; + char *buf = (char *)IO->shm.get(&shm_offset); + xstrncpy(buf, path_, SHMBUF_BLKSZ); + ioAway(); + int x = IO->send(_MQD_CREATE, + id, + this, + strlen(buf) + 1, + mode, + shm_offset, + NULL); + + if (x < 0) { + ioCompleted(); + errorOccured = true; + // IO->shm.put (shm_offset); + debug(79, 1) ("storeDiskdSend CREATE: %s\n", xstrerror()); + notifyClient(); + ioRequestor = NULL; + return; + } + + diskd_stats.create.ops++; +} + +void +DiskdFile::read(ReadRequest *aRead) +{ + assert (ioRequestor.getRaw() != NULL); + off_t shm_offset; + char *rbuf = (char *)IO->shm.get(&shm_offset); + assert(rbuf); + ioAway(); + int x = IO->send(_MQD_READ, + id, + this, + (int) aRead->len, + (int) aRead->offset, + shm_offset, + aRead); + + if (x < 0) { + ioCompleted(); + errorOccured = true; + // IO->shm.put (shm_offset); + debug(79, 1) ("storeDiskdSend READ: %s\n", xstrerror()); + notifyClient(); + ioRequestor = NULL; + return; + } + + diskd_stats.read.ops++; +} + +void +DiskdFile::close() +{ + debug (79,3)("DiskdFile::close: %p closing for %p\n", this, ioRequestor.getRaw()); + assert (ioRequestor.getRaw()); + ioAway(); + int x = IO->send(_MQD_CLOSE, + id, + this, + 0, + 0, + -1, + NULL); + + if (x < 0) { + ioCompleted(); + errorOccured = true; + debug(79, 1) ("storeDiskdSend CLOSE: %s\n", xstrerror()); + notifyClient(); + ioRequestor = NULL; + return; + } + + diskd_stats.close.ops++; +} + +bool +DiskdFile::error() const +{ + return errorOccured; +} + +bool +DiskdFile::canRead() const +{ + return !error(); +} + +bool +DiskdFile::canNotifyClient() const +{ + if (!ioRequestor.getRaw()) { + debug (79,3)("DiskdFile::canNotifyClient: No ioRequestor to notify\n"); + return false; + } + + return true; +} + +void +DiskdFile::notifyClient() +{ + if (!canNotifyClient()) { + return; + } + + ioRequestor->ioCompletedNotification(); +} + +void +DiskdFile::completed(diomsg *M) +{ + assert (M->newstyle); + + switch (M->mtype) { + + case _MQD_OPEN: + openDone(M); + break; + + case _MQD_CREATE: + createDone(M); + break; + + case _MQD_CLOSE: + closeDone(M); + break; + + case _MQD_READ: + readDone(M); + break; + + case _MQD_WRITE: + writeDone(M); + break; + + case _MQD_UNLINK: + assert (0); + break; + + default: + assert(0); + break; + } +} + +void +DiskdFile::openDone(diomsg *M) +{ + statCounter.syscalls.disk.opens++; + debug(79, 3) ("storeDiskdOpenDone: status %d\n", M->status); + + if (M->status < 0) { + diskd_stats.open.fail++; + errorOccured = true; + } else { + diskd_stats.open.success++; + } + + ioCompleted(); + notifyClient(); +} + +void +DiskdFile::createDone(diomsg *M) +{ + statCounter.syscalls.disk.opens++; + debug(79, 3) ("storeDiskdCreateDone: status %d\n", M->status); + + if (M->status < 0) { + diskd_stats.create.fail++; + errorOccured = true; + } else { + diskd_stats.create.success++; + } + + ioCompleted(); + notifyClient(); +} + +void +DiskdFile::write(WriteRequest *aRequest) +{ + debugs(79, 3, "DiskdFile::write: this " << (void *)this << ", buf " << (void *)aRequest->buf << ", off " << aRequest->offset << ", len " << aRequest->len); + off_t shm_offset; + char *sbuf = (char *)IO->shm.get(&shm_offset); + xmemcpy(sbuf, aRequest->buf, aRequest->len); + + if (aRequest->free_func) + aRequest->free_func(const_cast(aRequest->buf)); + + ioAway(); + + int x = IO->send(_MQD_WRITE, + id, + this, + (int) aRequest->len, + (int) aRequest->offset, + shm_offset, + aRequest); + + if (x < 0) { + ioCompleted() + ; + errorOccured = true; + debug(79, 1) ("storeDiskdSend WRITE: %s\n", xstrerror()); + // IO->shm.put (shm_offset); + notifyClient(); + ioRequestor = NULL; + return; + } + + diskd_stats.write.ops++; +} + +void +DiskdFile::ioAway() +{ + ++inProgressIOs; +} + +void +DiskdFile::ioCompleted() +{ + --inProgressIOs; +} + +void +DiskdFile::closeDone(diomsg * M) +{ + statCounter.syscalls.disk.closes++; + debug(79, 3) ("DiskdFile::closeDone: status %d\n", M->status); + + if (M->status < 0) { + diskd_stats.close.fail++; + errorOccured = true; + } else { + diskd_stats.close.success++; + } + + ioCompleted(); + + if (canNotifyClient()) + ioRequestor->closeCompleted(); + + ioRequestor = NULL; +} + +void +DiskdFile::readDone(diomsg * M) +{ + statCounter.syscalls.disk.reads++; + debug(79, 3) ("DiskdFile::readDone: status %d\n", M->status); + assert (M->requestor); + ReadRequest::Pointer readRequest = dynamic_cast(M->requestor); + /* remove the free protection */ + readRequest->RefCountDereference(); + + if (M->status < 0) { + diskd_stats.read.fail++; + ioCompleted(); + errorOccured = true; + ioRequestor->readCompleted(NULL, -1, DISK_ERROR, readRequest); + return; + } + + diskd_stats.read.success++; + + ioCompleted(); + ioRequestor->readCompleted (IO->shm.buf + M->shm_offset, M->status, DISK_OK, readRequest); +} + +void +DiskdFile::writeDone(diomsg *M) +{ + statCounter.syscalls.disk.writes++; + debug(79, 3) ("storeDiskdWriteDone: status %d\n", M->status); + assert (M->requestor); + WriteRequest::Pointer writeRequest = dynamic_cast(M->requestor); + /* remove the free protection */ + writeRequest->RefCountDereference(); + + if (M->status < 0) { + errorOccured = true; + diskd_stats.write.fail++; + ioCompleted(); + ioRequestor->writeCompleted (DISK_ERROR,0, writeRequest); + return; + } + + diskd_stats.write.success++; + ioCompleted(); + ioRequestor->writeCompleted (DISK_OK,M->status, writeRequest); +} + +bool +DiskdFile::ioInProgress()const +{ + return inProgressIOs != 0; +} diff --git a/src/DiskIO/DiskDaemon/DiskdFile.h b/src/DiskIO/DiskDaemon/DiskdFile.h new file mode 100644 index 0000000000..d83bbeddd1 --- /dev/null +++ b/src/DiskIO/DiskDaemon/DiskdFile.h @@ -0,0 +1,88 @@ + +/* + * $Id: DiskdFile.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 79 Squid-side DISKD I/O functions. + * AUTHOR: Duane Wessels + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef __STORE_DISKDFILE_H__ +#define __STORE_DISKDFILE_H__ + +#include "DiskIO/DiskFile.h" + +class DiskdIOStrategy; + +struct diomsg; + +class DiskdFile : public DiskFile +{ + +public: + void * operator new (size_t); + void operator delete (void *); + DiskdFile (char const *path, DiskdIOStrategy *); + ~DiskdFile(); + virtual void open (int, mode_t, RefCount); + virtual void create (int, mode_t, RefCount); + virtual void read(ReadRequest *); + virtual void write(WriteRequest *); + virtual void close (); + virtual bool error() const; + virtual bool canRead() const; + virtual bool ioInProgress()const; + + /* Temporary */ + int getID() const {return id;} + + void completed (diomsg *); + +private: + int id; + char const *path_; + bool errorOccured; + DiskdIOStrategy *IO; + RefCount ioRequestor; + CBDATA_CLASS(DiskdFile); + void openDone(diomsg *); + void createDone (diomsg *); + void readDone (diomsg *); + void writeDone (diomsg *); + void closeDone (diomsg *); + int mode; + void notifyClient(); + bool canNotifyClient() const; + void ioAway(); + void ioCompleted(); + size_t inProgressIOs; +}; + +#endif diff --git a/src/DiskIO/DiskDaemon/DiskdIOStrategy.cc b/src/DiskIO/DiskDaemon/DiskdIOStrategy.cc new file mode 100644 index 0000000000..49de2399ed --- /dev/null +++ b/src/DiskIO/DiskDaemon/DiskdIOStrategy.cc @@ -0,0 +1,637 @@ + +/* + * $Id: DiskdIOStrategy.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 79 Squid-side DISKD I/O functions. + * AUTHOR: Duane Wessels + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "squid.h" + +#include +#include +#include +#include "DiskdIOStrategy.h" +#include "ConfigOption.h" +#include "DiskIO/DiskFile.h" +#include "DiskdFile.h" +#include "diomsg.h" +/* for statfs */ +#include "Store.h" + +diskd_stats_t diskd_stats; + +size_t DiskdIOStrategy::nextInstanceID (0); +const int diomsg::msg_snd_rcv_sz = sizeof(diomsg) - sizeof(mtyp_t); + +size_t +DiskdIOStrategy::newInstance() +{ + return ++nextInstanceID; +} + +bool +DiskdIOStrategy::shedLoad() +{ + /* + * Fail on open() if there are too many requests queued. + */ + + if (away > magic1) { + debug(79, 3) ("storeDiskdIO::shedLoad: Shedding, too many requests away\n"); + + return true; + } + + return false; +} + +int +DiskdIOStrategy::load() +{ + /* Calculate the storedir load relative to magic2 on a scale of 0 .. 1000 */ + /* the parse function guarantees magic2 is positivie */ + return away * 1000 / magic2; +} + +void +DiskdIOStrategy::openFailed() +{ + diskd_stats.open_fail_queue_len++; +} + +DiskFile::Pointer +DiskdIOStrategy::newFile(char const *path) +{ + if (shedLoad()) { + openFailed(); + return NULL; + } + + return new DiskdFile (path, this); +} + +DiskdIOStrategy::DiskdIOStrategy() : magic1(64), magic2(72), away(0) , smsgid(-1), rmsgid(-1), wfd(-1) , instanceID(newInstance()) +{} + +void +DiskdIOStrategy::unlinkFile(char const *path) +{ + if (shedLoad()) { + /* Damn, we need to issue a sync unlink here :( */ + debug(79, 2) ("storeDiskUnlink: Out of queue space, sync unlink\n"); +#if USE_UNLINKD + + unlinkdUnlink(path); +#elif USE_TRUNCATE + + truncate(path, 0); +#else + + unlink(path); +#endif + + return; + } + + /* We can attempt a diskd unlink */ + int x; + + off_t shm_offset; + + char *buf; + + buf = (char *)shm.get(&shm_offset); + + xstrncpy(buf, path, SHMBUF_BLKSZ); + + x = send(_MQD_UNLINK, + 0, + (StoreIOState::Pointer )NULL, + 0, + 0, + shm_offset); + + if (x < 0) { + debug(79, 1) ("storeDiskdSend UNLINK: %s\n", xstrerror()); + ::unlink(buf); /* XXX EWW! */ + // shm.put (shm_offset); + } + + diskd_stats.unlink.ops++; +} + +void +DiskdIOStrategy::init() +{ + int x; + int rfd; + int ikey; + const char *args[5]; + char skey1[32]; + char skey2[32]; + char skey3[32]; + + ikey = (getpid() << 10) + (instanceID << 2); + ikey &= 0x7fffffff; + smsgid = msgget((key_t) ikey, 0700 | IPC_CREAT); + + if (smsgid < 0) { + debug(50, 0) ("storeDiskdInit: msgget: %s\n", xstrerror()); + fatal("msgget failed"); + } + + rmsgid = msgget((key_t) (ikey + 1), 0700 | IPC_CREAT); + + if (rmsgid < 0) { + debug(50, 0) ("storeDiskdInit: msgget: %s\n", xstrerror()); + fatal("msgget failed"); + } + + shm.init(ikey, magic2); + snprintf(skey1, 32, "%d", ikey); + snprintf(skey2, 32, "%d", ikey + 1); + snprintf(skey3, 32, "%d", ikey + 2); + args[0] = "diskd"; + args[1] = skey1; + args[2] = skey2; + args[3] = skey3; + args[4] = NULL; + x = ipcCreate(IPC_STREAM, + Config.Program.diskd, + args, + "diskd", + &rfd, + &wfd); + + if (x < 0) + fatalf("execl: %s", Config.Program.diskd); + + if (rfd != wfd) + comm_close(rfd); + + fd_note(wfd, "squid -> diskd"); + + commSetTimeout(wfd, -1, NULL, NULL); + + commSetNonBlocking(wfd); + + comm_quick_poll_required(); +} + +/* + * SHM manipulation routines + */ +void +SharedMemory::put (off_t offset) +{ + int i; + assert(offset >= 0); + assert(offset < nbufs * SHMBUF_BLKSZ); + i = offset / SHMBUF_BLKSZ; + assert(i < nbufs); + assert(CBIT_TEST(inuse_map, i)); + CBIT_CLR(inuse_map, i); + --diskd_stats.shmbuf_count; +} + +void * + +SharedMemory::get + (off_t * shm_offset) +{ + char *aBuf = NULL; + int i; + + for (i = 0; i < nbufs; i++) { + if (CBIT_TEST(inuse_map, i)) + continue; + + CBIT_SET(inuse_map, i); + + *shm_offset = i * SHMBUF_BLKSZ; + + aBuf = buf + (*shm_offset); + + break; + } + + assert(aBuf); + assert(aBuf >= buf); + assert(aBuf < buf + (nbufs * SHMBUF_BLKSZ)); + diskd_stats.shmbuf_count++; + + if (diskd_stats.max_shmuse < diskd_stats.shmbuf_count) + diskd_stats.max_shmuse = diskd_stats.shmbuf_count; + + return aBuf; +} + +void +SharedMemory::init(int ikey, int magic2) +{ + nbufs = (int)(magic2 * 1.3); + id = shmget((key_t) (ikey + 2), + nbufs * SHMBUF_BLKSZ, 0600 | IPC_CREAT); + + if (id < 0) { + debug(50, 0) ("storeDiskdInit: shmget: %s\n", xstrerror()); + fatal("shmget failed"); + } + + buf = (char *)shmat(id, NULL, 0); + + if (buf == (void *) -1) { + debug(50, 0) ("storeDiskdInit: shmat: %s\n", xstrerror()); + fatal("shmat failed"); + } + + inuse_map = (char *)xcalloc((nbufs + 7) / 8, 1); + diskd_stats.shmbuf_count += nbufs; + + for (int i = 0; i < nbufs; i++) { + CBIT_SET(inuse_map, i); + put (i * SHMBUF_BLKSZ); + } +} + +void +DiskdIOStrategy::unlinkDone(diomsg * M) +{ + debug(79, 3) ("storeDiskdUnlinkDone: file %s status %d\n",shm.buf + M->shm_offset, + M->status); + statCounter.syscalls.disk.unlinks++; + + if (M->status < 0) + diskd_stats.unlink.fail++; + else + diskd_stats.unlink.success++; +} + +void +DiskdIOStrategy::handle(diomsg * M) +{ + if (!cbdataReferenceValid (M->callback_data)) { + /* I.e. already closed file + * - say when we have a error opening after + * a read was already queued + */ + debug(79, 3) ("storeDiskdHandle: Invalid callback_data %p\n", + M->callback_data); + cbdataReferenceDone (M->callback_data); + return; + } + + + if (M->newstyle) { + DiskdFile *theFile = (DiskdFile *)M->callback_data; + theFile->RefCountDereference(); + theFile->completed (M); + } else + switch (M->mtype) { + + case _MQD_OPEN: + + case _MQD_CREATE: + + case _MQD_CLOSE: + + case _MQD_READ: + + case _MQD_WRITE: + assert (0); + break; + + case _MQD_UNLINK: + unlinkDone(M); + break; + + default: + assert(0); + break; + } + + cbdataReferenceDone (M->callback_data); +} + +int +DiskdIOStrategy::send(int mtype, int id, DiskdFile *theFile, int size, int offset, off_t shm_offset, RefCountable_ *requestor) +{ + int x; + diomsg M; + static int send_errors = 0; + static int last_seq_no = 0; + static int seq_no = 0; + M.mtype = mtype; + M.callback_data = cbdataReference(theFile); + theFile->RefCountReference(); + M.requestor = requestor; + + if (requestor) + requestor->RefCountReference(); + + M.size = size; + + M.offset = offset; + + M.status = -1; + + M.shm_offset = (int) shm_offset; + + M.id = id; + + M.seq_no = ++seq_no; + + M.newstyle = true; + + if (M.seq_no < last_seq_no) + debug(79, 1) ("WARNING: sequencing out of order\n"); + + debugs (79,9, "sending with" << smsgid <<" " << &M << " " < magic2) { + + struct timeval delay = {0, 1}; + + select(0, NULL, NULL, NULL, &delay); + storeDirCallback(); + + if (delay.tv_usec < 1000000) + delay.tv_usec <<= 1; + } + + return x; +} + +int +DiskdIOStrategy::send(int mtype, int id, StoreIOState::Pointer sio, int size, int offset, off_t shm_offset) +{ + int x; + diomsg M; + static int send_errors = 0; + static int last_seq_no = 0; + static int seq_no = 0; + M.mtype = mtype; + M.callback_data = cbdataReference(sio.getRaw()); + M.size = size; + M.offset = offset; + M.status = -1; + M.shm_offset = (int) shm_offset; + M.id = id; + M.seq_no = ++seq_no; + M.newstyle = false; + + if (M.seq_no < last_seq_no) + debug(79, 1) ("WARNING: sequencing out of order\n"); + + x = msgsnd(smsgid, &M, diomsg::msg_snd_rcv_sz, IPC_NOWAIT); + + last_seq_no = M.seq_no; + + if (0 == x) { + diskd_stats.sent_count++; + away++; + } else { + debug(79, 1) ("storeDiskdSend: msgsnd: %s\n", xstrerror()); + cbdataReferenceDone(M.callback_data); + assert(++send_errors < 100); + } + + /* + * We have to drain the queue here if necessary. If we don't, + * then we can have a lot of messages in the queue (probably + * up to 2*magic1) and we can run out of shared memory buffers. + */ + /* + * Note that we call storeDirCallback (for all SDs), rather + * than storeDiskdDirCallback for just this SD, so that while + * we're "blocking" on this SD we can also handle callbacks + * from other SDs that might be ready. + */ + while (away > magic2) { + + struct timeval delay = {0, 1}; + + select(0, NULL, NULL, NULL, &delay); + storeDirCallback(); + + if (delay.tv_usec < 1000000) + delay.tv_usec <<= 1; + } + + return x; +} + +ConfigOption * +DiskdIOStrategy::getOptionTree() const +{ + ConfigOptionVector *result = new ConfigOptionVector; + result->options.push_back(new ConfigOptionAdapter(*const_cast(this), &DiskdIOStrategy::optionQ1Parse, &DiskdIOStrategy::optionQ1Dump)); + result->options.push_back(new ConfigOptionAdapter(*const_cast(this), &DiskdIOStrategy::optionQ2Parse, &DiskdIOStrategy::optionQ2Dump)); + return result; +} + +bool +DiskdIOStrategy::optionQ1Parse(const char *name, const char *value, int reconfiguring) +{ + if (strcmp(name, "Q1") != 0) + return false; + + int old_magic1 = magic1; + + magic1 = atoi(value); + + if (!reconfiguring) + return true; + + if (old_magic1 < magic1) { + /* + * This is because shm.nbufs is computed at startup, when + * we call shmget(). We can't increase the Q1/Q2 parameters + * beyond their initial values because then we might have + * more "Q2 messages" than shared memory chunks, and this + * will cause an assertion in storeDiskdShmGet(). + */ + /* TODO: have DiskdIO hold a link to the swapdir, to allow detailed reporting again */ + debug(3, 1) ("WARNING: cannot increase cache_dir Q1 value while Squid is running.\n"); + magic1 = old_magic1; + return true; + } + + if (old_magic1 != magic1) + debug(3, 1) ("cache_dir new Q1 value '%d'\n", + magic1); + + return true; +} + +void +DiskdIOStrategy::optionQ1Dump(StoreEntry * e) const +{ + storeAppendPrintf(e, " Q1=%d", magic1); +} + +bool +DiskdIOStrategy::optionQ2Parse(const char *name, const char *value, int reconfiguring) +{ + if (strcmp(name, "Q2") != 0) + return false; + + int old_magic2 = magic2; + + magic2 = atoi(value); + + if (!reconfiguring) + return true; + + if (old_magic2 < magic2) { + /* See comments in Q1 function above */ + debug(3, 1) ("WARNING: cannot increase cache_dir Q2 value while Squid is running.\n"); + magic2 = old_magic2; + return true; + } + + if (old_magic2 != magic2) + debug(3, 1) ("cache_dir new Q2 value '%d'\n", + magic2); + + return true; +} + +void +DiskdIOStrategy::optionQ2Dump(StoreEntry * e) const +{ + storeAppendPrintf(e, " Q2=%d", magic2); +} + +/* + * Sync any pending data. We just sit around and read the queue + * until the data has finished writing. + */ +void +DiskdIOStrategy::sync() +{ + static time_t lastmsg = 0; + + while (away > 0) { + if (squid_curtime > lastmsg) { + debug(47, 1) ("storeDiskdDirSync: %d messages away\n", + away); + lastmsg = squid_curtime; + } + + callback(); + } +} + + +/* + * Handle callbacks. If we have more than magic2 requests away, we block + * until the queue is below magic2. Otherwise, we simply return when we + * don't get a message. + */ + +int +DiskdIOStrategy::callback() +{ + diomsg M; + int x; + int retval = 0; + + if (away >= magic2) { + diskd_stats.block_queue_len++; + retval = 1; + /* We might not have anything to do, but our queue + * is full.. */ + } + + if (diskd_stats.sent_count - diskd_stats.recv_count > + diskd_stats.max_away) { + diskd_stats.max_away = diskd_stats.sent_count - diskd_stats.recv_count; + } + + while (1) { +#ifdef ALWAYS_ZERO_BUFFERS + memset(&M, '\0', sizeof(M)); +#endif + + x = msgrcv(rmsgid, &M, diomsg::msg_snd_rcv_sz, 0, IPC_NOWAIT); + + if (x < 0) + break; + else if (x != diomsg::msg_snd_rcv_sz) { + debug(47, 1) ("storeDiskdDirCallback: msgget returns %d\n", + x); + break; + } + + diskd_stats.recv_count++; + --away; + handle(&M); + retval = 1; /* Return that we've actually done some work */ + + if (M.shm_offset > -1) + shm.put ((off_t) M.shm_offset); + } + + return retval; +} + +void +DiskdIOStrategy::statfs(StoreEntry & sentry)const +{ + storeAppendPrintf(&sentry, "Pending operations: %d\n", away); +} diff --git a/src/DiskIO/DiskDaemon/DiskdIOStrategy.h b/src/DiskIO/DiskDaemon/DiskdIOStrategy.h new file mode 100644 index 0000000000..2873a07ab8 --- /dev/null +++ b/src/DiskIO/DiskDaemon/DiskdIOStrategy.h @@ -0,0 +1,142 @@ + +/* + * $Id: DiskdIOStrategy.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 79 Squid-side DISKD I/O functions. + * AUTHOR: Duane Wessels + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef __STORE_DISKDIOSTRATEGY_H__ +#define __STORE_DISKDIOSTRATEGY_H__ + +/* + * magic2 is the point at which we start blocking on msgsnd/msgrcv. + * If a queue has magic2 (or more) messages away, then we read the + * queue until the level falls below magic2. Recommended value + * is 75% of SHMBUFS. magic1 is the number of messages away which we + * stop allowing open/create for. + */ + +struct diomsg; + +class SharedMemory +{ + +public: + void put(off_t); + + void *get + (off_t *); + + void init (int ikey, int magic2); + + int nbufs; + + char *buf; + + char *inuse_map; + + int id; +}; + +#include "DiskIO/DiskIOStrategy.h" + +class DiskFile; + +class DiskdFile; + +class ReadRequest; + +class DiskdIOStrategy : public DiskIOStrategy +{ + +public: + DiskdIOStrategy(); + virtual bool shedLoad(); + virtual int load(); + virtual RefCount newFile(char const *path); + virtual void unlinkFile (char const *); + virtual ConfigOption *getOptionTree() const; + virtual void init(); + virtual void sync(); + virtual int callback(); + virtual void statfs(StoreEntry & sentry)const; + int send(int mtype, int id, DiskdFile *theFile, int size, int offset, off_t shm_offset, RefCountable_ *); + /* public for accessing return address's */ + SharedMemory shm; + +private: + static size_t newInstance(); + static size_t nextInstanceID; + void openFailed(); + bool optionQ1Parse(char const *option, const char *value, int reconfiguring); + void optionQ1Dump(StoreEntry * e) const; + bool optionQ2Parse(char const *option, const char *value, int reconfiguring); + void optionQ2Dump(StoreEntry * e) const; + int send(int mtype, int id, RefCount sio, int size, int offset, off_t shm_offset); + void handle(diomsg * M); + void unlinkDone(diomsg * M); + int magic1; + int magic2; + int away; + int smsgid; + int rmsgid; + int wfd; + size_t instanceID; +}; + +#define SHMBUF_BLKSZ SM_PAGE_SIZE + + +struct diskd_stats_t +{ + int open_fail_queue_len; + int block_queue_len; + int max_away; + int max_shmuse; + int shmbuf_count; + int sent_count; + int recv_count; + int sio_id; + + struct + { + int ops; + int success; + int fail; + } + + open, create, close, unlink, read, write; +}; + +extern diskd_stats_t diskd_stats; + +#endif diff --git a/src/fs/diskd/dio.h b/src/DiskIO/DiskDaemon/diomsg.h similarity index 67% rename from src/fs/diskd/dio.h rename to src/DiskIO/DiskDaemon/diomsg.h index e78fad69d5..c35894e5cc 100644 --- a/src/fs/diskd/dio.h +++ b/src/DiskIO/DiskDaemon/diomsg.h @@ -1,13 +1,11 @@ /* - * dio.h + * diomsg.h * * Internal declarations for the diskd routines */ -#ifndef __STORE_DIO_H__ -#define __STORE_DIO_H__ - -typedef struct _diomsg diomsg; +#ifndef __SQUID_DIOMSG_H__ +#define __SQUID_DIOMSG_H__ enum { _MQD_NOP, @@ -19,19 +17,22 @@ enum { _MQD_UNLINK }; -struct _diomsg +struct RefCountable_; + +struct diomsg { mtyp_t mtype; int id; int seq_no; void * callback_data; + RefCountable_ * requestor; int size; int offset; int status; bool newstyle; int shm_offset; + static const int msg_snd_rcv_sz; }; -static const int msg_snd_rcv_sz = sizeof(diomsg) - sizeof(mtyp_t); #endif diff --git a/src/fs/diskd/diskd.cc b/src/DiskIO/DiskDaemon/diskd.cc similarity index 94% rename from src/fs/diskd/diskd.cc rename to src/DiskIO/DiskDaemon/diskd.cc index d9c8541e68..7bfd07702e 100644 --- a/src/fs/diskd/diskd.cc +++ b/src/DiskIO/DiskDaemon/diskd.cc @@ -1,5 +1,5 @@ /* - * $Id: diskd.cc,v 1.15 2003/02/21 22:50:40 robertc Exp $ + * $Id: diskd.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ * * DEBUG: section -- External DISKD process implementation. * AUTHOR: Harvest Derived @@ -39,13 +39,14 @@ #include #include -#include "dio.h" +#include "DiskIO/DiskDaemon/diomsg.h" #undef assert #include -#define DEBUG(LEVEL) if (LEVEL <= DebugLevel) +const int diomsg::msg_snd_rcv_sz = sizeof(diomsg) - sizeof(mtyp_t); +#define DEBUG(LEVEL) if ((LEVEL) <= DebugLevel) typedef struct _file_state file_state; @@ -61,7 +62,7 @@ struct _file_state static hash_table *hash = NULL; static pid_t mypid; static char *shmbuf; -static int DebugLevel = 0; +static int DebugLevel = 1; static int do_open(diomsg * r, int len, const char *buf) @@ -249,6 +250,7 @@ msg_handle(diomsg * r, int rl, diomsg * s) char *buf = NULL; s->mtype = r->mtype; s->callback_data = r->callback_data; + s->requestor = r->requestor; s->shm_offset = r->shm_offset; s->id = r->id; s->newstyle = r->newstyle; @@ -367,7 +369,10 @@ main(int argc, char *argv[]) for (;;) { alarm(1); memset(&rmsg, '\0', sizeof(rmsg)); - rlen = msgrcv(rmsgid, &rmsg, msg_snd_rcv_sz, 0, 0); + DEBUG(2) + fprintf(stderr, "msgrcv: %ld, %p, %u, %ld, %d \n", + rmsgid, &rmsg, diomsg::msg_snd_rcv_sz, 0, 0); + rlen = msgrcv(rmsgid, &rmsg, diomsg::msg_snd_rcv_sz, 0, 0); if (rlen < 0) { if (EINTR == errno) { @@ -392,7 +397,7 @@ main(int argc, char *argv[]) alarm(0); msg_handle(&rmsg, rlen, &smsg); - if (msgsnd(smsgid, &smsg, msg_snd_rcv_sz, 0) < 0) { + if (msgsnd(smsgid, &smsg, diomsg::msg_snd_rcv_sz, 0) < 0) { perror("msgsnd"); break; } diff --git a/src/DiskIO/DiskFile.h b/src/DiskIO/DiskFile.h new file mode 100644 index 0000000000..8c527ead18 --- /dev/null +++ b/src/DiskIO/DiskFile.h @@ -0,0 +1,69 @@ + +/* + * $Id: DiskFile.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003 Robert Collins + */ + +#ifndef SQUID_DISKFILE_H +#define SQUID_DISKFILE_H + +#include "squid.h" + +#include "RefCount.h" + +class IORequestor; + +class ReadRequest; + +class WriteRequest; + +class DiskFile : public RefCountable +{ + +public: + typedef RefCount Pointer; + virtual void open (int, mode_t, RefCount) = 0; + virtual void create (int, mode_t, RefCount) = 0; + virtual void read(ReadRequest *) = 0; + virtual void write(WriteRequest *) = 0; + virtual void close () = 0; + virtual bool canRead() const = 0; + virtual bool canWrite() const {return true;} + + /* During miogration only */ + virtual int getFD() const {return -1;} + + virtual bool error() const = 0; + + /* Inform callers if there is IO in progress */ + virtual bool ioInProgress() const = 0; +}; + +#endif /* SQUID_DISKFILE_H */ diff --git a/src/DiskIO/DiskIOModule.cc b/src/DiskIO/DiskIOModule.cc new file mode 100644 index 0000000000..2a4190f871 --- /dev/null +++ b/src/DiskIO/DiskIOModule.cc @@ -0,0 +1,109 @@ + +/* + * $Id: DiskIOModule.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 92 Storage File System + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "squid.h" +#include "DiskIOModule.h" + +Vector *DiskIOModule::_Modules = NULL; + +//DiskIOModule() : initialised (false) {} +DiskIOModule::DiskIOModule() +{ + /* We cannot call + * ModuleAdd(*this); + * here as the virtual methods are not yet available + */ +} + +void +DiskIOModule::SetupAllModules() +{ + for (iterator i = GetModules().begin(); i != GetModules().end(); ++i) + /* Call the FS to set up capabilities and initialize the FS driver */ + (*i)->init(); +} + +void +DiskIOModule::ModuleAdd(DiskIOModule &instance) +{ + iterator i = GetModules().begin(); + + while (i != GetModules().end()) { + assert(strcmp((*i)->type(), instance.type()) != 0); + ++i; + } + + GetModules().push_back (&instance); +} + +Vector const & +DiskIOModule::Modules() +{ + return GetModules(); +} + +Vector & +DiskIOModule::GetModules() +{ + if (!_Modules) + _Modules = new Vector; + + return *_Modules; +} + +/* + * called when a graceful shutdown is to occur + * of each fs module. + */ +void +DiskIOModule::FreeAllModules() +{ + while (GetModules().size()) { + DiskIOModule *fs = GetModules().back(); + GetModules().pop_back(); + fs->shutdown(); + } +} + +DiskIOModule * +DiskIOModule::Find(char const *type) +{ + for (iterator i = GetModules().begin(); i != GetModules().end(); ++i) + if (strcasecmp(type, (*i)->type()) == 0) + return *i; + + return NULL; +} diff --git a/src/DiskIO/DiskIOModule.h b/src/DiskIO/DiskIOModule.h new file mode 100644 index 0000000000..1b7e9f6fa8 --- /dev/null +++ b/src/DiskIO/DiskIOModule.h @@ -0,0 +1,74 @@ + +/* + * $Id: DiskIOModule.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_DISKIOMODULE_H +#define SQUID_DISKIOMODULE_H + +#include "squid.h" +#include "Array.h" + +class DiskIOStrategy; + +class DiskIOModule +{ + +public: + static void SetupAllModules(); + static void ModuleAdd(DiskIOModule &); + static void FreeAllModules(); + static DiskIOModule *Find(char const *type); + static Vector const &Modules(); + typedef Vector::iterator iterator; + typedef Vector::const_iterator const_iterator; + DiskIOModule(); + virtual ~DiskIOModule(){} + + virtual void init() = 0; + virtual void shutdown() = 0; + virtual DiskIOStrategy *createStrategy() = 0; + + virtual char const *type () const = 0; + // Not implemented + DiskIOModule(DiskIOModule const &); + DiskIOModule &operator=(DiskIOModule const&); + +protected: + //bool initialised; + +private: + static Vector &GetModules(); + static Vector *_Modules; +}; + + +#endif /* SQUID_DISKIOMODULE_H */ diff --git a/src/DiskIO/DiskIOStrategy.h b/src/DiskIO/DiskIOStrategy.h new file mode 100644 index 0000000000..8961a4b844 --- /dev/null +++ b/src/DiskIO/DiskIOStrategy.h @@ -0,0 +1,107 @@ + +/* + * $Id: DiskIOStrategy.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003 Robert Collins + */ + +#ifndef SQUID_DISKIOSTRATEGY_H +#define SQUID_DISKIOSTRATEGY_H + +#include "squid.h" + +#include "RefCount.h" + +class DiskFile; + +class ConfigOption; + +class DiskIOStrategy +{ + +public: + virtual ~DiskIOStrategy(){} + + /* Can the IO Strategy handle more requests ? */ + virtual bool shedLoad() = 0; + /* What is the current load? 999 = 99.9% */ + virtual int load() = 0; + /* Return a handle for performing IO operations */ + virtual RefCount newFile (char const *path) = 0; + /* flush all IO operations */ + virtual void sync() {} + + /* unlink a file by path */ + virtual void unlinkFile (char const *) = 0; + + /* perform any pending callbacks */ + virtual int callback() { return 0; } + + /* Init per-instance logic */ + virtual void init() {} + + /* cachemgr output on the IO instance stats */ + virtual void statfs(StoreEntry & sentry)const {} + + /* module specific options */ + virtual ConfigOption *getOptionTree() const { return NULL;} +}; + +/* Because we need the DiskFile definition for newFile. */ +#include "DiskFile.h" + +class SingletonIOStrategy : public DiskIOStrategy +{ + +public: + SingletonIOStrategy(DiskIOStrategy *anIO) : io(anIO){} + + virtual bool shedLoad() { return io->shedLoad(); } + + virtual int load() { return io->load(); } + + virtual RefCount newFile (char const *path) {return io->newFile(path); } + + virtual void sync() { io->sync(); } + + virtual void unlinkFile (char const *path) { io->unlinkFile(path); } + + virtual int callback() { return io->callback(); } + + virtual void init() { io->init(); } + + virtual void statfs(StoreEntry & sentry)const { io->statfs(sentry); } + + virtual ConfigOption *getOptionTree() const { return io->getOptionTree(); } + +private: + DiskIOStrategy *io; +}; + +#endif /* SQUID_DISKIOSTRATEGY_H */ diff --git a/src/fs/aufs/store_asyncufs.h b/src/DiskIO/DiskThreads/DiskThreads.h similarity index 51% rename from src/fs/aufs/store_asyncufs.h rename to src/DiskIO/DiskThreads/DiskThreads.h index 9e7284e7db..8ec3d7baa9 100644 --- a/src/fs/aufs/store_asyncufs.h +++ b/src/DiskIO/DiskThreads/DiskThreads.h @@ -1,11 +1,11 @@ /* - * store_aufs.h + * DiskThreads.h * - * Internal declarations for the aufs routines + * Internal declarations for the DiskThreads routines */ -#ifndef __STORE_ASYNCUFS_H__ -#define __STORE_ASYNCUFS_H__ +#ifndef __DISKTHREADS_H__ +#define __DISKTHREADS_H__ #ifdef AUFS_IO_THREADS #define NUMTHREADS AUFS_IO_THREADS @@ -38,7 +38,9 @@ enum _squidaio_request_type { }; typedef enum _squidaio_request_type squidaio_request_type; -struct _squidaio_result_t +typedef void AIOCB(int fd, void *cbdata, const char *buf, int aio_return, int aio_errno); + +struct squidaio_result_t { int aio_return; int aio_errno; @@ -47,9 +49,20 @@ struct _squidaio_result_t void *data; /* Available to the caller */ }; -typedef struct _squidaio_result_t squidaio_result_t; +struct squidaio_ctrl_t +{ -typedef void AIOCB(int fd, void *cbdata, const char *buf, int aio_return, int aio_errno); + struct squidaio_ctrl_t *next; + int fd; + int operation; + AIOCB *done_handler; + void *done_handler_data; + squidaio_result_t result; + int len; + char *bufp; + FREE *free_func; + dlink_node node; +}; int squidaio_cancel(squidaio_result_t *); int squidaio_open(const char *, int, mode_t, squidaio_result_t *); @@ -81,99 +94,30 @@ void aioUnlink(const char *, AIOCB *, void *); void aioTruncate(const char *, off_t length, AIOCB *, void *); int aioQueueSize(void); -#include "ufscommon.h" +#include "DiskIO/DiskFile.h" -class AufsIO; +class DiskThreadsIOStrategy; -class AUFSFile : public DiskFile +struct AIOCounts { - -public: - void * operator new (size_t); - void operator delete (void *); - AUFSFile (char const *path, AufsIO *); - ~AUFSFile(); - virtual void open (int, mode_t, IORequestor::Pointer); - virtual void create (int, mode_t, IORequestor::Pointer); - virtual void read(char *, off_t, size_t); - virtual void write(char const *buf, size_t size, off_t offset, FREE *free_func); - virtual void close (); - virtual bool error() const; - virtual int getFD() const { return fd;} - - virtual bool canRead() const; - virtual bool canWrite() const; - virtual bool ioInProgress()const; - -private: -#if ASYNC_READ - - static AIOCB ReadDone; -#else - - static DRCB ReadDone; -#endif -#if ASYNC_WRITE - - static AIOCB WriteDone; -#else - - static DWCB WriteDone; -#endif - - int fd; - bool errorOccured; - char const *path_; - AufsIO* IO; - size_t inProgressIOs; - static AIOCB OpenDone; - void openDone(int fd, const char *buf, int aio_return, int aio_errno); - IORequestor::Pointer ioRequestor; - CBDATA_CLASS(AUFSFile); - void doClose(); - - void readDone(int fd, const char *buf, int len, int errflag); - void writeDone (int fd, int errflag, size_t len); -}; - -/* - * Store IO stuff - */ - -class SwapDir; - -#include "fs/ufs/IOModule.h" - -class AufsIOModule : public IOModule -{ - -public: - static AufsIOModule &GetInstance(); - virtual void init(); - virtual void shutdown(); - virtual UFSStrategy *createSwapDirIOStrategy(); - -private: - static AufsIOModule *Instance; + int open_start; + int open_finish; + int close_start; + int close_finish; + int cancel; + int write_start; + int write_finish; + int read_start; + int read_finish; + int stat_start; + int stat_finish; + int unlink_start; + int unlink_finish; + int check_callback; }; -class AufsIO : public UFSStrategy -{ - -public: - AufsIO(); - virtual bool shedLoad(); - virtual int load(); - virtual StoreIOState::Pointer createState(SwapDir *SD, StoreEntry *e, STIOCB * callback, void *callback_data) const; - virtual DiskFile::Pointer newFile(char const *path); - virtual void unlinkFile (char const *); - virtual void sync(); - virtual int callback(); - void init(); - void done(); - static AufsIO Instance; - bool initialised; -}; +extern AIOCounts squidaio_counts; +extern dlink_list used_list; #endif diff --git a/src/DiskIO/DiskThreads/DiskThreadsDiskFile.cc b/src/DiskIO/DiskThreads/DiskThreadsDiskFile.cc new file mode 100644 index 0000000000..fe86b4176a --- /dev/null +++ b/src/DiskIO/DiskThreads/DiskThreadsDiskFile.cc @@ -0,0 +1,374 @@ + +/* + * $Id: DiskThreadsDiskFile.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 79 Disk IO Routines + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + + +#include "squid.h" +#include "DiskThreadsDiskFile.h" +#include "Store.h" +#include "SwapDir.h" +#include "Generic.h" +#include "DiskIO/IORequestor.h" +#include "DiskIO/ReadRequest.h" +#include "DiskIO/WriteRequest.h" + +/* === PUBLIC =========================================================== */ + +CBDATA_CLASS_INIT(DiskThreadsDiskFile); +void * +DiskThreadsDiskFile::operator new (size_t) +{ + CBDATA_INIT_TYPE(DiskThreadsDiskFile); + DiskThreadsDiskFile *result = cbdataAlloc(DiskThreadsDiskFile); + /* Mark result as being owned - we want the refcounter to do the delete + * call */ + return cbdataReference(result); +} + +void +DiskThreadsDiskFile::operator delete (void *address) +{ + DiskThreadsDiskFile *t = static_cast(address); + cbdataFree(address); + /* And allow the memory to be freed */ + cbdataReferenceDone (t); +} + +DiskThreadsDiskFile::DiskThreadsDiskFile (char const *aPath, DiskThreadsIOStrategy *anIO):fd(-1), errorOccured (false), IO(anIO), + inProgressIOs (0) +{ + assert (aPath); + debug (79,3)("UFSFile::UFSFile: %s\n", aPath); + path_ = xstrdup (aPath); +} + +DiskThreadsDiskFile::~DiskThreadsDiskFile() +{ + safe_free (path_); + doClose(); +} + +void +DiskThreadsDiskFile::open (int flags, mode_t mode, IORequestor::Pointer callback) +{ + statCounter.syscalls.disk.opens++; +#if !ASYNC_OPEN + + fd = file_open(path_, flags); + + if (fd < 0) { + debug(79, 3) ("DiskThreadsDiskFile::open: got failure (%d)\n", errno); + errorOccured = true; + return; + } + +#endif + Opening_FD++; + + ioRequestor = callback; + + ++inProgressIOs; + +#if ASYNC_OPEN + + aioOpen(path_, flags, mode, DiskThreadsDiskFile::OpenDone, this); + +#else + + openDone(fd, NULL, fd, 0); + +#endif +} + +void +DiskThreadsDiskFile::read(ReadRequest * request) +{ + debugs(79, 3, "DiskThreadsDiskFile::read: " << this << ", size " << request->len); + assert (fd > -1); + assert (ioRequestor.getRaw()); + statCounter.syscalls.disk.reads++; + ++inProgressIOs; +#if ASYNC_READ + + aioRead(fd, request->offset, request->len, ReadDone, new IoResult(this, request)); +#else + + file_read(fd, request->buf, request->len, request->offset, ReadDone, new IoResult(this, request)); +#endif +} + +void +DiskThreadsDiskFile::create (int flags, mode_t mode, IORequestor::Pointer callback) +{ + statCounter.syscalls.disk.opens++; +#if !ASYNC_CREATE + + int fd = file_open(path_, flags); + + if (fd < 0) { + debug(79, 3) ("storeAufsCreate: got failure (%d)\n", errno); + errorOccured = true; + return; + } + +#endif + Opening_FD++; + + ioRequestor = callback; + + ++inProgressIOs; + +#if ASYNC_CREATE + + aioOpen(path_, flags, mode, DiskThreadsDiskFile::OpenDone, this); + +#else + + openDone (fd, NULL, fd, 0); + +#endif +} + +bool +DiskThreadsDiskFile::error() const +{ + return errorOccured; +} + +void +DiskThreadsDiskFile::OpenDone(int fd, void *cbdata, const char *buf, int aio_return, int aio_errno) +{ + DiskThreadsDiskFile *myFile = static_cast(cbdata); + myFile->openDone (fd, buf, aio_return, aio_errno); +} + +void +DiskThreadsDiskFile::openDone(int unused, const char *unused2, int anFD, int errflag) +{ + debug(79, 3) ("DiskThreadsDiskFile::openDone: FD %d, errflag %d\n", anFD, errflag); + Opening_FD--; + + fd = anFD; + + if (errflag || fd < 0) { + errno = errflag; + debug(79, 0) ("DiskThreadsDiskFile::openDone: %s\n", xstrerror()); + debug(79, 1) ("\t%s\n", path_); + errorOccured = true; + } else { + store_open_disk_fd++; + commSetCloseOnExec(fd); + fd_open(fd, FD_FILE, path_); + } + + debug(79, 3) ("DiskThreadsDiskFile::openDone: exiting\n"); + + IORequestor::Pointer t = ioRequestor; + --inProgressIOs; + t->ioCompletedNotification(); +} + +void DiskThreadsDiskFile::doClose() +{ + if (fd > -1) { + statCounter.syscalls.disk.closes++; + aioClose(fd); + fd_close(fd); + store_open_disk_fd--; + fd = -1; + } +} + +void +DiskThreadsDiskFile::close () +{ + debug (79,3)("DiskThreadsDiskFile::close: %p closing for %p\n", this, ioRequestor.getRaw()); + + if (!ioInProgress()) { + doClose(); + assert (ioRequestor.getRaw()); + ioRequestor->closeCompleted(); + } +} + +bool +DiskThreadsDiskFile::canRead() const +{ + debug (79,3)("DiskThreadsDiskFile::canRead: fd is %d\n",fd); + return fd > -1; +} + +void +DiskThreadsDiskFile::write(WriteRequest * writeRequest) +{ + debug(79, 3) ("storeAufsWrite: FD %d\n", fd); + statCounter.syscalls.disk.writes++; + ++inProgressIOs; +#if ASYNC_WRITE + + aioWrite(fd, writeRequest->offset, (char *)writeRequest->buf, writeRequest->len, WriteDone, new IoResult(this, writeRequest), + writeRequest->free_func); +#else + + file_write(fd, writeRequest->offset, (char *)writeRequest->buf, writeRequest->len, WriteDone, new IoResult(this, writeRequest), + writeRequest->free_func); +#endif +} + +bool +DiskThreadsDiskFile::canWrite() const +{ + return fd > -1; +} + +bool +DiskThreadsDiskFile::ioInProgress()const +{ + return inProgressIOs > 0; +} + +/* === STATIC =========================================================== */ + +#if ASYNC_READ +void +DiskThreadsDiskFile::ReadDone(int fd, void *my_data, const char *buf, int len, int errflag) +#else +void +DiskThreadsDiskFile::ReadDone(int fd, const char *buf, int len, int errflag, void *my_data) +#endif +{ + IoResult * result = static_cast *>(my_data); + assert (result); + result->file->readDone(fd, buf, len, errflag, result->request); + delete result; +} + +void +DiskThreadsDiskFile::readDone(int rvfd, const char *buf, int len, int errflag, ReadRequest::Pointer request) +{ + debug (79,3)("DiskThreadsDiskFile::readDone: FD %d\n",rvfd); + assert (fd == rvfd); + + ssize_t rlen; + + if (errflag) { + debug(79, 3) ("DiskThreadsDiskFile::readDone: got failure (%d)\n", errflag); + rlen = -1; + } else { + rlen = (ssize_t) len; + } + +#if ASYNC_READ + /* translate errflag from errno to Squid disk error */ + errno = errflag; + + if (errflag) + errflag = DISK_ERROR; + else + errflag = DISK_OK; + +#else + + if (errflag == DISK_EOF) + errflag = DISK_OK; /* EOF is signalled by len == 0, not errors... */ + +#endif + + --inProgressIOs; + + ioRequestor->readCompleted(buf, rlen, errflag, request); +} + +void +DiskThreadsDiskFile:: +#if ASYNC_WRITE +WriteDone(int fd, void *my_data, int len, int errflag) +#else +WriteDone(int fd, int errflag, size_t len, void *my_data) +#endif +{ + IoResult * result = static_cast *>(my_data); + assert (result); + result->file->writeDone(fd, errflag, len, result->request); + delete result; +} + +void +DiskThreadsDiskFile::writeDone (int rvfd, int errflag, size_t len, WriteRequest::Pointer request) +{ + assert (rvfd == fd); + static int loop_detect = 0; + debug(79, 3) ("storeAufsWriteDone: FD %d, len %ld, err=%d\n", + fd, (long int) len, errflag); + +#if ASYNC_WRITE + /* Translate from errno to Squid disk error */ + errno = errflag; + + if (errflag) + errflag = errno == ENOSPC ? DISK_NO_SPACE_LEFT : DISK_ERROR; + else + errflag = DISK_OK; + +#endif + + assert(++loop_detect < 10); + + --inProgressIOs; + + ioRequestor->writeCompleted(errflag, len, request); + + --loop_detect; +} + +template +cbdata_type IoResult::CBDATA_IoResult = CBDATA_UNKNOWN; + +template +void * +IoResult::operator new (size_t) +{ + CBDATA_INIT_TYPE(IoResult); + IoResult *result = cbdataAlloc(IoResult); + return result; +} + +template +void +IoResult::operator delete (void *address) +{ + cbdataFree(address); +} + diff --git a/src/DiskIO/DiskThreads/DiskThreadsDiskFile.h b/src/DiskIO/DiskThreads/DiskThreadsDiskFile.h new file mode 100644 index 0000000000..23ebb2d0a2 --- /dev/null +++ b/src/DiskIO/DiskThreads/DiskThreadsDiskFile.h @@ -0,0 +1,116 @@ + +/* + * $Id: DiskThreadsDiskFile.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 79 Disk IO Routines + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_DISKTHREADSDISKFILE_H +#define SQUID_DISKTHREADSDISKFILE_H +#include "DiskIO/DiskFile.h" +#include "DiskThreads.h" + +class DiskThreadsDiskFile : public DiskFile +{ + +public: + void * operator new (size_t); + void operator delete (void *); + DiskThreadsDiskFile (char const *path, DiskThreadsIOStrategy *); + ~DiskThreadsDiskFile(); + virtual void open (int, mode_t, RefCount); + virtual void create (int, mode_t, RefCount); + virtual void read(ReadRequest *); + virtual void write(WriteRequest *); + virtual void close (); + virtual bool error() const; + virtual int getFD() const { return fd;} + + virtual bool canRead() const; + virtual bool canWrite() const; + virtual bool ioInProgress()const; + +private: +#if ASYNC_READ + + static AIOCB ReadDone; +#else + + static DRCB ReadDone; +#endif +#if ASYNC_WRITE + + static AIOCB WriteDone; +#else + + static DWCB WriteDone; +#endif + + int fd; + bool errorOccured; + char const *path_; + DiskThreadsIOStrategy *IO; + size_t inProgressIOs; + static AIOCB OpenDone; + void openDone(int fd, const char *buf, int aio_return, int aio_errno); + RefCount ioRequestor; + CBDATA_CLASS(DiskThreadsDiskFile); + void doClose(); + + void readDone(int fd, const char *buf, int len, int errflag, RefCount); + void writeDone (int fd, int errflag, size_t len, RefCount); +}; + +#include "DiskIO/ReadRequest.h" + +template + +class IoResult +{ + +public: + void * operator new (size_t); + void operator delete (void *); + IoResult(RefCount aFile, RefCount aRequest) : file(aFile), request(aRequest){} + + RefCount file; + RefCount request; + +private: + CBDATA_CLASS(IoResult); +}; + +template +IoResult +IOResult(RefCount aRequest, RefCount aFile) { return IoResult(aFile, aRequest);} + +#endif /* SQUID_DISKTHREADSDISKFILE_H */ diff --git a/src/DiskIO/DiskThreads/DiskThreadsDiskIOModule.cc b/src/DiskIO/DiskThreads/DiskThreadsDiskIOModule.cc new file mode 100644 index 0000000000..564018a9b9 --- /dev/null +++ b/src/DiskIO/DiskThreads/DiskThreadsDiskIOModule.cc @@ -0,0 +1,72 @@ + +/* + * $Id: DiskThreadsDiskIOModule.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "squid.h" +#include "DiskThreadsDiskIOModule.h" +#include "DiskThreadsIOStrategy.h" + +DiskThreadsDiskIOModule DiskThreadsDiskIOModule::Instance; +DiskThreadsDiskIOModule & +DiskThreadsDiskIOModule::GetInstance() +{ + return Instance; +} + +DiskThreadsDiskIOModule::DiskThreadsDiskIOModule() +{ + ModuleAdd(*this); +} + +void +DiskThreadsDiskIOModule::init() +{ + DiskThreadsIOStrategy::Instance.init(); +} + +void +DiskThreadsDiskIOModule::shutdown() +{ + DiskThreadsIOStrategy::Instance.done(); +} + +DiskIOStrategy * +DiskThreadsDiskIOModule::createStrategy() +{ + return new SingletonIOStrategy(&DiskThreadsIOStrategy::Instance); +} + +char const * +DiskThreadsDiskIOModule::type () const +{ + return "DiskThreads"; +} diff --git a/src/DiskIO/DiskThreads/DiskThreadsDiskIOModule.h b/src/DiskIO/DiskThreads/DiskThreadsDiskIOModule.h new file mode 100644 index 0000000000..a5fe10ca34 --- /dev/null +++ b/src/DiskIO/DiskThreads/DiskThreadsDiskIOModule.h @@ -0,0 +1,55 @@ + +/* + * $Id: DiskThreadsDiskIOModule.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef SQUID_DISKTHREADSDISKIOMODULE_H +#define SQUID_DISKTHREADSDISKIOMODULE_H + +#include "DiskIO/DiskIOModule.h" + +class DiskThreadsDiskIOModule : public DiskIOModule +{ + +public: + static DiskThreadsDiskIOModule &GetInstance(); + DiskThreadsDiskIOModule(); + virtual void init(); + virtual void shutdown(); + virtual char const *type () const; + virtual DiskIOStrategy* createStrategy(); + +private: + static DiskThreadsDiskIOModule Instance; +}; + + +#endif /* SQUID_DISKTHREADSDISKIOMODULE_H */ diff --git a/src/DiskIO/DiskThreads/DiskThreadsIOStrategy.cc b/src/DiskIO/DiskThreads/DiskThreadsIOStrategy.cc new file mode 100644 index 0000000000..5a4f72c000 --- /dev/null +++ b/src/DiskIO/DiskThreads/DiskThreadsIOStrategy.cc @@ -0,0 +1,265 @@ + +/* + * $Id: DiskThreadsIOStrategy.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 79 Squid-side Disk I/O functions. + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#include "squid.h" + +#include "DiskThreadsIOStrategy.h" +#include "DiskThreadsDiskFile.h" +/* for statfs */ +#include "Store.h" +#include "fde.h" + +//static MemAllocatorProxy *squidaio_ctrl_pool; + +void +DiskThreadsIOStrategy::init(void) +{ + if (initialised) + return; + + squidaio_ctrl_pool = new MemAllocatorProxy("aio_ctrl", sizeof(squidaio_ctrl_t)); + + cachemgrRegister("squidaio_counts", "Async IO Function Counters", + aioStats, 0, 1); + + initialised = true; +} + +void +DiskThreadsIOStrategy::done(void) +{ + if (!initialised) + return; + + delete squidaio_ctrl_pool; + + squidaio_ctrl_pool = NULL; + + initialised = false; +} + +int +DiskThreadsIOStrategy::callback() +{ + squidaio_result_t *resultp; + squidaio_ctrl_t *ctrlp; + int retval = 0; + + assert(initialised); + squidaio_counts.check_callback++; + + for (;;) { + if ((resultp = squidaio_poll_done()) == NULL) + break; + + ctrlp = (squidaio_ctrl_t *) resultp->data; + + switch (resultp->result_type) { + + case _AIO_OP_NONE: + + case _AIO_OP_TRUNCATE: + + case _AIO_OP_OPENDIR: + break; + + case _AIO_OP_OPEN: + ++squidaio_counts.open_finish; + break; + + case _AIO_OP_READ: + ++squidaio_counts.read_finish; + break; + + case _AIO_OP_WRITE: + ++squidaio_counts.write_finish; + break; + + case _AIO_OP_CLOSE: + ++squidaio_counts.close_finish; + break; + + case _AIO_OP_UNLINK: + ++squidaio_counts.unlink_finish; + break; + + case _AIO_OP_STAT: + ++squidaio_counts.stat_finish; + break; + } + + if (ctrlp == NULL) + continue; /* XXX Should not happen */ + + dlinkDelete(&ctrlp->node, &used_list); + + if (ctrlp->done_handler) { + AIOCB *callback = ctrlp->done_handler; + void *cbdata; + ctrlp->done_handler = NULL; + + if (cbdataReferenceValidDone(ctrlp->done_handler_data, &cbdata)) { + retval = 1; /* Return that we've actually done some work */ + callback(ctrlp->fd, cbdata, ctrlp->bufp, + ctrlp->result.aio_return, ctrlp->result.aio_errno); + } else { + if (ctrlp->operation == _AIO_OPEN) { + /* The open operation was aborted.. */ + int fd = ctrlp->result.aio_return; + + if (fd >= 0) + aioClose(fd); + } + } + } + + /* free data if requested to aioWrite() */ + if (ctrlp->free_func) + ctrlp->free_func(ctrlp->bufp); + + /* free temporary read buffer */ + if (ctrlp->operation == _AIO_READ) + squidaio_xfree(ctrlp->bufp, ctrlp->len); + + if (ctrlp->operation == _AIO_CLOSE) + aioFDWasClosed(ctrlp->fd); + + squidaio_ctrl_pool->free(ctrlp); + } + + return retval; +} + +/* Flush all pending I/O */ +void +DiskThreadsIOStrategy::sync() +{ + if (!initialised) + return; /* nothing to do then */ + + /* Flush all pending operations */ + debug(32, 1) ("aioSync: flushing pending I/O operations\n"); + + do { + callback(); + } while (squidaio_sync()); + + debug(32, 1) ("aioSync: done\n"); +} + +DiskThreadsIOStrategy::DiskThreadsIOStrategy() : initialised (false) {} + +void +DiskThreadsIOStrategy::aioStats(StoreEntry * sentry) +{ + storeAppendPrintf(sentry, "ASYNC IO Counters:\n"); + storeAppendPrintf(sentry, "Operation\t# Requests\tNumber serviced\n"); + storeAppendPrintf(sentry, "open\t%d\t%d\n", squidaio_counts.open_start, squidaio_counts.open_finish); + storeAppendPrintf(sentry, "close\t%d\t%d\n", squidaio_counts.close_start, squidaio_counts.close_finish); + storeAppendPrintf(sentry, "cancel\t%d\t-\n", squidaio_counts.cancel); + storeAppendPrintf(sentry, "write\t%d\t%d\n", squidaio_counts.write_start, squidaio_counts.write_finish); + storeAppendPrintf(sentry, "read\t%d\t%d\n", squidaio_counts.read_start, squidaio_counts.read_finish); + storeAppendPrintf(sentry, "stat\t%d\t%d\n", squidaio_counts.stat_start, squidaio_counts.stat_finish); + storeAppendPrintf(sentry, "unlink\t%d\t%d\n", squidaio_counts.unlink_start, squidaio_counts.unlink_finish); + storeAppendPrintf(sentry, "check_callback\t%d\t-\n", squidaio_counts.check_callback); + storeAppendPrintf(sentry, "queue\t%d\t-\n", squidaio_get_queue_len()); +} + +void +DiskThreadsIOStrategy::aioFDWasClosed(int fd) +{ + if (fd_table[fd].flags.closing) + fd_close(fd); +} + +DiskThreadsIOStrategy DiskThreadsIOStrategy::Instance; +bool +DiskThreadsIOStrategy::shedLoad() +{ + /* + * we should detect some 'too many files open' condition and return + * NULL here. + */ +#ifdef MAGIC2 + + if (aioQueueSize() > MAGIC2) + return true; + +#endif + + return false; +} + +int +DiskThreadsIOStrategy::load() +{ + int loadav; + int ql; + + ql = aioQueueSize(); + + if (ql == 0) + loadav = 0; + + loadav = ql * 1000 / MAGIC1; + + debug(47, 9) ("storeAufsDirCheckObj: load=%d\n", loadav); + + return loadav; +} + +DiskFile::Pointer +DiskThreadsIOStrategy::newFile (char const *path) +{ + if (shedLoad()) { + return NULL; + } + + return new DiskThreadsDiskFile (path, this); +} + +void +DiskThreadsIOStrategy::unlinkFile(char const *path) +{ + statCounter.syscalls.disk.unlinks++; +#if USE_TRUNCATE_NOT_UNLINK + + aioTruncate(path, NULL, NULL); +#else + + aioUnlink(path, NULL, NULL); +#endif +} diff --git a/src/DiskIO/DiskThreads/DiskThreadsIOStrategy.h b/src/DiskIO/DiskThreads/DiskThreadsIOStrategy.h new file mode 100644 index 0000000000..c5201331a0 --- /dev/null +++ b/src/DiskIO/DiskThreads/DiskThreadsIOStrategy.h @@ -0,0 +1,73 @@ + +/* + * $Id: DiskThreadsIOStrategy.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * DEBUG: section 79 Squid-side Disk I/O functions. + * AUTHOR: Robert Collins + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003, Robert Collins + */ + +#ifndef __STORE_DISKTHREADEDIOSTRATEGY_H__ +#define __STORE_DISKTHREADEDIOSTRATEGY_H__ + +#define _AIO_OPEN 0 +#define _AIO_READ 1 +#define _AIO_WRITE 2 +#define _AIO_CLOSE 3 +#define _AIO_UNLINK 4 +#define _AIO_TRUNCATE 4 +#define _AIO_OPENDIR 5 +#define _AIO_STAT 6 +#include "DiskIO/DiskIOStrategy.h" + +class DiskThreadsIOStrategy : public DiskIOStrategy +{ + +public: + DiskThreadsIOStrategy(); + virtual bool shedLoad(); + virtual int load(); + virtual RefCount newFile(char const *path); + virtual void unlinkFile (char const *); + virtual int callback(); + virtual void sync(); + virtual void init(); + void done(); + /* Todo: add access limitations */ + bool initialised; + static DiskThreadsIOStrategy Instance; + MemAllocatorProxy *squidaio_ctrl_pool; + +private: + static void aioStats(StoreEntry * sentry); + void aioFDWasClosed(int fd); +}; + +#endif diff --git a/src/fs/aufs/aiops.cc b/src/DiskIO/DiskThreads/aiops.cc similarity index 99% rename from src/fs/aufs/aiops.cc rename to src/DiskIO/DiskThreads/aiops.cc index 618d540ecb..40ab532b11 100644 --- a/src/fs/aufs/aiops.cc +++ b/src/DiskIO/DiskThreads/aiops.cc @@ -1,5 +1,5 @@ /* - * $Id: aiops.cc,v 1.25 2004/08/30 05:12:32 robertc Exp $ + * $Id: aiops.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ * * DEBUG: section 43 AIOPS * AUTHOR: Stewart Forster @@ -37,7 +37,7 @@ #endif #include "squid.h" -#include "store_asyncufs.h" +#include "DiskThreads.h" #include #include diff --git a/src/fs/aufs/async_io.cc b/src/DiskIO/DiskThreads/async_io.cc similarity index 50% rename from src/fs/aufs/async_io.cc rename to src/DiskIO/DiskThreads/async_io.cc index bdc62789b3..da40760ef3 100644 --- a/src/fs/aufs/async_io.cc +++ b/src/DiskIO/DiskThreads/async_io.cc @@ -1,6 +1,6 @@ /* - * $Id: async_io.cc,v 1.25 2004/08/30 05:12:32 robertc Exp $ + * $Id: async_io.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ * * DEBUG: section 32 Asynchronous Disk I/O * AUTHOR: Pete Bentley @@ -35,55 +35,13 @@ */ #include "squid.h" -#include "store_asyncufs.h" +#include "DiskThreads.h" #include "Store.h" #include "fde.h" +#include "DiskThreadsIOStrategy.h" +#include "Generic.h" -#define _AIO_OPEN 0 -#define _AIO_READ 1 -#define _AIO_WRITE 2 -#define _AIO_CLOSE 3 -#define _AIO_UNLINK 4 -#define _AIO_TRUNCATE 4 -#define _AIO_OPENDIR 5 -#define _AIO_STAT 6 - -typedef struct squidaio_ctrl_t -{ - - struct squidaio_ctrl_t *next; - int fd; - int operation; - AIOCB *done_handler; - void *done_handler_data; - squidaio_result_t result; - int len; - char *bufp; - FREE *free_func; - dlink_node node; -} - -squidaio_ctrl_t; - -static struct -{ - int open_start; - int open_finish; - int close_start; - int close_finish; - int cancel; - int write_start; - int write_finish; - int read_start; - int read_finish; - int stat_start; - int stat_finish; - int unlink_start; - int unlink_finish; - int check_callback; -} - -squidaio_counts; +AIOCounts squidaio_counts; typedef struct squidaio_unlinkq_t { @@ -94,53 +52,16 @@ typedef struct squidaio_unlinkq_t squidaio_unlinkq_t; -static dlink_list used_list; -static OBJH aioStats; -static MemAllocatorProxy *squidaio_ctrl_pool; -static void aioFDWasClosed(int fd); - -static void -aioFDWasClosed(int fd) -{ - if (fd_table[fd].flags.closing) - fd_close(fd); -} - -void -AufsIO::init(void) -{ - if (initialised) - return; - - squidaio_ctrl_pool = new MemAllocatorProxy("aio_ctrl", sizeof(squidaio_ctrl_t)); - - cachemgrRegister("squidaio_counts", "Async IO Function Counters", - aioStats, 0, 1); - - initialised = true; -} - -void -AufsIO::done(void) -{ - if (!initialised) - return; - - delete squidaio_ctrl_pool; - - squidaio_ctrl_pool = NULL; - - initialised = false; -} +dlink_list used_list; void aioOpen(const char *path, int oflag, mode_t mode, AIOCB * callback, void *callback_data) { squidaio_ctrl_t *ctrlp; - assert(AufsIO::Instance.initialised); + assert(DiskThreadsIOStrategy::Instance.initialised); squidaio_counts.open_start++; - ctrlp = (squidaio_ctrl_t *)squidaio_ctrl_pool->alloc(); + ctrlp = (squidaio_ctrl_t *)DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->alloc(); ctrlp->fd = -2; ctrlp->done_handler = callback; ctrlp->done_handler_data = cbdataReference(callback_data); @@ -156,10 +77,10 @@ aioClose(int fd) { squidaio_ctrl_t *ctrlp; - assert(AufsIO::Instance.initialised); + assert(DiskThreadsIOStrategy::Instance.initialised); squidaio_counts.close_start++; aioCancel(fd); - ctrlp = (squidaio_ctrl_t *)squidaio_ctrl_pool->alloc(); + ctrlp = (squidaio_ctrl_t *)DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->alloc(); ctrlp->fd = fd; ctrlp->done_handler = NULL; ctrlp->done_handler_data = NULL; @@ -176,7 +97,7 @@ aioCancel(int fd) squidaio_ctrl_t *ctrlp; dlink_node *m, *next; - assert(AufsIO::Instance.initialised); + assert(DiskThreadsIOStrategy::Instance.initialised); squidaio_counts.cancel++; for (m = used_list.head; m; m = next) { @@ -207,7 +128,7 @@ aioCancel(int fd) } dlinkDelete(m, &used_list); - squidaio_ctrl_pool->free(ctrlp); + DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->free(ctrlp); } } @@ -218,9 +139,9 @@ aioWrite(int fd, int offset, char *bufp, int len, AIOCB * callback, void *callba squidaio_ctrl_t *ctrlp; int seekmode; - assert(AufsIO::Instance.initialised); + assert(DiskThreadsIOStrategy::Instance.initialised); squidaio_counts.write_start++; - ctrlp = (squidaio_ctrl_t *)squidaio_ctrl_pool->alloc(); + ctrlp = (squidaio_ctrl_t *)DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->alloc(); ctrlp->fd = fd; ctrlp->done_handler = callback; ctrlp->done_handler_data = cbdataReference(callback_data); @@ -247,9 +168,9 @@ aioRead(int fd, int offset, int len, AIOCB * callback, void *callback_data) squidaio_ctrl_t *ctrlp; int seekmode; - assert(AufsIO::Instance.initialised); + assert(DiskThreadsIOStrategy::Instance.initialised); squidaio_counts.read_start++; - ctrlp = (squidaio_ctrl_t *)squidaio_ctrl_pool->alloc(); + ctrlp = (squidaio_ctrl_t *)DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->alloc(); ctrlp->fd = fd; ctrlp->done_handler = callback; ctrlp->done_handler_data = cbdataReference(callback_data); @@ -276,9 +197,9 @@ aioStat(char *path, struct stat *sb, AIOCB * callback, void *callback_data) { squidaio_ctrl_t *ctrlp; - assert(AufsIO::Instance.initialised); + assert(DiskThreadsIOStrategy::Instance.initialised); squidaio_counts.stat_start++; - ctrlp = (squidaio_ctrl_t *)squidaio_ctrl_pool->alloc(); + ctrlp = (squidaio_ctrl_t *)DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->alloc(); ctrlp->fd = -2; ctrlp->done_handler = callback; ctrlp->done_handler_data = cbdataReference(callback_data); @@ -293,9 +214,9 @@ void aioUnlink(const char *path, AIOCB * callback, void *callback_data) { squidaio_ctrl_t *ctrlp; - assert(AufsIO::Instance.initialised); + assert(DiskThreadsIOStrategy::Instance.initialised); squidaio_counts.unlink_start++; - ctrlp = (squidaio_ctrl_t *)squidaio_ctrl_pool->alloc(); + ctrlp = (squidaio_ctrl_t *)DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->alloc(); ctrlp->fd = -2; ctrlp->done_handler = callback; ctrlp->done_handler_data = cbdataReference(callback_data); @@ -309,9 +230,9 @@ void aioTruncate(const char *path, off_t length, AIOCB * callback, void *callback_data) { squidaio_ctrl_t *ctrlp; - assert(AufsIO::Instance.initialised); + assert(DiskThreadsIOStrategy::Instance.initialised); squidaio_counts.unlink_start++; - ctrlp = (squidaio_ctrl_t *)squidaio_ctrl_pool->alloc(); + ctrlp = (squidaio_ctrl_t *)DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->alloc(); ctrlp->fd = -2; ctrlp->done_handler = callback; ctrlp->done_handler_data = cbdataReference(callback_data); @@ -321,136 +242,8 @@ aioTruncate(const char *path, off_t length, AIOCB * callback, void *callback_dat dlinkAdd(ctrlp, &ctrlp->node, &used_list); } /* aioTruncate */ - -int -AufsIO::callback() -{ - squidaio_result_t *resultp; - squidaio_ctrl_t *ctrlp; - int retval = 0; - - assert(initialised); - squidaio_counts.check_callback++; - - for (;;) { - if ((resultp = squidaio_poll_done()) == NULL) - break; - - ctrlp = (squidaio_ctrl_t *) resultp->data; - - switch (resultp->result_type) { - - case _AIO_OP_NONE: - - case _AIO_OP_TRUNCATE: - - case _AIO_OP_OPENDIR: - break; - - case _AIO_OP_OPEN: - ++squidaio_counts.open_finish; - break; - - case _AIO_OP_READ: - ++squidaio_counts.read_finish; - break; - - case _AIO_OP_WRITE: - ++squidaio_counts.write_finish; - break; - - case _AIO_OP_CLOSE: - ++squidaio_counts.close_finish; - break; - - case _AIO_OP_UNLINK: - ++squidaio_counts.unlink_finish; - break; - - case _AIO_OP_STAT: - ++squidaio_counts.stat_finish; - break; - } - - if (ctrlp == NULL) - continue; /* XXX Should not happen */ - - dlinkDelete(&ctrlp->node, &used_list); - - if (ctrlp->done_handler) { - AIOCB *callback = ctrlp->done_handler; - void *cbdata; - ctrlp->done_handler = NULL; - - if (cbdataReferenceValidDone(ctrlp->done_handler_data, &cbdata)) { - retval = 1; /* Return that we've actually done some work */ - callback(ctrlp->fd, cbdata, ctrlp->bufp, - ctrlp->result.aio_return, ctrlp->result.aio_errno); - } else { - if (ctrlp->operation == _AIO_OPEN) { - /* The open operation was aborted.. */ - int fd = ctrlp->result.aio_return; - - if (fd >= 0) - aioClose(fd); - } - } - } - - /* free data if requested to aioWrite() */ - if (ctrlp->free_func) - ctrlp->free_func(ctrlp->bufp); - - /* free temporary read buffer */ - if (ctrlp->operation == _AIO_READ) - squidaio_xfree(ctrlp->bufp, ctrlp->len); - - if (ctrlp->operation == _AIO_CLOSE) - aioFDWasClosed(ctrlp->fd); - - squidaio_ctrl_pool->free(ctrlp); - } - - return retval; -} - -void -aioStats(StoreEntry * sentry) -{ - storeAppendPrintf(sentry, "ASYNC IO Counters:\n"); - storeAppendPrintf(sentry, "Operation\t# Requests\tNumber serviced\n"); - storeAppendPrintf(sentry, "open\t%d\t%d\n", squidaio_counts.open_start, squidaio_counts.open_finish); - storeAppendPrintf(sentry, "close\t%d\t%d\n", squidaio_counts.close_start, squidaio_counts.close_finish); - storeAppendPrintf(sentry, "cancel\t%d\t-\n", squidaio_counts.cancel); - storeAppendPrintf(sentry, "write\t%d\t%d\n", squidaio_counts.write_start, squidaio_counts.write_finish); - storeAppendPrintf(sentry, "read\t%d\t%d\n", squidaio_counts.read_start, squidaio_counts.read_finish); - storeAppendPrintf(sentry, "stat\t%d\t%d\n", squidaio_counts.stat_start, squidaio_counts.stat_finish); - storeAppendPrintf(sentry, "unlink\t%d\t%d\n", squidaio_counts.unlink_start, squidaio_counts.unlink_finish); - storeAppendPrintf(sentry, "check_callback\t%d\t-\n", squidaio_counts.check_callback); - storeAppendPrintf(sentry, "queue\t%d\t-\n", squidaio_get_queue_len()); -} - -/* Flush all pending I/O */ -void -AufsIO::sync() -{ - if (!initialised) - return; /* nothing to do then */ - - /* Flush all pending operations */ - debug(32, 1) ("aioSync: flushing pending I/O operations\n"); - - do { - callback(); - } while (squidaio_sync()); - - debug(32, 1) ("aioSync: done\n"); -} - -AufsIO::AufsIO() : initialised (false) {} - int aioQueueSize(void) { - return squidaio_ctrl_pool->inUseCount(); + return DiskThreadsIOStrategy::Instance.squidaio_ctrl_pool->inUseCount(); } diff --git a/src/DiskIO/IORequestor.h b/src/DiskIO/IORequestor.h new file mode 100644 index 0000000000..1f79dfa0d5 --- /dev/null +++ b/src/DiskIO/IORequestor.h @@ -0,0 +1,56 @@ + +/* + * $Id: IORequestor.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003 Robert Collins + */ + +#ifndef SQUID_IOREQUESTOR_H +#define SQUID_IOREQUESTOR_H + +#include "squid.h" + +#include "RefCount.h" + +class ReadRequest; + +class WriteRequest; + +class IORequestor : public RefCountable +{ + +public: + typedef RefCount Pointer; + virtual void ioCompletedNotification() = 0; + virtual void closeCompleted() = 0; + virtual void readCompleted(const char *buf, int len, int errflag, RefCount) = 0; + virtual void writeCompleted(int errflag, size_t len, RefCount) = 0; +}; + +#endif /* SQUID_IOREQUESTOR_H */ diff --git a/src/fs/ufs/IOModule.h b/src/DiskIO/ReadRequest.cc similarity index 76% rename from src/fs/ufs/IOModule.h rename to src/DiskIO/ReadRequest.cc index 87ccc12575..c76b24af50 100644 --- a/src/fs/ufs/IOModule.h +++ b/src/DiskIO/ReadRequest.cc @@ -1,6 +1,6 @@ /* - * $Id: IOModule.h,v 1.1 2003/07/22 15:23:14 robertc Exp $ + * $Id: ReadRequest.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ * * SQUID Web Proxy Cache http://www.squid-cache.org/ * ---------------------------------------------------------- @@ -28,26 +28,11 @@ * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. * - * Copyright (c) 2003, Robert Collins + * Copyright (c) 2003 Robert Collins */ -#ifndef SQUID_IOMODULE_H -#define SQUID_IOMODULE_H +#include "ReadRequest.h" -#include "squid.h" - -class UFSStrategy; - -class IOModule -{ - -public: - virtual ~IOModule(){} - - virtual void init() = 0; - virtual void shutdown() = 0; - virtual UFSStrategy *createSwapDirIOStrategy() = 0; -}; - - -#endif /* SQUID_IOMODULE_H */ +CBDATA_CLASS_INIT(ReadRequest); +ReadRequest::ReadRequest(char *aBuf, off_t anOffset, size_t aLen) : buf (aBuf), offset(anOffset), len(aLen) +{} diff --git a/src/DiskIO/ReadRequest.h b/src/DiskIO/ReadRequest.h new file mode 100644 index 0000000000..b0a7f8d474 --- /dev/null +++ b/src/DiskIO/ReadRequest.h @@ -0,0 +1,57 @@ + +/* + * $Id: ReadRequest.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003 Robert Collins + */ + +#ifndef SQUID_READREQUEST_H +#define SQUID_READREQUEST_H + +#include "squid.h" + +#include "RefCount.h" + +class ReadRequest : public RefCountable +{ + +public: + typedef RefCount Pointer; + ReadRequest(char *buf, off_t offset, size_t len); + virtual ~ReadRequest() {} + + char *buf; + off_t offset; + size_t len; + +private: + CBDATA_CLASS2(ReadRequest); +}; + +#endif /* SQUID_READREQUEST_H */ diff --git a/src/DiskIO/WriteRequest.cc b/src/DiskIO/WriteRequest.cc new file mode 100644 index 0000000000..e2f26595f1 --- /dev/null +++ b/src/DiskIO/WriteRequest.cc @@ -0,0 +1,38 @@ + +/* + * $Id: WriteRequest.cc,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003 Robert Collins + */ + +#include "WriteRequest.h" + +CBDATA_CLASS_INIT(WriteRequest); +WriteRequest::WriteRequest(char const *aBuf, off_t anOffset, size_t aLen, FREE *aFree) : buf (aBuf), offset(anOffset), len(aLen), free_func(aFree) +{} diff --git a/src/DiskIO/WriteRequest.h b/src/DiskIO/WriteRequest.h new file mode 100644 index 0000000000..52ef989e7d --- /dev/null +++ b/src/DiskIO/WriteRequest.h @@ -0,0 +1,58 @@ + +/* + * $Id: WriteRequest.h,v 1.1 2004/12/20 16:30:38 robertc Exp $ + * + * SQUID Web Proxy Cache http://www.squid-cache.org/ + * ---------------------------------------------------------- + * + * Squid is the result of efforts by numerous individuals from + * the Internet community; see the CONTRIBUTORS file for full + * details. Many organizations have provided support for Squid's + * development; see the SPONSORS file for full details. Squid is + * Copyrighted (C) 2001 by the Regents of the University of + * California; see the COPYRIGHT file for full details. Squid + * incorporates software developed and/or copyrighted by other + * sources; see the CREDITS file for full details. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. + * + * Copyright (c) 2003 Robert Collins + */ + +#ifndef SQUID_WRITEREQUEST_H +#define SQUID_WRITEREQUEST_H + +#include "squid.h" + +#include "RefCount.h" + +class WriteRequest : public RefCountable +{ + +public: + typedef RefCount Pointer; + WriteRequest(char const *buf, off_t offset, size_t len, FREE *); + virtual ~WriteRequest() {} + + char const *buf; + off_t offset; + size_t len; + FREE *free_func; + +private: + CBDATA_CLASS2(WriteRequest); +}; + +#endif /* SQUID_WRITEREQUEST_H */ diff --git a/src/ESI.cc b/src/ESI.cc index 3c92f5a530..7518e38091 100644 --- a/src/ESI.cc +++ b/src/ESI.cc @@ -1,6 +1,6 @@ /* - * $Id: ESI.cc,v 1.10 2004/12/20 14:52:25 robertc Exp $ + * $Id: ESI.cc,v 1.11 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 86 ESI processing * AUTHOR: Robert Collins diff --git a/src/ESIParser.cc b/src/ESIParser.cc index 96a73ae85e..a84bb15794 100644 --- a/src/ESIParser.cc +++ b/src/ESIParser.cc @@ -1,6 +1,6 @@ /* - * $Id: ESIParser.cc,v 1.3 2004/12/20 14:52:25 robertc Exp $ + * $Id: ESIParser.cc,v 1.4 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 86 ESI processing * AUTHOR: Robert Collins diff --git a/src/Generic.h b/src/Generic.h index 9dbb017ccb..6afbf31c4d 100644 --- a/src/Generic.h +++ b/src/Generic.h @@ -1,6 +1,6 @@ /* - * $Id: Generic.h,v 1.6 2003/09/22 08:50:51 robertc Exp $ + * $Id: Generic.h,v 1.7 2004/12/20 16:30:32 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ @@ -73,6 +73,38 @@ T& for_each(Stack const &collection, T& visitor) return visitor; }; +/* RBC 20030718 - use this to provide instance expecting classes a pointer to a + * singleton + */ + +template + +class InstanceToSingletonAdapter : public C +{ + +public: + void *operator new (size_t byteCount) { return ::operator new (byteCount);} + + void operator delete (void *address) { ::operator delete (address);} + + InstanceToSingletonAdapter(C const *instance) : theInstance (instance) {} + + C const * operator-> () const {return theInstance; } + + C * operator-> () {return const_cast(theInstance); } + + C const & operator * () const {return *theInstance; } + + C & operator * () {return *const_cast(theInstance); } + + operator C const * () const {return theInstance;} + + operator C *() {return const_cast(theInstance);} + +private: + C const *theInstance; +}; + template Visitor& for_each(InputIterator from, InputIterator to, Visitor& visitor) { diff --git a/src/HttpHeader.cc b/src/HttpHeader.cc index 25f0741132..464d07c55e 100644 --- a/src/HttpHeader.cc +++ b/src/HttpHeader.cc @@ -1,6 +1,6 @@ /* - * $Id: HttpHeader.cc,v 1.100 2004/12/20 14:52:25 robertc Exp $ + * $Id: HttpHeader.cc,v 1.101 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 55 HTTP Header * AUTHOR: Alex Rousskov diff --git a/src/HttpHeaderTools.cc b/src/HttpHeaderTools.cc index db95e87c12..c4a5384aa5 100644 --- a/src/HttpHeaderTools.cc +++ b/src/HttpHeaderTools.cc @@ -1,6 +1,6 @@ /* - * $Id: HttpHeaderTools.cc,v 1.47 2004/12/20 14:52:25 robertc Exp $ + * $Id: HttpHeaderTools.cc,v 1.48 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 66 HTTP Header Tools * AUTHOR: Alex Rousskov diff --git a/src/HttpReply.cc b/src/HttpReply.cc index 80d9b46603..8cce8fff76 100644 --- a/src/HttpReply.cc +++ b/src/HttpReply.cc @@ -1,6 +1,6 @@ /* - * $Id: HttpReply.cc,v 1.68 2004/12/20 14:52:25 robertc Exp $ + * $Id: HttpReply.cc,v 1.69 2004/12/20 16:30:32 robertc Exp $ * * DEBUG: section 58 HTTP Reply (Response) * AUTHOR: Alex Rousskov diff --git a/src/Makefile.am b/src/Makefile.am index 76072e1780..a38c3eb283 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1,7 +1,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.am,v 1.97 2004/12/20 14:52:26 robertc Exp $ +# $Id: Makefile.am,v 1.98 2004/12/20 16:30:32 robertc Exp $ # # Uncomment and customize the following to suit your needs: # @@ -167,14 +167,18 @@ AM_CXXFLAGS = @SQUID_CXXFLAGS@ SUBDIRS = fs repl auth -INCLUDES = -I/usr/include/libxml2 -I. -I$(srcdir) -I$(top_builddir)/include -I$(top_srcdir)/include -I$(top_srcdir)/lib/libTrie/include +EXTRA_LIBRARIES = libAIO.a libBlocking.a libDiskDaemon.a libDiskThreads.a +noinst_LIBRARIES = @DISK_LIBS@ + +INCLUDES = -I. -I$(srcdir) -I$(top_builddir)/include -I$(top_srcdir)/include -I$(top_srcdir)/lib/libTrie/include EXTRA_PROGRAMS = \ + DiskIO/DiskDaemon/diskd \ unlinkd \ pinger \ dnsserver \ recv-announce \ - ufsdump + ufsdump noinst_PROGRAMS = \ cf_gen @@ -189,6 +193,7 @@ bin_PROGRAMS = \ libexec_PROGRAMS = \ $(PINGER) \ $(DNSSERVER) \ + @DISK_PROGRAMS@ \ $(UNLINKD) \ cachemgr$(CGIEXT) @@ -205,6 +210,12 @@ all_FSMODULES = \ fs/null/StoreFSnull.cc \ fs/ufs/StoreFSufs.cc +all_DISKIOMODULES = \ + DiskIO/AIO/AIODiskIOModule.cc \ + DiskIO/Blocking/BlockingDiskIOModule.cc \ + DiskIO/DiskDaemon/DiskDaemonDiskIOModule.cc \ + DiskIO/DiskThreads/DiskThreadsDiskIOModule.cc + all_AUTHMODULES = \ auth/basic/basicScheme.cc \ auth/basic/basicScheme.h \ @@ -215,8 +226,10 @@ all_AUTHMODULES = \ EXTRA_squid_SOURCES = \ $(all_FSMODULES) \ + $(all_DISKIOMODULES) \ $(all_AUTHMODULES) \ $(ARP_ACL_ALL_SOURCE) \ + ConfigOption.h \ $(DELAY_POOL_ALL_SOURCE) \ dns.cc \ dnsserver.cc \ @@ -342,6 +355,7 @@ squid_SOURCES = \ comm_poll.cc \ comm_kqueue.cc \ CommRead.h \ + ConfigOption.cc \ ConfigParser.h \ ConnectionDetail.h \ debug.cc \ @@ -349,6 +363,9 @@ squid_SOURCES = \ defines.h \ $(DELAY_POOL_SOURCE) \ disk.cc \ + DiskIO/DiskIOModule.cc \ + DiskIO/ReadRequest.cc \ + DiskIO/WriteRequest.cc \ $(DNSSOURCE) \ $(EPOLL_SOURCE) \ enums.h \ @@ -479,8 +496,6 @@ squid_SOURCES = \ SwapDir.h \ tools.cc \ typedefs.h \ - ufscommon.cc \ - ufscommon.h \ $(UNLINKDSOURCE) \ url.cc \ urn.cc \ @@ -500,7 +515,6 @@ noinst_HEADERS = ACLChecklist.cci \ Store.cci \ String.cci \ SquidString.h \ - ufscommon.cci \ squid_windows.h nodist_squid_SOURCES = \ @@ -512,9 +526,11 @@ nodist_squid_SOURCES = \ squid_LDADD = \ -L../lib \ @XTRA_OBJS@ \ + @DISK_LINKOBJS@ \ @REPL_OBJS@ \ @STORE_LINKOBJS@ \ @STORE_OBJS@ \ + @DISK_LIBS@ \ @AUTH_LINKOBJS@ \ @AUTH_OBJS@ \ @CRYPTLIB@ \ @@ -526,6 +542,8 @@ squid_LDADD = \ @XTRA_LIBS@ \ @EPOLL_LIBS@ squid_DEPENDENCIES = $(top_builddir)/lib/libmiscutil.a @STORE_OBJS@ @STORE_LINKOBJS@ \ + @DISK_LIBS@ \ + @DISK_LINKOBJS@ \ @REPL_OBJS@ \ @AUTH_LINKOBJS@ \ @AUTH_OBJS@ @@ -677,8 +695,6 @@ ufsdump_SOURCES = debug.cc \ SwapDir.cc \ tools.cc \ typedefs.h \ - ufscommon.cc \ - ufscommon.h \ $(UNLINKDSOURCE) \ url.cc \ urn.cc \ @@ -727,6 +743,13 @@ data_DATA = \ LDADD = -L../lib -lmiscutil @XTRA_LIBS@ @EPOLL_LIBS@ +DISKIODIST = \ + DiskIO/DiskFile.h \ + DiskIO/DiskIOStrategy.h \ + DiskIO/IORequestor.h \ + DiskIO/DiskIOModule.h \ + DiskIO/ReadRequest.h + EXTRA_DIST = \ cf_gen_defines \ cf.data.pre \ @@ -734,7 +757,45 @@ EXTRA_DIST = \ mk-string-arrays.pl \ repl_modules.sh \ mib.txt \ - mime.conf.default + mime.conf.default \ + $(DISKIODIST) + +libAIO_a_SOURCES = \ + DiskIO/AIO/async_io.h \ + DiskIO/AIO/AIODiskFile.cc \ + DiskIO/AIO/AIODiskFile.h \ + DiskIO/AIO/AIODiskIOStrategy.cc \ + DiskIO/AIO/AIODiskIOStrategy.h \ + DiskIO/AIO/AIODiskIOModule.h + +libBlocking_a_SOURCES = \ + DiskIO/Blocking/BlockingFile.cc \ + DiskIO/Blocking/BlockingFile.h \ + DiskIO/Blocking/BlockingIOStrategy.cc \ + DiskIO/Blocking/BlockingIOStrategy.h \ + DiskIO/Blocking/BlockingDiskIOModule.h + +libDiskDaemon_a_SOURCES = \ + DiskIO/DiskDaemon/DiskdFile.cc \ + DiskIO/DiskDaemon/DiskdFile.h \ + DiskIO/DiskDaemon/DiskdIOStrategy.cc \ + DiskIO/DiskDaemon/DiskdIOStrategy.h \ + DiskIO/DiskDaemon/diomsg.h \ + DiskIO/DiskDaemon/DiskDaemonDiskIOModule.h + +libDiskThreads_a_SOURCES = \ + DiskIO/DiskThreads/aiops.cc \ + DiskIO/DiskThreads/async_io.cc \ + DiskIO/DiskThreads/DiskThreads.h \ + DiskIO/DiskThreads/DiskThreadsDiskFile.cc \ + DiskIO/DiskThreads/DiskThreadsDiskFile.h \ + DiskIO/DiskThreads/DiskThreadsDiskIOModule.h \ + DiskIO/DiskThreads/DiskThreadsIOStrategy.cc \ + DiskIO/DiskThreads/DiskThreadsIOStrategy.h + +DiskIO_DiskDaemon_diskd_SOURCES = DiskIO/DiskDaemon/diskd.cc +DiskIO_DiskDaemon_diskd_LDADD = $(top_builddir)/lib/libmiscutil.a @XTRA_LIBS@ + DEFAULT_PREFIX = $(prefix) DEFAULT_CONFIG_FILE = $(sysconfdir)/squid.conf diff --git a/src/Makefile.in b/src/Makefile.in index 07a3e8e214..a1e3f060c5 100644 --- a/src/Makefile.in +++ b/src/Makefile.in @@ -17,14 +17,15 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.328 2004/12/20 14:52:26 robertc Exp $ +# $Id: Makefile.in,v 1.329 2004/12/20 16:30:33 robertc Exp $ # # Uncomment and customize the following to suit your needs: # -SOURCES = $(cachemgr__CGIEXT__SOURCES) $(cf_gen_SOURCES) $(dnsserver_SOURCES) $(pinger_SOURCES) $(nodist_pinger_SOURCES) $(recv_announce_SOURCES) $(squid_SOURCES) $(EXTRA_squid_SOURCES) $(nodist_squid_SOURCES) $(squidclient_SOURCES) $(tests_testAuth_SOURCES) $(ufsdump_SOURCES) $(nodist_ufsdump_SOURCES) $(unlinkd_SOURCES) + +SOURCES = $(libAIO_a_SOURCES) $(libBlocking_a_SOURCES) $(libDiskDaemon_a_SOURCES) $(libDiskThreads_a_SOURCES) $(DiskIO_DiskDaemon_diskd_SOURCES) $(cachemgr__CGIEXT__SOURCES) $(cf_gen_SOURCES) $(dnsserver_SOURCES) $(pinger_SOURCES) $(nodist_pinger_SOURCES) $(recv_announce_SOURCES) $(squid_SOURCES) $(EXTRA_squid_SOURCES) $(nodist_squid_SOURCES) $(squidclient_SOURCES) $(tests_testAuth_SOURCES) $(ufsdump_SOURCES) $(nodist_ufsdump_SOURCES) $(unlinkd_SOURCES) srcdir = @srcdir@ top_srcdir = @top_srcdir@ @@ -49,13 +50,14 @@ POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ check_PROGRAMS = tests/testAuth$(EXEEXT) -EXTRA_PROGRAMS = unlinkd$(EXEEXT) pinger$(EXEEXT) dnsserver$(EXEEXT) \ - recv-announce$(EXEEXT) ufsdump$(EXEEXT) +EXTRA_PROGRAMS = DiskIO/DiskDaemon/diskd$(EXEEXT) unlinkd$(EXEEXT) \ + pinger$(EXEEXT) dnsserver$(EXEEXT) recv-announce$(EXEEXT) \ + ufsdump$(EXEEXT) noinst_PROGRAMS = cf_gen$(EXEEXT) sbin_PROGRAMS = squid$(EXEEXT) bin_PROGRAMS = squidclient$(EXEEXT) -libexec_PROGRAMS = $(am__EXEEXT_1) $(am__EXEEXT_2) $(am__EXEEXT_3) \ - cachemgr$(CGIEXT)$(EXEEXT) +libexec_PROGRAMS = $(am__EXEEXT_1) $(am__EXEEXT_2) @DISK_PROGRAMS@ \ + $(am__EXEEXT_3) cachemgr$(CGIEXT)$(EXEEXT) subdir = src DIST_COMMON = $(noinst_HEADERS) $(srcdir)/Makefile.am \ $(srcdir)/Makefile.in @@ -67,6 +69,31 @@ am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ mkinstalldirs = $(SHELL) $(top_srcdir)/cfgaux/mkinstalldirs CONFIG_HEADER = $(top_builddir)/include/autoconf.h CONFIG_CLEAN_FILES = +LIBRARIES = $(noinst_LIBRARIES) +ARFLAGS = cru +libAIO_a_AR = $(AR) $(ARFLAGS) +libAIO_a_LIBADD = +am__dirstamp = $(am__leading_dot)dirstamp +am_libAIO_a_OBJECTS = DiskIO/AIO/AIODiskFile.$(OBJEXT) \ + DiskIO/AIO/AIODiskIOStrategy.$(OBJEXT) +libAIO_a_OBJECTS = $(am_libAIO_a_OBJECTS) +libBlocking_a_AR = $(AR) $(ARFLAGS) +libBlocking_a_LIBADD = +am_libBlocking_a_OBJECTS = DiskIO/Blocking/BlockingFile.$(OBJEXT) \ + DiskIO/Blocking/BlockingIOStrategy.$(OBJEXT) +libBlocking_a_OBJECTS = $(am_libBlocking_a_OBJECTS) +libDiskDaemon_a_AR = $(AR) $(ARFLAGS) +libDiskDaemon_a_LIBADD = +am_libDiskDaemon_a_OBJECTS = DiskIO/DiskDaemon/DiskdFile.$(OBJEXT) \ + DiskIO/DiskDaemon/DiskdIOStrategy.$(OBJEXT) +libDiskDaemon_a_OBJECTS = $(am_libDiskDaemon_a_OBJECTS) +libDiskThreads_a_AR = $(AR) $(ARFLAGS) +libDiskThreads_a_LIBADD = +am_libDiskThreads_a_OBJECTS = DiskIO/DiskThreads/aiops.$(OBJEXT) \ + DiskIO/DiskThreads/async_io.$(OBJEXT) \ + DiskIO/DiskThreads/DiskThreadsDiskFile.$(OBJEXT) \ + DiskIO/DiskThreads/DiskThreadsIOStrategy.$(OBJEXT) +libDiskThreads_a_OBJECTS = $(am_libDiskThreads_a_OBJECTS) am__installdirs = "$(DESTDIR)$(bindir)" "$(DESTDIR)$(libexecdir)" \ "$(DESTDIR)$(sbindir)" "$(DESTDIR)$(datadir)" \ "$(DESTDIR)$(sysconfdir)" @@ -78,6 +105,12 @@ libexecPROGRAMS_INSTALL = $(INSTALL_PROGRAM) sbinPROGRAMS_INSTALL = $(INSTALL_PROGRAM) PROGRAMS = $(bin_PROGRAMS) $(libexec_PROGRAMS) $(noinst_PROGRAMS) \ $(sbin_PROGRAMS) +am_DiskIO_DiskDaemon_diskd_OBJECTS = \ + DiskIO/DiskDaemon/diskd.$(OBJEXT) +DiskIO_DiskDaemon_diskd_OBJECTS = \ + $(am_DiskIO_DiskDaemon_diskd_OBJECTS) +DiskIO_DiskDaemon_diskd_DEPENDENCIES = \ + $(top_builddir)/lib/libmiscutil.a am_cachemgr__CGIEXT__OBJECTS = cachemgr.$(OBJEXT) cachemgr__CGIEXT__OBJECTS = $(am_cachemgr__CGIEXT__OBJECTS) cachemgr__CGIEXT__LDADD = $(LDADD) @@ -128,28 +161,29 @@ am__squid_SOURCES_DIST = access_log.cc AccessLogEntry.h acl.cc ACL.h \ client_side.h client_side_reply.cc client_side_reply.h \ client_side_request.cc client_side_request.h clientStream.cc \ clientStream.h comm.cc comm.h comm_select.cc comm_poll.cc \ - comm_kqueue.cc CommRead.h ConfigParser.h ConnectionDetail.h \ - debug.cc Debug.h defines.h CommonPool.h CompositePoolNode.h \ - delay_pools.cc DelayId.cc DelayId.h DelayIdComposite.h \ - DelayBucket.cc DelayBucket.h DelayConfig.cc DelayConfig.h \ - DelayPool.cc DelayPool.h DelayPools.h DelaySpec.cc DelaySpec.h \ - DelayTagged.cc DelayTagged.h DelayUser.cc DelayUser.h \ - DelayVector.cc DelayVector.h NullDelayId.cc NullDelayId.h \ - disk.cc dns_internal.cc dns.cc comm_epoll.cc enums.h \ - errorpage.cc ElementList.h ESI.cc ESI.h ESIAssign.cc \ - ESIAssign.h ESIAttempt.h ESIContext.cc ESIContext.h \ - ESICustomParser.cc ESICustomParser.h ESIElement.h ESIExcept.h \ - ESIExpatParser.cc ESIExpatParser.h ESIExpression.cc \ - ESIExpression.h ESIInclude.cc ESIInclude.h ESILiteral.h \ - ESILibxml2Parser.cc ESILibxml2Parser.h ESIParser.cc \ - ESIParser.h ESISegment.cc ESISegment.h ESISequence.cc \ - ESISequence.h ESIVar.h ESIVarState.cc ESIVarState.h ETag.cc \ - event.cc external_acl.cc ExternalACL.h ExternalACLEntry.cc \ - ExternalACLEntry.h fd.cc fde.cc fde.h filemap.cc forward.cc \ - fqdncache.cc ftp.cc Generic.h globals.h gopher.cc helper.cc \ - helper.h HierarchyLogEntry.h htcp.cc htcp.h http.cc http.h \ - HttpStatusLine.cc HttpStatusLine.h HttpHdrCc.cc \ - HttpHdrRange.cc HttpHdrSc.cc HttpHdrScTarget.cc \ + comm_kqueue.cc CommRead.h ConfigOption.cc ConfigParser.h \ + ConnectionDetail.h debug.cc Debug.h defines.h CommonPool.h \ + CompositePoolNode.h delay_pools.cc DelayId.cc DelayId.h \ + DelayIdComposite.h DelayBucket.cc DelayBucket.h DelayConfig.cc \ + DelayConfig.h DelayPool.cc DelayPool.h DelayPools.h \ + DelaySpec.cc DelaySpec.h DelayTagged.cc DelayTagged.h \ + DelayUser.cc DelayUser.h DelayVector.cc DelayVector.h \ + NullDelayId.cc NullDelayId.h disk.cc DiskIO/DiskIOModule.cc \ + DiskIO/ReadRequest.cc DiskIO/WriteRequest.cc dns_internal.cc \ + dns.cc comm_epoll.cc enums.h errorpage.cc ElementList.h ESI.cc \ + ESI.h ESIAssign.cc ESIAssign.h ESIAttempt.h ESIContext.cc \ + ESIContext.h ESICustomParser.cc ESICustomParser.h ESIElement.h \ + ESIExcept.h ESIExpatParser.cc ESIExpatParser.h \ + ESIExpression.cc ESIExpression.h ESIInclude.cc ESIInclude.h \ + ESILiteral.h ESILibxml2Parser.cc ESILibxml2Parser.h \ + ESIParser.cc ESIParser.h ESISegment.cc ESISegment.h \ + ESISequence.cc ESISequence.h ESIVar.h ESIVarState.cc \ + ESIVarState.h ETag.cc event.cc external_acl.cc ExternalACL.h \ + ExternalACLEntry.cc ExternalACLEntry.h fd.cc fde.cc fde.h \ + filemap.cc forward.cc fqdncache.cc ftp.cc Generic.h globals.h \ + gopher.cc helper.cc helper.h HierarchyLogEntry.h htcp.cc \ + htcp.h http.cc http.h HttpStatusLine.cc HttpStatusLine.h \ + HttpHdrCc.cc HttpHdrRange.cc HttpHdrSc.cc HttpHdrScTarget.cc \ HttpHdrContRange.cc HttpHdrContRange.h HttpHeader.cc \ HttpHeader.h HttpHeaderRange.h HttpHeaderTools.cc HttpBody.cc \ HttpMsg.cc HttpReply.cc HttpReply.h HttpRequest.cc \ @@ -174,9 +208,8 @@ am__squid_SOURCES_DIST = access_log.cc AccessLogEntry.h acl.cc ACL.h \ StoreMetaUnpacker.cc StoreMetaUnpacker.h StoreMetaURL.cc \ StoreMetaURL.h StoreMetaVary.cc StoreMetaVary.h \ StoreSwapLogData.cc StoreSwapLogData.h structs.h SwapDir.cc \ - SwapDir.h tools.cc typedefs.h ufscommon.cc ufscommon.h \ - unlinkd.cc url.cc urn.cc useragent.cc wais.cc wccp.cc whois.cc \ - win32.cc + SwapDir.h tools.cc typedefs.h unlinkd.cc url.cc urn.cc \ + useragent.cc wais.cc wccp.cc whois.cc win32.cc am__objects_1 = ACLARP.$(OBJEXT) @ENABLE_ARP_ACL_TRUE@am__objects_2 = $(am__objects_1) am__objects_3 = $(am__objects_2) ACLASN.$(OBJEXT) ACLBrowser.$(OBJEXT) \ @@ -232,15 +265,18 @@ am_squid_OBJECTS = access_log.$(OBJEXT) acl.$(OBJEXT) \ cbdata.$(OBJEXT) client_db.$(OBJEXT) client_side.$(OBJEXT) \ client_side_reply.$(OBJEXT) client_side_request.$(OBJEXT) \ clientStream.$(OBJEXT) comm.$(OBJEXT) comm_select.$(OBJEXT) \ - comm_poll.$(OBJEXT) comm_kqueue.$(OBJEXT) debug.$(OBJEXT) \ - $(am__objects_5) disk.$(OBJEXT) $(am__objects_6) \ - $(am__objects_8) errorpage.$(OBJEXT) $(am__objects_10) \ - ETag.$(OBJEXT) event.$(OBJEXT) external_acl.$(OBJEXT) \ - ExternalACLEntry.$(OBJEXT) fd.$(OBJEXT) fde.$(OBJEXT) \ - filemap.$(OBJEXT) forward.$(OBJEXT) fqdncache.$(OBJEXT) \ - ftp.$(OBJEXT) gopher.$(OBJEXT) helper.$(OBJEXT) \ - $(am__objects_11) http.$(OBJEXT) HttpStatusLine.$(OBJEXT) \ - HttpHdrCc.$(OBJEXT) HttpHdrRange.$(OBJEXT) HttpHdrSc.$(OBJEXT) \ + comm_poll.$(OBJEXT) comm_kqueue.$(OBJEXT) \ + ConfigOption.$(OBJEXT) debug.$(OBJEXT) $(am__objects_5) \ + disk.$(OBJEXT) DiskIO/DiskIOModule.$(OBJEXT) \ + DiskIO/ReadRequest.$(OBJEXT) DiskIO/WriteRequest.$(OBJEXT) \ + $(am__objects_6) $(am__objects_8) errorpage.$(OBJEXT) \ + $(am__objects_10) ETag.$(OBJEXT) event.$(OBJEXT) \ + external_acl.$(OBJEXT) ExternalACLEntry.$(OBJEXT) fd.$(OBJEXT) \ + fde.$(OBJEXT) filemap.$(OBJEXT) forward.$(OBJEXT) \ + fqdncache.$(OBJEXT) ftp.$(OBJEXT) gopher.$(OBJEXT) \ + helper.$(OBJEXT) $(am__objects_11) http.$(OBJEXT) \ + HttpStatusLine.$(OBJEXT) HttpHdrCc.$(OBJEXT) \ + HttpHdrRange.$(OBJEXT) HttpHdrSc.$(OBJEXT) \ HttpHdrScTarget.$(OBJEXT) HttpHdrContRange.$(OBJEXT) \ HttpHeader.$(OBJEXT) HttpHeaderTools.$(OBJEXT) \ HttpBody.$(OBJEXT) HttpMsg.$(OBJEXT) HttpReply.$(OBJEXT) \ @@ -267,10 +303,9 @@ am_squid_OBJECTS = access_log.$(OBJEXT) acl.$(OBJEXT) \ StoreMetaSTD.$(OBJEXT) StoreMetaUnpacker.$(OBJEXT) \ StoreMetaURL.$(OBJEXT) StoreMetaVary.$(OBJEXT) \ StoreSwapLogData.$(OBJEXT) SwapDir.$(OBJEXT) tools.$(OBJEXT) \ - ufscommon.$(OBJEXT) $(am__objects_19) url.$(OBJEXT) \ - urn.$(OBJEXT) useragent.$(OBJEXT) wais.$(OBJEXT) \ - wccp.$(OBJEXT) whois.$(OBJEXT) $(am__objects_20) -am__dirstamp = $(am__leading_dot)dirstamp + $(am__objects_19) url.$(OBJEXT) urn.$(OBJEXT) \ + useragent.$(OBJEXT) wais.$(OBJEXT) wccp.$(OBJEXT) \ + whois.$(OBJEXT) $(am__objects_20) nodist_squid_OBJECTS = repl_modules.$(OBJEXT) globals.$(OBJEXT) \ string_arrays.$(OBJEXT) squid_OBJECTS = $(am_squid_OBJECTS) $(nodist_squid_OBJECTS) @@ -278,9 +313,9 @@ am_squidclient_OBJECTS = client.$(OBJEXT) squidclient_OBJECTS = $(am_squidclient_OBJECTS) squidclient_LDADD = $(LDADD) squidclient_DEPENDENCIES = -am__objects_23 = ../test-suite/test_tools.$(OBJEXT) +am__objects_24 = ../test-suite/test_tools.$(OBJEXT) am_tests_testAuth_OBJECTS = tests/testAuth.$(OBJEXT) \ - tests/testMain.$(OBJEXT) $(am__objects_23) \ + tests/testMain.$(OBJEXT) $(am__objects_24) \ AuthScheme.$(OBJEXT) globals.$(OBJEXT) authenticate.$(OBJEXT) \ AuthUser.$(OBJEXT) AuthUserRequest.$(OBJEXT) \ AuthConfig.$(OBJEXT) tests/stub_acl.$(OBJEXT) \ @@ -362,9 +397,8 @@ am__ufsdump_SOURCES_DIST = debug.cc int.cc ufsdump.cc store.cc \ StoreIOState.cc store_client.cc StoreClient.h store_digest.cc \ store_dir.cc store_key_md5.cc store_log.cc store_rebuild.cc \ store_swapin.cc store_swapmeta.cc store_swapout.cc structs.h \ - SwapDir.cc tools.cc typedefs.h ufscommon.cc ufscommon.h \ - unlinkd.cc url.cc urn.cc useragent.cc wais.cc wccp.cc whois.cc \ - win32.cc + SwapDir.cc tools.cc typedefs.h unlinkd.cc url.cc urn.cc \ + useragent.cc wais.cc wccp.cc whois.cc win32.cc am_ufsdump_OBJECTS = debug.$(OBJEXT) int.$(OBJEXT) ufsdump.$(OBJEXT) \ store.$(OBJEXT) StoreFileSystem.$(OBJEXT) StoreMeta.$(OBJEXT) \ StoreMetaMD5.$(OBJEXT) StoreMetaSTD.$(OBJEXT) \ @@ -407,9 +441,9 @@ am_ufsdump_OBJECTS = debug.$(OBJEXT) int.$(OBJEXT) ufsdump.$(OBJEXT) \ store_log.$(OBJEXT) store_rebuild.$(OBJEXT) \ store_swapin.$(OBJEXT) store_swapmeta.$(OBJEXT) \ store_swapout.$(OBJEXT) SwapDir.$(OBJEXT) tools.$(OBJEXT) \ - ufscommon.$(OBJEXT) $(am__objects_19) url.$(OBJEXT) \ - urn.$(OBJEXT) useragent.$(OBJEXT) wais.$(OBJEXT) \ - wccp.$(OBJEXT) whois.$(OBJEXT) $(am__objects_20) + $(am__objects_19) url.$(OBJEXT) urn.$(OBJEXT) \ + useragent.$(OBJEXT) wais.$(OBJEXT) wccp.$(OBJEXT) \ + whois.$(OBJEXT) $(am__objects_20) nodist_ufsdump_OBJECTS = repl_modules.$(OBJEXT) globals.$(OBJEXT) \ string_arrays.$(OBJEXT) ufsdump_OBJECTS = $(am_ufsdump_OBJECTS) $(nodist_ufsdump_OBJECTS) @@ -437,14 +471,20 @@ LTCXXCOMPILE = $(LIBTOOL) --tag=CXX --mode=compile $(CXX) $(DEFS) \ CXXLD = $(CXX) CXXLINK = $(LIBTOOL) --tag=CXX --mode=link $(CXXLD) $(AM_CXXFLAGS) \ $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ -SOURCES = $(cachemgr__CGIEXT__SOURCES) $(cf_gen_SOURCES) \ +SOURCES = $(libAIO_a_SOURCES) $(libBlocking_a_SOURCES) \ + $(libDiskDaemon_a_SOURCES) $(libDiskThreads_a_SOURCES) \ + $(DiskIO_DiskDaemon_diskd_SOURCES) \ + $(cachemgr__CGIEXT__SOURCES) $(cf_gen_SOURCES) \ $(dnsserver_SOURCES) $(pinger_SOURCES) \ $(nodist_pinger_SOURCES) $(recv_announce_SOURCES) \ $(squid_SOURCES) $(EXTRA_squid_SOURCES) \ $(nodist_squid_SOURCES) $(squidclient_SOURCES) \ $(tests_testAuth_SOURCES) $(ufsdump_SOURCES) \ $(nodist_ufsdump_SOURCES) $(unlinkd_SOURCES) -DIST_SOURCES = $(cachemgr__CGIEXT__SOURCES) $(cf_gen_SOURCES) \ +DIST_SOURCES = $(libAIO_a_SOURCES) $(libBlocking_a_SOURCES) \ + $(libDiskDaemon_a_SOURCES) $(libDiskThreads_a_SOURCES) \ + $(DiskIO_DiskDaemon_diskd_SOURCES) \ + $(cachemgr__CGIEXT__SOURCES) $(cf_gen_SOURCES) \ $(dnsserver_SOURCES) $(pinger_SOURCES) \ $(recv_announce_SOURCES) $(am__squid_SOURCES_DIST) \ $(EXTRA_squid_SOURCES) $(squidclient_SOURCES) \ @@ -501,6 +541,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ -DDEFAULT_CONFIG_FILE=\"$(DEFAULT_CONFIG_FILE)\" DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -588,9 +631,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ @@ -755,7 +796,9 @@ EPOLL_ALL_SOURCE = comm_epoll.cc AM_CFLAGS = @SQUID_CFLAGS@ AM_CXXFLAGS = @SQUID_CXXFLAGS@ SUBDIRS = fs repl auth -INCLUDES = -I/usr/include/libxml2 -I. -I$(srcdir) -I$(top_builddir)/include -I$(top_srcdir)/include -I$(top_srcdir)/lib/libTrie/include +EXTRA_LIBRARIES = libAIO.a libBlocking.a libDiskDaemon.a libDiskThreads.a +noinst_LIBRARIES = @DISK_LIBS@ +INCLUDES = -I. -I$(srcdir) -I$(top_builddir)/include -I$(top_srcdir)/include -I$(top_srcdir)/lib/libTrie/include cf_gen_SOURCES = cf_gen.cc defines.h nodist_cf_gen_HEADER = cf_gen_defines.h squidclient_SOURCES = client.cc @@ -767,6 +810,12 @@ all_FSMODULES = \ fs/null/StoreFSnull.cc \ fs/ufs/StoreFSufs.cc +all_DISKIOMODULES = \ + DiskIO/AIO/AIODiskIOModule.cc \ + DiskIO/Blocking/BlockingDiskIOModule.cc \ + DiskIO/DiskDaemon/DiskDaemonDiskIOModule.cc \ + DiskIO/DiskThreads/DiskThreadsDiskIOModule.cc + all_AUTHMODULES = \ auth/basic/basicScheme.cc \ auth/basic/basicScheme.h \ @@ -777,8 +826,10 @@ all_AUTHMODULES = \ EXTRA_squid_SOURCES = \ $(all_FSMODULES) \ + $(all_DISKIOMODULES) \ $(all_AUTHMODULES) \ $(ARP_ACL_ALL_SOURCE) \ + ConfigOption.h \ $(DELAY_POOL_ALL_SOURCE) \ dns.cc \ dnsserver.cc \ @@ -904,6 +955,7 @@ squid_SOURCES = \ comm_poll.cc \ comm_kqueue.cc \ CommRead.h \ + ConfigOption.cc \ ConfigParser.h \ ConnectionDetail.h \ debug.cc \ @@ -911,6 +963,9 @@ squid_SOURCES = \ defines.h \ $(DELAY_POOL_SOURCE) \ disk.cc \ + DiskIO/DiskIOModule.cc \ + DiskIO/ReadRequest.cc \ + DiskIO/WriteRequest.cc \ $(DNSSOURCE) \ $(EPOLL_SOURCE) \ enums.h \ @@ -1041,8 +1096,6 @@ squid_SOURCES = \ SwapDir.h \ tools.cc \ typedefs.h \ - ufscommon.cc \ - ufscommon.h \ $(UNLINKDSOURCE) \ url.cc \ urn.cc \ @@ -1062,7 +1115,6 @@ noinst_HEADERS = ACLChecklist.cci \ Store.cci \ String.cci \ SquidString.h \ - ufscommon.cci \ squid_windows.h nodist_squid_SOURCES = \ @@ -1074,9 +1126,11 @@ nodist_squid_SOURCES = \ squid_LDADD = \ -L../lib \ @XTRA_OBJS@ \ + @DISK_LINKOBJS@ \ @REPL_OBJS@ \ @STORE_LINKOBJS@ \ @STORE_OBJS@ \ + @DISK_LIBS@ \ @AUTH_LINKOBJS@ \ @AUTH_OBJS@ \ @CRYPTLIB@ \ @@ -1089,6 +1143,8 @@ squid_LDADD = \ @EPOLL_LIBS@ squid_DEPENDENCIES = $(top_builddir)/lib/libmiscutil.a @STORE_OBJS@ @STORE_LINKOBJS@ \ + @DISK_LIBS@ \ + @DISK_LINKOBJS@ \ @REPL_OBJS@ \ @AUTH_LINKOBJS@ \ @AUTH_OBJS@ @@ -1238,8 +1294,6 @@ ufsdump_SOURCES = debug.cc \ SwapDir.cc \ tools.cc \ typedefs.h \ - ufscommon.cc \ - ufscommon.h \ $(UNLINKDSOURCE) \ url.cc \ urn.cc \ @@ -1289,6 +1343,13 @@ data_DATA = \ mib.txt LDADD = -L../lib -lmiscutil @XTRA_LIBS@ @EPOLL_LIBS@ +DISKIODIST = \ + DiskIO/DiskFile.h \ + DiskIO/DiskIOStrategy.h \ + DiskIO/IORequestor.h \ + DiskIO/DiskIOModule.h \ + DiskIO/ReadRequest.h + EXTRA_DIST = \ cf_gen_defines \ cf.data.pre \ @@ -1296,8 +1357,44 @@ EXTRA_DIST = \ mk-string-arrays.pl \ repl_modules.sh \ mib.txt \ - mime.conf.default - + mime.conf.default \ + $(DISKIODIST) + +libAIO_a_SOURCES = \ + DiskIO/AIO/async_io.h \ + DiskIO/AIO/AIODiskFile.cc \ + DiskIO/AIO/AIODiskFile.h \ + DiskIO/AIO/AIODiskIOStrategy.cc \ + DiskIO/AIO/AIODiskIOStrategy.h \ + DiskIO/AIO/AIODiskIOModule.h + +libBlocking_a_SOURCES = \ + DiskIO/Blocking/BlockingFile.cc \ + DiskIO/Blocking/BlockingFile.h \ + DiskIO/Blocking/BlockingIOStrategy.cc \ + DiskIO/Blocking/BlockingIOStrategy.h \ + DiskIO/Blocking/BlockingDiskIOModule.h + +libDiskDaemon_a_SOURCES = \ + DiskIO/DiskDaemon/DiskdFile.cc \ + DiskIO/DiskDaemon/DiskdFile.h \ + DiskIO/DiskDaemon/DiskdIOStrategy.cc \ + DiskIO/DiskDaemon/DiskdIOStrategy.h \ + DiskIO/DiskDaemon/diomsg.h \ + DiskIO/DiskDaemon/DiskDaemonDiskIOModule.h + +libDiskThreads_a_SOURCES = \ + DiskIO/DiskThreads/aiops.cc \ + DiskIO/DiskThreads/async_io.cc \ + DiskIO/DiskThreads/DiskThreads.h \ + DiskIO/DiskThreads/DiskThreadsDiskFile.cc \ + DiskIO/DiskThreads/DiskThreadsDiskFile.h \ + DiskIO/DiskThreads/DiskThreadsDiskIOModule.h \ + DiskIO/DiskThreads/DiskThreadsIOStrategy.cc \ + DiskIO/DiskThreads/DiskThreadsIOStrategy.h + +DiskIO_DiskDaemon_diskd_SOURCES = DiskIO/DiskDaemon/diskd.cc +DiskIO_DiskDaemon_diskd_LDADD = $(top_builddir)/lib/libmiscutil.a @XTRA_LIBS@ DEFAULT_PREFIX = $(prefix) DEFAULT_CONFIG_FILE = $(sysconfdir)/squid.conf DEFAULT_MIME_TABLE = $(sysconfdir)/mime.conf @@ -1380,6 +1477,77 @@ $(top_srcdir)/configure: @MAINTAINER_MODE_TRUE@ $(am__configure_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(ACLOCAL_M4): @MAINTAINER_MODE_TRUE@ $(am__aclocal_m4_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh + +clean-noinstLIBRARIES: + -test -z "$(noinst_LIBRARIES)" || rm -f $(noinst_LIBRARIES) +DiskIO/AIO/$(am__dirstamp): + @$(mkdir_p) DiskIO/AIO + @: > DiskIO/AIO/$(am__dirstamp) +DiskIO/AIO/$(DEPDIR)/$(am__dirstamp): + @$(mkdir_p) DiskIO/AIO/$(DEPDIR) + @: > DiskIO/AIO/$(DEPDIR)/$(am__dirstamp) +DiskIO/AIO/AIODiskFile.$(OBJEXT): DiskIO/AIO/$(am__dirstamp) \ + DiskIO/AIO/$(DEPDIR)/$(am__dirstamp) +DiskIO/AIO/AIODiskIOStrategy.$(OBJEXT): DiskIO/AIO/$(am__dirstamp) \ + DiskIO/AIO/$(DEPDIR)/$(am__dirstamp) +libAIO.a: $(libAIO_a_OBJECTS) $(libAIO_a_DEPENDENCIES) + -rm -f libAIO.a + $(libAIO_a_AR) libAIO.a $(libAIO_a_OBJECTS) $(libAIO_a_LIBADD) + $(RANLIB) libAIO.a +DiskIO/Blocking/$(am__dirstamp): + @$(mkdir_p) DiskIO/Blocking + @: > DiskIO/Blocking/$(am__dirstamp) +DiskIO/Blocking/$(DEPDIR)/$(am__dirstamp): + @$(mkdir_p) DiskIO/Blocking/$(DEPDIR) + @: > DiskIO/Blocking/$(DEPDIR)/$(am__dirstamp) +DiskIO/Blocking/BlockingFile.$(OBJEXT): \ + DiskIO/Blocking/$(am__dirstamp) \ + DiskIO/Blocking/$(DEPDIR)/$(am__dirstamp) +DiskIO/Blocking/BlockingIOStrategy.$(OBJEXT): \ + DiskIO/Blocking/$(am__dirstamp) \ + DiskIO/Blocking/$(DEPDIR)/$(am__dirstamp) +libBlocking.a: $(libBlocking_a_OBJECTS) $(libBlocking_a_DEPENDENCIES) + -rm -f libBlocking.a + $(libBlocking_a_AR) libBlocking.a $(libBlocking_a_OBJECTS) $(libBlocking_a_LIBADD) + $(RANLIB) libBlocking.a +DiskIO/DiskDaemon/$(am__dirstamp): + @$(mkdir_p) DiskIO/DiskDaemon + @: > DiskIO/DiskDaemon/$(am__dirstamp) +DiskIO/DiskDaemon/$(DEPDIR)/$(am__dirstamp): + @$(mkdir_p) DiskIO/DiskDaemon/$(DEPDIR) + @: > DiskIO/DiskDaemon/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskDaemon/DiskdFile.$(OBJEXT): \ + DiskIO/DiskDaemon/$(am__dirstamp) \ + DiskIO/DiskDaemon/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskDaemon/DiskdIOStrategy.$(OBJEXT): \ + DiskIO/DiskDaemon/$(am__dirstamp) \ + DiskIO/DiskDaemon/$(DEPDIR)/$(am__dirstamp) +libDiskDaemon.a: $(libDiskDaemon_a_OBJECTS) $(libDiskDaemon_a_DEPENDENCIES) + -rm -f libDiskDaemon.a + $(libDiskDaemon_a_AR) libDiskDaemon.a $(libDiskDaemon_a_OBJECTS) $(libDiskDaemon_a_LIBADD) + $(RANLIB) libDiskDaemon.a +DiskIO/DiskThreads/$(am__dirstamp): + @$(mkdir_p) DiskIO/DiskThreads + @: > DiskIO/DiskThreads/$(am__dirstamp) +DiskIO/DiskThreads/$(DEPDIR)/$(am__dirstamp): + @$(mkdir_p) DiskIO/DiskThreads/$(DEPDIR) + @: > DiskIO/DiskThreads/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskThreads/aiops.$(OBJEXT): \ + DiskIO/DiskThreads/$(am__dirstamp) \ + DiskIO/DiskThreads/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskThreads/async_io.$(OBJEXT): \ + DiskIO/DiskThreads/$(am__dirstamp) \ + DiskIO/DiskThreads/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskThreads/DiskThreadsDiskFile.$(OBJEXT): \ + DiskIO/DiskThreads/$(am__dirstamp) \ + DiskIO/DiskThreads/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskThreads/DiskThreadsIOStrategy.$(OBJEXT): \ + DiskIO/DiskThreads/$(am__dirstamp) \ + DiskIO/DiskThreads/$(DEPDIR)/$(am__dirstamp) +libDiskThreads.a: $(libDiskThreads_a_OBJECTS) $(libDiskThreads_a_DEPENDENCIES) + -rm -f libDiskThreads.a + $(libDiskThreads_a_AR) libDiskThreads.a $(libDiskThreads_a_OBJECTS) $(libDiskThreads_a_LIBADD) + $(RANLIB) libDiskThreads.a install-binPROGRAMS: $(bin_PROGRAMS) @$(NORMAL_INSTALL) test -z "$(bindir)" || $(mkdir_p) "$(DESTDIR)$(bindir)" @@ -1478,6 +1646,11 @@ clean-sbinPROGRAMS: echo " rm -f $$p $$f"; \ rm -f $$p $$f ; \ done +DiskIO/DiskDaemon/diskd.$(OBJEXT): DiskIO/DiskDaemon/$(am__dirstamp) \ + DiskIO/DiskDaemon/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskDaemon/diskd$(EXEEXT): $(DiskIO_DiskDaemon_diskd_OBJECTS) $(DiskIO_DiskDaemon_diskd_DEPENDENCIES) DiskIO/DiskDaemon/$(am__dirstamp) + @rm -f DiskIO/DiskDaemon/diskd$(EXEEXT) + $(CXXLINK) $(DiskIO_DiskDaemon_diskd_LDFLAGS) $(DiskIO_DiskDaemon_diskd_OBJECTS) $(DiskIO_DiskDaemon_diskd_LDADD) $(LIBS) cachemgr$(CGIEXT)$(EXEEXT): $(cachemgr__CGIEXT__OBJECTS) $(cachemgr__CGIEXT__DEPENDENCIES) @rm -f cachemgr$(CGIEXT)$(EXEEXT) $(CXXLINK) $(cachemgr__CGIEXT__LDFLAGS) $(cachemgr__CGIEXT__OBJECTS) $(cachemgr__CGIEXT__LDADD) $(LIBS) @@ -1493,6 +1666,18 @@ pinger$(EXEEXT): $(pinger_OBJECTS) $(pinger_DEPENDENCIES) recv-announce$(EXEEXT): $(recv_announce_OBJECTS) $(recv_announce_DEPENDENCIES) @rm -f recv-announce$(EXEEXT) $(CXXLINK) $(recv_announce_LDFLAGS) $(recv_announce_OBJECTS) $(recv_announce_LDADD) $(LIBS) +DiskIO/$(am__dirstamp): + @$(mkdir_p) DiskIO + @: > DiskIO/$(am__dirstamp) +DiskIO/$(DEPDIR)/$(am__dirstamp): + @$(mkdir_p) DiskIO/$(DEPDIR) + @: > DiskIO/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskIOModule.$(OBJEXT): DiskIO/$(am__dirstamp) \ + DiskIO/$(DEPDIR)/$(am__dirstamp) +DiskIO/ReadRequest.$(OBJEXT): DiskIO/$(am__dirstamp) \ + DiskIO/$(DEPDIR)/$(am__dirstamp) +DiskIO/WriteRequest.$(OBJEXT): DiskIO/$(am__dirstamp) \ + DiskIO/$(DEPDIR)/$(am__dirstamp) fs/aufs/$(am__dirstamp): @$(mkdir_p) fs/aufs @: > fs/aufs/$(am__dirstamp) @@ -1533,6 +1718,17 @@ fs/ufs/$(DEPDIR)/$(am__dirstamp): @: > fs/ufs/$(DEPDIR)/$(am__dirstamp) fs/ufs/StoreFSufs.$(OBJEXT): fs/ufs/$(am__dirstamp) \ fs/ufs/$(DEPDIR)/$(am__dirstamp) +DiskIO/AIO/AIODiskIOModule.$(OBJEXT): DiskIO/AIO/$(am__dirstamp) \ + DiskIO/AIO/$(DEPDIR)/$(am__dirstamp) +DiskIO/Blocking/BlockingDiskIOModule.$(OBJEXT): \ + DiskIO/Blocking/$(am__dirstamp) \ + DiskIO/Blocking/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskDaemon/DiskDaemonDiskIOModule.$(OBJEXT): \ + DiskIO/DiskDaemon/$(am__dirstamp) \ + DiskIO/DiskDaemon/$(DEPDIR)/$(am__dirstamp) +DiskIO/DiskThreads/DiskThreadsDiskIOModule.$(OBJEXT): \ + DiskIO/DiskThreads/$(am__dirstamp) \ + DiskIO/DiskThreads/$(DEPDIR)/$(am__dirstamp) auth/basic/$(am__dirstamp): @$(mkdir_p) auth/basic @: > auth/basic/$(am__dirstamp) @@ -1608,6 +1804,24 @@ unlinkd$(EXEEXT): $(unlinkd_OBJECTS) $(unlinkd_DEPENDENCIES) mostlyclean-compile: -rm -f *.$(OBJEXT) -rm -f ../test-suite/test_tools.$(OBJEXT) + -rm -f DiskIO/AIO/AIODiskFile.$(OBJEXT) + -rm -f DiskIO/AIO/AIODiskIOModule.$(OBJEXT) + -rm -f DiskIO/AIO/AIODiskIOStrategy.$(OBJEXT) + -rm -f DiskIO/Blocking/BlockingDiskIOModule.$(OBJEXT) + -rm -f DiskIO/Blocking/BlockingFile.$(OBJEXT) + -rm -f DiskIO/Blocking/BlockingIOStrategy.$(OBJEXT) + -rm -f DiskIO/DiskDaemon/DiskDaemonDiskIOModule.$(OBJEXT) + -rm -f DiskIO/DiskDaemon/DiskdFile.$(OBJEXT) + -rm -f DiskIO/DiskDaemon/DiskdIOStrategy.$(OBJEXT) + -rm -f DiskIO/DiskDaemon/diskd.$(OBJEXT) + -rm -f DiskIO/DiskIOModule.$(OBJEXT) + -rm -f DiskIO/DiskThreads/DiskThreadsDiskFile.$(OBJEXT) + -rm -f DiskIO/DiskThreads/DiskThreadsDiskIOModule.$(OBJEXT) + -rm -f DiskIO/DiskThreads/DiskThreadsIOStrategy.$(OBJEXT) + -rm -f DiskIO/DiskThreads/aiops.$(OBJEXT) + -rm -f DiskIO/DiskThreads/async_io.$(OBJEXT) + -rm -f DiskIO/ReadRequest.$(OBJEXT) + -rm -f DiskIO/WriteRequest.$(OBJEXT) -rm -f auth/basic/basicScheme.$(OBJEXT) -rm -f auth/digest/digestScheme.$(OBJEXT) -rm -f auth/ntlm/ntlmScheme.$(OBJEXT) @@ -1671,6 +1885,7 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/AuthUser.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/AuthUserRequest.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/CacheDigest.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ConfigOption.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/DelayBucket.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/DelayConfig.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/DelayId.Po@am__quote@ @@ -1811,7 +2026,6 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/string_arrays.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tools.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tunnel.Po@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ufscommon.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ufsdump.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/unlinkd-SquidNew.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/unlinkd-unlinkd.Po@am__quote@ @@ -1823,6 +2037,24 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/wccp.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/whois.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/win32.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/$(DEPDIR)/DiskIOModule.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/$(DEPDIR)/ReadRequest.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/$(DEPDIR)/WriteRequest.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/AIO/$(DEPDIR)/AIODiskFile.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/AIO/$(DEPDIR)/AIODiskIOModule.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/AIO/$(DEPDIR)/AIODiskIOStrategy.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/Blocking/$(DEPDIR)/BlockingDiskIOModule.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/Blocking/$(DEPDIR)/BlockingFile.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/Blocking/$(DEPDIR)/BlockingIOStrategy.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskDaemon/$(DEPDIR)/DiskDaemonDiskIOModule.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskDaemon/$(DEPDIR)/DiskdFile.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskDaemon/$(DEPDIR)/DiskdIOStrategy.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskDaemon/$(DEPDIR)/diskd.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskThreads/$(DEPDIR)/DiskThreadsDiskFile.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskThreads/$(DEPDIR)/DiskThreadsDiskIOModule.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskThreads/$(DEPDIR)/DiskThreadsIOStrategy.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskThreads/$(DEPDIR)/aiops.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@DiskIO/DiskThreads/$(DEPDIR)/async_io.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@auth/basic/$(DEPDIR)/basicScheme.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@auth/digest/$(DEPDIR)/digestScheme.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@auth/ntlm/$(DEPDIR)/ntlmScheme.Po@am__quote@ @@ -2155,6 +2387,7 @@ check-TESTS: $(TESTS) else :; fi distdir: $(DISTFILES) + $(mkdir_p) $(distdir)/DiskIO @srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; \ topsrcdirstrip=`echo "$(top_srcdir)" | sed 's|.|.|g'`; \ list='$(DISTFILES)'; for file in $$list; do \ @@ -2201,7 +2434,7 @@ check-am: all-am $(MAKE) $(AM_MAKEFLAGS) check-TESTS check: $(BUILT_SOURCES) $(MAKE) $(AM_MAKEFLAGS) check-recursive -all-am: Makefile $(PROGRAMS) $(DATA) $(HEADERS) +all-am: Makefile $(LIBRARIES) $(PROGRAMS) $(DATA) $(HEADERS) installdirs: installdirs-recursive installdirs-am: for dir in "$(DESTDIR)$(bindir)" "$(DESTDIR)$(libexecdir)" "$(DESTDIR)$(sbindir)" "$(DESTDIR)$(datadir)" "$(DESTDIR)$(sysconfdir)"; do \ @@ -2230,6 +2463,16 @@ distclean-generic: -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) -rm -f ../test-suite/$(DEPDIR)/$(am__dirstamp) -rm -f ../test-suite/$(am__dirstamp) + -rm -f DiskIO/$(DEPDIR)/$(am__dirstamp) + -rm -f DiskIO/$(am__dirstamp) + -rm -f DiskIO/AIO/$(DEPDIR)/$(am__dirstamp) + -rm -f DiskIO/AIO/$(am__dirstamp) + -rm -f DiskIO/Blocking/$(DEPDIR)/$(am__dirstamp) + -rm -f DiskIO/Blocking/$(am__dirstamp) + -rm -f DiskIO/DiskDaemon/$(DEPDIR)/$(am__dirstamp) + -rm -f DiskIO/DiskDaemon/$(am__dirstamp) + -rm -f DiskIO/DiskThreads/$(DEPDIR)/$(am__dirstamp) + -rm -f DiskIO/DiskThreads/$(am__dirstamp) -rm -f auth/basic/$(DEPDIR)/$(am__dirstamp) -rm -f auth/basic/$(am__dirstamp) -rm -f auth/digest/$(DEPDIR)/$(am__dirstamp) @@ -2257,11 +2500,11 @@ maintainer-clean-generic: clean: clean-recursive clean-am: clean-binPROGRAMS clean-checkPROGRAMS clean-generic \ - clean-libexecPROGRAMS clean-libtool clean-noinstPROGRAMS \ - clean-sbinPROGRAMS mostlyclean-am + clean-libexecPROGRAMS clean-libtool clean-noinstLIBRARIES \ + clean-noinstPROGRAMS clean-sbinPROGRAMS mostlyclean-am distclean: distclean-recursive - -rm -rf ../test-suite/$(DEPDIR) ./$(DEPDIR) auth/basic/$(DEPDIR) auth/digest/$(DEPDIR) auth/ntlm/$(DEPDIR) fs/aufs/$(DEPDIR) fs/coss/$(DEPDIR) fs/diskd/$(DEPDIR) fs/null/$(DEPDIR) fs/ufs/$(DEPDIR) tests/$(DEPDIR) + -rm -rf ../test-suite/$(DEPDIR) ./$(DEPDIR) DiskIO/$(DEPDIR) DiskIO/AIO/$(DEPDIR) DiskIO/Blocking/$(DEPDIR) DiskIO/DiskDaemon/$(DEPDIR) DiskIO/DiskThreads/$(DEPDIR) auth/basic/$(DEPDIR) auth/digest/$(DEPDIR) auth/ntlm/$(DEPDIR) fs/aufs/$(DEPDIR) fs/coss/$(DEPDIR) fs/diskd/$(DEPDIR) fs/null/$(DEPDIR) fs/ufs/$(DEPDIR) tests/$(DEPDIR) -rm -f Makefile distclean-am: clean-am distclean-compile distclean-generic \ distclean-libtool distclean-tags @@ -2288,7 +2531,7 @@ install-man: installcheck-am: maintainer-clean: maintainer-clean-recursive - -rm -rf ../test-suite/$(DEPDIR) ./$(DEPDIR) auth/basic/$(DEPDIR) auth/digest/$(DEPDIR) auth/ntlm/$(DEPDIR) fs/aufs/$(DEPDIR) fs/coss/$(DEPDIR) fs/diskd/$(DEPDIR) fs/null/$(DEPDIR) fs/ufs/$(DEPDIR) tests/$(DEPDIR) + -rm -rf ../test-suite/$(DEPDIR) ./$(DEPDIR) DiskIO/$(DEPDIR) DiskIO/AIO/$(DEPDIR) DiskIO/Blocking/$(DEPDIR) DiskIO/DiskDaemon/$(DEPDIR) DiskIO/DiskThreads/$(DEPDIR) auth/basic/$(DEPDIR) auth/digest/$(DEPDIR) auth/ntlm/$(DEPDIR) fs/aufs/$(DEPDIR) fs/coss/$(DEPDIR) fs/diskd/$(DEPDIR) fs/null/$(DEPDIR) fs/ufs/$(DEPDIR) tests/$(DEPDIR) -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic @@ -2314,20 +2557,20 @@ uninstall-info: uninstall-info-recursive .PHONY: $(RECURSIVE_TARGETS) CTAGS GTAGS all all-am check check-TESTS \ check-am clean clean-binPROGRAMS clean-checkPROGRAMS \ clean-generic clean-libexecPROGRAMS clean-libtool \ - clean-noinstPROGRAMS clean-recursive clean-sbinPROGRAMS ctags \ - ctags-recursive distclean distclean-compile distclean-generic \ - distclean-libtool distclean-recursive distclean-tags distdir \ - dvi dvi-am html html-am info info-am install install-am \ - install-binPROGRAMS install-data install-data-am \ - install-data-local install-dataDATA install-exec \ - install-exec-am install-info install-info-am \ - install-libexecPROGRAMS install-man install-sbinPROGRAMS \ - install-strip install-sysconfDATA installcheck installcheck-am \ - installdirs installdirs-am maintainer-clean \ - maintainer-clean-generic maintainer-clean-recursive \ - mostlyclean mostlyclean-compile mostlyclean-generic \ - mostlyclean-libtool mostlyclean-recursive pdf pdf-am ps ps-am \ - tags tags-recursive uninstall uninstall-am \ + clean-noinstLIBRARIES clean-noinstPROGRAMS clean-recursive \ + clean-sbinPROGRAMS ctags ctags-recursive distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-recursive distclean-tags distdir dvi dvi-am html \ + html-am info info-am install install-am install-binPROGRAMS \ + install-data install-data-am install-data-local \ + install-dataDATA install-exec install-exec-am install-info \ + install-info-am install-libexecPROGRAMS install-man \ + install-sbinPROGRAMS install-strip install-sysconfDATA \ + installcheck installcheck-am installdirs installdirs-am \ + maintainer-clean maintainer-clean-generic \ + maintainer-clean-recursive mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool mostlyclean-recursive \ + pdf pdf-am ps ps-am tags tags-recursive uninstall uninstall-am \ uninstall-binPROGRAMS uninstall-dataDATA uninstall-info-am \ uninstall-libexecPROGRAMS uninstall-local \ uninstall-sbinPROGRAMS uninstall-sysconfDATA diff --git a/src/SwapDir.cc b/src/SwapDir.cc index 6022070316..0e12241eec 100644 --- a/src/SwapDir.cc +++ b/src/SwapDir.cc @@ -1,6 +1,6 @@ /* - * $Id: SwapDir.cc,v 1.5 2003/08/31 21:20:08 robertc Exp $ + * $Id: SwapDir.cc,v 1.6 2004/12/20 16:30:34 robertc Exp $ * * DEBUG: section ?? Swap Dir base object * AUTHOR: Robert Collins @@ -37,6 +37,7 @@ #include "SwapDir.h" #include "Store.h" #include "StoreFileSystem.h" +#include "ConfigOption.h" SwapDir::~SwapDir() { @@ -131,12 +132,12 @@ SwapDir::type() const /* NOT performance critical. Really. Don't bother optimising for speed * - RBC 20030718 */ -SwapDirOption * +ConfigOption * SwapDir::getOptionTree() const { - SwapDirOptionVector *result = new SwapDirOptionVector; - result->options.push_back(new SwapDirOptionAdapter(*const_cast(this), &SwapDir::optionReadOnlyParse, &SwapDir::optionReadOnlyDump)); - result->options.push_back(new SwapDirOptionAdapter(*const_cast(this), &SwapDir::optionMaxSizeParse, &SwapDir::optionMaxSizeDump)); + ConfigOptionVector *result = new ConfigOptionVector; + result->options.push_back(new ConfigOptionAdapter(*const_cast(this), &SwapDir::optionReadOnlyParse, &SwapDir::optionReadOnlyDump)); + result->options.push_back(new ConfigOptionAdapter(*const_cast(this), &SwapDir::optionMaxSizeParse, &SwapDir::optionMaxSizeDump)); return result; } @@ -146,7 +147,7 @@ SwapDir::parseOptions(int reconfiguring) unsigned int old_read_only = flags.read_only; char *name, *value; - SwapDirOption *newOption = getOptionTree(); + ConfigOption *newOption = getOptionTree(); while ((name = strtok(NULL, w_space)) != NULL) { value = strchr(name, '='); @@ -180,7 +181,7 @@ SwapDir::parseOptions(int reconfiguring) void SwapDir::dumpOptions(StoreEntry * entry) const { - SwapDirOption *newOption = getOptionTree(); + ConfigOption *newOption = getOptionTree(); if (newOption) newOption->dump(entry); @@ -238,34 +239,3 @@ SwapDir::optionMaxSizeDump(StoreEntry * e) const if (max_objsize != -1) storeAppendPrintf(e, " max-size=%ld", (long int) max_objsize); } - -SwapDirOptionVector::~SwapDirOptionVector() -{ - while (options.size()) { - delete options.back(); - options.pop_back(); - } -} - -bool -SwapDirOptionVector::parse(char const *option, const char *value, int reconfiguring) -{ - Vector::iterator i = options.begin(); - - while (i != options.end()) { - if ((*i)->parse(option,value, reconfiguring)) - return true; - - ++i; - } - - return false; -} - -void -SwapDirOptionVector::dump(StoreEntry * e) const -{ - for(Vector::const_iterator i = options.begin(); - i != options.end(); ++i) - (*i)->dump(e); -} diff --git a/src/SwapDir.h b/src/SwapDir.h index e13580123f..a4d2e98fc3 100644 --- a/src/SwapDir.h +++ b/src/SwapDir.h @@ -1,6 +1,6 @@ /* - * $Id: SwapDir.h,v 1.6 2003/07/22 15:23:01 robertc Exp $ + * $Id: SwapDir.h,v 1.7 2004/12/20 16:30:34 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ @@ -35,61 +35,12 @@ #define SQUID_SWAPDIR_H #include "StoreIOState.h" -#include "Array.h" - -/* cache option parsers */ - -class SwapDirOption -{ - -public: - virtual ~SwapDirOption() {} - - virtual bool parse(char const *option, const char *value, int reconfiguring) = 0; - virtual void dump (StoreEntry * e) const = 0; -}; - -class SwapDirOptionVector : public SwapDirOption -{ - -public: - virtual ~SwapDirOptionVector(); - virtual bool parse(char const *option, const char *value, int reconfiguring); - virtual void dump(StoreEntry * e) const; - Vectoroptions; -}; - -template - -class SwapDirOptionAdapter : public SwapDirOption -{ - -public: - SwapDirOptionAdapter (C& theObject, bool (C::*parseFP)(char const *option, const char *value, int reconfiguring), void (C::*dumpFP) (StoreEntry * e) const) : object(theObject), parser (parseFP), dumper(dumpFP) {} - - bool parse(char const *option, const char *value, int reconfiguring) - { - if (parser) - return (object.*parser)(option, value, reconfiguring); - - return false; - } - - void dump (StoreEntry * e) const - { - if (dumper) - (object.*dumper) (e); - } - -private: - C &object; - bool (C::*parser) (char const *option, const char *value, int reconfiguring) ; - void (C::*dumper)(StoreEntry * e) const; -}; /* Store dir configuration routines */ /* SwapDir *sd, char *path ( + char *opt later when the strtok mess is gone) */ +class ConfigOption; + class SwapDir { @@ -106,7 +57,7 @@ public: protected: void parseOptions(int reconfiguring); void dumpOptions(StoreEntry * e) const; - virtual SwapDirOption *getOptionTree() const; + virtual ConfigOption *getOptionTree() const; private: bool optionReadOnlyParse(char const *option, const char *value, int reconfiguring); diff --git a/src/access_log.cc b/src/access_log.cc index d7b1915217..b2dde4c123 100644 --- a/src/access_log.cc +++ b/src/access_log.cc @@ -1,6 +1,6 @@ /* - * $Id: access_log.cc,v 1.99 2004/12/20 14:52:26 robertc Exp $ + * $Id: access_log.cc,v 1.100 2004/12/20 16:30:34 robertc Exp $ * * DEBUG: section 46 Access Log * AUTHOR: Duane Wessels diff --git a/src/auth/Makefile.in b/src/auth/Makefile.in index 5359be0b3e..008a3cd0ae 100644 --- a/src/auth/Makefile.in +++ b/src/auth/Makefile.in @@ -120,6 +120,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -207,9 +210,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ @@ -273,7 +274,7 @@ target_alias = @target_alias@ # Makefile for authentication modules in the Squid Object Cache server # -# $Id: Makefile.in,v 1.45 2004/12/20 14:52:29 robertc Exp $ +# $Id: Makefile.in,v 1.46 2004/12/20 16:30:38 robertc Exp $ # AUTOMAKE_OPTIONS = subdir-objects AM_CFLAGS = @SQUID_CFLAGS@ diff --git a/src/auth/basic/auth_basic.cc b/src/auth/basic/auth_basic.cc index a6bca4501f..e170719a24 100644 --- a/src/auth/basic/auth_basic.cc +++ b/src/auth/basic/auth_basic.cc @@ -1,5 +1,5 @@ /* - * $Id: auth_basic.cc,v 1.33 2004/12/20 14:52:29 robertc Exp $ + * $Id: auth_basic.cc,v 1.34 2004/12/20 16:30:38 robertc Exp $ * * DEBUG: section 29 Authenticator * AUTHOR: Duane Wessels diff --git a/src/auth/digest/auth_digest.cc b/src/auth/digest/auth_digest.cc index e19d9e99d1..532d05a002 100644 --- a/src/auth/digest/auth_digest.cc +++ b/src/auth/digest/auth_digest.cc @@ -1,6 +1,6 @@ /* - * $Id: auth_digest.cc,v 1.37 2004/12/20 15:39:56 robertc Exp $ + * $Id: auth_digest.cc,v 1.38 2004/12/20 16:30:40 robertc Exp $ * * DEBUG: section 29 Authenticator * AUTHOR: Robert Collins diff --git a/src/auth/ntlm/auth_ntlm.cc b/src/auth/ntlm/auth_ntlm.cc index 0239f912fe..33c77928aa 100644 --- a/src/auth/ntlm/auth_ntlm.cc +++ b/src/auth/ntlm/auth_ntlm.cc @@ -1,6 +1,6 @@ /* - * $Id: auth_ntlm.cc,v 1.43 2004/12/20 14:52:30 robertc Exp $ + * $Id: auth_ntlm.cc,v 1.44 2004/12/20 16:30:40 robertc Exp $ * * DEBUG: section 29 NTLM Authenticator * AUTHOR: Robert Collins diff --git a/src/cache_cf.cc b/src/cache_cf.cc index dc8eddab72..4f0b102b7c 100644 --- a/src/cache_cf.cc +++ b/src/cache_cf.cc @@ -1,6 +1,6 @@ /* - * $Id: cache_cf.cc,v 1.459 2004/12/20 14:52:26 robertc Exp $ + * $Id: cache_cf.cc,v 1.460 2004/12/20 16:30:34 robertc Exp $ * * DEBUG: section 3 Configuration File Parsing * AUTHOR: Harvest Derived diff --git a/src/cf.data.pre b/src/cf.data.pre index bce8c9ca97..bd0f6261e3 100644 --- a/src/cf.data.pre +++ b/src/cf.data.pre @@ -1,6 +1,6 @@ # -# $Id: cf.data.pre,v 1.365 2004/12/20 14:52:26 robertc Exp $ +# $Id: cf.data.pre,v 1.366 2004/12/20 16:30:34 robertc Exp $ # # # SQUID Web Proxy Cache http://www.squid-cache.org/ diff --git a/src/cf_gen.cc b/src/cf_gen.cc index 7986817956..71e72e12b5 100644 --- a/src/cf_gen.cc +++ b/src/cf_gen.cc @@ -1,6 +1,6 @@ /* - * $Id: cf_gen.cc,v 1.53 2004/12/20 14:52:27 robertc Exp $ + * $Id: cf_gen.cc,v 1.54 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: none Generate squid.conf.default and cf_parser.h * AUTHOR: Max Okumoto diff --git a/src/clientStream.cc b/src/clientStream.cc index 8b058285b9..b8969cbc42 100644 --- a/src/clientStream.cc +++ b/src/clientStream.cc @@ -1,6 +1,6 @@ /* - * $Id: clientStream.cc,v 1.10 2004/12/20 14:52:27 robertc Exp $ + * $Id: clientStream.cc,v 1.11 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 87 Client-side Stream routines. * AUTHOR: Robert Collins diff --git a/src/client_db.cc b/src/client_db.cc index 8f7e56e541..19adde0015 100644 --- a/src/client_db.cc +++ b/src/client_db.cc @@ -1,6 +1,6 @@ /* - * $Id: client_db.cc,v 1.60 2004/12/20 14:52:27 robertc Exp $ + * $Id: client_db.cc,v 1.61 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 0 Client Database * AUTHOR: Duane Wessels diff --git a/src/client_side.cc b/src/client_side.cc index f13bc4ec6c..90c9947ab8 100644 --- a/src/client_side.cc +++ b/src/client_side.cc @@ -1,6 +1,6 @@ /* - * $Id: client_side.cc,v 1.676 2004/12/20 14:52:27 robertc Exp $ + * $Id: client_side.cc,v 1.677 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 33 Client-side Routines * AUTHOR: Duane Wessels diff --git a/src/client_side_reply.cc b/src/client_side_reply.cc index a0dba6aad0..68e8248759 100644 --- a/src/client_side_reply.cc +++ b/src/client_side_reply.cc @@ -1,6 +1,6 @@ /* - * $Id: client_side_reply.cc,v 1.77 2004/12/20 14:52:27 robertc Exp $ + * $Id: client_side_reply.cc,v 1.78 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 88 Client-side Reply Routines * AUTHOR: Robert Collins (Originally Duane Wessels in client_side.c) diff --git a/src/client_side_request.cc b/src/client_side_request.cc index 93dab649dc..eadecb0910 100644 --- a/src/client_side_request.cc +++ b/src/client_side_request.cc @@ -1,6 +1,6 @@ /* - * $Id: client_side_request.cc,v 1.41 2004/12/20 14:52:27 robertc Exp $ + * $Id: client_side_request.cc,v 1.42 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 85 Client-side Request Routines * AUTHOR: Robert Collins (Originally Duane Wessels in client_side.c) diff --git a/src/comm.cc b/src/comm.cc index 4160a2ba41..5165ce9f43 100644 --- a/src/comm.cc +++ b/src/comm.cc @@ -1,6 +1,6 @@ /* - * $Id: comm.cc,v 1.398 2004/12/20 14:52:27 robertc Exp $ + * $Id: comm.cc,v 1.399 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 5 Socket Functions * AUTHOR: Harvest Derived diff --git a/src/comm_select.cc b/src/comm_select.cc index b1159787e1..e3bfad6ca7 100644 --- a/src/comm_select.cc +++ b/src/comm_select.cc @@ -1,6 +1,6 @@ /* - * $Id: comm_select.cc,v 1.73 2004/12/20 14:52:27 robertc Exp $ + * $Id: comm_select.cc,v 1.74 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 5 Socket Functions * diff --git a/src/defines.h b/src/defines.h index ba50b834f5..0699417079 100644 --- a/src/defines.h +++ b/src/defines.h @@ -1,6 +1,6 @@ /* - * $Id: defines.h,v 1.114 2003/04/24 06:35:08 hno Exp $ + * $Id: defines.h,v 1.115 2004/12/20 16:30:35 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ @@ -279,10 +279,34 @@ #define cbdataInternalLock(a) cbdataInternalLockDbg(a,__FILE__,__LINE__) #define cbdataInternalUnlock(a) cbdataInternalUnlockDbg(a,__FILE__,__LINE__) #define cbdataReferenceValidDone(var, ptr) cbdataInternalReferenceDoneValidDbg((void **)&(var), (ptr), __FILE__,__LINE__) +#define CBDATA_CLASS2(type) \ + static cbdata_type CBDATA_##type; \ + public: \ + void *operator new(size_t size) { \ + assert(size == sizeof(type)); \ + (CBDATA_##type ? CBDATA_UNKNOWN : (CBDATA_##type = cbdataInternalAddType(CBDATA_##type, #type, sizeof(type), NULL))); \ + return cbdataInternalAllocDbg(CBDATA_##type,__FILE__,__LINE__); \ + } \ + void operator delete (void *address) { \ + if (address) cbdataInternalFreeDbg(address,__FILE__,__LINE__); \ + } \ + private: #else #define cbdataAlloc(type) ((type *)cbdataInternalAlloc(CBDATA_##type)) #define cbdataFree(var) do {if (var) {cbdataInternalFree(var); var = NULL;}} while(0) #define cbdataReferenceValidDone(var, ptr) cbdataInternalReferenceDoneValid((void **)&(var), (ptr)) +#define CBDATA_CLASS2(type) \ + static cbdata_type CBDATA_##type; \ + public: \ + void *operator new(size_t size) { \ + assert(size == sizeof(type)); \ + (CBDATA_##type ? CBDATA_UNKNOWN : (CBDATA_##type = cbdataInternalAddType(CBDATA_##type, #type, sizeof(type), NULL))); \ + return (type *)cbdataInternalAlloc(CBDATA_##type); \ + } \ + void operator delete (void *address) { \ + if (address) cbdataInternalFree(address);\ + } \ + private: #endif #define cbdataReference(var) (cbdataInternalLock(var), var) #define cbdataReferenceDone(var) do {if (var) {cbdataInternalUnlock(var); var = NULL;}} while(0) diff --git a/src/disk.cc b/src/disk.cc index 19c170f909..af51919297 100644 --- a/src/disk.cc +++ b/src/disk.cc @@ -1,6 +1,6 @@ /* - * $Id: disk.cc,v 1.167 2004/12/20 14:52:27 robertc Exp $ + * $Id: disk.cc,v 1.168 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 6 Disk I/O Routines * AUTHOR: Harvest Derived diff --git a/src/dns_internal.cc b/src/dns_internal.cc index e7c3140b0e..2d5ac0b115 100644 --- a/src/dns_internal.cc +++ b/src/dns_internal.cc @@ -1,6 +1,6 @@ /* - * $Id: dns_internal.cc,v 1.65 2004/12/20 14:52:27 robertc Exp $ + * $Id: dns_internal.cc,v 1.66 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 78 DNS lookups; interacts with lib/rfc1035.c * AUTHOR: Duane Wessels diff --git a/src/dnsserver.cc b/src/dnsserver.cc index 670118031c..3fdf6c7633 100644 --- a/src/dnsserver.cc +++ b/src/dnsserver.cc @@ -1,6 +1,6 @@ /* - * $Id: dnsserver.cc,v 1.66 2004/12/20 14:52:27 robertc Exp $ + * $Id: dnsserver.cc,v 1.67 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 0 DNS Resolver * AUTHOR: Harvest Derived diff --git a/src/enums.h b/src/enums.h index a89b4963a1..0ae29b85ee 100644 --- a/src/enums.h +++ b/src/enums.h @@ -1,6 +1,6 @@ /* - * $Id: enums.h,v 1.237 2004/12/20 14:52:27 robertc Exp $ + * $Id: enums.h,v 1.238 2004/12/20 16:30:35 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ diff --git a/src/errorpage.cc b/src/errorpage.cc index 178870399a..e1ab1072a4 100644 --- a/src/errorpage.cc +++ b/src/errorpage.cc @@ -1,6 +1,6 @@ /* - * $Id: errorpage.cc,v 1.198 2004/12/20 14:52:27 robertc Exp $ + * $Id: errorpage.cc,v 1.199 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 4 Error Generation * AUTHOR: Duane Wessels diff --git a/src/forward.cc b/src/forward.cc index fbd74b94cf..7b8063dbff 100644 --- a/src/forward.cc +++ b/src/forward.cc @@ -1,6 +1,6 @@ /* - * $Id: forward.cc,v 1.123 2004/12/20 14:52:27 robertc Exp $ + * $Id: forward.cc,v 1.124 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 17 Request Forwarding * AUTHOR: Duane Wessels diff --git a/src/fqdncache.cc b/src/fqdncache.cc index 39bb60e402..20d8377458 100644 --- a/src/fqdncache.cc +++ b/src/fqdncache.cc @@ -1,6 +1,6 @@ /* - * $Id: fqdncache.cc,v 1.161 2004/12/20 14:52:27 robertc Exp $ + * $Id: fqdncache.cc,v 1.162 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 35 FQDN Cache * AUTHOR: Harvest Derived diff --git a/src/fs/Makefile.am b/src/fs/Makefile.am index 784cb374e9..3b13d5bbfc 100644 --- a/src/fs/Makefile.am +++ b/src/fs/Makefile.am @@ -1,39 +1,32 @@ # Makefile for storage modules in the Squid Object Cache server # -# $Id: Makefile.am,v 1.10 2003/08/14 23:21:52 robertc Exp $ +# $Id: Makefile.am,v 1.11 2004/12/20 16:30:40 robertc Exp $ # AUTOMAKE_OPTIONS = subdir-objects AM_CFLAGS = @SQUID_CFLAGS@ AM_CXXFLAGS = @SQUID_CXXFLAGS@ -DIST_SUBDIRS = aufs coss diskd null ufs +DIST_SUBDIRS = coss null ufs -EXTRA_LIBRARIES = libaufs.a libcoss.a libdiskd.a libnull.a libufs.a +EXTRA_LIBRARIES = libcoss.a libnull.a libufs.a noinst_LIBRARIES = @STORE_LIBS@ -libaufs_a_SOURCES = aufs/aiops.cc aufs/async_io.cc aufs/store_asyncufs.h \ - aufs/store_io_aufs.cc -libcoss_a_SOURCES = coss/store_coss.h coss/store_io_coss.cc coss/store_dir_coss.cc \ - coss/async_io.cc coss/async_io.h -libdiskd_a_SOURCES = diskd/diskd.cc \ - diskd/store_diskd.h \ - diskd/store_io_diskd.cc - -EXTRA_PROGRAMS = diskd/diskd -libexec_PROGRAMS = @STORE_PROGRAMS@ -diskd_diskd_SOURCES = diskd/diskd.cc diskd/dio.h -diskd_diskd_LDADD = $(top_builddir)/lib/libmiscutil.a @XTRA_LIBS@ - -diskd_diskd_INCLUDES = -Idiskd -I$(top_builddir)/include \ - -I$(top_srcdir)/include -I$(top_srcdir)/src/ +libcoss_a_SOURCES = \ + coss/store_coss.h \ + coss/store_io_coss.cc \ + coss/store_dir_coss.cc \ + coss/CossSwapDir.h libnull_a_SOURCES = null/store_null.cc null/store_null.h -libufs_a_SOURCES = ufs/store_dir_ufs.cc ufs/store_io_ufs.cc ufs/store_ufs.h +libufs_a_SOURCES = ufs/store_dir_ufs.cc ufs/store_io_ufs.cc \ + ufs/ufscommon.cci \ + ufs/ufscommon.cc \ + ufs/ufscommon.h + EXTRA_DIST = \ coss/coss-notes.txt \ - ufs/IOModule.h \ coss/StoreFScoss.h \ ufs/StoreFSufs.h \ null/StoreFSnull.h @@ -42,12 +35,8 @@ INCLUDES = -I. -I$(top_builddir)/include -I$(top_srcdir)/include \ -I$(top_srcdir)/src/ ## targets below to emulate distributed makefiles -aufs/all: libaufs.a -aufs/clean: clean coss/all: libcoss.a coss/clean: clean -diskd/all: libdiskd.a -diskd/clean: clean null/all: libnull.a null/clean: clean ufs/all: libufs.a diff --git a/src/fs/Makefile.in b/src/fs/Makefile.in index 8cd0a570a3..9a36b996fd 100644 --- a/src/fs/Makefile.in +++ b/src/fs/Makefile.in @@ -16,11 +16,10 @@ # Makefile for storage modules in the Squid Object Cache server # -# $Id: Makefile.in,v 1.50 2004/12/20 14:52:30 robertc Exp $ +# $Id: Makefile.in,v 1.51 2004/12/20 16:30:40 robertc Exp $ # - -SOURCES = $(libaufs_a_SOURCES) $(libcoss_a_SOURCES) $(libdiskd_a_SOURCES) $(libnull_a_SOURCES) $(libufs_a_SOURCES) $(diskd_diskd_SOURCES) +SOURCES = $(libcoss_a_SOURCES) $(libnull_a_SOURCES) $(libufs_a_SOURCES) srcdir = @srcdir@ top_srcdir = @top_srcdir@ @@ -44,8 +43,6 @@ PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ -EXTRA_PROGRAMS = diskd/diskd$(EXEEXT) -libexec_PROGRAMS = @STORE_PROGRAMS@ subdir = src/fs SUBDIRS = DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in @@ -59,22 +56,12 @@ CONFIG_HEADER = $(top_builddir)/include/autoconf.h CONFIG_CLEAN_FILES = LIBRARIES = $(noinst_LIBRARIES) ARFLAGS = cru -libaufs_a_AR = $(AR) $(ARFLAGS) -libaufs_a_LIBADD = -am__dirstamp = $(am__leading_dot)dirstamp -am_libaufs_a_OBJECTS = aufs/aiops.$(OBJEXT) aufs/async_io.$(OBJEXT) \ - aufs/store_io_aufs.$(OBJEXT) -libaufs_a_OBJECTS = $(am_libaufs_a_OBJECTS) libcoss_a_AR = $(AR) $(ARFLAGS) libcoss_a_LIBADD = +am__dirstamp = $(am__leading_dot)dirstamp am_libcoss_a_OBJECTS = coss/store_io_coss.$(OBJEXT) \ - coss/store_dir_coss.$(OBJEXT) coss/async_io.$(OBJEXT) + coss/store_dir_coss.$(OBJEXT) libcoss_a_OBJECTS = $(am_libcoss_a_OBJECTS) -libdiskd_a_AR = $(AR) $(ARFLAGS) -libdiskd_a_LIBADD = -am_libdiskd_a_OBJECTS = diskd/diskd.$(OBJEXT) \ - diskd/store_io_diskd.$(OBJEXT) -libdiskd_a_OBJECTS = $(am_libdiskd_a_OBJECTS) libnull_a_AR = $(AR) $(ARFLAGS) libnull_a_LIBADD = am_libnull_a_OBJECTS = null/store_null.$(OBJEXT) @@ -82,14 +69,8 @@ libnull_a_OBJECTS = $(am_libnull_a_OBJECTS) libufs_a_AR = $(AR) $(ARFLAGS) libufs_a_LIBADD = am_libufs_a_OBJECTS = ufs/store_dir_ufs.$(OBJEXT) \ - ufs/store_io_ufs.$(OBJEXT) + ufs/store_io_ufs.$(OBJEXT) ufs/ufscommon.$(OBJEXT) libufs_a_OBJECTS = $(am_libufs_a_OBJECTS) -am__installdirs = "$(DESTDIR)$(libexecdir)" -libexecPROGRAMS_INSTALL = $(INSTALL_PROGRAM) -PROGRAMS = $(libexec_PROGRAMS) -am_diskd_diskd_OBJECTS = diskd/diskd.$(OBJEXT) -diskd_diskd_OBJECTS = $(am_diskd_diskd_OBJECTS) -diskd_diskd_DEPENDENCIES = $(top_builddir)/lib/libmiscutil.a DEFAULT_INCLUDES = -I. -I$(srcdir) -I$(top_builddir)/include depcomp = $(SHELL) $(top_srcdir)/cfgaux/depcomp am__depfiles_maybe = depfiles @@ -109,12 +90,10 @@ LTCOMPILE = $(LIBTOOL) --tag=CC --mode=compile $(CC) $(DEFS) \ CCLD = $(CC) LINK = $(LIBTOOL) --tag=CC --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ $(AM_LDFLAGS) $(LDFLAGS) -o $@ -SOURCES = $(libaufs_a_SOURCES) $(libcoss_a_SOURCES) \ - $(libdiskd_a_SOURCES) $(libnull_a_SOURCES) $(libufs_a_SOURCES) \ - $(diskd_diskd_SOURCES) -DIST_SOURCES = $(libaufs_a_SOURCES) $(libcoss_a_SOURCES) \ - $(libdiskd_a_SOURCES) $(libnull_a_SOURCES) $(libufs_a_SOURCES) \ - $(diskd_diskd_SOURCES) +SOURCES = $(libcoss_a_SOURCES) $(libnull_a_SOURCES) \ + $(libufs_a_SOURCES) +DIST_SOURCES = $(libcoss_a_SOURCES) $(libnull_a_SOURCES) \ + $(libufs_a_SOURCES) RECURSIVE_TARGETS = all-recursive check-recursive dvi-recursive \ html-recursive info-recursive install-data-recursive \ install-exec-recursive install-info-recursive \ @@ -155,6 +134,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -242,9 +224,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ @@ -308,29 +288,23 @@ target_alias = @target_alias@ AUTOMAKE_OPTIONS = subdir-objects AM_CFLAGS = @SQUID_CFLAGS@ AM_CXXFLAGS = @SQUID_CXXFLAGS@ -DIST_SUBDIRS = aufs coss diskd null ufs -EXTRA_LIBRARIES = libaufs.a libcoss.a libdiskd.a libnull.a libufs.a +DIST_SUBDIRS = coss null ufs +EXTRA_LIBRARIES = libcoss.a libnull.a libufs.a noinst_LIBRARIES = @STORE_LIBS@ -libaufs_a_SOURCES = aufs/aiops.cc aufs/async_io.cc aufs/store_asyncufs.h \ - aufs/store_io_aufs.cc - -libcoss_a_SOURCES = coss/store_coss.h coss/store_io_coss.cc coss/store_dir_coss.cc \ - coss/async_io.cc coss/async_io.h - -libdiskd_a_SOURCES = diskd/diskd.cc \ - diskd/store_diskd.h \ - diskd/store_io_diskd.cc - -diskd_diskd_SOURCES = diskd/diskd.cc diskd/dio.h -diskd_diskd_LDADD = $(top_builddir)/lib/libmiscutil.a @XTRA_LIBS@ -diskd_diskd_INCLUDES = -Idiskd -I$(top_builddir)/include \ - -I$(top_srcdir)/include -I$(top_srcdir)/src/ +libcoss_a_SOURCES = \ + coss/store_coss.h \ + coss/store_io_coss.cc \ + coss/store_dir_coss.cc \ + coss/CossSwapDir.h libnull_a_SOURCES = null/store_null.cc null/store_null.h -libufs_a_SOURCES = ufs/store_dir_ufs.cc ufs/store_io_ufs.cc ufs/store_ufs.h +libufs_a_SOURCES = ufs/store_dir_ufs.cc ufs/store_io_ufs.cc \ + ufs/ufscommon.cci \ + ufs/ufscommon.cc \ + ufs/ufscommon.h + EXTRA_DIST = \ coss/coss-notes.txt \ - ufs/IOModule.h \ coss/StoreFScoss.h \ ufs/StoreFSufs.h \ null/StoreFSnull.h @@ -374,22 +348,6 @@ $(ACLOCAL_M4): @MAINTAINER_MODE_TRUE@ $(am__aclocal_m4_deps) clean-noinstLIBRARIES: -test -z "$(noinst_LIBRARIES)" || rm -f $(noinst_LIBRARIES) -aufs/$(am__dirstamp): - @$(mkdir_p) aufs - @: > aufs/$(am__dirstamp) -aufs/$(DEPDIR)/$(am__dirstamp): - @$(mkdir_p) aufs/$(DEPDIR) - @: > aufs/$(DEPDIR)/$(am__dirstamp) -aufs/aiops.$(OBJEXT): aufs/$(am__dirstamp) \ - aufs/$(DEPDIR)/$(am__dirstamp) -aufs/async_io.$(OBJEXT): aufs/$(am__dirstamp) \ - aufs/$(DEPDIR)/$(am__dirstamp) -aufs/store_io_aufs.$(OBJEXT): aufs/$(am__dirstamp) \ - aufs/$(DEPDIR)/$(am__dirstamp) -libaufs.a: $(libaufs_a_OBJECTS) $(libaufs_a_DEPENDENCIES) - -rm -f libaufs.a - $(libaufs_a_AR) libaufs.a $(libaufs_a_OBJECTS) $(libaufs_a_LIBADD) - $(RANLIB) libaufs.a coss/$(am__dirstamp): @$(mkdir_p) coss @: > coss/$(am__dirstamp) @@ -400,26 +358,10 @@ coss/store_io_coss.$(OBJEXT): coss/$(am__dirstamp) \ coss/$(DEPDIR)/$(am__dirstamp) coss/store_dir_coss.$(OBJEXT): coss/$(am__dirstamp) \ coss/$(DEPDIR)/$(am__dirstamp) -coss/async_io.$(OBJEXT): coss/$(am__dirstamp) \ - coss/$(DEPDIR)/$(am__dirstamp) libcoss.a: $(libcoss_a_OBJECTS) $(libcoss_a_DEPENDENCIES) -rm -f libcoss.a $(libcoss_a_AR) libcoss.a $(libcoss_a_OBJECTS) $(libcoss_a_LIBADD) $(RANLIB) libcoss.a -diskd/$(am__dirstamp): - @$(mkdir_p) diskd - @: > diskd/$(am__dirstamp) -diskd/$(DEPDIR)/$(am__dirstamp): - @$(mkdir_p) diskd/$(DEPDIR) - @: > diskd/$(DEPDIR)/$(am__dirstamp) -diskd/diskd.$(OBJEXT): diskd/$(am__dirstamp) \ - diskd/$(DEPDIR)/$(am__dirstamp) -diskd/store_io_diskd.$(OBJEXT): diskd/$(am__dirstamp) \ - diskd/$(DEPDIR)/$(am__dirstamp) -libdiskd.a: $(libdiskd_a_OBJECTS) $(libdiskd_a_DEPENDENCIES) - -rm -f libdiskd.a - $(libdiskd_a_AR) libdiskd.a $(libdiskd_a_OBJECTS) $(libdiskd_a_LIBADD) - $(RANLIB) libdiskd.a null/$(am__dirstamp): @$(mkdir_p) null @: > null/$(am__dirstamp) @@ -442,70 +384,31 @@ ufs/store_dir_ufs.$(OBJEXT): ufs/$(am__dirstamp) \ ufs/$(DEPDIR)/$(am__dirstamp) ufs/store_io_ufs.$(OBJEXT): ufs/$(am__dirstamp) \ ufs/$(DEPDIR)/$(am__dirstamp) +ufs/ufscommon.$(OBJEXT): ufs/$(am__dirstamp) \ + ufs/$(DEPDIR)/$(am__dirstamp) libufs.a: $(libufs_a_OBJECTS) $(libufs_a_DEPENDENCIES) -rm -f libufs.a $(libufs_a_AR) libufs.a $(libufs_a_OBJECTS) $(libufs_a_LIBADD) $(RANLIB) libufs.a -install-libexecPROGRAMS: $(libexec_PROGRAMS) - @$(NORMAL_INSTALL) - test -z "$(libexecdir)" || $(mkdir_p) "$(DESTDIR)$(libexecdir)" - @list='$(libexec_PROGRAMS)'; for p in $$list; do \ - p1=`echo $$p|sed 's/$(EXEEXT)$$//'`; \ - if test -f $$p \ - || test -f $$p1 \ - ; then \ - f=`echo "$$p1" | sed 's,^.*/,,;$(transform);s/$$/$(EXEEXT)/'`; \ - echo " $(INSTALL_PROGRAM_ENV) $(LIBTOOL) --mode=install $(libexecPROGRAMS_INSTALL) '$$p' '$(DESTDIR)$(libexecdir)/$$f'"; \ - $(INSTALL_PROGRAM_ENV) $(LIBTOOL) --mode=install $(libexecPROGRAMS_INSTALL) "$$p" "$(DESTDIR)$(libexecdir)/$$f" || exit 1; \ - else :; fi; \ - done - -uninstall-libexecPROGRAMS: - @$(NORMAL_UNINSTALL) - @list='$(libexec_PROGRAMS)'; for p in $$list; do \ - f=`echo "$$p" | sed 's,^.*/,,;s/$(EXEEXT)$$//;$(transform);s/$$/$(EXEEXT)/'`; \ - echo " rm -f '$(DESTDIR)$(libexecdir)/$$f'"; \ - rm -f "$(DESTDIR)$(libexecdir)/$$f"; \ - done - -clean-libexecPROGRAMS: - @list='$(libexec_PROGRAMS)'; for p in $$list; do \ - f=`echo $$p|sed 's/$(EXEEXT)$$//'`; \ - echo " rm -f $$p $$f"; \ - rm -f $$p $$f ; \ - done -diskd/diskd$(EXEEXT): $(diskd_diskd_OBJECTS) $(diskd_diskd_DEPENDENCIES) diskd/$(am__dirstamp) - @rm -f diskd/diskd$(EXEEXT) - $(CXXLINK) $(diskd_diskd_LDFLAGS) $(diskd_diskd_OBJECTS) $(diskd_diskd_LDADD) $(LIBS) mostlyclean-compile: -rm -f *.$(OBJEXT) - -rm -f aufs/aiops.$(OBJEXT) - -rm -f aufs/async_io.$(OBJEXT) - -rm -f aufs/store_io_aufs.$(OBJEXT) - -rm -f coss/async_io.$(OBJEXT) -rm -f coss/store_dir_coss.$(OBJEXT) -rm -f coss/store_io_coss.$(OBJEXT) - -rm -f diskd/diskd.$(OBJEXT) - -rm -f diskd/store_io_diskd.$(OBJEXT) -rm -f null/store_null.$(OBJEXT) -rm -f ufs/store_dir_ufs.$(OBJEXT) -rm -f ufs/store_io_ufs.$(OBJEXT) + -rm -f ufs/ufscommon.$(OBJEXT) distclean-compile: -rm -f *.tab.c -@AMDEP_TRUE@@am__include@ @am__quote@aufs/$(DEPDIR)/aiops.Po@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@aufs/$(DEPDIR)/async_io.Po@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@aufs/$(DEPDIR)/store_io_aufs.Po@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@coss/$(DEPDIR)/async_io.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@coss/$(DEPDIR)/store_dir_coss.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@coss/$(DEPDIR)/store_io_coss.Po@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@diskd/$(DEPDIR)/diskd.Po@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@diskd/$(DEPDIR)/store_io_diskd.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@null/$(DEPDIR)/store_null.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@ufs/$(DEPDIR)/store_dir_ufs.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@ufs/$(DEPDIR)/store_io_ufs.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@ufs/$(DEPDIR)/ufscommon.Po@am__quote@ .cc.o: @am__fastdepCXX_TRUE@ depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`; \ @@ -706,12 +609,9 @@ distdir: $(DISTFILES) done check-am: all-am check: check-recursive -all-am: Makefile $(LIBRARIES) $(PROGRAMS) +all-am: Makefile $(LIBRARIES) installdirs: installdirs-recursive installdirs-am: - for dir in "$(DESTDIR)$(libexecdir)"; do \ - test -z "$$dir" || $(mkdir_p) "$$dir"; \ - done install: install-recursive install-exec: install-exec-recursive install-data: install-data-recursive @@ -732,12 +632,8 @@ clean-generic: distclean-generic: -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) - -rm -f aufs/$(DEPDIR)/$(am__dirstamp) - -rm -f aufs/$(am__dirstamp) -rm -f coss/$(DEPDIR)/$(am__dirstamp) -rm -f coss/$(am__dirstamp) - -rm -f diskd/$(DEPDIR)/$(am__dirstamp) - -rm -f diskd/$(am__dirstamp) -rm -f null/$(DEPDIR)/$(am__dirstamp) -rm -f null/$(am__dirstamp) -rm -f ufs/$(DEPDIR)/$(am__dirstamp) @@ -748,11 +644,11 @@ maintainer-clean-generic: @echo "it deletes files that may require special tools to rebuild." clean: clean-recursive -clean-am: clean-generic clean-libexecPROGRAMS clean-libtool \ - clean-noinstLIBRARIES mostlyclean-am +clean-am: clean-generic clean-libtool clean-noinstLIBRARIES \ + mostlyclean-am distclean: distclean-recursive - -rm -rf aufs/$(DEPDIR) coss/$(DEPDIR) diskd/$(DEPDIR) null/$(DEPDIR) ufs/$(DEPDIR) + -rm -rf coss/$(DEPDIR) null/$(DEPDIR) ufs/$(DEPDIR) -rm -f Makefile distclean-am: clean-am distclean-compile distclean-generic \ distclean-libtool distclean-tags @@ -769,7 +665,7 @@ info-am: install-data-am: -install-exec-am: install-libexecPROGRAMS +install-exec-am: install-info: install-info-recursive @@ -778,7 +674,7 @@ install-man: installcheck-am: maintainer-clean: maintainer-clean-recursive - -rm -rf aufs/$(DEPDIR) coss/$(DEPDIR) diskd/$(DEPDIR) null/$(DEPDIR) ufs/$(DEPDIR) + -rm -rf coss/$(DEPDIR) null/$(DEPDIR) ufs/$(DEPDIR) -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic @@ -795,33 +691,27 @@ ps: ps-recursive ps-am: -uninstall-am: uninstall-info-am uninstall-libexecPROGRAMS +uninstall-am: uninstall-info-am uninstall-info: uninstall-info-recursive .PHONY: $(RECURSIVE_TARGETS) CTAGS GTAGS all all-am check check-am \ - clean clean-generic clean-libexecPROGRAMS clean-libtool \ - clean-noinstLIBRARIES clean-recursive ctags ctags-recursive \ - distclean distclean-compile distclean-generic \ - distclean-libtool distclean-recursive distclean-tags distdir \ - dvi dvi-am html html-am info info-am install install-am \ - install-data install-data-am install-exec install-exec-am \ - install-info install-info-am install-libexecPROGRAMS \ - install-man install-strip installcheck installcheck-am \ - installdirs installdirs-am maintainer-clean \ + clean clean-generic clean-libtool clean-noinstLIBRARIES \ + clean-recursive ctags ctags-recursive distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-recursive distclean-tags distdir dvi dvi-am html \ + html-am info info-am install install-am install-data \ + install-data-am install-exec install-exec-am install-info \ + install-info-am install-man install-strip installcheck \ + installcheck-am installdirs installdirs-am maintainer-clean \ maintainer-clean-generic maintainer-clean-recursive \ mostlyclean mostlyclean-compile mostlyclean-generic \ mostlyclean-libtool mostlyclean-recursive pdf pdf-am ps ps-am \ - tags tags-recursive uninstall uninstall-am uninstall-info-am \ - uninstall-libexecPROGRAMS + tags tags-recursive uninstall uninstall-am uninstall-info-am -aufs/all: libaufs.a -aufs/clean: clean coss/all: libcoss.a coss/clean: clean -diskd/all: libdiskd.a -diskd/clean: clean null/all: libnull.a null/clean: clean ufs/all: libufs.a diff --git a/src/fs/aufs/StoreFSaufs.cc b/src/fs/aufs/StoreFSaufs.cc index 577e379749..5d30f414ea 100644 --- a/src/fs/aufs/StoreFSaufs.cc +++ b/src/fs/aufs/StoreFSaufs.cc @@ -1,6 +1,6 @@ /* - * $Id: StoreFSaufs.cc,v 1.1 2003/07/22 15:23:10 robertc Exp $ + * $Id: StoreFSaufs.cc,v 1.2 2004/12/20 16:30:40 robertc Exp $ * * DEBUG: section 47 Store Directory Routines * AUTHOR: Robert Collins @@ -37,6 +37,6 @@ #include "squid.h" #include "StoreFileSystem.h" #include "fs/ufs/StoreFSufs.h" -#include "fs/aufs/store_asyncufs.h" +#include "DiskIO/DiskIOModule.h" -static StoreFSufs AufsInstance(AufsIOModule::GetInstance(), "aufs"); +static StoreFSufs AufsInstance("DiskThreads", "aufs"); diff --git a/src/fs/aufs/store_io_aufs.cc b/src/fs/aufs/store_io_aufs.cc deleted file mode 100644 index 9ced197ee5..0000000000 --- a/src/fs/aufs/store_io_aufs.cc +++ /dev/null @@ -1,430 +0,0 @@ - -/* - * DEBUG 79 - */ - -#include "squid.h" -#include "store_asyncufs.h" -#include "Store.h" -#include "ufscommon.h" -#include "SwapDir.h" - -/* === PUBLIC =========================================================== */ - - - - -AufsIO AufsIO::Instance; -bool -AufsIO::shedLoad() -{ - /* - * we should detect some 'too many files open' condition and return - * NULL here. - */ -#ifdef MAGIC2 - - if (aioQueueSize() > MAGIC2) - return true; - -#endif - - return false; -} - -int -AufsIO::load() -{ - int loadav; - int ql; - - ql = aioQueueSize(); - - if (ql == 0) - loadav = 0; - - loadav = ql * 1000 / MAGIC1; - - debug(47, 9) ("storeAufsDirCheckObj: load=%d\n", loadav); - - return loadav; -} - -StoreIOState::Pointer -AufsIO::createState(SwapDir *SD, StoreEntry *e, STIOCB * callback, void *callback_data) const -{ - return new UFSStoreState (SD, e, callback, callback_data); -} - -DiskFile::Pointer -AufsIO::newFile (char const *path) -{ - return new AUFSFile (path, this); -} - -void -AufsIO::unlinkFile(char const *path) -{ - statCounter.syscalls.disk.unlinks++; -#if USE_TRUNCATE_NOT_UNLINK - - aioTruncate(path, NULL, NULL); -#else - - aioUnlink(path, NULL, NULL); -#endif -} - -AufsIOModule *AufsIOModule::Instance = NULL; -AufsIOModule & -AufsIOModule::GetInstance() -{ - if (!Instance) - Instance = new AufsIOModule; - - return *Instance; -} - -void -AufsIOModule::init() -{ - AufsIO::Instance.init(); -} - -void -AufsIOModule::shutdown() -{ - AufsIO::Instance.done(); -} - -UFSStrategy * -AufsIOModule::createSwapDirIOStrategy() -{ - return new InstanceToSingletonAdapter(&AufsIO::Instance); -} - -CBDATA_CLASS_INIT(AUFSFile); -void * -AUFSFile::operator new (size_t) -{ - CBDATA_INIT_TYPE(AUFSFile); - AUFSFile *result = cbdataAlloc(AUFSFile); - /* Mark result as being owned - we want the refcounter to do the delete - * call */ - return cbdataReference(result); -} - -void -AUFSFile::operator delete (void *address) -{ - AUFSFile *t = static_cast(address); - cbdataFree(address); - /* And allow the memory to be freed */ - cbdataReferenceDone (t); -} - -AUFSFile::AUFSFile (char const *aPath, AufsIO *anIO):fd(-1), errorOccured (false), IO(anIO), - inProgressIOs (0) -{ - assert (aPath); - debug (79,3)("UFSFile::UFSFile: %s\n", aPath); - path_ = xstrdup (aPath); -} - -AUFSFile::~AUFSFile() -{ - safe_free (path_); - doClose(); -} - -void -AUFSFile::open (int flags, mode_t mode, IORequestor::Pointer callback) -{ - statCounter.syscalls.disk.opens++; -#if !ASYNC_OPEN - - fd = file_open(path_, flags); - - if (fd < 0) { - debug(79, 3) ("AUFSFile::open: got failure (%d)\n", errno); - errorOccured = true; - return; - } - -#endif - Opening_FD++; - - ioRequestor = callback; - - ++inProgressIOs; - -#if ASYNC_OPEN - - aioOpen(path_, flags, mode, AUFSFile::OpenDone, this); - -#else - - openDone(fd, NULL, fd, 0); - -#endif -} - -void -AUFSFile::read(char *buf, off_t offset, size_t size) -{ - debugs(79, 3, "AUFSFile::read: " << this << ", size " << size); - assert (fd > -1); - assert (ioRequestor.getRaw()); - statCounter.syscalls.disk.reads++; - ++inProgressIOs; -#if ASYNC_READ - - aioRead(fd, offset, size, ReadDone, this); -#else - - file_read(fd, buf, size, offset, ReadDone, this); -#endif -} - -void -AUFSFile::create (int flags, mode_t mode, IORequestor::Pointer callback) -{ - statCounter.syscalls.disk.opens++; -#if !ASYNC_CREATE - - int fd = file_open(path_, flags); - - if (fd < 0) { - debug(79, 3) ("storeAufsCreate: got failure (%d)\n", errno); - errorOccured = true; - return; - } - -#endif - Opening_FD++; - - ioRequestor = callback; - - ++inProgressIOs; - -#if ASYNC_CREATE - - aioOpen(path_, flags, mode, AUFSFile::OpenDone, this); - -#else - - openDone (fd, NULL, fd, 0); - -#endif -} - -bool -AUFSFile::error() const -{ - return errorOccured; -} - -void -AUFSFile::OpenDone(int fd, void *cbdata, const char *buf, int aio_return, int aio_errno) -{ - AUFSFile *myFile = static_cast(cbdata); - myFile->openDone (fd, buf, aio_return, aio_errno); -} - -void -AUFSFile::openDone(int unused, const char *unused2, int anFD, int errflag) -{ - debug(79, 3) ("AUFSFile::openDone: FD %d, errflag %d\n", anFD, errflag); - Opening_FD--; - - fd = anFD; - - if (errflag || fd < 0) { - errno = errflag; - debug(79, 0) ("AUFSFile::openDone: %s\n", xstrerror()); - debug(79, 1) ("\t%s\n", path_); - errorOccured = true; - } else { - store_open_disk_fd++; - commSetCloseOnExec(fd); - fd_open(fd, FD_FILE, path_); - } - - debug(79, 3) ("AUFSFile::openDone: exiting\n"); - - IORequestor::Pointer t = ioRequestor; - --inProgressIOs; - t->ioCompletedNotification(); -} - -void AUFSFile::doClose() -{ - if (fd > -1) { - statCounter.syscalls.disk.closes++; - aioClose(fd); - fd_close(fd); - store_open_disk_fd--; - fd = -1; - } -} - -/* open for reading */ -StoreIOState::Pointer -storeAufsOpen(SwapDir * SD, StoreEntry * e, STFNCB * file_callback, - STIOCB * callback, void *callback_data) -{ - UFSStrategy *IO = dynamic_cast (((UFSSwapDir *)SD)->IO); - assert (IO); - return IO->open (SD, e, file_callback, callback, callback_data); -} - -/* open for creating */ -StoreIOState::Pointer -storeAufsCreate(SwapDir * SD, StoreEntry * e, STFNCB * file_callback, STIOCB * callback, void *callback_data) -{ - UFSStrategy *IO = dynamic_cast (((UFSSwapDir *)SD)->IO); - assert (IO); - return IO->create (SD, e, file_callback, callback, callback_data); -} - - -void -AUFSFile::close () -{ - debug (79,3)("AUFSFile::close: %p closing for %p\n", this, ioRequestor.getRaw()); - - if (!ioInProgress()) { - doClose(); - assert (ioRequestor.getRaw()); - ioRequestor->closeCompleted(); - } -} - -bool -AUFSFile::canRead() const -{ - debug (79,3)("AUFSFile::canRead: fd is %d\n",fd); - return fd > -1; -} - -void -AUFSFile::write(char const *buf, size_t size, off_t offset, FREE *free_func) -{ - debug(79, 3) ("storeAufsWrite: FD %d\n", fd); - statCounter.syscalls.disk.writes++; - ++inProgressIOs; -#if ASYNC_WRITE - - aioWrite(fd, offset, (char *)buf, size, WriteDone, this, - free_func); -#else - - file_write(fd, offset, (char *)buf, size, WriteDone, this, - free_func); -#endif -} - -bool -AUFSFile::canWrite() const -{ - return fd > -1; -} - -bool -AUFSFile::ioInProgress()const -{ - return inProgressIOs > 0; -} - -/* === STATIC =========================================================== */ - -#if ASYNC_READ -void -AUFSFile::ReadDone(int fd, void *my_data, const char *buf, int len, int errflag) -#else -void -AUFSFile::ReadDone(int fd, const char *buf, int len, int errflag, void *my_data) -#endif -{ - AUFSFile *myFile = static_cast(my_data); - assert (myFile); - myFile->readDone (fd, buf, len, errflag); -} - -void -AUFSFile::readDone(int rvfd, const char *buf, int len, int errflag) -{ - debug (79,3)("AUFSFile::readDone: FD %d\n",rvfd); - assert (fd == rvfd); - - ssize_t rlen; - - if (errflag) { - debug(79, 3) ("AUFSFile::readDone: got failure (%d)\n", errflag); - rlen = -1; - } else { - rlen = (ssize_t) len; - } - -#if ASYNC_READ - /* translate errflag from errno to Squid disk error */ - errno = errflag; - - if (errflag) - errflag = DISK_ERROR; - else - errflag = DISK_OK; - -#else - - if (errflag == DISK_EOF) - errflag = DISK_OK; /* EOF is signalled by len == 0, not errors... */ - -#endif - - --inProgressIOs; - - ioRequestor->readCompleted(buf, rlen, errflag); -} - -void -AUFSFile:: -#if ASYNC_WRITE -WriteDone(int fd, void *my_data, int len, int errflag) -#else -WriteDone(int fd, int errflag, size_t len, void *my_data) -#endif -{ - AUFSFile *aFile = static_cast(my_data); - assert (aFile); - aFile->writeDone (fd, errflag, len); -} - -void -AUFSFile::writeDone (int rvfd, int errflag, size_t len) -{ - assert (rvfd == fd); - static int loop_detect = 0; - debug(79, 3) ("storeAufsWriteDone: FD %d, len %ld, err=%d\n", - fd, (long int) len, errflag); - -#if ASYNC_WRITE - /* Translate from errno to Squid disk error */ - errno = errflag; - - if (errflag) - errflag = errno == ENOSPC ? DISK_NO_SPACE_LEFT : DISK_ERROR; - else - errflag = DISK_OK; - -#endif - - assert(++loop_detect < 10); - - --inProgressIOs; - - ioRequestor->writeCompleted(errflag, len); - - --loop_detect; -} - diff --git a/src/fs/coss/CossSwapDir.h b/src/fs/coss/CossSwapDir.h new file mode 100644 index 0000000000..31935a8c6f --- /dev/null +++ b/src/fs/coss/CossSwapDir.h @@ -0,0 +1,94 @@ +#ifndef __COSSSWAPDIR_H__ +#define __COSSSWAPDIR_H__ + +#include "SwapDir.h" + +#ifndef COSS_MEMBUF_SZ +#define COSS_MEMBUF_SZ 1048576 +#endif + +/* Note that swap_filen in sio/e are actually disk offsets too! */ + +/* What we're doing in storeCossAllocate() */ +#define COSS_ALLOC_ALLOCATE 1 +#define COSS_ALLOC_REALLOC 2 + +class CossSwapDir; + + +class CossMemBuf; + +class DiskIOStrategy; + +class DiskIOModule; + +class ConfigOptionVector; +#include "DiskIO/DiskFile.h" +#include "DiskIO/IORequestor.h" + +class CossSwapDir : public SwapDir, public IORequestor +{ + +public: + CossSwapDir(); + virtual void init(); + virtual void newFileSystem(); + virtual void dump(StoreEntry &)const; + ~CossSwapDir(); + virtual void unlink (StoreEntry &); + virtual void statfs (StoreEntry &)const; + virtual int canStore(StoreEntry const &)const; + virtual int callback(); + virtual void sync(); + virtual StoreIOState::Pointer createStoreIO(StoreEntry &, STFNCB *, STIOCB *, void *); + virtual StoreIOState::Pointer openStoreIO(StoreEntry &, STFNCB *, STIOCB *, void *); + virtual void openLog(); + virtual void closeLog(); + virtual int writeCleanStart(); + virtual void writeCleanDone(); + virtual void logEntry(const StoreEntry & e, int op) const; + virtual void parse (int index, char *path); + virtual void reconfigure (int, char *); + /* internals */ + virtual off_t storeCossFilenoToDiskOffset(sfileno); + virtual sfileno storeCossDiskOffsetToFileno(off_t); + virtual CossMemBuf *storeCossFilenoToMembuf(sfileno f); + /* IORequestor routines */ + virtual void ioCompletedNotification(); + virtual void closeCompleted(); + virtual void readCompleted(const char *buf, int len, int errflag, RefCount); + virtual void writeCompleted(int errflag, size_t len, RefCount); + //private: + int swaplog_fd; + int count; + dlink_list membufs; + + CossMemBuf *current_membuf; + size_t current_offset; /* in Blocks */ + int numcollisions; + dlink_list cossindex; + unsigned int blksz_bits; + unsigned int blksz_mask; /* just 1< theFile; + char *storeCossMemPointerFromDiskOffset(size_t offset, CossMemBuf ** mb); + void storeCossMemBufUnlock(storeIOState * e); + CossMemBuf *createMemBuf(size_t start, sfileno curfn, int *collision); + sfileno allocate(const StoreEntry * e, int which); + void startMembuf(); + +private: + void changeIO(DiskIOModule *module); + bool optionIOParse(char const *option, const char *value, int reconfiguring); + void optionIODump(StoreEntry * e) const; + void CossSwapDir::optionBlockSizeDump(StoreEntry *) const; + bool CossSwapDir::optionBlockSizeParse(const char *, const char *, int); + ConfigOption * getOptionTree() const; + const char *ioModule; + ConfigOptionVector *currentIOOptions; +}; + +extern void storeCossAdd(CossSwapDir *, StoreEntry *); +extern void storeCossRemove(CossSwapDir *, StoreEntry *); +extern void storeCossStartMembuf(CossSwapDir * SD); +#endif diff --git a/src/fs/coss/StoreFScoss.cc b/src/fs/coss/StoreFScoss.cc index c24a67c0b5..d1dfe7579c 100644 --- a/src/fs/coss/StoreFScoss.cc +++ b/src/fs/coss/StoreFScoss.cc @@ -1,6 +1,6 @@ /* - * $Id: StoreFScoss.cc,v 1.3 2004/08/30 05:12:32 robertc Exp $ + * $Id: StoreFScoss.cc,v 1.4 2004/12/20 16:30:41 robertc Exp $ * * DEBUG: section 47 Store Directory Routines * AUTHOR: Robert Collins @@ -35,17 +35,14 @@ */ #include "StoreFileSystem.h" -#include "fs/coss/StoreFScoss.h" -#include "store_coss.h" +#include "StoreFScoss.h" #include "Store.h" - -struct _coss_stats coss_stats; - -static void storeCossStats(StoreEntry *); +#include "CossSwapDir.h" +#include "store_coss.h" StoreFScoss StoreFScoss::_instance; -StoreFileSystem & +StoreFScoss & StoreFScoss::GetInstance() { return _instance; @@ -66,7 +63,7 @@ void StoreFScoss::done() { /* delete coss_index_pool;coss_index_pool = NULL; XXX Should be here? */ - cachemgrRegister("coss", "COSS Stats", storeCossStats, 0, 1); + cachemgrRegister("coss", "COSS Stats", Stats, 0, 1); initialised = false; } @@ -77,7 +74,6 @@ StoreFScoss::createSwapDir() return result; } - void StoreFScoss::setup() { @@ -87,33 +83,45 @@ StoreFScoss::setup() initialised = true; } -static void -storeCossStats(StoreEntry * sentry) +void +StoreFScoss::Stats(StoreEntry * sentry) +{ + GetInstance().stat(sentry); +} + +void +StoreFScoss::stat(StoreEntry *sentry) +{ + stats.stat(sentry); +} + +void +CossStats::stat(StoreEntry *sentry) { const char *tbl_fmt = "%10s %10d %10d %10d\n"; storeAppendPrintf(sentry, "\n OPS SUCCESS FAIL\n"); storeAppendPrintf(sentry, tbl_fmt, - "open", coss_stats.open.ops, coss_stats.open.success, coss_stats.open.fail); + "open", open.ops, open.success, open.fail); storeAppendPrintf(sentry, tbl_fmt, - "create", coss_stats.create.ops, coss_stats.create.success, coss_stats.create.fail); + "create", create.ops, create.success, create.fail); storeAppendPrintf(sentry, tbl_fmt, - "close", coss_stats.close.ops, coss_stats.close.success, coss_stats.close.fail); + "close", close.ops, close.success, close.fail); storeAppendPrintf(sentry, tbl_fmt, - "unlink", coss_stats.unlink.ops, coss_stats.unlink.success, coss_stats.unlink.fail); + "unlink", unlink.ops, unlink.success, unlink.fail); storeAppendPrintf(sentry, tbl_fmt, - "read", coss_stats.read.ops, coss_stats.read.success, coss_stats.read.fail); + "read", read.ops, read.success, read.fail); storeAppendPrintf(sentry, tbl_fmt, - "write", coss_stats.write.ops, coss_stats.write.success, coss_stats.write.fail); + "write", write.ops, write.success, write.fail); storeAppendPrintf(sentry, tbl_fmt, - "s_write", coss_stats.stripe_write.ops, coss_stats.stripe_write.success, coss_stats.stripe_write.fail); + "s_write", stripe_write.ops, stripe_write.success, stripe_write.fail); storeAppendPrintf(sentry, "\n"); - storeAppendPrintf(sentry, "stripes: %d\n", coss_stats.stripes); - storeAppendPrintf(sentry, "alloc.alloc: %d\n", coss_stats.alloc.alloc); - storeAppendPrintf(sentry, "alloc.realloc: %d\n", coss_stats.alloc.realloc); - storeAppendPrintf(sentry, "alloc.collisions: %d\n", coss_stats.alloc.collisions); - storeAppendPrintf(sentry, "disk_overflows: %d\n", coss_stats.disk_overflows); - storeAppendPrintf(sentry, "stripe_overflows: %d\n", coss_stats.stripe_overflows); - storeAppendPrintf(sentry, "open_mem_hits: %d\n", coss_stats.open_mem_hits); - storeAppendPrintf(sentry, "open_mem_misses: %d\n", coss_stats.open_mem_misses); + storeAppendPrintf(sentry, "stripes: %d\n", stripes); + storeAppendPrintf(sentry, "alloc.alloc: %d\n", alloc.alloc); + storeAppendPrintf(sentry, "alloc.realloc: %d\n", alloc.realloc); + storeAppendPrintf(sentry, "alloc.collisions: %d\n", alloc.collisions); + storeAppendPrintf(sentry, "disk_overflows: %d\n", disk_overflows); + storeAppendPrintf(sentry, "stripe_overflows: %d\n", stripe_overflows); + storeAppendPrintf(sentry, "open_mem_hits: %d\n", open_mem_hits); + storeAppendPrintf(sentry, "open_mem_misses: %d\n", open_mem_misses); } diff --git a/src/fs/coss/StoreFScoss.h b/src/fs/coss/StoreFScoss.h index 16cf45cb88..c289c3638b 100644 --- a/src/fs/coss/StoreFScoss.h +++ b/src/fs/coss/StoreFScoss.h @@ -1,6 +1,6 @@ /* - * $Id: StoreFScoss.h,v 1.1 2003/07/22 15:23:10 robertc Exp $ + * $Id: StoreFScoss.h,v 1.2 2004/12/20 16:30:41 robertc Exp $ * * SQUID Web Proxy Cache http://www.squid-cache.org/ * ---------------------------------------------------------- @@ -34,13 +34,44 @@ #ifndef SQUID_STOREFSCOSS_H #define SQUID_STOREFSCOSS_H -#include "squid.h" +#include "StoreFileSystem.h" + +class CossStats +{ + +public: + void stat(StoreEntry * sentry); + int stripes; + + struct + { + int alloc; + int realloc; + int collisions; + } + + alloc; + int disk_overflows; + int stripe_overflows; + int open_mem_hits; + int open_mem_misses; + + struct + { + int ops; + int success; + int fail; + } + + open, create, close, unlink, read, write, stripe_write; +}; class StoreFScoss : public StoreFileSystem { public: - static StoreFileSystem &GetInstance(); + static StoreFScoss &GetInstance(); + static void Stats(StoreEntry * sentry); StoreFScoss(); virtual ~StoreFScoss() {} @@ -51,6 +82,8 @@ public: /* Not implemented */ StoreFScoss (StoreFScoss const &); StoreFScoss &operator=(StoreFScoss const &); + void stat(StoreEntry * sentry); + CossStats stats; private: static StoreFScoss _instance; diff --git a/src/fs/coss/async_io.cc b/src/fs/coss/async_io.cc deleted file mode 100644 index 02d5608a27..0000000000 --- a/src/fs/coss/async_io.cc +++ /dev/null @@ -1,282 +0,0 @@ -/* - * async_io.c - some quick async IO routines for COSS - * - * Adrian Chadd - * - * These routines are simple plugin replacements for the file_* routines - * in disk.c . They back-end into the POSIX AIO routines to provide - * a nice and simple async IO framework for COSS. - * - * AIO is suitable for COSS - the only sync operations that the standard - * supports are read/write, and since COSS works on a single file - * per storedir it should work just fine. - * - * $Id: async_io.cc,v 1.15 2003/08/27 21:19:38 wessels Exp $ - */ - -#include "squid.h" -#include -#include - -#include "async_io.h" - -/* - * For the time being, we kinda don't need to have our own - * open/close. Just read/write (with the queueing), and callback - * with the dequeueing) - */ - - -/* Internal routines */ - -/* - * find a free aio slot. - * Return the index, or -1 if we can't find one. - */ -static int -a_file_findslot(async_queue_t * q) -{ - int i; - - /* Later we should use something a little more .. efficient :) */ - - for (i = 0; i < MAX_ASYNCOP; i++) { - if (q->aq_queue[i].aq_e_state == AQ_ENTRY_FREE) - /* Found! */ - return i; - } - - /* found nothing */ - return -1; -} - - - - -/* Exported routines */ - -void -a_file_read(async_queue_t * q, int fd, void *buf, int req_len, off_t offset, - DRCB * callback, void *data) -{ - int slot; - async_queue_entry_t *qe; - - assert(q->aq_state == AQ_STATE_SETUP); - - /* Find a free slot */ - slot = a_file_findslot(q); - - if (slot < 0) { - debug(79, 1) ("WARNING: out of aiocb slots!\n"); - /* fall back to blocking method */ - file_read(fd, (char *) buf, req_len, offset, callback, data); - return; - } - - /* Mark slot as ours */ - qe = &q->aq_queue[slot]; - - qe->aq_e_state = AQ_ENTRY_USED; - - qe->aq_e_callback.read = callback; - - qe->aq_e_callback_data = cbdataReference(data); - - qe->aq_e_type = AQ_ENTRY_READ; - - qe->aq_e_free = NULL; - - qe->aq_e_buf = buf; - - qe->aq_e_fd = fd; - - qe->aq_e_aiocb.aio_fildes = fd; - - qe->aq_e_aiocb.aio_nbytes = req_len; - - qe->aq_e_aiocb.aio_offset = offset; - - qe->aq_e_aiocb.aio_buf = buf; - - /* Account */ - q->aq_numpending++; - - /* Initiate aio */ - if (aio_read(&qe->aq_e_aiocb) < 0) { - debug(79, 1) ("WARNING: aio_read() returned error: %s\n", xstrerror()); - /* fall back to blocking method */ - file_read(fd, (char *) buf, req_len, offset, callback, data); - } -} - - -void -a_file_write(async_queue_t * q, int fd, off_t offset, void *buf, int len, - DWCB * callback, void *data, FREE * freefunc) -{ - int slot; - async_queue_entry_t *qe; - - assert(q->aq_state == AQ_STATE_SETUP); - - /* Find a free slot */ - slot = a_file_findslot(q); - - if (slot < 0) { - debug(79, 1) ("WARNING: out of aiocb slots!\n"); - /* fall back to blocking method */ - file_write(fd, offset, buf, len, callback, data, freefunc); - return; - } - - /* Mark slot as ours */ - qe = &q->aq_queue[slot]; - - qe->aq_e_state = AQ_ENTRY_USED; - - qe->aq_e_callback.write = callback; - - qe->aq_e_callback_data = cbdataReference(data); - - qe->aq_e_type = AQ_ENTRY_WRITE; - - qe->aq_e_free = freefunc; - - qe->aq_e_buf = buf; - - qe->aq_e_fd = fd; - - qe->aq_e_aiocb.aio_fildes = fd; - - qe->aq_e_aiocb.aio_nbytes = len; - - qe->aq_e_aiocb.aio_offset = offset; - - qe->aq_e_aiocb.aio_buf = buf; - - /* Account */ - q->aq_numpending++; - - /* Initiate aio */ - if (aio_write(&qe->aq_e_aiocb) < 0) { - debug(79, 1) ("WARNING: aio_write() returned error: %s\n", xstrerror()); - /* fall back to blocking method */ - file_write(fd, offset, buf, len, callback, data, freefunc); - } -} - - -/* - * Note: we grab the state and free the state before calling the callback - * because this allows us to cut down the amount of time it'll take - * to find a free slot (since if we call the callback first, we're going - * to probably be allocated the slot _after_ this one..) - * - * I'll make it much more optimal later. - */ -int -a_file_callback(async_queue_t * q) -{ - int i; - int completed = 0; - int retval, reterr; - DRCB *rc; - DWCB *wc; - FREE *freefunc; - void *cbdata; - int callback_valid; - void *buf; - int fd; - async_queue_entry_t *aqe; - async_queue_entry_type_t type; - - assert(q->aq_state == AQ_STATE_SETUP); - - /* Loop through all slots */ - - for (i = 0; i < MAX_ASYNCOP; i++) { - if (q->aq_queue[i].aq_e_state == AQ_ENTRY_USED) { - aqe = &q->aq_queue[i]; - /* Active, get status */ - reterr = aio_error(&aqe->aq_e_aiocb); - - if (reterr < 0) { - fatal("aio_error returned an error!\n"); - } - - if (reterr != EINPROGRESS) { - /* Get the return code */ - retval = aio_return(&aqe->aq_e_aiocb); - - /* Get the callback parameters */ - freefunc = aqe->aq_e_free; - rc = aqe->aq_e_callback.read; - wc = aqe->aq_e_callback.write; - buf = aqe->aq_e_buf; - fd = aqe->aq_e_fd; - type = aqe->aq_e_type; - callback_valid = cbdataReferenceValidDone(aqe->aq_e_callback_data, &cbdata); - - /* Free slot */ - bzero(aqe, sizeof(async_queue_entry_t)); - aqe->aq_e_state = AQ_ENTRY_FREE; - q->aq_numpending--; - - /* Callback */ - - if (callback_valid) { - if (type == AQ_ENTRY_READ) - rc(fd, (char *)buf, retval, reterr, cbdata); - - if (type == AQ_ENTRY_WRITE) - wc(fd, reterr, retval, cbdata); - } - - if (type == AQ_ENTRY_WRITE && freefunc) - freefunc(buf); - } - } - } - - return completed; -} - - -void -a_file_setupqueue(async_queue_t * q) -{ - /* Make sure the queue isn't setup */ - assert(q->aq_state == AQ_STATE_NONE); - - /* Loop through, blanking the queue entries */ - - /* Done */ - q->aq_state = AQ_STATE_SETUP; -} - - -void -a_file_syncqueue(async_queue_t * q) -{ - assert(q->aq_state == AQ_STATE_SETUP); - - /* - * Keep calling callback to complete ops until the queue is empty - * We can't quit when callback returns 0 - some calls may not - * return any completed pending events, but they're still pending! - */ - - while (q->aq_numpending) - a_file_callback(q); -} - - -void -a_file_closequeue(async_queue_t * q) -{ - assert(q->aq_state == AQ_STATE_SETUP); - - a_file_syncqueue(q); - q->aq_state = AQ_STATE_NONE; -} diff --git a/src/fs/coss/store_coss.h b/src/fs/coss/store_coss.h index ea887421c9..2f8c2aaa1c 100644 --- a/src/fs/coss/store_coss.h +++ b/src/fs/coss/store_coss.h @@ -2,7 +2,6 @@ #define __COSS_H__ #include "SwapDir.h" -#include "async_io.h" #ifndef COSS_MEMBUF_SZ #define COSS_MEMBUF_SZ 1048576 @@ -17,36 +16,13 @@ class CossSwapDir; -struct _coss_stats +class CossMemBuf { - int stripes; - struct - { - int alloc; - int realloc; - int collisions; - } - - alloc; - int disk_overflows; - int stripe_overflows; - int open_mem_hits; - int open_mem_misses; - - struct - { - int ops; - int success; - int fail; - } - - open, create, close, unlink, read, write, stripe_write; -}; - - -struct _cossmembuf -{ +public: + void describe(int level, int line); + void maybeWrite(CossSwapDir * SD); + void write(CossSwapDir * SD); dlink_node node; size_t diskstart; /* in blocks */ size_t diskend; /* in blocks */ @@ -105,19 +81,19 @@ unsigned int writing: flags; - struct _cossmembuf *locked_membuf; + CossMemBuf *locked_membuf; size_t st_size; void read_(char *buf, size_t size, off_t offset, STRCB * callback, void *callback_data); void write(char const *buf, size_t size, off_t offset, FREE * free_func); void close(); + void doCallback(int errflag); + void lockMemBuf(); CossSwapDir *SD; }; MEMPROXY_CLASS_INLINE(CossState) -typedef struct _cossmembuf CossMemBuf; - typedef struct _cossindex CossIndexNode; /* Whether the coss system has been setup or not */ @@ -125,55 +101,36 @@ extern int coss_initialised; extern MemAllocatorProxy *coss_membuf_pool; extern MemAllocatorProxy *coss_index_pool; -class CossSwapDir : public SwapDir +#include "DiskIO/ReadRequest.h" + +class CossRead : public ReadRequest { public: - CossSwapDir(); - virtual void init(); - virtual void newFileSystem(); - virtual void dump(StoreEntry &)const; - ~CossSwapDir(); - virtual void unlink (StoreEntry &); - virtual void statfs (StoreEntry &)const; - virtual int canStore(StoreEntry const &)const; - virtual int callback(); - virtual void sync(); - virtual StoreIOState::Pointer createStoreIO(StoreEntry &, STFNCB *, STIOCB *, void *); - virtual StoreIOState::Pointer openStoreIO(StoreEntry &, STFNCB *, STIOCB *, void *); - virtual void openLog(); - virtual void closeLog(); - virtual int writeCleanStart(); - virtual void writeCleanDone(); - virtual void logEntry(const StoreEntry & e, int op) const; - virtual void parse (int index, char *path); - virtual void reconfigure (int, char *); - virtual off_t storeCossFilenoToDiskOffset(sfileno); - virtual sfileno storeCossDiskOffsetToFileno(off_t); - virtual CossMemBuf *storeCossFilenoToMembuf(sfileno f); - virtual SwapDirOption *CossSwapDir::getOptionTree() const; - virtual void CossSwapDir::optionBlockSizeDump(StoreEntry *) const; - virtual bool CossSwapDir::optionBlockSizeParse(const char *, const char *, int); - - - //private: - int fd; - int swaplog_fd; - int count; - dlink_list membufs; - - struct _cossmembuf *current_membuf; - size_t current_offset; /* in Blocks */ - int numcollisions; - dlink_list cossindex; - async_queue_t aq; - unsigned int blksz_bits; - unsigned int blksz_mask; /* just 1<error()) { debug(47, 1) ("%s: %s\n", path, xstrerror()); fatal("storeCossDirInit: Failed to open a COSS file."); } +} + +void +CossSwapDir::closeCompleted() +{ + theFile = NULL; +} + +void +CossSwapDir::readCompleted(const char *buf, int len, int errflag, RefCount aRequest) +{ + CossRead* cossRead= dynamic_cast(aRequest.getRaw()); + assert (cossRead); + StoreIOState::Pointer sio = cossRead->sio; + void *cbdata; + STRCB *callback = sio->read.callback; + char *p; + CossState *cstate = dynamic_cast(sio.getRaw()); + ssize_t rlen; + + debug(79, 3) ("storeCossReadDone: fileno %d, len %d\n", + sio->swap_filen, len); + cstate->flags.reading = 0; + + if (errflag) { + StoreFScoss::GetInstance().stats.read.fail++; + + if (errflag > 0) { + errno = errflag; + debug(79, 1) ("storeCossReadDone: error: %s\n", xstrerror()); + } else { + debug(79, 1) ("storeCossReadDone: got failure (%d)\n", errflag); + } + + rlen = -1; + } else { + StoreFScoss::GetInstance().stats.read.success++; + + if (cstate->readbuffer == NULL) { + cstate->readbuffer = (char *)xmalloc(cstate->st_size); + p = storeCossMemPointerFromDiskOffset(storeCossFilenoToDiskOffset(sio->swap_filen), + NULL); + xmemcpy(cstate->readbuffer, p, cstate->st_size); + } + + sio->offset_ += len; + xmemcpy(cstate->requestbuf, &cstate->readbuffer[cstate->requestoffset], + cstate->requestlen); + rlen = (size_t) cstate->requestlen; + } + + assert(callback); + sio->read.callback = NULL; + + if (cbdataReferenceValidDone(sio->read.callback_data, &cbdata)) + callback(cbdata, cstate->requestbuf, rlen); +} + +void +CossSwapDir::writeCompleted(int errflag, size_t len, RefCount writeRequest) +{ + CossWrite* cossWrite= dynamic_cast(writeRequest.getRaw()); + assert (cossWrite); + + debug(79, 3) ("storeCossWriteMemBufDone: buf %p, len %ld\n", cossWrite->membuf, (long int) len); + + + if (errflag) { + StoreFScoss::GetInstance().stats.stripe_write.fail++; + debug(79, 1) ("storeCossWriteMemBufDone: got failure (%d)\n", errflag); + debug(79, 1) ("size=%x\n", cossWrite->membuf->diskend - cossWrite->membuf->diskstart); + } else { + StoreFScoss::GetInstance().stats.stripe_write.success++; + } + + + dlinkDelete(&cossWrite->membuf->node, &membufs); + cbdataFree(cossWrite->membuf); + StoreFScoss::GetInstance().stats.stripes--; +} + +void +CossSwapDir::changeIO(DiskIOModule *module) +{ + DiskIOStrategy *anIO = module->createStrategy(); + safe_free(ioModule); + ioModule = xstrdup(module->type()); + + delete io; + io = anIO; + /* Change the IO Options */ + + if (currentIOOptions->options.size() > 3) + delete currentIOOptions->options.pop_back(); + + /* TODO: factor out these 4 lines */ + ConfigOption *ioOptions = NULL; + + if (io) + ioOptions = io->getOptionTree(); + + if (ioOptions) + currentIOOptions->options.push_back(ioOptions); +} + +bool +CossSwapDir::optionIOParse(char const *option, const char *value, int reconfiguring) +{ + if (strcmp(option, "IOEngine") != 0) + return false; + + if (reconfiguring) + /* silently ignore this */ + return true; + + if (!value) + self_destruct(); + + DiskIOModule *module = DiskIOModule::Find(value); + + if (!module) + self_destruct(); + + changeIO(module); + + return true; +} + +void +CossSwapDir::optionIODump(StoreEntry * e) const +{ + storeAppendPrintf(e, " IOEngine=%s", ioModule); +} + +ConfigOption * +CossSwapDir::getOptionTree() const +{ + ConfigOption *parentResult = SwapDir::getOptionTree(); + + currentIOOptions->options.push_back(parentResult); + currentIOOptions->options.push_back(new ConfigOptionAdapter(*const_cast(this), &CossSwapDir::optionIOParse, &CossSwapDir::optionIODump)); + currentIOOptions->options.push_back( + new ConfigOptionAdapter(*const_cast(this), + &CossSwapDir::optionBlockSizeParse, + &CossSwapDir::optionBlockSizeDump)); + + + ConfigOption *ioOptions = NULL; + + if (io) + ioOptions = io->getOptionTree(); + + if (ioOptions) + currentIOOptions->options.push_back(ioOptions); + + return currentIOOptions; +} + +void +CossSwapDir::init() +{ + /* FIXME: SwapDirs aren't refcounted. We call IORequestor calls, which + * are refcounted. SO, we up our count once to avoid implicit delete's. + */ + RefCountReference(); + io->init(); + openLog(); + storeCossDirRebuild(this); + theFile = io->newFile(path); + theFile->open(O_RDWR | O_CREAT, 0644, this); - n_coss_dirs++; + ++n_coss_dirs; /* * fs.blksize is normally determined by calling statvfs() etc, * but we just set it here. It is used in accounting the @@ -200,7 +371,7 @@ storeCossRebuildComplete(void *data) { RebuildState *rb = (RebuildState *)data; CossSwapDir *sd = rb->sd; - storeCossStartMembuf(sd); + sd->startMembuf(); store_dirs_rebuilding--; storeCossDirCloseTmpSwapLog(rb->sd); storeRebuildComplete(&rb->counts); @@ -363,11 +534,6 @@ storeCossAddDiskRestore(CossSwapDir * SD, const cache_key * key, EBIT_CLR(e->flags, ENTRY_VALIDATED); storeHashInsert(e, key); /* do it after we clear KEY_PRIVATE */ storeCossAdd(SD, e); -#if USE_COSS_ALLOC_NOTIFY - - e->swap_filen = storeCossAllocate(SD, e, COSS_ALLOC_NOTIFY); -#endif - assert(e->swap_filen >= 0); return e; } @@ -514,7 +680,7 @@ class CossCleanLog : public SwapDir::CleanLog { public: - CossCleanLog(SwapDir *); + CossCleanLog(CossSwapDir *); virtual const StoreEntry *nextEntry(); virtual void write(StoreEntry const &); char *cur; @@ -524,12 +690,12 @@ public: off_t outbuf_offset; int fd; dlink_node *current; - SwapDir *sd; + CossSwapDir *sd; }; #define CLEAN_BUF_SZ 16384 -CossCleanLog::CossCleanLog(SwapDir *aSwapDir) : cur(NULL),newLog(NULL),cln(NULL),outbuf(NULL), +CossCleanLog::CossCleanLog(CossSwapDir *aSwapDir) : cur(NULL),newLog(NULL),cln(NULL),outbuf(NULL), outbuf_offset(0), fd(-1),current(NULL), sd(aSwapDir) {} @@ -611,24 +777,24 @@ CossCleanLog::write(StoreEntry const &e) s.refcount = e.refcount; s.flags = e.flags; xmemcpy(&s.key, e.key, MD5_DIGEST_CHARS); - xmemcpy(state->outbuf + state->outbuf_offset, &s, ss); - state->outbuf_offset += ss; + xmemcpy(outbuf + outbuf_offset, &s, ss); + outbuf_offset += ss; /* buffered write */ - if (state->outbuf_offset + ss > CLEAN_BUF_SZ) { - if (FD_WRITE_METHOD(state->fd, state->outbuf, state->outbuf_offset) < 0) { + if (outbuf_offset + ss > CLEAN_BUF_SZ) { + if (FD_WRITE_METHOD(fd, outbuf, outbuf_offset) < 0) { debug(50, 0) ("storeCossDirWriteCleanLogs: %s: write: %s\n", - state->newLog, xstrerror()); + newLog, xstrerror()); debug(50, 0) ("storeCossDirWriteCleanLogs: Current swap logfile not replaced.\n"); - file_close(state->fd); - state->fd = -1; - unlink(state->newLog); - delete state; + file_close(fd); + fd = -1; + unlink(newLog); sd->cleanLog = NULL; + delete state; return; } - state->outbuf_offset = 0; + outbuf_offset = 0; } } @@ -738,13 +904,13 @@ CossSwapDir::newFileSystem() /* we are shutting down, flush all membufs to disk */ CossSwapDir::~CossSwapDir() { - sync(); /* This'll call a_file_syncqueue() */ - a_file_closequeue(&aq); - file_close(fd); - fd = -1; + io->sync(); + theFile->close(); + delete io; closeLog(); n_coss_dirs--; + safe_free(ioModule); } /* @@ -758,7 +924,6 @@ CossSwapDir::~CossSwapDir() int CossSwapDir::canStore(StoreEntry const &e)const { - int loadav; /* Check if the object is a special object, we can't cache these */ @@ -767,9 +932,7 @@ CossSwapDir::canStore(StoreEntry const &e)const /* Otherwise, we're ok */ /* Return load, cs->aq.aq_numpending out of MAX_ASYNCOP */ - loadav = aq.aq_numpending * 1000 / MAX_ASYNCOP; - - return loadav; + return io->load(); } /* @@ -778,7 +941,7 @@ CossSwapDir::canStore(StoreEntry const &e)const int CossSwapDir::callback() { - return a_file_callback(&aq); + return io->callback(); } /* ========== LOCAL FUNCTIONS ABOVE, GLOBAL FUNCTIONS BELOW ========== */ @@ -799,7 +962,7 @@ CossSwapDir::statfs(StoreEntry & sentry) const percent(SD->map->n_files_in_map, SD->map->max_n_files)); #endif - storeAppendPrintf(&sentry, "Pending operations: %d out of %d\n", aq.aq_numpending, MAX_ASYNCOP); + // storeAppendPrintf(&sentry, "Pending operations: %d out of %d\n", io->aq.aq_numpending, MAX_ASYNCOP); storeAppendPrintf(&sentry, "Flags:"); if (flags.selected) @@ -887,8 +1050,7 @@ CossSwapDir::dump(StoreEntry &entry)const dumpOptions(&entry); } - -CossSwapDir::CossSwapDir() : SwapDir ("coss"), fd (-1), swaplog_fd(-1), count(0), current_membuf (NULL), current_offset(0), numcollisions(0), blksz_bits(0) +CossSwapDir::CossSwapDir() : SwapDir ("coss"), swaplog_fd(-1), count(0), current_membuf (NULL), current_offset(0), numcollisions(0), blksz_bits(0), io (NULL), ioModule(NULL), currentIOOptions(new ConfigOptionVector()) { membufs.head = NULL; membufs.tail = NULL; @@ -948,20 +1110,3 @@ CossSwapDir::optionBlockSizeDump(StoreEntry * e) const { storeAppendPrintf(e, " block-size=%d", 1 << blksz_bits); } - -SwapDirOption * -CossSwapDir::getOptionTree() const -{ - SwapDirOption *parentResult = SwapDir::getOptionTree(); - - SwapDirOptionVector *result = new SwapDirOptionVector(); - - result->options.push_back(parentResult); - - result->options.push_back( - new SwapDirOptionAdapter(*const_cast(this), - &CossSwapDir::optionBlockSizeParse, - &CossSwapDir::optionBlockSizeDump)); - - return result; -} diff --git a/src/fs/coss/store_io_coss.cc b/src/fs/coss/store_io_coss.cc index d7e9e5bb72..e3ef13a611 100644 --- a/src/fs/coss/store_io_coss.cc +++ b/src/fs/coss/store_io_coss.cc @@ -1,6 +1,6 @@ /* - * $Id: store_io_coss.cc,v 1.25 2004/08/30 05:12:33 robertc Exp $ + * $Id: store_io_coss.cc,v 1.26 2004/12/20 16:30:43 robertc Exp $ * * DEBUG: section 79 Storage Manager COSS Interface * AUTHOR: Eric Stern @@ -34,26 +34,14 @@ */ #include "squid.h" +#include "CossSwapDir.h" #include "Store.h" -#include -#include "async_io.h" #include "store_coss.h" #include "MemObject.h" #include "fde.h" #include "SwapDir.h" - -static DWCB storeCossWriteMemBufDone; -static DRCB storeCossReadDone; -static void storeCossIOCallback(storeIOState * sio, int errflag); -static char *storeCossMemPointerFromDiskOffset(CossSwapDir * SD, size_t offset, CossMemBuf ** mb); -static void storeCossMemBufLock(CossSwapDir * SD, storeIOState * e); -static void storeCossMemBufUnlock(CossSwapDir * SD, storeIOState * e); -static void storeCossWriteMemBuf(CossSwapDir * SD, CossMemBuf * t); -static void storeCossWriteMemBufDone(int, int errflag, size_t len, void *my_data); -static CossMemBuf *storeCossCreateMemBuf(CossSwapDir * SD, size_t start, - sfileno curfn, int *collision); -static void storeCossMaybeWriteMemBuf(CossSwapDir * SD, CossMemBuf * t); -static void storeCossMemBufDescribe(CossMemBuf * t, int level, int line); +#include "StoreFScoss.h" +#include "DiskIO/DiskIOStrategy.h" CBDATA_TYPE(CossMemBuf); @@ -70,8 +58,8 @@ CossState::CossState(CossSwapDir *aCSD):SD (aCSD) * to work.. * -- Adrian */ -static sfileno -storeCossAllocate(CossSwapDir * SD, const StoreEntry * e, int which) +sfileno +CossSwapDir::allocate(const StoreEntry * e, int which) { CossMemBuf *newmb; off_t retofs; @@ -83,10 +71,10 @@ storeCossAllocate(CossSwapDir * SD, const StoreEntry * e, int which) if (which == COSS_ALLOC_REALLOC) { checkf = e->swap_filen; - coss_stats.alloc.realloc++; + StoreFScoss::GetInstance().stats.alloc.realloc++; } else { checkf = -1; - coss_stats.alloc.alloc++; + StoreFScoss::GetInstance().stats.alloc.alloc++; } if (e->swap_file_sz > 0) @@ -94,39 +82,36 @@ storeCossAllocate(CossSwapDir * SD, const StoreEntry * e, int which) else allocsize = objectLen(e) + e->mem_obj->swap_hdr_sz; - /* Since we're not supporting NOTIFY anymore, lets fail */ - assert(which != COSS_ALLOC_NOTIFY); - /* Check if we have overflowed the disk .. */ - /* SD->max_size is int, so cast to (off_t) *before* bit-shifting */ - if ((off_t)(SD->current_offset + allocsize) > ((off_t)SD->max_size << 10)) { + /* max_size is int, so cast to (off_t) *before* bit-shifting */ + if ((current_offset + allocsize) > ((size_t)max_size << 10)) { /* * tried to allocate past the end of the disk, so wrap * back to the beginning */ - coss_stats.disk_overflows++; - SD->current_membuf->flags.full = 1; - SD->current_membuf->diskend = SD->current_offset; - storeCossMaybeWriteMemBuf(SD, SD->current_membuf); - SD->current_offset = 0; /* wrap back to beginning */ - debug(79, 2) ("storeCossAllocate: wrap to 0\n"); + StoreFScoss::GetInstance().stats.disk_overflows++; + current_membuf->flags.full = 1; + current_membuf->diskend = current_offset; + current_membuf->maybeWrite(this); + current_offset = 0; /* wrap back to beginning */ + debug(79, 2) ("CossSwapDir::allocate: wrap to 0\n"); - newmb = storeCossCreateMemBuf(SD, 0, checkf, &coll); - SD->current_membuf = newmb; + newmb = createMemBuf(0, checkf, &coll); + current_membuf = newmb; /* Check if we have overflowed the MemBuf */ - } else if ((SD->current_offset + allocsize) >= SD->current_membuf->diskend) { + } else if ((current_offset + allocsize) >= current_membuf->diskend) { /* * Skip the blank space at the end of the stripe. start over. */ - coss_stats.stripe_overflows++; - SD->current_membuf->flags.full = 1; - SD->current_offset = SD->current_membuf->diskend; - storeCossMaybeWriteMemBuf(SD, SD->current_membuf); - debug(79, 2) ("storeCossAllocate: New offset - %ld\n", - (long int) SD->current_offset); - newmb = storeCossCreateMemBuf(SD, SD->current_offset, checkf, &coll); - SD->current_membuf = newmb; + StoreFScoss::GetInstance().stats.stripe_overflows++; + current_membuf->flags.full = 1; + current_offset = current_membuf->diskend; + current_membuf->maybeWrite(this); + debug(79, 2) ("CossSwapDir::allocate: New offset - %ld\n", + (long int) current_offset); + newmb = createMemBuf(current_offset, checkf, &coll); + current_membuf = newmb; } /* @@ -134,14 +119,14 @@ storeCossAllocate(CossSwapDir * SD, const StoreEntry * e, int which) * and return it */ if (coll == 0) { - retofs = SD->current_offset; - SD->current_offset = retofs + allocsize; + retofs = current_offset; + current_offset = retofs + allocsize; /* round up to our blocksize */ - SD->current_offset = ((SD->current_offset + SD->blksz_mask) >> SD->blksz_bits ) << SD->blksz_bits; - return SD->storeCossDiskOffsetToFileno(retofs); + current_offset = ((current_offset + blksz_mask) >> blksz_bits ) << blksz_bits; + return storeCossDiskOffsetToFileno(retofs); } else { - coss_stats.alloc.collisions++; - debug(79, 3) ("storeCossAllocate: Collision\n"); + StoreFScoss::GetInstance().stats.alloc.collisions++; + debug(79, 3) ("CossSwapDir::allocate: Collision\n"); return -1; } } @@ -150,8 +135,8 @@ void CossSwapDir::unlink(StoreEntry & e) { debug(79, 3) ("storeCossUnlink: offset %d\n", e.swap_filen); - coss_stats.unlink.ops++; - coss_stats.unlink.success++; + StoreFScoss::GetInstance().stats.unlink.ops++; + StoreFScoss::GetInstance().stats.unlink.success++; storeCossRemove(this, &e); } @@ -169,20 +154,20 @@ CossSwapDir::createStoreIO(StoreEntry &e, STFNCB * file_callback, STIOCB * callb * the squid code is broken */ assert(e.mem_obj->object_sz != -1); - coss_stats.create.ops++; + StoreFScoss::GetInstance().stats.create.ops++; /* - * this one is kinda strange - Eric called storeCossAllocate(), then + * this one is kinda strange - Eric called allocate(), then * storeCossOpen(O_RDONLY) .. weird. Anyway, I'm allocating this now. */ cstate->st_size = objectLen(&e) + e.mem_obj->swap_hdr_sz; sio->swap_dirn = index; - sio->swap_filen = storeCossAllocate(this, &e, COSS_ALLOC_ALLOCATE); + sio->swap_filen = allocate(&e, COSS_ALLOC_ALLOCATE); debug(79, 3) ("storeCossCreate: offset %ld, size %ld, end %ld\n", (long int) storeCossFilenoToDiskOffset(sio->swap_filen), (long int) cstate->st_size, (long int) (sio->swap_filen + cstate->st_size)); - /* assume storeCossAllocate() always succeeds */ + /* assume allocate() always succeeds */ assert(-1 != sio->swap_filen); sio->callback = callback; @@ -198,8 +183,8 @@ CossSwapDir::createStoreIO(StoreEntry &e, STFNCB * file_callback, STIOCB * callb /* Now add it into the index list */ storeCossAdd(this, &e); - storeCossMemBufLock(this, sio.getRaw()); - coss_stats.create.success++; + cstate->lockMemBuf(); + StoreFScoss::GetInstance().stats.create.success++; return sio; } @@ -212,7 +197,7 @@ CossSwapDir::openStoreIO(StoreEntry & e, STFNCB * file_callback, sfileno f = e.swap_filen; debug(79, 3) ("storeCossOpen: offset %d\n", f); - coss_stats.open.ops++; + StoreFScoss::GetInstance().stats.open.ops++; StoreIOState::Pointer sio = new CossState (this); cstate = dynamic_cast(sio.getRaw()); @@ -231,20 +216,20 @@ CossSwapDir::openStoreIO(StoreEntry & e, STFNCB * file_callback, cstate->flags.reading = 0; cstate->readbuffer = NULL; cstate->reqdiskoffset = -1; - p = storeCossMemPointerFromDiskOffset(this, storeCossFilenoToDiskOffset(f), NULL); + p = storeCossMemPointerFromDiskOffset(storeCossFilenoToDiskOffset(f), NULL); /* make local copy so we don't have to lock membuf */ if (p) { cstate->readbuffer = (char *)xmalloc(cstate->st_size); xmemcpy(cstate->readbuffer, p, cstate->st_size); - coss_stats.open_mem_hits++; + StoreFScoss::GetInstance().stats.open_mem_hits++; } else { /* Do the allocation */ /* this is the first time we've been called on a new sio * read the whole object into memory, then return the * requested amount */ - coss_stats.open_mem_misses++; + StoreFScoss::GetInstance().stats.open_mem_misses++; /* * This bit of code actually does the LRU disk thing - we realloc * a place for the object here, and the file_read() reads the object @@ -252,11 +237,11 @@ CossSwapDir::openStoreIO(StoreEntry & e, STFNCB * file_callback, */ cstate->reqdiskoffset = storeCossFilenoToDiskOffset(sio->swap_filen); sio->swap_filen = -1; - sio->swap_filen = storeCossAllocate(this, &e, COSS_ALLOC_REALLOC); + sio->swap_filen = allocate(&e, COSS_ALLOC_REALLOC); if (sio->swap_filen == -1) { /* We have to clean up neatly .. */ - coss_stats.open.fail++; + StoreFScoss::GetInstance().stats.open.fail++; numcollisions++; debug(79, 2) ("storeCossOpen: Reallocation of %d/%d failed\n", e.swap_dirn, e.swap_filen); /* XXX XXX XXX Will squid call storeUnlink for this object? */ @@ -270,7 +255,7 @@ CossSwapDir::openStoreIO(StoreEntry & e, STFNCB * file_callback, * lock the buffer so it doesn't get swapped out on us * this will get unlocked in storeCossClose */ - storeCossMemBufLock(this, sio.getRaw()); + cstate->lockMemBuf(); /* * Do the index magic to keep the disk and memory LRUs identical @@ -286,7 +271,7 @@ CossSwapDir::openStoreIO(StoreEntry & e, STFNCB * file_callback, */ } - coss_stats.open.success++; + StoreFScoss::GetInstance().stats.open.success++; return sio; } @@ -295,10 +280,10 @@ CossState::close() { debug(79, 3) ("storeCossClose: offset %d\n", swap_filen); - coss_stats.close.ops++; - coss_stats.close.success++; - storeCossMemBufUnlock(SD, this); - storeCossIOCallback(this, 0); + StoreFScoss::GetInstance().stats.close.ops++; + StoreFScoss::GetInstance().stats.close.success++; + SD->storeCossMemBufUnlock(this); + doCallback(0); } void @@ -307,7 +292,7 @@ CossState::read_(char *buf, size_t size, off_t offset, STRCB * callback, void *c char *p; CossSwapDir *SD = (CossSwapDir *)INDEXSD(swap_dirn); - coss_stats.read.ops++; + StoreFScoss::GetInstance().stats.read.ops++; assert(read.callback == NULL); assert(read.callback_data == NULL); read.callback = callback; @@ -326,23 +311,17 @@ CossState::read_(char *buf, size_t size, off_t offset, STRCB * callback, void *c requestoffset = offset; if (readbuffer == NULL) { - p = storeCossMemPointerFromDiskOffset(SD, SD->storeCossFilenoToDiskOffset(swap_filen), NULL); - a_file_read(&SD->aq, SD->fd, - p, - st_size, - reqdiskoffset, - storeCossReadDone, - this); + p = SD->storeCossMemPointerFromDiskOffset(SD->storeCossFilenoToDiskOffset(swap_filen), NULL); + sfileno tempReqdiskoffset = reqdiskoffset; reqdiskoffset = 0; /* XXX */ + SD->theFile->read(new CossRead(ReadRequest(p, st_size, tempReqdiskoffset), this)); } else { /* * It was copied from memory in storeCossOpen() */ - storeCossReadDone(SD->fd, - readbuffer, - st_size, - 0, - this); + ReadRequest::Pointer readRequest = new CossRead(ReadRequest( + (char *)readbuffer,st_size, 0), this); + SD->readCompleted(readbuffer, st_size, 0, readRequest); } } @@ -358,12 +337,12 @@ CossState::write(char const *buf, size_t size, off_t offset, FREE * free_func) * the squid code is broken */ assert(e->mem_obj->object_sz != -1); - coss_stats.write.ops++; + StoreFScoss::GetInstance().stats.write.ops++; debug(79, 3) ("storeCossWrite: offset %ld, len %lu\n", (long int) offset_, (unsigned long int) size); diskoffset = SD->storeCossFilenoToDiskOffset(swap_filen) + offset_; CossSwapDir *SD = (CossSwapDir *)INDEXSD(swap_dirn); - dest = storeCossMemPointerFromDiskOffset(SD, diskoffset, &membuf); + dest = SD->storeCossMemPointerFromDiskOffset(diskoffset, &membuf); assert(dest != NULL); xmemcpy(dest, buf, size); offset_ += size; @@ -371,7 +350,7 @@ CossState::write(char const *buf, size_t size, off_t offset, FREE * free_func) if (free_func) (free_func) ((char *)buf); - coss_stats.write.success++; + StoreFScoss::GetInstance().stats.write.success++; } off_t @@ -407,80 +386,57 @@ CossSwapDir::storeCossFilenoToMembuf(sfileno f) /* === STATIC =========================================================== */ -static void -storeCossReadDone(int rvfd, const char *buf, int len, int errflag, void *my_data) +CBDATA_CLASS_INIT(CossRead); +void * +CossRead::operator new (size_t) { - storeIOState *sio = (storeIOState *)my_data; - char *p; - STRCB *callback = sio->read.callback; - void *cbdata; - CossSwapDir *SD = (CossSwapDir *)INDEXSD(sio->swap_dirn); - CossState *cstate = dynamic_cast(sio); - ssize_t rlen; - - debug(79, 3) ("storeCossReadDone: fileno %d, FD %d, len %d\n", - sio->swap_filen, rvfd, len); - cstate->flags.reading = 0; - - if (errflag) { - coss_stats.read.fail++; - - if (errflag > 0) { - errno = errflag; - debug(79, 1) ("storeCossReadDone: error: %s\n", xstrerror()); - } else { - debug(79, 1) ("storeCossReadDone: got failure (%d)\n", errflag); - } - - rlen = -1; - } else { - coss_stats.read.success++; - - if (cstate->readbuffer == NULL) { - cstate->readbuffer = (char *)xmalloc(cstate->st_size); - p = storeCossMemPointerFromDiskOffset(SD, - SD->storeCossFilenoToDiskOffset(sio->swap_filen), - NULL); - xmemcpy(cstate->readbuffer, p, cstate->st_size); - } + CBDATA_INIT_TYPE(CossRead); + CossRead *result = cbdataAlloc(CossRead); + return result; +} - sio->offset_ += len; - xmemcpy(cstate->requestbuf, &cstate->readbuffer[cstate->requestoffset], - cstate->requestlen); - rlen = (size_t) cstate->requestlen; - } +void +CossRead::operator delete (void *address) +{ + cbdataFree(address); +} - assert(callback); - sio->read.callback = NULL; +CBDATA_CLASS_INIT(CossWrite); +void * +CossWrite::operator new (size_t) +{ + CBDATA_INIT_TYPE(CossWrite); + CossWrite *result = cbdataAlloc(CossWrite); + return result; +} - if (cbdataReferenceValidDone(sio->read.callback_data, &cbdata)) - callback(cbdata, cstate->requestbuf, rlen); +void +CossWrite::operator delete (void *address) +{ + cbdataFree(address); } -static void -storeCossIOCallback(storeIOState * sio, int errflag) +void +CossState::doCallback(int errflag) { - CossState *cstate = dynamic_cast(sio); - STIOCB *callback = sio->callback; + STIOCB *callback = this->callback; void *cbdata; - debug(79, 3) ("storeCossIOCallback: errflag=%d\n", errflag); - assert(NULL == cstate->locked_membuf); - xfree(cstate->readbuffer); - sio->callback = NULL; - - if (cbdataReferenceValidDone(sio->callback_data, &cbdata)) - callback(cbdata, errflag, sio); + debug(79, 3) ("CossState::doCallback: errflag=%d\n", errflag); + assert(NULL == locked_membuf); + xfree(readbuffer); + this->callback = NULL; - cbdataFree(sio); + if (cbdataReferenceValidDone(callback_data, &cbdata)) + callback(cbdata, errflag, this); } -static char * -storeCossMemPointerFromDiskOffset(CossSwapDir * SD, size_t offset, CossMemBuf ** mb) +char * +CossSwapDir::storeCossMemPointerFromDiskOffset(size_t offset, CossMemBuf ** mb) { CossMemBuf *t; dlink_node *m; - for (m = SD->membufs.head; m; m = m->next) { + for (m = membufs.head; m; m = m->next) { t = (CossMemBuf *)m->data; if ((offset >= t->diskstart) && (offset < t->diskend)) { @@ -497,20 +453,19 @@ storeCossMemPointerFromDiskOffset(CossSwapDir * SD, size_t offset, CossMemBuf ** return NULL; } -static void -storeCossMemBufLock(CossSwapDir * SD, storeIOState * sio) +void +CossState::lockMemBuf() { - CossMemBuf *t = SD->storeCossFilenoToMembuf(sio->swap_filen); - CossState *cstate = dynamic_cast(sio); - debug(79, 3) ("storeCossMemBufLock: locking %p, lockcount %d\n", t, t->lockcount); - cstate->locked_membuf = t; - t->lockcount++; + CossMemBuf *t = SD->storeCossFilenoToMembuf(swap_filen); + debug(79, 3) ("CossState::lockMemBuf: locking %p, lockcount %d\n", t, t->lockcount); + locked_membuf = t; + ++t->lockcount; } -static void -storeCossMemBufUnlock(CossSwapDir * SD, storeIOState * sio) +void +CossSwapDir::storeCossMemBufUnlock(storeIOState * sio) { - CossMemBuf *t = SD->storeCossFilenoToMembuf(sio->swap_filen); + CossMemBuf *t = storeCossFilenoToMembuf(sio->swap_filen); CossState *cstate = dynamic_cast(sio); if (NULL == t) @@ -522,7 +477,7 @@ storeCossMemBufUnlock(CossSwapDir * SD, storeIOState * sio) cstate->locked_membuf = NULL; - storeCossMaybeWriteMemBuf(SD, t); + t->maybeWrite(this); } void @@ -533,7 +488,7 @@ CossSwapDir::sync() int end; /* First, flush pending IO ops */ - a_file_syncqueue(&aq); + io->sync(); /* Then, flush any in-memory partial membufs */ @@ -548,65 +503,44 @@ CossSwapDir::sync() sleep(5); /* XXX EEEWWW! */ } - lseek(fd, t->diskstart, SEEK_SET); - end = (t == current_membuf) ? current_offset : t->diskend; - FD_WRITE_METHOD(fd, t->buffer, end - t->diskstart); + if ((size_t)end > t->diskstart) + theFile->write(new CossWrite(WriteRequest((char const *)&t->buffer, t->diskstart, end - t->diskstart, NULL), t)); + + /* and flush */ + io->sync(); } } -static void -storeCossMaybeWriteMemBuf(CossSwapDir * SD, CossMemBuf * t) +void +CossMemBuf::maybeWrite(CossSwapDir * SD) { - storeCossMemBufDescribe(t, 3, __LINE__); - - if (!t->flags.full) - debug(79, 3) ("membuf %p not full\n", t); - else if (t->flags.writing) - debug(79, 3) ("membuf %p writing\n", t); - else if (t->lockcount) - debug(79, 3) ("membuf %p lockcount=%d\n", t, t->lockcount); + describe(3, __LINE__); + + if (!flags.full) + debug(79, 3) ("membuf %p not full\n", this); + else if (flags.writing) + debug(79, 3) ("membuf %p writing\n", this); + else if (lockcount) + debug(79, 3) ("membuf %p lockcount=%d\n", this, lockcount); else - storeCossWriteMemBuf(SD, t); -} - -static void -storeCossWriteMemBuf(CossSwapDir * SD, CossMemBuf * t) -{ - coss_stats.stripe_write.ops++; - debug(79, 3) ("storeCossWriteMemBuf: offset %ld, len %ld\n", - (long int) t->diskstart, (long int) (t->diskend - t->diskstart)); - t->flags.writing = 1; - a_file_write(&SD->aq, SD->fd, t->diskstart, &t->buffer, - t->diskend - t->diskstart, storeCossWriteMemBufDone, t, NULL); + write(SD); } - -static void -storeCossWriteMemBufDone(int rvfd, int errflag, size_t len, void *my_data) +void +CossMemBuf::write(CossSwapDir * SD) { - CossMemBuf *t = (CossMemBuf *)my_data; - - debug(79, 3) ("storeCossWriteMemBufDone: buf %p, len %ld\n", t, (long int) len); - - if (errflag) { - coss_stats.stripe_write.fail++; - debug(79, 1) ("storeCossWriteMemBufDone: got failure (%d)\n", errflag); - debug(79, 1) ("FD %d, size=%x\n", rvfd, t->diskend - t->diskstart); - } else { - coss_stats.stripe_write.success++; - } - - - dlinkDelete(&t->node, &t->SD->membufs); - cbdataFree(t); - coss_stats.stripes--; + StoreFScoss::GetInstance().stats.stripe_write.ops++; + debug(79, 3) ("CossMemBuf::write: offset %ld, len %ld\n", + (long int) diskstart, (long int) (diskend - diskstart)); + flags.writing = 1; + /* XXX Remember that diskstart/diskend are block offsets! */ + SD->theFile->write(new CossWrite(WriteRequest((char const *)&buffer, diskstart, diskend - diskstart, NULL), this)); } -static CossMemBuf * -storeCossCreateMemBuf(CossSwapDir * SD, size_t start, - sfileno curfn, int *collision) +CossMemBuf * +CossSwapDir::createMemBuf(size_t start, sfileno curfn, int *collision) { CossMemBuf *newmb; CossMemBuf *t; @@ -617,33 +551,33 @@ storeCossCreateMemBuf(CossSwapDir * SD, size_t start, CBDATA_INIT_TYPE_FREECB(CossMemBuf, NULL); newmb = cbdataAlloc(CossMemBuf); newmb->diskstart = start; - debug(79, 3) ("storeCossCreateMemBuf: creating new membuf at %ld\n", (long int) newmb->diskstart); - debug(79, 3) ("storeCossCreateMemBuf: at %p\n", newmb); + debug(79, 3) ("CossSwapDir::createMemBuf: creating new membuf at %ld\n", (long int) newmb->diskstart); + debug(79, 3) ("CossSwapDir::createMemBuf: at %p\n", newmb); newmb->diskend = newmb->diskstart + COSS_MEMBUF_SZ; newmb->flags.full = 0; newmb->flags.writing = 0; newmb->lockcount = 0; - newmb->SD = SD; + newmb->SD = this; /* XXX This should be reversed, with the new buffer last in the chain */ - dlinkAdd(newmb, &newmb->node, &SD->membufs); + dlinkAdd(newmb, &newmb->node, &membufs); /* Print out the list of membufs */ - debug(79, 3) ("storeCossCreateMemBuf: membuflist:\n"); + debug(79, 3) ("CossSwapDir::createMemBuf: membuflist:\n"); - for (m = SD->membufs.head; m; m = m->next) { + for (m = membufs.head; m; m = m->next) { t = (CossMemBuf *)m->data; - storeCossMemBufDescribe(t, 3, __LINE__); + t->describe(3, __LINE__); } /* * Kill objects from the tail to make space for a new chunk */ - for (m = SD->cossindex.tail; m; m = prev) { + for (m = cossindex.tail; m; m = prev) { off_t o; prev = m->prev; e = (StoreEntry *)m->data; - o = SD->storeCossFilenoToDiskOffset(e->swap_filen); + o = storeCossFilenoToDiskOffset(e->swap_filen); if (curfn == e->swap_filen) *collision = 1; /* Mark an object alloc collision */ @@ -656,9 +590,9 @@ storeCossCreateMemBuf(CossSwapDir * SD, size_t start, } if (numreleased > 0) - debug(79, 3) ("storeCossCreateMemBuf: this allocation released %d storeEntries\n", numreleased); + debug(79, 3) ("CossSwapDir::createMemBuf: this allocation released %d storeEntries\n", numreleased); - coss_stats.stripes++; + StoreFScoss::GetInstance().stats.stripes++; return newmb; } @@ -667,12 +601,12 @@ storeCossCreateMemBuf(CossSwapDir * SD, size_t start, * Creates the initial membuf after rebuild */ void -storeCossStartMembuf(CossSwapDir * sd) +CossSwapDir::startMembuf() { CossMemBuf *newmb; - newmb = storeCossCreateMemBuf(sd, sd->current_offset, -1, NULL); - assert(!sd->current_membuf); - sd->current_membuf = newmb; + newmb = createMemBuf(current_offset, -1, NULL); + assert(!current_membuf); + current_membuf = newmb; } /* @@ -681,14 +615,14 @@ storeCossStartMembuf(CossSwapDir * sd) CossState::~CossState() {} -static void -storeCossMemBufDescribe(CossMemBuf * t, int level, int line) +void +CossMemBuf::describe(int level, int line) { debug(79, level) ("membuf %p, LC:%02d, ST:%010lu, FL:%c%c\n", - t, - t->lockcount, - (unsigned long) t->diskstart, - t->flags.full ? 'F' : '.', - t->flags.writing ? 'W' : '.'); + this, + lockcount, + (unsigned long) diskstart, + flags.full ? 'F' : '.', + flags.writing ? 'W' : '.'); } diff --git a/src/fs/diskd/StoreFSdiskd.cc b/src/fs/diskd/StoreFSdiskd.cc index 63ff45cc8b..a5072cca7f 100644 --- a/src/fs/diskd/StoreFSdiskd.cc +++ b/src/fs/diskd/StoreFSdiskd.cc @@ -1,6 +1,6 @@ /* - * $Id: StoreFSdiskd.cc,v 1.1 2003/07/22 15:23:11 robertc Exp $ + * $Id: StoreFSdiskd.cc,v 1.2 2004/12/20 16:30:43 robertc Exp $ * * DEBUG: section 47 Store Directory Routines * AUTHOR: Robert Collins @@ -37,6 +37,6 @@ #include "StoreFileSystem.h" #include "fs/ufs/StoreFSufs.h" -#include "fs/diskd/store_diskd.h" +#include "DiskIO/DiskIOModule.h" -static StoreFSufs DiskdInstance(DiskdIOModule::GetInstance(), "diskd"); +static StoreFSufs DiskdInstance("DiskDaemon", "diskd"); diff --git a/src/fs/diskd/store_diskd.h b/src/fs/diskd/store_diskd.h deleted file mode 100644 index d2522493ef..0000000000 --- a/src/fs/diskd/store_diskd.h +++ /dev/null @@ -1,173 +0,0 @@ -/* - * store_diskd.h - * - * Internal declarations for the diskd routines - */ - -#ifndef __STORE_DISKD_H__ -#define __STORE_DISKD_H__ - -#include "ufscommon.h" - -/* - * magic2 is the point at which we start blocking on msgsnd/msgrcv. - * If a queue has magic2 (or more) messages away, then we read the - * queue until the level falls below magic2. Recommended value - * is 75% of SHMBUFS. magic1 is the number of messages away which we - * stop allowing open/create for. - */ - -typedef struct _diomsg diomsg; - -class DiskdIO; - -class DiskdFile : public DiskFile -{ - -public: - void * operator new (size_t); - void operator delete (void *); - DiskdFile (char const *path, DiskdIO *); - ~DiskdFile(); - virtual void open (int, mode_t, IORequestor::Pointer); - virtual void create (int, mode_t, IORequestor::Pointer); - virtual void read(char *, off_t, size_t); - virtual void write(char const *buf, size_t size, off_t offset, FREE *free_func); - virtual void close (); - virtual bool error() const; - virtual bool canRead() const; - virtual bool ioInProgress()const; - - /* Temporary */ - int getID() const {return id;} - - void completed (diomsg *); - -private: - int id; - char const *path_; - bool errorOccured; - DiskdIO *IO; - IORequestor::Pointer ioRequestor; - CBDATA_CLASS(DiskdFile); - void openDone(diomsg *); - void createDone (diomsg *); - void readDone (diomsg *); - void writeDone (diomsg *); - void closeDone (diomsg *); - int mode; - void notifyClient(); - bool canNotifyClient() const; - void ioAway(); - void ioCompleted(); - size_t inProgressIOs; -}; - -class SharedMemory -{ - -public: - void put(off_t); - - void *get - (off_t *); - - void init (int ikey, int magic2); - - int nbufs; - - char *buf; - - char *inuse_map; - - int id; -}; - -#include "dio.h" - -struct _diskd_stats -{ - int open_fail_queue_len; - int block_queue_len; - int max_away; - int max_shmuse; - int shmbuf_count; - int sent_count; - int recv_count; - int sio_id; - - struct - { - int ops; - int success; - int fail; - } - - open, create, close, unlink, read, write; -}; - -typedef struct _diskd_stats diskd_stats_t; - -class SwapDir; -#define SHMBUF_BLKSZ SM_PAGE_SIZE - -extern diskd_stats_t diskd_stats; - -#include "fs/ufs/IOModule.h" - -class DiskdIOModule : public IOModule -{ - -public: - static DiskdIOModule &GetInstance(); - DiskdIOModule(); - virtual void init(); - virtual void shutdown(); - virtual UFSStrategy *createSwapDirIOStrategy(); - -private: - static DiskdIOModule *Instance; - bool initialised; -}; - -/* Per SwapDir instance */ - -class DiskdIO : public UFSStrategy -{ - -public: - DiskdIO(); - virtual bool shedLoad(); - virtual void openFailed(); - virtual int load(); - virtual StoreIOState::Pointer createState(SwapDir *SD, StoreEntry *e, STIOCB * callback, void *callback_data) const; - virtual DiskFile::Pointer newFile (char const *path); - virtual SwapDirOption *getOptionTree() const; - virtual void unlinkFile (char const *); - void storeDiskdHandle(diomsg * M); - virtual void init(); - virtual int callback(); - virtual void sync(); - virtual void statfs(StoreEntry & sentry)const; - int away; - int magic1; - int magic2; - int smsgid; - int rmsgid; - int wfd; - SharedMemory shm; - -private: - static size_t newInstance(); - bool optionQ1Parse(char const *option, const char *value, int reconfiguring); - void optionQ1Dump(StoreEntry * e) const; - bool optionQ2Parse(char const *option, const char *value, int reconfiguring); - void optionQ2Dump(StoreEntry * e) const; - void unlinkDone(diomsg * M); - static size_t nextInstanceID; - size_t instanceID; -}; - -extern void storeDiskdStats(StoreEntry * sentry); - -#endif diff --git a/src/fs/diskd/store_io_diskd.cc b/src/fs/diskd/store_io_diskd.cc deleted file mode 100644 index 85a0b042c6..0000000000 --- a/src/fs/diskd/store_io_diskd.cc +++ /dev/null @@ -1,1072 +0,0 @@ - -/* - * $Id: store_io_diskd.cc,v 1.41 2004/12/20 14:52:32 robertc Exp $ - * - * DEBUG: section 79 Squid-side DISKD I/O functions. - * AUTHOR: Duane Wessels - * - * SQUID Web Proxy Cache http://www.squid-cache.org/ - * ---------------------------------------------------------- - * - * Squid is the result of efforts by numerous individuals from - * the Internet community; see the CONTRIBUTORS file for full - * details. Many organizations have provided support for Squid's - * development; see the SPONSORS file for full details. Squid is - * Copyrighted (C) 2001 by the Regents of the University of - * California; see the COPYRIGHT file for full details. Squid - * incorporates software developed and/or copyrighted by other - * sources; see the CREDITS file for full details. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write to the Free Software - * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. - * - */ - -#include "config.h" -#include "squid.h" -#include "Store.h" - -#include -#include -#include - -#include "store_diskd.h" -#include "SwapDir.h" - -size_t DiskdIO::nextInstanceID (0); - -static int storeDiskdSend(int, DiskdIO *, int, StoreIOState::Pointer, int, int, off_t); -static int storeDiskdSend(int, DiskdIO *, int, DiskdFile *, int, int, off_t); - -/* === PUBLIC =========================================================== */ -DiskdIO::DiskdIO() : away (0), magic1(64), magic2(72), instanceID(newInstance()) -{} - -size_t -DiskdIO::newInstance() -{ - return ++nextInstanceID; -} - -bool -DiskdIO::shedLoad() -{ - /* - * Fail on open() if there are too many requests queued. - */ - - if (away > magic1) { - debug(79, 3) ("storeDiskdIO::shedLoad: Shedding, too many requests away\n"); - - return true; - } - - return false; -} - -int -DiskdIO::load() -{ - /* Calculate the storedir load relative to magic2 on a scale of 0 .. 1000 */ - /* the parse function guarantees magic2 is positivie */ - return away * 1000 / magic2; -} - -void -DiskdIO::openFailed() -{ - diskd_stats.open_fail_queue_len++; -} - -StoreIOState::Pointer -DiskdIO::createState(SwapDir *SD, StoreEntry *e, STIOCB * callback, void *callback_data) const -{ - return new UFSStoreState (SD, e, callback, callback_data); -} - -DiskFile::Pointer -DiskdIO::newFile (char const *path) -{ - return new DiskdFile (path, this); -} - - -void -DiskdIO::unlinkFile(char const *path) -{ - if (shedLoad()) { - /* Damn, we need to issue a sync unlink here :( */ - debug(79, 2) ("storeDiskUnlink: Out of queue space, sync unlink\n"); -#if USE_UNLINKD - - unlinkdUnlink(path); -#elif USE_TRUNCATE - - truncate(path, 0); -#else - - unlink(path); -#endif - - return; - } - - /* We can attempt a diskd unlink */ - int x; - - off_t shm_offset; - - char *buf; - - buf = (char *)shm.get(&shm_offset); - - xstrncpy(buf, path, SHMBUF_BLKSZ); - - x = storeDiskdSend(_MQD_UNLINK, - this, - 0, - (StoreIOState::Pointer )NULL, - 0, - 0, - shm_offset); - - if (x < 0) { - debug(79, 1) ("storeDiskdSend UNLINK: %s\n", xstrerror()); - ::unlink(buf); /* XXX EWW! */ - shm.put (shm_offset); - } - - diskd_stats.unlink.ops++; -} - -/* - * SHM manipulation routines - */ -void -SharedMemory::put (off_t offset) -{ - int i; - assert(offset >= 0); - assert(offset < nbufs * SHMBUF_BLKSZ); - i = offset / SHMBUF_BLKSZ; - assert(i < nbufs); - assert(CBIT_TEST(inuse_map, i)); - CBIT_CLR(inuse_map, i); - --diskd_stats.shmbuf_count; -} - -void * - -SharedMemory::get - (off_t * shm_offset) -{ - char *aBuf = NULL; - int i; - - for (i = 0; i < nbufs; i++) { - if (CBIT_TEST(inuse_map, i)) - continue; - - CBIT_SET(inuse_map, i); - - *shm_offset = i * SHMBUF_BLKSZ; - - aBuf = buf + (*shm_offset); - - break; - } - - assert(aBuf); - assert(aBuf >= buf); - assert(aBuf < buf + (nbufs * SHMBUF_BLKSZ)); - diskd_stats.shmbuf_count++; - - if (diskd_stats.max_shmuse < diskd_stats.shmbuf_count) - diskd_stats.max_shmuse = diskd_stats.shmbuf_count; - - return aBuf; -} - -void -SharedMemory::init(int ikey, int magic2) -{ - nbufs = (int)(magic2 * 1.3); - id = shmget((key_t) (ikey + 2), - nbufs * SHMBUF_BLKSZ, 0600 | IPC_CREAT); - - if (id < 0) { - debug(50, 0) ("storeDiskdInit: shmget: %s\n", xstrerror()); - fatal("shmget failed"); - } - - buf = (char *)shmat(id, NULL, 0); - - if (buf == (void *) -1) { - debug(50, 0) ("storeDiskdInit: shmat: %s\n", xstrerror()); - fatal("shmat failed"); - } - - inuse_map = (char *)xcalloc((nbufs + 7) / 8, 1); - diskd_stats.shmbuf_count += nbufs; - - for (int i = 0; i < nbufs; i++) { - CBIT_SET(inuse_map, i); - put (i * SHMBUF_BLKSZ); - } -} - -CBDATA_CLASS_INIT(DiskdFile); - -void * -DiskdFile::operator new (size_t) -{ - CBDATA_INIT_TYPE(DiskdFile); - DiskdFile *result = cbdataAlloc(DiskdFile); - debug (79,3)("diskdFile with base %p allocating\n", result); - /* Mark result as being owned - we want the refcounter to do the delete - * call */ - return cbdataReference(result); -} - -void -DiskdFile::operator delete (void *address) -{ - debug (79,3)("diskdFile with base %p deleting\n",address); - DiskdFile *t = static_cast(address); - cbdataFree(address); - /* And allow the memory to be freed */ - cbdataReferenceDone (t); -} - -DiskdFile::DiskdFile (char const *aPath, DiskdIO *anIO) : errorOccured (false), IO(anIO), - inProgressIOs (0) -{ - assert (aPath); - debug (79,3)("DiskdFile::DiskdFile: %s\n", aPath); - path_ = xstrdup (aPath); - id = diskd_stats.sio_id++; -} - -DiskdFile::~DiskdFile() -{ - assert (inProgressIOs == 0); - safe_free (path_); -} - -void -DiskdFile::open (int flags, mode_t aMode, IORequestor::Pointer callback) -{ - debug (79,3)("DiskdFile::open: %p opening for %p\n", this, callback.getRaw()); - assert (ioRequestor.getRaw() == NULL); - ioRequestor = callback; - assert (callback.getRaw()); - mode = flags; - off_t shm_offset; - char *buf = (char *)IO->shm.get(&shm_offset); - xstrncpy(buf, path_, SHMBUF_BLKSZ); - ioAway(); - int x = storeDiskdSend(_MQD_OPEN, - IO, - id, - this, - strlen(buf) + 1, - mode, - shm_offset); - - if (x < 0) { - ioCompleted(); - errorOccured = true; - IO->shm.put (shm_offset); - ioRequestor->ioCompletedNotification(); - ioRequestor = NULL; - } - - diskd_stats.open.ops++; -} - -void -DiskdFile::create (int flags, mode_t aMode, IORequestor::Pointer callback) -{ - debug (79,3)("DiskdFile::create: %p creating for %p\n", this, callback.getRaw()); - assert (ioRequestor.getRaw() == NULL); - ioRequestor = callback; - assert (callback.getRaw()); - mode = flags; - off_t shm_offset; - char *buf = (char *)IO->shm.get(&shm_offset); - xstrncpy(buf, path_, SHMBUF_BLKSZ); - ioAway(); - int x = storeDiskdSend(_MQD_CREATE, - IO, - id, - this, - strlen(buf) + 1, - mode, - shm_offset); - - if (x < 0) { - ioCompleted(); - errorOccured = true; - IO->shm.put (shm_offset); - debug(79, 1) ("storeDiskdSend CREATE: %s\n", xstrerror()); - notifyClient(); - ioRequestor = NULL; - return; - } - - diskd_stats.create.ops++; -} - -void -DiskdFile::read(char *buf, off_t offset, size_t size) -{ - assert (ioRequestor.getRaw() != NULL); - off_t shm_offset; - IO->shm.get(&shm_offset); - ioAway(); - int x = storeDiskdSend(_MQD_READ, - IO, - id, - this, - (int) size, - (int) offset, - shm_offset); - - if (x < 0) { - ioCompleted(); - errorOccured = true; - IO->shm.put (shm_offset); - debug(79, 1) ("storeDiskdSend READ: %s\n", xstrerror()); - notifyClient(); - ioRequestor = NULL; - return; - } - - diskd_stats.read.ops++; -} - -void -DiskdFile::close() -{ - debug (79,3)("DiskdFile::close: %p closing for %p\n", this, ioRequestor.getRaw()); - assert (ioRequestor.getRaw()); - ioAway(); - int x = storeDiskdSend(_MQD_CLOSE, - IO, - id, - this, - 0, - 0, - -1); - - if (x < 0) { - ioCompleted(); - errorOccured = true; - debug(79, 1) ("storeDiskdSend CLOSE: %s\n", xstrerror()); - notifyClient(); - ioRequestor = NULL; - return; - } - - diskd_stats.close.ops++; -} - -bool -DiskdFile::error() const -{ - return errorOccured; -} - -bool -DiskdFile::canRead() const -{ - return !error(); -} - -bool -DiskdFile::canNotifyClient() const -{ - if (!ioRequestor.getRaw()) { - debug (79,3)("DiskdFile::canNotifyClient: No ioRequestor to notify\n"); - return false; - } - - return true; -} - -void -DiskdFile::notifyClient() -{ - if (!canNotifyClient()) { - return; - } - - ioRequestor->ioCompletedNotification(); -} - -void -DiskdFile::completed(diomsg *M) -{ - assert (M->newstyle); - - switch (M->mtype) { - - case _MQD_OPEN: - openDone(M); - break; - - case _MQD_CREATE: - createDone(M); - break; - - case _MQD_CLOSE: - closeDone(M); - break; - - case _MQD_READ: - readDone(M); - break; - - case _MQD_WRITE: - writeDone(M); - break; - - case _MQD_UNLINK: - assert (0); - break; - - default: - assert(0); - break; - } -} - -void -DiskdFile::openDone(diomsg *M) -{ - statCounter.syscalls.disk.opens++; - debug(79, 3) ("storeDiskdOpenDone: status %d\n", M->status); - assert (FILE_MODE(mode) == O_RDONLY); - - if (M->status < 0) { - diskd_stats.open.fail++; - errorOccured = true; - } else { - diskd_stats.open.success++; - } - - ioCompleted(); - notifyClient(); -} - -void -DiskdFile::createDone(diomsg *M) -{ - statCounter.syscalls.disk.opens++; - debug(79, 3) ("storeDiskdCreateDone: status %d\n", M->status); - - if (M->status < 0) { - diskd_stats.create.fail++; - errorOccured = true; - } else { - diskd_stats.create.success++; - } - - ioCompleted(); - notifyClient(); -} - -void -DiskdFile::write(char const *buf, size_t size, off_t offset, FREE *free_func) -{ - debugs(79, 3, "DiskdFile::write: this " << (void *)this << ", buf " << (void *)buf << ", off " << offset << ", len " << size); - off_t shm_offset; - char *sbuf = (char *)IO->shm.get(&shm_offset); - xmemcpy(sbuf, buf, size); - - if (free_func) - free_func(const_cast(buf)); - - ioAway(); - - int x = storeDiskdSend(_MQD_WRITE, - IO, - id, - this, - (int) size, - (int) offset, - shm_offset); - - if (x < 0) { - ioCompleted(); - errorOccured = true; - debug(79, 1) ("storeDiskdSend WRITE: %s\n", xstrerror()); - IO->shm.put (shm_offset); - notifyClient(); - ioRequestor = NULL; - return; - } - - diskd_stats.write.ops++; -} - - -/* === STATIC =========================================================== */ - -void -DiskdFile::ioAway() -{ - ++inProgressIOs; -} - -void -DiskdFile::ioCompleted() -{ - --inProgressIOs; -} - -void -DiskdFile::closeDone(diomsg * M) -{ - statCounter.syscalls.disk.closes++; - debug(79, 3) ("DiskdFile::closeDone: status %d\n", M->status); - - if (M->status < 0) { - diskd_stats.close.fail++; - errorOccured = true; - } else { - diskd_stats.close.success++; - } - - ioCompleted(); - - if (canNotifyClient()) - ioRequestor->closeCompleted(); - - ioRequestor = NULL; -} - -void -DiskdFile::readDone(diomsg * M) -{ - statCounter.syscalls.disk.reads++; - debug(79, 3) ("DiskdFile::readDone: status %d\n", M->status); - - if (M->status < 0) { - diskd_stats.read.fail++; - ioCompleted(); - errorOccured = true; - ioRequestor->readCompleted(NULL, -1, DISK_ERROR); - return; - } - - diskd_stats.read.success++; - - ioCompleted(); - ioRequestor->readCompleted (IO->shm.buf + M->shm_offset, M->status, DISK_OK); -} - -void -DiskdFile::writeDone(diomsg *M) -{ - statCounter.syscalls.disk.writes++; - debug(79, 3) ("storeDiskdWriteDone: status %d\n", M->status); - - if (M->status < 0) { - errorOccured = true; - diskd_stats.write.fail++; - ioCompleted(); - ioRequestor->writeCompleted (DISK_ERROR,0); - return; - } - - diskd_stats.write.success++; - ioCompleted(); - ioRequestor->writeCompleted (DISK_OK,M->status); -} - -bool -DiskdFile::ioInProgress()const -{ - return inProgressIOs != 0; -} - -void -DiskdIO::unlinkDone(diomsg * M) -{ - debug(79, 3) ("storeDiskdUnlinkDone: file %s status %d\n",shm.buf + M->shm_offset, - M->status); - statCounter.syscalls.disk.unlinks++; - - if (M->status < 0) - diskd_stats.unlink.fail++; - else - diskd_stats.unlink.success++; -} - -void -DiskdIO::storeDiskdHandle(diomsg * M) -{ - if (!cbdataReferenceValid (M->callback_data)) { - debug(79, 3) ("storeDiskdHandle: Invalid callback_data %p\n", - M->callback_data); - cbdataReferenceDone (M->callback_data); - return; - } - - - if (M->newstyle) { - DiskdFile *theFile = (DiskdFile *)M->callback_data; - theFile->RefCountDereference(); - theFile->completed (M); - } else - switch (M->mtype) { - - case _MQD_OPEN: - - case _MQD_CREATE: - - case _MQD_CLOSE: - - case _MQD_READ: - - case _MQD_WRITE: - assert (0); - break; - - case _MQD_UNLINK: - unlinkDone(M); - break; - - default: - assert(0); - break; - } - - cbdataReferenceDone (M->callback_data); -} - -int -storeDiskdSend(int mtype, DiskdIO *IO, int id, DiskdFile *theFile, int size, int offset, off_t shm_offset) -{ - int x; - diomsg M; - static int send_errors = 0; - static int last_seq_no = 0; - static int seq_no = 0; - M.mtype = mtype; - M.callback_data = cbdataReference(theFile); - theFile->RefCountReference(); - M.size = size; - M.offset = offset; - M.status = -1; - M.shm_offset = (int) shm_offset; - M.id = id; - M.seq_no = ++seq_no; - M.newstyle = true; - - if (M.seq_no < last_seq_no) - debug(79, 1) ("WARNING: sequencing out of order\n"); - - x = msgsnd(IO->smsgid, &M, msg_snd_rcv_sz, IPC_NOWAIT); - - last_seq_no = M.seq_no; - - if (0 == x) { - diskd_stats.sent_count++; - IO->away++; - } else { - debug(79, 1) ("storeDiskdSend: msgsnd: %s\n", xstrerror()); - cbdataReferenceDone(M.callback_data); - - if (++send_errors > 100) - fatal ("over 100 errors sending to the daemon - aborting\n"); - - IO->shm.put (shm_offset); - } - - /* - * We have to drain the queue here if necessary. If we don't, - * then we can have a lot of messages in the queue (probably - * up to 2*magic1) and we can run out of shared memory buffers. - */ - /* - * Note that we call storeDirCallback (for all SDs), rather - * than storeDiskdDirCallback for just this SD, so that while - * we're "blocking" on this SD we can also handle callbacks - * from other SDs that might be ready. - */ - while (IO->away > IO->magic2) { - - struct timeval delay = {0, 1}; - - select(0, NULL, NULL, NULL, &delay); - storeDirCallback(); - - if (delay.tv_usec < 1000000) - delay.tv_usec <<= 1; - } - - return x; -} - -static int -storeDiskdSend(int mtype, DiskdIO *IO, int id, StoreIOState::Pointer sio, int size, int offset, off_t shm_offset) -{ - int x; - diomsg M; - static int send_errors = 0; - static int last_seq_no = 0; - static int seq_no = 0; - M.mtype = mtype; - M.callback_data = cbdataReference(sio.getRaw()); - M.size = size; - M.offset = offset; - M.status = -1; - M.shm_offset = (int) shm_offset; - M.id = id; - M.seq_no = ++seq_no; - M.newstyle = false; - - if (M.seq_no < last_seq_no) - debug(79, 1) ("WARNING: sequencing out of order\n"); - - x = msgsnd(IO->smsgid, &M, msg_snd_rcv_sz, IPC_NOWAIT); - - last_seq_no = M.seq_no; - - if (0 == x) { - diskd_stats.sent_count++; - IO->away++; - } else { - debug(79, 1) ("storeDiskdSend: msgsnd: %s\n", xstrerror()); - cbdataReferenceDone(M.callback_data); - - if (++send_errors > 100) - fatal ("over 100 errors sending to the daemon - aborting\n"); - } - - /* - * We have to drain the queue here if necessary. If we don't, - * then we can have a lot of messages in the queue (probably - * up to 2*magic1) and we can run out of shared memory buffers. - */ - /* - * Note that we call storeDirCallback (for all SDs), rather - * than storeDiskdDirCallback for just this SD, so that while - * we're "blocking" on this SD we can also handle callbacks - * from other SDs that might be ready. - */ - while (IO->away > IO->magic2) { - - struct timeval delay = {0, 1}; - - select(0, NULL, NULL, NULL, &delay); - storeDirCallback(); - - if (delay.tv_usec < 1000000) - delay.tv_usec <<= 1; - } - - return x; -} - -SwapDirOption * -DiskdIO::getOptionTree() const -{ - SwapDirOptionVector *result = new SwapDirOptionVector; - result->options.push_back(new SwapDirOptionAdapter(*const_cast(this), &DiskdIO::optionQ1Parse, &DiskdIO::optionQ1Dump)); - result->options.push_back(new SwapDirOptionAdapter(*const_cast(this), &DiskdIO::optionQ2Parse, &DiskdIO::optionQ2Dump)); - return result; -} - -bool -DiskdIO::optionQ1Parse(const char *name, const char *value, int reconfiguring) -{ - if (strcmp(name, "Q1") != 0) - return false; - - int old_magic1 = magic1; - - magic1 = atoi(value); - - if (!reconfiguring) - return true; - - if (old_magic1 < magic1) { - /* - * This is because shm.nbufs is computed at startup, when - * we call shmget(). We can't increase the Q1/Q2 parameters - * beyond their initial values because then we might have - * more "Q2 messages" than shared memory chunks, and this - * will cause an assertion in storeDiskdShmGet(). - */ - /* TODO: have DiskdIO hold a link to the swapdir, to allow detailed reporting again */ - debug(3, 1) ("WARNING: cannot increase cache_dir Q1 value while Squid is running.\n"); - magic1 = old_magic1; - return true; - } - - if (old_magic1 != magic1) - debug(3, 1) ("cache_dir new Q1 value '%d'\n", - magic1); - - return true; -} - -void -DiskdIO::optionQ1Dump(StoreEntry * e) const -{ - storeAppendPrintf(e, " Q1=%d", magic1); -} - -bool -DiskdIO::optionQ2Parse(const char *name, const char *value, int reconfiguring) -{ - if (strcmp(name, "Q2") != 0) - return false; - - int old_magic2 = magic2; - - magic2 = atoi(value); - - if (!reconfiguring) - return true; - - if (old_magic2 < magic2) { - /* See comments in Q1 function above */ - debug(3, 1) ("WARNING: cannot increase cache_dir Q2 value while Squid is running.\n"); - magic2 = old_magic2; - return true; - } - - if (old_magic2 != magic2) - debug(3, 1) ("cache_dir new Q2 value '%d'\n", - magic2); - - return true; -} - -void -DiskdIO::optionQ2Dump(StoreEntry * e) const -{ - storeAppendPrintf(e, " Q2=%d", magic2); -} - -void -DiskdIO::init() -{ - int x; - int rfd; - int ikey; - const char *args[5]; - char skey1[32]; - char skey2[32]; - char skey3[32]; - - ikey = (getpid() << 10) + (instanceID << 2); - ikey &= 0x7fffffff; - smsgid = msgget((key_t) ikey, 0700 | IPC_CREAT); - - if (smsgid < 0) { - debug(50, 0) ("storeDiskdInit: msgget: %s\n", xstrerror()); - fatal("msgget failed"); - } - - rmsgid = msgget((key_t) (ikey + 1), 0700 | IPC_CREAT); - - if (rmsgid < 0) { - debug(50, 0) ("storeDiskdInit: msgget: %s\n", xstrerror()); - fatal("msgget failed"); - } - - shm.init(ikey, magic2); - snprintf(skey1, 32, "%d", ikey); - snprintf(skey2, 32, "%d", ikey + 1); - snprintf(skey3, 32, "%d", ikey + 2); - args[0] = "diskd"; - args[1] = skey1; - args[2] = skey2; - args[3] = skey3; - args[4] = NULL; - x = ipcCreate(IPC_STREAM, - Config.Program.diskd, - args, - "diskd", - &rfd, - &wfd); - - if (x < 0) - fatalf("execl: %s", Config.Program.diskd); - - if (rfd != wfd) - comm_close(rfd); - - fd_note(wfd, "squid -> diskd"); - - commSetTimeout(wfd, -1, NULL, NULL); - - commSetNonBlocking(wfd); - - comm_quick_poll_required(); -} - -/* - * Sync any pending data. We just sit around and read the queue - * until the data has finished writing. - */ -void -DiskdIO::sync() -{ - static time_t lastmsg = 0; - - while (away > 0) { - if (squid_curtime > lastmsg) { - debug(47, 1) ("storeDiskdDirSync: %d messages away\n", - away); - lastmsg = squid_curtime; - } - - callback(); - } -} - - -/* - * Handle callbacks. If we have more than magic2 requests away, we block - * until the queue is below magic2. Otherwise, we simply return when we - * don't get a message. - */ -int -DiskdIO::callback() -{ - diomsg M; - int x; - int retval = 0; - - DiskdIO *DIO = this;//dynamic_cast(IO); - - if (DIO->away >= DIO->magic2) { - diskd_stats.block_queue_len++; - retval = 1; - /* We might not have anything to do, but our queue - * is full.. */ - } - - if (diskd_stats.sent_count - diskd_stats.recv_count > - diskd_stats.max_away) { - diskd_stats.max_away = diskd_stats.sent_count - diskd_stats.recv_count; - } - - while (1) { -#ifdef ALWAYS_ZERO_BUFFERS - memset(&M, '\0', sizeof(M)); -#endif - - x = msgrcv(DIO->rmsgid, &M, msg_snd_rcv_sz, 0, IPC_NOWAIT); - - if (x < 0) - break; - else if (x != msg_snd_rcv_sz) { - debug(47, 1) ("storeDiskdDirCallback: msgget returns %d\n", - x); - break; - } - - diskd_stats.recv_count++; - --DIO->away; - DIO->storeDiskdHandle(&M); - retval = 1; /* Return that we've actually done some work */ - - if (M.shm_offset > -1) - DIO->shm.put ((off_t) M.shm_offset); - } - - return retval; -} - -void -DiskdIO::statfs(StoreEntry & sentry)const -{ - storeAppendPrintf(&sentry, "Pending operations: %d\n", away); -} - -DiskdIOModule::DiskdIOModule() : initialised(false) {} - -DiskdIOModule & -DiskdIOModule::GetInstance() -{ - if (!Instance) - Instance = new DiskdIOModule; - - return *Instance; -} - -void -DiskdIOModule::init() -{ - /* We may be reused - for instance in coss - eventually. - * When we do, we either need per-using-module stats ( - * no singleton pattern), or we need to refcount the - * initialisation level and handle multiple clients. - * RBC - 20030718. - */ - assert(!initialised); - memset(&diskd_stats, '\0', sizeof(diskd_stats)); - cachemgrRegister("diskd", "DISKD Stats", storeDiskdStats, 0, 1); - - debug(47, 1) ("diskd started\n"); - initialised = true; -} - -void -DiskdIOModule::shutdown() -{ - initialised = false; -} - -UFSStrategy * -DiskdIOModule::createSwapDirIOStrategy() -{ - return new DiskdIO; -} - -DiskdIOModule *DiskdIOModule::Instance = NULL; - -diskd_stats_t diskd_stats; - -void -storeDiskdStats(StoreEntry * sentry) -{ - storeAppendPrintf(sentry, "sent_count: %d\n", diskd_stats.sent_count); - storeAppendPrintf(sentry, "recv_count: %d\n", diskd_stats.recv_count); - storeAppendPrintf(sentry, "max_away: %d\n", diskd_stats.max_away); - storeAppendPrintf(sentry, "max_shmuse: %d\n", diskd_stats.max_shmuse); - storeAppendPrintf(sentry, "open_fail_queue_len: %d\n", diskd_stats.open_fail_queue_len); - storeAppendPrintf(sentry, "block_queue_len: %d\n", diskd_stats.block_queue_len); - diskd_stats.max_away = diskd_stats.max_shmuse = 0; - storeAppendPrintf(sentry, "\n OPS SUCCESS FAIL\n"); - storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", - "open", diskd_stats.open.ops, diskd_stats.open.success, diskd_stats.open.fail); - storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", - "create", diskd_stats.create.ops, diskd_stats.create.success, diskd_stats.create.fail); - storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", - "close", diskd_stats.close.ops, diskd_stats.close.success, diskd_stats.close.fail); - storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", - "unlink", diskd_stats.unlink.ops, diskd_stats.unlink.success, diskd_stats.unlink.fail); - storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", - "read", diskd_stats.read.ops, diskd_stats.read.success, diskd_stats.read.fail); - storeAppendPrintf(sentry, "%7s %7d %7d %7d\n", - "write", diskd_stats.write.ops, diskd_stats.write.success, diskd_stats.write.fail); -} diff --git a/src/fs/ufs/StoreFSufs.cc b/src/fs/ufs/StoreFSufs.cc index ab74d6ff62..547fb3490d 100644 --- a/src/fs/ufs/StoreFSufs.cc +++ b/src/fs/ufs/StoreFSufs.cc @@ -1,7 +1,7 @@ /* - * $Id: StoreFSufs.cc,v 1.1 2003/07/22 15:23:14 robertc Exp $ + * $Id: StoreFSufs.cc,v 1.2 2004/12/20 16:30:45 robertc Exp $ * * DEBUG: section 47 Store Directory Routines * AUTHOR: Robert Collins @@ -38,7 +38,7 @@ #include "StoreFileSystem.h" #include "fs/ufs/StoreFSufs.h" -#include "fs/ufs/store_ufs.h" +#include "DiskIO/DiskIOModule.h" -static StoreFSufs UfsInstance(UfsIOModule::GetInstance(), "ufs"); +static StoreFSufs UfsInstance("Blocking", "ufs"); diff --git a/src/fs/ufs/StoreFSufs.h b/src/fs/ufs/StoreFSufs.h index cc80047c67..931cfa0eb6 100644 --- a/src/fs/ufs/StoreFSufs.h +++ b/src/fs/ufs/StoreFSufs.h @@ -1,6 +1,6 @@ /* - * $Id: StoreFSufs.h,v 1.2 2004/11/07 23:29:51 hno Exp $ + * $Id: StoreFSufs.h,v 1.3 2004/12/20 16:30:45 robertc Exp $ * * SQUID Web Proxy Cache http://www.squid-cache.org/ * ---------------------------------------------------------- @@ -35,8 +35,9 @@ #define SQUID_STOREFSUFS_H #include "squid.h" +#include "ufscommon.h" -#include "fs/ufs/IOModule.h" +class DiskIOModule; template @@ -45,7 +46,7 @@ class StoreFSufs : public StoreFileSystem public: static StoreFileSystem &GetInstance(); - StoreFSufs(IOModule &, char const *label); + StoreFSufs(char const *DefaultModuleType, char const *label); virtual ~StoreFSufs() {} virtual char const *type() const; @@ -57,12 +58,16 @@ public: StoreFSufs &operator=(StoreFSufs const &); protected: - IOModule &IO; + DiskIOModule *IO; + char const *moduleName; char const *label; + +private: + void checkIO(); }; template -StoreFSufs::StoreFSufs(IOModule &anIO, char const *aLabel) : IO(anIO), label(aLabel) +StoreFSufs::StoreFSufs(char const *defaultModuleName, char const *aLabel) : IO(NULL), moduleName(defaultModuleName), label(aLabel) { FsAdd(*this); } @@ -78,8 +83,9 @@ template SwapDir * StoreFSufs::createSwapDir() { - C *result = new C(type()); - result->IO = IO.createSwapDirIOStrategy(); + C *result = new C(type(), moduleName); + checkIO(); + result->IO = new UFSStrategy(IO->createStrategy()); return result; } @@ -87,7 +93,6 @@ template void StoreFSufs::done() { - IO.shutdown(); initialised = false; } @@ -97,7 +102,16 @@ StoreFSufs::setup() { assert(!initialised); initialised = true; - IO.init(); +} + +template +void +StoreFSufs::checkIO() +{ + if (IO) + return; + + IO = DiskIOModule::Find(moduleName); } #endif /* SQUID_STOREFSUFS_H */ diff --git a/src/fs/ufs/store_dir_ufs.cc b/src/fs/ufs/store_dir_ufs.cc index cdfc0013e5..b2a4e314a1 100644 --- a/src/fs/ufs/store_dir_ufs.cc +++ b/src/fs/ufs/store_dir_ufs.cc @@ -1,6 +1,6 @@ /* - * $Id: store_dir_ufs.cc,v 1.66 2004/12/20 14:52:32 robertc Exp $ + * $Id: store_dir_ufs.cc,v 1.67 2004/12/20 16:30:45 robertc Exp $ * * DEBUG: section 47 Store Directory Routines * AUTHOR: Duane Wessels @@ -36,9 +36,11 @@ #include "squid.h" #include "Store.h" #include "fde.h" -#include "store_ufs.h" #include "ufscommon.h" #include "StoreSwapLogData.h" +#include "ConfigOption.h" +#include "DiskIO/DiskIOStrategy.h" +#include "DiskIO/DiskIOModule.h" #include "SwapDir.h" int UFSSwapDir::NumberOfUFSDirs = 0; @@ -130,22 +132,70 @@ UFSSwapDir::parse (int anIndex, char *aPath) parseOptions(0); } -SwapDirOption * -UFSSwapDir::getOptionTree() const +void +UFSSwapDir::changeIO(DiskIOModule *module) +{ + DiskIOStrategy *anIO = module->createStrategy(); + safe_free(ioType); + ioType = xstrdup(module->type()); + + delete IO->io; + IO->io = anIO; + /* Change the IO Options */ + + if (currentIOOptions->options.size() > 2) + delete currentIOOptions->options.pop_back(); + + /* TODO: factor out these 4 lines */ + ConfigOption *ioOptions = IO->io->getOptionTree(); + + if (ioOptions) + currentIOOptions->options.push_back(ioOptions); +} + +bool +UFSSwapDir::optionIOParse(char const *option, const char *value, int reconfiguring) { - SwapDirOption *parentResult = SwapDir::getOptionTree(); - SwapDirOption *ioOptions = IO->getOptionTree(); + if (strcmp(option, "IOEngine") != 0) + return false; - if (!ioOptions) - return parentResult; + if (reconfiguring) + /* silently ignore this */ + return true; + + if (!value) + self_destruct(); - SwapDirOptionVector *result = new SwapDirOptionVector(); + DiskIOModule *module = DiskIOModule::Find(value); - result->options.push_back(parentResult); + if (!module) + self_destruct(); - result->options.push_back(ioOptions); + changeIO(module); - return result; + return true; +} + +void +UFSSwapDir::optionIODump(StoreEntry * e) const +{ + storeAppendPrintf(e, " IOEngine=%s", ioType); +} + +ConfigOption * +UFSSwapDir::getOptionTree() const +{ + ConfigOption *parentResult = SwapDir::getOptionTree(); + + currentIOOptions->options.push_back(parentResult); + currentIOOptions->options.push_back(new ConfigOptionAdapter(*const_cast(this), &UFSSwapDir::optionIOParse, &UFSSwapDir::optionIODump)); + + ConfigOption *ioOptions = IO->io->getOptionTree(); + + if (ioOptions) + currentIOOptions->options.push_back(ioOptions); + + return currentIOOptions; } /* @@ -154,6 +204,8 @@ UFSSwapDir::getOptionTree() const void UFSSwapDir::init() { + /* Parsing is finished - force to NULL, don't delete */ + currentIOOptions = NULL; static int started_clean_event = 0; static const char *errmsg = "\tFailed to verify one of the swap directories, Check cache.log\n" @@ -185,7 +237,7 @@ UFSSwapDir::newFileSystem() createSwapSubDirs(); } -UFSSwapDir::UFSSwapDir(char const *aType) : SwapDir(aType), IO(NULL), map(NULL), suggest(0), swaplog_fd (-1) {} +UFSSwapDir::UFSSwapDir(char const *aType, const char *anIOType) : SwapDir(aType), IO(NULL), map(NULL), suggest(0), swaplog_fd (-1), currentIOOptions(new ConfigOptionVector()), ioType(xstrdup(anIOType)) {} UFSSwapDir::~UFSSwapDir() { @@ -200,6 +252,8 @@ UFSSwapDir::~UFSSwapDir() delete IO; IO = NULL; + + safe_free(ioType); } void diff --git a/src/fs/ufs/store_io_ufs.cc b/src/fs/ufs/store_io_ufs.cc index e6f3641812..7916c63bf2 100644 --- a/src/fs/ufs/store_io_ufs.cc +++ b/src/fs/ufs/store_io_ufs.cc @@ -1,6 +1,6 @@ /* - * $Id: store_io_ufs.cc,v 1.27 2004/12/20 14:52:32 robertc Exp $ + * $Id: store_io_ufs.cc,v 1.28 2004/12/20 16:30:45 robertc Exp $ * * DEBUG: section 79 Storage Manager UFS Interface * AUTHOR: Duane Wessels @@ -34,50 +34,53 @@ */ #include "squid.h" -#include "store_ufs.h" #include "Store.h" #include "ufscommon.h" +#include "Generic.h" +#include "DiskIO/DiskFile.h" +#include "DiskIO/DiskIOStrategy.h" +#include "DiskIO/ReadRequest.h" +#include "DiskIO/WriteRequest.h" #include "SwapDir.h" -UfsIO UfsIO::Instance; bool -UfsIO::shedLoad() +UFSStrategy::shedLoad() { - return false; + return io->shedLoad(); } int -UfsIO::load() +UFSStrategy::load() { - /* Return 999 (99.9%) constant load */ - return 999; + return io->load(); +} + +UFSStrategy::UFSStrategy (DiskIOStrategy *anIO) : io(anIO) +{} + +UFSStrategy::~UFSStrategy () +{ + delete io; } StoreIOState::Pointer -UfsIO::createState(SwapDir *SD, StoreEntry *e, STIOCB * callback, void *callback_data) const +UFSStrategy::createState(SwapDir *SD, StoreEntry *e, STIOCB * callback, void *callback_data) const { return new UFSStoreState (SD, e, callback, callback_data); } DiskFile::Pointer -UfsIO::newFile (char const *path) +UFSStrategy::newFile (char const *path) { - return new UFSFile (path); + return io->newFile(path); } + void -UfsIO::unlinkFile(char const *path) +UFSStrategy::unlinkFile(char const *path) { -#if USE_UNLINKD - unlinkdUnlink(path); -#elif USE_TRUNCATE - - truncate(path, 0); -#else - - ::unlink(path); -#endif + io->unlinkFile(path); } CBDATA_CLASS_INIT(UFSStoreState); @@ -95,105 +98,6 @@ UFSStoreState::operator delete (void *address) cbdataFree(address); } -CBDATA_CLASS_INIT(UFSFile); -void * -UFSFile::operator new (size_t) -{ - CBDATA_INIT_TYPE(UFSFile); - UFSFile *result = cbdataAlloc(UFSFile); - /* Mark result as being owned - we want the refcounter to do the delete - * call */ - return cbdataReference(result); -} - -void -UFSFile::operator delete (void *address) -{ - UFSFile *t = static_cast(address); - cbdataFree(address); - /* And allow the memory to be freed */ - cbdataReferenceDone (t); -} - -UFSFile::UFSFile (char const *aPath) : fd (-1), closed (true), error_(false) -{ - assert (aPath); - debug (79,3)("UFSFile::UFSFile: %s\n", aPath); - path_ = xstrdup (aPath); -} - -UFSFile::~UFSFile() -{ - safe_free (path_); - doClose(); -} - -void -UFSFile::open (int flags, mode_t mode, IORequestor::Pointer callback) -{ - /* Simulate async calls */ - fd = file_open(path_ , flags); - ioRequestor = callback; - - if (fd < 0) { - debug(79, 1) ("UFSFile::open: Failed to open %s (%s)\n", path_, xstrerror()); - error(true); - } else { - closed = false; - store_open_disk_fd++; - debug(79, 3) ("UFSFile::open: opened FD %d\n", fd); - } - - callback->ioCompletedNotification(); -} - -void -UFSFile::create (int flags, mode_t mode, IORequestor::Pointer callback) -{ - /* We use the same logic path for open */ - open(flags, mode, callback); -} - - -void UFSFile::doClose() -{ - if (fd > -1) { - closed = true; - file_close(fd); - store_open_disk_fd--; - fd = -1; - } -} - -void -UFSFile::close () -{ - debug (79,3)("UFSFile::close: %p closing for %p\n", this, ioRequestor.getRaw()); - doClose(); - assert (ioRequestor.getRaw()); - ioRequestor->closeCompleted(); -} - -bool -UFSFile::canRead() const -{ - return fd > -1; -} - -bool -UFSFile::error() const -{ - if ((fd < 0 && !closed) || error_) - return true; - - return false; -} - -void UFSFile::error(bool const &aBool) -{ - error_ = aBool; -} - void UFSStoreState::ioCompletedNotification() { @@ -293,37 +197,9 @@ UFSStoreState::read_(char *buf, size_t size, off_t offset, STRCB * callback, voi offset_ = offset; read_buf = buf; reading = true; - theFile->read(buf, offset, size); -} - -void -UFSFile::read(char *buf, off_t offset, size_t size) -{ - assert (fd > -1); - assert (ioRequestor.getRaw()); - file_read(fd, buf, size, offset, ReadDone, this); + theFile->read(new ReadRequest(buf,offset,size)); } -void -UFSFile::ReadDone(int fd, const char *buf, int len, int errflag, void *my_data) -{ - UFSFile *myFile = static_cast(my_data); - assert (myFile); - myFile->readDone (fd, buf, len, errflag); -} - -void -UFSFile::write(char const *buf, size_t size, off_t offset, FREE *free_func) -{ - debug(79, 3) ("storeUfsWrite: FD %d\n",fd); - file_write(fd, - offset, - (char *)buf, - size, - WriteDone, - this, - free_func); -} void UFSStoreState::write(char const *buf, size_t size, off_t offset, FREE * free_func) @@ -337,42 +213,13 @@ UFSStoreState::write(char const *buf, size_t size, off_t offset, FREE * free_fun } writing = true; - theFile->write(buf,size,offset,free_func); -} - -bool -UFSFile::ioInProgress()const -{ - /* IO is never pending with UFS */ - return false; -} - -/* === STATIC =========================================================== */ - -void -UFSFile::readDone(int rvfd, const char *buf, int len, int errflag) -{ - debug (79,3)("UFSFile::readDone: FD %d\n",rvfd); - assert (fd == rvfd); - - ssize_t rlen; - - if (errflag) { - debug(79, 3) ("UFSFile::readDone: got failure (%d)\n", errflag); - rlen = -1; - } else { - rlen = (ssize_t) len; - } - - if (errflag == DISK_EOF) - errflag = DISK_OK; /* EOF is signalled by len == 0, not errors... */ - - ioRequestor->readCompleted(buf, rlen, errflag); + theFile->write(new WriteRequest(buf, offset, size, free_func)); } void -UFSStoreState::readCompleted(const char *buf, int len, int errflag) +UFSStoreState::readCompleted(const char *buf, int len, int errflag, RefCount result) { + assert (result.getRaw()); reading = false; debug(79, 3) ("storeDiskdReadDone: dirno %d, fileno %08x len %d\n", swap_dirn, swap_filen, len); @@ -410,31 +257,7 @@ UFSStoreState::readCompleted(const char *buf, int len, int errflag) } void -UFSFile::WriteDone (int fd, int errflag, size_t len, void *me) -{ - UFSFile *aFile = static_cast(me); - aFile->writeDone (fd, errflag, len); -} - -void -UFSFile::writeDone(int rvfd, int errflag, size_t len) -{ - assert (rvfd == fd); - debug(79, 3) ("storeUfsWriteDone: FD %d, len %ld\n", - fd, (long int) len); - - if (errflag) { - debug(79, 0) ("storeUfsWriteDone: got failure (%d)\n", errflag); - doClose(); - ioRequestor->writeCompleted (DISK_ERROR,0); - return; - } - - ioRequestor->writeCompleted(DISK_OK, len); -} - -void -UFSStoreState::writeCompleted(int errflag, size_t len) +UFSStoreState::writeCompleted(int errflag, size_t len, RefCount writeRequest) { debug(79, 3) ("storeUfsWriteDone: dirno %d, fileno %08X, len %ld\n", swap_dirn, swap_filen, (long int) len); @@ -589,11 +412,6 @@ UFSStrategy::open(SwapDir * SD, StoreEntry * e, STFNCB * file_callback, assert (((UFSSwapDir *)SD)->IO == this); debug(79, 3) ("UFSStrategy::open: fileno %08X\n", e->swap_filen); - if (shedLoad()) { - openFailed(); - return NULL; - } - /* to consider: make createstate a private UFSStrategy call */ StoreIOState::Pointer sio = createState (SD, e, callback, callback_data); @@ -607,6 +425,9 @@ UFSStrategy::open(SwapDir * SD, StoreEntry * e, STFNCB * file_callback, DiskFile::Pointer myFile = newFile (path); + if (myFile.getRaw() == NULL) + return NULL; + state->theFile = myFile; state->opening = true; @@ -628,12 +449,6 @@ UFSStrategy::create(SwapDir * SD, StoreEntry * e, STFNCB * file_callback, sfileno filn = ((UFSSwapDir *)SD)->mapBitAllocate(); debug(79, 3) ("UFSStrategy::create: fileno %08X\n", filn); - if (shedLoad()) { - openFailed(); - ((UFSSwapDir *)SD)->mapBitReset (filn); - return NULL; - } - /* Shouldn't we handle a 'bitmap full' error here? */ StoreIOState::Pointer sio = createState (SD, e, callback, callback_data); @@ -650,6 +465,11 @@ UFSStrategy::create(SwapDir * SD, StoreEntry * e, STFNCB * file_callback, DiskFile::Pointer myFile = newFile (path); + if (myFile.getRaw() == NULL) { + ((UFSSwapDir *)SD)->mapBitReset (filn); + return NULL; + } + state->theFile = myFile; state->creating = true; @@ -667,27 +487,27 @@ UFSStrategy::create(SwapDir * SD, StoreEntry * e, STFNCB * file_callback, return sio; } -UfsIOModule & -UfsIOModule::GetInstance() +int +UFSStrategy::callback() { - if (!Instance) - Instance = new UfsIOModule; - - return *Instance; + return io->callback(); } void -UfsIOModule::init() -{} +UFSStrategy::init() +{ + io->init(); +} void -UfsIOModule::shutdown() -{} +UFSStrategy::sync() +{ + io->sync(); +} -UFSStrategy * -UfsIOModule::createSwapDirIOStrategy() +void +UFSStrategy::statfs(StoreEntry & sentry)const { - return new InstanceToSingletonAdapter(&UfsIO::Instance); + io->statfs(sentry); } -UfsIOModule *UfsIOModule::Instance = NULL; diff --git a/src/fs/ufs/store_ufs.h b/src/fs/ufs/store_ufs.h deleted file mode 100644 index e60643e69d..0000000000 --- a/src/fs/ufs/store_ufs.h +++ /dev/null @@ -1,79 +0,0 @@ -/* - * store_ufs.h - * - * Internal declarations for the ufs routines - */ - -#ifndef __STORE_UFS_H__ -#define __STORE_UFS_H__ - -#include "ufscommon.h" - -class UFSFile : public DiskFile -{ - -public: - void *operator new(size_t); - void operator delete(void *); - UFSFile (char const *path); - ~UFSFile(); - virtual void open (int, mode_t, IORequestor::Pointer); - virtual void create (int, mode_t, IORequestor::Pointer); - virtual void read(char *, off_t, size_t); - virtual void write(char const *buf, size_t size, off_t offset, FREE *free_func); - virtual void close (); - virtual bool error() const; - virtual int getFD() const { return fd;} - - virtual bool canRead() const; - virtual bool ioInProgress()const; - -private: - static DRCB ReadDone; - static DWCB WriteDone; - CBDATA_CLASS(UFSFile); - int fd; - bool closed; - void error (bool const &); - bool error_; - char const *path_; - IORequestor::Pointer ioRequestor; - void doClose(); - void readDone(int fd, const char *buf, int len, int errflag); - void writeDone(int fd, int errflag, size_t len); -}; - -#include "SwapDir.h" -/* - * Store IO stuff - */ -/* For things that aren't factored well yet */ - -class UfsIO : public UFSStrategy -{ - -public: - virtual bool shedLoad(); - virtual int load(); - virtual StoreIOState::Pointer createState(SwapDir *SD, StoreEntry *e, STIOCB * callback, void *callback_data) const; - virtual DiskFile::Pointer newFile (char const *path); - virtual void unlinkFile (char const *); - static UfsIO Instance; -}; - -#include "fs/ufs/IOModule.h" - -class UfsIOModule : public IOModule -{ - -public: - static UfsIOModule &GetInstance(); - virtual void init(); - virtual void shutdown(); - virtual UFSStrategy *createSwapDirIOStrategy(); - -private: - static UfsIOModule *Instance; -}; - -#endif diff --git a/src/ufscommon.cc b/src/fs/ufs/ufscommon.cc similarity index 99% rename from src/ufscommon.cc rename to src/fs/ufs/ufscommon.cc index bbd9021b1e..3946e4af5a 100644 --- a/src/ufscommon.cc +++ b/src/fs/ufs/ufscommon.cc @@ -1,5 +1,5 @@ /* - * $Id: ufscommon.cc,v 1.13 2004/12/20 14:52:28 robertc Exp $ + * $Id: ufscommon.cc,v 1.1 2004/12/20 16:30:45 robertc Exp $ * * DEBUG: section 47 Store Directory Routines * AUTHOR: Robert Collins diff --git a/src/ufscommon.cci b/src/fs/ufs/ufscommon.cci similarity index 96% rename from src/ufscommon.cci rename to src/fs/ufs/ufscommon.cci index 56314c2f1a..436398f225 100644 --- a/src/ufscommon.cci +++ b/src/fs/ufs/ufscommon.cci @@ -1,5 +1,5 @@ /* - * $Id: ufscommon.cci,v 1.2 2003/01/23 20:59:10 robertc Exp $ + * $Id: ufscommon.cci,v 1.1 2004/12/20 16:30:45 robertc Exp $ * * DEBUG: section 47 Store Directory Routines * AUTHOR: Duane Wessels diff --git a/src/ufscommon.h b/src/fs/ufs/ufscommon.h similarity index 69% rename from src/ufscommon.h rename to src/fs/ufs/ufscommon.h index 082ebf98a1..2fd1e3d650 100644 --- a/src/ufscommon.h +++ b/src/fs/ufs/ufscommon.h @@ -1,6 +1,6 @@ /* - * $Id: ufscommon.h,v 1.9 2004/08/30 05:12:31 robertc Exp $ + * $Id: ufscommon.h,v 1.1 2004/12/20 16:30:45 robertc Exp $ * * SQUID Web Proxy Cache http://www.squid-cache.org/ * ---------------------------------------------------------- @@ -44,6 +44,10 @@ class UFSStrategy; +class ConfigOptionVector; + +class DiskIOModule; + class UFSSwapDir : public SwapDir { @@ -52,7 +56,7 @@ public: static int DirClean(int swap_index); static int FilenoBelongsHere(int fn, int F0, int F1, int F2); - UFSSwapDir(char const *aType); + UFSSwapDir(char const *aType, const char *aModuleType); virtual void init(); virtual void newFileSystem(); virtual void dump(StoreEntry &) const; @@ -100,7 +104,7 @@ public: int clean); int validFileno(sfileno filn, int flag) const; int mapBitAllocate(); - virtual SwapDirOption *getOptionTree() const; + virtual ConfigOption *getOptionTree() const; void *fsdata; @@ -130,139 +134,66 @@ private: void createSwapSubDirs(); void dumpEntry(StoreEntry &) const; char *logFile(char const *ext = NULL)const; + void changeIO(DiskIOModule *); + bool optionIOParse(char const *option, const char *value, int reconfiguring); + void optionIODump(StoreEntry * e) const; + ConfigOptionVector *currentIOOptions; + char const *ioType; }; #include "RefCount.h" +#include "DiskIO/IORequestor.h" -class IORequestor : public RefCountable -{ - -public: - typedef RefCount Pointer; - virtual void ioCompletedNotification() = 0; - virtual void closeCompleted() = 0; - virtual void readCompleted(const char *buf, int len, int errflag) = 0; - virtual void writeCompleted(int errflag, size_t len) = 0; -}; - -class DiskFile : public RefCountable -{ +/* UFS dir specific IO calls */ -public: - typedef RefCount Pointer; - virtual void open (int, mode_t, IORequestor::Pointer) = 0; - virtual void create (int, mode_t, IORequestor::Pointer) = 0; - virtual void read(char *, off_t, size_t) = 0; - virtual void write(char const *buf, size_t size, off_t offset, FREE *free_func) = 0; - virtual void close () = 0; - virtual bool canRead() const = 0; - virtual bool canWrite() const {return true;} - - /* During miogration only */ - virtual int getFD() const {return -1;} - - virtual bool error() const = 0; - - /* Inform callers if there is IO in progress */ - virtual bool ioInProgress() const = 0; -}; +class DiskIOStrategy; -/* UFS dir specific IO calls */ +class DiskFile; class UFSStrategy { public: - virtual bool shedLoad() = 0; - virtual void openFailed(){} + UFSStrategy (DiskIOStrategy *); + virtual ~UFSStrategy (); + /* Not implemented */ + UFSStrategy (UFSStrategy const &); + UFSStrategy &operator=(UFSStrategy const &); - virtual int load(){return -1;} + virtual bool shedLoad(); - virtual StoreIOState::Pointer createState(SwapDir *, StoreEntry *, STIOCB *, void *)const = 0; + virtual int load(); + + StoreIOState::Pointer createState(SwapDir *SD, StoreEntry *e, STIOCB * callback, void *callback_data) const; /* UFS specific */ - virtual DiskFile::Pointer newFile (char const *path) = 0; + virtual RefCount newFile (char const *path); StoreIOState::Pointer open(SwapDir *, StoreEntry *, STFNCB *, STIOCB *, void *); StoreIOState::Pointer create(SwapDir *, StoreEntry *, STFNCB *, STIOCB *, void *); - /* virtual void strategyStats(StoreEntry *sentry) const = 0; */ - /* virtual void dumpCacheDirParams(StoreEntry * e, const char *option) const = 0; */ - virtual SwapDirOption *getOptionTree() const { return NULL;} - - virtual void unlinkFile (char const *) = 0; - virtual void sync() {} - - virtual int callback() { return 0; } - /* Init per-instance logic */ - virtual void init() {} - - /* cachemgr output on the IO instance stats */ - virtual void statfs(StoreEntry & sentry)const {}} - -; - -class IOStrategy -{ - -public: - virtual ~IOStrategy(){} - - /* Can the IO Strategy handle more requests ? */ - virtual bool shedLoad() = 0; - /* What is the current load? 999 = 99.9% */ - virtual int load() = 0; - /* Return a handle for performing IO operations */ - virtual DiskFile::Pointer newFile (char const *path) = 0; - /* flush all IO operations */ - virtual void sync() {} + virtual void unlinkFile (char const *); + virtual void sync(); - /* perform any pending callbacks */ - virtual int callback() { return 0; } + virtual int callback(); /* Init per-instance logic */ - virtual void init() {} + virtual void init(); /* cachemgr output on the IO instance stats */ - virtual void statfs(StoreEntry & sentry)const {}} - -; - -/* RBC 20030718 - use this to provide instance expecting classes a pointer to a - * singleton - */ - -template - -class InstanceToSingletonAdapter : public C -{ - -public: - void *operator new (size_t byteCount) { return ::operator new (byteCount);} - - void operator delete (void *address) { ::operator delete (address);} - - InstanceToSingletonAdapter(C const *instance) : theInstance (instance) {} - - C const * operator-> () const {return theInstance; } - - C * operator-> () {return const_cast(theInstance); } - - C const & operator * () const {return *theInstance; } + virtual void statfs(StoreEntry & sentry)const; - C & operator * () {return *const_cast(theInstance); } - - operator C const * () const {return theInstance;} - - operator C *() {return const_cast(theInstance);} +protected: -private: - C const *theInstance; + friend class UFSSwapDir; + DiskIOStrategy *io; }; /* Common ufs-store-dir logic */ +class ReadRequest; + class UFSStoreState : public storeIOState, public IORequestor { @@ -275,9 +206,9 @@ public: virtual void closeCompleted(); // protected: virtual void ioCompletedNotification(); - virtual void readCompleted(const char *buf, int len, int errflag); - virtual void writeCompleted(int errflag, size_t len); - DiskFile::Pointer theFile; + virtual void readCompleted(const char *buf, int len, int errflag, RefCount); + virtual void writeCompleted(int errflag, size_t len, RefCount); + RefCount theFile; bool opening; bool creating; bool closing; diff --git a/src/http.cc b/src/http.cc index 7ddbfd5f79..273fa26f66 100644 --- a/src/http.cc +++ b/src/http.cc @@ -1,6 +1,6 @@ /* - * $Id: http.cc,v 1.439 2004/12/20 14:52:27 robertc Exp $ + * $Id: http.cc,v 1.440 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 11 Hypertext Transfer Protocol (HTTP) * AUTHOR: Harvest Derived diff --git a/src/ipc.cc b/src/ipc.cc index 0a65568ce6..80208358b6 100644 --- a/src/ipc.cc +++ b/src/ipc.cc @@ -1,6 +1,6 @@ /* - * $Id: ipc.cc,v 1.38 2004/12/20 14:52:27 robertc Exp $ + * $Id: ipc.cc,v 1.39 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 54 Interprocess Communication * AUTHOR: Duane Wessels diff --git a/src/ipcache.cc b/src/ipcache.cc index 36f370432e..d3a70c52bd 100644 --- a/src/ipcache.cc +++ b/src/ipcache.cc @@ -1,6 +1,6 @@ /* - * $Id: ipcache.cc,v 1.248 2004/12/20 14:52:27 robertc Exp $ + * $Id: ipcache.cc,v 1.249 2004/12/20 16:30:35 robertc Exp $ * * DEBUG: section 14 IP Cache * AUTHOR: Harvest Derived diff --git a/src/main.cc b/src/main.cc index 3d531882cb..0149c47d45 100644 --- a/src/main.cc +++ b/src/main.cc @@ -1,6 +1,6 @@ /* - * $Id: main.cc,v 1.395 2004/12/20 14:52:27 robertc Exp $ + * $Id: main.cc,v 1.396 2004/12/20 16:30:36 robertc Exp $ * * DEBUG: section 1 Startup and Main Loop * AUTHOR: Harvest Derived @@ -44,6 +44,7 @@ #include "ACL.h" #include "htcp.h" #include "StoreFileSystem.h" +#include "DiskIO/DiskIOModule.h" #include "comm.h" #if USE_WIN32_SERVICE @@ -988,6 +989,9 @@ main(int argc, char **argv) storeFsInit(); /* required for config parsing */ + /* May not be needed for parsing, have not audited for such */ + DiskIOModule::SetupAllModules(); + /* Shouldn't be needed for config parsing, but have not audited for such */ StoreFileSystem::SetupAllFs(); @@ -1505,6 +1509,7 @@ SquidShutdown(void *unused) storeDirSync(); /* Flush log close */ StoreFileSystem::FreeAllFs(); + DiskIOModule::FreeAllModules(); #if PURIFY || XMALLOC_TRACE configFreeMemory(); diff --git a/src/peer_select.cc b/src/peer_select.cc index 644a6d99b5..1c92a2fce9 100644 --- a/src/peer_select.cc +++ b/src/peer_select.cc @@ -1,6 +1,6 @@ /* - * $Id: peer_select.cc,v 1.134 2004/12/20 14:52:27 robertc Exp $ + * $Id: peer_select.cc,v 1.135 2004/12/20 16:30:36 robertc Exp $ * * DEBUG: section 44 Peer Selection Algorithm * AUTHOR: Duane Wessels diff --git a/src/protos.h b/src/protos.h index 2d4b35995f..14280aabcc 100644 --- a/src/protos.h +++ b/src/protos.h @@ -1,6 +1,6 @@ /* - * $Id: protos.h,v 1.500 2004/12/20 14:52:27 robertc Exp $ + * $Id: protos.h,v 1.501 2004/12/20 16:30:36 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ diff --git a/src/redirect.cc b/src/redirect.cc index bc4aab60ac..b16e75bddf 100644 --- a/src/redirect.cc +++ b/src/redirect.cc @@ -1,6 +1,6 @@ /* - * $Id: redirect.cc,v 1.108 2004/12/20 14:52:28 robertc Exp $ + * $Id: redirect.cc,v 1.109 2004/12/20 16:30:36 robertc Exp $ * * DEBUG: section 61 Redirector * AUTHOR: Duane Wessels diff --git a/src/refresh.cc b/src/refresh.cc index 3e1a3fa470..411d21321a 100644 --- a/src/refresh.cc +++ b/src/refresh.cc @@ -1,6 +1,6 @@ /* - * $Id: refresh.cc,v 1.65 2004/12/20 14:52:28 robertc Exp $ + * $Id: refresh.cc,v 1.66 2004/12/20 16:30:36 robertc Exp $ * * DEBUG: section 22 Refresh Calculation * AUTHOR: Harvest Derived diff --git a/src/repl/Makefile.in b/src/repl/Makefile.in index ada2da28cb..c219a54571 100644 --- a/src/repl/Makefile.in +++ b/src/repl/Makefile.in @@ -16,7 +16,7 @@ # Makefile for storage modules in the Squid Object Cache server # -# $Id: Makefile.in,v 1.45 2004/12/20 14:52:32 robertc Exp $ +# $Id: Makefile.in,v 1.46 2004/12/20 16:30:47 robertc Exp $ # SOURCES = $(libheap_a_SOURCES) $(liblru_a_SOURCES) @@ -120,6 +120,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -207,9 +210,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@ diff --git a/src/repl/heap/store_repl_heap.cc b/src/repl/heap/store_repl_heap.cc index 1cd9f6aee8..fde8227e16 100644 --- a/src/repl/heap/store_repl_heap.cc +++ b/src/repl/heap/store_repl_heap.cc @@ -1,6 +1,6 @@ /* - * $Id: store_repl_heap.cc,v 1.16 2004/12/20 14:52:32 robertc Exp $ + * $Id: store_repl_heap.cc,v 1.17 2004/12/20 16:30:47 robertc Exp $ * * DEBUG: section ? HEAP based removal policies * AUTHOR: Henrik Nordstrom diff --git a/src/squid.h b/src/squid.h index a7e976ec0c..37d3b81e1c 100644 --- a/src/squid.h +++ b/src/squid.h @@ -1,6 +1,6 @@ /* - * $Id: squid.h,v 1.240 2004/12/20 14:52:28 robertc Exp $ + * $Id: squid.h,v 1.241 2004/12/20 16:30:36 robertc Exp $ * * AUTHOR: Duane Wessels * diff --git a/src/ssl_support.cc b/src/ssl_support.cc index 6add9a82e8..212977f863 100644 --- a/src/ssl_support.cc +++ b/src/ssl_support.cc @@ -1,6 +1,6 @@ /* - * $Id: ssl_support.cc,v 1.19 2004/12/20 14:52:28 robertc Exp $ + * $Id: ssl_support.cc,v 1.20 2004/12/20 16:30:36 robertc Exp $ * * AUTHOR: Benno Rice * DEBUG: section 83 SSL accelerator support diff --git a/src/stat.cc b/src/stat.cc index 5fe7faa91c..a06c254e4c 100644 --- a/src/stat.cc +++ b/src/stat.cc @@ -1,5 +1,5 @@ /* - * $Id: stat.cc,v 1.383 2004/12/20 14:52:28 robertc Exp $ + * $Id: stat.cc,v 1.384 2004/12/20 16:30:36 robertc Exp $ * * DEBUG: section 18 Cache Manager Statistics * AUTHOR: Harvest Derived diff --git a/src/store_client.cc b/src/store_client.cc index 8746945e41..1e1763b745 100644 --- a/src/store_client.cc +++ b/src/store_client.cc @@ -1,6 +1,6 @@ /* - * $Id: store_client.cc,v 1.137 2004/12/20 14:52:28 robertc Exp $ + * $Id: store_client.cc,v 1.138 2004/12/20 16:30:36 robertc Exp $ * * DEBUG: section 90 Storage Manager Client-Side Interface * AUTHOR: Duane Wessels diff --git a/src/structs.h b/src/structs.h index 8634903895..af521c1c38 100644 --- a/src/structs.h +++ b/src/structs.h @@ -1,6 +1,6 @@ /* - * $Id: structs.h,v 1.497 2004/12/20 14:52:28 robertc Exp $ + * $Id: structs.h,v 1.498 2004/12/20 16:30:36 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ @@ -1359,8 +1359,12 @@ struct _iostats /* Removal policies */ -struct _RemovalPolicyNode +class RemovalPolicyNode { + +public: + RemovalPolicyNode() : data(NULL) {} + void *data; }; diff --git a/src/test_cache_digest.cc b/src/test_cache_digest.cc index 7b91b4236c..459813fc2b 100644 --- a/src/test_cache_digest.cc +++ b/src/test_cache_digest.cc @@ -1,6 +1,6 @@ /* - * $Id: test_cache_digest.cc,v 1.32 2004/12/20 14:52:28 robertc Exp $ + * $Id: test_cache_digest.cc,v 1.33 2004/12/20 16:30:36 robertc Exp $ * * AUTHOR: Alex Rousskov * diff --git a/src/tools.cc b/src/tools.cc index ee66d7c155..cc0e07d2c5 100644 --- a/src/tools.cc +++ b/src/tools.cc @@ -1,6 +1,6 @@ /* - * $Id: tools.cc,v 1.246 2004/12/20 14:52:28 robertc Exp $ + * $Id: tools.cc,v 1.247 2004/12/20 16:30:37 robertc Exp $ * * DEBUG: section 21 Misc Functions * AUTHOR: Harvest Derived diff --git a/src/typedefs.h b/src/typedefs.h index 66fb8c8621..6cecef275f 100644 --- a/src/typedefs.h +++ b/src/typedefs.h @@ -1,6 +1,6 @@ /* - * $Id: typedefs.h,v 1.175 2004/08/30 03:28:59 robertc Exp $ + * $Id: typedefs.h,v 1.176 2004/12/20 16:30:37 robertc Exp $ * * * SQUID Web Proxy Cache http://www.squid-cache.org/ @@ -259,8 +259,6 @@ typedef struct _RemovalPolicyWalker RemovalPolicyWalker; typedef struct _RemovalPurgeWalker RemovalPurgeWalker; -typedef struct _RemovalPolicyNode RemovalPolicyNode; - typedef struct _RemovalPolicySettings RemovalPolicySettings; #if SQUID_SNMP diff --git a/src/win32.cc b/src/win32.cc index 93429c7942..436f829a59 100644 --- a/src/win32.cc +++ b/src/win32.cc @@ -1,6 +1,6 @@ /* - * $Id: win32.cc,v 1.12 2004/12/20 14:52:28 robertc Exp $ + * $Id: win32.cc,v 1.13 2004/12/20 16:30:37 robertc Exp $ * * * * * * * * * Legal stuff * * * * * * * * diff --git a/test-suite/Makefile.in b/test-suite/Makefile.in index 256e1291f7..3609d3bfa5 100644 --- a/test-suite/Makefile.in +++ b/test-suite/Makefile.in @@ -17,7 +17,7 @@ # # Makefile for the Squid Object Cache server # -# $Id: Makefile.in,v 1.42 2004/12/20 14:52:32 robertc Exp $ +# $Id: Makefile.in,v 1.43 2004/12/20 16:30:49 robertc Exp $ # SOURCES = $(ESIExpressions_SOURCES) $(MemPoolTest_SOURCES) $(StackTest_SOURCES) $(VirtualDeleteOperator_SOURCES) $(debug_SOURCES) $(http_range_test_SOURCES) $(mem_hdr_test_SOURCES) $(mem_node_test_SOURCES) membanger.c $(refcount_SOURCES) $(rfc1738_SOURCES) $(splay_SOURCES) $(syntheticoperators_SOURCES) tcp-banger2.c @@ -191,6 +191,9 @@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DIGEST_AUTH_HELPERS = @DIGEST_AUTH_HELPERS@ +DISK_LIBS = @DISK_LIBS@ +DISK_LINKOBJS = @DISK_LINKOBJS@ +DISK_PROGRAMS = @DISK_PROGRAMS@ ECHO = @ECHO@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ @@ -278,9 +281,7 @@ SQUID_CXXFLAGS = @SQUID_CXXFLAGS@ SSLLIB = @SSLLIB@ STORE_LIBS = @STORE_LIBS@ STORE_LINKOBJS = @STORE_LINKOBJS@ -STORE_MODULE_SUBDIRS = @STORE_MODULE_SUBDIRS@ STORE_OBJS = @STORE_OBJS@ -STORE_PROGRAMS = @STORE_PROGRAMS@ STRIP = @STRIP@ TRUE = @TRUE@ USE_DELAY_POOLS_FALSE = @USE_DELAY_POOLS_FALSE@