From: Amos Jeffries Date: Thu, 26 Feb 2015 13:19:35 +0000 (-0800) Subject: Parser-NG: HTTP Response Parser upgrade X-Git-Tag: merge-candidate-3-v1~240 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=6fe0370;p=thirdparty%2Fsquid.git Parser-NG: HTTP Response Parser upgrade 1) convert the HTTP server read buffer to an SBuf using the same design and Comm::Read API implemented previously for the client connections. The buffer remains default initialized at 16KB per connection but is no longer absolutely limited to 256KB. Instead it is limited by configuration options controlling maximum server input sizes on read_ahead_gap and response message headers. The Client API has been extended with a new method to estimate size requirements of an SBuf I/O buffer. Modelled on and deprecating the existing MemBuf estimator. The Comm::ReadNow() API is extended to allow limited-size read(2) operations by setting the CommIoCbParams::size parameter. The HttpStateData buffer is partially detached from StoreEntry::delayAwareRead() API due to requirements of the Comm::ReadNow() API. Instead StoreEntry::bytesWanted() is used directly to determine read(2) size, and DeferredRead are generated only when ReadNow() is actually and immediately to be deferred. Theoretically this means less read operations get deferred in some high load cases. Practically it means there is no longer an AsyncCall queue plus socket wait delay between delay_pools promising a read size, doing the read(2), and accounting for the bytes received - accuracy should be much improved under load. This introduces one temporary performance regression converting the SBuf content to MemBuf for chunked decoder to process. 2) add Http1::ResponseParser class for parsing HTTP response messages. Modelled on the same design as used for the HTTP RequestParser, and inheriting from a mutual parent Http1::Parser. The Parser is Tokeniser based, incremental and 'consumes' bytes out of the buffer as they are parsed. The Tokenizer int64 API is updated to handle limited-length scans and optional +/- symbols. This Parser class recognises HTTP/1.x and ICY/1 syntax messages. Any unknown syntax input is assumed to be HTTP "0.9" and it will gateway/transform the response to HTTP/1.1. NOTE: these are all semantic actions performed by the code being replaced in (3). Only the form and OO scoping has changed. The mime header block detection operation is generalized into the Http1::Parser for use by both RequestParser and ResponseParser. The request_parse_status error code has also been adapted for shared use. 3) integrate the HTTP1::ResponseParser with HttpStateData server response processing. This is largely code shuffling. Though I have extended the EOF \r\n hack such that it enables Squid to parse truncated response headers now. --- 6fe0370242ce54e81b1dd196fc4468743c3079d1