From: Willy Tarreau <w@1wt.eu>
Date: Fri, 28 Feb 2020 13:09:12 +0000 (+0100)
Subject: MINOR: rawsock: always mark the FD not ready when we're certain it happens
X-Git-Tag: v2.2-dev4~105
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=8dd348c90c5c6c5c7ea4a304a1e44692cd4a84ae;p=thirdparty%2Fhaproxy.git

MINOR: rawsock: always mark the FD not ready when we're certain it happens

This partially reverts commit 1113116b4a ("MEDIUM: raw-sock: remove
obsolete calls to fd_{cant,cond,done}_{send,recv}") so that we can mark
the FD not ready as required since commit 19bc201c9f ("MEDIUM: connection:
remove the intermediary polling state from the connection"). Indeed, with
the removal of the latter we don't have any other reliable indication that
the FD is blocked, which explains why there are so many EAGAIN in traces.

It's worth noting that a short read or a short write are also reliable
indicators of exhausted buffers and are even documented as such in the
epoll man page in case of edge-triggered mode. That's why we also report
the FD as blocked in such a case.

With this change we completely got rid of EAGAIN in keep-alive tests, but
they were expectedly transferred to epoll_ctl:

  $ ./h1load -n 100000 -t 4 -c 1000 -T 20 -F 127.0.0.1:8001/?s=1k/t=20

  before:
   266331 epoll_ctl 1
   200000 sendto 1
   200000 recvfrom 1
   135757 recvfrom -1
     8626 epoll_wait 1

  after:
   394865 epoll_ctl 1
   200000 sendto 1
   200000 recvfrom 1
    10748 epoll_wait 1
     1999 recvfrom -1
---

diff --git a/src/raw_sock.c b/src/raw_sock.c
index ab03a8fd69..ca22d206e1 100644
--- a/src/raw_sock.c
+++ b/src/raw_sock.c
@@ -118,6 +118,8 @@ int raw_sock_to_pipe(struct connection *conn, void *xprt_ctx, struct pipe *pipe,
 					conn->flags |= CO_FL_WAIT_ROOM;
 					break;
 				}
+				/* socket buffer exhausted */
+				fd_cant_recv(conn->handle.fd);
 				break;
 			}
 			else if (errno == ENOSYS || errno == EINVAL || errno == EBADF) {
@@ -196,6 +198,7 @@ int raw_sock_from_pipe(struct connection *conn, void *xprt_ctx, struct pipe *pip
 
 		if (ret <= 0) {
 			if (ret == 0 || errno == EAGAIN) {
+				fd_cant_send(conn->handle.fd);
 				break;
 			}
 			else if (errno == EINTR)
@@ -274,6 +277,9 @@ static size_t raw_sock_to_buf(struct connection *conn, void *xprt_ctx, struct bu
 			b_add(buf, ret);
 			done += ret;
 			if (ret < try) {
+				/* socket buffer exhausted */
+				fd_cant_recv(conn->handle.fd);
+
 				/* unfortunately, on level-triggered events, POLL_HUP
 				 * is generally delivered AFTER the system buffer is
 				 * empty, unless the poller supports POLL_RDHUP. If
@@ -300,6 +306,8 @@ static size_t raw_sock_to_buf(struct connection *conn, void *xprt_ctx, struct bu
 			goto read0;
 		}
 		else if (errno == EAGAIN || errno == ENOTCONN) {
+			/* socket buffer exhausted */
+			fd_cant_recv(conn->handle.fd);
 			break;
 		}
 		else if (errno != EINTR) {
@@ -382,13 +390,16 @@ static size_t raw_sock_from_buf(struct connection *conn, void *xprt_ctx, const s
 			done += ret;
 
 			/* if the system buffer is full, don't insist */
-			if (ret < try)
+			if (ret < try) {
+				fd_cant_send(conn->handle.fd);
 				break;
+			}
 			if (!count)
 				fd_stop_send(conn->handle.fd);
 		}
 		else if (ret == 0 || errno == EAGAIN || errno == ENOTCONN || errno == EINPROGRESS) {
 			/* nothing written, we need to poll for write first */
+			fd_cant_send(conn->handle.fd);
 			break;
 		}
 		else if (errno != EINTR) {