Willy Tarreau [Thu, 11 Jun 2020 07:14:49 +0000 (09:14 +0200)]
REORG: include: make list-t.h part of the base API
There are list definitions everywhere in the code, let's drop the need
for including list-t.h to declare them. The rest of the list manipulation
is huge however and not needed everywhere so using the list walking macros
still requires to include list.h.
Willy Tarreau [Tue, 9 Jun 2020 07:07:15 +0000 (09:07 +0200)]
CLEANUP: include: tree-wide alphabetical sort of include files
This patch fixes all the leftovers from the include cleanup campaign. There
were not that many (~400 entries in ~150 files) but it was definitely worth
doing it as it revealed a few duplicates.
Willy Tarreau [Thu, 11 Jun 2020 06:33:02 +0000 (08:33 +0200)]
REORG: include: move THREAD_LOCAL and __decl_thread() to compiler.h
Since these are used as type attributes or conditional clauses, they
are used about everywhere and should not require a dependency on
thread.h. Moving them to compiler.h along with other similar statements
like ALIGN() etc looks more logical; this way they become part of the
base API. This allowed to remove thread-t.h from ~12 files, one was
found to only require thread-t and not thread and dict.c was found to
require thread.h.
Willy Tarreau [Thu, 11 Jun 2020 06:14:01 +0000 (08:14 +0200)]
REORG: include: move MAX_THREADS to defaults.h
That's already where MAX_PROCS is set, and we already handle the case of
the default value so there is no reason for placing it in thread.h given
that most call places don't need the rest of the threads definitions. The
include was removed from global-t.h and activity.c.
Willy Tarreau [Thu, 11 Jun 2020 06:22:01 +0000 (08:22 +0200)]
CLEANUP: compiler: add a THREAD_ALIGNED macro and use it where appropriate
Sometimes we need to align a struct member or a struct's size only when
threads are enabled. This is the case on fdtab for example. Instead of
using ugly ifdefs in the code itself, let's have a THREAD_ALIGNED() macro
performing the alignment only when threads are enabled. For now this was
only applied to fd-t.h as it was the only place found.
Willy Tarreau [Fri, 5 Jun 2020 18:04:36 +0000 (20:04 +0200)]
BUILD: reorder objects in the Makefile for faster builds
Splitting large files and changing includes has changed the per-file
build time. After a careful reordering based on build time, we're now
down to 5.8s at -O0 on the PC at -j8 and 2.4-2.6s on the farm at -j120.
Some room for at least one file name was left on each line to ease
future additions.
Willy Tarreau [Fri, 5 Jun 2020 15:27:29 +0000 (17:27 +0200)]
REORG: include: move the error reporting functions to from log.h to errors.h
Most of the files dealing with error reports have to include log.h in order
to access ha_alert(), ha_warning() etc. But while these functions don't
depend on anything, log.h depends on a lot of stuff because it deals with
log-formats and samples. As a result it's impossible not to embark long
dependencies when using ha_warning() or qfprintf().
This patch moves these low-level functions to errors.h, which already
defines the error codes used at the same places. About half of the users
of log.h could be adjusted, sometimes revealing other issues such as
missing tools.h. Interestingly the total preprocessed size shrunk by
4%.
Willy Tarreau [Fri, 5 Jun 2020 14:54:16 +0000 (16:54 +0200)]
CLEANUP: include: move sample_data out of sample-t.h
The struct sample_data is used by pattern, map and vars, and currently
requires to include sample-t which comes with many other dependencies.
Let's move sample_data into its own file to shorten the dependency tree.
This revealed a number of issues in adjacent files which were hidden by
the fact that sample-t.h brought everything that was missing.
Willy Tarreau [Fri, 5 Jun 2020 13:37:34 +0000 (15:37 +0200)]
CLEANUP: include: don't include stddef.h directly
Directly including stddef.h in many files results in it being processed
multiple times while it can be centralized in api-t.h and be guarded
against multiple inclusions. Doing so reduces the number of preprocessed
lines by 1200!
Willy Tarreau [Fri, 5 Jun 2020 10:25:38 +0000 (12:25 +0200)]
REORG: check: move tcpchecks away from check.c
Checks.c remains one of the largest file of the project and it contains
too many things. The tcpchecks code represents half of this file, and
both parts are relatively isolated, so let's move it away into its own
file. We now have tcpcheck.c, tcpcheck{,-t}.h.
Doing so required to export quite a number of functions because check.c
has almost everything made static, which really doesn't help to split!
Willy Tarreau [Fri, 5 Jun 2020 09:40:38 +0000 (11:40 +0200)]
REORG: check: move the e-mail alerting code to mailers.c
check.c is one of the largest file and contains too many things. The
e-mail alerting code is stored there while nothing is in mailers.c.
Let's move this code out. That's only 4% of the code but a good start.
In order to do so, a few tcp-check functions had to be exported.
Willy Tarreau [Fri, 5 Jun 2020 07:05:31 +0000 (09:05 +0200)]
CLEANUP: hpack: export debug functions and move inlines to .h
When building contrib/hpack there is a warning about an unused static
function. Actually it makes no sense to make it static, instead it must
be regularly exported. Similarly there is hpack_dht_get_tail() which is
inlined in the C file and which would make more sense with all other ones
in the H file.
Willy Tarreau [Thu, 4 Jun 2020 22:00:29 +0000 (00:00 +0200)]
REORG: include: move cfgparse.h to haproxy/cfgparse.h
There's no point splitting the file in two since only cfgparse uses the
types defined there. A few call places were updated and cleaned up. All
of them were in C files which register keywords.
There is nothing left in common/ now so this directory must not be used
anymore.
Willy Tarreau [Thu, 4 Jun 2020 21:46:14 +0000 (23:46 +0200)]
REORG: include: move stream.h to haproxy/stream{,-t}.h
This one was not easy because it was embarking many includes with it,
which other files would automatically find. At least global.h, arg.h
and tools.h were identified. 93 total locations were identified, 8
additional includes had to be added.
In the rare files where it was possible to finalize the sorting of
includes by adjusting only one or two extra lines, it was done. But
all files would need to be rechecked and cleaned up now.
It was the last set of files in types/ and proto/ and these directories
must not be reused anymore.
Willy Tarreau [Thu, 4 Jun 2020 21:20:13 +0000 (23:20 +0200)]
REORG: include: move server.h to haproxy/server{,-t}.h
extern struct dict server_name_dict was moved from the type file to the
main file. A handful of inlined functions were moved at the bottom of
the file. Call places were updated to use server-t.h when relevant, or
to simply drop the entry when not needed.
Willy Tarreau [Thu, 4 Jun 2020 20:50:02 +0000 (22:50 +0200)]
REORG: include: move backend.h to haproxy/backend{,-t}.h
The files remained mostly unchanged since they were OK. However, half of
the users didn't need to include them, and about as many actually needed
to have it and used to find functions like srv_currently_usable() through
a long chain that broke when moving the file.
Willy Tarreau [Thu, 4 Jun 2020 20:29:18 +0000 (22:29 +0200)]
REORG: include: move proxy.h to haproxy/proxy{,-t}.h
This one is particularly difficult to split because it provides all the
functions used to manipulate a proxy state and to retrieve names or IDs
for error reporting, and as such, it was included in 73 files (down to
68 after cleanup). It would deserve a small cleanup though the cut points
are not obvious at the moment given the number of structs involved in
the struct proxy itself.
Willy Tarreau [Thu, 4 Jun 2020 20:01:04 +0000 (22:01 +0200)]
REORG: include: move log.h to haproxy/log{,-t}.h
The current state of the logging is a real mess. The main problem is
that almost all files include log.h just in order to have access to
the alert/warning functions like ha_alert() etc, and don't care about
logs. But log.h also deals with real logging as well as log-format and
depends on stream.h and various other things. As such it forces a few
heavy files like stream.h to be loaded early and to hide missing
dependencies depending where it's loaded. Among the missing ones is
syslog.h which was often automatically included resulting in no less
than 3 users missing it.
Among 76 users, only 5 could be removed, and probably 70 don't need the
full set of dependencies.
A good approach would consist in splitting that file in 3 parts:
- one for error output ("errors" ?).
- one for log_format processing
- and one for actual logging.
Willy Tarreau [Thu, 4 Jun 2020 19:21:03 +0000 (21:21 +0200)]
REORG: include: move http_ana.h to haproxy/http_ana{,-t}.h
It was moved without any change, however many callers didn't need it at
all. This was a consequence of the split of proto_http.c into several
parts that resulted in many locations to still reference it.
Willy Tarreau [Thu, 4 Jun 2020 18:19:54 +0000 (20:19 +0200)]
REORG: include: move cli.h to haproxy/cli{,-t}.h
Almost no change except moving the cli_kw struct definition after the
defines. Almost all users had both types&proto included, which is not
surprizing since this code is old and it used to be the norm a decade
ago. These places were cleaned.
Willy Tarreau [Thu, 4 Jun 2020 17:58:55 +0000 (19:58 +0200)]
REORG: include: move stats.h to haproxy/stats{,-t}.h
Just some minor reordering, and the usual cleanup of call places for
those which didn't need it. We don't include the whole tools.h into
stats-t anymore but just tools-t.h.
Willy Tarreau [Thu, 4 Jun 2020 17:42:41 +0000 (19:42 +0200)]
REORG: move applet.h to haproxy/applet{,-t}.h
The type file was slightly tidied. The cli-specific APPCTX_CLI_ST1_* flag
definitions were moved to cli.h. The type file was adjusted to include
buf-t.h and not the huge buf.h. A few call places were fixed because they
did not need this include.
Willy Tarreau [Thu, 4 Jun 2020 17:27:34 +0000 (19:27 +0200)]
REORG: include: split common/uri_auth.h into haproxy/uri_auth{,-t}.h
Initially it looked like this could have been placed into auth.h or
stats.h but it's not the case as it's what makes the link between them
and the HTTP layer. However the file needed to be split in two. Quite
a number of call places were dropped because these were mostly leftovers
from the early days where the stats and cli were packed together.
Willy Tarreau [Thu, 4 Jun 2020 17:11:43 +0000 (19:11 +0200)]
REORG: include: move acl.h to haproxy/acl.h{,-t}.h
The files were moved almost as-is, just dropping arg-t and auth-t from
acl-t but keeping arg-t in acl.h. It was useful to revisit the call places
since a handful of files used to continue to include acl.h while they did
not need it at all. Struct stream was only made a forward declaration
since not otherwise needed.
Willy Tarreau [Thu, 4 Jun 2020 16:46:44 +0000 (18:46 +0200)]
REORG: include: move stick_table.h to haproxy/stick_table{,-t}.h
The stktable_types[] array declaration was moved to the main file as
it had nothing to do in the types. A few declarations were reordered
in the types file so that defines were before the structs. Thread-t
was added since there are a few __decl_thread(). The loss of peers.h
revealed that cfgparse-listen needed it.
Willy Tarreau [Thu, 4 Jun 2020 16:38:21 +0000 (18:38 +0200)]
REORG: include: move peers.h to haproxy/peers{,-t}.h
The cfg_peers external declaration was moved to the main file instead
of the type one. A few types were still missing from the proto, causing
warnings in the functions prototypes (proxy, stick_table).
Willy Tarreau [Thu, 4 Jun 2020 16:21:56 +0000 (18:21 +0200)]
REORG: include: move checks.h to haproxy/check{,-t}.h
All includes that were not absolutely necessary were removed because
checks.h happens to very often be part of dependency loops. A warning
was added about this in check-t.h. The fields, enums and structs were
a bit tidied because it's particularly tedious to find anything there.
It would make sense to split this in two or more files (at least
extract tcp-checks).
The file was renamed to the singular because it was one of the rare
exceptions to have an "s" appended to its name compared to the struct
name.
Willy Tarreau [Thu, 4 Jun 2020 16:02:10 +0000 (18:02 +0200)]
REORG: include: move connection.h to haproxy/connection{,-t}.h
The type file is becoming a mess, half of it is for the proxy protocol,
another good part describes conn_streams and mux ops, it would deserve
being split again. At least it was reordered so that elements are easier
to find, with the PP-stuff left at the end. The MAX_SEND_FD macro was moved
to compat.h as it's said to be the value for Linux.
Willy Tarreau [Thu, 4 Jun 2020 15:25:40 +0000 (17:25 +0200)]
REORG: include: move task.h to haproxy/task{,-t}.h
The TASK_IS_TASKLET() macro was moved to the proto file instead of the
type one. The proto part was a bit reordered to remove a number of ugly
forward declaration of static inline functions. About a tens of C and H
files had their dependency dropped since they were not using anything
from task.h.
Willy Tarreau [Thu, 4 Jun 2020 15:05:57 +0000 (17:05 +0200)]
REORG: include: split global.h into haproxy/global{,-t}.h
global.h was one of the messiest files, it has accumulated tons of
implicit dependencies and declares many globals that make almost all
other file include it. It managed to silence a dependency loop between
server.h and proxy.h by being well placed to pre-define the required
structs, forcing struct proxy and struct server to be forward-declared
in a significant number of files.
It was split in to, one which is the global struct definition and the
few macros and flags, and the rest containing the functions prototypes.
The UNIX_MAX_PATH definition was moved to compat.h.
Willy Tarreau [Thu, 4 Jun 2020 14:06:59 +0000 (16:06 +0200)]
REORG: include: move protocol_buffers.h to haproxy/protobuf{,-t}.h
There is no C file for this one, the code was placed into sample.c which
thus has a dependency on this file which itself includes sample.h. Probably
that it would be wise to split that later.
Willy Tarreau [Thu, 4 Jun 2020 13:33:47 +0000 (15:33 +0200)]
REORG: include: move sample.h to haproxy/sample{,-t}.h
This one is particularly tricky to move because everyone uses it
and it depends on a lot of other types. For example it cannot include
arg-t.h and must absolutely only rely on forward declarations to avoid
dependency loops between vars -> sample_data -> arg. In order to address
this one, it would be nice to split the sample_data part out of sample.h.
Willy Tarreau [Thu, 4 Jun 2020 13:13:30 +0000 (15:13 +0200)]
REORG: include: move payload.h to haproxy/payload.h
There's no type file, it only contains fetch_rdp_cookie_name() and
val_payload_lv() which probably ought to move somewhere else instead
of staying there.
Willy Tarreau [Thu, 4 Jun 2020 13:06:28 +0000 (15:06 +0200)]
REORG: include: move pattern.h to haproxy/pattern{,-t}.h
It was moved as-is, except for extern declaration of pattern_reference.
A few C files used to include it but didn't need it anymore after having
been split apart so this was cleaned.
Willy Tarreau [Thu, 4 Jun 2020 12:07:37 +0000 (14:07 +0200)]
REORG: include: move mworker.h to haproxy/mworker{,-t}.h
One function prototype makes reference to struct mworker_proc which was
not defined there but in global.h instead. This definition, along with
the PROC_O_* fields were moved to mworker-t.h instead.
Willy Tarreau [Thu, 4 Jun 2020 09:09:42 +0000 (11:09 +0200)]
REORG: include: split mailers.h into haproxy/mailers{,-t}.h
The file mostly contained struct definitions but there was also a
variable export. Most of the stuff currently lies in checks.h and
should definitely move here!
Willy Tarreau [Thu, 4 Jun 2020 08:36:03 +0000 (10:36 +0200)]
REORG: include: move auth.h to haproxy/auth{,-t}.h
The STATS_DEFAULT_REALM and STATS_DEFAULT_URI were moved to defaults.h.
It was required to include types/pattern.h and types/sample.h since they
are mentioned in function prototypes.
It would be wise to merge this with uri_auth.h later.
Willy Tarreau [Thu, 4 Jun 2020 07:08:41 +0000 (09:08 +0200)]
REORG: include: move http_htx.h to haproxy/http_htx{,-t}.h
A few includes had to be added, namely list-t.h in the type file and
types/proxy.h in the proto file. actions.h was including http-htx.h
but didn't need it so it was dropped.
Willy Tarreau [Wed, 3 Jun 2020 18:02:28 +0000 (20:02 +0200)]
REORG: include: move sink.h to haproxy/sink{,-t}.h
The sink files could be moved with almost no change at since they
didn't rely on anything fancy. ssize_t required sys/types.h and
thread.h was needed for the locks.
Willy Tarreau [Wed, 3 Jun 2020 17:33:00 +0000 (19:33 +0200)]
REORG: include: move fd.h to haproxy/fd{,-t}.h
A few includes were missing in each file. A definition of
struct polled_mask was moved to fd-t.h. The MAX_POLLERS macro was
moved to defaults.h
Stdio used to be silently inherited from whatever path but it's needed
for list_pollers() which takes a FILE* and which can thus not be
forward-declared.
Willy Tarreau [Wed, 3 Jun 2020 16:09:46 +0000 (18:09 +0200)]
REORG: tools: split common/standard.h into haproxy/tools{,-t}.h
And also rename standard.c to tools.c. The original split between
tools.h and standard.h dates from version 1.3-dev and was mostly an
accident. This patch moves the files back to what they were expected
to be, and takes care of not changing anything else. However this
time tools.h was split between functions and types, because it contains
a small number of commonly used macros and structures (e.g. name_desc)
which in turn cause the massive list of includes of tools.h to conflict
with the callers.
They remain the ugliest files of the whole project and definitely need
to be cleaned and split apart. A few types are defined there only for
functions provided there, and some parts are even OS-specific and should
move somewhere else, such as the symbol resolution code.
Willy Tarreau [Wed, 3 Jun 2020 13:26:55 +0000 (15:26 +0200)]
REORG: include: move protocol.h to haproxy/protocol{,-t}.h
The protocol.h files are pretty low in the dependency and (sadly) used
by some files from common/. Almost nothing was changed except lifting a
few comments.
Willy Tarreau [Wed, 3 Jun 2020 12:56:08 +0000 (14:56 +0200)]
REORG: include: move common/fcgi.h to haproxy/
The file was moved almost verbatim (only stdio.h was dropped as useless).
It was not split between types and functions because it's only included
from direct C code (fcgi.c and mux_fcgi.c) as well as fcgi_app.h, included
from the same ones, which should also be remerged as a single one.
Willy Tarreau [Wed, 3 Jun 2020 07:09:57 +0000 (09:09 +0200)]
REORG: include: move hpack*.h to haproxy/ and split hpack-tbl
The various hpack files are self-contained, but hpack-tbl was one of
those showing difficulties when pools were added because that began
to add quite some dependencies. Now when built in standalone mode,
it still uses the bare minimum pool definitions and doesn't require
to know the prototypes anymore when only the structures are needed.
Thus the files were moved verbatim except for hpack-tbl which was
split between types and prototypes.
Willy Tarreau [Wed, 3 Jun 2020 06:44:35 +0000 (08:44 +0200)]
REORG: include: split common/htx.h into haproxy/htx{,-t}.h
Most of the file was a large set of HTX elements manipulation functions
and few types, so splitting them allowed to further reduce dependencies
and shrink the build time. Doing so revealed that a few files (h2.c,
mux_pt.c) needed haproxy/buf.h and were previously getting it through
htx.h. They were fixed.
Willy Tarreau [Tue, 2 Jun 2020 17:33:08 +0000 (19:33 +0200)]
REORG: include: move common/h1.h to haproxy/h1.h
The file was moved as-is. There was a wrong dependency on dynbuf.h
instead of buf.h which was addressed. There was no benefit to
splitting this between types and functions.
Willy Tarreau [Tue, 2 Jun 2020 17:25:28 +0000 (19:25 +0200)]
REORG: include: split common/http-hdr.h into haproxy/http-hdr{,-t}.h
There's only one struct and 2 inline functions. It could have been
merged into http.h but that would have added a massive dependency on
the hpack parts for nothing, so better keep it this way since hpack
is already freestanding and portable.
Willy Tarreau [Tue, 2 Jun 2020 17:11:26 +0000 (19:11 +0200)]
REORG: include: split common/http.h into haproxy/http{,-t}.h
So the enums and structs were placed into http-t.h and the functions
into http.h. This revealed that several files were dependeng on http.h
but not including it, as it was silently inherited via other files.