]>
Commit | Line | Data |
---|---|---|
f83133a5 RL |
1 | Design document for the unified scheme data |
2 | =========================================== | |
3 | ||
4 | How are things connected? | |
5 | ------------------------- | |
6 | ||
7 | The unified scheme takes all its data from the build.info files seen | |
8 | throughout the source tree. These files hold the minimum information | |
9 | needed to build end product files from diverse sources. See the | |
10 | section on build.info files below. | |
11 | ||
12 | From the information in build.info files, Configure builds up an | |
13 | information database as a hash table called %unified_info, which is | |
14 | stored in configdata.pm, found at the top of the build tree (which may | |
15 | or may not be the same as the source tree). | |
16 | ||
17 | Configurations/common.tmpl uses the data from %unified_info to | |
18 | generate the rules for building end product files as well as | |
19 | intermediary files with the help of a few functions found in the | |
20 | build-file templates. See the section on build-file templates further | |
21 | down for more information. | |
22 | ||
23 | build.info files | |
24 | ---------------- | |
25 | ||
26 | As mentioned earlier, build.info files are meant to hold the minimum | |
27 | information needed to build output files, and therefore only (with a | |
28 | few possible exceptions [1]) have information about end products (such | |
29 | as scripts, library files and programs) and source files (such as C | |
30 | files, C header files, assembler files, etc). Intermediate files such | |
b6453a68 | 31 | as object files are rarely directly referred to in build.info files (and |
f83133a5 | 32 | when they are, it's always with the file name extension .o), they are |
b6453a68 | 33 | inferred by Configure. By the same rule of minimalism, end product |
f83133a5 | 34 | file name extensions (such as .so, .a, .exe, etc) are never mentioned |
b6453a68 | 35 | in build.info. Their file name extensions will be inferred by the |
f83133a5 RL |
36 | build-file templates, adapted for the platform they are meant for (see |
37 | sections on %unified_info and build-file templates further down). | |
38 | ||
39 | The variables PROGRAMS, LIBS, ENGINES and SCRIPTS are used to declare | |
40 | end products. | |
41 | ||
42 | The variables SOURCE, DEPEND, INCLUDE and ORDINALS are indexed by a | |
43 | produced file, and their values are the source used to produce that | |
44 | particular produced file, extra dependencies, include directories | |
45 | needed, and ordinal files (explained further below. | |
46 | ||
47 | All their values in all the build.info throughout the source tree are | |
48 | collected together and form a set of programs, libraries, engines and | |
49 | scripts to be produced, source files, dependencies, etc etc etc. | |
50 | ||
51 | Let's have a pretend example, a very limited contraption of OpenSSL, | |
52 | composed of the program 'apps/openssl', the libraries 'libssl' and | |
53 | 'libcrypto', an engine 'engines/ossltest' and their sources and | |
54 | dependencies. | |
55 | ||
56 | # build.info | |
57 | LIBS=libcrypto libssl | |
58 | ORDINALS[libcrypto]=crypto | |
59 | ORDINALS[libssl]=ssl | |
60 | INCLUDE[libcrypto]=include | |
61 | INCLUDE[libssl]=include | |
62 | DEPEND[libssl]=libcrypto | |
63 | ||
64 | This is the top directory build.info file, and it tells us that two | |
65 | libraries are to be built, there are some ordinals to be used to | |
66 | declare what symbols in those libraries are seen as public, the | |
67 | include directory 'include/' shall be used throughout when building | |
68 | anything that will end up in each library, and that the library | |
69 | 'libssl' depend on the library 'libcrypto' to function properly. | |
70 | ||
71 | # apps/build.info | |
72 | PROGRAMS=openssl | |
73 | SOURCE[openssl]=openssl.c | |
74 | INCLUDE[openssl]=.. ../include | |
75 | DEPEND[openssl]=../libssl | |
76 | ||
77 | This is the build.info file in 'apps/', one may notice that all file | |
78 | paths mentioned are relative to the directory the build.info file is | |
79 | located in. This one tells us that there's a program to be built | |
80 | called 'apps/openssl' (the file name extension will depend on the | |
81 | platform and is therefore not mentioned in the build.info file). It's | |
82 | built from one source file, 'apps/openssl.c', and building it requires | |
83 | the use of '.' and 'include' include directories (both are declared | |
84 | from the point of view of the 'apps/' directory), and that the program | |
85 | depends on the library 'libssl' to function properly. | |
86 | ||
87 | # crypto/build.info | |
88 | LIBS=../libcrypto | |
89 | SOURCE[../libcrypto]=aes.c evp.c cversion.c | |
90 | DEPEND[cversion.o]=buildinf.h | |
91 | ||
92 | BEGINRAW[Makefile(unix)] | |
93 | crypto/buildinf.h : Makefile | |
94 | perl util/mkbuildinf.h "$(CC) $(CFLAGS)" "$(PLATFORM)" \ | |
95 | > crypto/buildinf.h | |
96 | ENDRAW[Makefile(unix)] | |
97 | ||
98 | This is the build.info file in 'crypto', and it tells us a little more | |
99 | about what's needed to produce 'libcrypto'. LIBS is used again to | |
100 | declare that 'libcrypto' is to be produced. This declaration is | |
101 | really unnecessary as it's already mentioned in the top build.info | |
102 | file, but can make the info file easier to understand. This is to | |
103 | show that duplicate information isn't an issue. | |
104 | ||
105 | This build.info file informs us that 'libcrypto' is built from a few | |
106 | source files, 'crypto/aes.c', 'crypto/evp.c' and 'crypto/cversion.c'. | |
107 | It also shows us that building the object file inferred from | |
108 | 'crypto/cversion.c' depends on 'crypto/buildinf.h'. Finally, it | |
109 | also shows the possibility to include raw build-file statements in a | |
110 | build.info file, in this case showing how 'buildinf.h' is built on | |
111 | Unix-like operating systems. | |
112 | ||
113 | Two things are worth an extra note: | |
114 | ||
b6453a68 | 115 | 'DEPEND[cversion.o]' mentions an object file. DEPEND indexes is the |
f83133a5 RL |
116 | only location where it's valid to mention them |
117 | ||
118 | Lines in 'BEGINRAW'..'ENDRAW' sections must always mention files as | |
119 | seen from the top directory, no exception. | |
120 | ||
121 | # ssl/build.info | |
122 | LIBS=../libssl | |
123 | SOURCE[../libssl]=tls.c | |
124 | ||
125 | This is the build.info file in 'ssl/', and it tells us that the | |
126 | library 'libssl' is built from the source file 'ssl/tls.c'. | |
127 | ||
128 | # engines/build.info | |
129 | ENGINES=libossltest | |
130 | SOURCE[libossltest]=e_ossltest.c | |
131 | DEPEND[libossltest]=../libcrypto | |
132 | INCLUDE[libossltest]=../include | |
133 | ||
134 | This is the build.info file in 'engines/', telling us that an engine | |
135 | called 'engines/libossltest' shall be built, that it's source is | |
136 | 'engines/e_ossltest.c' and that the include directory 'include/' may | |
137 | be used when building anything that will be part of this engine. | |
138 | Finally, the engine 'engines/libossltest' depends on the library | |
139 | 'libcrypto' to function properly. | |
140 | ||
141 | When Configure digests these build.info files, the accumulated | |
142 | information comes down to this: | |
143 | ||
144 | LIBS=libcrypto libssl | |
145 | ORDINALS[libcrypto]=crypto | |
146 | SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c | |
147 | DEPEND[crypto/cversion.o]=crypto/buildinf.h | |
148 | INCLUDE[libcrypto]=include | |
149 | ORDINALS[libssl]=ssl | |
150 | SOURCE[libssl]=ssl/tls.c | |
151 | INCLUDE[libssl]=include | |
152 | DEPEND[libssl]=libcrypto | |
153 | ||
154 | PROGRAMS=apps/openssl | |
155 | SOURCE[apps/openssl]=apps/openssl.c | |
156 | INCLUDE[apps/openssl]=. include | |
157 | DEPEND[apps/openssl]=libssl | |
158 | ||
159 | ENGINES=engines/libossltest | |
160 | SOURCE[engines/libossltest]=engines/e_ossltest.c | |
161 | DEPEND[engines/libossltest]=libcrypto | |
162 | INCLUDE[engines/libossltest]=include | |
163 | ||
164 | BEGINRAW[Makefile(unix)] | |
165 | crypto/buildinf.h : Makefile | |
166 | perl util/mkbuildinf.h "$(CC) $(CFLAGS)" "$(PLATFORM)" \ | |
167 | > crypto/buildinf.h | |
168 | ENDRAW[Makefile(unix)] | |
169 | ||
170 | ||
171 | A few notes worth mentioning: | |
172 | ||
173 | LIBS may be used to declare routine libraries only. | |
174 | ||
175 | PROGRAMS may be used to declare programs only. | |
176 | ||
177 | ENGINES may be used to declare engines only. | |
178 | ||
179 | The indexes for SOURCE, INCLUDE and ORDINALS must only be end product | |
180 | files, such as libraries, programs or engines. The values of SOURCE | |
181 | variables must only be source files (possibly generated) | |
182 | ||
183 | DEPEND shows a relationship between different end product files, such | |
184 | as a program depending on a library, or between an object file and | |
185 | some extra source file. | |
186 | ||
187 | When Configure processes the build.info files, it will take it as | |
188 | truth without question, and will therefore perform very few checks. | |
189 | If the build tree is separate from the source tree, it will assume | |
190 | that all built files and up in the build directory and that all source | |
191 | files are to be found in the source tree, if they can be found there. | |
192 | Configure will assume that source files that can't be found in the | |
193 | source tree (such as 'crypto/bildinf.h' in the example above) are | |
194 | generated and will be found in the build tree. | |
195 | ||
196 | ||
197 | The %unified_info database | |
198 | -------------------------- | |
199 | ||
200 | The information in all the build.info get digested by Configure and | |
201 | collected into the %unified_info database, divided into the following | |
202 | indexes: | |
203 | ||
204 | depends => a hash table containing 'file' => [ 'dependency' ... ] | |
205 | pairs. These are directly inferred from the DEPEND | |
206 | variables in build.info files. | |
207 | ||
208 | engines => a list of engines. These are directly inferred from | |
209 | the ENGINES variable in build.info files. | |
210 | ||
211 | includes => a hash table containing 'file' => [ 'include' ... ] | |
212 | pairs. These are directly inferred from the INCLUDE | |
213 | variables in build.info files. | |
214 | ||
215 | libraries => a list of libraries. These are directly inferred from | |
216 | the LIBS variable in build.info files. | |
217 | ||
218 | ordinals => a hash table containing 'file' => [ 'word', 'ordfile' ] | |
219 | pairs. 'file' and 'word' are directly inferred from | |
220 | the ORDINALS variables in build.info files, while the | |
221 | file 'ofile' comes from internal knowledge in | |
222 | Configure. | |
223 | ||
224 | programs => a list of programs. These are directly inferred from | |
225 | the PROGRAMS variable in build.info files. | |
226 | ||
227 | rawlines => a list of build-file lines. These are a direct copy of | |
228 | the BEGINRAW..ENDRAW lines in build.info files. Note: | |
229 | only the BEGINRAW..ENDRAW section for the current | |
230 | platform are copied, the rest are ignored. | |
231 | ||
232 | scripts => a list of scripts. There are directly inferred from | |
233 | the SCRIPTS variable in build.info files. | |
234 | ||
235 | sources => a hash table containing 'file' => [ 'sourcefile' ... ] | |
236 | pairs. These are indirectly inferred from the SOURCE | |
237 | variables in build.info files. Object files are | |
238 | mentioned in this hash table, with source files from | |
239 | SOURCE variables, and AS source files for programs and | |
240 | libraries. | |
241 | ||
242 | As an example, here is how the build.info files example from the | |
243 | section above would be digested into a %unified_info table: | |
244 | ||
245 | our %unified_info = ( | |
246 | "depends" => | |
247 | { | |
248 | "apps/openssl" => | |
249 | [ | |
250 | "libssl", | |
251 | ], | |
252 | "crypto/cversion.o" => | |
253 | [ | |
254 | "crypto/buildinf.h", | |
255 | ], | |
256 | "engines/libossltest" => | |
257 | [ | |
258 | "libcrypto", | |
259 | ], | |
260 | "libssl" => | |
261 | [ | |
262 | "libcrypto", | |
263 | ], | |
264 | }, | |
265 | "engines" => | |
266 | [ | |
267 | "engines/libossltest", | |
268 | ], | |
269 | "includes" => | |
270 | { | |
271 | "apps/openssl" => | |
272 | [ | |
273 | ".", | |
274 | "include", | |
275 | ], | |
276 | "engines/libossltest" => | |
277 | [ | |
278 | "include" | |
279 | ], | |
280 | "libcrypto" => | |
281 | [ | |
282 | "include", | |
283 | ], | |
284 | "libssl" => | |
285 | [ | |
286 | "include", | |
287 | ], | |
288 | } | |
289 | "libraries" => | |
290 | [ | |
291 | "libcrypto", | |
292 | "libssl", | |
293 | ], | |
294 | "ordinals" => | |
295 | { | |
296 | "libcrypto" => | |
297 | [ | |
298 | "crypto", | |
6928b617 | 299 | "util/libcrypto.num", |
f83133a5 RL |
300 | ], |
301 | "libssl" => | |
302 | [ | |
303 | "ssl", | |
6928b617 | 304 | "util/libssl.num", |
f83133a5 RL |
305 | ], |
306 | }, | |
307 | "programs" => | |
308 | [ | |
309 | "apps/openssl", | |
310 | ], | |
311 | "rawlines" => | |
312 | [ | |
313 | "crypto/buildinf.h : Makefile", | |
314 | " perl util/mkbuildinf.h \"\$(CC) \$(CFLAGS)\" \"\$(PLATFORM)\" \\" | |
315 | " > crypto/buildinf.h" | |
316 | ], | |
317 | "sources" => | |
318 | { | |
319 | "apps/openssl" => | |
320 | [ | |
321 | "apps/openssl.o", | |
322 | ], | |
323 | "apps/openssl.o" => | |
324 | [ | |
325 | "apps/openssl.c", | |
326 | ], | |
327 | "crypto/aes.o" => | |
328 | [ | |
329 | "crypto/aes.c", | |
330 | ], | |
331 | "crypto/cversion.o" => | |
332 | [ | |
333 | "crypto/cversion.c", | |
334 | ], | |
335 | "crypto/evp.o" => | |
336 | [ | |
337 | "crypto/evp.c", | |
338 | ], | |
339 | "engines/e_ossltest.o" => | |
340 | [ | |
341 | "engines/e_ossltest.c", | |
342 | ], | |
343 | "engines/libossltest" => | |
344 | [ | |
345 | "engines/e_ossltest.o", | |
346 | ], | |
347 | "libcrypto" => | |
348 | [ | |
349 | "crypto/aes.c", | |
350 | "crypto/cversion.c", | |
351 | "crypto/evp.c", | |
352 | ], | |
353 | "libssl" => | |
354 | [ | |
355 | "ssl/tls.c", | |
356 | ], | |
357 | "ssl/tls.o" => | |
358 | [ | |
359 | "ssl/tls.c", | |
360 | ], | |
361 | }, | |
362 | ); | |
363 | ||
b6453a68 | 364 | As can be seen, everything in %unified_info is fairly simple suggest |
f83133a5 RL |
365 | of information. Still, it tells us that to build all programs, we |
366 | must build 'apps/openssl', and to build the latter, we will need to | |
367 | build all its sources ('apps/openssl.o' in this case) and all the | |
368 | other things it depends on (such as 'libssl'). All those dependencies | |
369 | need to be built as well, using the same logic, so to build 'libssl', | |
370 | we need to build 'ssl/tls.o' as well as 'libcrypto', and to build the | |
371 | latter... | |
372 | ||
373 | ||
374 | Build-file templates | |
375 | -------------------- | |
376 | ||
377 | Build-file templates are essentially build-files (such as Makefile on | |
378 | Unix) with perl code fragments mixed in. Those perl code fragment | |
379 | will generate all the configuration dependent data, including all the | |
380 | rules needed to build end product files and intermediary files alike. | |
381 | At a minimum, there must be a perl code fragment that defines a set of | |
382 | functions that are used to generates specific build-file rules, to | |
383 | build static libraries from object files, to build shared libraries | |
384 | from static libraries, to programs from object files and libraries, | |
385 | etc. | |
386 | ||
f83133a5 RL |
387 | src2obj - function that produces build file lines to build an |
388 | object file from source files and associated data. | |
389 | ||
390 | It's called like this: | |
391 | ||
392 | src2obj(obj => "PATH/TO/objectfile", | |
393 | srcs => [ "PATH/TO/sourcefile", ... ], | |
394 | deps => [ "dep1", ... ], | |
45502bfe RL |
395 | incs => [ "INCL/PATH", ... ] |
396 | intent => one of "lib", "dso", "bin" ); | |
f83133a5 RL |
397 | |
398 | 'obj' has the intended object file *without* | |
399 | extension, src2obj() is expected to add that. | |
400 | 'srcs' has the list of source files to build the | |
401 | object file, with the first item being the source | |
402 | file that directly corresponds to the object file. | |
45502bfe RL |
403 | 'deps' is a list of explicit dependencies. 'incs' |
404 | is a list of include file directories. Finally, | |
405 | 'intent' indicates what this object file is going | |
406 | to be used for. | |
f83133a5 RL |
407 | |
408 | obj2lib - function that produces build file lines to build a | |
409 | static library file ("libfoo.a" in Unix terms) from | |
410 | object files. | |
411 | ||
412 | called like this: | |
413 | ||
414 | obj2lib(lib => "PATH/TO/libfile", | |
415 | objs => [ "PATH/TO/objectfile", ... ]); | |
416 | ||
417 | 'lib' has the intended library file name *without* | |
418 | extension, obj2lib is expected to add that. 'objs' | |
419 | has the list of object files (also *without* | |
420 | extension) to build this library. | |
421 | ||
422 | libobj2shlib - function that produces build file lines to build a | |
423 | shareable object library file ("libfoo.so" in Unix | |
424 | terms) from the corresponding static library file | |
425 | or object files. | |
426 | ||
427 | called like this: | |
428 | ||
429 | libobj2shlib(shlib => "PATH/TO/shlibfile", | |
430 | lib => "PATH/TO/libfile", | |
431 | objs => [ "PATH/TO/objectfile", ... ], | |
432 | deps => [ "PATH/TO/otherlibfile", ... ], | |
433 | ordinals => [ "word", "/PATH/TO/ordfile" ]); | |
434 | ||
435 | 'lib' has the intended library file name *without* | |
436 | extension, libobj2shlib is expected to add that. | |
b6453a68 | 437 | 'shlib' has the corresponding shared library name |
f83133a5 RL |
438 | *without* extension. 'deps' has the list of other |
439 | libraries (also *without* extension) this library | |
440 | needs to be linked with. 'objs' has the list of | |
441 | object files (also *without* extension) to build | |
442 | this library. 'ordinals' MAY be present, and when | |
443 | it is, its value is an array where the word is | |
444 | "crypto" or "ssl" and the file is one of the ordinal | |
6928b617 | 445 | files util/libcrypto.num or util/libssl.num in the |
f83133a5 RL |
446 | source directory. |
447 | ||
448 | This function has a choice; it can use the | |
449 | corresponding static library as input to make the | |
450 | shared library, or the list of object files. | |
451 | ||
452 | obj2dynlib - function that produces build file lines to build a | |
453 | dynamically loadable library file ("libfoo.so" on | |
454 | Unix) from object files. | |
455 | ||
456 | called like this: | |
457 | ||
458 | obj2dynlib(lib => "PATH/TO/libfile", | |
459 | objs => [ "PATH/TO/objectfile", ... ], | |
460 | deps => [ "PATH/TO/otherlibfile", | |
461 | ... ]); | |
462 | ||
463 | This is almost the same as libobj2shlib, but the | |
464 | intent is to build a shareable library that can be | |
465 | loaded in runtime (a "plugin"...). The differences | |
466 | are subtle, one of the most visible ones is that the | |
467 | resulting shareable library is produced from object | |
468 | files only. | |
469 | ||
470 | obj2bin - function that produces build file lines to build an | |
471 | executable file from object files. | |
472 | ||
473 | called like this: | |
474 | ||
475 | obj2bin(bin => "PATH/TO/binfile", | |
476 | objs => [ "PATH/TO/objectfile", ... ], | |
477 | deps => [ "PATH/TO/libfile", ... ]); | |
478 | ||
479 | 'bin' has the intended executable file name | |
480 | *without* extension, obj2bin is expected to add | |
481 | that. 'objs' has the list of object files (also | |
482 | *without* extension) to build this library. 'deps' | |
483 | has the list of library files (also *without* | |
484 | extension) that the programs needs to be linked | |
485 | with. | |
486 | ||
487 | in2script - function that produces build file lines to build a | |
488 | script file from some input. | |
489 | ||
490 | called like this: | |
491 | ||
492 | in2script(script => "PATH/TO/scriptfile", | |
493 | sources => [ "PATH/TO/infile", ... ]); | |
494 | ||
495 | 'script' has the intended script file name. | |
496 | 'sources' has the list of source files to build the | |
497 | resulting script from. | |
498 | ||
499 | Along with the build-file templates is the driving engine | |
500 | Configurations/common.tmpl, which looks through all the information in | |
501 | %unified_info and generates all the rulesets to build libraries, | |
502 | programs and all intermediate files, using the rule generating | |
503 | functions defined in the build-file template. | |
504 | ||
505 | As an example with the smaller build.info set we've seen as an | |
506 | example, producing the rules to build 'libssl' would result in the | |
507 | following calls: | |
508 | ||
509 | # Note: libobj2shlib will only be called if shared libraries are | |
510 | # to be produced. | |
511 | # Note 2: libobj2shlib gets both the name of the static library | |
512 | # and the names of all the object files that go into it. It's up | |
513 | # to the implementation to decide which to use as input. | |
514 | libobj2shlib(shlib => "libssl", | |
515 | lib => "libssl", | |
516 | objs => [ "ssl/tls.o" ], | |
517 | deps => [ "libcrypto" ] | |
6928b617 | 518 | ordinals => [ "ssl", "util/libssl.num" ]); |
f83133a5 RL |
519 | |
520 | obj2lib(lib => "libssl" | |
521 | objs => [ "ssl/tls.o" ]); | |
522 | ||
523 | # Note 3: common.tmpl peals off the ".o" extension, as the | |
524 | # platform at hand may have a different one. | |
525 | src2obj(obj => "ssl/tls" | |
526 | srcs => [ "ssl/tls.c" ], | |
527 | deps => [ ], | |
528 | incs => [ "include" ]); | |
529 | ||
530 | src2dep(obj => "ssl/tls" | |
531 | srcs => [ "ssl/tls.c" ], | |
532 | incs => [ "include" ]); | |
533 | ||
534 | The returned strings from all those calls are then concatenated | |
535 | together and written to the resulting build-file. |