unified build scheme: add a "unified" template for VMS descrip.mms

[thirdparty/openssl.git] / Configurations / README

Configurations of OpenSSL target platforms
==========================================

Target configurations are a collection of facts that we know about
different platforms and their capabilities.  We organise them in a
hash table, where each entry represent a specific target.

In each table entry, the following keys are significant:

        inherit_from    => Other targets to inherit values from.
                           Explained further below. [1]
        template        => Set to 1 if this isn't really a platform
                           target.  Instead, this target is a template
                           upon which other targets can be built.
                           Explained further below.  [1]

        sys_id          => System identity for systems where that
                           is difficult to determine automatically.

        cc              => The compiler command, usually one of "cc",
                           "gcc" or "clang".  This command is normally
                           also used to link object files and
                           libraries into the final program.
        cflags          => Flags that are used at all times when
                           compiling.
        defines         => As an alternative, macro definitions may be
                           present here instead of in `cflags'.  If
                           given here, they MUST be as an array of the
                           string such as "MACRO=value", or just
                           "MACRO" for definitions without value.
        debug_cflags    => Extra compilation flags used when making a
                           debug build (when Configure receives the
                           --debug option).  Typically something like
                           "-g -O0".
        debug_defines   => Similarly to `debug_cflags', this gets
                           combined with `defines' during a debug
                           build.  The value here MUST also be an
                           array of the same form as for `defines'.
        release_cflags  => Extra compilation flags used when making a
                           release build (when Configure receives the
                           --release option, or doesn't receive the
                           --debug option).  Typically something like
                           "-O" or "-O3".
        release_defines => Similarly to `release_cflags', this gets
                           combined with `defines' during a release
                           build.  The value here MUST also be an
                           array of the same form as for `defines'.
        thread_cflags   => Extra compilation flags used when
                           compiling with threading enabled.
                           Explained further below.  [2]
        thread_defines  => Similarly to `thread_cflags', this gets
                           combined with `defines' when threading is
                           enabled.  The value here MUST also be an
                           array of the same form as for `defines'.
        shared_cflag    => Extra compilation flags used when
                           compiling for shared libraries, typically
                           something like "-fPIC".

        (linking is a complex thing, see [3] below)
        ld              => Linker command, usually not defined
                           (meaning the compiler command is used
                           instead).
                           (NOTE: this is here for future use, it's
                           not implemented yet)
        lflags          => Flags that are used when linking apps.
        shared_ldflag   => Flags that are used when linking shared
                           or dynamic libraries.
        plib_lflags     => Extra linking flags to appear just before
                           the libraries on the command line.
        ex_libs         => Extra libraries that are needed when
                           linking.

        debug_lflags    => Like debug_cflags, but used when linking.
        release_lflags  => Like release_cflags, but used when linking.

        ar              => The library archive command, the default is
                           "ar".
                           (NOTE: this is here for future use, it's
                           not implemented yet)
        arflags         => Flags to be used with the library archive
                           command.

        ranlib          => The library archive indexing command, the
                           default is 'ranlib' it it exists.

        unistd          => An alternative header to the typical
                           '<unistd.h>'.  This is very rarely needed.

        shared_extension => File name extension used for shared
                            libraries. 
        obj_extension   => File name extension used for object files.
                           On unix, this defaults to ".o" (NOTE: this
                           is here for future use, it's not
                           implemented yet)
        exe_extension   => File name extension used for executable
                           files.  On unix, this defaults to "" (NOTE:
                           this is here for future use, it's not
                           implemented yet)

        dso_scheme      => The type of dynamic shared objects to build
                           for.  This mostly comes into play with
                           engines, but can be used for other purposes
                           as well.  Valid values are "DLFCN"
                           (dlopen() et al), "DLFCN_NO_H" (for systems
                           that use dlopen() et al but do not have
                           fcntl.h), "DL" (shl_load() et al), "WIN32"
                           and "VMS".
        perlasm_scheme  => The perlasm method used to created the
                           assembler files used when compiling with
                           assembler implementations.
        shared_target   => The shared library building method used.
                           This is a target found in Makefile.shared.
        build_scheme    => The scheme used to build up a Makefile.
                           In its simplest form, the value is a string
                           with the name of the build scheme.
                           The value may also take the form of a list
                           of strings, if the build_scheme is to have
                           some options.  In this case, the first
                           string in the list is the name of the build
                           scheme.
                           Currently recognised build schemes are
                           "mk1mf" and "unixmake" and "unified".
                           For the "unified" build scheme, this item
                           *must* be an array with the first being the
                           word "unified" and the second being a word
                           to identify the platform family.

        multilib        => On systems that support having multiple
                           implementations of a library (typically a
                           32-bit and a 64-bit variant), this is used
                           to have the different variants in different
                           directories.

        bn_ops          => Building options (was just bignum options
                           in the earlier history of this option,
                           hence the name).  This a string of words
                           that describe properties on the designated
                           target platform, such as the type of
                           integers used to build up the bitnum,
                           different ways to implement certain ciphers
                           and so on.  To fully comprehend the
                           meaning, the best is to read the affected
                           source.
                           The valid words are:

                           BN_LLONG     use 'unsigned long long' in
                                        some bignum calculations.
                                        This has no value when
                                        SIXTY_FOUR_BIT or
                                        SIXTY_FOUR_BIT_LONG is given.
                           RC4_CHAR     makes the basic RC4 unit of
                                        calculation an unsigned char.
                           SIXTY_FOUR_BIT       processor registers
                                                are 64 bits, long is
                                                32 bits, long long is
                                                64 bits.
                           SIXTY_FOUR_BIT_LONG  processor registers
                                                are 64 bits, long is
                                                64 bits.
                           THIRTY_TWO_BIT       processor registers
                                                are 32 bits.
                           EXPORT_VAR_AS_FN     for shared libraries,
                                                export vars as
                                                accessor functions.

        apps_extra_src  => Extra source to build apps/openssl, as
                           needed by the target.
        cpuid_asm_src   => assembler implementation of cpuid code as
                           well as OPENSSL_cleanse().
                           Default to mem_clr.c
        bn_asm_src      => Assembler implementation of core bignum
                           functions.
                           Defaults to bn_asm.c
        ec_asm_src      => Assembler implementation of core EC
                           functions.
        des_asm_src     => Assembler implementation of core DES
                           encryption functions.
                           Defaults to 'des_enc.c fcrypt_b.c'
        aes_asm_src     => Assembler implementation of core AES
                           functions.
                           Defaults to 'aes_core.c aes_cbc.c'
        bf_asm_src      => Assembler implementation of core BlowFish
                           functions.
                           Defaults to 'bf_enc.c'
        md5_asm_src     => Assembler implementation of core MD5
                           functions.
        sha1_asm_src    => Assembler implementation of core SHA1,
                           functions, and also possibly SHA256 and
                           SHA512 ones.
        cast_asm_src    => Assembler implementation of core CAST
                           functions.
                           Defaults to 'c_enc.c'
        rc4_asm_src     => Assembler implementation of core RC4
                           functions.
                           Defaults to 'rc4_enc.c rc4_skey.c'
        rmd160_asm_src  => Assembler implementation of core RMD160
                           functions.
        rc5_asm_src     => Assembler implementation of core RC5
                           functions.
                           Defaults to 'rc5_enc.c'
        wp_asm_src      => Assembler implementation of core WHIRLPOOL
                           functions.
        cmll_asm_src    => Assembler implementation of core CAMELLIA
                           functions.
                           Defaults to 'camellia.c cmll_misc.c cmll_cbc.c'
        modes_asm_src   => Assembler implementation of cipher modes,
                           currently the functions gcm_gmult_4bit and
                           gcm_ghash_4bit.
        padlock_asm_src => Assembler implementation of core parts of
                           the padlock engine.  This is mandatory on
                           any platform where the padlock engine might
                           actually be built.


[1] as part of the target configuration, one can have a key called
    'inherit_from' that indicate what other configurations to inherit
    data from.  These are resolved recursively.

    Inheritance works as a set of default values that can be overriden
    by corresponding key values in the inheriting configuration.

    Note 1: any configuration table can be used as a template.
    Note 2: pure templates have the attribute 'template => 1' and
            cannot be used as build targets.

    If several configurations are given in the 'inherit_from' array,
    the values of same attribute are concatenated with space
    separation.  With this, it's possible to have several smaller
    templates for different configuration aspects that can be combined
    into a complete configuration.

    instead of a scalar value or an array, a value can be a code block
    of the form 'sub { /* your code here */ }'.  This code block will
    be called with the list of inherited values for that key as
    arguments.  In fact, the concatenation of strings is really done
    by using 'sub { join(" ",@_) }' on the list of inherited values.

    An example:

        "foo" => {
                template => 1,
                haha => "ha ha",
                hoho => "ho",
                ignored => "This should not appear in the end result",
        },
        "bar" => {
                template => 1,
                haha => "ah",
                hoho => "haho",
                hehe => "hehe"
        },
        "laughter" => {
                inherit_from => [ "foo", "bar" ],
                hehe => sub { join(" ",(@_,"!!!")) },
                ignored => "",
        }

        The entry for "laughter" will become as follows after processing:

        "laughter" => {
                haha => "ha ha ah",
                hoho => "ho haho",
                hehe => "hehe !!!",
                ignored => ""
        }

[2] OpenSSL is built with threading capabilities unless the user
    specifies 'no-threads'.  The value of the key 'thread_cflags' may
    be "(unknown)", in which case the user MUST give some compilation
    flags to Configure.

[3] OpenSSL has three types of things to link from object files or
    static libraries:

    - shared libraries; that would be libcrypto and libssl.
    - shared objects (sometimes called dynamic libraries);  that would
      be the engines.
    - applications; those are apps/openssl and all the test apps.

    Very roughly speaking, linking is done like this (words in braces
    represent the configuration settings documented at the beginning
    of this file):

    shared libraries:
        {ld} $(CFLAGS) {shared_ldflag} -shared -o libfoo.so \
            -Wl,--whole-archive libfoo.a -Wl,--no-whole-archive \
            {plib_lflags} -lcrypto {ex_libs}

    shared objects:
        {ld} $(CFLAGS) {shared_ldflag} -shared -o libeng.so \
            blah1.o blah2.o {plib_lflags} -lcrypto {ex_libs}

    applications:
        {ld} $(CFLAGS) {lflags} -o app \
            app1.o utils.o {plib_lflags} -lssl -lcrypto {ex_libs}


Historically, the target configurations came in form of a string with
values separated by colons.  This use is deprecated.  The string form
looked like this:

   "target" => "{cc}:{cflags}:{unistd}:{thread_cflag}:{sys_id}:{lflags}:{bn_ops}:{cpuid_obj}:{bn_obj}:{ec_obj}:{des_obj}:{aes_obj}:{bf_obj}:{md5_obj}:{sha1_obj}:{cast_obj}:{rc4_obj}:{rmd160_obj}:{rc5_obj}:{wp_obj}:{cmll_obj}:{modes_obj}:{padlock_obj}:{perlasm_scheme}:{dso_scheme}:{shared_target}:{shared_cflag}:{shared_ldflag}:{shared_extension}:{ranlib}:{arflags}:{multilib}"


Build info files
================

The build.info files that are spread over the source tree contain the
minimum information needed to build and distribute OpenSSL.  It uses a
simple and yet fairly powerful language to determine what needs to be
built, from what sources, and other relationships between files.

For every build.info file, all file references are relative to the
directory of the build.info file for source files, and the
corresponding build directory for built files if the build tree
differs from the source tree.

When processed, every line is processed with the perl module
Text::Template, using the delimiters "{-" and "-}".  The hashes
%config and %target are passed to the perl fragments, along with
$sourcedir and $builddir, which are the locations of the source
directory for the current build.info file and the corresponding build
directory, all relative to the top of the build tree.

To begin with, things to be built are declared by setting specific
variables:

    PROGRAMS=foo bar
    LIBS=libsomething
    ENGINES=libeng
    SCRIPTS=myhack
    EXTRA=file1 file2

Note that the files mentioned for PROGRAMS, LIBS and ENGINES *must* be
without extensions.  The build file templates will figure them out.

For each thing to be built, it is then possible to say what sources
they are built from:

    PROGRAMS=foo bar
    SOURCE[foo]=foo.c common.c
    SOURCE[bar]=bar.c extra.c common.c

It's also possible to tell some other dependencies:

    DEPEND[foo]=libsomething
    DEPEND[libbar]=libsomethingelse

(it could be argued that 'libsomething' and 'libsomethingelse' are
source as well.  However, the files given through SOURCE are expected
to be located in the source tree while files given through DEPEND are
expected to be located in the build tree)

For some libraries, we maintain files with public symbols and their
slot in a transfer vector (important on some platforms).  It can be
declared like this:

    ORDINALS[libcrypto]=crypto

The value is not the name of the file in question, but rather the
argument to util/mkdef.pl that indicates which file to use.

One some platforms, shared libraries come with a name that's different
from their static counterpart.  That's declared as follows:

    SHARED_NAME[libfoo]=cygfoo-{- $config{shlibver} -}

The example is from Cygwin, which has a required naming convention.

Sometimes, it makes sense to rename an output file, for example a
library:

    RENAME[libfoo]=libbar

That lines has "libfoo" get renamed to "libbar".  While it makes no
sense at all to just have a rename like that (why not just use
"libbar" everywhere?), it does make sense when it can be used
conditionally.  See a little further below for an example.

For any file to be built, it's also possible to tell what extra
include paths the build of their source files should use:

    INCLUDE[foo]=include

It's possible to have raw build file lines, between BEGINRAW and
ENDRAW lines as follows:

    BEGINRAW[Makefile(unix)]
    haha.h: {- $builddir -}/Makefile
        echo "/* haha */" > haha.h
    ENDRAW[Makefile(unix)]

The word withing square brackets is the build_file configuration item
or the build_file configuration item followed by the second word in the
build_scheme configuration item for the configured target within
parenthesis as shown above.  For example, with the following relevant
configuration items:

   build_file   => "build.ninja"
   build_scheme => [ "unified", "unix" ]

... these lines will be considered:

   BEGINRAW[build.ninja]
   build haha.h: echo "/* haha */" > haha.h
   ENDRAW[build.ninja]

   BEGINRAW[build.ninja(unix)]
   build hoho.h: echo "/* hoho */" > hoho.h
   ENDRAW[build.ninja(unix)]

See the documentation further up for more information on configuration
items.

Finally, you can have some simple conditional use of the build.info
information, looking like this:

    IF[1]
     something
    ELSIF[2]
     something other
    ELSE
     something else
    ENDIF

The expression in square brackets is interpreted as a string in perl,
and will be seen as true if perl thinks it is, otherwise false.  For
example, the above would have "something" used, since 1 is true.

Together with the use of Text::Template, this can be used as
conditions based on something in the passed variables, for example:

    IF[{- $config{no_shared} -}]
      LIBS=libcrypto
      SOURCE[libcrypto]=...
    ELSE
      LIBS=libfoo
      SOURCE[libfoo]=...
    ENDIF

or:

    # VMS has a cultural standard where all libraries are prefixed.
    # For OpenSSL, the choice is 'ossl_'
    IF[{- $config{target} =~ /^vms/ -}]
     RENAME[libcrypto]=ossl_libcrypto
     RENAME[libssl]=ossl_libssl
    ENDIF


Build-file programming with the "unified" build system
======================================================

"Build files" are called "Makefile" on Unix-like operating systems,
"descrip.mms" for MMS on VMS, "makefile" for nmake on Windows, etc.

To use the "unified" build system, the target configuration needs to
set the three items 'build_scheme', 'build_file' and 'build_command'.
In the rest of this section, we will assume that 'build_scheme' is set
to "unified" (see the configurations documentation above for the
details).

For any name given by 'build_file', the "unified" system expects a
template file in Configurations/ named like the build file, with
".tmpl" appended, or in case of possible ambiguity, a combination of
the second 'build_scheme' list item and the 'build_file' name.  For
example, if 'build_file' is set to "Makefile", the template could be
Configurations/Makefile.tmpl or Configurations/unix-Makefile.tmpl.
In case both Configurations/unix-Makefile.tmpl and
Configurations/Makefile.tmpl are present, the former takes
precedence.

The build-file template is processed with the perl module
Text::Template, using "{-" and "-}" as delimiters that enclose the
perl code fragments that generate configuration-dependent content.
Those perl fragments have access to all the hash variables from
configdata.pem.

The build-file template is expected to define at least the following
perl functions in a perl code fragment enclosed with "{-" and "-}".
They are all expected to return a string with the lines they produce.

    src2dep     - function that produces build file lines to get the
                  dependencies for an object file into a dependency
                  file.

                  It's called like this:

                        src2dep(obj => "PATH/TO/objectfile",
                                srcs => [ "PATH/TO/sourcefile", ... ],
                                incs => [ "INCL/PATH", ... ]);

                  'obj' has the dependent object file as well as
                  object file the dependencies are for; it's *without*
                  extension, src2dep() is expected to add that.
                  'srcs' has the list of source files to build the
                  object file, with the first item being the source
                  file that directly corresponds to the object file.
                  'incs' is a list of include file directories.

    src2obj     - function that produces build file lines to build an
                  object file from source files and associated data.

                  It's called like this:

                        src2obj(obj => "PATH/TO/objectfile",
                                srcs => [ "PATH/TO/sourcefile", ... ],
                                deps => [ "dep1", ... ],
                                incs => [ "INCL/PATH", ... ]);

                  'obj' has the intended object file *without*
                  extension, src2obj() is expected to add that.
                  'srcs' has the list of source files to build the
                  object file, with the first item being the source
                  file that directly corresponds to the object file.
                  'deps' is a list of dependencies.  'incs' is a list
                  of include file directories.

    obj2lib     - function that produces build file lines to build a
                  static library file ("libfoo.a" in Unix terms) from
                  object files.

                  called like this:

                        obj2lib(lib => "PATH/TO/libfile",
                                objs => [ "PATH/TO/objectfile", ... ]);

                  'lib' has the intended library file name *without*
                  extension, obj2lib is expected to add that.  'objs'
                  has the list of object files (also *without*
                  extension) to build this library.

    libobj2shlib - function that produces build file lines to build a
                  shareable object library file ("libfoo.so" in Unix
                  terms) from the corresponding static library file
                  or object files.

                  called like this:

                        libobj2shlib(shlib => "PATH/TO/shlibfile",
                                     lib => "PATH/TO/libfile",
                                     objs => [ "PATH/TO/objectfile", ... ],
                                     deps => [ "PATH/TO/otherlibfile", ... ],
                                     ordinals => [ "word", "/PATH/TO/ordfile" ]);

                  'lib' has the intended library file name *without*
                  extension, libobj2shlib is expected to add that.
                  'shlib' has the correcponding shared library name
                  *without* extension.  'deps' has the list of other
                  libraries (also *without* extension) this library
                  needs to be linked with.  'objs' has the list of
                  object files (also *without* extension) to build
                  this library.  'ordinals' MAY be present, and when
                  it is, its value is an array where the word is
                  "crypto" or "ssl" and the file is one of the ordinal
                  files util/libeay.num or util/ssleay.num in the
                  source directory.

                  This function has a choice; it can use the
                  corresponding static library as input to make the
                  shared library, or the list of object files.

    obj2dynlib  - function that produces build file lines to build a
                  dynamically loadable library file ("libfoo.so" on
                  Unix) from object files.

                  called like this:

                        obj2dynlib(lib => "PATH/TO/libfile",
                                   objs => [ "PATH/TO/objectfile", ... ],
                                   deps => [ "PATH/TO/otherlibfile",
                                   ... ]);

                  This is almost the same as libobj2shlib, but the
                  intent is to build a shareable library that can be
                  loaded in runtime (a "plugin"...).  The differences
                  are subtle, one of the most visible ones is that the
                  resulting shareable library is produced from object
                  files only.

    obj2bin     - function that produces build file lines to build an
                  executable file from object files.

                  called like this:

                        obj2bin(bin => "PATH/TO/binfile",
                                objs => [ "PATH/TO/objectfile", ... ],
                                deps => [ "PATH/TO/libfile", ... ]);

                  'bin' has the intended executable file name
                  *without* extension, obj2bin is expected to add
                  that.  'objs' has the list of object files (also
                  *without* extension) to build this library.  'deps'
                  has the list of library files (also *without*
                  extension) that the programs needs to be linked
                  with.

    in2script   - function that produces build file lines to build a
                  script file from some input.

                  called like this:

                        in2script(script => "PATH/TO/scriptfile",
                                  sources => [ "PATH/TO/infile", ... ]);

                  'script' has the intended script file name.
                  'sources' has the list of source files to build the
                  resulting script from.

In all cases, file file paths are relative to the build tree top, and
the build file actions run with the build tree top as current working
directory.

Make sure to end the section with these functions with a string that
you thing is apropriate for the resulting build file.  If nothing
else, end it like this:

      "";       # Make sure no lingering values end up in the Makefile
    -}