1 Running external test suites with OpenSSL
2 =========================================
4 It is possible to integrate external test suites into OpenSSL's `make test`.
5 This capability is considered a developer option and does not work on all
8 Python PYCA/Cryptography test suite
9 ===================================
11 This python test suite runs cryptographic tests with a local OpenSSL build as
14 First checkout the `PYCA/Cryptography` module into `./pyca-cryptography` using:
16 $ git submodule update --init
18 Then configure/build OpenSSL compatible with the python module:
20 $ ./config shared enable-external-tests
23 The tests will run in a python virtual environment which requires virtualenv
26 $ make test VERBOSE=1 TESTS=test_external_pyca
28 Test failures and suppressions
29 ------------------------------
31 Some tests target older (<=1.0.2) versions so will not run. Other tests target
32 other crypto implementations so are not relevant. Currently no tests fail.
37 Much like the PYCA/Cryptography test suite, this builds and runs the krb5
38 tests against the local OpenSSL build.
40 You will need a git checkout of krb5 at the top level:
42 $ git clone https://github.com/krb5/krb5
44 krb5's master has to pass this same CI, but a known-good version is
45 krb5-1.15.1-final if you want to be sure.
48 $ git checkout krb5-1.15.1-final
51 OpenSSL must be built with external tests enabled:
53 $ ./config enable-external-tests
56 krb5's tests will then be run as part of the rest of the suite, or can be
57 explicitly run (with more debugging):
59 $ VERBOSE=1 make TESTS=test_external_krb5 test
61 Test-failures suppressions
62 --------------------------
64 krb5 will automatically adapt its test suite to account for the configuration
65 of your system. Certain tests may require more installed packages to run. No
66 tests are expected to fail.
68 GOST engine test suite
71 Much like the PYCA/Cryptography test suite, this builds and runs the GOST engine
72 tests against the local OpenSSL build.
74 You will need a git checkout of gost-engine at the top level:
76 $ git submodule update --init
78 Then configure/build OpenSSL enabling external tests:
80 $ ./config shared enable-external-tests
83 GOST engine requires CMake for the build process.
85 GOST engine tests will then be run as part of the rest of the suite, or can be
86 explicitly run (with more debugging):
88 $ make test VERBOSE=1 TESTS=test_external_gost_engine
90 OQSprovider test suite
91 ======================
93 Much like the PYCA/Cryptography test suite, this builds and runs the OQS
94 (OpenQuantumSafe -- www.openquantumsafe.org) provider tests against the
97 You will need a git checkout of oqsprovider at the top level:
99 $ git submodule update --init
101 Then configure/build OpenSSL enabling external tests:
103 $ ./config shared enable-external-tests
106 oqsprovider requires CMake for the build process.
108 OQSprovider tests will then be run as part of the rest of the suite, or can be
109 explicitly run (with more debugging):
111 $ make test VERBOSE=1 TESTS=test_external_oqsprovider
113 The environment variable `OQS_SKIP_TESTS` can be set to select tests and
114 algorithms to be skipped, e.g. as follows:
116 OQS_SKIP_TESTS=kyber make test TESTS=test_external_oqsprovider
118 The names of all supported quantum-safe algorithms are available at
119 <https://github.com/open-quantum-safe/oqs-provider#algorithms>
124 To update the commit for any of the above test suites:
126 - Make sure the submodules are cloned locally:
128 $ git submodule update --init --recursive
130 - Enter subdirectory and pull from the repository (use a specific branch/tag if required):
132 $ cd `<submodule-dir>`
133 $ git pull origin master
135 - Go to root directory, there should be a new git status:
140 # modified: `<submodule-dir>` (new commits)
143 - Add/commit/push the update
145 $ git add `<submodule-dir>`
146 $ git commit -m `"Updated <submodule> to latest commit"`