1393 Commits

Author SHA1 Message Date
Gilles Peskine
d7d5e9a328
Merge pull request #6536 from davidhorstmann-arm/2.28-change-test-templating-syntax
[Backport 2.28] Change test templating syntax to be valid C
2022-11-10 12:05:58 +01:00
Gilles Peskine
618f989954 Don't use test_suite_mpi as an example
It just got renamed, and it's also not the most canonical example since it's
a somewhat deprecated interface in the development branch. Make a different
module the example.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-11-09 21:00:28 +01:00
Gilles Peskine
5b68608b40 Rename test_suite_bignum for consistency with bignum.{h,c}
Align the name of the bignum test suite with the source module (which was
renamed from mpi.c to bignum.c in the PolarSSL 1.x days). This also brings
it into line with the test suites for the low-level bignum interfaces.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-11-09 21:00:28 +01:00
David Horstmann
8eff06f1d6 Minor improvements to test code script
Signed-off-by: David Horstmann <david.horstmann@arm.com>
2022-11-09 17:31:08 +00:00
Manuel Pégourié-Gonnard
78dea63de9 Improve test suite detection in run-test-suites.pl
Looking for executables causes problems with leftover compiled test
suites from other branches when we forget to run make clean before
switching branches. Using the .data files is more robust as most of them
are tracked, so will be removed when switching branches.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2022-11-09 12:06:43 +01:00
David Horstmann
14bae83ca0 Change test templating syntax to be valid C
For the benefit of auto-formatting tools, move from the '$placeholder'
templating syntax to a new syntax of the form:

__MBEDTLS_TEST_TEMPLATE__PLACEHOLDER

This change allows the test code template to be almost entirely valid C.

Signed-off-by: David Horstmann <david.horstmann@arm.com>
2022-11-07 09:55:35 +00:00
Gilles Peskine
83264bed63 Group cpp_dummy_build test into an existing component
No need to do yet another build just to compile an additional trivial program.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-31 11:45:17 +00:00
Gilles Peskine
e86cdcb7bf Remove redundant build-only driver interface component
component_build_psa_accel_alg_ecdsa is subsumed by
component_test_psa_crypto_config_accel_ecdsa, which has the same
configuration and additionally runs the unit tests.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-31 11:41:47 +00:00
Andrzej Kurek
c5acdb9e92 Remove unused perl dependency scripts
curves.pl, depends-hashes.pl, key-exchanges.pl and depends-pkalgs.pl are now superseded by depends.py.
Update all references to them accordingly.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-27 07:44:13 -04:00
Andrzej Kurek
20d8a5f688 Add all.sh components running depends.py without MBEDTLS_USE_PSA_CRYPTO
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-27 07:44:13 -04:00
Andrzej Kurek
2b44a929a4 depends.py: add a config option to unset MBEDTLS_USE_PSA
This lets us perform any test without MBEDTLS_USE_PSA
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-27 07:44:13 -04:00
Gilles Peskine
031c8c2815 Add build with a typical configuration for a PSA crypto service
Disable non-crypto features that can't be called through the PSA API, as
well as algorithms that have no PSA interface.

This serves as a non-regression test for #6408 and #6409.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-25 21:09:49 +02:00
Gilles Peskine
8df2748432 Improve test component name
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-25 20:32:35 +02:00
Gilles Peskine
21503df752 Add build with a typical configuration for a PSA crypto service
Disable non-crypto features that can't be called through the PSA API, as
well as algorithms that have no PSA interface.

This serves as a non-regression test for #6408 and #6409.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-25 20:32:20 +02:00
Andrzej Kurek
467a0f2025 Adjust depends.py to the 2.28 branch
SHA224 and SHA384 does not exist.
ARC4 and BLOWFISH requirements had to be added. 
Changed mbedtls_config.h to config.h
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
113952de01 depends.py: rename domains argument to tasks
Tasks can consist of domains and/or jobs.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
ffbc8f5d4a depends.py documentation fixes
Now that the format of exclusive groups has been changed,
update the documentation using it too.

Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
3146bb4f61 depends.py: remove config options that are unset anyway
Over the lifespan of this script these options have been removed
from the "full" configuration.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
bec35a18c1 depends.py: rename config_pl usage to config_py
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
9cbdf102d6 Change the way exclusive groups are defined in depends.py
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:37:00 -04:00
Andrzej Kurek
0325ced8cc depends.py: fix TLS 1.3 requirements
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:37:00 -04:00
Andrzej Kurek
8b7a157fbe Improve depends.py readability
Switch from SHA224 & SHA384 testing to SHA256 and SHA512.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:37:00 -04:00
Andrzej Kurek
cddae3b3fa Add MBEDTLS_DES_C exclusive group dependencies
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:37:00 -04:00
Andrzej Kurek
110fc48070 depends.py: Add script documentation
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:24:19 -04:00
Andrzej Kurek
5b327555d5 Disable MBEDTLS_TEST_HOOKS in depends.py
This option was increasing testing duration
by about 40%.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:24:14 -04:00
Andrzej Kurek
068a73f46d Rework depends.py to run more tests with hashes
The test coverage reduction introduced in
dc25cee lowered the coverage of hash tests
due to intertwining dependencies.
This commit introduces a new class for building
a domain using both the complementary
and exclusive classes.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:24:09 -04:00
Andrzej Kurek
2e105b53f9 depends.py: disable part of the test jobs
Disable exclusive jobs that run with a single
config disabled. A lot more bugs should be found by running jobs with only one config
of a family enabled.
This will also lessen the burden on the CI.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:24:02 -04:00
Andrzej Kurek
85d69309e8 Split depends.py all.sh job into seven
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:56 -04:00
Andrzej Kurek
fb3e27e178 Use upper case for constants in depends.py
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:51 -04:00
Andrzej Kurek
b95ba9a2db Improve depends.py structrue
Apply most improvements suggested by pylint.
Use config.py instead of config.pl.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:45 -04:00
Andrzej Kurek
798f5c28c9 Fix python formatting and indentation
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:40 -04:00
Andrzej Kurek
3cdd08cf84 Add an all.sh component running depends.pl
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:34 -04:00
Andrzej Kurek
9068625a57 Update depends.py
Remove old and add new dependencies.
Introduce a way to handle non-trivial problems 
stemming from exclusive group testing.
Exclude SHA256 and SHA512, as these are tested
in SHA224 and SHA384 jobs, respectively.
Change config.h to mbedtls_config.h).
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:29 -04:00
Gilles Peskine
3ce0e32d4b Add domains for symmetric ciphers
Add a domain for cipher base algorithms (block permutations and stream
ciphers), a domain for block cipher chaining modes and a domain for
block cipher padding modes.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
7088a73296 When exercising key exchanges, don't build the test suites
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
b81f406742 Declare more reverse dependencies
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
3dd0dab7fb Don't test builds with only deprecated hashes
Don't try to build with only SHA-1 or with only RIPEMD160 or with only
MD{2,4,5}.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
88e8dd6f5d Use the full config as the baseline for all jobs
Start each job from the full config minus some memory management
settings and the job-specific settings.

The original content of config.h no longer influences the
configurations used for the jobs (but it still influences what jobs
may run, in that the set of jobs is partly built by parsing #define
and //#define lines in config.h).
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
dc68f618cc Simplify final passed/failed reporting
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
fd1d69c77c Add basic support for colored output
Show "pass" lines in green and "fail" lines in red.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
d43ce2b4c1 Factor running config.pl into its own function
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
e6a60db285 Flush log output after each line
Otherwise the output can be out of order when redirected.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
f5ea1978d4 New script to exercise compilation options
Unify curves.pl, key-exchanges.pl, depends-pkalgs.pl and
depends-hashes.pl into a single, newly-written script.

For curves, key exchanges and hashes, in addition to testing
all-but-one settings in the group like the old scripts, also run the
tests with a single option in the group.

Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
5d01cc04a0 Don't use parallel jobs for pylint
When pylint runs in parallel, it loses the ability to detect duplicated code
across modules. Duplicated code is usually a bad thing, so give pylint the
opportunity to let us know.

This makes pylint slightly slower, but going from 2 threads to 1 does not
make it anywhere close to twice as slow. On my machine, with Python 3.5,
pylint -j2 takes about 12s while single-threaded pylint takes about 16s
of wall clock time.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-14 15:34:06 +02:00
Gilles Peskine
7ff4766115 Unify check_repo_path
We had 4 identical copies of the check_repo_path function. Replace them by a
single copy in the build_tree module where it naturally belongs.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-14 15:34:06 +02:00
Gilles Peskine
ca980c037f Move implementation detail from docstring to comment
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-14 15:34:06 +02:00
Gilles Peskine
69feebd178 More precise name for test data generation
We have Python code both for test code generation
(tests/scripts/generate_test_code.py) and now for test data generation.
Avoid the ambiguous expression "test generation".

This commit renames the Python module and adjusts all references to it. A
subsequent commit will adjust the documentation.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-14 15:24:30 +02:00
Manuel Pégourié-Gonnard
3495ca309b
Merge pull request #6415 from mprse/aead_driver_test_2_28_backport
2.28 backport: Enable testing of AEAD drivers with libtestdriver1
2022-10-14 11:11:06 +02:00
Gilles Peskine
279188f3f3
Merge pull request #6396 from gilles-peskine-arm/platform.h-unconditional-2.28
Backport 2.28: Include platform.h unconditionally
2022-10-13 10:19:25 +02:00
Manuel Pégourié-Gonnard
a129babb83
Merge pull request #6412 from mpg/print-failed-suites-2.28
[backport 2.28] Print the list of failed suites in verbose mode
2022-10-13 09:59:53 +02:00