1352 Commits

Author SHA1 Message Date
Dave Rodgman
b400fb0b76
Merge pull request #6452 from AndrzejKurek/depends-py-reloaded-2-28
[Backport 2.28] Unified tests/scripts/depends.py - reloaded
2022-11-24 10:59:31 +00:00
Andrzej Kurek
f53d0ba529 depends.py: remove tls defines in SHA-512-only test
TLS 1.2 requires either SHA-256 or SHA-384.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-11-23 05:54:46 -05:00
Gilles Peskine
05dcb1300d
Merge pull request #6613 from gilles-peskine-arm/run-test-suites-out-of-tree-2.28
Backport 2.28: Fix run-test-suites.pl in out-of-tree builds
2022-11-22 15:01:21 +01:00
Andrzej Kurek
aa11281c07 Add a SHA512_NO_SHA384_C test to depends.py
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-11-22 08:13:45 -05:00
Gilles Peskine
6b7c3ccfed Use .datax for make test, not .data
Looking for the .data file doesn't work in out-of-tree builds. Use the
.datax file instead. `make clean` removes all .datax files, so this resolves
the issue of executables not present on the current branch being left behind
after a branch change followed by a `make clean`.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-11-16 10:35:20 +01:00
Gilles Peskine
83763ab6b1 Pacify pylint
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-11-10 15:15:25 +01:00
Gilles Peskine
92c5d31b44 Add negative zero as an input to automatically generated tests
Although negative zero is officially unsupported, we've had bugs related to
it in the past. So do test functions with a negative zero input.

There will likely be cases where we don't want to accept negative zero as if
it was valid, because it's too hard to handle. We'll add exceptions on a
case by case basis.

For the functions that are currently tested by the generated tests, the new
test cases pass.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-11-10 15:15:25 +01:00
Gilles Peskine
ee78b6e642 For binary operations, test both x op y and y op x
This exposes a bug in mbedtls_mpi_add_mpi() and mbedtls_mpi_sub_mpi() which
will be fixed in a subsequent commit.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-11-10 15:15:25 +01:00
Gilles Peskine
676df8ec88
Merge pull request #6578 from gilles-peskine-arm/bignum-test-suite-names-2.28
Backport 2.28: Rename test_suite_bignum for consistency
2022-11-10 14:40:16 +01:00
Gilles Peskine
d7d5e9a328
Merge pull request #6536 from davidhorstmann-arm/2.28-change-test-templating-syntax
[Backport 2.28] Change test templating syntax to be valid C
2022-11-10 12:05:58 +01:00
Gilles Peskine
618f989954 Don't use test_suite_mpi as an example
It just got renamed, and it's also not the most canonical example since it's
a somewhat deprecated interface in the development branch. Make a different
module the example.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-11-09 21:00:28 +01:00
Gilles Peskine
5b68608b40 Rename test_suite_bignum for consistency with bignum.{h,c}
Align the name of the bignum test suite with the source module (which was
renamed from mpi.c to bignum.c in the PolarSSL 1.x days). This also brings
it into line with the test suites for the low-level bignum interfaces.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-11-09 21:00:28 +01:00
David Horstmann
8eff06f1d6 Minor improvements to test code script
Signed-off-by: David Horstmann <david.horstmann@arm.com>
2022-11-09 17:31:08 +00:00
Manuel Pégourié-Gonnard
78dea63de9 Improve test suite detection in run-test-suites.pl
Looking for executables causes problems with leftover compiled test
suites from other branches when we forget to run make clean before
switching branches. Using the .data files is more robust as most of them
are tracked, so will be removed when switching branches.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2022-11-09 12:06:43 +01:00
David Horstmann
14bae83ca0 Change test templating syntax to be valid C
For the benefit of auto-formatting tools, move from the '$placeholder'
templating syntax to a new syntax of the form:

__MBEDTLS_TEST_TEMPLATE__PLACEHOLDER

This change allows the test code template to be almost entirely valid C.

Signed-off-by: David Horstmann <david.horstmann@arm.com>
2022-11-07 09:55:35 +00:00
Gilles Peskine
83264bed63 Group cpp_dummy_build test into an existing component
No need to do yet another build just to compile an additional trivial program.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-31 11:45:17 +00:00
Gilles Peskine
e86cdcb7bf Remove redundant build-only driver interface component
component_build_psa_accel_alg_ecdsa is subsumed by
component_test_psa_crypto_config_accel_ecdsa, which has the same
configuration and additionally runs the unit tests.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-31 11:41:47 +00:00
Andrzej Kurek
c5acdb9e92 Remove unused perl dependency scripts
curves.pl, depends-hashes.pl, key-exchanges.pl and depends-pkalgs.pl are now superseded by depends.py.
Update all references to them accordingly.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-27 07:44:13 -04:00
Andrzej Kurek
20d8a5f688 Add all.sh components running depends.py without MBEDTLS_USE_PSA_CRYPTO
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-27 07:44:13 -04:00
Andrzej Kurek
2b44a929a4 depends.py: add a config option to unset MBEDTLS_USE_PSA
This lets us perform any test without MBEDTLS_USE_PSA
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-27 07:44:13 -04:00
Gilles Peskine
031c8c2815 Add build with a typical configuration for a PSA crypto service
Disable non-crypto features that can't be called through the PSA API, as
well as algorithms that have no PSA interface.

This serves as a non-regression test for #6408 and #6409.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-25 21:09:49 +02:00
Gilles Peskine
8df2748432 Improve test component name
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-25 20:32:35 +02:00
Gilles Peskine
21503df752 Add build with a typical configuration for a PSA crypto service
Disable non-crypto features that can't be called through the PSA API, as
well as algorithms that have no PSA interface.

This serves as a non-regression test for #6408 and #6409.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-10-25 20:32:20 +02:00
Andrzej Kurek
467a0f2025 Adjust depends.py to the 2.28 branch
SHA224 and SHA384 does not exist.
ARC4 and BLOWFISH requirements had to be added. 
Changed mbedtls_config.h to config.h
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
113952de01 depends.py: rename domains argument to tasks
Tasks can consist of domains and/or jobs.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
ffbc8f5d4a depends.py documentation fixes
Now that the format of exclusive groups has been changed,
update the documentation using it too.

Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
3146bb4f61 depends.py: remove config options that are unset anyway
Over the lifespan of this script these options have been removed
from the "full" configuration.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
bec35a18c1 depends.py: rename config_pl usage to config_py
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:38:12 -04:00
Andrzej Kurek
9cbdf102d6 Change the way exclusive groups are defined in depends.py
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:37:00 -04:00
Andrzej Kurek
0325ced8cc depends.py: fix TLS 1.3 requirements
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:37:00 -04:00
Andrzej Kurek
8b7a157fbe Improve depends.py readability
Switch from SHA224 & SHA384 testing to SHA256 and SHA512.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:37:00 -04:00
Andrzej Kurek
cddae3b3fa Add MBEDTLS_DES_C exclusive group dependencies
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:37:00 -04:00
Andrzej Kurek
110fc48070 depends.py: Add script documentation
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:24:19 -04:00
Andrzej Kurek
5b327555d5 Disable MBEDTLS_TEST_HOOKS in depends.py
This option was increasing testing duration
by about 40%.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:24:14 -04:00
Andrzej Kurek
068a73f46d Rework depends.py to run more tests with hashes
The test coverage reduction introduced in
dc25cee lowered the coverage of hash tests
due to intertwining dependencies.
This commit introduces a new class for building
a domain using both the complementary
and exclusive classes.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:24:09 -04:00
Andrzej Kurek
2e105b53f9 depends.py: disable part of the test jobs
Disable exclusive jobs that run with a single
config disabled. A lot more bugs should be found by running jobs with only one config
of a family enabled.
This will also lessen the burden on the CI.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:24:02 -04:00
Andrzej Kurek
85d69309e8 Split depends.py all.sh job into seven
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:56 -04:00
Andrzej Kurek
fb3e27e178 Use upper case for constants in depends.py
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:51 -04:00
Andrzej Kurek
b95ba9a2db Improve depends.py structrue
Apply most improvements suggested by pylint.
Use config.py instead of config.pl.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:45 -04:00
Andrzej Kurek
798f5c28c9 Fix python formatting and indentation
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:40 -04:00
Andrzej Kurek
3cdd08cf84 Add an all.sh component running depends.pl
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:34 -04:00
Andrzej Kurek
9068625a57 Update depends.py
Remove old and add new dependencies.
Introduce a way to handle non-trivial problems 
stemming from exclusive group testing.
Exclude SHA256 and SHA512, as these are tested
in SHA224 and SHA384 jobs, respectively.
Change config.h to mbedtls_config.h).
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-20 06:23:29 -04:00
Gilles Peskine
3ce0e32d4b Add domains for symmetric ciphers
Add a domain for cipher base algorithms (block permutations and stream
ciphers), a domain for block cipher chaining modes and a domain for
block cipher padding modes.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
7088a73296 When exercising key exchanges, don't build the test suites
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
b81f406742 Declare more reverse dependencies
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
3dd0dab7fb Don't test builds with only deprecated hashes
Don't try to build with only SHA-1 or with only RIPEMD160 or with only
MD{2,4,5}.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
88e8dd6f5d Use the full config as the baseline for all jobs
Start each job from the full config minus some memory management
settings and the job-specific settings.

The original content of config.h no longer influences the
configurations used for the jobs (but it still influences what jobs
may run, in that the set of jobs is partly built by parsing #define
and //#define lines in config.h).
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
dc68f618cc Simplify final passed/failed reporting
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
fd1d69c77c Add basic support for colored output
Show "pass" lines in green and "fail" lines in red.
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00
Gilles Peskine
d43ce2b4c1 Factor running config.pl into its own function
Signed-off-by: Andrzej Kurek <andrzej.kurek@arm.com>
2022-10-18 06:13:04 -04:00