Run metatests in some components, covering both GCC and Clang, with ASan,
MSan or neither.
Note that this commit does not cover constant-flow testing builds or Valgrind.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Enforce a specific copyright statement and a specific SPDX license
identifier where they are present.
Binary files, third-party modules and a few other exceptions are not
checked.
There is currently no check that copyright statements and license
identifiers are present.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Some basic test coverage for now:
* Nominal operation.
* Larger output buffer.
* Clone an operation and use it after the original operation stops.
Generate test data automatically. For the time being, only do that for
hashes that Python supports natively. Supporting all algorithms is future
work.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
This will let us use these features from other modules (yet to be created).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
walk_compat_sh and walk_opt_sh are basically the same now, so:
* Merge them into one function.
* Use the --list-test-cases option for both of them.
* Rename this merged function as collect_from_script which seems
more appropriate as since it isn't iterating the script but
calling it.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The official spelling of the trade mark changed from all-lowercase "mbed"
to normal proper noun capitalization "Mbed" a few years ago. We've been
using the new spelling in new text but still have the old spelling in a
lot of text. This commit updates most occurrences of "mbed TLS":
```
sed -i -e 's/mbed TLS/Mbed TLS/g' $(git ls-files ':!ChangeLog' ':!tests/data_files/**' ':!tests/suites/*.data' ':!programs/x509/*' ':!configs/tfm*')
```
Justification for the omissions:
* `ChangeLog`: historical text.
* `test/data_files/**`, `tests/suites/*.data`, `programs/x509/*`: many
occurrences are significant names in certificates and such. Changing
the spelling would invalidate many signatures and tests.
* `configs/tfm*`: this is an imported file. We'll follow the upstream
updates.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This commit includes:
- use subprocess.check_output to report error and capture return
value
- add comment as a reminder for option --list-test-case
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Test case description is printed by different block of code. This
causes code maintenance harder since we need to maintain two parts
of code with same functionality. print_test_title is used to
control test case description in compat.sh
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Test case description in compat.sh is in format of
[ogm]->[ogm] TLSmode, VERIFY CIPHERSUITE_NAME
This program calls compat.sh to list all potential test case
descriptions then checks test case duplication.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
It isn't used anywhere.
Keep the command line options of all.sh to avoid breaking any wrapper
scripts that people might have.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
None of the tests actually need GNUTLS_LEGACY (3.3.8): GNUTLS (3.4.10)
works.
Only single-DES actually needs OPENSSL_LEGACY (1.0.1j). For the rest,
OPENSSL (1.0.2g) works.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Explain how PSA_CRYPTO_DRIVER_TEST_ALL works and why we have it. Note that
it is incompatible with MBEDTLS_PSA_CRYPTO_CONFIG.
MBEDTLS_PSA_CRYPTO_BUILTIN_KEYS is in the full config, so there's no need to
add it explicitly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
* Turn the warnings produced when finding non-executed tests that
are not in the allow list into errors.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
Introduce the --require-full-coverage in analyze_outcomes.py so that
when analyze_outcomes.py --require-full-coverage is called, those
tests that are not executed and are not in the allowed list issue an
error instead of a warning.
Note that it is useful to run analyze_outcomes.py on incomplete test
results, so this error mode needs to remain optional in the long
term.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The allow list explicits which test cases are allowed to not be
executed when testing. This may be, for example, because a feature
is yet to be developed but the test for that feature is already in
our code base.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
When MBEDTLS_USE_PSA_CRYPTO is disabled on a base of full, mention it.
Don't explicitly mention MBEDTLS_PSA_CRYPTO_CONFIG when also mentioning
PSA_WANT_xxx, since PSA_WANT symbols are only meaningful when
MBEDTLS_PSA_CRYPTO_CONFIG is enabled.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Having a wrapper made it harder to use: incompatible with setting
MBEDTLS_CONFIG_FILE, harder to combine with other settings. It was also
surprising since it was the only test config that was structured in that
way.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The original goal (https://github.com/Mbed-TLS/mbedtls/pull/5072) was to run
a test with ChaChaPoly disabled in PSA. It was actually implemented with GCM
also partially disabled (legacy GCM enabled but PSA GCM disabled), which
distracted from the objective. It's actually useful to test both with and
without GCM, so test both. Don't test inconsistencies between legacy and PSA
support because that's not a common case and not one we have particular
reasons to test.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
