Remove unnecessary "../library" prefix from test suite include. This
aligns the test suite with the development branch
Signed-off-by: David Horstmann <david.horstmann@arm.com>
When calling `add_xxx_ciphersuites`, we have to set MODE properly.
This commit adjusts order to address this issue in list_test_case
which matches what we do in a normal execution.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
This commit includes:
- use subprocess.check_output to report error and capture return
value
- add comment as a reminder for option --list-test-case
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
uniform_title is used to print identical format of $TITLE between
--list-test-case and run_client. In such way, no matter how $TITLE
is developed, --list-test-case will in the same format of test case
description as stored in OUTCOME.CSV.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Test case description is printed by different block of code. This
causes code maintenance harder since we need to maintain two parts
of code with same functionality. print_test_title is used to
control test case description in compat.sh
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Test case description in compat.sh is in format of
[ogm]->[ogm] TLSmode, VERIFY CIPHERSUITE_NAME
This program calls compat.sh to list all potential test case
descriptions then checks test case duplication.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
The option --list-test-case lists all potential test cases without
executing them. The test case description is identical with $TITLE
during test case execution.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
* Turn the warnings produced when finding non-executed tests that
are not in the allow list into errors.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
Introduce the --require-full-coverage in analyze_outcomes.py so that
when analyze_outcomes.py --require-full-coverage is called, those
tests that are not executed and are not in the allowed list issue an
error instead of a warning.
Note that it is useful to run analyze_outcomes.py on incomplete test
results, so this error mode needs to remain optional in the long
term.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The allow list explicits which test cases are allowed to not be
executed when testing. This may be, for example, because a feature
is yet to be developed but the test for that feature is already in
our code base.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The hardware module name otherName SAN contains 2 OIDs:
OtherName ::= SEQUENCE {
type-id OBJECT IDENTIFIER,
value [0] EXPLICIT ANY DEFINED BY type-id }
HardwareModuleName ::= SEQUENCE {
hwType OBJECT IDENTIFIER,
hwSerialNum OCTET STRING }
The first, type-id, is the one that identifies the otherName as a
HardwareModuleName. The second, hwType, identifies the type of hardware.
This change fixes 2 issues:
1. We were erroneously trying to identify HardwareModuleNames by looking
at hwType, not type-id.
2. We accidentally inverted the check so that we were checking that
hwType did NOT match HardwareModuleName.
This fix ensures that type-id is correctly checked to make sure that it
matches the OID for HardwareModuleName.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
