walk_compat_sh and walk_opt_sh are basically the same now, so:
* Merge them into one function.
* Use the --list-test-cases option for both of them.
* Rename this merged function as collect_from_script which seems
more appropriate as since it isn't iterating the script but
calling it.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The official spelling of the trade mark changed from all-lowercase "mbed"
to normal proper noun capitalization "Mbed" a few years ago. We've been
using the new spelling in new text but still have the old spelling in a
lot of text. This commit updates most occurrences of "mbed TLS":
```
sed -i -e 's/mbed TLS/Mbed TLS/g' $(git ls-files ':!ChangeLog' ':!tests/data_files/**' ':!tests/suites/*.data' ':!programs/x509/*' ':!configs/tfm*')
```
Justification for the omissions:
* `ChangeLog`: historical text.
* `test/data_files/**`, `tests/suites/*.data`, `programs/x509/*`: many
occurrences are significant names in certificates and such. Changing
the spelling would invalidate many signatures and tests.
* `configs/tfm*`: this is an imported file. We'll follow the upstream
updates.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
TLS-RSA-WITH-NULL-SHA256, like other SHA256-based cipher suites, was first
introduced in TLS 1.2. Mbed TLS accepts it in earlier protocol versions as
well. This is technically a bug, which older versions of GnuTLS also have.
GnuTLS 3.4.7 fixed this bug. Adapt compat.sh to automatically omit
TLS-RSA-WITH-NULL-SHA256 in invalid protocol versions if GnuTLS doesn't
support it. It's already not included in invalid protocol versions in
OpenSSL interoperability testing.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When calling `add_xxx_ciphersuites`, we have to set MODE properly.
This commit adjusts order to address this issue in list_test_case
which matches what we do in a normal execution.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
This commit includes:
- use subprocess.check_output to report error and capture return
value
- add comment as a reminder for option --list-test-case
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
uniform_title is used to print identical format of $TITLE between
--list-test-case and run_client. In such way, no matter how $TITLE
is developed, --list-test-case will in the same format of test case
description as stored in OUTCOME.CSV.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Test case description is printed by different block of code. This
causes code maintenance harder since we need to maintain two parts
of code with same functionality. print_test_title is used to
control test case description in compat.sh
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
The option --list-test-case lists all potential test cases without
executing them. The test case description is identical with $TITLE
during test case execution.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
If the environment variable MBEDTLS_TEST_OUTCOME_FILE is set,
the test outcome file records each test case in a single line
with the format of
PLATFORM;CONFIGURATION;compat;TEST CASE DESCRIPTION;RESULT;[CAUSE]
- CONFIGURATION comes from MBEDTLS_TEST_CONFIGURATION to record
configuration of each test case
- PLATFORM is either set by users or calculated from test
platform
- RESULT is one of PASS, FAIL or SKIP. If test case fails,
srv_out/cli_out follows as FAILURE CAUSE.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
This commit add support to detect if openssl used for testing
supports static ECDH key exchange. Skip the ciphersutes if
openssl doesn't support them.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
Since PSK cipher suites do not allow client certificate verification,
PSK test cases should be executed under VERIFY=NO. SUB_VERIFIES is
used to constrain verification option for PSK tests.
With aforementioned change, the latter check of
$VERIFY=YES && $TYPE!=PSK is redundant so it's removed.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
With the introduction of PSK_TESTS,
- Either `compat.sh -V NO` or `compat.sh -V YES` runs the PSK tests
- `compat.sh` or `compat.sh -V "NO YES"` runs PSK tests only once
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
There is no need to provide CA file in PSK. Thus VERIFY is
meaningless for PSK. This change omits the arguments passed to
the client and server for $VERIFY=YES.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Under Ubuntu-22.04, wait command prints out Terminated message.
Therefore server process is handled with identical ways like other
processes in compat.sh. In addition, PROCESS_ID is renamed as
SRV_PID to improve code readability.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Under Ubuntu-22.04, wait command prints out Terminated message
if the process has been killed by kill command. This messes up
the output in compat.sh
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
These variables were both uses to select the default version of OpenSSL
to use for tests:
- when running compat.sh or ssl-opt.sh directly, OPENSSL_CMD was used;
- when running all.sh, OPENSSL was used.
This caused surprising situations if you had one but not the other set
in your environment. For example I used to have OPENSSL_CMD set but not
OPENSSL, so ssl-opt.sh was failing in some all.sh components but passing
when I ran it manually in the same configuration and build, a rather
unpleasant experience.
The natural name would be OPENSSL, and that's what set in the Docker
images used by the CI. However back in the 1.3.x days, that name was
already used in library/Makefile, so it was preferable to pick a
different one, hence OPENSSL_CMD. However the build system has not been
using this name since at least Mbed TLS 2.0.0, so it's now free for use
again (as demonstrated by the fact that it's been set in the CI without
causing any trouble).
So, unify things and use OPENSSL everywhere. Just leave an error message
for the benefit of developers which might have OPENSSL_CMD, not OPENSSL,
set in their environment from the old days.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Restore compatibiltiy testing against OpenSSL for
(D)TLS versions smaller that 1.2.
. Fix the check for support in OpenSSL for these versions
. For test cases for (D)TLS version smaller than 1.2,
restore the configuration of OpenSSL client/server
with the given TLS version, as it was before #5660
that broke it.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
OpenSSL may be configured to support features such as cipher suites or
protocol versions that are disabled by default. Enable them all: we're
testing, we don't care about enabling insecure stuff. This is not needed
with the builds of OpenSSL that we're currently using on the Jenkins CI, but
it's needed with more recent versions such as typically found on developer
machines, and with future CI additions.
The syntax to do that was only introduced in OpenSSL 1.1.0; fortunately we
don't need to do anything special with earlier versions.
With OpenSSL 1.1.1f on Ubuntu 20.04, this is needed to enable TLS 1.0, TLS
1.1 and DTLS 1.0. This also allows SHA-1 in certificates, which is still
needed for a few test cases in ssl-opt.sh. Curiously, this is also needed
for the cipher suite TLS-DHE-PSK-WITH-ARIA-128-GCM-SHA256 (and no other,
including other DHE-PSK or ARIA cipher suites).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Our interoperability tests fail with a recent OpenSSL server. The
reason is that they force 1024-bit Diffie-Hellman parameters, which
recent OpenSSL (e.g. 1.1.1f on Ubuntu 20.04) reject:
```
140072814650688:error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small:../ssl/s3_lib.c:3782:
```
We've been passing custom DH parameters since
6195767554da332e9f81e6510b07f7565ff8a538 because OpenSSL <=1.0.2a
requires it. This is only concerns the version we use as
OPENSSL_LEGACY. So only use custom DH parameters for that version. In
compat.sh, use it based on the observed version of $OPENSSL_CMD.
This way, ssl-opt.sh and compat.sh work (barring other issues) for all
our reference versions of OpenSSL as well as for a modern system OpenSSL.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Fix `printf "$foo"` which treats the value of `foo` as a printf format
rather than a string.
I used the following command to find potentially problematic lines:
```
git ls-files '*.sh' | xargs egrep 'printf +("?[^"]*|[^ ]*)\$'
```
The remaining ones are false positives for this regexp.
The errors only had minor consequences: the output of `ssl-opt.sh`
contained lines like
```
Renegotiation: gnutls server strict, client-initiated .................. ./tests/ssl-opt.sh: 741: printf: %S: invalid directive
PASS
```
and in case of failure the GnuTLS command containing a substring like
`--priority=NORMAL:%SAFE_RENEGOTIATION` was not included in the log
file. With the current tests, there was no risk of a test failure
going undetected.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Replace server2.crt with server2-sha256.crt which, as the name implies, is
just the SHA-256 version of the same certificate.
Replace server1.crt with cert_sha256.crt which, as the name doesn't imply, is
associated with the same key and just have a slightly different Subject Name,
which doesn't matter in this instance.
The other certificates used in this script (server5.crt and server6.crt) are
already signed with SHA-256.
This change is motivated by the fact that recent versions of GnuTLS (or older
versions with the Debian patches) reject SHA-1 in certificates by default, as
they should. There are options to still accept it (%VERIFY_ALLOW_BROKEN and
%VERIFY_ALLOW_SIGN_WITH_SHA1) but:
- they're not available in all versions that reject SHA-1-signed certs;
- moving to SHA-2 just seems cleaner anyway.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Recent GnuTLS packages on Ubuntu 16.04 have them disabled.
From /usr/share/doc/libgnutls30/changelog.Debian.gz:
gnutls28 (3.4.10-4ubuntu1.5) xenial-security; urgency=medium
* SECURITY UPDATE: Lucky-13 issues
[...]
- debian/patches/CVE-2018-1084x-4.patch: hmac-sha384 and sha256
ciphersuites were removed from defaults in lib/gnutls_priority.c,
tests/priorities.c.
Since we do want to test the ciphersuites, explicitly re-enable them in the
server's priority string. (This is a no-op with versions of GnuTLS where those
are already enabled by default.)
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
As a result, the copyright of contributors other than Arm is now
acknowledged, and the years of publishing are no longer tracked in the
source files.
Also remove the now-redundant lines declaring that the files are part of
MbedTLS.
This commit was generated using the following script:
# ========================
#!/bin/sh
# Find files
find '(' -path './.git' -o -path './3rdparty' ')' -prune -o -type f -print | xargs sed -bi '
# Replace copyright attribution line
s/Copyright.*Arm.*/Copyright The Mbed TLS Contributors/I
# Remove redundant declaration and the preceding line
$!N
/This file is part of Mbed TLS/Id
P
D
'
# ========================
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
This commit was generated using the following script:
# ========================
#!/bin/sh
# Find scripts
find -path './.git' -prune -o '(' -name '*.gdb' -o -name '*.pl' -o -name '*.py' -o -name '*.sh' ')' -print | xargs sed -i '
# Remove Mbed TLS declaration if it occurs before the copyright line
1,/Copyright.*Arm/I {
/This file is part of/,$ {
/Copyright.*Arm/I! d
}
}
# Convert non-standard header in scripts/abi_check.py to the format used in the other scripts
/"""/,/"""/ {
# Cut copyright declaration
/Copyright.*Arm/I {
h
N
d
}
# Paste copyright declaration
/"""/ {
x
/./ {
s/^/# / # Add #
x # Replace orignal buffer with Copyright declaration
p # Print original buffer, insert newline
i\
s/.*// # Clear original buffer
}
x
}
}
/Copyright.*Arm/I {
# Print copyright declaration
p
# Read the two lines immediately following the copyright declaration
N
N
# Insert Apache header if it is missing
/SPDX/! {
i\
# SPDX-License-Identifier: Apache-2.0\
#\
# Licensed under the Apache License, Version 2.0 (the "License"); you may\
# not use this file except in compliance with the License.\
# You may obtain a copy of the License at\
#\
# http://www.apache.org/licenses/LICENSE-2.0\
#\
# Unless required by applicable law or agreed to in writing, software\
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT\
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\
# See the License for the specific language governing permissions and\
# limitations under the License.
# Insert Mbed TLS declaration if it is missing
/This file is part of/! i\
#\
# This file is part of Mbed TLS (https://tls.mbed.org)
}
# Clear copyright declaration from buffer
D
}
'
# ========================
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
compat.sh used to skip OpenSSL altogether for DTLS 1.2, because older
versions of OpenSSL didn't support it. But these days it is supported.
We don't want to use DTLS 1.2 with OpenSSL unconditionally, because we
still use legacy versions of OpenSSL to test with legacy ciphers. So
check whether the version we're using supports it.
Limit log output in compat.sh and ssl-opt.sh, in case of failures with these
scripts where they may output seemingly unlimited length error logs.
Note that ulimit -f uses units of 512 bytes, so we use 10 * 1024 * 1024 * 2 to
get 10 GiB.
