mirror of
				https://github.com/cuberite/polarssl.git
				synced 2025-10-30 19:20:40 -04:00 
			
		
		
		
	Document test case descriptions
This commit is contained in:
		
							parent
							
								
									717cd76e8a
								
							
						
					
					
						commit
						e94bc87ebe
					
				| @ -4,6 +4,36 @@ This document is an overview of the Mbed TLS test framework and test tools. | |||||||
| 
 | 
 | ||||||
| This document is incomplete. You can help by expanding it. | This document is incomplete. You can help by expanding it. | ||||||
| 
 | 
 | ||||||
|  | ## Unit tests | ||||||
|  | 
 | ||||||
|  | See https://tls.mbed.org/kb/development/test_suites | ||||||
|  | 
 | ||||||
|  | ### Unit test descriptions | ||||||
|  | 
 | ||||||
|  | Each test case has a description which succinctly describes for a human audience what the test does. The first non-comment line of each paragraph in a `.data` file is the test description. The following rules and guidelines apply: | ||||||
|  | 
 | ||||||
|  | * Test descriptions may not contain semicolons, line breaks and other control characters, or non-ASCII characters. <br> | ||||||
|  |   Rationale: keep the tools that process test descriptions (`generate_test_code.py`, [outcome file](#outcome-file) tools) simple. | ||||||
|  | * Test descriptions must be unique within a `.data` file. If you can't think of a better description, the convention is to append `#1`, `#2`, etc. <br> | ||||||
|  |   Rationale: make it easy to relate a failure log to the test data. Avoid confusion between cases in the [outcome file](#outcome-file). | ||||||
|  | * Test descriptions should be a maximum of **66 characters**. <br> | ||||||
|  |   Rationale: 66 characters is what our various tools assume (leaving room for 14 more characters on an 80-column line). Longer descriptions may be truncated or may break a visual alignment. <br> | ||||||
|  |   We have a lot of test cases with longer descriptions, but they should be avoided. At least please make sure that the first 66 characters describe the test uniquely. | ||||||
|  | * Make the description descriptive. “foo: x=2, y=4” is more descriptive than “foo #2”. “foo: 0<x<y, both even” is even better if these inequalities and parities are why this particular test data was chosen. | ||||||
|  | * Avoid changing the description of an existing test case without a good reason. This breaks the tracking of failures across CI runs, since this tracking is based on the descriptions. | ||||||
|  | 
 | ||||||
|  | `tests/scripts/check-test-cases.py` enforces some rules and warns if some guidelines are violated. | ||||||
|  | 
 | ||||||
|  | ## TLS tests | ||||||
|  | 
 | ||||||
|  | ### SSL extension tests | ||||||
|  | 
 | ||||||
|  | #### SSL test case descriptions | ||||||
|  | 
 | ||||||
|  | Each test case in `ssl-opt.sh` has a description which succinctly describes for a human audience what the test does. The test description is the first parameter to `run_tests`. | ||||||
|  | 
 | ||||||
|  | The same rules and guidelines apply as for [unit test descriptions](#unit-test-descriptions). In addition, the description must be written on the same line as `run_test`, in double quotes, for the sake of `check-test-cases.py`. | ||||||
|  | 
 | ||||||
| ## Running tests | ## Running tests | ||||||
| 
 | 
 | ||||||
| ### Outcome file | ### Outcome file | ||||||
|  | |||||||
		Loading…
	
	
			
			x
			
			
		
	
		Reference in New Issue
	
	Block a user
	 Gilles Peskine
						Gilles Peskine