1  The HAL test suite
 2  ~~~~~~~~~~~~~~~~~~~
 3  The tests in these directories serve to test the behavior of HAL components.
 4  
 5  Each subdirectory of this directory may contain a test item.  The runtests
 6  script recurses through the directory structure, so multiple tests could
 7  be structured as
 8  	tests/
 9  		xyz.0
10  		xyz.1
11  		xyz.2
12  or
13  	tests/
14  		xyz/
15  			0	
16  			1	
17  			2	
18  
19  
20  Two types of tests are supported: Regression tests, in which the output is
21  tested against a "known good" output, and functional tests, in which the
22  output is fed to a program that can determine whether it is correct or not
23  
24  
25  Running the tests
26  ~~~~~~~~~~~~~~~~~
27  Currently, tests only work with the "run in place" configuration.  They
28  can be run by executing (from the top emc2 directory)
29  	scripts/runtests tests
30  A subset of the tests can also be run:
31  	scripts/runtests tests/xyz tests/a*
32  The directories named on the commandline are searched recursively for
33  'test.hal' or 'test.sh' files, and a directory with such a file is
34  assumed to contain a regression test or a functional test.
35  
36  Tests may contain files other than the ones specified below.  For instance,
37  when using 'streamer' data as test input, a shell script with
38  "halstreamer<<EOF" and a "here document" will generally be present.
39  (see and-or-not-mux.0/runstreamer for an example)
40  
41  Regression Tests
42  ~~~~~~~~~~~~~~~~
43  A regression test should consist of these three files:
44  	README
45  		A human-readable file describing the test
46  	test.hal *or* test.sh *or* test
47  		The test script to execute.  test.hal is executed with
48  		'halrun -f', test.sh is executed with 'bash -x', and
49  		test is executed as ./test
50  	expected
51  		A file whose contents are compared with the stdout of
52  			halrun -f test.hal
53  
54  A typical regression test will load several components, usually including
55  'threads' and 'sampler', and often including 'streamer', then collect samples
56  from some number of calls to the realtime thread, then exit.
57  
58  Regression test "test.hal" files will almost always include the line
59  	setexact_for_test_suite_only
60  which causes HAL to act as though the requested base_period was available.
61  Otherwise, results will differ slightly depending on the actual base_period
62  and regression tests will fail.
63  
64  The test passes if the expected and actual output are identical.
65  Otherwise, the test fails.
66  
67  
68  Functional Tests
69  ~~~~~~~~~~~~~~~~
70  A functional test should consist of three files:
71  	README
72  		A human-readable file describing the test
73  	test.hal *or* test.sh *or* test
74  		The test script to execute.  test.hal is executed with
75  		'halrun -f', test.sh is executed with 'bash -x', and
76  		test is executed as ./test
77  	checkresult
78  		An executable file (such as a shell or python script)
79  		which determines if the stdout of
80  			halrun -f test.hal
81  		indicates success or failure
82  
83  Regression test "test.hal" files will often include the line
84  	setexact_for_test_suite_only
85  which causes HAL to act as though the requested base_period was available.
86  Otherwise, results will differ slightly depending on the actual base_period,
87  which could affect whether 'checkresult' gives an accurate result.
88  
89  A typical regression test will load several components, usually including
90  'threads' and 'sample', then collect samples from some number of calls
91  to the realtime thread, then exit.  'checkresult' will look at the output
92  and see if it indicates success.
93  
94  The test passes if the command "checkresult actual" returns a shell
95  success value (exit code 0).  Otherwise, the test fails.