[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
gnattest
is an ASIS-based utility that creates unit-test skeletons
as well as a test driver infrastructure (harness). gnattest
creates
a skeleton for each visible subprogram in the packages under consideration when
they do not exist already.
In order to process source files from a project, gnattest
has to
semantically analyze the sources. Therefore, test skeletons can only be
generated for legal Ada units. If a unit is dependent on other units,
those units should be among the source files of the project or of other projects
imported by this one.
Generated skeletons and harnesses are based on the AUnit testing framework.
AUnit is an Ada adaptation of the xxxUnit testing frameworks, similar to JUnit
for Java or CppUnit for C++. While it is advised that gnattest users read
the AUnit manual, deep knowledge of AUnit is not necessary for using gnattest.
For correct operation of gnattest
, AUnit should be installed and
aunit.gpr must be on the project path. This happens automatically when Aunit
is installed at its default location.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
gnattest
has a command-line interface of the form
$ gnattest -Pprojname [--harness-dir=dirname] [switches] [filename] [-cargs gcc_switches] |
where
gcc
. These switches will be passed on to all compiler invocations
made by gnattest
to generate a set of ASIS trees. Here you can provide
`-I' switches to form the source search path,
use the `-gnatec' switch to set the configuration file,
use the `-gnat05' switch if sources should be compiled in
Ada 2005 mode, etc.
gnattest
results can be found in two different places.
test_driver.gpr
. Tests can be compiled and run
using a command such as:
gnatmake -P<harness-dir>/test_driver test_runner |
Note that you might need to specify the necessary values of scenario variables when you are not using the AUnit defaults.
Note that if the project already has both my_unit.ads and my_unit-test_data.ads, this will cause a name conflict with the generated test package.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
gnattest
creates a harness for all
sources, treating them as test packages.
`--tests_root', `--subdir' and `--tests-dir' switches are mutually exclusive.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
Most of the command-line options can also be passed to the tool by adding special attributes to the project file. Those attributes should be put in package gnattest. Here is the list of attributes:
Each of those attributes can be overridden from the command line if needed.
Other gnattest
switches can also be passed via the project
file as an attribute list called GNATtest_Switches.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Let's take a very simple example using the first gnattest
example
located in:
<install_prefix>/share/examples/gnattest/simple |
This project contains a simple package containing one subprogram. By running gnattest:
$ gnattest --harness-dir=driver -Psimple.gpr |
a test driver is created in directory "driver". It can be compiled and run:
$ cd obj/driver $ gnatmake -Ptest_driver $ test_runner |
One failed test with diagnosis "test not implemented" is reported. Since no special output option was specified, the test package Simple.Tests is located in:
<install_prefix>/share/examples/gnattest/simple/obj/gnattest/tests |
For each package containing visible subprograms, a child test package is generated. It contains one test routine per tested subprogram. Each declaration of a test subprogram has a comment specifying which tested subprogram it corresponds to. Bodies of test routines are placed in test package bodies and are surrounded by special comment sections. Those comment sections should not be removed or modified in order for gnattest to be able to regenerate test packages and keep already written tests in place. The test routine Test_Inc_5eaee3 located at simple-test_data-tests.adb contains a single statement: a call to procedure Assert. It has two arguments: the Boolean expression we want to check and the diagnosis message to display if the condition is false.
That is where actual testing code should be written after a proper setup. An actual check can be performed by replacing the Assert call with:
Assert (Inc (1) = 2, "wrong incrementation"); |
After recompiling and running the test driver, one successfully passed test is reported.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Besides test routines themselves, each test package has a parent package Test_Data that has two procedures: Set_Up and Tear_Down. This package is never overwritten by the tool. Set_Up is called before each test routine of the package and Tear_Down is called after each test routine. Those two procedures can be used to perform necessary initialization and finalization, memory allocation, etc. Test type declared in Test_Data package is parent type for the test type of test package and can have user-defined components whose values can be set by Set_Up routine and used in test routines afterwards.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Bodies of test routines and test_data packages are never overridden after they have been created once. As long as the name of the subprogram, full expanded Ada names, and the order of its parameters is the same, and comment sections are intact the old test routine will fit in its place and no test skeleton will be generated for the subprogram.
This can be demonstrated with the previous example. By uncommenting declaration
and body of function Dec in simple.ads and simple.adb, running
gnattest
on the project, and then running the test driver:
gnattest --harness-dir=driver -Psimple.gpr cd obj/driver gnatmake -Ptest_driver test_runner |
the old test is not replaced with a stub, nor is it lost, but a new test skeleton is created for function Dec.
The only way of regenerating tests skeletons is to remove the previously created tests together with corresponding comment sections.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The generated test driver can treat unimplemented tests in two ways: either count them all as failed (this is useful to see which tests are still left to implement) or as passed (to sort out unimplemented ones from those actually failing).
The test driver accepts a switch to specify this behavior:
--skeleton-default=val, where val is either "pass" or "fail" (exactly as for
gnattest
).
The default behavior of the test driver is set with the same switch as passed to gnattest when generating the test driver.
Passing it to the driver generated on the first example:
test_runner --skeleton-default=pass |
makes both tests pass, even the unimplemented one.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Creation of test skeletons for primitive operations of tagged types entails a number of features. Test routines for all primitives of a given tagged type are placed in a separate child package named according to the tagged type. For example, if you have tagged type T in package P, all tests for primitives of T will be in P.T_Test_Data.T_Tests.
Consider running gnattest on the second example (note: actual tests for this example already exist, so there's no need to worry if the tool reports that no new stubs were generated):
cd <install_prefix>/share/examples/gnattest/tagged_rec gnattest --harness-dir=driver -Ptagged_rec.gpr |
Taking a closer look at the test type declared in the test package Speed1.Controller_Test_Data is necessary. It is declared in:
<install_prefix>/share/examples/gnattest/tagged_rec/obj/gnattest/tests |
Test types are direct or indirect descendants of AUnit.Test_Fixtures.Test_Fixture type. In the case of nonprimitive tested subprograms, the user doesn't need to be concerned with them. However, when generating test packages for primitive operations, there are some things the user needs to know.
Type Test_Controller has components that allow assignment of various derivations of type Controller. And if you look at the specification of package Speed2.Auto_Controller, you will see that Test_Auto_Controller actually derives from Test_Controller rather than AUnit type Test_Fixture. Thus, test types mirror the hierarchy of tested types.
The Set_Up procedure of Test_Data package corresponding to a test package of primitive operations of type T assigns to Fixture a reference to an object of that exact type T. Notice, however, that if the tagged type has discriminants, the Set_Up only has a commented template for setting up the fixture, since filling the discriminant with actual value is up to the user.
The knowledge of the structure of test types allows additional testing without additional effort. Those possibilities are described below.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Since the test type hierarchy mimics the hierarchy of tested types, the inheritance of tests takes place. An example of such inheritance can be seen by running the test driver generated for the second example. As previously mentioned, actual tests are already written for this example.
cd obj/driver gnatmake -Ptest_driver test_runner |
There are 6 passed tests while there are only 5 testable subprograms. The test routine for function Speed has been inherited and run against objects of the derived type.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Tagged Types Substitutability Testing is a way of verifying the global type consistency by testing. Global type consistency is a principle stating that if S is a subtype of T (in Ada, S is a derived type of tagged type T), then objects of type T may be replaced with objects of type S (that is, objects of type S may be substituted for objects of type T), without altering any of the desirable properties of the program. When the properties of the program are expressed in the form of subprogram preconditions and postconditions (let's call them pre and post), the principle is formulated as relations between the pre and post of primitive operations and the pre and post of their derived operations. The pre of a derived operation should not be stronger than the original pre, and the post of the derived operation should not be weaker than the original post. Those relations ensure that verifying if a dispatching call is safe can be done just by using the pre and post of the root operation.
Verifying global type consistency by testing consists of running all the unit tests associated with the primitives of a given tagged type with objects of its derived types.
In the example used in the previous section, there was clearly a violation of type consistency. The overriding primitive Adjust_Speed in package Speed2 removes the functionality of the overridden primitive and thus doesn't respect the consistency principle. Gnattest has a special option to run overridden parent tests against objects of the type which have overriding primitives:
gnattest --harness-dir=driver --validate-type-extensions -Ptagged_rec.gpr cd obj/driver gnatmake -Ptest_driver test_runner |
While all the tests pass by themselves, the parent test for Adjust_Speed fails against objects of the derived type.
Non-overridden tests are already inherited for derived test types, so the --validate-type-extensions enables the application of overriden tests to objects of derived types.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
supports pragmas Precondition, Postcondition, and Test_Case,
as well as corresponding aspects.
Test routines are generated, one per each Test_Case associated with a tested
subprogram. Those test routines have special wrappers for tested functions
that have composition of pre- and postcondition of the subprogram with
"requires" and "ensures" of the Test_Case (depending on the mode, pre and post
either count for Nominal mode or do not count for Robustness mode).
The third example demonstrates how this works:
cd <install_prefix>/share/examples/gnattest/contracts gnattest --harness-dir=driver -Pcontracts.gpr |
Putting actual checks within the range of the contract does not cause any error reports. For example, for the test routine which corresponds to test case 1:
Assert (Sqrt (9.0) = 3.0, "wrong sqrt"); |
and for the test routine corresponding to test case 2:
Assert (Sqrt (-5.0) = -1.0, "wrong error indication"); |
are acceptable:
cd obj/driver gnatmake -Ptest_driver test_runner |
However, by changing 9.0 to 25.0 and 3.0 to 5.0, for example, you can get a precondition violation for test case one. Also, by using any otherwise correct but positive pair of numbers in the second test routine, you can also get a precondition violation. Postconditions are checked and reported the same way.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
can add user-written tests to the main suite of the test
driver. gnattest
traverses the given packages and searches for test
routines. All procedures with a single in out parameter of a type which is
derived from AUnit.Test_Fixtures.Test_Fixture and that are declared in package
specifications are added to the suites and are then executed by the test driver.
(Set_Up and Tear_Down are filtered out.)
An example illustrates two ways of creating test harnesses for user-written tests. Directory additional_tests contains an AUnit-based test driver written by hand.
<install_prefix>/share/examples/gnattest/additional_tests/ |
To create a test driver for already-written tests, use the --harness-only option:
gnattest -Padditional/harness/harness.gpr --harness-dir=harness_only \ --harness-only gnatmake -Pharness_only/test_driver.gpr harness_only/test_runner |
Additional tests can also be executed together with generated tests:
gnattest -Psimple.gpr --additional-tests=additional/harness/harness.gpr \ --harness-dir=mixing gnatmake -Pmixing/test_driver.gpr mixing/test_runner |
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
can be used to generate the test harness for platforms
and run-time libraries others than the default native target with the
default full run-time. For example, when using a limited run-time library
such as Zero FootPrint (ZFP), a simplified harness is generated.
Two variables are used to tell the underlying AUnit framework how to generate
the test harness: PLATFORM
, which identifies the target, and
RUNTIME
, used to determine the run-time library for which the harness
is generated. Corresponding prefix should also be used when calling
gnattest
for non-native targets. For example, the following options
are used to generate the AUnit test harness for a PowerPC ELF target using
the ZFP run-time library:
powerpc-elf-gnattest -Psimple.gpr -XPLATFORM=powerpc-elf -XRUNTIME=zfp |
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The tool currently does not support following features:
[ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |