This module allows executing test cases. The test runner executes
the test cases and sum-marizes the results. The test cases and the
results are certain facts in the knowledge base. The test runner
can be invoked via the predicate runner_batch/0. The coverage
depends on how well the test cases are designed in respect of the
probed execution paths.
:- multifile(test_ref/4). :- discontiguous(test_ref/4).
% test_ref(Fun, Arity, Suite, Ref). :- multifile(test_case/4). :- discontiguous(test_case/4). % test_case(Fun, Arity, Suite, Number) :- Body.
Note: Fun/Arity is used
to denote predicates, and Fun/-Arity-1 is used to denote evaluable
functions. The test steps and the test validation points need to
be implemented in the body of the predicate test_case/4. The body
is assumed to terminate, the test runner doesn't impose some
timeout currently. The body can be used to check a multitude of
% Check whether the goal succeeds: Goal % Check whether the determinitic goal succeeds with result Value: Goal, Var==Value % Check whether the non-determinitic goal first succeeds with result Value: Goal, !, Var==Value % Check whether the goal fails: \+ Goal: % Check whether the goal succeeds on redo: findall(,Goal,[_,_|_]) % Check whether the goal succeeds on redo with result Value: findall(Var,Goal,[_,Var|_]), Var==Value % Check whether the goal fails on redo: findall(,Goal,[_]) % Check whether the goal throws the exception Value: catch(Goal,error(Var,_),true), Var==Value.
The testing is not limited to the above example scenarios.
A particular application domain might need additional test
helper predicates to express the desired test steps and test
validation points. Examples are the CLP(FD) test cases which
use the predicate call_residue/2. The test results are ok and
not-ok counts, a non-conclusive count is currently not provided.
:- public result_summary/1.
:- dynamic result_summary/1.
:- public result_suite/2.
:- dynamic result_suite/2.
% result_suite(Suite, OkNok).
:- public result_predicate/4.
:- dynamic result_predicate/4.
% result_predicate(Fun, Arity, Suite, OkNok).
:- public result/5.
:- dynamic result/5.
% result(Fun, Arity, Suite, Number, OkNok).
The results are accessible by the diagnose module or the report module from the Jekejeke Prolog development environment. Or the tester might code its own analysis based on these facts. In the future we might as well provide additional tools, such as a coverage analysis tool or similar.
The following runner predicates are provided: