In this section we will describe our testing approach in more
detail. In particular we will give details how we measured the
test cases and under what circumstances we run the test cases. The
test harness was used to perform internal and external tests. In
the internal tests we compared Jekejeke Prolog in different
configurations. In the external tests we compared the optimal
Jekejeke Prolog configuration with different existing products.
The test scope included the following features of a Prolog
The test scope did not include the following features of a Prolog interpreter:
To perform the internal and external tests we used the same set
of test programs and the same kind of harness. Each test program
consists of an entry point that will compute the solution to a
problem. This entry point was then iterated via the test harness.
The number of iterations was chosen in such a way that for each
test program a time measure of the same order can be obtained. The
following test programs and number of iterations were used:
|6'001||qsort||Quick sort of a list.|
|31||mtak||McCarthy Tak function.|
When doing the internal tests we varied some Prolog flags. We used a Java class and our Prolog API as the main entry point so that the runtime library is tested. For the external tests we used the Java class again. For each compared Prolog system we had to adapt the code that retrieves the elapsed runtime. Details on the test harness and the test programs can be found in the appendix.