Test Scope

In this section we will describe our testing approach in more detail. In particular we will give details how we measured the test cases and under what circumstances we run the test cases. The test harness was used to perform internal and external tests. In the internal tests we compared Jekejeke Prolog in different configurations. In the external tests we compared the optimal Jekejeke Prolog configuration with different existing products.

The test scope included the following features of a Prolog interpreter:

The test scope did not include the following features of a Prolog interpreter:

To perform the internal and external tests we used the same set of test programs and the same kind of harness. Each test program consists of an entry point that will compute the solution to a problem. This entry point was then iterated via the test harness. The number of iterations was chosen in such a way that for each test program a time measure of the same order can be obtained. The following test programs and number of iterations were used:

Table 1: Iterations of the Test Programs
Iterations
Name
Description
6'001 nrev Naïve reverse
301 crypt Crypt arithmetic.
30'001 deriv Symbolic derivation.
61 poly Polynomial reduction.
6'001 qsort Quick sort of a list.
11
tictac Tic-Tac-Toe game.
16 queens 9-queens problem.
3'001 query Database query.
31 mtak McCarthy Tak function.
16 perfect Perfect numbers.
20'001 calc
DCG calculator.

When doing the internal tests we varied some Prolog flags. We used a Java class and our Prolog API as the main entry point so that the runtime library is tested. For the external tests we used the Java class again. For each compared Prolog system we had to adapt the code that retrieves the elapsed runtime. Details on the test harness and the test programs can be found in the appendix.

Comments