All of the systems that competed in CASC-16 passed the
soundness testing
that was performed before the competition.
However, in August 1999, i.e., about a month after the competition,
E 0.5 and
E-SETHEO(-FLOTTER) 99csp were found to be unsound in certain
rare circumstances.
The unsoundness was due to a bug in E, which was also used as a component
of E-SETHEO.
Unsoundness is unacceptable, and the competition panel retrospectively
disqualified the two systems from being ranked in the competition.
It must be noted that the unsoundness was entirely accidental, and that
there was no attempt to deceive.
The circumstances in which the two systems are unsound did not occur in
the competition, and therefore the performance of these systems in the
competition was not affected.
Subsequent testing of the repaired systems verified that the same
problems could be solved.
The results of these two unranked systems are therefore still of interest,
and are presented here.
Results for MIXProblems
| E-SETHEO 99csp | Vampire 0.0 | SPASS 1.00T | E 0.5 | Gandalf c-1.8c | Gandalf c-2.0 | Bliksem 1.10 | SCOTT 4.6.4 | OtterMACE 437 | FDP 0.9
|
|---|
| Attempted
| 75 | 75 | 75 | 75 | 75 | 75 | 75 | 75 | 75 | 75
|
|---|
| Solved
| 54 | 51 | 50 | 47 | 39 | 36 | 24 | 18 | 16 | 8
|
|---|
| Av. Time
| 70.33 | 33.66 | 40.95 | 47.85 | 59.99 | 56.74 | 16.40 | 103.93 | 75.18 | 153.16
|
|---|
Results for UEQProblems
| Waldmeister 799 | Waldmeister 798 | SCOTT 4.6.4 | E 0.5 | SPASS 1.00T | Fiesta 2 | OtterMACE 437 | E-SETHEO 99csp | Gandalf c-2.0 | Vampire 0.0 | Bliksem 1.10
|
|---|
| Attempted
| 30 | 30 | 30 | 30 | 30 | 30 | 30 | 30 | 30 | 30 | 30
|
|---|
| Solved
| 30 | 19 | 13 | 10 | 7 | 5 | 4 | 3 | 3 | 1 | 1
|
|---|
| Av. Time
| 22.71 | 12.22 | 95.22 | 159.78 | 32.07 | 77.98 | 44.65 | 99.77 | 106.13 | 27.80 | 61.90
|
|---|
Results for SATProblems
| OtterMACE 437 | SPASS 1.00T | SPASS 0.95T | FDP 0.9 | E-SETHEO 99csp | Bliksem 1.10
|
|---|
| Attempted
| 30 | 30 | 30 | 30 | 30 | 30
|
|---|
| Solved
| 16 | 11 | 9 | 9 | 8 | 3
|
|---|
| Av. Time
| 8.47 | 22.16 | 16.69 | 56.21 | 48.51 | 0.00
|
|---|
Results for FOFProblems
| SPASS 1.00T | E-SETHEO 99csp | SPASS 0.95T | Bliksem 1.10 | MUSCADET 2.1 | OtterMACE 437
|
|---|
| Attempted
| 30 | 30 | 30 | 30 | 30 | 30
|
|---|
| Solved
| 22 | 22 | 19 | 12 | 9 | 9
|
|---|
| Av. Time
| 25.46 | 32.30 | 28.36 | 1.58 | 1.48 | 7.54
|
|---|
Results for HEQProblems
| E 0.5 | E-SETHEO 99csp | Vampire 0.0 | SCOTT 4.6.4 | SPASS 1.00T | Gandalf c-1.8c | OtterMACE 437 | Gandalf c-2.0 | Bliksem 1.10 | FDP 0.9
|
|---|
| Attempted
| 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15
|
|---|
| Solved
| 13 | 10 | 8 | 8 | 5 | 3 | 3 | 2 | 0 | 0
|
|---|
| Av. Time
| 72.12 | 131.36 | 35.20 | 92.51 | 55.50 | 46.63 | 82.40 | 59.65 | - | -
|
|---|
Results for HNEProblems
| Vampire 0.0 | E 0.5 | E-SETHEO 99csp | Gandalf c-2.0 | Gandalf c-1.8c | SPASS 1.00T | Bliksem 1.10 | OtterMACE 437 | SCOTT 4.6.4 | FDP 0.9
|
|---|
| Attempted
| 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15
|
|---|
| Solved
| 11 | 10 | 10 | 10 | 9 | 8 | 4 | 3 | 2 | 0
|
|---|
| Av. Time
| 31.33 | 54.55 | 73.35 | 81.11 | 73.12 | 18.14 | 56.52 | 185.03 | 144.55 | -
|
|---|
Results for NEQProblems
| Gandalf c-2.0 | Gandalf c-1.8c | SPASS 1.00T | E-SETHEO 99csp | E 0.5 | Vampire 0.0 | Bliksem 1.10 | OtterMACE 437 | SCOTT 4.6.4 | FDP 0.9
|
|---|
| Attempted
| 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15
|
|---|
| Solved
| 13 | 13 | 12 | 10 | 9 | 9 | 6 | 6 | 5 | 1
|
|---|
| Av. Time
| 47.15 | 59.23 | 28.64 | 69.50 | 10.74 | 14.39 | 0.77 | 39.00 | 105.64 | 0.70
|
|---|
Results for NNEProblems
| SPASS 1.00T | Vampire 0.0 | E-SETHEO 99csp | FDP 0.9 | Gandalf c-1.8c | Gandalf c-2.0 | Bliksem 1.10 | E 0.5 | OtterMACE 437 | SCOTT 4.6.4
|
|---|
| Attempted
| 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15
|
|---|
| Solved
| 14 | 11 | 10 | 7 | 3 | 3 | 1 | 1 | 0 | 0
|
|---|
| Av. Time
| 65.01 | 74.53 | 38.31 | 174.94 | 40.57 | 67.40 | 1.60 | 2.80 | - | -
|
|---|
Results for PEQProblems
| E 0.5 | E-SETHEO 99csp | Bliksem 1.10 | Vampire 0.0 | SPASS 1.00T | Gandalf c-1.8c | Gandalf c-2.0 | OtterMACE 437 | SCOTT 4.6.4 | FDP 0.9
|
|---|
| Attempted
| 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15
|
|---|
| Solved
| 14 | 14 | 13 | 12 | 11 | 11 | 8 | 4 | 3 | 0
|
|---|
| Av. Time
| 47.60 | 48.04 | 12.41 | 11.78 | 33.75 | 59.07 | 37.14 | 41.65 | 104.43 | -
|
|---|
Results for FEQProblems
| MUSCADET 2.1 | SPASS 1.00T | E-SETHEO 99csp | Bliksem 1.10 | SPASS 0.95T | OtterMACE 437
|
|---|
| Attempted
| 15 | 15 | 15 | 15 | 15 | 15
|
|---|
| Solved
| 9 | 8 | 7 | 5 | 5 | 4
|
|---|
| Av. Time
| 1.48 | 0.76 | 28.91 | 0.20 | 2.12 | 16.78
|
|---|
Results for FNEProblems
| E-SETHEO 99csp | SPASS 0.95T | SPASS 1.00T | Bliksem 1.10 | OtterMACE 437 | MUSCADET 2.1
|
|---|
| Attempted
| 15 | 15 | 15 | 15 | 15 | 15
|
|---|
| Solved
| 15 | 14 | 14 | 7 | 5 | 0
|
|---|
| Av. Time
| 33.88 | 37.74 | 39.57 | 2.57 | 0.16 | -
|
|---|