changelog download home install notes performance

Metis Performance Benchmarks

CASC 2008

Metis 2.1 was entered in the competition with this System Description.

CASC 2007

Metis 2.0 was entered in the competition with this System Description.

FOF Attempted Solved Av. Time Solutions
Vampire 9.0 300 270 20.84 270
Vampire 8.1 300 266 16.37 266
Fampire 1.3 300 262 12.19 0
E 0.999 300 248 13.36 0
EP 0.999 300 225 13.10 221
iProver 0.2 300 201 9.57 0
Equinox 1.2 300 173 14.92 0
leanCoP 2.0 300 160 39.50 0
Otter 3.3 300 138 25.04 138
Metis 2.0 300 117 16.06 89
Geo 2007f 300 104 29.42 104
ileanCoP 1.2 300 103 44.23 0
Muscadet 2.7 300 37 0.76 0

In the FOF division the time limit was 360 seconds, and Metis 2.0 solved 117 out of 300 problems with an average time of 16.06 seconds. The number of solutions is only 89 because the other 28 problems were solved by the FOF → CNF normalization.

The competition website includes the division results, individual problem results and performance graphs.

Here is the paragraph on Metis 2.0 contributed to the CASC 21 report.

CASC 2003

Metis 1.5 was not entered in the official competition, but was run afterwards on the problems used in the competition.

MIX Attempted Solved Av. Time Solutions
Vampire 6.0 140 120 65.64 120
E-SETHEO csp03 140 119 34.65 0
E 0.8 140 113 20.91 0
Vampire 5.0 140 113 23.08 113
EP 0.8 140 113 25.35 110
Gandalf c-2.6 140 102 67.75 0
Gandalf c-2.6-PRF 140 79 30.94 79
DCTP 10.2p 140 72 43.28 0
Metis 1.5 140 71 81.07 71
DCTP 1.3 140 55 18.15 0
THEO J2003 140 49 45.59 47
Otter 3.2 140 34 99.12 34
CARINE 0.72 140 7 104.86 7

In the MIX division the time limit was 600 seconds, and Metis 1.5 solved 71 out of 140 problems with an average time of 81.07 seconds.

In the UEQ division the time limit was 600 seconds, and Metis 1.5 solved 21 out of 70 problems with an average time of 68.68 seconds.

In the FOF division the time limit was 600 seconds, and Metis 1.5 solved 32 out of 70 problems with an average time of 49.87 seconds.

Metis is running on a RedHat 7.1 GNU/Linux box with a Pentium III 600Mhz processor and 512Mb of main memory; the other provers are running on Dell Precision 330 workstations (GNU/Linux boxes with Pentium III 1Ghz processors and 512Mb of main memory).

CASC 2002

Metis 1.5 was not entered in the official competition, but was run afterwards on the problems used in the competition.

MIX Attempted Solved Av. Time Solutions
Vampire 5.0 175 158 25.25 158
Vampire 5.0-CASC 175 157 27.28 157
Vampire 2.0-CASC 175 152 57.33 152
E-SETHEO csp02 175 145 57.60 0
E 0.7 175 131 21.35 0
EP 0.7 175 129 25.58 129
Gandalf c-2.5 175 129 82.92 0
Gandalf c-2.5-PROOF 175 125 77.47 125
Metis 1.5 175 97 77.42 97
Bliksem 1.12a 175 88 43.99 88
DCTP 10.1p 175 76 41.18 0
SCOTT 6.1 175 63 74.48 63
Otter 3.2 175 60 56.34 60
DCTP 1.2 175 56 45.03 0

In the MIX division the time limit was 600 seconds, and Metis 1.5 solved 97 out of 175 problems with an average time of 77.42 seconds.

In the UEQ division the time limit was 300 seconds, and Metis 1.5 solved 7 out of 70 problems with an average time of 50.45 seconds.

In the FOF division the time limit was 300 seconds, and Metis 1.5 solved 35 out of 70 problems with an average time of 32.09 seconds.

Metis is running on a RedHat 7.1 GNU/Linux box with a Pentium III 600Mhz processor and 512Mb of main memory; the other provers are running on Dell Precision 330 workstations (GNU/Linux boxes with Pentium III 1Ghz processors and 512Mb of main memory).


Metis the moon of Jupiter