Higher-order Theorems Satallax
 2.8
LEO‑II
 1.6.2
Satallax‑MaLeS
 1.3
Isabelle
 2015
Solved/400 271/400 195/400 285/400 267/400
Av. CPU Time 14.96 12.25 21.67 61.02
Solutions 268/400 191/400 0/400 0/400
μEfficiency 390 366 240 30
SOTAC 0.36 0.36 0.35 0.38
Core Usage 0.92 0.91 0.94 0.99
New Solved 4/7 3/7 4/7 4/7
Higher-order Non-theorems Nitpick
 2015
Refute
 2015
Satallax
 2.8
Solved/200 200/200 74/200 49/200
Av. CPU Time 7.92 24.34 0.05
Solutions 200/200 0/200 0/200
μEfficiency 130 29 242
SOTAC 0.70 0.49 0.48
Core Usage 0.92 0.98 0.04
New Solved 5/5 2/5 2/5
Typed First-order Theorems +*-/ VampireZ3
 1.0
CVC4
 TFA‑1.5
Vampire
 4.0
Beagle
 0.9.22
SPASS+T
 2.2.22
ZenonArith
 0.1.0
Princess
 20150706
CVC4
 1.4‑TFF
Solved/200 172/200 163/200 160/200 131/200 108/200 60/200 143/200 131/200
Av. CPU Time 11.85 17.27 10.75 21.76 10.04 2.86 17.38 10.67
Solutions 172/200 163/200 160/200 131/200 108/200 60/200 0/200 0/200
μEfficiency 478 424 422 275 186 281 152 395
SOTAC 0.21 0.21 0.20 0.18 0.14 0.14 0.18 0.17
Core Usage 0.91 0.93 0.90 1.20 1.05 0.71 1.58 0.90
New Solved 4/5 3/5 4/5 0/5 2/5 2/5 0/5 4/5
Typed First-order Non-theorems +*-/ CVC4
 TFN‑1.5
Princess
 20150706
Beagle
 0.9.22
Solved/20 10/20 6/20 6/20
Av. CPU Time 0.00 0.97 1.33
Solutions 0/20 0/20 0/20
μEfficiency 500 200 183
SOTAC 0.50 0.42 0.42
Core Usage 0.00 0.33 0.56
New Solved 0/0 0/0 0/0
First-order Theorems Vampire
 4.0
Vampire
 2.6
E
 1.9.1
ET
 0.2
CVC4
 FOF‑1.5
iProver
 2.0
leanCoP
 2.2
iProverModulo
 0.7‑0.3
Prover9
 1109a
ePrincess
 1.0
Muscadet
 4.5
Geo‑III
 2015E
Solved/400 380/400 371/400 316/400 303/400 257/400 222/400 159/400 127/400 111/400 113/400 37/400 37/400
Av. CPU Time 12.20 14.86 20.18 20.96 33.40 21.12 46.76 30.15 28.01 48.39 7.32 38.47
Solutions 374/400 368/400 316/400 303/400 256/400 217/400 159/400 127/400 111/400 88/400 37/400 37/400
μEfficiency 552 574 450 344 287 215 150 136 138 19 70 50
SOTAC 0.20 0.19 0.17 0.16 0.15 0.14 0.12 0.12 0.14 0.13 0.11 0.14
Core Usage 0.91 0.92 0.94 0.94 0.96 0.95 0.97 0.96 0.96 1.38 0.85 0.97
New Solved 1/1 1/1 1/1 0/1 0/1 1/1 0/1 1/1 0/1 0/1 0/1 0/1
First-order Non-theorems Vampire
 SAT‑4.0
iProver
 SAT‑2.0
iProver
 SAT‑1.0
CVC4
 FNT‑1.5
E
 FNT‑1.9.1
Geo‑III
 2015E
Solved/200 195/200 163/200 134/200 71/200 51/200 38/200
Av. CPU Time 38.95 44.11 79.93 57.78 9.62 21.89
Solutions 195/200 163/200 134/200 71/200 51/200 38/200
μEfficiency 368 226 39 121 127 112
SOTAC 0.37 0.29 0.28 0.25 0.33 0.20
Core Usage 0.97 2.38 0.98 0.98 0.89 0.95
New Solved 2/2 2/2 2/2 2/2 0/2 0/2
Effectively Propositional CNF Vampire
 4.0
iProver
 0.9
iProver
 2.0
E
 1.9.1
Geo‑III
 2015E
Solved/200 192/200 161/200 153/200 101/200 9/200
Av. CPU Time 27.61 27.91 36.57 11.09 86.71
Solutions 192/200 48/200 152/200 101/200 9/200
μEfficiency 445 250 242 327 11
SOTAC 0.39 0.30 0.28 0.25 0.21
Core Usage 0.96 0.96 0.96 0.90 0.99
New Solved 0/0 0/0 0/0 0/0 0/0
Large Theory Batch Problems Vampire
 4.0‑LTB
MaLARea
 0.5
E
 1.9.1‑LTB
iProver
 2.0‑LTB
Solved/1600 1208/1600 837/1600 799/1600 352/1600
Av. CPU Time 24.08 27.91 27.75 30.42
Av. WC Time 6.51 8.30 7.40 13.04
Solutions 1208/1600 837/1600 799/1600 352/1600
μEfficiency 387 96 214 51
SOTAC 0.49 0.41 0.36 0.29
Core Usage 3.70 3.36 3.75 2.33
New Solved 0/0 0/0 0/0 0/0
THF without Equality Satallax
 2.8
LEO‑II
 1.6.2
Satallax‑MaLeS
 1.3
Isabelle
 2015
Solved/100 71/100 49/100 71/100 66/100
Av. CPU Time 3.35 7.47 19.22 34.35
Solutions 71/100 49/100 0/100 0/100
μEfficiency 448 404 256 50
SOTAC 0.34 0.31 0.34 0.35
Core Usage 0.72 0.86 0.93 0.99
New Solved 0/0 0/0 0/0 0/0
THF with Equality Satallax
 2.8
LEO‑II
 1.6.2
Satallax‑MaLeS
 1.3
Isabelle
 2015
Solved/300 200/300 146/300 214/300 201/300
Av. CPU Time 19.08 13.85 22.48 69.78
Solutions 197/300 142/300 0/300 0/300
μEfficiency 370 353 234 23
SOTAC 0.37 0.38 0.35 0.39
Core Usage 0.93 0.92 0.94 0.99
New Solved 4/7 3/7 4/7 4/7
THN without Equality Nitpick
 2015
Refute
 2015
Satallax
 2.8
Solved/25 25/25 25/25 5/25
Av. CPU Time 7.02 24.67 0.00
Solutions 25/25 0/25 0/25
μEfficiency 135 94 200
SOTAC 0.47 0.47 0.33
Core Usage 0.91 0.98 0.00
New Solved 0/0 0/0 0/0
THN with Equality Nitpick
 2015
Refute
 2015
Satallax
 2.8
Solved/175 175/175 49/175 44/175
Av. CPU Time 8.05 24.18 0.06
Solutions 175/175 0/175 0/175
μEfficiency 129 20 249
SOTAC 0.73 0.50 0.50
Core Usage 0.93 0.97 0.05
New Solved 5/5 2/5 2/5
TFA using Integers VampireZ3
 1.0
CVC4
 TFA‑1.5
Vampire
 4.0
Beagle
 0.9.22
SPASS+T
 2.2.22
ZenonArith
 0.1.0
Princess
 20150706
CVC4
 1.4‑TFF
Solved/150 126/150 116/150 114/150 81/150 60/150 12/150 100/150 84/150
Av. CPU Time 16.04 24.27 15.09 34.69 17.19 14.23 22.18 16.64
Solutions 126/150 116/150 114/150 81/150 60/150 12/150 0/150 0/150
μEfficiency 354 253 257 100 88 55 106 213
SOTAC 0.24 0.24 0.23 0.18 0.16 0.17 0.20 0.20
Core Usage 0.93 0.95 0.93 1.24 1.13 0.93 1.63 0.93
New Solved 4/5 3/5 4/5 0/5 2/5 2/5 0/5 4/5
TFA using Rationals CVC4
 TFA‑1.5
ZenonArith
 0.1.0
Beagle
 0.9.22
SPASS+T
 2.2.22
Vampire
 4.0
VampireZ3
 1.0
CVC4
 1.4‑TFF
Princess
 20150706
Solved/35 35/35 35/35 35/35 35/35 34/35 34/35 35/35 33/35
Av. CPU Time 0.00 0.01 0.64 1.11 0.00 0.27 0.00 5.61
Solutions 35/35 35/35 35/35 35/35 34/35 34/35 0/35 0/35
μEfficiency 1000 1000 834 500 971 914 1000 315
SOTAC 0.13 0.13 0.13 0.13 0.13 0.13 0.13 0.13
Core Usage 0.00 0.01 0.29 0.45 0.00 0.19 0.00 1.18
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
TFA using Reals Beagle
 0.9.22
ZenonArith
 0.1.0
SPASS+T
 2.2.22
CVC4
 TFA‑1.5
Vampire
 4.0
VampireZ3
 1.0
CVC4
 1.4‑TFF
Princess
 20150706
Solved/15 15/15 13/15 13/15 12/15 12/15 12/15 12/15 10/15
Av. CPU Time 1.20 0.02 1.12 0.00 0.00 0.66 0.00 8.21
Solutions 15/15 13/15 13/15 12/15 12/15 12/15 0/15 0/15
μEfficiency 723 867 433 800 800 700 800 237
SOTAC 0.25 0.14 0.14 0.13 0.13 0.13 0.13 0.13
Core Usage 0.50 0.02 0.42 0.00 0.00 0.37 0.00 1.41
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
TFN using Integers CVC4
 TFN‑1.5
Princess
 20150706
Beagle
 0.9.22
Solved/15 8/15 6/15 4/15
Av. CPU Time 0.00 0.97 0.59
Solutions 0/15 0/15 0/15
μEfficiency 533 267 200
SOTAC 0.50 0.42 0.38
Core Usage 0.00 0.33 0.29
New Solved 0/0 0/0 0/0
TFN using Rationals CVC4
 TFN‑1.5
Beagle
 0.9.22
Princess
 20150706
Solved/3 1/3 1/3 0/3
Av. CPU Time 0.00 2.81 -
Solutions 0/3 0/3 0/3
μEfficiency 333 111 -
SOTAC 0.50 0.50 -
Core Usage 0.00 0.93 -
New Solved 0/0 0/0 0/0
TFN using Reals CVC4
 TFN‑1.5
Beagle
 0.9.22
Princess
 20150706
Solved/2 1/2 1/2 0/2
Av. CPU Time 0.01 2.81 -
Solutions 0/2 0/2 0/2
μEfficiency 500 167 -
SOTAC 0.50 0.50 -
Core Usage 0.01 0.93 -
New Solved 0/0 0/0 0/0
FOF Theorems without Equality Vampire
 2.6
Vampire
 4.0
E
 1.9.1
iProver
 2.0
ET
 0.2
CVC4
 FOF‑1.5
iProverModulo
 0.7‑0.3
leanCoP
 2.2
ePrincess
 1.0
Prover9
 1109a
Muscadet
 4.5
Geo‑III
 2015E
Solved/150 144/150 139/150 122/150 125/150 119/150 111/150 91/150 85/150 35/150 29/150 18/150 15/150
Av. CPU Time 7.24 10.76 12.14 12.75 19.15 35.59 23.48 45.95 51.49 16.65 6.52 17.08
Solutions 141/150 137/150 122/150 122/150 119/150 110/150 91/150 85/150 33/150 29/150 18/150 15/150
μEfficiency 719 591 543 339 404 248 234 199 11 112 83 55
SOTAC 0.17 0.16 0.14 0.14 0.13 0.14 0.12 0.12 0.10 0.11 0.11 0.16
Core Usage 0.85 0.90 0.91 0.92 0.93 0.96 0.95 0.98 1.40 0.93 0.83 0.94
New Solved 1/1 1/1 1/1 1/1 0/1 0/1 1/1 0/1 0/1 0/1 0/1 0/1
FOF Theorems with Equality Vampire
 4.0
Vampire
 2.6
E
 1.9.1
ET
 0.2
CVC4
 FOF‑1.5
iProver
 2.0
Prover9
 1109a
leanCoP
 2.2
ePrincess
 1.0
iProverModulo
 0.7‑0.3
Geo‑III
 2015E
Muscadet
 4.5
Solved/250 241/250 227/250 194/250 184/250 146/250 97/250 82/250 74/250 78/250 36/250 22/250 19/250
Av. CPU Time 13.03 19.70 25.23 22.12 31.74 31.89 32.03 47.69 47.00 47.02 53.05 8.08
Solutions 237/250 227/250 194/250 184/250 146/250 95/250 82/250 74/250 55/250 36/250 22/250 19/250
μEfficiency 529 487 395 307 310 140 154 120 23 77 47 62
SOTAC 0.22 0.20 0.20 0.18 0.16 0.15 0.15 0.13 0.14 0.13 0.12 0.12
Core Usage 0.91 0.94 0.95 0.94 0.96 0.98 0.96 0.97 1.37 0.97 0.98 0.87
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Non-theorems without Equality Vampire
 SAT‑4.0
iProver
 SAT‑2.0
iProver
 SAT‑1.0
E
 FNT‑1.9.1
CVC4
 FNT‑1.5
Geo‑III
 2015E
Solved/110 106/110 92/110 77/110 32/110 21/110 8/110
Av. CPU Time 54.19 49.67 76.74 9.15 88.01 30.34
Solutions 106/110 92/110 77/110 32/110 21/110 8/110
μEfficiency 117 94 55 96 33 20
SOTAC 0.36 0.31 0.30 0.36 0.28 0.20
Core Usage 0.98 2.60 0.99 0.89 0.99 0.96
New Solved 2/2 2/2 2/2 0/2 2/2 0/2
FOF Non-theorems with Equality Vampire
 SAT‑4.0
iProver
 SAT‑2.0
iProver
 SAT‑1.0
CVC4
 FNT‑1.5
Geo‑III
 2015E
E
 FNT‑1.9.1
Solved/90 89/90 71/90 57/90 50/90 30/90 19/90
Av. CPU Time 20.80 36.90 84.23 45.09 19.63 10.42
Solutions 89/90 71/90 57/90 50/90 30/90 19/90
μEfficiency 675 387 20 228 225 165
SOTAC 0.37 0.27 0.24 0.23 0.20 0.29
Core Usage 0.94 2.08 0.98 0.97 0.94 0.89
New Solved 0/0 0/0 0/0 0/0 0/0 0/0
EPR Unsatisfiable CNF Vampire
 4.0
iProver
 0.9
iProver
 2.0
E
 1.9.1
Geo‑III
 2015E
Solved/125 123/125 106/125 96/125 61/125 1/125
Av. CPU Time 35.29 31.17 37.80 5.62 0.04
Solutions 123/125 0/125 95/125 61/125 1/125
μEfficiency 428 199 213 382 8
SOTAC 0.39 0.31 0.29 0.25 0.20
Core Usage 0.97 0.96 0.97 0.83 0.03
New Solved 0/0 0/0 0/0 0/0 0/0
EPR Satisfiable CNF Vampire
 4.0
iProver
 2.0
iProver
 0.9
E
 1.9.1
Geo‑III
 2015E
Solved/75 69/75 57/75 55/75 40/75 8/75
Av. CPU Time 13.93 34.51 21.63 19.45 97.55
Solutions 69/75 57/75 48/75 40/75 8/75
μEfficiency 473 289 334 234 16
SOTAC 0.37 0.28 0.28 0.25 0.21
Core Usage 0.92 0.96 0.95 0.93 0.99
New Solved 0/0 0/0 0/0 0/0 0/0
LTB HOL Light Theorems Vampire
 4.0‑LTB
MaLARea
 0.5
E
 1.9.1‑LTB
iProver
 2.0‑LTB
Solved/400 279/400 243/400 154/400 59/400
Av. CPU Time 26.76 24.41 34.75 32.41
Av. WC Time 6.93 7.24 8.96 10.64
Solutions 279/400 243/400 154/400 59/400
μEfficiency 328 114 187 36
SOTAC 0.49 0.46 0.35 0.26
Core Usage 3.86 3.37 3.88 3.05
New Solved 0/0 0/0 0/0 0/0
LTB HOL 4 Theorems Vampire
 4.0‑LTB
E
 1.9.1‑LTB
iProver
 2.0‑LTB
MaLARea
 0.5
Solved/400 322/400 214/400 77/400 9/400
Av. CPU Time 25.38 31.66 59.00 11.90
Av. WC Time 7.03 8.55 18.36 3.95
Solutions 322/400 214/400 77/400 9/400
μEfficiency 361 175 24 7
SOTAC 0.64 0.46 0.33 0.26
Core Usage 3.61 3.70 3.21 3.01
New Solved 0/0 0/0 0/0 0/0
LTB Isabelle Theorems Vampire
 4.0‑LTB
MaLARea
 0.5
E
 1.9.1‑LTB
iProver
 2.0‑LTB
Solved/400 319/400 280/400 202/400 21/400
Av. CPU Time 29.37 40.14 34.79 49.07
Av. WC Time 8.09 11.49 9.54 28.51
Solutions 319/400 280/400 202/400 21/400
μEfficiency 329 66 78 2
SOTAC 0.46 0.41 0.34 0.26
Core Usage 3.63 3.49 3.65 1.72
New Solved 0/0 0/0 0/0 0/0
LTB Mizar Theorems MaLARea
 0.5
Vampire
 4.0‑LTB
E
 1.9.1‑LTB
iProver
 2.0‑LTB
Solved/400 305/400 288/400 229/400 195/400
Av. CPU Time 19.94 14.19 13.18 16.52
Av. WC Time 6.33 3.74 3.40 10.00
Solutions 305/400 288/400 229/400 195/400
μEfficiency 196 532 417 143
SOTAC 0.37 0.34 0.29 0.29
Core Usage 3.15 3.79 3.87 1.65
New Solved 0/0 0/0 0/0 0/0