Higher-order Theorems Satallax_MaLeS
 1.3
Satallax
 2.7
Isabelle
 2013
LEO‑II
 1.6.2
agsyHOL
 1.0
HOLyHammer
 140616
cocATP
 0.2.0
Solved/400 333/400 324/400 316/400 269/400 236/400 96/400 74/400
Av. CPU Time 18.88 5.68 38.91 7.21 5.83 25.89 4.65
Solutions 0/400 0/400 0/400 265/400 0/400 0/400 74/400
μEfficiency 313 576 113 554 499 130 139
SOTAC 0.24 0.25 0.23 0.23 0.24 0.23 0.21
Core Usage 1.00 0.99 0.97 0.99 1.00 1.00 0.99
New Solved 8/8 8/8 5/8 7/8 5/8 0/8 1/8
Typed First-order Theorems +*-/ CVC4
 1.4‑TFA
Princess
 140704
SPASS+T
 2.2.19
SPASS+T
 2.2.20
Beagle
 0.9
Zipperposition
 0.4‑TFF
Solved/200 179/200 176/200 173/200 173/200 173/200 80/200
Av. CPU Time 4.47 11.81 3.44 3.57 5.49 6.57
Solutions 0/200 0/200 173/200 173/200 0/200 80/200
μEfficiency 797 307 402 402 623 313
SOTAC 0.22 0.21 0.19 0.19 0.20 0.27
Core Usage 1.30 1.19 1.83 1.79 1.21 0.99
New Solved 33/50 35/50 30/50 30/50 28/50 44/50
First-order Theorems Vampire
 2.6
ET
 0.1
E
 1.9
VanHElsing
 1.0
CVC4
 1.4‑FOF
iProver
 1.4
leanCoP
 2.2
Prover9
 1109a
Zipperposition
 0.4‑FOF
Muscadet
 4.4
Princess
 140704
Solved/400 375/400 339/400 321/400 310/400 215/400 216/400 158/400 95/400 73/400 32/400 134/400
Av. CPU Time 13.19 29.31 22.88 17.29 46.03 18.11 55.15 41.45 28.81 19.74 69.31
Solutions 372/400 339/400 321/400 310/400 215/400 214/400 158/400 95/400 73/400 30/400 0/400
μEfficiency 571 361 466 168 228 216 129 119 75 47 17
SOTAC 0.22 0.18 0.17 0.17 0.15 0.16 0.14 0.14 0.13 0.12 0.13
Core Usage 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.99 1.22
New Solved 5/6 5/6 0/6 0/6 0/6 6/6 0/6 0/6 0/6 0/6 0/6
First-order Non-theorems iProver
 1.0‑SAT
iProver
 1.4‑SAT
Crossbow
 0.1
CVC4
 1.4‑FNT
E
 1.9‑FNT
Solved/200 150/200 133/200 124/200 116/200 82/200
Av. CPU Time 59.12 63.37 21.66 24.38 7.23
Solutions 150/200 133/200 124/200 116/200 82/200
μEfficiency 146 186 107 374 292
SOTAC 0.30 0.29 0.32 0.25 0.38
Core Usage 1.00 1.00 1.00 1.00 0.99
New Solved 4/10 4/10 9/10 6/10 0/10
Effectively Propositional CNF iProver
 0.9
iProver
 1.4
E
 1.9
Solved/200 183/200 180/200 88/200
Av. CPU Time 22.96 22.80 31.02
Solutions 62/200 174/200 88/200
μEfficiency 400 405 98
SOTAC 0.44 0.43 0.36
Core Usage 1.00 1.00 1.00
New Solved 10/10 9/10 5/10
Unsatisfiable Unit Equality CNF Waldmeister
 710
E
 1.9
Solved/200 194/200 168/200
Av. CPU Time 11.75 13.08
Solutions 194/200 168/200
μEfficiency 746 633
SOTAC 0.57 0.50
Core Usage 1.00 1.00
New Solved 0/0 0/0
THF without Equality Satallax
 2.7
Satallax_MaLeS
 1.3
Isabelle
 2013
LEO‑II
 1.6.2
agsyHOL
 1.0
cocATP
 0.2.0
HOLyHammer
 140616
Solved/150 128/150 126/150 126/150 106/150 87/150 37/150 27/150
Av. CPU Time 3.52 9.45 23.89 5.81 3.65 1.50 19.58
Solutions 0/150 0/150 0/150 106/150 0/150 37/150 0/150
μEfficiency 648 356 168 607 539 206 128
SOTAC 0.23 0.23 0.23 0.20 0.23 0.22 0.21
Core Usage 0.98 0.99 1.01 0.99 0.99 0.95 0.98
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
THF with Equality Satallax_MaLeS
 1.3
Satallax
 2.7
Isabelle
 2013
LEO‑II
 1.6.2
agsyHOL
 1.0
HOLyHammer
 140616
cocATP
 0.2.0
Solved/250 207/250 196/250 190/250 163/250 149/250 69/250 37/250
Av. CPU Time 24.62 7.10 48.88 8.13 7.11 28.36 7.79
Solutions 0/250 0/250 0/250 159/250 0/250 0/250 37/250
μEfficiency 287 533 80 522 475 132 99
SOTAC 0.24 0.26 0.24 0.24 0.24 0.24 0.19
Core Usage 1.00 0.99 0.97 0.99 1.00 1.00 0.99
New Solved 8/8 8/8 5/8 7/8 5/8 0/8 1/8
TFA using Integers Princess
 140704
Zipperposition
 0.4‑TFF
CVC4
 1.4‑TFA
SPASS+T
 2.2.19
SPASS+T
 2.2.20
Beagle
 0.9
Solved/100 81/100 80/100 80/100 75/100 75/100 73/100
Av. CPU Time 20.30 6.57 10.00 6.51 6.80 12.57
Solutions 0/100 80/100 0/100 75/100 75/100 0/100
μEfficiency 291 626 605 314 314 325
SOTAC 0.22 0.27 0.24 0.18 0.18 0.18
Core Usage 1.19 0.99 1.30 1.83 1.79 1.21
New Solved 35/50 44/50 33/50 30/50 30/50 28/50
TFA using Rationals CVC4
 1.4‑TFA
Beagle
 0.9
SPASS+T
 2.2.19
SPASS+T
 2.2.20
Princess
 140704
Zipperposition
 0.4‑TFF
Solved/50 50/50 50/50 50/50 50/50 49/50 0/50
Av. CPU Time 0.00 0.22 1.09 1.09 3.53 -
Solutions 0/50 0/50 50/50 50/50 0/50 0/50
μEfficiency 1000 945 500 500 344 -
SOTAC 0.20 0.20 0.20 0.20 0.20 -
Core Usage 0.03 1.01 0.48 0.48 1.79 -
New Solved 0/0 0/0 0/0 0/0 0/0 0/0
TFA using Reals Beagle
 0.9
CVC4
 1.4‑TFA
SPASS+T
 2.2.20
SPASS+T
 2.2.19
Princess
 140704
Zipperposition
 0.4‑TFF
Solved/50 50/50 49/50 48/50 48/50 46/50 0/50
Av. CPU Time 0.44 0.00 1.09 1.09 5.67 -
Solutions 0/50 0/50 48/50 48/50 0/50 0/50
μEfficiency 899 980 480 480 301 -
SOTAC 0.22 0.21 0.20 0.20 0.20 -
Core Usage 1.15 0.02 0.49 0.49 1.87 -
New Solved 0/0 0/0 0/0 0/0 0/0 0/0
FOF Theorems without Equality Vampire
 2.6
E
 1.9
ET
 0.1
iProver
 1.4
VanHElsing
 1.0
leanCoP
 2.2
CVC4
 1.4‑FOF
Zipperposition
 0.4‑FOF
Prover9
 1109a
Muscadet
 4.4
Princess
 140704
Solved/150 141/150 116/150 115/150 116/150 111/150 84/150 83/150 26/150 21/150 6/150 47/150
Av. CPU Time 9.19 9.55 19.65 9.69 11.68 49.87 77.71 41.62 3.21 39.95 109.14
Solutions 139/150 116/150 115/150 114/150 111/150 84/150 83/150 26/150 21/150 6/150 0/150
μEfficiency 684 560 383 341 180 220 131 47 109 3 4
SOTAC 0.24 0.15 0.15 0.16 0.15 0.14 0.16 0.16 0.12 0.13 0.12
Core Usage 0.99 0.99 1.00 1.00 1.00 1.00 1.00 1.00 0.96 0.99 1.28
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Theorems with Equality Vampire
 2.6
ET
 0.1
E
 1.9
VanHElsing
 1.0
CVC4
 1.4‑FOF
iProver
 1.4
Prover9
 1109a
leanCoP
 2.2
Zipperposition
 0.4‑FOF
Muscadet
 4.4
Princess
 140704
Solved/250 234/250 224/250 205/250 199/250 132/250 100/250 74/250 74/250 47/250 26/250 87/250
Av. CPU Time 15.60 34.27 30.43 20.42 26.11 27.88 52.30 61.15 21.72 15.07 47.79
Solutions 233/250 224/250 205/250 199/250 132/250 100/250 74/250 74/250 47/250 24/250 0/250
μEfficiency 504 348 409 161 286 140 125 75 92 74 26
SOTAC 0.21 0.20 0.19 0.18 0.15 0.15 0.14 0.13 0.12 0.11 0.13
Core Usage 1.00 1.00 1.00 1.00 0.99 1.00 1.00 1.00 0.99 0.99 1.17
New Solved 5/6 5/6 0/6 0/6 0/6 6/6 0/6 0/6 0/6 0/6 0/6
FOF Non-theorems without Equality iProver
 1.0‑SAT
Crossbow
 0.1
iProver
 1.4‑SAT
CVC4
 1.4‑FNT
E
 1.9‑FNT
Solved/100 79/100 77/100 77/100 54/100 42/100
Av. CPU Time 47.38 24.86 52.17 34.94 2.15
Solutions 79/100 77/100 77/100 54/100 42/100
μEfficiency 241 143 322 266 265
SOTAC 0.31 0.30 0.30 0.23 0.33
Core Usage 1.00 1.00 0.99 1.00 0.95
New Solved 0/0 0/0 0/0 0/0 0/0
FOF Non-theorems with Equality iProver
 1.0‑SAT
CVC4
 1.4‑FNT
iProver
 1.4‑SAT
Crossbow
 0.1
E
 1.9‑FNT
Solved/100 71/100 62/100 56/100 47/100 40/100
Av. CPU Time 72.18 15.18 78.77 16.42 12.56
Solutions 71/100 62/100 56/100 47/100 40/100
μEfficiency 51 481 49 72 319
SOTAC 0.30 0.28 0.28 0.36 0.43
Core Usage 1.00 0.99 1.00 0.99 0.99
New Solved 4/10 6/10 4/10 9/10 0/10
EPR Unsatisfiable CNF iProver
 0.9
iProver
 1.4
E
 1.9
Solved/125 116/125 114/125 56/125
Av. CPU Time 26.58 22.46 33.08
Solutions 0/125 108/125 56/125
μEfficiency 355 361 61
SOTAC 0.43 0.42 0.36
Core Usage 1.00 1.00 0.99
New Solved 10/10 9/10 5/10
EPR Satisfiable CNF iProver
 0.9
iProver
 1.4
E
 1.9
Solved/75 67/75 66/75 32/75
Av. CPU Time 16.70 23.38 27.43
Solutions 62/75 66/75 32/75
μEfficiency 474 479 160
SOTAC 0.45 0.44 0.35
Core Usage 0.99 1.00 1.00
New Solved 0/0 0/0 0/0
Unsatisfiable Unit Equality CNF Waldmeister
 710
E
 1.9
Solved/200 194/200 168/200
Av. CPU Time 11.75 13.08
Solutions 194/200 168/200
μEfficiency 746 633
SOTAC 0.57 0.50
Core Usage 1.00 1.00
New Solved 0/0 0/0