Higher-order Theorems Zipperpin
2.1
Zipperpin
2.1.999
E
3.0
Vampire
4.7
Leo-III
1.7.0
Satallax
3.4
cvc5
1.0
Lash
1.12
LEO-II
1.7.0
Solved/500 460/500 456/500 419/500 367/500 336/500 329/500 282/500 280/500 174/500
Av. WC Time 8.73 9.15 3.10 7.54 6.11 20.28 5.93 7.57 3.78
Av. CPU Time 61.47 65.00 13.76 54.45 19.83 20.25 5.70 7.41 3.60
Solutions 460 92% 456 91% 419 83% 367 73% 335 67% 329 65% 282 56% 262 52% 173 34%
μWCEfficiency 527 529 684 488 228 308 425 410 307
μEfficiency 436 423 634 396 124 317 431 419 310
SotAC - 0.34 0.28 0.21 0.15 0.15 0.10 0.10 0.03
Core Usage 3.74 3.72 1.39 3.43 2.51 0.74 0.62 0.61 0.53
New Solved 128/142 133/142 121/142 109/142 92/142 94/142 73/142 77/142 43/142
Typed First-order Theorems +*-/ SnakeForV4.7
1.0
cvc5
1.0
Vampire
4.5
Vampire
4.7
iProver
3.6
Solved/250 218/250 195/250 192/250 187/250 138/250
Av. WC Time 5.46 20.45 5.56 4.35 17.74
Av. CPU Time 37.85 20.23 18.83 29.63 127.43
Solutions 218 87% 195 78% 192 76% 187 74% 137 54%
μWCEfficiency 620 289 513 495 130
μEfficiency 346 291 295 337 69
SotAC 0.27 0.24 - 0.17 0.06
Core Usage 4.06 0.83 3.63 3.87 5.20
New Solved 0/0 0/0 0/0 0/0 0/0
First-order Theorems SnakeForV4.7
1.0
Vampire
4.7
Vampire
4.6
E
3.0
iProver
3.6
CSE_E
1.4
GKC
0.7
Zipperpin
2.1.999
cvc5
1.0
Drodi
3.3.3
CSE
1.5
Prover9
1109a
Goeland
1.0.0
Etableau
0.67
Solved/500 460/500 451/500 448/500 384/500 365/500 361/500 335/500 294/500 271/500 205/500 128/500 123/500 7/500 279/500
Av. WC Time 4.99 5.44 6.15 8.47 15.79 11.50 8.18 12.79 23.47 7.83 31.62 14.62 6.53 6.29
Av. CPU Time 32.02 35.63 30.77 54.05 103.95 11.40 57.51 81.65 23.26 57.00 31.55 14.33 36.65 36.58
Solutions 460 92% 451 90% 448 89% 384 76% 365 73% 361 72% 335 67% 294 58% 271 54% 205 41% 128 25% 123 24% 7  1% 0  0%
μWCEfficiency 672 664 687 479 191 396 420 286 229 271 76 93 7 359
μEfficiency 450 524 518 397 87 399 261 206 235 183 75 119 6 272
SotAC 0.36 0.35 - 0.26 0.23 0.23 0.20 0.16 0.16 0.09 0.05 0.05 0.00 0.15
Core Usage 3.36 2.82 2.69 3.00 4.63 0.91 4.12 3.82 0.82 3.97 0.95 0.87 3.40 3.04
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
First-order Non-theorems Vampire
4.6
Vampire
4.7
SnakeForV4.7
1.0
cvc5
1.0
iProver
3.6
E
3.0
Solved/250 167/250 160/250 159/250 78/250 63/250 62/250
Av. WC Time 16.23 18.08 16.40 31.62 20.49 5.13
Av. CPU Time 55.99 138.71 126.87 31.42 152.10 19.01
Solutions 167 66% 160 64% 159 63% 78 31% 63 25% 62 24%
μWCEfficiency 322 325 340 90 60 186
μEfficiency 230 227 227 94 28 157
SotAC - 0.25 0.25 0.09 0.07 0.05
Core Usage 3.54 4.78 4.68 0.91 5.51 2.21
New Solved 0/0 0/0 0/0 0/0 0/0 0/0
Unit Equality CNF Twee
2.4.1
Twee
2.4
E
3.0
SnakeForV4.7
1.0
Vampire
4.7
iProver
3.6
GKC
0.7
Drodi
3.3.3
Toma
0.2
Solved/250 216/250 215/250 212/250 207/250 152/250 142/250 120/250 116/250 1/250
Av. WC Time 7.37 7.54 11.07 6.37 8.98 14.38 11.35 5.75 2.22
Av. CPU Time 55.53 56.85 74.14 45.18 67.32 101.19 85.32 43.31 1.99
Solutions 216 86% 215 86% 212 84% 207 82% 152 60% 142 56% 120 48% 116 46% 1  0%
μWCEfficiency 585 574 527 546 381 158 230 295 1
μEfficiency 396 375 392 305 218 71 137 199 2
SotAC 0.31 - 0.29 0.27 0.13 0.11 0.09 0.07 0.00
Core Usage 4.02 4.19 3.72 4.41 4.66 4.83 5.18 4.28 0.90
New Solved 38/46 36/46 43/46 32/46 20/46 23/46 19/46 19/46 1/46
SLedgeHammer Theorems E
3.0
Ehoh
2.7
Zipperpin
2.1.999
Vampire
4.7
cvc5
1.0
Solved/720 655/720 655/720 654/720 629/720 595/720
Av. WC Time 1.94 2.18 1.30 3.43 0.97
Av. CPU Time 1.72 1.99 5.41 3.30 0.81
Solutions 655 90% 655 90% 654 90% 629 87% 595 82%
μWCEfficiency 788 745 529 486 778
μEfficiency 780 738 688 463 763
SotAC 0.23 - 0.21 0.19 0.15
Core Usage 0.62 0.60 2.83 0.86 0.79
New Solved 0/0 0/0 0/0 0/0 0/0
Large Theory Batch Vampire
4.7
Vampire
4.6
iProver
3.6
E
3.0
Solved/8000 3729/8000 3725/8000 3446/7999 3292/8000
Av. WC Time 46.34 46.38 50.12 52.49
Av. CPU Time 366.72 367.90 352.76 105.63
Solutions 3729 46% 3725 46% 3446 43% 3290 41%
μWCEfficiency 146 150 27 90
μEfficiency 154 163 126 116
SotAC 0.09 - 0.07 0.05
Core Usage 5.69 5.66 6.06 1.64
New Solved 0/0 0/0 0/0 0/0
THF without Equality Zipperpin
2.1
Zipperpin
2.1.999
E
3.0
Vampire
4.7
Satallax
3.4
Leo-III
1.7.0
cvc5
1.0
Lash
1.12
LEO-II
1.7.0
Solved/100 82/100 79/100 63/100 59/100 54/100 51/100 38/100 41/100 27/100
Av. WC Time 14.07 16.62 1.77 8.12 36.16 4.63 2.73 4.61 1.73
Av. CPU Time 105.18 125.46 11.25 61.22 36.04 16.84 2.53 4.47 1.52
Solutions 82 82% 79 79% 63 63% 59 59% 54 54% 51 51% 38 38% 37 37% 27 27%
μWCEfficiency 399 394 574 414 116 178 314 294 226
μEfficiency 327 331 536 375 123 98 314 294 243
SotAC - 0.33 0.24 0.20 0.17 0.16 0.10 0.11 0.03
Core Usage 4.53 4.34 1.40 3.00 0.91 2.64 0.60 0.69 0.59
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
THF with Equality Zipperpin
2.1
Zipperpin
2.1.999
E
3.0
Vampire
4.7
Leo-III
1.7.0
Satallax
3.4
cvc5
1.0
Lash
1.12
LEO-II
1.7.0
Solved/400 378/400 377/400 356/400 308/400 285/400 275/400 244/400 239/400 147/400
Av. WC Time 7.58 7.58 3.33 7.43 6.37 17.16 6.43 8.07 4.16
Av. CPU Time 51.99 52.33 14.21 53.15 20.36 17.15 6.20 7.91 3.98
Solutions 378 94% 377 94% 356 89% 308 77% 284 71% 275 68% 244 61% 225 56% 146 36%
μWCEfficiency 558 563 712 506 241 356 453 439 327
μEfficiency 463 446 659 401 131 366 460 450 327
SotAC - 0.35 0.29 0.21 0.15 0.14 0.10 0.09 0.03
Core Usage 3.57 3.59 1.39 3.51 2.49 0.70 0.62 0.60 0.52
New Solved 128/142 133/142 121/142 109/142 92/142 94/142 73/142 77/142 43/142
TFA using Integers SnakeForV4.7
1.0
cvc5
1.0
Vampire
4.5
Vampire
4.7
iProver
3.6
Solved/225 195/225 173/225 171/225 166/225 123/225
Av. WC Time 5.55 22.53 6.02 4.85 19.17
Av. CPU Time 38.22 22.31 20.80 33.30 138.14
Solutions 195 86% 173 76% 171 76% 166 73% 122 54%
μWCEfficiency 593 248 481 458 116
μEfficiency 304 250 241 288 61
SotAC 0.28 0.24 - 0.16 0.06
Core Usage 4.33 0.86 4.01 4.26 5.43
New Solved 0/0 0/0 0/0 0/0 0/0
TFA using Reals SnakeForV4.7
1.0
cvc5
1.0
Vampire
4.7
Vampire
4.5
iProver
3.6
Solved/25 23/25 22/25 21/25 21/25 15/25
Av. WC Time 4.70 4.07 0.40 1.82 6.01
Av. CPU Time 34.78 3.85 0.54 2.81 39.61
Solutions 23 92% 22 88% 21 84% 21 84% 15 60%
μWCEfficiency 860 659 820 801 254
μEfficiency 718 659 779 781 146
SotAC 0.25 0.21 0.19 - 0.08
Core Usage 1.83 0.57 0.81 0.55 3.32
New Solved 0/0 0/0 0/0 0/0 0/0
FOF Theorems without Equality Vampire
4.7
Vampire
4.6
SnakeForV4.7
1.0
iProver
3.6
CSE_E
1.4
E
3.0
GKC
0.7
Zipperpin
2.1.999
cvc5
1.0
Drodi
3.3.3
CSE
1.5
Prover9
1109a
Goeland
1.0.0
Etableau
0.67
Solved/100 94/100 92/100 91/100 88/100 79/100 78/100 76/100 66/100 52/100 46/100 41/100 19/100 2/100 58/100
Av. WC Time 5.47 3.20 3.01 9.47 10.39 6.05 3.88 16.98 56.17 6.42 30.31 11.16 6.21 6.23
Av. CPU Time 36.16 12.51 18.48 59.27 10.28 39.63 26.79 111.82 55.93 47.25 30.21 10.89 11.96 37.17
Solutions 94 94% 92 92% 91 91% 88 88% 79 79% 78 78% 76 76% 66 66% 52 52% 46 46% 41 41% 19 19% 2  2% 0  0%
μWCEfficiency 743 798 714 237 445 412 590 264 120 282 165 90 11 341
μEfficiency 637 627 466 126 441 398 451 197 120 153 165 116 10 255
SotAC 0.33 - 0.31 0.29 0.24 0.22 0.22 0.16 0.13 0.10 0.08 0.03 0.01 0.14
Core Usage 2.35 2.08 3.08 4.35 0.89 2.56 2.88 4.09 0.86 4.71 0.91 0.84 1.44 3.18
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Theorems with Equality SnakeForV4.7
1.0
Vampire
4.7
Vampire
4.6
E
3.0
CSE_E
1.4
iProver
3.6
GKC
0.7
Zipperpin
2.1.999
cvc5
1.0
Drodi
3.3.3
Prover9
1109a
CSE
1.5
Goeland
1.0.0
Etableau
0.67
Solved/400 369/400 357/400 356/400 306/400 282/400 277/400 259/400 228/400 219/400 159/400 104/400 87/400 5/400 221/400
Av. WC Time 5.48 5.43 6.91 9.08 11.81 17.80 9.44 11.57 15.70 8.23 15.25 32.23 6.66 6.30
Av. CPU Time 35.36 35.49 35.48 57.73 11.71 118.15 66.53 72.91 15.51 59.83 14.96 32.18 46.52 36.42
Solutions 369 92% 357 89% 356 89% 306 76% 282 70% 277 69% 259 64% 228 57% 219 54% 159 39% 104 26% 87 21% 5  1% 0  0%
μWCEfficiency 661 645 660 496 383 179 377 292 257 268 94 54 7 363
μEfficiency 445 496 491 396 389 77 214 209 264 190 120 53 5 276
SotAC 0.38 0.36 - 0.27 0.23 0.22 0.20 0.16 0.16 0.08 0.05 0.05 0.00 0.15
Core Usage 3.43 2.95 2.85 3.11 0.92 4.72 4.48 3.74 0.81 3.75 0.87 0.97 4.19 3.00
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Non-theorems without Equality Vampire
4.6
SnakeForV4.7
1.0
Vampire
4.7
cvc5
1.0
iProver
3.6
E
3.0
Solved/100 69/100 67/100 61/100 43/100 26/100 16/100
Av. WC Time 31.18 32.27 35.81 35.34 15.60 10.08
Av. CPU Time 87.28 252.96 276.87 35.13 113.67 34.22
Solutions 69 69% 67 67% 61 61% 43 43% 26 26% 16 16%
μWCEfficiency 227 240 223 80 77 74
μEfficiency 168 166 163 80 50 48
SotAC - 0.26 0.22 0.16 0.06 0.03
Core Usage 2.68 5.63 5.53 0.90 4.53 3.87
New Solved 0/0 0/0 0/0 0/0 0/0 0/0
FOF Non-theorems with Equality Vampire
4.7
Vampire
4.6
SnakeForV4.7
1.0
E
3.0
iProver
3.6
cvc5
1.0
Solved/150 99/150 98/150 92/150 46/150 37/150 35/150
Av. WC Time 7.16 5.71 4.84 3.41 23.93 27.04
Av. CPU Time 53.58 33.96 35.05 13.73 179.11 26.85
Solutions 99 66% 98 65% 92 61% 46 30% 37 24% 35 23%
μWCEfficiency 393 386 406 261 49 96
μEfficiency 269 272 267 229 13 103
SotAC 0.27 - 0.24 0.07 0.08 0.04
Core Usage 4.33 4.14 3.98 1.64 6.21 0.92
New Solved 0/0 0/0 0/0 0/0 0/0 0/0
Unit Equality CNF Twee
2.4.1
Twee
2.4
E
3.0
SnakeForV4.7
1.0
Vampire
4.7
iProver
3.6
GKC
0.7
Drodi
3.3.3
Toma
0.2
Solved/250 216/250 215/250 212/250 207/250 152/250 142/250 120/250 116/250 1/250
Av. WC Time 7.37 7.54 11.07 6.37 8.98 14.38 11.35 5.75 2.22
Av. CPU Time 55.53 56.85 74.14 45.18 67.32 101.19 85.32 43.31 1.99
Solutions 216 86% 215 86% 212 84% 207 82% 152 60% 142 56% 120 48% 116 46% 1  0%
μWCEfficiency 585 574 527 546 381 158 230 295 1
μEfficiency 396 375 392 305 218 71 137 199 2
SotAC 0.31 - 0.29 0.27 0.13 0.11 0.09 0.07 0.00
Core Usage 4.02 4.19 3.72 4.41 4.66 4.83 5.18 4.28 0.90
New Solved 38/46 36/46 43/46 32/46 20/46 23/46 19/46 19/46 1/46
SLedgeHammer Theorems E
3.0
Ehoh
2.7
Zipperpin
2.1.999
Vampire
4.7
cvc5
1.0
Solved/720 655/720 655/720 654/720 629/720 595/720
Av. WC Time 1.94 2.18 1.30 3.43 0.97
Av. CPU Time 1.72 1.99 5.41 3.30 0.81
Solutions 655 90% 655 90% 654 90% 629 87% 595 82%
μWCEfficiency 788 745 529 486 778
μEfficiency 780 738 688 463 763
SotAC 0.23 - 0.21 0.19 0.15
Core Usage 0.62 0.60 2.83 0.86 0.79
New Solved 0/0 0/0 0/0 0/0 0/0
LTB Van Emde Boas Tree Theorems Vampire
4.7
Vampire
4.6
iProver
3.6
E
3.0
Solved/8000 3729/8000 3725/8000 3446/7999 3292/8000
Av. WC Time 46.34 46.38 50.12 52.49
Av. CPU Time 366.72 367.90 352.76 105.63
Solutions 3729 46% 3725 46% 3446 43% 3290 41%
μWCEfficiency 154 163 126 116
μEfficiency 146 150 27 90
SotAC 0.09 - 0.07 0.05
Core Usage 5.69 5.66 6.06 1.64
New Solved 0/0 0/0 0/0 0/0