Higher-order Theorems Zipperpin
2.1 		Zipperpin
2.0 		Vampire
4.6 		Leo‑III
1.6 		Ehoh
2.7 		cvc5
1.0 		LEO‑II
1.7.0
Solved/500 467/500 450/500 386/500 357/500 300/500 239/500 95/500
Av. WC Time 9.37 11.30 4.65 11.80 15.24 7.62 6.69
Av. CPU Time 65.52 71.90 34.90 34.00 15.20 7.57 6.65
Solutions 467 93% 450 90% 386 77% 357 71% 300 60% 239 47% 94 18%
μWCEfficiency 476 428 566 142 410 344 144
μEfficiency 358 327 371 79 410 346 144
SotAC 0.35 - 0.24 0.20 0.17 0.10 0.03
Core Usage 4.55 4.32 4.22 2.51 0.94 0.91 0.90
New Solved 161/170 159/170 153/170 136/170 147/170 108/170 6/170
First-order Theorems Vampire
4.6 		Vampire
4.5 		iProver
3.5 		CSE_E
1.3 		E
2.6 		GKC
0.7 		Zipperpin
2.1 		Etableau
0.67 		cvc5
1.0 		SATCoP
0.1 		Drodi
3.1.5 		Prover9
1109a 		CSE
1.4 		CSE‑F
1.0 		Twee
2.4 		JavaRes
1.3.0 		RPx
1.0
Solved/500 448/500 439/500 373/500 373/500 368/500 342/500 310/500 291/500 281/500 217/500 204/500 138/500 128/500 112/500 98/500 16/500 16/500
Av. WC Time 6.52 3.44 12.15 13.43 13.50 7.27 10.51 4.82 21.99 7.09 11.64 13.63 23.56 37.86 11.16 35.88 19.01
Av. CPU Time 32.80 18.67 84.28 13.47 13.44 52.57 68.00 26.49 21.92 53.08 87.19 13.49 23.66 37.94 86.91 45.93 20.07
Solutions 448 89% 439 87% 373 74% 373 74% 368 73% 342 68% 310 62% 291 58% 280 56% 217 43% 204 40% 138 27% 128 25% 112 22% 98 19% 16  3% 0  0%
μWCEfficiency 706 702 219 407 402 460 338 410 233 305 236 112 68 66 84 10 10
μEfficiency 546 498 101 399 403 304 254 313 232 221 149 125 67 63 43 7 10
SotAC 0.44 - 0.32 0.32 0.31 0.28 0.24 0.22 0.23 0.16 0.13 0.09 0.07 0.06 0.07 0.00 0.01
Core Usage 2.95 3.27 4.71 1.03 0.90 4.18 3.73 3.16 0.94 3.99 4.90 0.92 1.02 1.02 6.20 1.69 1.06
New Solved 44/50 38/50 30/50 27/50 29/50 32/50 24/50 24/50 16/50 20/50 7/50 13/50 9/50 11/50 14/50 0/50 0/50
First-order Non-theorems Vampire
SAT‑4.5 		Vampire
SAT‑4.6 		iProver
SAT‑3.5 		cvc5
SAT‑1.0 		E
FNT‑2.6 		Etableau
0.67 		RPx
1.0
Solved/250 230/250 229/250 145/250 79/250 65/250 58/250 0/250
Av. WC Time 14.47 14.85 8.87 19.36 4.38 3.08 -
Av. CPU Time 48.40 47.12 58.00 19.30 4.31 19.44 -
Solutions 230 92% 229 91% 145 58% 79 31% 65 26% 58 23% 0  0%
μWCEfficiency 534 536 192 129 189 171 -
μEfficiency 408 422 93 129 189 143 -
SotAC - 0.53 0.26 0.11 0.07 0.06 -
Core Usage 3.46 3.29 4.52 0.94 0.81 3.06 -
New Solved 3/3 3/3 3/3 1/3 0/3 0/3 0/3
Unit Equality CNF Twee
2.4 		E
2.5 		E
2.6 		Vampire
4.6 		Etableau
0.67 		iProver
3.5 		GKC
0.7 		Drodi
3.1.5
Solved/250 227/250 197/250 195/250 157/250 152/250 138/250 122/250 118/250
Av. WC Time 8.17 13.91 12.15 10.47 5.61 17.41 13.69 8.98
Av. CPU Time 62.95 13.88 12.10 32.53 36.50 125.77 104.97 67.05
Solutions 227 90% 197 78% 195 78% 157 62% 152 60% 138 55% 122 48% 118 47%
μWCEfficiency 572 503 498 334 444 146 202 258
μEfficiency 343 502 501 213 355 58 111 155
SotAC 0.44 - 0.28 0.15 0.15 0.11 0.07 0.05
Core Usage 5.06 0.93 0.87 2.92 3.39 5.22 5.95 5.15
New Solved 29/42 25/42 24/42 16/42 16/42 18/42 14/42 14/42
SLedgeHammer Theorems Zipperpin
SLH‑2.1 		Ehoh
SLH‑2.7 		Vampire
SLH‑4.6 		Leo‑III
1.6 		cvc5
SLH‑1.0
Solved/720 675/720 655/720 626/720 499/720 310/720
Av. WC Time 2.67 2.09 3.48 5.88 2.11
Av. CPU Time 2.44 2.03 3.45 17.83 2.01
Solutions 675 93% 655 90% 626 86% 499 69% 310 43%
μWCEfficiency 574 742 496 42 326
μEfficiency 550 742 493 127 324
SotAC 0.27 0.25 0.23 0.15 0.05
Core Usage 0.90 0.83 0.93 3.18 0.83
New Solved 0/0 0/0 0/0 0/0 0/0
Large Theory Batch Vampire
LTB‑4.6 		iProver
LTB‑3.5 		Zipperpin
LTB‑2.1 		E
LTB‑2.5 		E
LTB‑2.6 		GKC
LTB‑0.7 		Leo‑III
LTB‑1.6
Solved/10000 7891/10000 7013/9993 6952/10000 6921/10000 6833/10000 6545/10000 2481/10000
Av. WC Time 21.89 19.74 9.57 24.95 15.75 26.40 69.65
Av. CPU Time 309.51 18.51 0.17 13.73 10.45 15.64 170.62
Solutions 7891 78% 7013 70% 6952 69% 6921 69% 6833 68% 6545 65% 2481 24%
μWCEfficiency 428 402 695 532 548 544 23
μEfficiency 408 115 695 473 470 413 4
SotAC 0.21 0.13 0.13 0.12 0.12 0.11 0.02
Core Usage 4.38 6.43 1.01 2.16 2.30 3.23 6.10
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
THF without Equality Zipperpin
2.1 		Zipperpin
2.0 		Vampire
4.6 		Leo‑III
1.6 		cvc5
1.0 		LEO‑II
1.7.0 		Ehoh
2.7
Solved/100 88/100 83/100 70/100 61/100 40/100 36/100 30/100
Av. WC Time 12.79 15.00 3.05 6.05 2.01 1.05 4.20
Av. CPU Time 95.64 100.33 22.56 14.54 1.96 1.01 4.16
Solutions 88 88% 83 83% 70 70% 61 61% 40 40% 36 36% 30 30%
μWCEfficiency 433 422 479 126 340 330 274
μEfficiency 499 464 556 195 340 330 274
SotAC 0.43 - 0.26 0.19 0.08 0.06 0.04
Core Usage 4.06 3.76 2.97 2.01 0.84 0.88 0.90
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
THF with Equality Zipperpin
2.1 		Zipperpin
2.0 		Vampire
4.6 		Leo‑III
1.6 		Ehoh
2.7 		cvc5
1.0 		LEO‑II
1.7.0
Solved/400 379/400 367/400 316/400 296/400 270/400 199/400 59/400
Av. WC Time 8.58 10.46 5.01 12.99 16.47 8.75 10.14
Av. CPU Time 58.52 65.47 37.63 38.01 16.43 8.70 10.09
Solutions 379 94% 367 91% 316 79% 296 74% 270 67% 199 49% 58 14%
μWCEfficiency 339 303 343 68 444 348 97
μEfficiency 470 419 568 129 444 346 97
SotAC 0.34 - 0.23 0.21 0.19 0.11 0.02
Core Usage 4.66 4.44 4.49 2.62 0.94 0.93 0.92
New Solved 161/170 159/170 153/170 136/170 147/170 108/170 6/170
FOF Theorems without Equality Vampire
4.5 		Vampire
4.6 		iProver
3.5 		CSE_E
1.3 		E
2.6 		GKC
0.7 		Zipperpin
2.1 		SATCoP
0.1 		Etableau
0.67 		Drodi
3.1.5 		cvc5
1.0 		CSE
1.4 		CSE‑F
1.0 		Prover9
1109a 		Twee
2.4 		JavaRes
1.3.0 		RPx
1.0
Solved/100 93/100 92/100 90/100 85/100 83/100 80/100 73/100 69/100 68/100 64/100 56/100 44/100 39/100 25/100 18/100 12/100 9/100
Av. WC Time 1.39 2.73 6.37 8.83 9.97 3.32 8.84 2.75 3.48 7.79 40.21 22.44 45.70 8.97 14.16 44.60 16.56
Av. CPU Time 5.47 10.70 40.39 8.87 9.91 21.08 56.14 19.44 16.24 57.87 39.90 22.51 45.77 8.80 110.40 56.22 17.37
Solutions 93 93% 92 92% 90 90% 85 85% 83 83% 80 80% 73 73% 69 69% 68 68% 64 64% 56 56% 44 44% 39 39% 25 25% 18 18% 12 12% 0  0%
μWCEfficiency 682 690 167 560 431 581 318 522 387 250 153 134 123 150 38 23 34
μEfficiency 824 816 312 561 428 687 414 584 461 396 158 134 129 120 58 24 34
SotAC - 0.37 0.35 0.31 0.29 0.28 0.23 0.23 0.21 0.19 0.19 0.10 0.09 0.06 0.03 0.01 0.02
Core Usage 2.34 2.16 3.99 1.04 0.90 2.36 3.23 2.26 2.77 4.55 0.93 1.03 1.03 0.90 6.40 1.38 1.02
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Theorems with Equality Vampire
4.6 		Vampire
4.5 		CSE_E
1.3 		E
2.6 		iProver
3.5 		GKC
0.7 		Zipperpin
2.1 		cvc5
1.0 		Etableau
0.67 		SATCoP
0.1 		Drodi
3.1.5 		Prover9
1109a 		CSE
1.4 		Twee
2.4 		CSE‑F
1.0 		JavaRes
1.3.0 		RPx
1.0
Solved/400 356/400 346/400 288/400 285/400 283/400 262/400 237/400 225/400 223/400 148/400 140/400 113/400 84/400 80/400 73/400 4/400 7/400
Av. WC Time 7.50 3.99 14.79 14.52 13.99 8.47 11.03 17.45 5.23 9.11 13.40 14.65 24.14 10.48 33.66 9.73 22.16
Av. CPU Time 38.52 22.22 14.83 14.47 98.24 62.18 71.66 17.45 29.61 68.76 100.59 14.53 24.27 81.62 33.76 15.06 23.54
Solutions 356 89% 346 86% 288 72% 285 71% 283 70% 262 65% 237 59% 224 56% 223 55% 148 37% 140 35% 113 28% 84 21% 80 20% 73 18% 4  1% 0  0%
μWCEfficiency 511 452 359 396 84 234 238 252 294 146 124 119 50 45 49 3 4
μEfficiency 678 672 369 396 196 403 319 252 398 236 196 110 52 90 51 6 4
SotAC 0.46 - 0.32 0.32 0.32 0.28 0.24 0.24 0.22 0.14 0.12 0.10 0.06 0.07 0.05 0.00 0.00
Core Usage 3.15 3.52 1.03 0.90 4.94 4.74 3.89 0.94 3.27 4.79 5.05 0.93 1.02 6.16 1.02 2.63 1.12
New Solved 44/50 38/50 27/50 29/50 30/50 32/50 24/50 16/50 24/50 20/50 7/50 13/50 9/50 14/50 11/50 0/50 0/50
FOF Non-theorems without Equality Vampire
SAT‑4.5 		Vampire
SAT‑4.6 		E
FNT‑2.6 		iProver
SAT‑3.5 		Etableau
0.67 		cvc5
SAT‑1.0 		RPx
1.0
Solved/75 69/75 68/75 20/75 19/75 17/75 8/75 0/75
Av. WC Time 39.15 39.63 5.00 14.49 8.80 1.31 -
Av. CPU Time 104.72 105.77 4.97 93.01 58.19 1.24 -
Solutions 69 92% 68 90% 20 26% 19 25% 17 22% 8 10% 0  0%
μWCEfficiency 167 173 112 53 49 95 -
μEfficiency 226 224 112 80 88 95 -
SotAC - 0.61 0.10 0.10 0.08 0.04 -
Core Usage 2.89 2.80 0.92 3.76 5.48 0.77 -
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Non-theorems with Equality Vampire
SAT‑4.5 		Vampire
SAT‑4.6 		iProver
SAT‑3.5 		cvc5
SAT‑1.0 		E
FNT‑2.6 		Etableau
0.67 		RPx
1.0
Solved/175 161/175 161/175 126/175 71/175 45/175 41/175 0/175
Av. WC Time 3.89 4.38 8.02 21.39 4.10 0.71 -
Av. CPU Time 24.26 22.35 52.72 21.33 4.02 3.38 -
Solutions 161 92% 161 92% 126 72% 71 40% 45 25% 41 23% 0  0%
μWCEfficiency 512 529 110 144 221 183 -
μEfficiency 666 670 240 144 221 207 -
SotAC - 0.50 0.33 0.15 0.06 0.05 -
Core Usage 3.71 3.49 4.63 0.96 0.76 2.06 -
New Solved 3/3 3/3 3/3 1/3 0/3 0/3 0/3
Unit Equality CNF Twee
2.4 		E
2.5 		E
2.6 		Vampire
4.6 		Etableau
0.67 		iProver
3.5 		GKC
0.7 		Drodi
3.1.5
Solved/250 227/250 197/250 195/250 157/250 152/250 138/250 122/250 118/250
Av. WC Time 8.17 13.91 12.15 10.47 5.61 17.41 13.69 8.98
Av. CPU Time 62.95 13.88 12.10 32.53 36.50 125.77 104.97 67.05
Solutions 227 90% 197 78% 195 78% 157 62% 152 60% 138 55% 122 48% 118 47%
μWCEfficiency 572 503 498 334 444 146 202 258
μEfficiency 343 502 501 213 355 58 111 155
SotAC 0.44 - 0.28 0.15 0.15 0.11 0.07 0.05
Core Usage 5.06 0.93 0.87 2.92 3.39 5.22 5.95 5.15
New Solved 29/42 25/42 24/42 16/42 16/42 18/42 14/42 14/42
SLedgeHammer Theorems Zipperpin
SLH‑2.1 		Ehoh
SLH‑2.7 		Vampire
SLH‑4.6 		Leo‑III
1.6 		cvc5
SLH‑1.0
Solved/720 675/720 655/720 626/720 499/720 310/720
Av. WC Time 2.67 2.09 3.48 5.88 2.11
Av. CPU Time 2.44 2.03 3.45 17.83 2.01
Solutions 675 93% 655 90% 626 86% 499 69% 310 43%
μWCEfficiency 574 742 496 42 326
μEfficiency 550 742 493 127 324
SotAC 0.27 0.25 0.23 0.15 0.05
Core Usage 0.90 0.83 0.93 3.18 0.83
New Solved 0/0 0/0 0/0 0/0 0/0
LTB JinjaThreads Theorems Vampire
LTB‑4.6 		iProver
LTB‑3.5 		Zipperpin
LTB‑2.1 		E
LTB‑2.5 		E
LTB‑2.6 		GKC
LTB‑0.7 		Leo‑III
LTB‑1.6
Solved/10000 7891/10000 7013/9993 6952/10000 6921/10000 6833/10000 6545/10000 2481/10000
Av. WC Time 21.89 19.74 9.57 24.95 15.75 26.40 69.65
Av. CPU Time 309.51 18.51 0.17 13.73 10.45 15.64 170.62
Solutions 7891 78% 7013 70% 6952 69% 6921 69% 6833 68% 6545 65% 2481 24%
μWCEfficiency 428 402 695 532 548 544 23
μEfficiency 408 115 695 473 470 413 4
SotAC 0.21 0.13 0.13 0.12 0.12 0.11 0.02
Core Usage 4.38 6.43 1.01 2.16 2.30 3.23 6.10
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0