Higher-order Theorems Zipperpin
2.0 		Satallax
3.4 		Satallax
3.5 		Vampire
4.5 		Leo‑III
1.5 		CVC4
1.8 		LEO‑II
1.7.0
Solved/500 424/500 323/500 319/500 299/500 287/500 194/500 112/500
Av. WC Time 14.78 19.95 20.09 3.70 14.70 8.81 7.36
Av. CPU Time 94.26 19.98 20.06 27.43 41.73 8.75 7.32
Solutions 424 84% 323 64% 319 63% 299 59% 287 57% 194 38% 111 22%
μWCEfficiency 382 214 212 400 90 277 159
μEfficiency 288 215 211 270 45 278 159
SotAC 0.35 - 0.23 0.19 0.17 0.10 0.04
Core Usage 4.44 0.99 0.97 4.57 2.47 0.86 0.91
New Solved 31/51 7/51 8/51 23/51 15/51 15/51 3/51
Typed First-order Theorems +*-/ Vampire
4.5 		Vampire
4.4 		CVC4
1.8
Solved/250 191/250 190/250 187/250
Av. WC Time 4.61 17.88 17.91
Av. CPU Time 16.72 17.98 17.80
Solutions 191 76% 190 76% 187 74%
μWCEfficiency 538 284 302
μEfficiency 307 288 302
SotAC 0.20 - 0.18
Core Usage 4.06 0.94 0.88
New Solved 0/0 0/0 0/0
First-order Theorems Vampire
4.5 		Vampire
4.4 		Enigma
0.5.1 		E
2.5 		CSE_E
1.2 		iProver
3.3 		GKC
0.5.1 		CVC4
1.8 		Zipperpin
2.0 		Etableau
0.2 		Prover9
1109a 		CSE
1.3 		leanCoP
2.2 		Twee
2.2.1 		PyRes
1.3 		lazyCoP
0.1
Solved/500 429/500 416/500 401/500 351/500 316/500 312/500 289/500 275/500 237/500 162/500 146/500 124/500 111/500 68/500 26/500 94/500
Av. WC Time 3.89 12.05 10.16 15.17 8.98 10.34 3.09 19.73 22.63 9.08 12.94 41.99 32.03 9.77 21.37 8.79
Av. CPU Time 21.64 12.04 63.38 15.12 9.06 78.29 21.19 19.69 22.53 9.04 12.76 42.08 32.54 75.81 21.37 62.62
Solutions 429 85% 416 83% 401 80% 351 70% 316 63% 312 62% 289 57% 275 55% 237 47% 162 32% 146 29% 124 24% 111 22% 68 13% 26  5% 0  0%
μWCEfficiency 669 440 183 398 387 296 406 270 174 200 123 67 53 70 19 111
μEfficiency 457 452 55 398 380 84 261 272 175 200 141 63 94 47 19 76
SotAC 0.43 - 0.37 0.30 0.26 0.25 0.22 0.22 0.16 0.10 0.09 0.06 0.06 0.04 0.01 0.04
Core Usage 3.48 0.92 4.40 0.95 1.05 5.99 4.13 0.89 0.96 0.96 0.91 1.02 0.77 5.33 1.01 4.45
New Solved 35/52 32/52 25/52 22/52 21/52 26/52 19/52 17/52 17/52 15/52 9/52 7/52 7/52 7/52 0/52 4/52
First-order Non-theorems Vampire
SAT‑4.5 		Vampire
SAT‑4.4 		iProver
SAT‑3.3 		CVC4
SAT‑1.8 		E
FNT‑2.5 		PyRes
1.3
Solved/250 238/250 226/250 182/250 98/250 63/250 13/250
Av. WC Time 5.74 8.62 10.50 13.60 5.07 5.08
Av. CPU Time 24.23 8.64 78.78 13.50 5.07 5.08
Solutions 238 95% 226 90% 182 72% 98 39% 63 25% 13  5%
μWCEfficiency 670 299 357 211 157 13
μEfficiency 522 316 152 207 157 13
SotAC 0.50 - 0.32 0.13 0.08 0.00
Core Usage 3.34 0.98 5.59 0.91 0.97 0.97
New Solved 7/8 7/8 6/8 6/8 0/8 0/8
Unit Equality CNF E
2.5 		Twee
2.2.1 		E
2.4 		Vampire
4.5 		Etableau
0.2 		GKC
0.5.1 		iProver
3.3 		lazyCoP
0.1
Solved/250 202/250 197/250 185/250 162/250 148/250 128/250 124/250 20/250
Av. WC Time 7.80 6.04 5.39 8.38 2.63 8.30 7.41 0.80
Av. CPU Time 7.79 45.63 5.34 24.88 2.60 63.12 55.13 5.43
Solutions 202 80% 197 78% 185 74% 162 64% 148 59% 128 51% 124 49% 0  0%
μWCEfficiency 595 532 579 426 465 289 293 70
μEfficiency 595 373 579 302 467 186 95 65
SotAC 0.27 0.26 - 0.16 0.14 0.10 0.09 0.01
Core Usage 0.95 4.41 0.93 2.59 0.93 5.05 5.31 2.36
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Large Theory Batch Problems MaLARea
0.9 		E
LTB‑2.5 		iProver
LTB‑3.3 		Zipperpin
LTB‑2.0 		Leo‑III
LTB‑1.5 		ATPBoost
1.0 		GKC
LTB‑0.5.1 		Leo‑III
LTB‑1.4
Solved/10000 7054/10000 3393/10000 3164/10000 1699/10000 1413/10000 1237/10000 493/10000 134/10000
Av. WC Time 3.87 4.61 5.53 38.51 20.63 0.01 3.71 15.69
Av. CPU Time 18.89 13.58 34.94 120.82 147.49 0.01 24.92 113.04
Solutions 7054 70% 3393 33% 3163 31% 1699 16% 1413 14% 1237 12% 493  4% 134  1%
μWCEfficiency 262 127 98 50 9 124 33 1
μEfficiency 65 87 27 31 1 124 24 0
SotAC 0.48 0.17 0.15 0.06 0.06 0.04 0.01 0.00
Core Usage 4.66 2.57 6.04 4.17 7.35 1.00 4.07 7.30
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
THF without Equality Zipperpin
2.0 		Satallax
3.4 		Satallax
3.5 		Leo‑III
1.5 		Vampire
4.5 		CVC4
1.8 		LEO‑II
1.7.0
Solved/100 83/100 72/100 71/100 65/100 64/100 43/100 35/100
Av. WC Time 14.88 13.86 14.22 7.96 1.84 4.77 4.25
Av. CPU Time 99.61 13.86 14.17 20.37 12.88 4.70 4.21
Solutions 83 83% 72 72% 71 71% 65 65% 64 64% 43 43% 35 35%
μWCEfficiency 407 252 254 74 456 351 275
μEfficiency 460 252 259 138 530 351 275
SotAC 0.30 - 0.22 0.17 0.16 0.10 0.04
Core Usage 3.81 1.00 0.97 2.24 2.82 0.75 0.89
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
THF with Equality Zipperpin
2.0 		Satallax
3.4 		Satallax
3.5 		Vampire
4.5 		Leo‑III
1.5 		CVC4
1.8 		LEO‑II
1.7.0
Solved/400 341/400 251/400 248/400 235/400 222/400 151/400 77/400
Av. WC Time 14.75 21.69 21.77 4.20 16.68 9.96 8.78
Av. CPU Time 92.96 21.74 21.75 31.39 47.99 9.91 8.74
Solutions 341 85% 251 62% 248 62% 235 58% 222 55% 151 37% 76 19%
μWCEfficiency 258 205 200 224 38 260 130
μEfficiency 363 204 200 367 78 259 130
SotAC 0.37 - 0.23 0.20 0.17 0.10 0.04
Core Usage 4.59 0.99 0.98 5.05 2.54 0.89 0.92
New Solved 31/51 7/51 8/51 23/51 15/51 15/51 3/51
TFA using Integers Vampire
4.5 		Vampire
4.4 		CVC4
1.8
Solved/225 170/225 169/225 167/225
Av. WC Time 5.15 19.85 19.75
Av. CPU Time 18.75 19.97 19.64
Solutions 170 75% 169 75% 167 74%
μWCEfficiency 251 253 259
μEfficiency 505 250 259
SotAC 0.21 - 0.19
Core Usage 4.46 0.94 0.90
New Solved 0/0 0/0 0/0
TFA using Reals Vampire
4.5 		Vampire
4.4 		CVC4
1.8
Solved/25 21/25 21/25 20/25
Av. WC Time 0.26 2.01 2.50
Av. CPU Time 0.23 1.99 2.42
Solutions 21 84% 21 84% 20 80%
μWCEfficiency 820 601 688
μEfficiency 840 587 688
SotAC 0.17 - 0.08
Core Usage 0.84 0.93 0.68
New Solved 0/0 0/0 0/0
FOF Theorems without Equality Vampire
4.5 		Vampire
4.4 		Enigma
0.5.1 		iProver
3.3 		E
2.5 		CSE_E
1.2 		GKC
0.5.1 		Zipperpin
2.0 		CVC4
1.8 		Etableau
0.2 		CSE
1.3 		leanCoP
2.2 		Prover9
1109a 		Twee
2.2.1 		PyRes
1.3 		lazyCoP
0.1
Solved/100 90/100 90/100 85/100 85/100 83/100 77/100 75/100 57/100 53/100 45/100 45/100 37/100 25/100 10/100 8/100 38/100
Av. WC Time 1.48 7.57 7.44 7.82 10.54 5.57 1.36 20.78 30.82 6.16 57.48 34.19 6.10 21.29 38.28 5.16
Av. CPU Time 5.59 7.46 40.98 58.86 10.50 5.63 7.34 20.60 30.69 6.13 57.56 34.76 5.82 166.18 38.28 34.17
Solutions 90 90% 90 90% 85 85% 85 85% 83 83% 77 77% 75 75% 57 57% 53 53% 45 45% 45 45% 37 37% 25 25% 10 10% 8  8% 0  0%
μWCEfficiency 636 656 59 115 531 474 484 130 151 270 66 155 137 17 12 188
μEfficiency 782 622 198 401 531 486 626 130 150 270 71 88 122 31 12 257
SotAC 0.37 - 0.33 0.33 0.31 0.28 0.26 0.17 0.18 0.12 0.12 0.09 0.06 0.02 0.01 0.09
Core Usage 2.49 0.89 4.25 5.99 0.95 1.04 2.81 0.96 0.96 0.96 1.02 0.79 0.89 6.55 1.00 3.81
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Theorems with Equality Vampire
4.5 		Vampire
4.4 		Enigma
0.5.1 		E
2.5 		CSE_E
1.2 		iProver
3.3 		CVC4
1.8 		GKC
0.5.1 		Zipperpin
2.0 		Prover9
1109a 		Etableau
0.2 		CSE
1.3 		leanCoP
2.2 		Twee
2.2.1 		PyRes
1.3 		lazyCoP
0.1
Solved/400 339/400 326/400 316/400 268/400 239/400 227/400 222/400 214/400 180/400 121/400 117/400 79/400 74/400 58/400 18/400 56/400
Av. WC Time 4.53 13.28 10.89 16.60 10.08 11.28 17.08 3.70 23.22 14.35 10.20 33.17 30.96 7.79 13.85 11.24
Av. CPU Time 25.91 13.30 69.41 16.56 10.16 85.56 17.07 26.05 23.14 14.19 10.17 33.27 31.44 60.23 13.85 81.92
Solutions 339 84% 326 81% 316 79% 268 67% 239 59% 227 56% 222 55% 214 53% 180 45% 121 30% 117 29% 79 19% 74 18% 58 14% 18  4% 0  0%
μWCEfficiency 413 401 54 365 357 76 303 206 186 142 183 62 79 54 21 48
μEfficiency 640 394 179 365 362 270 300 351 186 123 183 66 45 80 21 74
SotAC 0.44 - 0.39 0.29 0.25 0.24 0.23 0.21 0.16 0.10 0.10 0.05 0.05 0.05 0.01 0.03
Core Usage 3.75 0.93 4.44 0.95 1.06 5.98 0.88 4.59 0.95 0.91 0.96 1.03 0.76 5.12 1.01 4.89
New Solved 35/52 32/52 25/52 22/52 21/52 26/52 17/52 19/52 17/52 9/52 15/52 7/52 7/52 7/52 0/52 4/52
FOF Non-theorems without Equality Vampire
SAT‑4.5 		Vampire
SAT‑4.4 		iProver
SAT‑3.3 		E
FNT‑2.5 		CVC4
SAT‑1.8 		PyRes
1.3
Solved/75 72/75 65/75 55/75 32/75 14/75 1/75
Av. WC Time 11.46 8.78 17.65 5.79 12.56 5.58
Av. CPU Time 30.75 8.74 133.42 5.80 12.50 5.62
Solutions 72 96% 65 86% 55 73% 32 42% 14 18% 1  1%
μWCEfficiency 487 443 129 167 123 2
μEfficiency 605 383 314 168 123 2
SotAC 0.50 - 0.35 0.19 0.06 0.00
Core Usage 2.61 0.97 5.73 0.99 0.86 1.01
New Solved 6/6 6/6 6/6 0/6 6/6 0/6
FOF Non-theorems with Equality Vampire
SAT‑4.5 		Vampire
SAT‑4.4 		iProver
SAT‑3.3 		CVC4
SAT‑1.8 		E
FNT‑2.5 		PyRes
1.3
Solved/175 166/175 161/175 127/175 84/175 31/175 12/175
Av. WC Time 3.26 8.55 7.41 13.77 4.32 5.04
Av. CPU Time 21.40 8.59 55.12 13.67 4.31 5.04
Solutions 166 94% 161 92% 127 72% 84 48% 31 17% 12  6%
μWCEfficiency 538 262 162 244 154 18
μEfficiency 699 264 375 248 153 18
SotAC 0.50 - 0.31 0.17 0.04 0.00
Core Usage 3.66 0.98 5.53 0.92 0.95 0.97
New Solved 1/2 1/2 0/2 0/2 0/2 0/2
Unit Equality CNF E
2.5 		Twee
2.2.1 		E
2.4 		Vampire
4.5 		Etableau
0.2 		GKC
0.5.1 		iProver
3.3 		lazyCoP
0.1
Solved/250 202/250 197/250 185/250 162/250 148/250 128/250 124/250 20/250
Av. WC Time 7.80 6.04 5.39 8.38 2.63 8.30 7.41 0.80
Av. CPU Time 7.79 45.63 5.34 24.88 2.60 63.12 55.13 5.43
Solutions 202 80% 197 78% 185 74% 162 64% 148 59% 128 51% 124 49% 0  0%
μWCEfficiency 595 532 579 426 465 289 293 70
μEfficiency 595 373 579 302 467 186 95 65
SotAC 0.27 0.26 - 0.16 0.14 0.10 0.09 0.01
Core Usage 0.95 4.41 0.93 2.59 0.93 5.05 5.31 2.36
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
LTB HOL4 Theorems MaLARea
0.9 		E
LTB‑2.5 		iProver
LTB‑3.3 		Zipperpin
LTB‑2.0 		Leo‑III
LTB‑1.5 		ATPBoost
1.0 		GKC
LTB‑0.5.1 		Leo‑III
LTB‑1.4
Solved/10000 7054/10000 3393/10000 3164/10000 1699/10000 1413/10000 1237/10000 493/10000 134/10000
Av. WC Time 3.87 4.61 5.53 38.51 20.63 0.01 3.71 15.69
Av. CPU Time 18.89 13.58 34.94 120.82 147.49 0.01 24.92 113.04
Solutions 7054 70% 3393 33% 3163 31% 1699 16% 1413 14% 1237 12% 493  4% 134  1%
μWCEfficiency 262 127 98 50 9 124 33 1
μEfficiency 65 87 27 31 1 124 24 0
SotAC 0.48 0.17 0.15 0.06 0.06 0.04 0.01 0.00
Core Usage 4.66 2.57 6.04 4.17 7.35 1.00 4.07 7.30
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0