Colour key: 1st place 2nd place 3rd place Demonstration
Higher-order Theorems Vampire
4.9
Vampire
4.8
Zipperpin
2.1.9999
E
3.2.0
cvc5
1.1.3
Leo-III
1.7.15
LEO-II
1.7.0
Solved/500 463/500 447/500 433/500 411/500 299/500 268/500 88/500
Av. WC Time 5.41 2.62 13.08 11.87 13.14 12.47 8.65
Av. CPU Time 30.93 17.78 94.64 39.08 12.96 38.57 8.44
Solutions/500 463/500 447/500 433/500 411/500 299/500 268/500 88/500
μWCEfficiency 674 680 412 624 351 139 130
μEfficiency 514 499 321 575 363 72 132
SotAC 0.32 - 0.28 0.25 0.15 0.11 0.02
Core Usage/> 1.0 4.54/260 5.65/243 6.24/297 4.08/168 1.10/1 2.70/268 0.00/0
Core U. ≤ 1.0 203 204 136 243 298 0 88
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Unique Solved 3/17 0/17 5/17 7/17 2/17 0/17 0/17
Typed First-order Theorems +*-/ Vampire
4.9
cvc5
1.1.3
Vampire
4.8
iProver
3.9
Solved/200 164/200 151/200 136/200 93/200
Av. WC Time 4.85 23.56 5.74 16.77
Av. CPU Time 29.11 23.29 36.95 125.90
Solutions/200 164/200 151/200 136/200 90/200
μWCEfficiency 553 308 466 181
μEfficiency 336 308 341 138
SotAC 0.29 0.25 - 0.04
Core Usage/> 1.0 5.06/118 0.00/0 5.39/83 5.31/76
Core U. ≤ 1.0 46 151 53 17
New Solved 4/10 7/10 2/10 2/10
Unique Solved 8/29 18/29 3/29 0/29
Typed First-order Non-theorems iProver
3.9
cvc5
1.1.3
Vampire
4.8
Vampire
4.9
Solved/150 85/150 48/150 106/150 107/150
Av. WC Time 23.18 41.91 2.99 4.27
Av. CPU Time 174.50 41.69 17.94 27.59
Solutions/150 85/150 0/150 0/150 0/150
μWCEfficiency 175 173 504 519
μEfficiency 208 176 301 308
SotAC 0.10 0.02 0.18 0.19
Core Usage/> 1.0 6.34/59 0.00/0 5.42/71 4.93/75
Core U. ≤ 1.0 26 48 35 32
New Solved 1/2 0/2 1/2 1/2
Unique Solved 0/0 0/0 0/0 0/0
First-order Theorems Vampire
4.9
Vampire
4.8
E
3.2.0
CSI_E
1.0
CSE_E
1.6
iProver
3.9
CSG_E
1.0
GKC
0.8
Drodi
3.6.0
Zipperpin
2.1.9999
cvc5
1.1.3
Prover9
1109a
ConnectPP
0.6.0
CSE
1.7
Solved/500 471/500 370/500 370/500 366/500 360/500 343/500 331/500 326/500 303/500 288/499 279/500 131/500 110/500 105/500
Av. WC Time 3.97 4.12 12.45 13.04 11.26 22.35 16.62 10.62 11.63 13.19 19.62 18.86 49.10 54.93
Av. CPU Time 22.41 25.01 80.02 75.88 64.03 161.11 16.51 77.14 87.49 84.16 19.40 18.57 48.92 54.88
Solutions/500 471/500 370/500 370/500 365/500 360/500 342/500 331/500 326/500 303/500 288/499 279/500 131/500 110/500 105/500
μWCEfficiency 716 512 485 459 450 182 344 402 372 296 260 95 53 54
μEfficiency 538 383 391 395 399 89 347 243 274 217 267 117 54 54
SotAC 0.38 - 0.21 0.21 0.20 0.19 0.17 0.17 0.14 0.13 0.15 0.04 0.03 0.02
Core Usage/> 1.0 4.41/267 5.00/225 4.83/233 4.72/218 4.33/197 5.68/327 1.02/46 5.38/250 6.49/183 5.14/201 1.04/1 0.00/0 0.00/0 1.01/37
Core U. ≤ 1.0 204 145 137 148 163 16 285 76 120 87 278 131 110 68
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Unique Solved 28/32 1/32 0/32 0/32 0/32 0/32 0/32 0/32 0/32 0/32 3/32 0/32 0/32 0/32
Unit Equality CNF Vampire
4.9
Twee
2.5.0
Twee
2.4.2
E
3.2.0
iProver
3.9
Drodi
3.6.0
GKC
0.8
Solved/300 265/300 257/300 254/300 230/300 202/300 115/300 102/300
Av. WC Time 10.27 16.92 14.58 17.35 34.22 21.75 21.75
Av. CPU Time 67.41 129.87 112.02 121.65 261.39 170.49 168.63
Solutions/300 265/300 256/300 254/300 230/300 202/300 115/300 102/300
μWCEfficiency 492 455 461 414 121 155 118
μEfficiency 273 247 248 259 46 64 49
SotAC 0.35 0.32 - 0.26 0.19 0.04 0.03
Core Usage/> 1.0 5.29/218 6.47/209 6.39/205 5.33/192 6.70/197 6.80/105 6.75/95
Core U. ≤ 1.0 47 48 49 38 5 10 7
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Unique Solved 9/18 1/18 0/18 8/18 0/18 0/18 0/18
SLedgeHammer Theorems Vampire
4.9
E
3.2.0
E
3.1
cvc5
1.1.3
Solved/1000 463/1000 429/1000 426/1000 363/1000
Av. WC Time 2.86 4.37 4.19 3.15
Av. CPU Time 2.65 4.15 4.06 2.98
Solutions/1000 463/1000 429/1000 426/1000 363/1000
μWCEfficiency 332 246 248 149
μEfficiency 309 214 239 137
SotAC 0.11 0.09 - 0.06
Core Usage/> 1.0 1.01/37 0.00/0 1.04/51 0.00/0
Core U. ≤ 1.0 426 429 375 363
New Solved 0/0 0/0 0/0 0/0
Unique Solved 39/80 1/80 1/80 39/80
I Challenge yoU Vampire
4.9
CSI_E
1.0
E
3.2.0
iProver
3.9
CSE_E
1.6
Drodi
3.6.0
CSG_E
1.0
ConnectPP
0.6.0
Solved/80 59/80 43/80 42/80 21/80 20/80 15/80 11/80 3/80
Av. WC Time 27.76 118.02 127.75 46.53 239.42 48.92 185.88 295.42
Av. CPU Time 175.51 915.25 972.29 350.46 1000.32 386.49 185.74 295.13
Solutions/80 59/80 43/80 42/80 21/80 20/80 15/80 11/80 3/80
μWCEfficiency 225 94 108 17 1 95 1 4
μEfficiency 102 69 56 3 1 32 1 4
SotAC 0.45 0.28 0.27 0.12 0.11 0.06 0.05 0.00
Core Usage/> 1.0 6.03/54 7.21/39 7.03/40 7.07/21 3.28/20 6.43/14 1.00/4 0.00/0
Core U. ≤ 1.0 5 4 2 0 0 1 7 3
New Solved 8/12 12/12 12/12 3/12 3/12 5/12 5/12 1/12
Unique Solved 14/14 0/14 0/14 0/14 0/14 0/14 0/14 0/14
THF without Equality Vampire
4.9
Vampire
4.8
Zipperpin
2.1.9999
E
3.2.0
Leo-III
1.7.15
cvc5
1.1.3
LEO-II
1.7.0
Solved/100 87/100 86/100 83/100 71/100 57/100 52/100 34/100
Av. WC Time 2.81 0.95 10.45 2.35 7.52 7.82 5.54
Av. CPU Time 17.53 5.25 75.80 15.71 21.56 7.61 5.35
Solutions/100 87/100 86/100 83/100 71/100 57/100 52/100 34/100
μWCEfficiency 750 780 445 653 205 342 291
μEfficiency 617 589 368 619 116 342 291
SotAC 0.33 - 0.28 0.23 0.12 0.11 0.02
Core Usage/> 1.0 4.78/33 5.22/35 6.24/51 4.50/15 2.42/57 0.00/0 0.00/0
Core U. ≤ 1.0 54 51 32 56 0 52 34
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Unique Solved 1/7 0/7 0/7 5/7 0/7 1/7 0/7
THF with Equality Vampire
4.9
Vampire
4.8
Zipperpin
2.1.9999
E
3.2.0
cvc5
1.1.3
Leo-III
1.7.15
LEO-II
1.7.0
Solved/400 376/400 361/400 350/400 340/400 247/400 211/400 54/400
Av. WC Time 6.01 3.01 13.71 13.86 14.26 13.80 10.60
Av. CPU Time 34.03 20.76 99.10 43.95 14.09 43.16 10.38
Solutions/400 376/400 361/400 350/400 340/400 247/400 211/400 54/400
μWCEfficiency 655 655 403 617 354 122 90
μEfficiency 488 476 309 564 369 61 92
SotAC 0.32 - 0.28 0.26 0.15 0.10 0.01
Core Usage/> 1.0 4.51/227 5.73/208 6.24/246 4.04/153 1.10/1 2.77/211 0.00/0
Core U. ≤ 1.0 149 153 104 187 246 0 54
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Unique Solved 2/10 0/10 5/10 2/10 1/10 0/10 0/10
TFA using Integers Vampire
4.9
cvc5
1.1.3
Vampire
4.8
iProver
3.9
Solved/175 143/175 132/175 117/175 78/175
Av. WC Time 5.49 26.50 6.61 19.72
Av. CPU Time 33.18 26.24 42.90 148.94
Solutions/175 143/175 132/175 117/175 75/175
μWCEfficiency 515 264 424 157
μEfficiency 301 264 287 127
SotAC 0.30 0.26 - 0.04
Core Usage/> 1.0 5.18/108 0.00/0 5.48/80 5.66/63
Core U. ≤ 1.0 35 132 37 15
New Solved 4/10 7/10 2/10 2/10
Unique Solved 8/29 18/29 3/29 0/29
TFA using Reals Vampire
4.9
Vampire
4.8
cvc5
1.1.3
iProver
3.9
Solved/25 21/25 19/25 19/25 15/25
Av. WC Time 0.53 0.39 3.07 1.43
Av. CPU Time 1.36 0.29 2.84 6.04
Solutions/25 21/25 19/25 19/25 15/25
μWCEfficiency 820 760 612 346
μEfficiency 583 720 612 211
SotAC 0.22 - 0.17 0.06
Core Usage/> 1.0 3.75/10 2.85/3 0.00/0 3.61/13
Core U. ≤ 1.0 11 16 19 2
New Solved 0/0 0/0 0/0 0/0
Unique Solved 0/0 0/0 0/0 0/0
Typed First-order Non-theorems iProver
3.9
cvc5
1.1.3
Vampire
4.8
Vampire
4.9
Solved/150 85/150 48/150 106/150 107/150
Av. WC Time 23.18 41.91 2.99 4.27
Av. CPU Time 174.50 41.69 17.94 27.59
Solutions/150 85/150 0/150 0/150 0/150
μWCEfficiency 175 173 504 519
μEfficiency 208 176 301 308
SotAC 0.10 0.02 0.18 0.19
Core Usage/> 1.0 6.34/59 0.00/0 5.42/71 4.93/75
Core U. ≤ 1.0 26 48 35 32
New Solved 1/2 0/2 1/2 1/2
Unique Solved 0/0 0/0 0/0 0/0
FOF Theorems without Equality Vampire
4.9
Vampire
4.8
iProver
3.9
E
3.2.0
CSI_E
1.0
CSE_E
1.6
CSG_E
1.0
GKC
0.8
Drodi
3.6.0
Zipperpin
2.1.9999
ConnectPP
0.6.0
cvc5
1.1.3
CSE
1.7
Prover9
1109a
Solved/75 71/75 70/75 69/75 63/75 62/75 61/75 59/75 58/75 55/75 50/75 48/75 43/75 33/75 20/75
Av. WC Time 3.23 2.26 9.95 10.30 15.82 8.23 5.73 2.80 5.84 8.44 54.56 37.80 60.97 5.89
Av. CPU Time 17.12 11.45 68.45 71.63 80.42 46.33 5.63 17.59 44.04 53.34 54.40 37.69 60.93 5.60
Solutions/75 71/75 70/75 69/75 63/75 62/75 61/75 59/75 58/75 55/75 50/75 48/75 43/75 33/75 20/75
μWCEfficiency 854 761 310 613 566 607 468 617 515 393 201 122 116 166
μEfficiency 727 607 185 556 509 574 475 418 386 336 203 123 116 215
SotAC 0.27 - 0.25 0.18 0.17 0.16 0.14 0.14 0.12 0.08 0.10 0.11 0.04 0.02
Core Usage/> 1.0 3.17/34 4.26/34 5.15/63 4.69/31 4.44/32 4.26/29 1.02/7 4.27/38 6.24/30 4.82/28 0.00/0 1.04/1 1.00/9 0.00/0
Core U. ≤ 1.0 37 36 6 32 30 32 52 20 25 22 48 42 24 20
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Unique Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Theorems with Equality Vampire
4.9
E
3.2.0
CSI_E
1.0
Vampire
4.8
CSE_E
1.6
iProver
3.9
CSG_E
1.0
GKC
0.8
Drodi
3.6.0
Zipperpin
2.1.9999
cvc5
1.1.3
Prover9
1109a
CSE
1.7
ConnectPP
0.6.0
Solved/425 400/425 307/425 304/425 300/425 299/425 274/425 272/425 268/425 248/425 238/424 236/425 111/425 72/425 62/425
Av. WC Time 4.10 12.89 12.48 4.55 11.88 25.47 18.98 12.31 12.92 14.18 16.30 21.20 52.17 44.87
Av. CPU Time 23.34 81.74 74.95 28.17 67.64 184.45 18.87 90.03 97.12 90.63 16.07 20.91 52.10 44.68
Solutions/425 400/425 307/425 303/425 300/425 299/425 273/425 272/425 268/425 248/425 238/424 236/425 111/425 72/425 62/425
μWCEfficiency 692 462 440 469 422 159 322 364 347 279 284 82 43 27
μEfficiency 505 362 374 343 368 72 325 212 254 196 292 100 43 28
SotAC 0.40 0.22 0.22 - 0.21 0.18 0.17 0.18 0.15 0.14 0.16 0.05 0.02 0.02
Core Usage/> 1.0 4.60/233 4.85/202 4.77/186 5.13/191 4.34/168 5.80/264 1.02/39 5.58/212 6.54/153 5.19/173 0.00/0 0.00/0 1.01/28 0.00/0
Core U. ≤ 1.0 167 105 118 109 131 10 233 56 95 65 236 111 44 62
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Unique Solved 28/32 0/32 0/32 1/32 0/32 0/32 0/32 0/32 0/32 0/32 3/32 0/32 0/32 0/32
Unit Equality CNF Vampire
4.9
Twee
2.5.0
Twee
2.4.2
E
3.2.0
iProver
3.9
Drodi
3.6.0
GKC
0.8
Solved/300 265/300 257/300 254/300 230/300 202/300 115/300 102/300
Av. WC Time 10.27 16.92 14.58 17.35 34.22 21.75 21.75
Av. CPU Time 67.41 129.87 112.02 121.65 261.39 170.49 168.63
Solutions/300 265/300 256/300 254/300 230/300 202/300 115/300 102/300
μWCEfficiency 492 455 461 414 121 155 118
μEfficiency 273 247 248 259 46 64 49
SotAC 0.35 0.32 - 0.26 0.19 0.04 0.03
Core Usage/> 1.0 5.29/218 6.47/209 6.39/205 5.33/192 6.70/197 6.80/105 6.75/95
Core U. ≤ 1.0 47 48 49 38 5 10 7
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Unique Solved 9/18 1/18 0/18 8/18 0/18 0/18 0/18
SLedgeHammer Theorems Vampire
4.9
E
3.2.0
E
3.1
cvc5
1.1.3
Solved/1000 463/1000 429/1000 426/1000 363/1000
Av. WC Time 2.86 4.37 4.19 3.15
Av. CPU Time 2.65 4.15 4.06 2.98
Solutions/1000 463/1000 429/1000 426/1000 363/1000
μWCEfficiency 332 246 248 149
μEfficiency 309 214 239 137
SotAC 0.11 0.09 - 0.06
Core Usage/> 1.0 1.01/37 0.00/0 1.04/51 0.00/0
Core U. ≤ 1.0 426 429 375 363
New Solved 0/0 0/0 0/0 0/0
Unique Solved 39/80 1/80 1/80 39/80
I Challenge yoU Vampire
4.9
CSI_E
1.0
E
3.2.0
iProver
3.9
CSE_E
1.6
Drodi
3.6.0
CSG_E
1.0
ConnectPP
0.6.0
Solved/80 59/80 43/80 42/80 21/80 20/80 15/80 11/80 3/80
Av. WC Time 27.76 118.02 127.75 46.53 239.42 48.92 185.88 295.42
Av. CPU Time 175.51 915.25 972.29 350.46 1000.32 386.49 185.74 295.13
Solutions/80 59/80 43/80 42/80 21/80 20/80 15/80 11/80 3/80
μWCEfficiency 225 94 108 17 1 95 1 4
μEfficiency 102 69 56 3 1 32 1 4
SotAC 0.45 0.28 0.27 0.12 0.11 0.06 0.05 0.00
Core Usage/> 1.0 6.03/54 7.21/39 7.03/40 7.07/21 3.28/20 6.43/14 1.00/4 0.00/0
Core U. ≤ 1.0 5 4 2 0 0 1 7 3
New Solved 8/12 12/12 12/12 3/12 3/12 5/12 5/12 1/12
Unique Solved 14/14 0/14 0/14 0/14 0/14 0/14 0/14 0/14