Colour key: 1st place 2nd place 3rd place Demonstration
Higher-order Theorems Vampire
4.8 		Zipperpin
2.1.9999 		Zipperpin
2.1.999 		E
3.1 		Leo-III
1.7.8 		Satallax
3.4 		cvc5
1.0.5 		Lash
1.13 		LEO-II
1.7.0 		Duper
1.0
Solved/500 452/500 440/500 438/500 407/500 302/500 268/500 258/500 208/500 58/500 36/500
Av. WC Time 4.52 11.76 12.42 7.24 8.77 39.86 12.97 19.36 4.72 17.67
Av. CPU Time 32.17 85.33 90.24 37.64 33.13 39.82 12.71 19.21 4.53 17.50
Solutions 452 90% 440 88% 438 87% 407 81% 302 60% 268 53% 258 51% 196 39% 58 11% 36  7%
μWCEfficiency 674 437 434 655 168 197 345 231 85 10
μEfficiency 506 335 353 607 81 200 362 235 87 10
SotAC 0.38 0.37 - 0.33 0.20 0.17 0.17 0.12 0.03 0.01
Core Usage/> 1.0 5.46/255 6.32/295 6.51/277 4.49/117 2.95/302 1.01/46 0.00/0 1.01/14 0.00/0 0.00/0
≤ 1.0 Cores 197 145 161 290 0 222 258 194 58 36
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Typed First-order Theorems +*-/ Vampire
4.8 		cvc5
1.0 		cvc5
1.0.5 		iProver
3.8 		Princess
230619
Solved/200 154/200 148/200 144/200 84/200 95/200
Av. WC Time 16.46 24.67 23.87 18.71 5.66
Av. CPU Time 111.58 24.44 23.61 139.42 35.71
Solutions 154 77% 148 74% 144 72% 83 41% 68 34%
μWCEfficiency 457 316 327 169 183
μEfficiency 362 317 327 148 69
SotAC 0.27 - 0.24 0.09 0.12
Core Usage/> 1.0 6.03/90 0.00/0 0.00/0 5.26/63 4.66/95
≤ 1.0 Cores 64 148 144 21 0
New Solved 1/1 1/1 0/1 0/1 0/1
Typed First-order Non-theorems Vampire
4.8 		cvc5
1.0.5 		iProver
3.8 		E
3.1 		Beagle
0.9.47
Solved/150 108/150 58/150 91/150 28/150 22/150
Av. WC Time 3.96 15.95 13.66 19.43 1.27
Av. CPU Time 25.59 15.74 101.92 62.74 2.32
Solutions 87 58% 58 38% 57 38% 28 18% 0  0%
μWCEfficiency 516 257 197 160 88
μEfficiency 329 259 220 160 60
SotAC 0.36 0.12 0.27 0.02 0.00
Core Usage/> 1.0 5.45/69 0.00/0 6.25/63 3.06/5 1.74/22
≤ 1.0 Cores 39 58 28 23 0
New Solved 90/131 51/131 77/131 28/131 22/131
First-order Theorems Vampire
4.8 		Vampire
4.7 		E
3.1 		iProver
3.8 		CSE_E
1.5 		GKC
0.8 		Drodi
3.5.1 		cvc5
1.0.5 		Zipperpin
2.1.9999 		Prover9
1109a 		CSE
1.6 		Princess
230619 		Duper
1.0
Solved/500 451/500 448/500 393/500 355/500 346/500 310/500 301/500 297/500 269/500 127/500 119/500 158/500 15/500
Av. WC Time 15.87 6.66 13.22 22.14 19.62 10.73 15.42 25.12 13.35 21.04 40.44 17.64 47.40
Av. CPU Time 105.63 45.97 90.67 159.35 19.48 78.97 118.06 24.85 85.01 20.76 40.35 114.44 47.23
Solutions 451 90% 448 89% 393 78% 354 70% 346 69% 310 62% 301 60% 297 59% 269 53% 127 25% 119 23% 114 22% 15  3%
μWCEfficiency 524 622 483 186 326 377 355 261 258 87 59 58 2
μEfficiency 371 470 389 108 329 232 260 271 186 114 61 11 2
SotAC 0.39 - 0.30 0.25 0.24 0.20 0.18 0.20 0.15 0.06 0.05 0.08 0.00
Core Usage/> 1.0 5.45/310 5.50/257 5.38/241 5.77/328 1.02/52 5.33/242 6.46/190 1.03/3 5.18/194 0.00/0 1.01/22 6.33/158 0.00/0
≤ 1.0 Cores 141 191 152 27 294 68 111 294 75 127 97 0 15
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
First-order Non-theorems Vampire
4.7 		Vampire
4.8 		cvc5
1.0.5 		E
3.1 		iProver
3.8
Solved/150 142/150 144/150 66/150 59/150 82/150
Av. WC Time 18.86 13.32 35.12 9.85 15.22
Av. CPU Time 140.47 88.55 34.93 45.59 111.86
Solutions 142 94% 113 75% 66 44% 59 39% 4  2%
μWCEfficiency 497 513 163 210 194
μEfficiency 378 395 164 148 175
SotAC 0.32 0.33 0.12 0.08 0.14
Core Usage/> 1.0 6.72/91 5.88/94 0.00/0 5.45/40 5.37/65
≤ 1.0 Cores 51 50 66 19 17
New Solved 0/0 0/0 0/0 0/0 0/0
Unit Equality CNF Twee
2.4.1 		Twee
2.4.2 		Vampire
4.8 		E
3.1 		iProver
3.8 		Drodi
3.5.1 		GKC
0.8 		Toma
0.4
Solved/300 260/300 260/300 255/300 242/300 137/300 125/300 103/300 8/300
Av. WC Time 14.34 13.70 12.88 19.54 43.92 32.24 20.70 7.51
Av. CPU Time 109.39 104.24 87.02 137.13 339.95 254.06 160.24 7.25
Solutions 260 86% 258 86% 255 85% 242 80% 137 45% 125 41% 103 34% 8  2%
μWCEfficiency 457 459 427 413 71 126 115 10
μEfficiency 236 251 229 255 32 63 54 11
SotAC - 0.36 0.35 0.31 0.11 0.09 0.07 0.00
Core Usage/> 1.0 6.45/215 6.42/210 5.81/212 5.75/198 7.09/132 7.16/113 6.83/94 0.00/0
≤ 1.0 Cores 45 50 43 44 5 12 9 8
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
SLedgeHammer Theorems E
3.0 		E
3.1 		Zipperpin
2.1.9999 		Vampire
4.8 		cvc5
1.0.5 		Satallax
3.4 		Lash
1.13 		Leo-III
1.7.8 		Duper
1.0
Solved/1000 467/1000 467/1000 462/1000 454/1000 362/1000 278/1000 219/1000 125/1000 51/1000
Av. WC Time 3.75 3.93 5.31 3.35 2.65 8.90 7.47 4.76 7.54
Av. CPU Time 3.60 3.78 4.75 3.26 2.78 8.72 7.26 6.48 7.40
Solutions 467 46% 467 46% 462 46% 454 45% 362 36% 278 27% 219 21% 125 12% 51  5%
μWCEfficiency 277 273 146 291 173 74 129 32 8
μEfficiency 262 260 137 273 188 69 128 36 8
SotAC 0.18 0.18 0.18 0.17 0.13 0.08 0.06 0.02 0.01
Core Usage/> 1.0 1.02/54 1.03/56 0.00/0 1.03/100 1.09/345 1.02/10 1.01/8 1.39/97 0.00/0
≤ 1.0 Cores 413 411 462 354 17 268 211 28 51
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
THF without Equality Vampire
4.8 		Zipperpin
2.1.9999 		Zipperpin
2.1.999 		E
3.1 		Satallax
3.4 		Leo-III
1.7.8 		Lash
1.13 		cvc5
1.0.5 		LEO-II
1.7.0 		Duper
1.0
Solved/100 90/100 87/100 86/100 66/100 58/100 55/100 49/100 40/100 28/100 16/100
Av. WC Time 3.78 10.46 18.50 2.26 52.24 4.36 32.29 3.26 1.87 7.97
Av. CPU Time 27.46 77.73 137.16 14.95 52.17 15.01 32.17 3.05 1.70 7.82
Solutions 90 90% 87 87% 86 86% 66 66% 58 58% 55 55% 44 44% 40 40% 28 28% 16 16%
μWCEfficiency 743 426 427 604 123 204 264 339 244 24
μEfficiency 575 317 357 549 129 112 269 349 244 25
SotAC 0.39 0.37 - 0.23 0.19 0.17 0.14 0.10 0.05 0.01
Core Usage/> 1.0 5.53/41 6.53/60 6.75/53 4.36/17 1.01/8 2.66/55 1.01/5 0.00/0 0.00/0 0.00/0
≤ 1.0 Cores 49 27 33 49 50 0 44 40 28 16
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
THF with Equality Vampire
4.8 		Zipperpin
2.1.9999 		Zipperpin
2.1.999 		E
3.1 		Leo-III
1.7.8 		cvc5
1.0.5 		Satallax
3.4 		Lash
1.13 		LEO-II
1.7.0 		Duper
1.0
Solved/400 362/400 353/400 352/400 341/400 247/400 218/400 210/400 159/400 30/400 20/400
Av. WC Time 4.70 12.08 10.93 8.20 9.75 14.75 36.43 15.38 7.38 25.42
Av. CPU Time 33.35 87.20 78.78 42.03 37.16 14.49 36.42 15.22 7.18 25.24
Solutions 362 90% 353 88% 352 88% 341 85% 247 61% 218 54% 210 52% 152 38% 30  7% 20  5%
μWCEfficiency 657 439 436 667 159 346 215 222 46 6
μEfficiency 488 339 352 621 73 365 218 226 48 6
SotAC 0.38 0.36 - 0.35 0.20 0.18 0.16 0.11 0.02 0.01
Core Usage/> 1.0 5.45/214 6.26/235 6.46/224 4.51/100 3.02/247 0.00/0 1.01/38 1.01/9 0.00/0 0.00/0
≤ 1.0 Cores 148 118 128 241 0 218 172 150 30 20
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
TFA using Integers Vampire
4.8 		cvc5
1.0 		cvc5
1.0.5 		iProver
3.8 		Princess
230619
Solved/175 131/175 125/175 122/175 71/175 93/175
Av. WC Time 18.11 28.65 27.61 21.51 5.73
Av. CPU Time 122.86 28.41 27.35 161.14 36.19
Solutions 131 74% 125 71% 122 69% 71 40% 66 37%
μWCEfficiency 408 252 270 158 205
μEfficiency 299 252 270 131 77
SotAC 0.26 - 0.22 0.08 0.14
Core Usage/> 1.0 6.00/87 0.00/0 0.00/0 5.42/55 4.65/93
≤ 1.0 Cores 44 125 122 16 0
New Solved 0/0 0/0 0/0 0/0 0/0
TFA using Reals cvc5
1.0 		Vampire
4.8 		cvc5
1.0.5 		iProver
3.8 		Princess
230619
Solved/25 23/25 23/25 22/25 13/25 2/25
Av. WC Time 3.02 7.10 3.13 3.38 2.43
Av. CPU Time 2.81 47.31 2.88 20.81 13.66
Solutions 23 92% 23 92% 22 88% 12 48% 2  8%
μWCEfficiency 770 803 730 246 30
μEfficiency 770 800 730 267 10
SotAC - 0.34 0.31 0.12 0.01
Core Usage/> 1.0 0.00/0 6.96/3 0.00/0 4.16/8 5.04/2
≤ 1.0 Cores 23 20 22 5 0
New Solved 1/1 1/1 0/1 0/1 0/1
Typed First-order Non-theorems Vampire
4.8 		cvc5
1.0.5 		iProver
3.8 		E
3.1 		Beagle
0.9.47
Solved/150 108/150 58/150 91/150 28/150 22/150
Av. WC Time 3.96 15.95 13.66 19.43 1.27
Av. CPU Time 25.59 15.74 101.92 62.74 2.32
Solutions 87 58% 58 38% 57 38% 28 18% 0  0%
μWCEfficiency 516 257 197 160 88
μEfficiency 329 259 220 160 60
SotAC 0.36 0.12 0.27 0.02 0.00
Core Usage/> 1.0 5.45/69 0.00/0 6.25/63 3.06/5 1.74/22
≤ 1.0 Cores 39 58 28 23 0
New Solved 90/131 51/131 77/131 28/131 22/131
FOF Theorems without Equality Vampire
4.7 		Vampire
4.8 		iProver
3.8 		E
3.1 		CSE_E
1.5 		GKC
0.8 		Drodi
3.5.1 		cvc5
1.0.5 		Zipperpin
2.1.9999 		CSE
1.6 		Prover9
1109a 		Princess
230619 		Duper
1.0
Solved/75 72/75 72/75 67/75 62/75 59/75 56/75 54/75 46/75 45/75 36/75 16/75 15/75 4/75
Av. WC Time 4.03 6.99 14.55 10.50 12.84 2.73 8.81 64.76 6.23 37.80 7.74 29.03 50.21
Av. CPU Time 22.82 40.85 101.25 65.13 12.76 16.71 67.54 64.59 36.81 37.73 7.47 206.03 50.02
Solutions 72 96% 72 96% 67 89% 62 82% 59 78% 56 74% 54 72% 46 61% 45 60% 36 48% 16 21% 9 12% 4  5%
μWCEfficiency 750 722 251 638 429 588 469 122 310 183 97 41 3
μEfficiency 635 524 140 588 429 400 348 123 253 183 120 9 3
SotAC - 0.37 0.32 0.27 0.25 0.22 0.21 0.19 0.14 0.11 0.03 0.04 0.01
Core Usage/> 1.0 4.77/33 4.29/45 5.49/63 5.78/19 1.01/13 3.95/42 6.29/32 1.03/1 4.38/30 1.00/6 0.00/0 6.27/15 0.00/0
≤ 1.0 Cores 39 27 4 43 46 14 22 45 15 30 16 0 4
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Theorems with Equality Vampire
4.8 		Vampire
4.7 		E
3.1 		CSE_E
1.5 		iProver
3.8 		GKC
0.8 		cvc5
1.0.5 		Drodi
3.5.1 		Zipperpin
2.1.9999 		Prover9
1109a 		Princess
230619 		CSE
1.6 		Duper
1.0
Solved/425 379/425 376/425 331/425 287/425 288/425 254/425 251/425 247/425 224/425 111/425 143/425 83/425 11/425
Av. WC Time 17.56 7.16 13.73 21.01 23.91 12.49 17.86 16.86 14.79 22.96 16.44 41.58 46.38
Av. CPU Time 117.93 50.41 95.45 20.86 172.87 92.70 17.57 129.10 94.70 22.67 104.84 41.48 46.21
Solutions 379 89% 376 88% 331 77% 287 67% 287 67% 254 59% 251 59% 247 58% 224 52% 111 26% 105 24% 83 19% 11  2%
μWCEfficiency 489 599 456 308 174 340 286 335 249 85 61 37 1
μEfficiency 344 440 354 311 103 202 298 245 174 112 12 39 1
SotAC 0.39 - 0.31 0.24 0.24 0.19 0.20 0.18 0.15 0.06 0.09 0.05 0.00
Core Usage/> 1.0 5.65/265 5.61/224 5.35/222 1.02/39 5.83/265 5.62/200 1.03/2 6.49/158 5.32/164 0.00/0 6.33/143 1.01/16 0.00/0
≤ 1.0 Cores 114 152 109 248 23 54 249 89 60 111 0 67 11
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
FOF Non-theorems without Equality Vampire
4.7 		Vampire
4.8 		cvc5
1.0.5 		E
3.1 		iProver
3.8
Solved/100 95/100 96/100 54/100 36/100 66/100
Av. WC Time 20.71 13.63 25.28 8.74 13.40
Av. CPU Time 159.87 91.98 25.09 42.30 96.54
Solutions 95 95% 86 86% 54 54% 36 36% 0  0%
μWCEfficiency 524 555 233 158 256
μEfficiency 425 457 235 104 255
SotAC 0.29 0.30 0.15 0.07 0.17
Core Usage/> 1.0 6.89/56 5.87/57 0.00/0 5.76/27 4.97/49
≤ 1.0 Cores 39 39 54 9 17
New Solved 0/0 0/0 0/0 0/0 0/0
FOF Non-theorems with Equality Vampire
4.7 		Vampire
4.8 		E
3.1 		cvc5
1.0.5 		iProver
3.8
Solved/50 47/50 48/50 23/50 12/50 16/50
Av. WC Time 15.14 12.71 11.58 79.41 22.74
Av. CPU Time 101.26 81.68 50.75 79.21 175.08
Solutions 47 94% 27 54% 23 46% 12 24% 4  8%
μWCEfficiency 443 430 314 23 69
μEfficiency 285 271 235 23 17
SotAC 0.38 0.40 0.10 0.04 0.09
Core Usage/> 1.0 6.45/35 5.88/37 4.81/13 0.00/0 6.58/16
≤ 1.0 Cores 12 11 10 12 0
New Solved 0/0 0/0 0/0 0/0 0/0
Unit Equality CNF Twee
2.4.1 		Twee
2.4.2 		Vampire
4.8 		E
3.1 		iProver
3.8 		Drodi
3.5.1 		GKC
0.8 		Toma
0.4
Solved/300 260/300 260/300 255/300 242/300 137/300 125/300 103/300 8/300
Av. WC Time 14.34 13.70 12.88 19.54 43.92 32.24 20.70 7.51
Av. CPU Time 109.39 104.24 87.02 137.13 339.95 254.06 160.24 7.25
Solutions 260 86% 258 86% 255 85% 242 80% 137 45% 125 41% 103 34% 8  2%
μWCEfficiency 457 459 427 413 71 126 115 10
μEfficiency 236 251 229 255 32 63 54 11
SotAC - 0.36 0.35 0.31 0.11 0.09 0.07 0.00
Core Usage/> 1.0 6.45/215 6.42/210 5.81/212 5.75/198 7.09/132 7.16/113 6.83/94 0.00/0
≤ 1.0 Cores 45 50 43 44 5 12 9 8
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
SLedgeHammer Theorems E
3.0 		E
3.1 		Zipperpin
2.1.9999 		Vampire
4.8 		cvc5
1.0.5 		Satallax
3.4 		Lash
1.13 		Leo-III
1.7.8 		Duper
1.0
Solved/1000 467/1000 467/1000 462/1000 454/1000 362/1000 278/1000 219/1000 125/1000 51/1000
Av. WC Time 3.75 3.93 5.31 3.35 2.65 8.90 7.47 4.76 7.54
Av. CPU Time 3.60 3.78 4.75 3.26 2.78 8.72 7.26 6.48 7.40
Solutions 467 46% 467 46% 462 46% 454 45% 362 36% 278 27% 219 21% 125 12% 51  5%
μWCEfficiency 277 273 146 291 173 74 129 32 8
μEfficiency 262 260 137 273 188 69 128 36 8
SotAC 0.18 0.18 0.18 0.17 0.13 0.08 0.06 0.02 0.01
Core Usage/> 1.0 1.02/54 1.03/56 0.00/0 1.03/100 1.09/345 1.02/10 1.01/8 1.39/97 0.00/0
≤ 1.0 Cores 413 411 462 354 17 268 211 28 51
New Solved 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0