Higher-order Theorems | Zipperpin 2.1 | Zipperpin 2.1.999 | E 3.0 | Vampire 4.7 | Leo-III 1.7.0 | Satallax 3.4 | cvc5 1.0 | Lash 1.12 | LEO-II 1.7.0 |
---|---|---|---|---|---|---|---|---|---|
Solved/500 | 460/500 | 456/500 | 419/500 | 367/500 | 336/500 | 329/500 | 282/500 | 280/500 | 174/500 |
Av. WC Time | 8.73 | 9.15 | 3.10 | 7.54 | 6.11 | 20.28 | 5.93 | 7.57 | 3.78 |
Av. CPU Time | 61.47 | 65.00 | 13.76 | 54.45 | 19.83 | 20.25 | 5.70 | 7.41 | 3.60 |
Solutions | 460 92% | 456 91% | 419 83% | 367 73% | 335 67% | 329 65% | 282 56% | 262 52% | 173 34% |
μWCEfficiency | 527 | 529 | 684 | 488 | 228 | 308 | 425 | 410 | 307 |
μEfficiency | 436 | 423 | 634 | 396 | 124 | 317 | 431 | 419 | 310 |
SotAC | - | 0.34 | 0.28 | 0.21 | 0.15 | 0.15 | 0.10 | 0.10 | 0.03 |
Core Usage | 3.74 | 3.72 | 1.39 | 3.43 | 2.51 | 0.74 | 0.62 | 0.61 | 0.53 |
New Solved | 128/142 | 133/142 | 121/142 | 109/142 | 92/142 | 94/142 | 73/142 | 77/142 | 43/142 |
Typed First-order Theorems +*-/ | SnakeForV4.7 1.0 | cvc5 1.0 | Vampire 4.5 | Vampire 4.7 | iProver 3.6 |
---|---|---|---|---|---|
Solved/250 | 218/250 | 195/250 | 192/250 | 187/250 | 138/250 |
Av. WC Time | 5.46 | 20.45 | 5.56 | 4.35 | 17.74 |
Av. CPU Time | 37.85 | 20.23 | 18.83 | 29.63 | 127.43 |
Solutions | 218 87% | 195 78% | 192 76% | 187 74% | 137 54% |
μWCEfficiency | 620 | 289 | 513 | 495 | 130 |
μEfficiency | 346 | 291 | 295 | 337 | 69 |
SotAC | 0.27 | 0.24 | - | 0.17 | 0.06 |
Core Usage | 4.06 | 0.83 | 3.63 | 3.87 | 5.20 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
First-order Theorems | SnakeForV4.7 1.0 | Vampire 4.7 | Vampire 4.6 | E 3.0 | iProver 3.6 | CSE_E 1.4 | GKC 0.7 | Zipperpin 2.1.999 | cvc5 1.0 | Drodi 3.3.3 | CSE 1.5 | Prover9 1109a | Goeland 1.0.0 | Etableau 0.67 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Solved/500 | 460/500 | 451/500 | 448/500 | 384/500 | 365/500 | 361/500 | 335/500 | 294/500 | 271/500 | 205/500 | 128/500 | 123/500 | 7/500 | 279/500 |
Av. WC Time | 4.99 | 5.44 | 6.15 | 8.47 | 15.79 | 11.50 | 8.18 | 12.79 | 23.47 | 7.83 | 31.62 | 14.62 | 6.53 | 6.29 |
Av. CPU Time | 32.02 | 35.63 | 30.77 | 54.05 | 103.95 | 11.40 | 57.51 | 81.65 | 23.26 | 57.00 | 31.55 | 14.33 | 36.65 | 36.58 |
Solutions | 460 92% | 451 90% | 448 89% | 384 76% | 365 73% | 361 72% | 335 67% | 294 58% | 271 54% | 205 41% | 128 25% | 123 24% | 7 1% | 0 0% |
μWCEfficiency | 672 | 664 | 687 | 479 | 191 | 396 | 420 | 286 | 229 | 271 | 76 | 93 | 7 | 359 |
μEfficiency | 450 | 524 | 518 | 397 | 87 | 399 | 261 | 206 | 235 | 183 | 75 | 119 | 6 | 272 |
SotAC | 0.36 | 0.35 | - | 0.26 | 0.23 | 0.23 | 0.20 | 0.16 | 0.16 | 0.09 | 0.05 | 0.05 | 0.00 | 0.15 |
Core Usage | 3.36 | 2.82 | 2.69 | 3.00 | 4.63 | 0.91 | 4.12 | 3.82 | 0.82 | 3.97 | 0.95 | 0.87 | 3.40 | 3.04 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
First-order Non-theorems | Vampire 4.6 | Vampire 4.7 | SnakeForV4.7 1.0 | cvc5 1.0 | iProver 3.6 | E 3.0 |
---|---|---|---|---|---|---|
Solved/250 | 167/250 | 160/250 | 159/250 | 78/250 | 63/250 | 62/250 |
Av. WC Time | 16.23 | 18.08 | 16.40 | 31.62 | 20.49 | 5.13 |
Av. CPU Time | 55.99 | 138.71 | 126.87 | 31.42 | 152.10 | 19.01 |
Solutions | 167 66% | 160 64% | 159 63% | 78 31% | 63 25% | 62 24% |
μWCEfficiency | 322 | 325 | 340 | 90 | 60 | 186 |
μEfficiency | 230 | 227 | 227 | 94 | 28 | 157 |
SotAC | - | 0.25 | 0.25 | 0.09 | 0.07 | 0.05 |
Core Usage | 3.54 | 4.78 | 4.68 | 0.91 | 5.51 | 2.21 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
Unit Equality CNF | Twee 2.4.1 | Twee 2.4 | E 3.0 | SnakeForV4.7 1.0 | Vampire 4.7 | iProver 3.6 | GKC 0.7 | Drodi 3.3.3 | Toma 0.2 |
---|---|---|---|---|---|---|---|---|---|
Solved/250 | 216/250 | 215/250 | 212/250 | 207/250 | 152/250 | 142/250 | 120/250 | 116/250 | 1/250 |
Av. WC Time | 7.37 | 7.54 | 11.07 | 6.37 | 8.98 | 14.38 | 11.35 | 5.75 | 2.22 |
Av. CPU Time | 55.53 | 56.85 | 74.14 | 45.18 | 67.32 | 101.19 | 85.32 | 43.31 | 1.99 |
Solutions | 216 86% | 215 86% | 212 84% | 207 82% | 152 60% | 142 56% | 120 48% | 116 46% | 1 0% |
μWCEfficiency | 585 | 574 | 527 | 546 | 381 | 158 | 230 | 295 | 1 |
μEfficiency | 396 | 375 | 392 | 305 | 218 | 71 | 137 | 199 | 2 |
SotAC | 0.31 | - | 0.29 | 0.27 | 0.13 | 0.11 | 0.09 | 0.07 | 0.00 |
Core Usage | 4.02 | 4.19 | 3.72 | 4.41 | 4.66 | 4.83 | 5.18 | 4.28 | 0.90 |
New Solved | 38/46 | 36/46 | 43/46 | 32/46 | 20/46 | 23/46 | 19/46 | 19/46 | 1/46 |
SLedgeHammer Theorems | E 3.0 | Ehoh 2.7 | Zipperpin 2.1.999 | Vampire 4.7 | cvc5 1.0 |
---|---|---|---|---|---|
Solved/720 | 655/720 | 655/720 | 654/720 | 629/720 | 595/720 |
Av. WC Time | 1.94 | 2.18 | 1.30 | 3.43 | 0.97 |
Av. CPU Time | 1.72 | 1.99 | 5.41 | 3.30 | 0.81 |
Solutions | 655 90% | 655 90% | 654 90% | 629 87% | 595 82% |
μWCEfficiency | 788 | 745 | 529 | 486 | 778 |
μEfficiency | 780 | 738 | 688 | 463 | 763 |
SotAC | 0.23 | - | 0.21 | 0.19 | 0.15 |
Core Usage | 0.62 | 0.60 | 2.83 | 0.86 | 0.79 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
Large Theory Batch | Vampire 4.7 | Vampire 4.6 | iProver 3.6 | E 3.0 |
---|---|---|---|---|
Solved/8000 | 3729/8000 | 3725/8000 | 3446/7999 | 3292/8000 |
Av. WC Time | 46.34 | 46.38 | 50.12 | 52.49 |
Av. CPU Time | 366.72 | 367.90 | 352.76 | 105.63 |
Solutions | 3729 46% | 3725 46% | 3446 43% | 3290 41% |
μWCEfficiency | 146 | 150 | 27 | 90 |
μEfficiency | 154 | 163 | 126 | 116 |
SotAC | 0.09 | - | 0.07 | 0.05 |
Core Usage | 5.69 | 5.66 | 6.06 | 1.64 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 |
THF without Equality | Zipperpin 2.1 | Zipperpin 2.1.999 | E 3.0 | Vampire 4.7 | Satallax 3.4 | Leo-III 1.7.0 | cvc5 1.0 | Lash 1.12 | LEO-II 1.7.0 |
---|---|---|---|---|---|---|---|---|---|
Solved/100 | 82/100 | 79/100 | 63/100 | 59/100 | 54/100 | 51/100 | 38/100 | 41/100 | 27/100 |
Av. WC Time | 14.07 | 16.62 | 1.77 | 8.12 | 36.16 | 4.63 | 2.73 | 4.61 | 1.73 |
Av. CPU Time | 105.18 | 125.46 | 11.25 | 61.22 | 36.04 | 16.84 | 2.53 | 4.47 | 1.52 |
Solutions | 82 82% | 79 79% | 63 63% | 59 59% | 54 54% | 51 51% | 38 38% | 37 37% | 27 27% |
μWCEfficiency | 399 | 394 | 574 | 414 | 116 | 178 | 314 | 294 | 226 |
μEfficiency | 327 | 331 | 536 | 375 | 123 | 98 | 314 | 294 | 243 |
SotAC | - | 0.33 | 0.24 | 0.20 | 0.17 | 0.16 | 0.10 | 0.11 | 0.03 |
Core Usage | 4.53 | 4.34 | 1.40 | 3.00 | 0.91 | 2.64 | 0.60 | 0.69 | 0.59 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
THF with Equality | Zipperpin 2.1 | Zipperpin 2.1.999 | E 3.0 | Vampire 4.7 | Leo-III 1.7.0 | Satallax 3.4 | cvc5 1.0 | Lash 1.12 | LEO-II 1.7.0 |
---|---|---|---|---|---|---|---|---|---|
Solved/400 | 378/400 | 377/400 | 356/400 | 308/400 | 285/400 | 275/400 | 244/400 | 239/400 | 147/400 |
Av. WC Time | 7.58 | 7.58 | 3.33 | 7.43 | 6.37 | 17.16 | 6.43 | 8.07 | 4.16 |
Av. CPU Time | 51.99 | 52.33 | 14.21 | 53.15 | 20.36 | 17.15 | 6.20 | 7.91 | 3.98 |
Solutions | 378 94% | 377 94% | 356 89% | 308 77% | 284 71% | 275 68% | 244 61% | 225 56% | 146 36% |
μWCEfficiency | 558 | 563 | 712 | 506 | 241 | 356 | 453 | 439 | 327 |
μEfficiency | 463 | 446 | 659 | 401 | 131 | 366 | 460 | 450 | 327 |
SotAC | - | 0.35 | 0.29 | 0.21 | 0.15 | 0.14 | 0.10 | 0.09 | 0.03 |
Core Usage | 3.57 | 3.59 | 1.39 | 3.51 | 2.49 | 0.70 | 0.62 | 0.60 | 0.52 |
New Solved | 128/142 | 133/142 | 121/142 | 109/142 | 92/142 | 94/142 | 73/142 | 77/142 | 43/142 |
TFA using Integers | SnakeForV4.7 1.0 | cvc5 1.0 | Vampire 4.5 | Vampire 4.7 | iProver 3.6 |
---|---|---|---|---|---|
Solved/225 | 195/225 | 173/225 | 171/225 | 166/225 | 123/225 |
Av. WC Time | 5.55 | 22.53 | 6.02 | 4.85 | 19.17 |
Av. CPU Time | 38.22 | 22.31 | 20.80 | 33.30 | 138.14 |
Solutions | 195 86% | 173 76% | 171 76% | 166 73% | 122 54% |
μWCEfficiency | 593 | 248 | 481 | 458 | 116 |
μEfficiency | 304 | 250 | 241 | 288 | 61 |
SotAC | 0.28 | 0.24 | - | 0.16 | 0.06 |
Core Usage | 4.33 | 0.86 | 4.01 | 4.26 | 5.43 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
TFA using Reals | SnakeForV4.7 1.0 | cvc5 1.0 | Vampire 4.7 | Vampire 4.5 | iProver 3.6 |
---|---|---|---|---|---|
Solved/25 | 23/25 | 22/25 | 21/25 | 21/25 | 15/25 |
Av. WC Time | 4.70 | 4.07 | 0.40 | 1.82 | 6.01 |
Av. CPU Time | 34.78 | 3.85 | 0.54 | 2.81 | 39.61 |
Solutions | 23 92% | 22 88% | 21 84% | 21 84% | 15 60% |
μWCEfficiency | 860 | 659 | 820 | 801 | 254 |
μEfficiency | 718 | 659 | 779 | 781 | 146 |
SotAC | 0.25 | 0.21 | 0.19 | - | 0.08 |
Core Usage | 1.83 | 0.57 | 0.81 | 0.55 | 3.32 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
FOF Theorems without Equality | Vampire 4.7 | Vampire 4.6 | SnakeForV4.7 1.0 | iProver 3.6 | CSE_E 1.4 | E 3.0 | GKC 0.7 | Zipperpin 2.1.999 | cvc5 1.0 | Drodi 3.3.3 | CSE 1.5 | Prover9 1109a | Goeland 1.0.0 | Etableau 0.67 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Solved/100 | 94/100 | 92/100 | 91/100 | 88/100 | 79/100 | 78/100 | 76/100 | 66/100 | 52/100 | 46/100 | 41/100 | 19/100 | 2/100 | 58/100 |
Av. WC Time | 5.47 | 3.20 | 3.01 | 9.47 | 10.39 | 6.05 | 3.88 | 16.98 | 56.17 | 6.42 | 30.31 | 11.16 | 6.21 | 6.23 |
Av. CPU Time | 36.16 | 12.51 | 18.48 | 59.27 | 10.28 | 39.63 | 26.79 | 111.82 | 55.93 | 47.25 | 30.21 | 10.89 | 11.96 | 37.17 |
Solutions | 94 94% | 92 92% | 91 91% | 88 88% | 79 79% | 78 78% | 76 76% | 66 66% | 52 52% | 46 46% | 41 41% | 19 19% | 2 2% | 0 0% |
μWCEfficiency | 743 | 798 | 714 | 237 | 445 | 412 | 590 | 264 | 120 | 282 | 165 | 90 | 11 | 341 |
μEfficiency | 637 | 627 | 466 | 126 | 441 | 398 | 451 | 197 | 120 | 153 | 165 | 116 | 10 | 255 |
SotAC | 0.33 | - | 0.31 | 0.29 | 0.24 | 0.22 | 0.22 | 0.16 | 0.13 | 0.10 | 0.08 | 0.03 | 0.01 | 0.14 |
Core Usage | 2.35 | 2.08 | 3.08 | 4.35 | 0.89 | 2.56 | 2.88 | 4.09 | 0.86 | 4.71 | 0.91 | 0.84 | 1.44 | 3.18 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
FOF Theorems with Equality | SnakeForV4.7 1.0 | Vampire 4.7 | Vampire 4.6 | E 3.0 | CSE_E 1.4 | iProver 3.6 | GKC 0.7 | Zipperpin 2.1.999 | cvc5 1.0 | Drodi 3.3.3 | Prover9 1109a | CSE 1.5 | Goeland 1.0.0 | Etableau 0.67 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Solved/400 | 369/400 | 357/400 | 356/400 | 306/400 | 282/400 | 277/400 | 259/400 | 228/400 | 219/400 | 159/400 | 104/400 | 87/400 | 5/400 | 221/400 |
Av. WC Time | 5.48 | 5.43 | 6.91 | 9.08 | 11.81 | 17.80 | 9.44 | 11.57 | 15.70 | 8.23 | 15.25 | 32.23 | 6.66 | 6.30 |
Av. CPU Time | 35.36 | 35.49 | 35.48 | 57.73 | 11.71 | 118.15 | 66.53 | 72.91 | 15.51 | 59.83 | 14.96 | 32.18 | 46.52 | 36.42 |
Solutions | 369 92% | 357 89% | 356 89% | 306 76% | 282 70% | 277 69% | 259 64% | 228 57% | 219 54% | 159 39% | 104 26% | 87 21% | 5 1% | 0 0% |
μWCEfficiency | 661 | 645 | 660 | 496 | 383 | 179 | 377 | 292 | 257 | 268 | 94 | 54 | 7 | 363 |
μEfficiency | 445 | 496 | 491 | 396 | 389 | 77 | 214 | 209 | 264 | 190 | 120 | 53 | 5 | 276 |
SotAC | 0.38 | 0.36 | - | 0.27 | 0.23 | 0.22 | 0.20 | 0.16 | 0.16 | 0.08 | 0.05 | 0.05 | 0.00 | 0.15 |
Core Usage | 3.43 | 2.95 | 2.85 | 3.11 | 0.92 | 4.72 | 4.48 | 3.74 | 0.81 | 3.75 | 0.87 | 0.97 | 4.19 | 3.00 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
FOF Non-theorems without Equality | Vampire 4.6 | SnakeForV4.7 1.0 | Vampire 4.7 | cvc5 1.0 | iProver 3.6 | E 3.0 |
---|---|---|---|---|---|---|
Solved/100 | 69/100 | 67/100 | 61/100 | 43/100 | 26/100 | 16/100 |
Av. WC Time | 31.18 | 32.27 | 35.81 | 35.34 | 15.60 | 10.08 |
Av. CPU Time | 87.28 | 252.96 | 276.87 | 35.13 | 113.67 | 34.22 |
Solutions | 69 69% | 67 67% | 61 61% | 43 43% | 26 26% | 16 16% |
μWCEfficiency | 227 | 240 | 223 | 80 | 77 | 74 |
μEfficiency | 168 | 166 | 163 | 80 | 50 | 48 |
SotAC | - | 0.26 | 0.22 | 0.16 | 0.06 | 0.03 |
Core Usage | 2.68 | 5.63 | 5.53 | 0.90 | 4.53 | 3.87 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
FOF Non-theorems with Equality | Vampire 4.7 | Vampire 4.6 | SnakeForV4.7 1.0 | E 3.0 | iProver 3.6 | cvc5 1.0 |
---|---|---|---|---|---|---|
Solved/150 | 99/150 | 98/150 | 92/150 | 46/150 | 37/150 | 35/150 |
Av. WC Time | 7.16 | 5.71 | 4.84 | 3.41 | 23.93 | 27.04 |
Av. CPU Time | 53.58 | 33.96 | 35.05 | 13.73 | 179.11 | 26.85 |
Solutions | 99 66% | 98 65% | 92 61% | 46 30% | 37 24% | 35 23% |
μWCEfficiency | 393 | 386 | 406 | 261 | 49 | 96 |
μEfficiency | 269 | 272 | 267 | 229 | 13 | 103 |
SotAC | 0.27 | - | 0.24 | 0.07 | 0.08 | 0.04 |
Core Usage | 4.33 | 4.14 | 3.98 | 1.64 | 6.21 | 0.92 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
Unit Equality CNF | Twee 2.4.1 | Twee 2.4 | E 3.0 | SnakeForV4.7 1.0 | Vampire 4.7 | iProver 3.6 | GKC 0.7 | Drodi 3.3.3 | Toma 0.2 |
---|---|---|---|---|---|---|---|---|---|
Solved/250 | 216/250 | 215/250 | 212/250 | 207/250 | 152/250 | 142/250 | 120/250 | 116/250 | 1/250 |
Av. WC Time | 7.37 | 7.54 | 11.07 | 6.37 | 8.98 | 14.38 | 11.35 | 5.75 | 2.22 |
Av. CPU Time | 55.53 | 56.85 | 74.14 | 45.18 | 67.32 | 101.19 | 85.32 | 43.31 | 1.99 |
Solutions | 216 86% | 215 86% | 212 84% | 207 82% | 152 60% | 142 56% | 120 48% | 116 46% | 1 0% |
μWCEfficiency | 585 | 574 | 527 | 546 | 381 | 158 | 230 | 295 | 1 |
μEfficiency | 396 | 375 | 392 | 305 | 218 | 71 | 137 | 199 | 2 |
SotAC | 0.31 | - | 0.29 | 0.27 | 0.13 | 0.11 | 0.09 | 0.07 | 0.00 |
Core Usage | 4.02 | 4.19 | 3.72 | 4.41 | 4.66 | 4.83 | 5.18 | 4.28 | 0.90 |
New Solved | 38/46 | 36/46 | 43/46 | 32/46 | 20/46 | 23/46 | 19/46 | 19/46 | 1/46 |
SLedgeHammer Theorems | E 3.0 | Ehoh 2.7 | Zipperpin 2.1.999 | Vampire 4.7 | cvc5 1.0 |
---|---|---|---|---|---|
Solved/720 | 655/720 | 655/720 | 654/720 | 629/720 | 595/720 |
Av. WC Time | 1.94 | 2.18 | 1.30 | 3.43 | 0.97 |
Av. CPU Time | 1.72 | 1.99 | 5.41 | 3.30 | 0.81 |
Solutions | 655 90% | 655 90% | 654 90% | 629 87% | 595 82% |
μWCEfficiency | 788 | 745 | 529 | 486 | 778 |
μEfficiency | 780 | 738 | 688 | 463 | 763 |
SotAC | 0.23 | - | 0.21 | 0.19 | 0.15 |
Core Usage | 0.62 | 0.60 | 2.83 | 0.86 | 0.79 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
LTB Van Emde Boas Tree Theorems | Vampire 4.7 | Vampire 4.6 | iProver 3.6 | E 3.0 |
---|---|---|---|---|
Solved/8000 | 3729/8000 | 3725/8000 | 3446/7999 | 3292/8000 |
Av. WC Time | 46.34 | 46.38 | 50.12 | 52.49 |
Av. CPU Time | 366.72 | 367.90 | 352.76 | 105.63 |
Solutions | 3729 46% | 3725 46% | 3446 43% | 3290 41% |
μWCEfficiency | 154 | 163 | 126 | 116 |
μEfficiency | 146 | 150 | 27 | 90 |
SotAC | 0.09 | - | 0.07 | 0.05 |
Core Usage | 5.69 | 5.66 | 6.06 | 1.64 |
New Solved | 0/0 | 0/0 | 0/0 | 0/0 |