| Higher-order Theorems | Satallax_MaLeS 1.3 | Satallax 2.7 | Isabelle 2013 | LEO‑II 1.6.2 | agsyHOL 1.0 | HOLyHammer 140616 | cocATP 0.2.0 |
|---|---|---|---|---|---|---|---|
| Solved/400 | 333/400 | 324/400 | 316/400 | 269/400 | 236/400 | 96/400 | 74/400 |
| Av. CPU Time | 18.88 | 5.68 | 38.91 | 7.21 | 5.83 | 25.89 | 4.65 |
| Solutions | 0/400 | 0/400 | 0/400 | 265/400 | 0/400 | 0/400 | 74/400 |
| μEfficiency | 313 | 576 | 113 | 554 | 499 | 130 | 139 |
| SOTAC | 0.24 | 0.25 | 0.23 | 0.23 | 0.24 | 0.23 | 0.21 |
| Core Usage | 1.00 | 0.99 | 0.97 | 0.99 | 1.00 | 1.00 | 0.99 |
| New Solved | 8/8 | 8/8 | 5/8 | 7/8 | 5/8 | 0/8 | 1/8 |
| Typed First-order Theorems +*-/ | CVC4 1.4‑TFA | Princess 140704 | SPASS+T 2.2.19 | SPASS+T 2.2.20 | Beagle 0.9 | Zipperposition 0.4‑TFF |
|---|---|---|---|---|---|---|
| Solved/200 | 179/200 | 176/200 | 173/200 | 173/200 | 173/200 | 80/200 |
| Av. CPU Time | 4.47 | 11.81 | 3.44 | 3.57 | 5.49 | 6.57 |
| Solutions | 0/200 | 0/200 | 173/200 | 173/200 | 0/200 | 80/200 |
| μEfficiency | 797 | 307 | 402 | 402 | 623 | 313 |
| SOTAC | 0.22 | 0.21 | 0.19 | 0.19 | 0.20 | 0.27 |
| Core Usage | 1.30 | 1.19 | 1.83 | 1.79 | 1.21 | 0.99 |
| New Solved | 33/50 | 35/50 | 30/50 | 30/50 | 28/50 | 44/50 |
| First-order Theorems | Vampire 2.6 | ET 0.1 | E 1.9 | VanHElsing 1.0 | CVC4 1.4‑FOF | iProver 1.4 | leanCoP 2.2 | Prover9 1109a | Zipperposition 0.4‑FOF | Muscadet 4.4 | Princess 140704 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Solved/400 | 375/400 | 339/400 | 321/400 | 310/400 | 215/400 | 216/400 | 158/400 | 95/400 | 73/400 | 32/400 | 134/400 |
| Av. CPU Time | 13.19 | 29.31 | 22.88 | 17.29 | 46.03 | 18.11 | 55.15 | 41.45 | 28.81 | 19.74 | 69.31 |
| Solutions | 372/400 | 339/400 | 321/400 | 310/400 | 215/400 | 214/400 | 158/400 | 95/400 | 73/400 | 30/400 | 0/400 |
| μEfficiency | 571 | 361 | 466 | 168 | 228 | 216 | 129 | 119 | 75 | 47 | 17 |
| SOTAC | 0.22 | 0.18 | 0.17 | 0.17 | 0.15 | 0.16 | 0.14 | 0.14 | 0.13 | 0.12 | 0.13 |
| Core Usage | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.99 | 1.22 |
| New Solved | 5/6 | 5/6 | 0/6 | 0/6 | 0/6 | 6/6 | 0/6 | 0/6 | 0/6 | 0/6 | 0/6 |
| First-order Non-theorems | iProver 1.0‑SAT | iProver 1.4‑SAT | Crossbow 0.1 | CVC4 1.4‑FNT | E 1.9‑FNT |
|---|---|---|---|---|---|
| Solved/200 | 150/200 | 133/200 | 124/200 | 116/200 | 82/200 |
| Av. CPU Time | 59.12 | 63.37 | 21.66 | 24.38 | 7.23 |
| Solutions | 150/200 | 133/200 | 124/200 | 116/200 | 82/200 |
| μEfficiency | 146 | 186 | 107 | 374 | 292 |
| SOTAC | 0.30 | 0.29 | 0.32 | 0.25 | 0.38 |
| Core Usage | 1.00 | 1.00 | 1.00 | 1.00 | 0.99 |
| New Solved | 4/10 | 4/10 | 9/10 | 6/10 | 0/10 |
| Effectively Propositional CNF | iProver 0.9 | iProver 1.4 | E 1.9 |
|---|---|---|---|
| Solved/200 | 183/200 | 180/200 | 88/200 |
| Av. CPU Time | 22.96 | 22.80 | 31.02 |
| Solutions | 62/200 | 174/200 | 88/200 |
| μEfficiency | 400 | 405 | 98 |
| SOTAC | 0.44 | 0.43 | 0.36 |
| Core Usage | 1.00 | 1.00 | 1.00 |
| New Solved | 10/10 | 9/10 | 5/10 |
| Unsatisfiable Unit Equality CNF | Waldmeister 710 | E 1.9 |
|---|---|---|
| Solved/200 | 194/200 | 168/200 |
| Av. CPU Time | 11.75 | 13.08 |
| Solutions | 194/200 | 168/200 |
| μEfficiency | 746 | 633 |
| SOTAC | 0.57 | 0.50 |
| Core Usage | 1.00 | 1.00 |
| New Solved | 0/0 | 0/0 |
| THF without Equality | Satallax 2.7 | Satallax_MaLeS 1.3 | Isabelle 2013 | LEO‑II 1.6.2 | agsyHOL 1.0 | cocATP 0.2.0 | HOLyHammer 140616 |
|---|---|---|---|---|---|---|---|
| Solved/150 | 128/150 | 126/150 | 126/150 | 106/150 | 87/150 | 37/150 | 27/150 |
| Av. CPU Time | 3.52 | 9.45 | 23.89 | 5.81 | 3.65 | 1.50 | 19.58 |
| Solutions | 0/150 | 0/150 | 0/150 | 106/150 | 0/150 | 37/150 | 0/150 |
| μEfficiency | 648 | 356 | 168 | 607 | 539 | 206 | 128 |
| SOTAC | 0.23 | 0.23 | 0.23 | 0.20 | 0.23 | 0.22 | 0.21 |
| Core Usage | 0.98 | 0.99 | 1.01 | 0.99 | 0.99 | 0.95 | 0.98 |
| New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
| THF with Equality | Satallax_MaLeS 1.3 | Satallax 2.7 | Isabelle 2013 | LEO‑II 1.6.2 | agsyHOL 1.0 | HOLyHammer 140616 | cocATP 0.2.0 |
|---|---|---|---|---|---|---|---|
| Solved/250 | 207/250 | 196/250 | 190/250 | 163/250 | 149/250 | 69/250 | 37/250 |
| Av. CPU Time | 24.62 | 7.10 | 48.88 | 8.13 | 7.11 | 28.36 | 7.79 |
| Solutions | 0/250 | 0/250 | 0/250 | 159/250 | 0/250 | 0/250 | 37/250 |
| μEfficiency | 287 | 533 | 80 | 522 | 475 | 132 | 99 |
| SOTAC | 0.24 | 0.26 | 0.24 | 0.24 | 0.24 | 0.24 | 0.19 |
| Core Usage | 1.00 | 0.99 | 0.97 | 0.99 | 1.00 | 1.00 | 0.99 |
| New Solved | 8/8 | 8/8 | 5/8 | 7/8 | 5/8 | 0/8 | 1/8 |
| TFA using Integers | Princess 140704 | Zipperposition 0.4‑TFF | CVC4 1.4‑TFA | SPASS+T 2.2.19 | SPASS+T 2.2.20 | Beagle 0.9 |
|---|---|---|---|---|---|---|
| Solved/100 | 81/100 | 80/100 | 80/100 | 75/100 | 75/100 | 73/100 |
| Av. CPU Time | 20.30 | 6.57 | 10.00 | 6.51 | 6.80 | 12.57 |
| Solutions | 0/100 | 80/100 | 0/100 | 75/100 | 75/100 | 0/100 |
| μEfficiency | 291 | 626 | 605 | 314 | 314 | 325 |
| SOTAC | 0.22 | 0.27 | 0.24 | 0.18 | 0.18 | 0.18 |
| Core Usage | 1.19 | 0.99 | 1.30 | 1.83 | 1.79 | 1.21 |
| New Solved | 35/50 | 44/50 | 33/50 | 30/50 | 30/50 | 28/50 |
| TFA using Rationals | CVC4 1.4‑TFA | Beagle 0.9 | SPASS+T 2.2.19 | SPASS+T 2.2.20 | Princess 140704 | Zipperposition 0.4‑TFF |
|---|---|---|---|---|---|---|
| Solved/50 | 50/50 | 50/50 | 50/50 | 50/50 | 49/50 | 0/50 |
| Av. CPU Time | 0.00 | 0.22 | 1.09 | 1.09 | 3.53 | - |
| Solutions | 0/50 | 0/50 | 50/50 | 50/50 | 0/50 | 0/50 |
| μEfficiency | 1000 | 945 | 500 | 500 | 344 | - |
| SOTAC | 0.20 | 0.20 | 0.20 | 0.20 | 0.20 | - |
| Core Usage | 0.03 | 1.01 | 0.48 | 0.48 | 1.79 | - |
| New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
| TFA using Reals | Beagle 0.9 | CVC4 1.4‑TFA | SPASS+T 2.2.20 | SPASS+T 2.2.19 | Princess 140704 | Zipperposition 0.4‑TFF |
|---|---|---|---|---|---|---|
| Solved/50 | 50/50 | 49/50 | 48/50 | 48/50 | 46/50 | 0/50 |
| Av. CPU Time | 0.44 | 0.00 | 1.09 | 1.09 | 5.67 | - |
| Solutions | 0/50 | 0/50 | 48/50 | 48/50 | 0/50 | 0/50 |
| μEfficiency | 899 | 980 | 480 | 480 | 301 | - |
| SOTAC | 0.22 | 0.21 | 0.20 | 0.20 | 0.20 | - |
| Core Usage | 1.15 | 0.02 | 0.49 | 0.49 | 1.87 | - |
| New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
| FOF Theorems without Equality | Vampire 2.6 | E 1.9 | ET 0.1 | iProver 1.4 | VanHElsing 1.0 | leanCoP 2.2 | CVC4 1.4‑FOF | Zipperposition 0.4‑FOF | Prover9 1109a | Muscadet 4.4 | Princess 140704 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Solved/150 | 141/150 | 116/150 | 115/150 | 116/150 | 111/150 | 84/150 | 83/150 | 26/150 | 21/150 | 6/150 | 47/150 |
| Av. CPU Time | 9.19 | 9.55 | 19.65 | 9.69 | 11.68 | 49.87 | 77.71 | 41.62 | 3.21 | 39.95 | 109.14 |
| Solutions | 139/150 | 116/150 | 115/150 | 114/150 | 111/150 | 84/150 | 83/150 | 26/150 | 21/150 | 6/150 | 0/150 |
| μEfficiency | 684 | 560 | 383 | 341 | 180 | 220 | 131 | 47 | 109 | 3 | 4 |
| SOTAC | 0.24 | 0.15 | 0.15 | 0.16 | 0.15 | 0.14 | 0.16 | 0.16 | 0.12 | 0.13 | 0.12 |
| Core Usage | 0.99 | 0.99 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.96 | 0.99 | 1.28 |
| New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
| FOF Theorems with Equality | Vampire 2.6 | ET 0.1 | E 1.9 | VanHElsing 1.0 | CVC4 1.4‑FOF | iProver 1.4 | Prover9 1109a | leanCoP 2.2 | Zipperposition 0.4‑FOF | Muscadet 4.4 | Princess 140704 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Solved/250 | 234/250 | 224/250 | 205/250 | 199/250 | 132/250 | 100/250 | 74/250 | 74/250 | 47/250 | 26/250 | 87/250 |
| Av. CPU Time | 15.60 | 34.27 | 30.43 | 20.42 | 26.11 | 27.88 | 52.30 | 61.15 | 21.72 | 15.07 | 47.79 |
| Solutions | 233/250 | 224/250 | 205/250 | 199/250 | 132/250 | 100/250 | 74/250 | 74/250 | 47/250 | 24/250 | 0/250 |
| μEfficiency | 504 | 348 | 409 | 161 | 286 | 140 | 125 | 75 | 92 | 74 | 26 |
| SOTAC | 0.21 | 0.20 | 0.19 | 0.18 | 0.15 | 0.15 | 0.14 | 0.13 | 0.12 | 0.11 | 0.13 |
| Core Usage | 1.00 | 1.00 | 1.00 | 1.00 | 0.99 | 1.00 | 1.00 | 1.00 | 0.99 | 0.99 | 1.17 |
| New Solved | 5/6 | 5/6 | 0/6 | 0/6 | 0/6 | 6/6 | 0/6 | 0/6 | 0/6 | 0/6 | 0/6 |
| FOF Non-theorems without Equality | iProver 1.0‑SAT | Crossbow 0.1 | iProver 1.4‑SAT | CVC4 1.4‑FNT | E 1.9‑FNT |
|---|---|---|---|---|---|
| Solved/100 | 79/100 | 77/100 | 77/100 | 54/100 | 42/100 |
| Av. CPU Time | 47.38 | 24.86 | 52.17 | 34.94 | 2.15 |
| Solutions | 79/100 | 77/100 | 77/100 | 54/100 | 42/100 |
| μEfficiency | 241 | 143 | 322 | 266 | 265 |
| SOTAC | 0.31 | 0.30 | 0.30 | 0.23 | 0.33 |
| Core Usage | 1.00 | 1.00 | 0.99 | 1.00 | 0.95 |
| New Solved | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 |
| FOF Non-theorems with Equality | iProver 1.0‑SAT | CVC4 1.4‑FNT | iProver 1.4‑SAT | Crossbow 0.1 | E 1.9‑FNT |
|---|---|---|---|---|---|
| Solved/100 | 71/100 | 62/100 | 56/100 | 47/100 | 40/100 |
| Av. CPU Time | 72.18 | 15.18 | 78.77 | 16.42 | 12.56 |
| Solutions | 71/100 | 62/100 | 56/100 | 47/100 | 40/100 |
| μEfficiency | 51 | 481 | 49 | 72 | 319 |
| SOTAC | 0.30 | 0.28 | 0.28 | 0.36 | 0.43 |
| Core Usage | 1.00 | 0.99 | 1.00 | 0.99 | 0.99 |
| New Solved | 4/10 | 6/10 | 4/10 | 9/10 | 0/10 |
| EPR Unsatisfiable CNF | iProver 0.9 | iProver 1.4 | E 1.9 |
|---|---|---|---|
| Solved/125 | 116/125 | 114/125 | 56/125 |
| Av. CPU Time | 26.58 | 22.46 | 33.08 |
| Solutions | 0/125 | 108/125 | 56/125 |
| μEfficiency | 355 | 361 | 61 |
| SOTAC | 0.43 | 0.42 | 0.36 |
| Core Usage | 1.00 | 1.00 | 0.99 |
| New Solved | 10/10 | 9/10 | 5/10 |
| EPR Satisfiable CNF | iProver 0.9 | iProver 1.4 | E 1.9 |
|---|---|---|---|
| Solved/75 | 67/75 | 66/75 | 32/75 |
| Av. CPU Time | 16.70 | 23.38 | 27.43 |
| Solutions | 62/75 | 66/75 | 32/75 |
| μEfficiency | 474 | 479 | 160 |
| SOTAC | 0.45 | 0.44 | 0.35 |
| Core Usage | 0.99 | 1.00 | 1.00 |
| New Solved | 0/0 | 0/0 | 0/0 |
| Unsatisfiable Unit Equality CNF | Waldmeister 710 | E 1.9 |
|---|---|---|
| Solved/200 | 194/200 | 168/200 |
| Av. CPU Time | 11.75 | 13.08 |
| Solutions | 194/200 | 168/200 |
| μEfficiency | 746 | 633 |
| SOTAC | 0.57 | 0.50 |
| Core Usage | 1.00 | 1.00 |
| New Solved | 0/0 | 0/0 |