| Higher-order Theorems | Satallax 2.8 | LEO‑II 1.6.2 | Satallax‑MaLeS 1.3 | Isabelle 2015 |
|---|---|---|---|---|
| Solved/400 | 273/400 | 198/400 | 288/400 | 272/400 |
| Av. CPU Time | 17.56 | 15.08 | 24.16 | 63.20 |
| Solutions | 271/400 | 193/400 | 0/400 | 0/400 |
| Higher-order Non-theorems | Nitpick 2015 | Refute 2015 | Satallax 2.8 |
|---|---|---|---|
| Solved/200 | 200/200 | 74/200 | 49/200 |
| Av. CPU Time | 7.88 | 24.48 | 0.06 |
| Typed First-order Theorems +*-/ | VampireZ3 1.0 | CVC4 TFA‑1.5 | Vampire 4.0 | Beagle 0.9.22 | SPASS+T 2.2.22 | ZenonArith 0.1.0 | Princess 20150706 | CVC4 1.4‑TFF |
|---|---|---|---|---|---|---|---|---|
| Solved/200 | 172/200 | 163/200 | 161/200 | 132/200 | 107/200 | 60/200 | 143/200 | 131/200 |
| Av. CPU Time | 11.87 | 17.46 | 11.61 | 21.82 | 10.20 | 2.70 | 17.42 | 10.83 |
| Solutions | 172/200 | 163/200 | 161/200 | 132/200 | 107/200 | 60/200 | 0/200 | 0/200 |
| Typed First-order Non-theorems +*-/ | CVC4 TFN‑1.5 | Princess 20150706 | Beagle 0.9.22 |
|---|---|---|---|
| Solved/20 | 10/20 | 6/20 | 6/20 |
| Av. CPU Time | 0.00 | 1.19 | 1.28 |
| First-order Theorems | Vampire 4.0 | Vampire 2.6 | E 1.9.1 | ET 0.2 | CVC4 FOF‑1.5 | iProver 2.0 | leanCoP 2.2 | iProverModulo 0.7‑0.3 | Prover9 1109a | ePrincess 1.0 | Muscadet 4.5 | Geo‑III 2015E |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Solved/400 | 383/400 | 374/400 | 322/400 | 309/400 | 263/400 | 226/400 | 162/400 | 127/400 | 117/400 | 115/400 | 37/400 | 37/400 |
| Av. CPU Time | 12.85 | 15.52 | 23.34 | 24.35 | 38.48 | 23.91 | 48.38 | 32.24 | 39.92 | 54.74 | 6.98 | 19.62 |
| Solutions | 377/400 | 371/400 | 322/400 | 309/400 | 262/400 | 220/400 | 162/400 | 127/400 | 117/400 | 88/400 | 37/400 | 37/400 |
| First-order Non-theorems | Vampire SAT‑4.0 | iProver SAT‑2.0 | iProver SAT‑1.0 | CVC4 FNT‑1.5 | E FNT‑1.9.1 | Geo‑III 2015E |
|---|---|---|---|---|---|---|
| Solved/200 | 195/200 | 163/200 | 135/200 | 71/200 | 51/200 | 39/200 |
| Av. CPU Time | 39.82 | 44.13 | 84.73 | 58.07 | 9.71 | 23.68 |
| Solutions | 195/200 | 163/200 | 134/200 | 71/200 | 51/200 | 39/200 |
| Effectively Propositional CNF | Vampire 4.0 | iProver 0.9 | iProver 2.0 | E 1.9.1 | Geo‑III 2015E |
|---|---|---|---|---|---|
| Solved/200 | 193/200 | 163/200 | 153/200 | 101/200 | 6/200 |
| Av. CPU Time | 27.84 | 29.79 | 36.27 | 11.14 | 59.56 |
| Large Theory Batch Problems | Vampire 4.0‑LTB | MaLARea 0.5 | E 1.9.1‑LTB | iProver 2.0‑LTB |
|---|---|---|---|---|
| Solved/1600 | 1208/1600 | 837/1600 | 799/1600 | 352/1600 |
| Av. WC Time | 6.51 | 8.30 | 7.40 | 13.04 |
| Solutions | 1208/1600 | 837/1600 | 799/1600 | 352/1600 |