Evaluating General Purpose ATP Systems
Abstract
A key concern of ATP research is the development of more powerful systems,
capable of solving more difficult problems within the same resource limits.
In order to build more powerful systems, it is important to understand which
systems, and hence which techniques, work well for what types of problems.
This paper deals with the empirical evaluation of general purpose ATP systems,
to determine which systems work well for what types of problems.
This requires also dealing with the issues of assigning ATP problems into
classes that are reasonably homogeneous with respect to the ATP systems that
(attempt to) solve the problems, and assigning ratings to problems based on
their difficulty.