Overview of the Empirical Assessment

The Assessment Period

• From TPTP v6.3.0, released on 28th November 2015
• To TPTP v8.2.0, released on 13th June 2023

Principles

• Analysis over time, not one point in time
• Ability to solve problems
• Changing problem profile
• Comparable solution data
• Resources used to find solutions
• Hardware context

Summary

• There has been progress
• Stronger progress 2015-2018
• Less progress 2019-2022
• More progress 2023
• First solutions
• Progress in individual ATP systems