This was last updated on 14.6.96, i.e., 2 weeks before the competition. There were some minor differences between what was predicted below and what happened on the day.

Before the competition day

• Entrants must be familiar with the competition design, as explained in the Competition Technical Report.
• The workstations to be used are SPARC 20 machines with 50Mhz processors and 64MB memory, running SunOS 4.1.3. Gnu C v2.6.3, Sun Pro C, Allegro Common Lisp 4.1, Eclipse Prolog, and BinProlog 5.00 are available, and other languages can be supplied by entrants.
• Entrants will be given access to the competition machines on 15/7/96, and must install their ATP system by 26/7/96. Once installed, the ATP system must be handed over to the competition organizers.
• The installed system must be executable by a single command line, with the TPTP file name, the time limit (if required by the entrant), and entrant specified system switches (the same for all theorems) as command line arguments.
• Iff a proof is found, the ATP system must output some distinguished string to stdout (possibly among other output).
• The installed system should be interruptable by SIGXCPU (signal 30). If an ATP system does not respond to SIGXCPU, in the competition the time limit will measured on the wall clock (i.e., less CPU time), so that a SIGKILL can be used.
• The ATP systems will be tested for soundness, using a set of satisfiable problems. Systems which fail this test, or are found to be unsound at any time during or after the competition, will be disqualified.
• The non-proof time of each ATP system will be estimated by measuring the time that it takes to prove a trivial theorem, e.g., p v ~p. The geometric average of the systems' non-proof times will be taken to determine the non-proof time. The current estimate is 1 or 2 seconds.
• The eligible theorems for each competition category will be chosen from the TPTP.
• The tptp2X utility will be used to process the eligible TPTP problems:
  • Each problem will have its clauses and literals randomly reordered (the same ordering for all ATP systems).
  • Equality axioms will be removed as required by each ATP system.
  • Each problem will be written to a disk file in the format required by each ATP system. If the tptp2X utility does not support the format required, then it is the entrant's resposibility to provide a reformating program. The most acceptable form for this program is a tptp2X format module. If the ATP system is in the public domain, then the competition organizers may be able to produce such a module.
• The number of theorems to be used will be determined. The number of theorems is currently unknown because the eligible theorems are not yet known, and the non-proof time is unknown. The current estimate is between 50 and 100 theorems.
• The time limit will be determined. This time limit is currently unknown because the number of theorems is unknown. The current estimate is between 160 and 320 seconds.

On the competition day, before the competition starts

• All entrants must meet at 10:30am for a final briefing.
• The competition theorems will be selected randomly from the eligible theorems in the TPTP.
• Bribes will not be accepted at this stage :-)

On the competition day, during the competition

• Entrants need not attend continuously, but must appear regularly in case there is a problem with their ATP system.
• Each system will be tested on each problem, using a shell script that invokes the system with the TPTP file name, the time limit (if required by the entrant), and entrant specified system switches (the same for all theorems) as command line arguments.
• SIGXCPU will be sent when the CPU time limit is reached (SIGKILL will be used for systems that ignore SIGXCPU, using wall clock timing).
• The following rules will apply to the timing in the competition:
  • If an ATP system erroneously claims that no proof exists for a particular theorem, the runtime will be set to the time limit.
  • If an ATP system produces a system error (e.g., a core dump) for a particular theorem, the runtime will be set to the time limit.
  • The timing will be done in units of 1 second, and the minimal time to find a proof is 1 second.
• Iff a proof is found, the ATP system must output some distinguished string to stdout (possibly among other output). Timing will not stop when the ATP system outputs its distinguished string, but only when the system terminates.
• The ATP systems are not required to return proof objects. However, in the presentation of results, it will be noted which of the systems do produce complete proof objects.
• For every proof found, the proof process must be reproducible by running the system again. If randomness plays a role, reproducibility must still be ensured, e.g., by using pseudo-random numbers and recording the seeds.

On the competition day, after the competition

• The competition results will be announced by the competition panel, at 17:35pm.
• The presentation of the prizes will be made hopefully at the banquet that evening (being negotiated).

After the competition day

• Entrants will be invited to submit contributions to a special edition of the Journal of Automated Reasoning, documenting the CADE-13 ATP System Competition.