SOTA System Performance, part I
SOTA Systems
- A state-of-the-art (SOTA) system has, for every problem,
the performance of the best individual system
- A SOTA system solves every problem solved by some system
- A SOTA system really can be built
- SOTA system performance curves also have a PPP
- Compare performance of SOTA systems over time, on common problems
SOTA System v2.0.0 vs v2.3.0
| SPC
| Problems
| v2.0.0
| v2.3.0
|
| THM_EPR_CNF
| 304
| 301
| 303
|
| THM_EPR_FOF
| 157
| 145
| 150
|
| THM_RFO_NEQ_CNF_HRN
| 327
| 265
| 281
|
| THM_RFO_NEQ_CNF_NHN
| 131
| 105
| 113
|
| THM_RFO_SEQ_CNF_HRN
| 213
| 176
| 195
|
| THM_RFO_SEQ_CNF_NHN
| 314
| 181
| 219
|
| THM_RFO_PEQ_CNF_NUE
| 112
| 69
| 93
|
| THM_RFO_PEQ_CNF_UEQ
| 358
| 309
| 334
|