Machine-translated by Google. Questions are authored in English.
Top

Discussion

What is the role of benchmarks in artificial intelligence research?

  • A.They allow methods to be compared on a common task and data set
  • B.They prove that a method will work on any similar problem
  • C.They replace the need for evaluation on deployment data
  • D.They establish the theoretical limits of what is achievable

Answer: A

A shared benchmark makes results comparable and progress measurable, which has driven much of the field's advance, but a method that tops a benchmark has been shown to do exactly that and no more. Overfitting to a benchmark through repeated use is a recognised hazard. Theoretical limits come from analysis rather than from measurement. A measure that becomes a target ceases to be a good measure, and the field has learned that lesson repeatedly.

No comment is present. Be the first to comment.
Loading…

Post your comment

Please log in or register to post a comment.