Machine-translated by Google. Questions are authored in English.
Top

Discussion

Why is evaluation harder for open ended tasks than for games?

  • A.The tasks require more computation to attempt at all
  • B.The outputs cannot be recorded and compared afterwards
  • C.Human judges cannot be found to assess the outputs
  • D.There is no rule determining whether the output is correct

Answer: D

A translation or a summary may be good in several incompatible ways and bad in others, so no rule decides it and human judgement, with all its variability and cost, becomes the measure. Automatic proxies correlate imperfectly. Computation, recording and judge availability are practical matters. When the criterion of success is itself contested, progress becomes harder to establish, and that difficulty is inherited by everything built on the benchmark.

No comment is present. Be the first to comment.
Loading…

Post your comment

Please log in or register to post a comment.