Discussion
What is the risk of a widely used benchmark?
Answer: B
Repeated evaluation on the same data amounts to fitting the test set through the accumulated choices of the community, so reported gains overstate what would be seen on fresh data. Held out challenge sets and periodic replacement are the usual responses. Hardware, cost and data integrity are practical issues rather than the fundamental one. Selection pressure applied to a fixed target eventually optimises for the target rather than for what it was meant to represent.
No comment is present. Be the first to comment.
Loading…
Post your comment
Please log in or register to post a comment.
Examination Categories
- SSC Exams
- Banking Exams
- Teaching Exams
- Civil Services Exams
- Railways Exams
- Engineering Recruitment Exams
- Defence Exams
- State Government Exams
- Police Exams
- Insurance Exams
- Nursing Exams
- Other Government Exams
- NRA CET
- PG Entrance Exams
- Campus Placement
- MBA Entrance Exams
- CUET and UG Entrance
- Judiciary Exams
- Regulatory Body Exams
- Software Development