In plain words
An evaluation tests a model or system against defined criteria. It can measure correctness, usefulness, reliability, safety, cost, or other properties. A benchmark is a standardized collection of tasks or measurements used for comparison.
A closer look
Useful evaluations combine representative inputs with an appropriate way to judge outputs. Some answers can be checked automatically; others need human review or a carefully validated model-based judge. Agent evaluations may inspect the final state of a task as well as the sequence of actions that led there.
Scores depend on the test set, instructions, tools, and scoring method. Training-data contamination, judge errors, and small sample sizes can make results misleading. Repeated trials matter when outputs vary. For a product, tests should reflect the conditions its users actually encounter, including important failure cases.
In practice
For a document assistant, an evaluation might check whether an answer is correct, whether the cited passage supports it, and whether the assistant admits when the document lacks the information.
A useful distinction
One benchmark score is not a universal intelligence score. A model that wins on coding puzzles may still perform poorly on your organization’s documents or on tasks requiring external tools.