A test dataset is a portion of labeled data set aside and used only once, after a model is fully trained, to report its final, honest performance. Unlike the validation dataset, which gets checked repeatedly during training to guide decisions, the test set stays completely untouched until the very end.

‍

‍Why it matters

If a team keeps checking a model against the same dataset over and over while tuning it, that dataset stops being a fair test, the team ends up unintentionally optimizing the model to perform well on that specific data, the same way studying the exact questions on a practice exam stops testing real understanding. The test set exists specifically to avoid this, it's the one honest, uncontaminated measure of how a model will likely perform on data it has genuinely never seen or been adjusted toward.

‍

‍Why it has to stay separate

Because of this, teams treat the test set almost like a locked box during development, using the validation set for every decision made while training, and only touching the test set once, right before reporting final results. If a team finds themselves checking test set performance repeatedly and adjusting the model based on it, they've effectively turned it into a second validation set, and the reported final number stops being trustworthy.