Repeatedly selecting the best settings on the same evaluation data gradually turns that data into part of the development process. The resulting best score can be optimistic.
Scikit-learn distinguishes parameter-search procedures from evaluation of the selected model. Scikit-learn — grid search.
Record the search
For a visual classifier, identify which settings were explored and which data informed the choice. A single final model file does not explain how many comparisons led to it.
Preserve a credible estimate
Use an appropriate held-out or nested evaluation design for the project's purpose. Do not report the best trial's development score as though it were an untouched estimate of future performance. Keep model selection and the final assessment distinguishable in the record.
Source reviewed 25 October 2026. Interpretation and original diagrams: PLUSHIP / Qunhe. Examples are illustrative.
Related reading
Permutation Importance Needs Care With Correlated Features
Image Smoothing Can Hide the Detail an Inspection Needs
Instance Masks Separate Individual Objects in a Product Photograph