Before buying more annotations, examine whether additional data is addressing the observed problem. A learning curve compares performance as the training-set size changes.
Scikit-learn describes learning curves that relate training size to training and validation scores. Scikit-learn — learning curve.
Keep the comparison controlled
Use comparable data selection and evaluation conditions. If one point contains entirely different products or cleaner labels, the curve may reflect more than quantity. Record the sampling basis for each point.
Use it as evidence for a trial
A curve can suggest a useful next experiment, not guarantee a future return on annotation spending. Inspect the errors and label quality alongside it. More data of the wrong kind may leave the practical limitation untouched.
Source reviewed 25 October 2026. Interpretation and original diagrams: PLUSHIP / Qunhe. Examples are illustrative.
Related reading
A Decision Tree Learns Splits From Data Rather Than Writing a Specification
A Crop Can Remove the Evidence Needed for a Vision Decision