The usefulness of the validation error as an indicator for the performance of a hypothesis depends on the size of the validation set.

Experiment with different ML methods and datasets to find out the minimum required size for the validation set.