ML Researcher Questions Validity of Validation Loss Overfitting Answer

There’s only one answer: “ INSUFFICIENT DATA FOR A MEANINGFUL ANSWER ”

at the end of which the candidate shall reject the interviewer’s question, who will then in disbelief about their own mental model of the universe will go onto a 7 year long spiritual journey on a remote Greek island and then the Himalayas, to only come back with a single answer

INSUFFICIENT DATA FOR A MEANINGFUL ANSWER

@prajdabre Not great, you should expect the training loss to generally be lower than the validation loss. Only when the gap between them increases or is already quite big can you assume there is some overfitting

@prajdabre It's the correct answer but could be explained more deeply

Training is divided into 3 stages Early , good fit(both are decreasing) , overfit

After a certain time training loss continues to decrease while validation loss stops improving and begins to increase

@prajdabre its simply a wrong answer?

@prajdabre when the model is giving good results ( or low error ) on training data, but much higher error on test set comparitively.

@prajdabre instead of comparing the value of losses, we should compare the trends. If the validation loss is increasing but training loss is decreasing , then it is overfitting

@prajdabre Is that answer from a candidate? Make them into a PM, you know, to limit the damage.

@prajdabre half right. val > train is normal for basically every model. overfitting is val loss reversing and climbing while train keeps dropping