Overfitting, Hyperparameters

Overfitting

- If the neral network is large and deep enough, the neral network can achieve zero loss for the training dataset.

 - It can memorize the answer to every problem in the dataset.

  - Without a proper understanding of the problem

  - We call it Overfitting

 - It can even memorize random trash dataset.

  - A neural network can reduce the training loss that has no pattern at all.

 

https://developers.google.com/machine-learning/crash-course/generalization/peril-of-overfitting

Generalization: Peril of Overfitting  |  Machine Learning  |  Google Developers

 

Hyperprameters

- Hyperparameters are settings that can affect the training of parameters.

- Some examples:

  Number of layers, Size of each layer, Structure of neural network, Learing rate, Number of training iteration, Loss          function

 

Hyperparameter Overfitting

- The specific hyperparameters that we have used was tested only with a specific test set.

 - We cannot assure that the same setting works well with anoter unseen dataset.

- Therefore, we use another dataset called the validation set.

 - So dataset is split into train / valid / test

 - Instead of checking the accuracy on the test set, we use the validation set.

 - We must not checking the accuracy until the very last stage.

- Parameters are trained with the Training set, and Hyperparameters are selected with the validation set.

 

번외)

머신 러닝에 관한 좋은 레퍼런스 링크

https://developers.google.com/machine-learning/crash-course/ml-intro

머신러닝 소개  |  Machine Learning  |  Google Developers

 

Accuracy in Machine Learning

https://developers.google.com/machine-learning/crash-course/classification/accuracy

분류: 정확성  |  Machine Learning  |  Google Developers