Batch
- Batch is a unit of data samples that is fed as a grouped matrix to a neural network.
- If we feed 4 data to a neural network, it will return 4 outputs
- During the training, a batch is a unit that we calculate loss, do backpropagation, and make update.
Epoch
- If our training stage used all data samples for once, we call it an epoch.
- Training 10 epochs means that each training data sample was seen by model for 10 times.
- Number of update(iteration) per epoch is decided by [Number of total data sample] / [Batch Size]
[ML] Epoch, Batch size, Iterations 용어 정리
Epoch 모든 훈련 데이터셋을 학습하는 횟수입니다. 주의 사항 너무 많은 Epoch는 overfitting의 위험이 있습니다. Batch Size 훈련 데이터셋 중 몇 개의 데이터를 묶어서 가중치 값을 갱신할 것인지에 대
wooono.tistory.com
Full Batch
- Use entire training data samples for every single iteration
- Pros:
- Calculated gradient is accurate because it sees a lot of data
- Cons:
- Takes more time to calculate a single batch
- Takes more time to update a model
- Demands more memory to put every data on RAM
One Data per Batch
- Use only one training data samples for every iteration
- Pros:
- Each calculation takes short amount of time
- Can make a lot more updates for a limited time
- Cons:
- Calculated loss is only good for the specific data sample
- The direction of gradient can be wrong
Mini Batch
- Use certain amount of samples for every single iteration
Mix Pros & Cons of Full and Other Mini Batch
Comparision
- Small batch size can make more iteration for a given time
- Large batch size provides more accurate gradient to solve the problem
참고할 만한 링크:
[딥러닝] 배치 사이즈(batch size) vs 에포크(epoch) vs 반복(iteration)의 차이
배치 사이즈(batch size) vs 에포크(epoch) vs 반복(iteration)의 차이
losskatsu.github.io
'정리 > Data&AI' 카테고리의 다른 글
| 활성화 함수와 손실 함수 (0) | 2022.10.20 |
|---|---|
| Overfitting, Hyperparameters (0) | 2022.10.04 |
| 손실 함수, 경사 하강법, 역전파 (0) | 2022.09.22 |
| Python에서 OOP하기 (0) | 2022.09.14 |
| Jupyter Notebook관련 이것 저것 (0) | 2022.09.14 |