Refactoring CClassifierPyTorch training loop
The current version of CClassifierPyTorch._fit
method reports training statistics every 2000 batches. This is not really useful if you're dealing with small datasets, having fewer batches, moreover, it does not provide any statistic per-epoch.
# print statistics
running_loss += loss.item()
if i % 2000 == 1999: # print every 2000 mini-batches
self.logger.info('[%d, %5d] loss: %.3f' %
(epoch + 1, i + 1, running_loss / 2000))
running_loss = 0.0
Implementation plan
I propose to replace the current training loop with a more standard one, reporting model performance on an epoch-basis.
def _fit(self, x, y):
"""Fit PyTorch model.
Parameters
----------
x : CArray
Array to be used for training with shape (n_samples, n_features).
y : CArray
Array of shape (n_samples,) containing the class labels.
"""
if any([self._optimizer is None,
self._loss is None]):
raise ValueError("Optimizer and loss should both be defined "
"in order to fit the model.")
train_loader = self._data_loader(x, y, batch_size=self._batch_size,
num_workers=self.n_jobs - 1, transform=self._transform_train)
for epoch in range(self._epochs):
train_loss = 0.0
batches = 0
for data in train_loader:
batches += 1
inputs, labels = data
inputs = inputs.to(self._device)
labels = labels.to(self._device)
self._optimizer.zero_grad()
outputs = self._model(inputs)
loss = self._loss(outputs, labels)
loss.backward()
self._optimizer.step()
# accumulate (Simple Moving Average)
train_loss += (1/batches) * (loss.item() - train_loss)
# print statistics
if epoch % 10 == 0:
# Logging
self.logger.info('[epoch: %d] TR loss: %.3f' % (epoch + 1, train_loss))
if self._optimizer_scheduler is not None:
self._optimizer_scheduler.step()
self._trained = True
return self._model
Edited by Marco Melis