training the bear classifier with different dataset sizes

vishal bakshi
september 5, 2020

jeremy howard urges students (and practitioners) not to overestimate the amount of data that is needed to train a high performing model given that we can use transfer learning. i want to experiment with that concept by training an image classifier for the following number of images and compare the results:

  • 10
  • 20
  • 50
  • 100
In [29]:
#!tar chvfz notebook.tar.gz *
In [30]:
from utils import *
from fastai.vision.all import *
from fastai.vision.widgets import *

first, i'll check to see that i placed the images into the correct folders

In [2]:
Image.open('bears2/grizzly/00000000.jpg').to_thumb(128,128)
Out[2]:
In [3]:
Image.open('bears2/black/00000000.jpg').to_thumb(128,128)
Out[3]:
In [4]:
Image.open('bears2/teddy/00000000.jpg').to_thumb(128,128)
Out[4]:
In [71]:
bears = DataBlock(
    blocks=(ImageBlock, CategoryBlock),
    get_items=get_image_files,
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    get_y=parent_label,
    item_tfms=Resize(128))

before i move on, i want to understand the DataBlock a bit more. so far, it's an empty container with two blocks (ImageBlock and CategoryBlock), a function to get items to store in the DataBlock (get_image_files), a function to split the items into a training and validation set RandomSplitter, a function to get the dependent variable y (parent_label) and a transform to apply to each item (Resize())

In [6]:
path = Path('bears2')
print('# of total images:',len(bears.get_items(path)))
print('# of training images:',len(bears.splitter(bears.get_items(path))[0]))
print('# of validation images:',len(bears.splitter(bears.get_items(path))[1]))
print('image labels: ',
      bears.get_y(bears.get_items(path)[0]),",",
      bears.get_y(bears.get_items(path)[10]),",",
      bears.get_y(bears.get_items(path)[20]))
print('image after item_tfms is applied to it',bears.item_tfms[1](bears.datasets(path)[0]))
plt.imshow(bears.item_tfms[1](bears.datasets(path)[0])[0])

# of total images: 30
# of training images: 24
# of validation images: 6
image labels:  black , grizzly , teddy
image after item_tfms is applied to it (PILImage mode=RGB size=128x128, TensorCategory(0))
Out[6]:
<matplotlib.image.AxesImage at 0x7f99aa002fa0>

10 images per class

In [22]:
dls = bears.dataloaders(path, batch_size=2)
In [23]:
dls.valid.show_batch(max_n=5,nrows=1)
In [24]:
learn = cnn_learner(dls, resnet18, metrics=error_rate)
learn.fine_tune(4)
epoch train_loss valid_loss error_rate time
0 2.640413 2.023208 0.500000 00:02
epoch train_loss valid_loss error_rate time
0 1.604320 1.237319 0.500000 00:02
1 1.296324 0.575800 0.333333 00:02
2 0.932348 0.318487 0.333333 00:02
3 1.005771 0.664139 0.333333 00:02
In [25]:
interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix()
In [26]:
interp.plot_top_losses(4, nrows=2)