Part2 lesson 10, 04 callbacks | fastai 2019 course -v3

Since fastai instructor people highly emphasized to copy the code from scratch, I’ve been using that approach.
It was fine with Part1, but from Part2, applying same strategy which was trying to copy nbs from blank cell without peeking up original was not enough.
First, there were lots of sub-topics worth to dig in, since we’ve moved to bottom-up phase. And I couldn’t inspect deep inside of each topic(sometimes even don’t know what I was doing) only to replicating the code. It would be okay with part1 session, but seemed insufficient with part2
Second, literally it was too difficult to rely on my own memory. I should have open and close up the Jeremy’s too many times.
So, I thought it would be great if I could draw overall picture of each notebook, making replicating process easier. And it was. (At least with 01_matmul, 02_fully_connected, 02b_initializing, 03_minibatch_training)
So I’d like to share my own supplementary, hoping it could help you to replicate code much easily and (hopefully) grasp the lesson that Jeremy intended.

---

Original notebook link

My replicated Notebook Link

• Initial Setup
• DataBunch / Learner
• CallbackHandler
  • Basic
• Runner
• Notes

Initial Setup

1. Fastai for Colab env

2. Hyper parames
   1. Number of output unit = value max
   2. Number of hidden unit = 50
   3. Number of hidden size = 1
   4. Number of batch size = 64

3. loss function - cross entropy ¹

4. train_ds, valid_ds - instance from Dataset() class

DataBunch / Learner

To make our model polymorphic, we will replace the arguments of fit function.

fit(epochs, model, loss_fun, opt, train_dl, valid_dl) -> fit(const:epoch, learner) ²

1. Make Databunch class
   1. instance initializes with train/valid dataloader and label
      1. default label as None
   2. has methods which can get dataset property
   3. make data instance
2. re-define get_model function
   1. params - databunch, learning rate, hidden layer nods
   2. retur model and optimizer using arguments
3. make class Learner
   1. instance initializes with model, optimizer, loss function, data bunch instance
4. re-define fit function
   1. parameters - epochs, learner object
   2. now it trains using uses learner’s attribute to access data/parameters
   3. return avg loss, acc (* suppose batch size is not variant)

CallbackHandler

(Issue: I’m still trying to grasp overall content, so the contents would be in mess)

* Not satisfying if not is normal. (i.e. satisfying if not condition means something bad happened) ³

Basic

fit(1, learn, cb = CallbackHandler([TestCallback()]))

1. fn one_batch
   1. begin batch, after loss, after backward, after step
2. fn all_batches
   1. do_stop
3. fn fit
   1. begin fit, begin epoch, begin validate, ~ do stop & after epoch, after fit
4. class Callback()
   1. just as mentioned above except do_stop
   2. return input / boolean (I think specifics are not yet implemented) after saving inputs
5. class CallbackHandler()
   1. init - callback, type: list
   2. directly called at training process
   3. Does not save inputs ⁴
6. class TestCallback(Callback)
   1. inherit callback, but interrupt targets, i.e. iteration.

Runner

we will split callback class to clases composed of functions it had.

1. make function which changes camel to snake
2. Callback
   1. set_runner, get attr, change attribute of instance which is name using 1
3. TrainEvalCallback: switch the model in train/valid, count iterations, iteration status
4. TestCallback: run just for 10 iterations

3,4 class inherit Callback, and see the changed name

1. listify function which makes inputs to type list
2. Runner class
   1. get additional callbacks and set them as attributes, merging with TrainEvalCallback
   2. set opt model loss_fun, data method and make those available to be used as property
   3. one_batch, all_batches, fit functions
   4. dunder call that checks the availability of callback
3. AvgStatsCallback
   1. make AvgStats class which save, track and calculate the loss and metrics.
   2. begin epoch, after loss, after epoch - reset, get value, print out the result stats
4. see the results with stats callback
5. see the results with stats callback function

Notes

1. See the document of torch ↩

2. why are factored out arguments problem? ↩

3. See the questions from lesson09 ↩

4. Why handler set res value ? It could just return boolean ↩