, lesson03

Follow Feb 08, 2020 · 1 min read lesson03
Share this



Unet model

  • similar with convolutional auto-encoder
  • but Unet has connection between symmetry element when max-pooling.

Q. what kind of connection does it have? weight sharing?

freeze/unfreeze, what should I choose as learning rate?


What is fit_one_cycle

recorder.plot_losses() #show you loss depends on time
recorder.plot_lr() #show you lr chnange depends on time (on epoch)

When you visualize these it seems like this. (errors go up and down)

Then why error does not go down at the first time?

  • Why fit_one_cycle is good for learning?

⭐️ When you increase the learning rate, it’s easy to explore whole area and find the place which is not bumpy, so it train faster and can generalize well

what should we do when underfitting

1) train longer 2) train last bit at a lower learning rate 3) decrease learning rate - weight decay - drop out - data-augmentation

Min numerical gradient vs Min loss divided by 10

  • which learning rate is better?

what should I do if run out of memory?

  • Restart the kernel
  • If you want to be able to do more without needing to restart your notebook
    • Learner.destroy

      Free the Learner internals, leaving just an empty shell that consumes no memory If you need to free the memory consumed by the Learner object, call this method. It can also be automatically invoked through Learner.export via its destroy=True argument.

  • Mixed precision training