Learning to continually learn (Beaulieu et al., 2020)

A paper titled 'Learning to continually learn' was recently published in ECAI (European Conference on Artificial Intelligence).

The paper talks about a  method for training deep learning models to mitigate the problem of catastrophic forgetting. 

Humans have the ability to learn skill after skill without forgetting previously learned skills. For example, if I learned to play chess last month and volleyball this month, I would still be good at playing chess. The process of learning volleyball wouldn't make me forget chess. Machine learning models on the other hand suffer from catastrophic forgetting i.e., when a machine learning model learns a new task, it does it at the expense of previously learned tasks. 

Some previous work on mitigating catastrophic forgetting have focused on sparse representations for using model capacity more conservatively to leave room for future learning. However, such models have not shown great promise. 

Figure source: 'Meta-Learning Representations for Continual Learning' NeurIPS 2019. 

A 2019 NIPS paper, titled 'Meta-Learning Representations for Continual Learning' by Khurram Javed and Martha White introduced Online Meta Learning. They proposed have two training loops (an outer and inner) for learning a representation learning network  (see figure above) whose weights are fixed during the testing phase. The final few layers of the network (that follow the representation learning network) are the only ones that are trained during testing. By learning a representation learning network, they are able to learn new tasks while not forgetting previously learning tasks, with limited success.

The current paper takes it a step further by training a second neural network that performs neuromodulation on the 1st network. The 2nd network essentially acts like a gate and directs which neurons in the primary neural network should activate for a given input.

Figure source: 'Learning to Continually Learn' ECAI 2020

The cool thing about both these papers (OML and ANML) are that they learn sparse representations even though they were not explicitly trained to learn sparse representations. 

Figure source: 'Learning to Continually Learn' ECAI 2020

Finally they tested their methods on the OMNIGLOT dataset on 600 new sequential classes and found that their network is able to recollect previously learned classes pretty well.