meta learning

https://people.idsia.ch/~juergen/metalearning.html

Goal: General purpose meta-learning, i.e. discovering learning algorithms that generalize very well.

The objective for human-engineered learning algorithms (LAs) is fixed, given by the algorithm.
For meta learning algorithms, like MetaGenRL, it is parametrized by a neural network:

Meta-learning Spectrum: Less structure $\leftrightarrow$ More structure

More structure: Learned Optimizers (Adam - adapts gradient landscape during learning)
Gradient-based optimization with learned initialization (MAML)
Gradient based with learned objective functions (MetaGenRL, LPG)
black-box optimization with parameter-sharing and symmetries (VSML, SymLA)
Least structure: Black-box (MetaRNN, RL², transformer, GPICL)

Summary from The Sensory Neuron as a Transformer Permutation-Invariant Neural Networks for Reinforcement Learning

Meta-learning RNNs have been proposed to approach the problem of learning the learning rules for a neural network using the reward or error signal, enabling meta-learners to learn to solve problems presented outside of their original training domains. The goals are to enable agents to continually learn from their environments in a single lifetime episode, and to obtain much better data efficiency than conventional learning methods such as SGD. A meta-learned policy that can adapt the weights of a neural network to its inputs during inference time have been proposed in fast weights, associative weights, hypernetworks, and hebbian learning approaches. Recently works (VSML, Meta-Learning Bidirectional Update Rules) combine ideas of self-organization with meta-learning RNNs, and have demonstrated that modular meta-learning RNN systems not only can learn to perform SGD-like learning rules, but can also discover more general learning rules that transfer to classification tasks on unseen datasets.

The variable ratio problem: $\frac{\text{learned variables}{V}_L}{\text{meta variables}{V}_M}$

Meta RNNs are simple, but they have much more meta variables than learned variables ($|V_L|\in O(N),|V_M|\in O(N^2)⟹|V_L|\ll |V_M|$) leading them to be overparametrized, prone to overfit.

Learned learning rules / Fast Weight Networks have $|V_L|\gg |V_M|$, but introduce a lot of complexity in the meta-learning network etc.

→A variable-sharing and sparcity principle can be used to unify these approaches into a simple framework: VSML.

Link to original

… but transformers should improve this, as they have a much bigger effective state size / param ratio, as GPICL shows, right?

General purpose in context learners need diverse task distributions! (duh)

Link to original

If you meta learn on enough tasks, suddenly you can generalize to all tasks.

Link to original

Definition

Each task is defined by its dataset $\mathcal{D}=\{(x_i,y_i){\}}_{i=1}^n$, . The optimal model parameters are:

$$\begin{array}{r}{\theta }^{\ast }=\arg \underset{\theta }{\min }{\mathbb{E}}_{\mathcal{D}\sim p(\mathcal{D})}[{\mathcal{L}}_{\theta }(\mathcal{D})]\end{array}$$

Just like regular SL, but instead of sampling examples form one dataset, we sample datasets (representing different tasks) from a distribution of datasets.
→ The goal is to achieve good performance on out of distribution tasks.
→ Kinda everything is meta-learning in that sense, on a continuous spectrum, but the bar is continually moving higher.

Few-Shot Classification

Training a classifier

We train a classifier $f_{\theta }$ with parameters $\theta$ on a dataset $\mathcal{D}=\{({\mathbf{x}}_i,y_i){\}}_{i=1}^n$ to output the probability of a datapoint belonging to the class $y$ given the feature vector $x$, $P_{\theta }(y|\mathbf{x})$.
The optimal parameters maximize the probability of the true labels:

$$\begin{array}{r}{\theta }^{\ast }=\arg \underset{\theta }{\max }{\mathbb{E}}_{(\mathbf{x},y)\in \mathcal{D}}[P_{\theta }(y|\mathbf{x})])]\end{array}$$

When training with mini-batches $B\subset \mathcal{D}$:

$$\begin{array}{r}{\theta }^{\ast }=\arg \underset{\theta }{\max }{\mathbb{E}}_{B\subset \mathcal{D}}\left[\sum _{(\mathbf{x},y)\in B}{P}_{\theta }(y|\mathbf{x})\right]\end{array}$$

Link to original

In few-shot classification, we sample two disjoint sets from ${\mathcal{D}}^T\subset \mathcal{D}$ each epoch, where $T\subset \mathcal{T}$ is a subset of labels (tasks) of $D$: A support set $S\subset \mathcal{D}$ and a training batch $B\subset \mathcal{D}$.
The support set is part of the model input. The goal is that the model learns to generalize to other datasets, by learning how to learn from the support set.

$$\begin{array}{r}\theta =\arg \underset{\theta }{\max }{\mathbb{E}}_{T\subset \mathcal{T}}{\mathbb{E}}_{(S,B)\subset {\mathcal{D}}^T}\left[\sum _{(\mathbf{x},y)\in B}{P}_{\theta }(y|\mathbf{x},S)\right]\end{array}$$

How would such a network represent never-seen classes? It would need to output embeddings.

Link to original

Viewing meta-learning as learner and meta-learner

The learner $f_{\theta }$ is trained to perform well on a given task.
The meta learner $g_{\varphi }$ learns how to update the learner’s parameters via the support set $S$: ${\theta }^{\prime }=g_{\varphi }(\theta ,S)$
For a classification task, $\theta$ and $\varphi$ are optimized to maximize the objective:

$$\begin{array}{r}\theta ,\varphi =\arg \underset{\theta ,\varphi }{\max }{\mathbb{E}}_{T\subset \mathcal{T}}{\mathbb{E}}_{(S,B)\subset {\mathcal{D}}^T}\left[\sum _{(\mathbf{x},y)\in B}{P}_{g_{\varphi }(\theta ,S)}(y|\mathbf{x})\right]\end{array}$$

Common approaches to meta-learning – how to learn $P(y|\mathbf{x},S)$

model-based meta-learning: $P_{\theta }(y|\mathbf{x},S)=f_{\theta }(\mathbf{x},S)$ –
metric-based meta-learning: $P_{\theta }(y|\mathbf{x},S)={\sum }_{({\mathbf{x}}_i,y_i)}{k}_{\theta }(\mathbf{x},{\mathbf{x}}_{\mathbf{i}})y_i$ – metric learning, $k_{\theta }$ is a similarity kernel
optimization-based meta-learning: $P_{\theta }(y|\mathbf{x},S)=f_{g_{\varphi }(\theta ,S)}(y|\mathbf{x}))$ – gradient descent updates of $\theta$ based on $S$

(Some) Meta-learning approaches touch all fundamental aspect of machine learning at once

data $\{x,y{\}}_i$: obtaining new tasks, …
model $y\approx f_{\theta }(x)$: data efficiency, …
loss $\mathcal{L}(\theta )={\sum }_{i=1}^{N}l(f_{\theta }(x_i),y_i)$: …
optimization ${\theta }^{\ast }=\arg {\min }_{\theta }\mathcal{L}(\theta )$: …

References

(2023) Louis Kirsch - Towards Automating ML Research with general-purpose meta-learners @ UCL DARK

(2021) Oriol Vinyals: Perspective and Frontiers of Meta-Learning

https://lilianweng.github.io/posts/2018-11-30-meta-learning/