A meta-RL model is trained over a distribution of MDPs, and at test time, it is able to learn to solve a new task quickly. The goal of meta-RL is ambitious, taking one step further towards general algorithms.

_Meta Reinforcement Learning_, in short, is to do [meta-learning](https://lilianweng.github.io/posts/2018-11-30-meta-learning/) in the field of [reinforcement learning](https://lilianweng.github.io/posts/2018-02-19-rl-overview/). Usually the train and test tasks are different but drawn from the same family of problems; i.e., experiments in the papers included multi-armed bandit with different reward probabilities, mazes with different layouts, **same robots but with different physical parameters in simulator**, and many others.


## Key Components

There are three key components in Meta-RL:

> ⭐ **A Model with Memory**  
> A recurrent neural network maintains a hidden state. Thus, it could acquire and memorize the knowledge about the current task by updating the hidden state during rollouts. Without memory, meta-RL would not work.

> ⭐ **Meta-learning Algorithm**  
> A meta-learning algorithm refers to how we can update the model weights to optimize for the purpose of solving an unseen task fast at test time. In both Meta-RL and RL^2 papers, the meta-learning algorithm is the ordinary gradient descent update of LSTM with hidden state reset between a switch of MDPs.

> ⭐ **A Distribution of MDPs**  
> While the agent is exposed to a variety of environments and tasks during training, it has to learn how to adapt to different MDPs.



```
@article{weng2019metaRL,
  title   = "Meta Reinforcement Learning",
  author  = "Weng, Lilian",
  journal = "lilianweng.github.io",
  year    = "2019",
  url     = "https://lilianweng.github.io/posts/2019-06-23-meta-rl/"
}

```