Meta-training and adaptation
Meta-RL trains across to support rapid learning from limited feedback in a new task. State the task family, train/test split, visible context, and adaptation budget. Interpolation, changed dynamics, and entirely new task families are different generalization claims.
| Mechanism | What changes at test time | Required record |
|---|---|---|
| Optimization-based adaptation | Model parameters | Gradient steps, data, initialization |
| Context/task inference | Task representation or conditioning | Available context and inference |
| Recurrent adaptation | Hidden state | Cross-episode retention and resets |
The short source cites Lilian Weng and highlights memory in recurrent Meta-RL. This edition narrows that claim: an RNN is not a requirement for every meta-RL method. Parameter updates and contextual inference can also carry task information.
Exploration and control
An action can earn reward and reveal task dynamics or reward structure. Meta-training may learn both identification and control. Task IDs or privileged inputs that reveal the answer must be disclosed; otherwise fast adaptation can be overstated.
Connections and a proposed comparison
Read RL² for hidden-state adaptation, MetaVLA for demonstration conditioning, and π_RL for parameter updating. Compare zero-shot behavior, context inference, and gradient adaptation within a fixed task family while accounting for cumulative interaction. This is a proposed experiment, not a completed reproduction.