Make the research claim measurable
The sources cover manipulation, locomotion, Meta-RL, LLM/VLA post-training, and benchmarks. Organize them through tasks, data, adaptation, execution, and evaluation. State the capability being changed before selecting algorithms.
| Question | Controlled comparison | Main measurements |
|---|---|---|
| Few-shot adaptation | Fixed task family and demonstrations | Test success, extra interaction, adaptation time |
| Reward shaping | Same policy and original task | Original success, failures, learning efficiency |
| RL recovery/generalization | Fixed pretrained checkpoint | ID/OOD, recovery, total supervised and RL data |
| Asynchronous execution | Same model and hardware | Observation age, executed horizon, task time |
| Model-based planning | Fixed environment and planning budget | Multi-step error, actual success, extrapolation |
These are proposed comparisons, not completed experiments.
Language post-training and physical control
Token generation can be treated as sequential decision-making. RLHF, rule-reward RL, and preference optimization supply different signals. DPO-style objectives should not all be labeled online environment-interaction RL. Tool agents additionally need states, rollouts, termination, and feedback.
VLAs connect visual/language conditioning to physical actions. Control can involve continuous or discrete commands, contacts, observation delays, reset costs, and low-level controllers. Ordinary RL is not inherently discrete, and robot policy/PD frequencies are not universal constants.
Data and task splits
Count pretraining, demonstrations, offline replay, simulated interaction, human corrections, and real deployment separately. Meta-RL is not merely hyperparameter tuning: context, hidden state, and parameter adaptation need separate records. Splits over scenes, objects, instructions, tasks, and dynamics determine the scope of generalization.
A reading route
Start with values, policy gradients, and evaluation. Continue through RL², MetaVLA, π_RL, and RL Token, then RTC and DynamicVLA. Use the cross-paper roadmap to check interface compatibility.