SALE versus the complete TD7 system
The source links For SALE to arXiv:2306.02451 and NeurIPS 2023. SALE is a state–action representation mechanism; TD7 combines it with TD3, checkpoints, replay, and stability choices. Whole-system gains cannot all be attributed to the encoder.
Representation objective
Write and . A conceptual objective predicts the next-state embedding:
Actual normalization, stop-gradient, fixed/target representation versions, and schedules matter. Critics can retain original states and actions. Low-dimensional observations can still involve complex dynamics, but embedding prediction alone does not establish a homomorphism or optimal control.
| Component | Purpose | Required comparison |
|---|---|---|
| Decoupled representation | Keep value losses from arbitrarily reshaping the encoder | End-to-end and encoder-free controls |
| Fixed embeddings/normalization | Stabilize targets and scales | Schedules and gradient boundaries |
| Replay/loss design including LAP | Change sample use and weighting | Separate from representation ablations |
| Policy checkpoints | Select a training-time policy copy | Assessment versus independent evaluation |
| Value clipping | Limit selected extrapolated values | Thresholds and underestimation |
| Offline BC term | Restrict fixed-data policy drift | Separate offline and online protocols |
Evidence and scope
The long source describes online MuJoCo, offline D4RL, component ablations, and runtime. Its score tables refer to reported paper configurations; no PDF was uploaded or independently checked here, so they are not presented as new verification.
The central evidence concerns low-dimensional continuous control. Visual inputs, multitask VLAs, dynamic robots, and real deployment require further experiments. Account for compute relative to base TD3.
Related reading: checkpoint mechanisms, abstraction, and extrapolation.