A checkpoint is part of training
TD7’s policy checkpoint is more than a training-recovery file: it is a policy copy selected through training-time assessment. The current learner, data-collection policy, and reported policy may differ.
The source discusses selecting candidates through minimum assessment return. Let be the saved score and the candidate’s minimum after k episodes. Additional episodes cannot increase that minimum:
A candidate already below the threshold can be rejected early. Acceptance still requires the intended assessment length and implementation-specific tie rule; a few favorable trials are insufficient.
Interpreting the minimum
Minimum return favors consistency in the observed sample, but depends on episode count, randomness, and extremes. It is not automatically a CVaR estimator or a guarantee of future worst-case performance. Assessment scheduling is therefore part of the algorithm.
| Record | Role |
|---|---|
| Current actor/critic/encoder | Optimization and recovery |
| Saved policy plus required representation | Consistent action generation |
| Assessment experience | Candidate selection |
| Independent evaluation | Performance and uncertainty reporting |
Save dependent embeddings, normalization, and configuration with the actor. Selection episodes consume budget and must not be reused as independent final-test evidence.
Reproduction records
Record acceptance criteria, maximum assessment length, early rejection, exploration noise, and when the candidate is frozen. Compare current and checkpoint policy curves so selection effects are not attributed entirely to representation learning.
Related reading: TD7/SALE, statistics and tuning, and evaluation.