The vertical axis is time t (bottom = pure noise, top = clean action), the horizontal axis is action a. The trajectory climbs upward one step at a time.