"You know, it's time for him to get up and work on his memoirs."