"And people would catch us talking to him,"