"And it's not even his death that haunts me."