"It wasn't Robert who had the affair."