"I told you I wouldn't take this job if it got in the way of us,"