"I shall be the judge of that, Prime Minister."