"Maybe he's the one who needs to be worried."