"Either we get rid of him now or we lose the next election"