"This is hurting him worse than it's hurting us."