"I want you to leave this alone for now, especially with the children,"