"That you wouldn't be able to handle the kids leaving"