"And he was always between jobs. I hardly ever got to see the kids."