"It was being out of the White House that Katherine couldn't stand."