"He thought we were mad to live here"