"You got to give that up and concentrate on golf."