"If you don't want to spend time with me, Peter, that's fine."