"But l don't want that to be the reason we're not together."