"You can't just keep walking in and out of my life like this."