"Lady Mary could be hours, so there's no point in waiting."