The only way to get married was in church.
The only way to get divorced was by an act of parliament- nothing to do with the church. Even if the state thinks you are single again, the church does not - " those whom God hath joined, let no man put asunder", as the ceremony goes. The church cannot marry you again, if you are already married to someone else.
I said woman because I was feeling cynical. After all, in any case of adultery, which was the normal grounds for divorce, the woman was automatically in the wrong , which explains JA's comment about it always going harder for the woman in cases such as the Rushworth-Bertram one.