] So what would have been wrong with saying "I fear I could not give you the happiness you deserve and may I please have the ring back?"

This is the one thing he couldn't do. I believe (legal regency scholars please correct me if I'm wrong) that a promise to marry formed a binding contract and if the man chose to break the engagement the woman was legally entitled to keep the ring, if any. She was also able to sue for breach of promise and could be awarded a settlement equivalent to damages in a court of law.
Not sure if a man had same legal right? I can see why woman would have right to pursue legal action as her "reputation" was at stake (less so with male partner) and having recourse to the law would allow her to re-establish this.

This inequity in law as a reflection of social beliefs would also explain why it was considered worse for a man to dump the woman. It stems from women being originally the "chattals" of their father so they could not ask a man to marry them and therefore could not solicit a contract of marriage.

Weddings of any standing were usually preluded by a form of pre-nuptial legal contract drawn up between the man's lawyers and the woman's father's lawyers. These very often protected the woman from a completely unscrupulous husband and allowed her to both inherit and dispose of some property and money independent of her husband. (Of course, unscrupulous husbands developed ways of placing pressure on said wives to sign money over.) In the absence of such a contract e.g. secret engagements, elopement where no wedding resulted, "understandings" where no actual promise was made there was no proof and so one had to rely on the honour of the man involved.