Most jurisdicitons order mediation before allowing litigation. It can work if the parties are relatively close in what they want. My mediation was a complete joke, given that I have a narcissistic bully of an ex, who had no intention of compromising. It was actually the single most stressful, dramatic day of the entire two years of the custody case, due in no small part to having a condescending (to me) mediator.
Generally a mediation consists of the parties, their attorneys and the mediator meeting and discussing division of assets, custody, visitation, child support and related matters. I chose to request that we caucus, which meant that ex and his attorney were in one room and my attorney and I were in another room, with the mediator going back and forth between the rooms relating the various offers each side was making. I chose this because my ex had a history of being extremely confrontational with me several times in several public settings. This is not the norm, however. Most mediations occur with all the parties in the same room.
Mediation left a sour taste in my mouth, but I know that that had everything to do with the personalities involved. Some people find it very helpful. It's also possible to reach a partial agreement in mediation, wherein the parties agree on some of the issues at hand, but still wind up in court to settle the remaining issues.