Often when someone leaves the Amish church, they'll metaphorically "torch it" as they go leaving behind a smoldering ruin. I've read many accounts of people who have left for various reasons and many - to be sure, not all - paint their decision in stark black and white monotones. For instance, someone leaving, may be fleeing an abusive family member or very overbearing parents. But keep in mind, abusive parents can be found in any religious group. So in many cases, someone isn't fleeing the Amish life or church, they're fleeing a bad parent, person, or situation which could well be found in any church.
It's a subtle distinction, but one that bears examining. There are people who do leave the Amish church based on honest doctrinal differences or even because they are chafing under the church's limitations on education or gender biases. Those people are, indeed, fleeing the Amish church as is their right in this wonderful country. But someone fleeing an abusive parent is often fleeing an abusive parent. I'm not certain that is reflective upon the whole church, as horrible as a situation like that might be.
Mennonite minister Harvey Yoder wrote an excellent piece about the topic of leaving the Amish. It was also reprinted in Mennonite World Review.
The decision to leave one's faith is very nuanced, very individual, with each case being very different and I think Harvey Yoder explains this quite well.