Social Engineering Should Be Tested First

The best intentioned reformers often make things even worse.  But so would anyone trying to solve massive, complicated problems on the first try. It’s actually more surprising anything ever goes right.
It amazes me looking back over history to see how reformers and revolutionaries try to apply their ideologies without ever having tested them. Imagine a tech company releasing a new device without extensively testing it first or a computer programmer writing code for an entire program without ever trying to compile it. Ridiculous, yet that’s what people try to do all the time. Too often the result is disaster.
The higher castes have greater agency through which they deal with greater responsibilities. They can’t just say “oops” when there’s a big logistical screwup and a couple million people starve to death.
Any responsible sentient being in power naturally has a system to test new ways of organization before implementing them on a large scale.
Observing differences between local governments and the study of history provides lots of fertile material for hypotheses, but the devil is in the details.

There would have to be some sort of R and D department for trying out new social technologies. Perhaps there could even be reality shows of a sort where in the first round groups of maybe 150 or so live under a hypothetical social model then those groups that make it past the elimination get expanded up to 1000 and so on. There would be rules to keep it ethical. People who “die” in the experiment would just be “voted off the island” and sent home. Not being “real” would of course distort the data, but perhaps money or other incentives could make the results worthwhile. Someone who “dies” might lose all their prize money, representing a total loss.

Or to make this simpler maybe a reform simply gets tried first in a small town or a single city first and upvoted or nexted based on results. Perhaps there might be an actual experimental province set aside with discrete zones. Those who chose to live there would simply vote with their feet. In the absence of any Berlin walls, it would quickly become evident which zones people like and which they avoid and what type of people or demographics prefer different systems. Of course, the experimental province might give unrepresentative data if they attracted outliers of the population, but it could be a start. Not to mention, there would probably have to be incentives to get people to choose to live in experimental land.  Perhaps they’d sign contracts to stick around in experimentland for a year or two or else they lose all their bonuses.
As enough information was amassed from real life experiments maybe computer simulations would become more effective at projecting results and maybe programs could be written to project hypotheses for ideal social organizations taking every aspect of human nature into account that maximize both raw competitiveness and creativity/adaptability to new stressors.

Throughout history, groups have settled on something that works for the time period and then try to perpetuate it ad nauseum across milennia.  Talmudic Judaism was a brilliant way to coordinate a particular Semite tribe over 2000 years ago.  Islam turned out to be the right solution for quarreling Arab city states about 1300 years ago.  But one of the things we immediately notice is that all these systems buy a professional suite of anti-virus software to prevent change to that successful formula, even if it’s a thousand years later.
Sadly, social technologies tend to stagnate because they only ascend to apotheosis in the first place because they have serious protection against change.
The challenge before us then is how to design a society to be both resilient and highly adaptable to new stressors, so that when the next big asteroid hits, we aren’t among the dinosaurs.