I think this is a good question to ask if you are openminded enough to accept that a voluntary society is more moral in theory over a coercive one. From this perspective, one is testing their philosophical position with the real world.
If this is not the case, and one is just throwing out roadblocks in an attempt to confuse the moral issue, I wouldn't bother continuing the conversation.
This seems to be an analogous scenario to the question of voluntaryists escaping society to the woods. I guess one could start a freedom loving society in the wilderness as well. I have considered what it would mean to "head to the hills" in search of true freedom. What could I expect? No access to health care, fuel, culture, fresh fruit and vegetables, gum, coffee, and all of the other great technology living cooperatively within our larger society provides. Now, some people may enjoy this. I don't see them rising to any influencial or dangerous position as they would be unable to resupply or enter into any contracts with the outside world.
I think the question of whether this isolated sub-culture is voluntaryist or a dictatorship is a separate issue. In a free society, nature would select good ideas over bad. Could children suffer? Yes. Do children suffer today? Yes, much more.
In my opinion, a greatly underappreciated effect of living without a government, has got to be this overwhelming sense of frailty within our cosmos. It seems to me that one of the many failings of our perverted "capitalist" system is that greed is not balanced with fear. To remove the concepts of country and government, and more consciously comprehend that one's quality of existence is dependent on the value that that individual brings to society is crucial to aligning perception with reality. Modern society is incomprehensively interconnected. To understand this should influence which hair brained cults or communes one may choose to join. If one does not appreciate the potential dire consequences of failure, one will be more likely to choose the bizarre.
Great questions to ask, unless the goal is to cloud the moral argument.
Sudden burst of heat, burning source of life
Masterful destruction.
Power's not an act, it's understanding truth
Changing my direction.
-Dream Theater