The Concept of Institution Humans have invented the institution of the government, the electoral systems, concepts of rule-of-law. These truths are unlike the physical truths which are physically observable under any instrument and describable using simple relationships: for example the fact that a rock is in the river is objectively true even if humans are extinct. Yet the concept of rock and the concept of river, and the word "rock" and "river" are examples of human institutions, in the sense of shared, often common, knowledge among different humans that help them doing something, and have often unclear but in some way consistent boundaries. These institutions are deemed in unanimity as human created in the modern conception, even though historically some institutions, for example that of kingship was regarded as God chosen. However, while we now know most of the fundamental physical laws that govern the natural world, we have great difficulty in understanding if similar rules in the human interaction sphere exist and how they are spelled out, even given most of humans agree that implicitly these rules exist. Norms, Conventions and Instructions Even if rules exists, they exist in different formats. Some rules, called norms, are general, often implicit, and span over a wide variety of concrete actions that follow that norm. These rules often follow along conventions, which are understood as a similar concept, just without the universability need of the norm, but work within the local community that agreed to it. Examples of conventions are car driving direction, naming conventions whose rule is just make search faster or interpreting red as a stopping signal: they work by merely solving a coordination problem, but don't impose any . Other rules are instead precise, we call them instructions, they tell you exactly how to act given some context. Examples of these rules are the codes that machines execute. They are able to operate 1+1=2 given their physical constraining, and every instruction is interpreted as one unambiguous physical action in terms of transistors, registers etc... The rules that computers understand are fixed and unchangeable from a single computer's point of reference, even a shared virtual language exists and translation to unambiguous instructions for each specific computer is possible.

Rules in Humans and Machines Human legal system has a mix of these rules. Some rules are norms, like principle of fairness, of equality, of human dignity; these don't specify how humans should act case by case, but are generalizable to new cases. Other rules are instead very precise, like the computation of the income tax. Computer rules have been up until recently always consisting in very precise instructions. They needed highly specialized workers, the software engineers, to interact with these systems correctly. Talking to computers in english was not a thing. But recently, from personal experience since December of 2025, language models finally became fluent enough in computer talk and human language intention understanding to be able to allow humans to program in english. Humans can mostly develop a big chunk of the old inscrutable nerdy profession of the software engineer without actually needing to read any line of code. The role of the software engineer is now shifted from being a custodian of the application of this knowledge to intention-gap translator, that is moving the business or client requirements to actually implementable and efficient software. The software engineer still needs broad understanding of the architecture and classical notions in CS, but strictly does not need anymore to understand the workings of the code. This is a pivotal change in humanity.

Language Model Constitutions have a classification problem. The ability to seemingly understand human language opens the avenue for another language to describe and act by the rules: encoded norms. This direction was used by frontier lab's Constitution or Model Spec approaches to encode general morality inside the systems. Yet we don't have clear ways to classify when a model infringes some high-level norm and we don't have guaranteed compliance for all the rules, especially when rules have some exclusive tradeoffs in some instantiation. There is also evidence for impossibility of defending against weird framing (Goldstein et al. 2025). Open-Textured Property of Law However, the nice property of these rules, not present with computer instructions, is the possible generalizability of those rules, they are open-textured (Hart & Green 2012). One can now write rules at different levels of abstraction, and expect a consistent model to behave following such norms. This means if a norm prescribes always good actions under some definition of good, the agent that follows the norm will always and only commit good actions. The problem with the norms, however, is that the notion of good may shift in time, at the point that with enough time the action that was considered good at the time of writing can be considered evil at the time of judgement. As an example consider historical witch trials, while the intention was good -- protecting from spiritual harm -- the actions were later condemned as homicide. Many other cases exist in history. Fixing Distributional Shift The solution was then attempt to fix the intention of the original writer, so that the goodness concept does not shift, and extend from this inferred intention: if the intention is clear, one can supersede or extend the rule by analogy, keeping the same intention. However, it is clear that intention was not clearly available for all laws, and one needs to understand historical context well to balance the whole thinking process of the legislator at the time, information that is often not available in a prompt format, in addition, bloats the simple law with lots and lots of additional information that is difficult to digest efficiently for humans. But this is not true in modernity. It is now possible to track down the whole process of the thinking process and have automated systems extracting information from this data. One thing to be requested by citizens is to have well-known open auditing tools to understand clearly and easily how the legislative process happened so that the future process might be informed on the harms the rule was intended to protect and the rights it was intended to preserve.

A Future of Artificial Societies

Autonomy of Artificial Societies Artificial Agents herald autonomy. They started from simple text engines that provided knowledge, and humans still needed to understand that knowledge to act in the real world. Recently, in the last year, more and more focus has been invested on their autonomy, allowing them to act directly on the digital environment, and soon also in the physical environment (humanoid robots). I don't see anything preventing them into gaining more and more autonomy as the economy will greatly benefit from this high agentic intelligence. How hard is really developing an artificial economy? The next step that will become a reality is then agents that dynamically discover each other, come into a negotiation of some kind of service, and implement the service (Tomasev et al. 2025) and continue to sustain long horizon collaboration. The institutions that allowed humans to develop planetary-level collaboration took millennia to develop, and we think it will be very easy to develop the equivalent in short time to allow artificial economies to develop. Observing the state of blockchain economy and observations as the solipsistic state of artificial intelligence (Trivedi et al. 2026), I currently believe this problem is harder than what is expected. Institutions need first shared knowledge of its existence, and then need legitimacy of the agents participating in it. As Harari calls it, it is a inter-subjective reality that exists purely because of the shared belief of its existence. One Research Agenda: Grounding Multi-Agent Prediction in Intelligent SocietiesTestability: the benefit of the modern technology is that we can actually test social theories out with known assumptions. For the first time in history theories from sociology can be tested with the experimental rigor of the physics. Before, most agents we considered were only rule-based or with very simple models; building an realistic text-based intention based multi-agent system was impossible. Now it's possible to engineer intentionality, social relationships, and test out the predictions of the given theories using intelligent agents. I argue this direction is a precondition that is needed to understand before a more broader large scale systems can be deployed safely. This science would then entail game-theoretic incentive analysis, social/psychological studies of interactions between one agent and other, along with human interactions, institutional design to prevent classical failure mechanism such as tragedy-of-the-commons (Ostrom 1990), engineered law for such society, and any discipline that can have predictive power to systematically understand behaviour of such complex systems. I may call this new field Societal Engineering for the moment, Societal Engineering this branch of engineering that concerns rigorous analysis of social science to create environments where agents, human or not, can interact with known desiderata.

References

[1] Goldstein et al. “Jailbreaking Large Language Models in Infinitely Many Ways” arXiv preprint arXiv:2501.10800 2025

[2] Hart & Green “The Concept of Law” OUP Oxford 2012

[3] Tomasev et al. “Virtual Agent Economies” 2025

[4] Trivedi et al. “Solipsistic Superintelligence Is Unlikely to Be Cooperative” arXiv preprint arXiv:2606.03237 2026

[5] Ostrom “Governing the Commons: The Evolution of Institutions for Collective Action” Cambridge University Press 1990