It seems that for the long history of humanity, a big part of our moral abilities was crafted out evolutionarily (Tomasello 2016,Binmore 1994).
Yet the systems that we build don’t have morality in the human sense. They are mostly compliant with the code that we engineer into it, and don’t have a choice between cooperation and defection. They cooperate in the only ways humans want them to do, which is with the precision and unambiguity that computer code inherently has, by construction. It’s the first time in human history that we cannot predict before deployment what kind of decisions a computer system can make in any given scenario. Yet their broad capability in most of the tasks that historically have always required deep human intelligence is indeed something new that has great value, and VCs, investors and the broader society have intuitively grasped this importance. This begs the question of what and how best to extract this value from these models, and what use cases we expect them to solve.

LLM Expectations

The thing we need to discover is:
What are we expecting LLM agents to do? In other words, what are the social norms that we appoint to such systems? The general answer would be just “I expect the system to work”, yet what it “works” on is still something that we, as humanity, are discovering. It depends on what they actually can do, and we are very quickly expanding their capabilities and the ability to evaluate them in doing so.
There are many ways that we expect them to work.

In the AI era with the AGI in view, it looks like we are expecting them to do everything a human can do, plus something more informed by the mastodontic amount of knowledge that they have. We are expecting them to improve more and more and get more autonomy than ever, and especially have some autonomous economic output. This means communicating and coordinating with other agents, with humans, and creating systems to access computer systems and use them well enough to reach some specific goal.

Another expectation is following our command; this is the general problem of AI alignment (Gabriel 2020). We expect them to comply with our requests, even if our request is under-specified, incomplete in the Huang et al. 2026 sense. And as a society, we also expect them to refuse harmful requests that could help the user gather information to enact dangerous acts on society.

In more specific usages, particularly in workflows, we expect them to solve a very well-defined, limited use case, and only be specialized to do that. We thus incur in some specialisation-generalization trade-off: how much can we use an off-the-shelf foundation model to solve our problem, or do we need more specialized workflows, maybe some test time-based methods (Hübotter et al. 2026)?

AI Social Norms

Social norms anticipate competition in potentially disruptive situations and make it clear how individuals must behave in such situations in order to cooperate. ~Tomasello 2016

As a society, we are discovering what to expect from these models. The question of what norms to imbue models with seems still to be far, but eventually, it is one that needs to be engaged with. As models become more autonomous, they will get into the cooperative-competitive dilemmas that humans have implicitly struggled with in the past few millennia, especially if we have at some point autonomous economies à la Tomašev et al. 2025.
For some people, the process of discovering the expectations is quite dreadful. Tech leaders are talking about AI getting the job of most entry-level jobs. Some jobs were also thought to be among the most secure jobs from computer automation, such as lawyers, economists and other white-collar jobs.

I don’t have a clear answer on how these norms should be defined and what we should build into them. It seems that lots of people have quite different ideas of what AI should do, and I don’t have a clear prescriptive framework to think about this topic. But the reality looks like the tools are already shaping how we are relating to technology, and as social media, also shaping how we will relate to other people. So giving some thoughts about this topic is an important one in this era.

References

[1] Tomasello “A Natural History of Human Morality” Harvard University Press 2016

[2] Binmore “Game Theory and the Social Contract: Just Playing” MIT Press 1994

[3] Gabriel “Artificial Intelligence, Values, and Alignment” Minds and Machines Vol. 30(3), pp. 411–437 2020

[4] Huang et al. “Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AI” 2026

[5] Hübotter et al. “Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models” arXiv preprint arXiv:2509.24510 2026

[6] Tomašev et al. “Distributional AGI Safety” arXiv preprint arXiv:2512.16856 2025