No Peerage for AI

In spite of their flaws, I enjoy grand strategy games enough to spend a lot of time thinking about how to develop them. For someone unfamiliar with the genre, a Grand Strategy Game is a kind of a strategy game that attempts a simulation of society, both deeper than the board game-like rules of the 4X genre and broader than the narrow military scope of RTS or tactical strategy games. This combination of simulation depth and breadth creates both interesting (in theory) strategic choices and (hopefully) dynamic worlds.

While the exact fidelity of the simulation varies, a hallmark of a grand strategy game is an attempt at a very granular simulation - one may say even a more realistic simulation (whether or not it’s actually more realistic is a question beyond the scope of this essay, but that is the appearance cultivated by the paratext of the genre). Stellaris, a sci-fi grand strategy game set in space, offers discrete “POPs” - population groups defined by their species and occupation, with their own ideologies and dietary needs. Production, warfare, and governance are about managing these clusters of citizens, not just manipulating high-level statistics. Building on top of POPs, Stellaris also presents a wide array of government forms that a civilization can attain, starting from a simple liberal representative democracy, to a technocratic dictatorship, a hive mind, some combination of those, or something completely different. These different forms have various unique effects that affect the simulation on a granular level - democracies have elections in which POPs directly participate, technocracies have portions of its leadership POPs designated as “Science Directors” producing research, while hive minds have a completely unique production chain and their POPs ignore happiness mechanics altogether.

Victoria 3, set during the industrial age, goes even further - their POPs, while limited only to the human species, are notably more dynamic in their political demands, and are both shaped by and drive the historical changes of the turbulent era. Peasants that move to urban jobs discover collective action, the bourgeoisie lobby their governments and invest abroad and the free market is simulated to the point where every single starving serf in Siberian wastelands can have their causes of starvation understood in detail, from racial and religious discrimination to a sudden famine brought on by a mismanagement of food exports.

For comparison - a different style of strategy game would not be interested in those social factors. In RTS (Real Time Strategy) games like the Command and Conquer series, the focus is more on immediate tactics, positioning of specific troops. The economic gameplay loop tends to be limited to one resource only (money or something similar), and the overall “simulation” is at most a simulation of pathfinding for an army to its enemy. On the other hand, a 4X game like Civilization attempts to model history not unlike the grand strategy games that arrived later, but their modelling is akin to a board game; for example, a population unit in Civilization is closer to a token or a resource than an agent in the game world. Neither of these genres will attempt to simulate any tradeoffs in the process of taxation - the player gains money from a tax, but only in a grand strategy game does that tax make simulated populations poorer as a result.

These detailed simulations hint at greatness, but unfortunately these games often end up with repetitive and shallow gameplay, providing some enjoyable hours of gameplay that eventually leads to stagnation once one familiarizes themselves with the broad strokes of how the simulation usually resolves. There are different ways you can critique these games - different people find things interesting in different ways. My specific interests in grand strategy games are not shared fully with the rest of the community, and I would like to resist making overly objective claims. Even something as relatively simple as a first person shooter can interest one person with good gunplay and then alienate someone else with a pedestrian storyline. Grand strategy games are sprawling, thus to me, the most interesting part to critique would be its simulationist side; the side that I established earlier to be its unique attribute. Problems in the fundamental simulation bleed into strategic gameplay in general. In fact, this very tension between strategy gameplay and simulation is perhaps one of the more important causes of design problems on both sides.

Among many things that grand strategy inherits from strategy genre as a whole is its notion of a peer opponent - essentially that players in one game should be symmetrical in gameplay. For example, in the 4X Civilization series, every player civilization is at their core the same type of actor with slightly different attributes. This differs from grand strategy games to a degree, where asymmetry in starting positions between actors can get quite extreme. A "Russia" and a "United States" in Civilization, for instance, will have different perks and stats but overall have a generally similar experience, balanced around a competitive equilibrium - their unique perks and quirks are designed to, at least in theory, balance each other out overall. The competitiveness between two countries in Victoria 3, however, is closer to reality - the United States is leagues ahead of Russia for roughly historical reasons (higher literacy rates, a head start at industrialization, a massive ocean separating it from most enemies, the list goes on). Players of similar skill will have outcomes where the USA outranks Russia in almost every game.

This asymmetry, however, is in a transitory state; it still inherits the limitations of the strategy genre. One of the biggest limitations is that these asymmetries have difficult implications for AI design. In most grand strategy games, actors are mostly differentiated by starting position. While some options are unavailable to one or the other (for example, a Russian player is functionally blocked from encountering slavery mechanics, and a USA player is unable to bring back serfdom), the vast majority of options are equally attainable by those actors. Russia may not start with the same military innovations as the USA, but they can research them eventually. They may start with less industrialization, but that can be resolved via a good grasp on the game's industrialization loop. The laws that separate Russia from the USA can be fairly quickly changed out - an autocratic monarchy can become a somewhat liberal republic in a couple of in-game decades.

There are several issues in this arrangement. The first one is that since every country is a potentially equal actor, they must play by the same rules. If you are playing as Russia, then the interactions available to you must also be available to the USA and the dozens of other player countries, whether human or AI. This is a primitive game AI (very different from the LLMs that have captured our zeitgeist), that is also notoriously incompetent. To my knowledge, there is no grand strategy game with an AI implementation on par with a ‘good player' - only, at times, passable. The complexities of the simulation are almost always beyond the grasp of the AI, yet the symmetry maintains the assumption that the simulation can only make sense if the AI uses the simulation in a human-like or competitive way. This results, unfortunately, in simulations that just don't make sense, where AI actors allow the player country to become ahistorically strong, while playing their historic roles so poorly that certain expected events never come to pass - the unification of Germany within its historic borders and timeline is a rare occurrence in Victoria 3. Those that come to the grand strategy genre for strategic gameplay are likewise left unsatisfied - incompetent AI leaves such players dissatisfied and wanting for an engaging challenge. But the design space for complex mechanics is limited - after all, if you create mechanics that require a lot of creative strategic thinking, you will have to find a way to teach an AI actor to do it as well. When the grand strategy games do implement such complex mechanics, they become exclusively used by the players as the AI tends to be incapable of engaging with them at all.

Given that AI symmetry hinders the complexity of the game design, the player has an astoundingly meagre toolbox of options that impact the simulation. While it is true that a large number of population groups and attributes are tracked in the simulation, their net effect tends to be tied to only a handful of higher-level gameplay loops. In Victoria 3, this is the loop of constructing buildings - places where populations can be hired to work a job. Within this loop, the player may build a tooling workshop, a glass factory or a mine. All of those will result in roughly the same political outcomes - less peasantry, less aristocracy which are converted into the workers and the bourgeoisie. Building agrarian buildings makes the agrarian population more prevalent. And that's about it. For historical simulation reasons urban industrial buildings are overall better than agrarian ones, so the already limited player 'choices' end up funneled from two very weak choices into one non-choice - industrialization. The pace of industrialization does depend on the exact prices of construction resources and taxation, but these factors are also not particularly dynamic either.

There are other ways that influence the simulation. Events allow the player to pick between one of several outcomes, which usually aren't highly impactful and are fairly random. Another is the process of law passing, which reflects changes in a country’s political makeup with new buildings and jobs being opened. The complexity of ideological factors that the game tracks - racism, political opinions, radicalization - all of these have to be made simple enough for the AI and thus they become individually meaningless.

For instance, laws relating to cultural discrimination, for all their innumerable modifiers that they produce, can be reduced in terms of actual decisions to "which broad cultural groups experience racism and which broad cultural groups will get angry about it." Taxation laws are likewise broad brushstrokes that cannot capture the actual historic nuances of conflicts that flare up as a result of even the smallest changes to the system. Instead, they are reduced simply to “do we tax the peasants, the workers or the rich.” This is an acceptable decision tree for a simpler strategy game and a waste of potential for a simulation as supposedly deep as Victoria 3.

There is a deeper conceptual problem with this arrangement. AI may be incompetent, but it’s also unclear AI should act in the first place. In history, actors make bad decisions that no competent player would do, and yet there is always an assumption that the AI should be as competent and challenging as a human player, an assumption that I’ve come to call the “Peer AI Design Paradigm” or simply the “Peer AI.” This leaves the design of these games in a perpetually difficult contradiction - an AI that acts in a way that seems historically realistic (frequently making mistakes, unable to act on information it has due to that information being acquired in a gamey fashion, etc), versus an AI that acts like a ruthless game player (keenly aware of game’s more gamey aspects and exploiting them fully while also acting with the knowledge of certain historic events in advance that would be a surprise to actors within the setting). This dichotomy results in both a deficiency in gameplay quality and a deficiency in simulation, which despite its flaws is so ingrained into the core assumptions of the genre that counterexamples are almost unheard of beyond highly experimental and controversial fan-made modifications.

Peer AI has profoundly negative implications on game design of grand strategy games. A common sense notion in strategy game communities is that "espionage mechanics" are hard or impossible to implement well. Besides pure intelligence gathering, espionage mechanics hit the player with consequences of something they had no idea about and that they could not prepare for. To have control taken away tends to be unfun, so the most common way "espionage mechanics" are implemented end up being either annoying to the players or, more commonly, are so ineffective that they are de-facto non-existent. As a result, actions that should sound fun, like fomenting revolts, assassinating undesired leaders and messing with foreign political groups are neutered in grand strategy games.

There are many reasons for why grand strategy simulations remain seemingly complex yet weak. Despite the fact that simulationism is a major distinctive feature of the genre, it isn't the only feature. Players want different things after all, and judging by how well the genre sells, there's a general satisfaction with the overall package. Creating a fully fledged simulation for everything I've discussed prior takes up time and money - designer time, programmer time, artist time and then testing time. The design restrictions from the Peer AI paradigm increases how expensive implementing those simulations may be, and closes the door on some of the more interesting mechanics. Perhaps it is time for the next generation of the grand strategy games to give no Peerage to AI?