📊 Expected value and Rational choice in games

What is expected value?

Expected value (EV) is the average outcome of a random event, weighted by probabilities. The formula for a discrete random variable is: \[ \mathbb{E}[X] = \sum_{i} p_i \cdot x_i \] where \(x_i\) are the possible outcomes and \(p_i\) are their respective probabilities. In games, EV allows one to compare different strategies and choose the one that yields the best result on average, even if a loss is possible in any individual attempt.

Example: In a turn-based strategy game, a soldier attacks with a 70% chance to hit, dealing 10 damage. The EV of the damage is \(0.7 \times 10 = 7\). If a grenade that guarantees 6 damage is available, then the attack’s EV is higher (7 > 6), making it the better choice (provided there are no other risks (such as return fire)).

Decision-making based on expected value

In XCOM, Darkest Dungeon, and many RPGs, every action has a probabilistic outcome. Expected damage is calculated as follows: \[ \text{EV} = (\text{hit chance}) \times (\text{damage}) + (\text{crit chance}) \times (\text{crit damage}) \] However, EV is not the only criterion: when playing for survival, variance (risk) matters. But for the purpose of maximizing points or resources, the expected value is the gold standard.

An example from card games (Gwent, Hearthstone): A player has a 2/2 creature, while their opponent has a 3/3. The player can cast a "Fireball" spell (which has a 50% chance to deal 4 damage and a 50% chance to deal 1 damage). The EV is \(0.5 \times 4 + 0.5 \times 1 = 2.5\) damage. On average, this is enough to kill the enemy (who has 3 health). However, in 50% of cases, the player will fail to secure the kill, and in those instances, will lose the game. If risking losing the entire match based on a single unfavorable outcome, it might be better to choose a guaranteed action with a lower EV.

Expected damage and critical hit

A critical hit is a specific type of damage modifier. Let a base hit deal \(D\) damage, let the critical hit chance be \(p_{\text{crit}}\), and let the critical multiplier be \(k\) (typically 2, less frequently 1.5 or 3). Then, the expected damage per attack is: \[ \text{EV} = (1 - p_{\text{crit}}) \cdot D + p_{\text{crit}} \cdot (k \cdot D) = D \cdot \big(1 + p_{\text{crit}}(k - 1)\big) \] For example, with \(D=100\), \(p_{\text{crit}}=0.2\), and \(k=2\): \(\text{EV} = 100 \cdot (1 + 0.2) = 120\). In other words, critical hits increase the average damage by 20%.

Comparing two rings: +5% critical hit chance (with \(k=2\)) provides an increase in EV equal to \(D \cdot 0.05\). A ring with +5 Strength (which increases base damage by 5) provides an increase of \(5 \cdot (1 + p_{\text{crit}})\). With \(p_{\text{crit}}=0.2\), the Strength bonus amounts to 6, while the critical hit chance bonus is \(0.05 \times D\). If \(D\) is sufficiently large (for example, \(D \ge 120\), then the +5% critical hit chance is more beneficial. Players often underestimate the linear growth in EV that comes from critical hits.

Interactive demonstration: crits and EV accumulation

Click the button several times to simulate a series of hits. The current arithmetic mean of the damage is displayed on the right. It tends toward a theoretical EV of 120. Note that even after 50 hits, the average may still fluctuate, but with each additional hit, it draws closer to the expected value (the law of large numbers).

Expected value in games with loot and gacha mechanics

In games like Genshin Impact, Brawl Stars or collectible card games, the concept of the "expected number of attempts to obtain a legendary item" is frequently used. If the drop rate for a legendary item is 1%, it will take an average of 100 chest openings to get one (though the actual number can vary significantly). EV allows for budget planning: if a single opening costs $10, the expected cost of a legendary item is $1,000. This helps players (and developers) set prices and gauge "luck".

Another example is the "roulette" mechanic found in games like Borderlands or Destiny. Weapons come with randomized attributes (such as the barrel, magazine or stock) that affect their performance. The expected value for DPS (damage per second) accounts not only for base damage but also for factors like the probability of missing, the chance of landing a critical hit and the rate of fire. The formula is: \[ \text{EV}_{\text{DPS}} = \text{fire rate} \times \big( \text{hit rate} \times \text{EV}_{\text{hit damage}} \big) \]

When is the expected value misleading?

St. Petersburg paradox: a coin-toss game might offer infinite EV, yet no one would pay a large sum to participate. This is because the utility of money diminishes and a real person cannot afford to risk enormous amounts. In games, there is also the risk of total failure (or "bankruptcy") where a single unfortunate move can destroy a character (for instance, in XCOM’s Ironman mode). In such scenarios, players often apply the Kelly criterion or simply steer clear of high-variance actions, even when the EV is slightly higher than that of the alternatives.

Gambler's fallacy and underestimation of the tails of distribution are common pitfalls. It is always useful to evaluate not just the average outcome, but also the probability of an extremely poor result.