Ars D&D.sci: Mysteries of Mana

post by aphyer · 2022-07-09T12:19:36.510Z · LW · GW · 13 comments

Contents

  STORY (skippable)
  DATA & OBJECTIVES
None
14 comments

This is an entry in the 'Dungeons & Data Science' series, a set of puzzles where players are given a dataset to analyze and an objective to pursue using information from that dataset. 

STORY (skippable)

For thousands of years the Order of Athena has preserved magical knowledge, passing it down from masters to apprentices, in a lineage of knowledge unmarred by countless centuries.

Those hidebound old fools!  Your thesis, 'Calculating Levels of Ambient Mana,' could have revolutionized all magic!  You found a way to determine how powerful various types of mana used to power spells were at various times, via bold modern Data Science techniques these antediluvian wizards would never have dreamed of.

Sadly, these senile old mages failed to appreciate your contribution, and have assigned you to a standard apprenticeship under a training mage, intending to have you spend the next fifteen years studying primeval books under his tutelage before they let you call yourself a mage.  Ridiculous!  Modern techniques are far more applicable, and only by looking forward can we hope to become more powerful than the past!  These mages are doomed to only ever pass on the knowledge of the past, but you can learn more and grow stronger than they could ever be!

Of course, first you'll need to convince your new 'master' to let you pass early.  He has  at least agreed to give you a chance to demonstrate the use of your new techniques.  Among these mages' traditions is to hold frequent non-lethal duels between one another, and he has one such duel coming up.  He's offered to allow you to use your new techniques to choose which spells he should prepare for that duel. 

With your new knowledge of mana levels, and an extensive history of duels, you expect you can choose excellent spells and maximize his odds of winning.  Then, perhaps, you'll be able to spend the next fifteen years pursuing your own research instead of learning Latin so that you can read cobwebbed old books!

DATA & OBJECTIVES

I'll aim to post the ruleset and results on July 18th (giving one week and both weekends for players).  If you find yourself wanting extra time, comment below and I can push this deadline back.

As usual, working together is allowed, but for the sake of anyone who wants to work alone, please spoiler parts of your answers (type a '>' followed by a '!' at the start of a line to open a spoiler block) that contain information or questions about the dataset. 

13 comments

Comments sorted by top scores.

comment by simon · 2022-07-10T07:12:05.500Z · LW(p) · GW(p)

Thanks for making this!

Initially I've been looking at the mana data (which I've already figured out according to the story, but is only presented in the data in oracular fashion):

Typically, all mana strengths stay the same for some number of duels, then multiple mana strengths change. The 232501 total duels can be separated into 14475 groups of duels where the mana strengths stay the same in the group. The most common number of duels in such a group is 1 (871 examples) but the number of duels can go as high as 144. Since 1 duel before a change is common, probably 0 is as well meaning we are likely missing a bunch of mana data points which do not have a corresponding duel.

All mana strengths are anticorrelated, but fire and water are especially anticorrelated and so are earth and air.

Graphing fire+water and earth+air (by group number) shows what visually looks like slowly varying sine waves about 120 degrees out of phase. These sums are in the range from 34 to 66. The individual mana types vary much more frequently and look random. So, I guess that the sine waves are deterministic, then an allocation into individual mana types is made in a relatively random manner.

But what about light? It seems symmetrical with the other mana types apart from not being in a group... so a natural hypothesis is that it is part of a group with another non-shown mana type to complete the symmetry, let's call the hypothesized mana type "dark".

Since all mana types average to about 25, I set "dark" to 150 minus the sum of all known mana types. This indeed produced the expected effect of a third slowly varying sine wave like shape for light+dark 120 degrees out of phase with the other two. The statistics for "dark" mana do look similar to those for known types, except that while known types vary from 7 to 53, "dark" as calculated varies from 6 to 54. I am inclined to put this down to rounding, though haven't checked the actual instances of 6 and 54 to verify if this is likely.

Conclusion on mana strengths: Mana strength including a hypothetical "dark" mana type has a total budget of 150 (though the sum will not necessarily be this due to rounding) allocated between fire+water, air+earth and light+dark sine waves 120 degrees out of phase (each sine wave varying from 34 to 66, average 50), and some kind of random-looking allocation occurs between the mana types of each wave, so the individual mana types then vary from 7 to 53 (average 25). Then, different numbers of duels occur at each mana strength data point, which could be 0 resulting in some mana strength data points not showing up.

Replies from: simon
↑ comment by simon · 2022-07-13T04:07:58.034Z · LW(p) · GW(p)

Further remarks:

on the spells in general:

I read GuySrinivasan's comment about offensive and defensive spells, everyone picking at least one offensive and at least on defensive, and their mana affinities being distributed fairly, and confirm this with my own observations. 

Checking how individual spells vary based on individual mana strengths (todo - how do they vary with combinations of strengths), indeed each one varies strongest with the two affinities you'd expect. Also, I noticed that if you form a chain of offensive spells based on common affinities, you form a single loop. The defensive spells on the other hand form two loops of three spells each.

Also, comparing individual offensive spells against individual defensive spells and ignoring mana strengths, it seems that each spell counters two spells and is countered by two spells.

These counters can be represented geometrically if you arrange the spells in a hexagonal pattern based on their mana affinities and the single loop formed by the offensive spells. Each offensive spell does better against the two nearest defensive abilities and worse against the two furthest. See this imgur link. Edit: this relationship is explained by GuySrinivasan who says "It looks like defenses of an element do not protect well against offenses of that element, and defenses of an opposed element protect very well."

Note that opposite mana types are opposite on the hexagonal pattern, so the sine wave component of the mana strength changes can be imagined as a rotation of the hexagon.

countering the opponent's spells part 1:

Now to apply this to the current situation. Unfortunately, the opponent's offensive spells, Flambeau's Flying Fireball and Merinita's Mud Missiles are opposite to one another. So, any defensive spell that counters one will be countered by the other. 

Unlike GuySrinivasan, I am not eager to experiment with 3 offensive - that's a great idea to test once I'm an official mage, but two risky now imo.

So, I looked at the cases where one duelist picked these 2 offensive spells, and found the win rates of each defensive skill against it, both solo (and specifically in the case where there was only one defensive pick by the opponent) and in each combination.

What I found was that, in the case of picking a single defensive skill against the Flambeau's Flying Fireball and Merinita's Mud Missiles combination, Vaporous Vambrace of Verditius was only a little bit behind the top picks (Citadel and Levee) and will likely be the best once mana strengths are taken into account.

Looking at pairs of defensive spells, the best pairs did better, but not enormously so, and this was with whatever offensive spells happened to be picked. Also, the best pairs tend to include weak mana affinities. I expect the freedom to pick two offensive spells to counter the opponent, plus the strong mana affinities of Vambrace, will outweigh the slight winrate advantage of the best pairs of defensive spells.

guessing an answer prematurely:

So, the next step is to similarly check for the best offensive spell pair to counter Solomon's Solar Shield (and then to try to figure out how mana strengths affect all this....). However, I am going to make a preliminary guess before checking this. The two direct counters for SSS are Flambeau's Flying Fireball and Hylenion's Hammer of the Heavens. Fireball is doing okay mana-strength wise and is an obvious choice. Hammer on the other hand has weak light mana affinity. So the question is, should we sacrifice the counter for better mana type? Obvious choices are Rainbow Rays of Rahl (one strong mana and one weak, not countered by SSS) and Merinita's Mud missiles (one strong, one normal, but countered by SSS). However, this latter choice would make the matchup a mirror except for the defensive skill choice. And I already looked at FFF+MMM's win rates against all defensive abilities including SSS! It did worse than VVV, but not by a massive amount. So, our advantage in that case would be the slightly better defensive counter, plus whatever we gain from defensive mana affinity to stronger mana types. It didn't seem that defensive abilities scaled all that well from mana affinity, and the current strength variations are relatively mild, so I am not sure that the advantage would be all that huge.

So while Flying Fireball+Mud Missiles+Vaporous Vambrace would be a safe bet for at least some advantage over the opponent, my preliminary guess will be to assume that abstractapplic is wrong about mana being more important and to go for double counters to SSS, by picking Fireball+Hammer+Vambrace. Likely to change once I do more analysis!

I also note that the intermediate choice of Rainbow Rays of Rahl mentioned above, if picked instead, would cause my answer to match apstractapplic's "new best guess". We'll see if I end up agreeing!

countering the opponent's spells part 2:

Edit: I've checked the offensive pairs against Solar Shield and did end up agreeing with abstractapplic. F+R is the second best pair after F+H against Solar and the difference is likely small enough to be compensated for by R's higher mana strength. Still need to study multiple mana strength interactions and mana strength-counter interactions.

My assumption that being able to optimize two offensive skills against the opponent's defense would bring an advantage seems likely wrong (just one of F or H gets good winrate against O as a solo defense if picked with 2 defensive skills), so I may further look into 2 defense+1 offense.

Current pick:

Flambeau's Flying Fireball + Rainbow Rays of Rahl + Vaporous Vambrace of Verditius, same as abstractapplic.

comment by abstractapplic · 2022-07-10T02:25:22.066Z · LW(p) · GW(p)

My provisional answer is:

Fireball, Levee, Hammer

This is supported by the reasoning that:

Levee (Fire/Earth) does a passably mediocre job protecting against Missiles (Earth/Water) and Fireball (Air/Fire); Fireball (Air/Fire) and Hammer (Light/Air) can both sneak past Solar (Fire/Light) by sharing an element.

And more prosaically by the fact that:

When I filtered the dataset to have Wizard A with the opponent's spell list, the spells which most raised Wizard B's winrate were those three.

However:

I've had a hard time figuring out how to weight "counter the opponent's element choices!" vs "go with what has the most ambient mana!" vs "go with what blocks the opponent's highest-mana attacks!". It's entirely possible that I should replace Hammer with Missiles, Rays or Vambrace; I hope to look into these possibilities later on.

Additionally:

The opponent picked a pretty good set of spells for the conditions in play; as such, I'm seriously questioning whether I can get my master even a >66% winrate.

Replies from: abstractapplic, abstractapplic
↑ comment by abstractapplic · 2022-07-10T11:36:06.894Z · LW(p) · GW(p)

I took an ML-based approach which gave me radically different answers; the machine seems to think that

Matching currently-strong mana types is much more important than countering your opponent's choices.

As such, my new best guess is 

Fireball, Rays, Vambrace

Which should

give my master roughly 2:1 odds in favor.

↑ comment by abstractapplic · 2022-07-10T02:28:42.828Z · LW(p) · GW(p)

Oh, also:

I deduced the existence of Darkness Mana, determined that it almost certainly has a value in the 16-18 range, and then . . . couldn't figure out any clever way to use that information when strategizing. I suspect I'm missing something here.

comment by GuySrinivasan · 2022-07-11T08:06:36.352Z · LW(p) · GW(p)

Half the spells are for offense, half for defense. A wizard never goes to a duel with pure offense or pure defense prepared.

As per other spoilers, Dark is an element, elements come in anti-correlated pairs, and total mana level is 150.

Each spell has elemental affinities that make sense. For example, Fireball works better with high levels of Fire and Air mana and Abyssal works better with high levels of Water and Dark mana. (3C2)*(2^2)=12.

Offensive and defensive spells' affinities are divided "fairly".

Something lets weaker-mana spells win sometimes. It looks like defenses of an element do not protect well against offenses of that element, and defenses of an opposed element protect very well.

I'd like to figure out precisely what protecting well or not well means, but haven't yet. I think the most likely candidates for best spells to prepare are:

(current choice because YOLO Edit: not anymore) Fireball, Mud Missiles, and Rays. No defense. What could go wrong. No one ever does it because in real combat you need defenses but in duels all-out-offense turns out to be technique.

(hope this ends up being right) Fireball, Lava Levee, Vambraces. Hoping counters trump weaknesses.

(would be a bit sad if right) Fireball, Mud Missiles, Abyssal Armor. Just high stats with an obvious weakness.

Replies from: GuySrinivasan
↑ comment by GuySrinivasan · 2022-07-11T16:55:17.377Z · LW(p) · GW(p)

Wondering about:

Are mana levels actually normalized to 150 total, or is that an effect of our measurement/prediction process?

How do duels actually work? They could be as simple as "add up mana values of elements on offense, modify defense mana values according to effectiveness, subtract; compare results; higher wins" or much more complicated like bucketizing mana levels, choosing a random person to attack first, choosing a random attack and a random defense, doing [something like the simple one] to determine damage, switch attackers, repeat until someone has taken X damage, also wizards have native attack/defense/elemental bonuses, also wizards have different HP so X1 and X2 not just X".

Replies from: GuySrinivasan
↑ comment by GuySrinivasan · 2022-07-13T22:23:41.609Z · LW(p) · GW(p)

Seeing if I can figure out how duels work, now. First guess is not correct: that mages pick an offensive spell and defensive spell at random, then calculate damage, and whoever hits for more, wins; damage is determined by for each offensive element, taking the mana level, reducing it by half the mana levels of the defensive elements, adjusting defensive x2 if element opposes and x0 if element is the same, summing the two results.

I need more randomness, maybe; my calibration curve looks decent, but overconfident at both ends. In particular this predicts certain duels are 100% determined, but they do not always work out that way.

How about picking an element at random, too? The problem with this approach is that it doesn't really do much re expected value. I'd expect that it's more like "whoever does X damage first wins" so that repeatedly pinging for 8 can be better than a 50/50 chance of hitting for 13. That may make the difference between Levee and Vambraces for defense, for example, as the opponent's expected mana-levels-getting-through vs Levee is ~1.5 less than vs Vambraces, but Levee is more likely to let some damage through.

Trying the original, but letting the probability of winning be proportional to the respective damage dealt, yields decent calibration but underconfident in the 5-25% and 75-95% ranges. This seems very suggestive of "deal damage until X"; a huge advantage translates to a win, a tiny advantage translates to a tiny advantage, and a moderate advantage, repeated, translates to probably-a-win?

In any case, it seems very likely that the structure has something like "opponents choose an offense vs a defense spell, then ??", so I think I need to abandon my YOLO. :( I would like to go with "maybe higher EV? beats reliably blocking?" and choose Lava Levees for my defenses, with Fireball and Rainbow Rays for offense. I think Fireball is the very clear choice for offense, but Mud Missiles might be better than Rainbow Rays.

Current submission: Fireball, Lava, Rays.

comment by gammagurke · 2022-07-18T01:14:21.259Z · LW(p) · GW(p)

I'm recommending Fireball+Levee+Vambrace.

It seems like it's a turn-based format where wizards don't have control over which of 2 spells of a type(offensive or defensive) they're using, if they have more than 1 prepared. So with knowledge of what our opponent has prepared, it would be best if we could only prepare the best 2 spells (1 of each type) to counter his, and not have to randomly switch to the 2nd best some of the time. With that, we need to find out what is the least bad 2nd-best option for either type.

In defensive spells, there is nothing that counters both our opponent's offensive spells. Since it's presumably random which one they're using, we can't really pick anything that gets countered by either of the opponent's offensive spells, leaving two options. Levee, which can counter some of the water damage on the missiles, and vambrace, which can use the high water mana itself. Not sure which one to prefer here.

In offensive spells, there are the spells that counter our opponent's solar shield, one of them (Fireball) with a reasonably higher mana total than the other, and spells that use the high water mana, one of them matching the shield on light, the other having higher mana with earth. Out of these, I am reasonably confident that Fireball gives the best dpr (or whatever the relevant metric turns out to be).

Taking all that into account, it seems like we get the smallest loss in picking the 2 defensive spells. If we can recommend our master to only prepare 2 spells or have a 100%-0% split instead of a 50%-50% between 2 spells of a type, I'd recommend to cut the Levee and only rely on the Vambrace for defense.

comment by Yonge · 2022-07-17T20:36:05.995Z · LW(p) · GW(p)

I only had time for a very superficial analysis, however hopefully the following isn't too bad:

  • Rainbow Rays of Rahl
  • Alatar's Abyssal Armor
  • Solomon's Solar Shield
comment by Pattern · 2022-07-09T18:16:55.732Z · LW(p) · GW(p)

Spoilering/hiding questions. Interesting.

Do the rules of the wizards' duels change depending on the date?

I'll aim to post the ruleset and results on July 18th (giving one week and both weekends for players).  If you find yourself wanting extra time, comment below and I can push this deadline back.

The dataset might not have enough info for this/rules might not be deep enough, but a wizards duel between analysts, or 'players', also sounds like it could be fun.

Replies from: aphyer
↑ comment by aphyer · 2022-07-09T20:17:36.138Z · LW(p) · GW(p)
  • Based on how stuck-in-the-past all these wizards are,  and how stable the Laws of Magic seem to be, you don't expect that wizard strategies/spell selection/how a duel works/what each spell does should change over time - the only differences you expect between duels are what spells each wizard brought and what the mana levels were.

To clarify, this part of the post is intended as an out-of-game statement to players.  Mana levels are different between duels, but nothing other than mana levels should change between duels - the rules of the duels remain the same.  You don't need to worry about 'what if the rules of duels changed over time', 'what if a better version of this spell was developed halfway through the dataset', etc.

Replies from: Pattern
↑ comment by Pattern · 2022-07-14T01:59:01.079Z · LW(p) · GW(p)

I was thinking

The rules don't change over time, but what if on...the equivalent of the summer solstice, fire spells get +1 fire mana or something. i.e, periodic behavior. Wait, I misread that. I meant more like, rules might be different, say, once every hundred years (anniversary of something important) - like there's more duels that day, so you might have to fight multiple opponents, or something. 

This is a place where people might look at the game flux, and go 'the rules don't change'.