D&D.Sci: Whom Shall You Call? [Evaluation and Ruleset]

post by abstractapplic · 2024-07-17T22:34:25.111Z · LW · GW · 5 comments

Contents

  Ruleset
    Ghost Generation
      Traits
      Categories
    Exorcism Prices
      Phantom Pummelers
      Wraith Wranglers
      Spectre Slayers
      Entity Eliminators
      Demon Destroyers
      Mundanifying Mystics
  Strategy 
  Reflections 
  Scheduling
None
5 comments

This is a followup to the D&D.Sci post [LW · GW]I made twelve days ago; if you haven’t already read it, you should do so now before spoiling yourself.

Here is the web interactive I built to let you evaluate your solution; below is an explanation of the rules used to generate the dataset (my full generation code is available here, in case you’re curious about details I omitted). You’ll probably want to test your answer before reading any further.


Ruleset

(Note: to make writing this easier, I’m using standard D&D dice notation, in which “3+4d8” means “roll four eight-sided dice, sum the results, then add three”.)

Ghost Generation

Traits

Ghosts have nine traits which can affect the price of an exorcism: Intellect, Corporeality, Sliminess, Hostility, Grotesqueness, Deviousness, Perversity, Paradoxicality, and Unholiness, each a positive integer with a max of 100. Of these, the PPEK only measures the first five.

For the typical ghost (kind of; see next section), these traits are generated per the below table:

TraitGenerator
Intellect1d80 + min(1d20, 1d20)
Corporeality1d60 + 1d20 + min(1d20, 1d20)
Sliminess[Corporeality]/5 + 1d60 + min(1d20, 1d20)
Hostility1d80 + min(1d10, 1d10) + 1d10
Grotesquenessmin(1d20, [Hostility]/5) + [Corporeality]/10 + 1d10 + 1d60
Deviousness[Intellect]/10 + 1d10 + 1d80
Perversity1d60 + min(1d40, 1d40)
Paradoxicality1d100
Unholiness1d60 + 1d40

Categories

A ghost is either a Wisp, Spirit, or Horror. Wisps, when generating each trait, roll three times and pick the lowest result; Spirits roll once; Horrors roll three times and pick the highest.

43.3% of ghosts are Wisps, 19.3% are Spirits, and 37.4% are Horrors.

Exorcism Prices

The cost of an exorcism varies randomly in a triangular distribution from 50% to 150% of the mean.

Phantom Pummelers

The Pummelers, as the name suggests, tend to solve problems with their fists. They are aided slightly by Corporeality (they have ways to drag enemies into the physical plane, but find life somewhat simpler when they get to skip that step), hindered greatly by Sliminess, and impeded by Perversity and Unholiness.

Wraith Wranglers

The Wranglers believe in talking things out, seeing their task as gently helping ghosts to pass on. Their ideal client has high Intellect, low Deviousness, low Paradoxicality, and - most importantly - low Hostility.

Spectre Slayers

The Slayers are overwhelmingly powerful but hamstrung by conscience. High Intellect makes things much harder for them; Grotesque and Hostile targets make their job slightly easier to bear; Deviousness helps their victims figure out how best to play on their sympathies.

Entity Eliminators

The Eliminators are capable, but bad at finishing blows (to an extent which leads the Slayers to accuse them of toying with their prey). Sliminess, Deviousness, Paradoxicality and above all Perversity enable their enemies to take advantage of this deficiency.

Demon Destroyers

The Destroyers are hard to defy or reason with, but easy to make uncomfortable. Grotesqueness is a ghost’s best defense against them; Hostility and Perversity also make them squeamish; Unholiness, by contrast, places them on reassuringly familiar ground.

Mundanifying Mystics

The Mystics mostly ignore their quarry, endlessly and expensively recycling their reliable rituals to purge occult influences. Unholiness gives said rituals something to latch onto, and makes their task somewhat easier; no other trait affects them.

Strategy 

The constraints on your decisionmaking bind less stringently than it might seem. The Eliminators – while decent all-rounders – are never the best pick, so you can freely choose the Slayers over them; there are only three ghosts for which the Pummelers are an optimal choice, so their limitation hangs loose; the Destroyers are comfortably worth their call-out fee when used right. As such, you’re actually free to pick the best exorcist group for each ghost, with average prices for each (rounded to two decimal places) shown in the table below:

GhostE(MM)E(SS)E(PP)E(EE)E(DD)E(WW)
A3396.811901.262056.662068.502693.442502.37
B3384.412025.802564.992216.153002.251628.38
C2788.924682.773789.214470.754277.744646.54
D3396.162963.712214.362139.741793.431863.84
E3346.413168.963180.842471.862386.681839.22
F2770.175456.684027.434580.404364.484477.89
G3402.092463.631744.992025.341331.521846.44
H3327.613917.901909.042332.502014.392967.68
I3093.945072.782742.223279.222415.872255.53
J3396.452727.512480.052191.721895.431907.17
K2809.013676.954459.544479.504458.014661.17
L2767.514462.524317.314604.584985.455235.91
M3392.061797.531833.772084.842348.373878.10
N3285.463739.931713.362425.503093.762477.91
O3357.812320.113471.582461.642656.851858.46
P2764.894546.324977.144735.624517.415485.83
Q3359.702823.812708.562297.152686.881629.20
R2770.934748.164358.624588.804277.494682.19
S3350.611587.072482.332283.413070.744142.74
T2763.455513.314223.694624.584550.395375.08
U3397.502317.791682.922048.132276.453021.81
V3393.121667.813380.632274.201493.382140.71
W3397.972660.472252.422130.521820.462103.76

Reflections 

The intended goal of this challenge was to exhibit the ways in which absent columns can complicate and obfuscate a problem. For each exorcist, average cost is a simple linear function of a handful of features. But half of those features were missing, and had meaningful correlations with those that weren’t; this distorted the linearity and independence of remaining features’ effects[1], and forced irrelevant features back into play (Hostility has 0 coefficient when calculating Eliminator prices; it only matters to an Eliminator insofar as it’s evidence of ghost type, which itself only matters as evidence of other features; but removing Hostility from your Eliminator price model still makes it legitimately less predictive).

What the scenario actually ended up exhibiting was how weak and manageable these obfuscating effects can be, and that a good-enough first-order approximation can be the best way to reach a perfect answer. Congratulations to simon, aphyer, and new (?) player Jonathan Paulsen for successfully steamrolling the problem with Linear Regression; condolences to Yonge for reaching a slightly worse answer by using a more nuanced modelling approach after successfully detecting nonlinearities in feature effects; and apologies[2] to anyone expecting a harder challenge (I also thought this one would be more difficult than it was!). Feedback on this point, and on all other points, is greatly appreciated.

Scheduling

I find myself occupied by work, and distracted by other projects: I may run a game starting 2nd August, but don't count on it. (Please feel free to snipe me if you’d like to take next month’s slot!)

  1. ^

    Things which aren’t there, and yet continue to meaningfully influence events by indirect and arcane mechanisms . . . seems like a metaphor for something, doesn’t it?

  2. ^

    In my defense, I did confirm that using simple models resulted in systematic mis-estimations; unfortunately I didn’t think to check that relying on these mis-estimations would ever lead to a suboptimal decision. I'll know better next time.

5 comments

Comments sorted by top scores.

comment by Jonathan Paulson (jpaulson) · 2024-07-18T01:14:33.477Z · LW(p) · GW(p)

For row V, why is SS highlighted but DD is lower?

Replies from: abstractapplic
comment by abstractapplic · 2024-07-18T08:17:10.502Z · LW(p) · GW(p)

Good catch; fixed now; ty.

comment by Christian Z R · 2024-09-27T13:27:23.227Z · LW(p) · GW(p)

Thanks for a good one, where I finally could use a bunch of linear regressions. Steamrolling! (I was sure there would be some devious trap, but I guess sometimes the basics actually do work, which is how they became the basics)

One thing that perhaps would make it easier was if the web interactive could tell whether or not your selection was the optimal one directly, and possibly how higher your expected price was than the optimal price (I first plugged mine in, then had to double check with your table out here)

Anyway, greetings, and looking forward to seeing the next one. Will train on the older ones until then

Replies from: abstractapplic
comment by abstractapplic · 2024-09-27T17:16:22.936Z · LW(p) · GW(p)

Thanks for a good one

 

I'm glad you feel that way about this scenario. I wish I did . . .

(For future reference, on the off-chance you haven't seen it: there's a compilation of all the past scenarios here [LW · GW], handily rated by quality and steamrollability.)

One thing that perhaps would make it easier was if the web interactive could tell whether or not your selection was the optimal one directly, and possibly how higher your expected price was than the optimal price (I first plugged mine in, then had to double check with your table out here)

. . . huh. I feel conflicted about this on aesthetic grounds - like, Reality doesn't come with big flashing signs saying "EV-maxxing solution reached!" when you reach an EV-maxxing solution - but it does sound both convenient to have and easy to set up. Might try adding this functionality to the interactive for the next one; would be curious to hear what anyone else who happens to be reading this comment thinks.

Anyway, greetings, and looking forward to seeing the next one.

Good to have you on board!

comment by NoriMori1992 · 2024-10-09T13:52:12.128Z · LW(p) · GW(p)

I normally just read these for fun and make no effort to solve them (I know nothing about data science or data analysis). This time I fooled around with the dataset for about half an hour, and managed to get a small inkling that the Phantom Pummelers disliked Sliminess and maybe liked Corporeality. I feel inordinately proud of myself for that minor achievement. (Full disclosure, I also got some inklings that were wrong, like thinking PP also disliked Hostility.)

Feels weird to be at the end. Looking forward to the next one. Might actually try to solve it, even though I will have absolutely no idea what I'm doing.