D&D.Sci December 2022 Evaluation and Ruleset

post by abstractapplic · 2022-12-12T21:21:08.781Z · LW · GW · 8 comments

Contents

  Ruleset
    Snark Sub-Species
      Average
      Waking-Time
    Which Snarks are Hunted?
  Strategy
  Leaderboardplot
  Reflections
None
8 comments

This is a followup to the D&D.Sci post [LW · GW] I made ten days ago; if you haven’t already read it, you should do so now before spoiling yourself.

Here is the web interactive I built to let you evaluate your solution; below is an explanation of the rules used to generate the dataset (my full generation code is available here, in case you’re curious about details I omitted). You’ll probably want to test your answer before reading any further.

---

Ruleset

Snark Sub-Species

There are thirteen distinct types of Snark; three of these are Boojums. Typical characteristics for each sub-species (which are frequently deviated from; see my generation code for details) are summarized in the table below:

Name	Freq	Boojum?	Average Waking-Time	Other Characteristics
Vorpal	19%	No	2:27pm	Hollow yet Crisp taste Extreme Fondness Moderate Cleanliness Moderate Phobia
Frumious	7%	No	2:00pm	Crumbling yet Blunt taste Mild/Moderate Fondness Moderate Cleanliness Extreme Phobia
Slythy	14%	No	4:20pm	Hollow/Artless taste Crisp/Neat taste Mild Everything
Mimsy	4%	No	4:10pm	Artless/Meagre taste Bright taste Moderate Everything
Manxome	4%	No	2:41pm	Hollow/Haunting taste Blunt taste Unusually specific sleep schedule (Very!) Mild Fondness Moderate Cleanliness Extreme/Moderate Phobia
Whiffling	6%	No	1:39pm	Hollow yet Bright taste Relatively specific sleep schedule Extreme Fondness (Very!) Mild Cleanliness Moderate Phobia
Burbling	8%	No	4:33pm	Artless yet Crisp/Clear taste Extreme Fondness Moderate Phobia
Gyring	10%	No	3:22pm	Meagre yet Neat taste Moderate Fondness Moderate Cleanliness Extreme Phobia
Gimbling	11%	No	3:23pm	Artless/Meagre taste Extreme Fondness Moderate Cleanliness Mild Phobia
Cromulent	5%	Yes	4:01pm	Hollow/Crumbling taste Blunt taste Mild Cleanliness Phobia almost never Moderate
Snippid	2%	Yes	2:44pm	Meagre/Haunting taste Clear taste Moderate Fondness Moderate/Extreme Phobia
Scrumbling	3%	Yes	4:22pm	Crumbling yet Blunt taste Mild Fondness Moderate/Mild Cleanliness Moderate/Mild Phobia

Which Snarks are Hunted?

2% of all sighted Snarks are left unhunted due to logistical problems or gluts of potential targets.

Conventional wisdom in the Snark-hunting community is that Snarks with a Taste containing “Crumbling” and (to a lesser extent) “Blunt” are much more likely to be Boojums, and so should not be hunted. Six-sevenths of Snark-hunters follow this advice regarding “Blunt”, and everyone follows it regarding “Crumbling”.

Strategy

The risk associated with each Snark in the list is as follows:

Snark	Risk
A	1.75%
B	3.39%
C	2.41%
D	0.29%
E	98.93%
F	77.46%
G	0.32%
H	0.07%
I	3.71%
J	1.34%
K	2.21%
L	1.21%
M	0.86%
N	0.14%
O	0.55%
P	1.26%
Q	0.03%
R	0.38%
S	0%
T	0.01%
U	0.03%
V	0.05%
W	4.58%
X	7.13%
Y	0.04%

Which Snarks are worth hunting is a function of your own appetite for risk; the only certainty (given you assign money sublinear utility, only care about killing Snarks so you can spend the reward money, and value your life and the lives of the crew at >0), is that E, F and X aren't worth it.

Leaderboardplot

The solutions provided for the Bonus Task look like this.

Player (listed alphabetically)	Snarks Selected	P(Survival)	EV
aphyer	ABGHLPQVWY	87.899%	8.79 Snarks
simon	YNVGQRPHDBL	93.004%	10.23 Snarks
Thomas Sepulchre	BPGYHQTS	94.959%	7.60 Snarks
Yonge	BGHPQY	94.964%	5.69 Snarks

 

In classic Carroll-ian fashion, all have won and all must have prizes. Everyone but aphyer is on the Efficient Frontier, and I was planning to offer [UNSPECIFIED BENEFIT (UNDERWHELMING)] to him just for being a fellow D&D.Sci creator; you'll all be contacted when it's ready.

Reflections

In my view, this challenge did nothing it attempted to do, while facilitating several worthwhile things it attempted to facilitate, and a few it didn’t.

The bad news first. Most critically, I provided many too many rows, and far too few columns (continuous columns in particular), enabling people to successfully select Snarks by simply subsetting (sorry!). The central conceit of trying to optimize conflicting objectives under uncertainty also lacked a necessary basis: without a good sense of how much a Snark-corpse is worth, and how much the lives of the crew (should?) matter, the intended tension between these targets hangs loose.

However, the challenge ended up serving as a fruitful basis itself. It (completely unintentionally!) provided a demonstration of how much easier optimizing for EV is than minimizing risk, and how much harder it is to fairly evaluate. It allowed the creation of an interactive with a novel gimmick, tempting players to either hyperbolically discount and give up early, or to take on more risk than they planned in order to reach the next story event. It represented a cautionary tale for makers of future D&D.Sci games. And, finally, the intro poem was (in my humble opinion) an absolute banger.

In summary, this game was much better at having Artistic Merit than at actually being worth playing (my congratulations and condolences to the people who did play it for their excellent work). That was accidentally a really good match for the time of year, since everyone with Data skills spends December frantically trying to hit end-of-year targets (c.f. the muted response to aphyer’s excellent How the Grinch Pessimized Christmas [LW · GW]). I’ll be sure to take seasonality into account when planning future games.

. . . I usually end these sections with “feedback on these points, and any other point, is greatly appreciated”, but this time I’m already pretty sure what went well/badly. That said, if you disagree with any of the above, I’d still like to be corrected.

8 comments

Comments sorted by top scores.