D&D.Sci June 2022: A Goddess Tried To Reincarnate Me Into A Fantasy World, But I Insisted On Using Data Science To Select An Optimal Combination Of Cheat Skills!

This has been a fun problem so far. I expect that the inclusion of the Eldritch Abomination and Chaos Deity data are intended as Data Science Cheat Skills - though, the Eldritch Abomination so far doesn't seem that useful, in contrast to the awesome Chaos Deity Cheat.

At the start, I thought to myself that the Eldritch Abomination data was probably intended to be used to learn about trait interactions in order to debias the no-collaborator data. (Though you could just use Chaos Deity data). I figured that in order for that to work, there had to be not much in the way of trait-skill interactions. But, in that case, the ordering of skill combos shouldn't depend much on the traits, so the Chaos Deity data ignoring data should be about right.

 So, I found the best skill combos according to Chaos Deity interactions and started by looking at what was best ignoring traits in the Chaos Deity data. I also noted some skill synergies and anti-synergies, noting that . Well, easy problem, now I just have to check if it works taking into account traits, then switch to just getting observations for the goddess right?

Wrong. I check all the data with the protagonist's specific traits, and the skill combo relative win rates are very different. 

But, now I think almost done though, because there's so much data that even a single trait+skill combination has a decent number of data points, enough to be strongly suggestive of what's best and while not enough to really be sure, but I'll just have to check some "nearby" combos to see if it's likely a fluke and then I'm done, right?

Wrong again. Because before I did that, first I wanted to just check to see if some hidden variables (which affect skill selection but also win rates) were affecting the result. So to do that I compared the results with Chaos Deity (for the protagonist's traits) and they look - kind of different. Maybe not for sure different, but enough to be pretty suspicious.

So I look for more evidence of hidden variables, by (eventually) making a list of all the trait+skill combos and sorting by how unlikely[1] the chaos data (or a more extreme deviation from the Chaos+None data) would occur, assuming that the true distribution is given by the Chaos+None total data. And - holy crap, there is a freight train's worth of evidence for hidden variables. Lots of outliers where specific skill combos show up consistently. In order to try to make sense of it, one theory I come up with classifies the skills into three groups - which I'll discuss below. It is still seems flawed though, and I'm running out of time so I checked out the comments in this thread. 

1 The formula I used was a cumulative binomial distribution with a conditional to determine which side I check. I am not confident this is the correct way to do it and it feels like it could easily be off by a factor of two or something (because some two-sided test might be more appropriate), but I guess it should give a reasonably accurate ordering of unlikeliness.

I see aphyer's made a detailed set of observations, and he and Yonge have both noted the Chaos Deity/no collaborator discrepancies to some extent. 

What really excites me about aphyer's comment though is a classification of the skills into three groups - Mental, Defense and Utility. Because these groups have a lot of similarity with the (flawed) classification I came up with from looking at discrepancies between the Chaos Deity and None data. (they also jive with my much more limited initial notes on synergies, which I had slipped out of mind).

My first group was Enlightenment/Radiant Splendor/Uncanny Luck, because for the trait/skill combos with the first 14 most unlikely discrepancies, both skills were from this group (and they all did better with no collaborator than with Chaos Deity). I called this these "light side" skills and, apart from the omission of Mind Palace, they correspond to aphyer's "Mental" group.

My second group was Barrier Conjuration/Hypercompetent Dark Side/Monstrous Regeneration/Anomalous Agility.  That was because, in the list of most unlikely discrepancies, up until spot 136 where these skills showed up they always did better with Chaos Deity than no collaborator. I called these "dark side" skills and, apart from the inclusion of Hypercompetent Dark Side, they match aphyer's "Combat" (or "Defense") group.

My third group was the remaining skills of Shapeshifting/Rapid XP Gain/Temporal Distortion/Mind Palace. I called these "neutral" skills and apart from the inclusion of Mind Palace and omission of Hypercompetent Dark Side, they match aphyer's "Utility" group.

I had attempted to use these groups to predict which combos would do better with chaos and which with no collaborator. My working hypothesis had been: if the combination includes two "light side" skills, or a "light side" and a "neutral" skill, then there will be better results with no collaborator, otherwise there are better results with Chaos Deity. This correctly predicts which of the 82 most unlikely discrepancies favor Chaos Deity v. None. But, that may not be that great in terms of implications for correctness of the theory since some skills and skill combos are not showing up very much in the outliers extreme enough to be confident of non-randomness. For example, it's not until the 99th most unlikely discrepancy that a dark+light combo shows up. And at this point there's a decent fraction of a percent chance of this happening at random in any skill/trait combo making it very likely this will show up by chance. 

One obvious question is, what happens if I modify my groups to match aphyer's? Keeping the same rule, this fails on the 24th unlikeliest outlier, where Temporal Distortion + Mind Palace favors Chaos, but ought to favor None with the shift of Mind Palace. There are several other added failures in the first 100 or so, and all are attributable to Mind Palace. (And a number of these are not with the Hikkikomori trait). So, I think Mind Palace has to stay as I originally placed it, for this particular purpose of determining outliers).

Putting both "Mind Palace" and "Hypercompetent Dark Side" as "Utility", the same as my original grouping but with HDS moved to utility: this rule makes the same decisions as the first up to the no. 177 point, so there is relatively weak evidence to distinguish which group HDS is in. It does look a little worse for this purpose to have it as "Utility"/"Neutral" than as "Defense"/"Dark Side", but it could be either way. But the case against Mind Palace being in the "Light Side"/"Mental" group for this purpose seems relatively solid.

So anyway, I am trying to figure out more about this discrepancy information, but don't have time. I would like to at least investigate the relationship to traits. So far all my analysis of it has no effect on my decision apart from confirming the existence of hidden variables. I also haven't gotten around to looking at "nearby" trait combinations to look for trends/consistencies. 

Anyway, as others have noted, it may be best to avoid a combo that did badly with the Chaos Deity, since that suggests that there's some hidden variable that leads to selecting that combo and also to succeeding it. On the other hand, maybe these variables might not be hidden to the protagonist? If I feel a strong urge to select Enlightenment+Radiant Splendor, might as well go for it. I guess Temporal Distortion+Anomalous Agility otherwise, as the second best scorer for the protagonist's exact traits.

Also, while the goddess's reaction suggests that data analysis is unusual, I should probably ask just to verify. If not, that could change the implications of the hidden variables.

There is an enormous amount of evidence of one or more hidden variables affecting both win rate and skill choice (as evidenced by discrepancies between the chaos deity and no-collaborator data for particular skill-and-trait combinations).

This includes both skill combinations that do better if you chose them yourself, and skill combinations that do worse.

This seems to be in some way connected to the groups that aphyer found, except that in this case Mind Palace seems to count as a "Utility" skill and maybe Hypercompetent Dark Side counts as a "Combat" skill (I use "Light", "Dark" and "Neutral" instead of "Mental", "Combat" and "Utility" for these altered groups) . Light being associated with doing better if you chose them yourself, and Dark (or double neutral, or maybe light+dark) doing worse. This rule is imperfect and the real rule could be completely different than what I think.

All of the above hasn't actually had all that much practical impact yet. For now I pick:

Temporal Distortion + Anomalous Agility

(Same reasons as Yonge)

But I would pick Enlightenment + Radiant Splendor if that's what I was going to pick without doing the analysis.

Also, is it possible to add some time?

Notes:

-Traits:

Most nerds are otakus and vice versa. Nerds are great at time travel, otakus are great at everything except having a dark side. Postulating some underlying stats (and tentatively drawing the causal arrow stats -> traits, because of the uneven and correlated distribution of traits), it would seem that anyone with enough points in different stats, probably including something we can call intelligence, has a high chance of being an otaku, while only intelligence contributes to the chance of being a nerd. A dark side just switches your underlying stats to something much-better-distributed than normal, which is why it's often a good thing for non-otakus to take and bad for otakus, who already have good stats and have better uses for the skill slot.

Most office workers aren't Hikkikomoris and vice versa. It would make sense for this to be linked to some sort of sociability stat, though curiously sociopaths aren't part of this interaction. Office worker special power: The Grind (reliably gain power through XP gain), Hikkikomori special power: Death from isolation (high chance to get trapped in one's own mind palace and instantly die (or something))

-Skills (most of this agrees with aphyer's analysis):

If you want to stay alive, you'll want to specialise in some way to defend. There are 3 ways of doing this: Shielding yourself from damage (works best at medium hp and tankiness), Lots of regeneration (works best at high hp and tankiness), or just Dodging everything that comes at you (works best at low hp and tankiness).

You'll also want some way of gaining power and doing damage, otherwise you'll never work your way up to defeating the Demon King. If your stats are bad, the best thing you can do is just swap them out for a powerful and mysterious dark side. Enough Intelligence and the power of time travel might work for you. With hard work and diligence, you can try to amass enough XP to get there all by yourself. If you don't mind the opinions of others or if you're into that kind of thing, you can turn yourself into a big and tanky dinosaur (which then means regeneration is the best defense to specialise in). It's possible you can scrape by just getting more lucky than others would, but usually there are better options. Similarly, a Mind Palace can be sufficient for some people, but usually there are better options, especially for Hikkikomoris.

Anyone good enough or righteous enough or something to feel compelled to pick some combination of Enlightenment, Splendour or Luck should follow that impulse, everyone else should stay clear.

My choice:

Temporal Distortion, Anomalous Agility

Unfortunately, we lack any information about ourselves other than the two traits. We don't know whether enlightenment/splendour is a newcomb-like or smoker's lesion-like decision or whether we've lost our chance at goodly and righteously declaring that of course we'll be taking enlightenment/splendour by just taking the time to look at the data.
As for the decision between Agility and Barriers, and possibly even between Temporal and XP or Mind palace, it seems to be down to what our underlying stats are, which we know nothing about, so we just have to go by the average from the data.

 Hmm, seems like something bad happens if you pick Hypercompetent Dark Side and Enlightenment.

 I ranked each possible trait and cheat by the effect it has on your win rate relative to not having it.  From best to worst:

Otaku: + 40.6%

Nerd: +28.5%

Radiant Splendor: +15.7%

Enlightenment: +15.6%

Shapeshifting: + 11.3%

Sociopath: + 9.1%

Office Worker: +8.5%

Uncanny Luck: + 7.6%

Rapid XP Gain: + 3.2%

Barrier: -2.1%

Regeneration: -3.3%

Anomalous Agility: -5.4%

Temporal Distortion: -6.4%

Dark Side: -10.7%

Hikkikoh god i can't spell this am I going to have to spell it everywhere in my code: -15.3%

Mind Palace: -34.2%

 This doesn't consider interaction effects - is being a Nerd good, or is it just correlated with Otakus who have a knowledge of the underlying world?  Is Mind Palace bad for everyone, or only for Hikkikowhatevers who sit inside their Mind Palace all day and don't do anything?  Still, at a first glance, we should probably take at least one of Enlightenment and Radiant Splendor, and probably not take Mind Palace. 

For heros with our traits:

- Temporal Distortion and Rapid XP gain are the best 2 overall which seem to work well with anything.  The best single combination however is Enlightenment + Radiant Splendor (94 percent win rate). Shapeshifting is just terrible and should always be avoided. The others work well with some traits, but not with others;
The following combinations are very poor (< 1/2 win rate) and should not be given to other heroes like me:
- Anomalous Agility + Barrier Conjuration
- Anomalous Agility + Monstrous Regeneration
- Anomalous Agility + Uncanny Luck
- Barrier Conjuration + Monstrous Regeneration
- Barrier Conjuration + Uncanny Luck
- Enlightenment + Hyper Competent Dark Side
- Enlightenment + Monstrous Regeneration
- Hyper Competent dark Side + Mind Place
- Hyper Competent Dark Side + Radiant Splendor
- Hyper Competent Dark Side + Uncanny Luck
- Mind Place + Uncanny Luck
- Monstrous Regeneration + Radiant Splendor
- Monstrous Regeneration + Uncanny Luck
- Rapid XP Gain + Uncanny Luck
 
When looking at records for all trait a different picture emerges. Here MindPlace is the ability that is terrible, and shapeshifting does reasonably well with anything except MindPlace. The only terrible combos (< 1/2 chance of success ) which don't involve MindPlace are:
- Anomalous Agility + Barrier Conjuration
- Barrier Conjuration + Monstrous Regeneration
- Enlightenment + Hyper Competent Dark Side
- Hyper Compenetent Dark Side + Radiant Splendor
- Hyper competent Dark Side + Uncanny Luck
 
 Radiant Splendor and Enlightenment still does best overall though.
 
In general our traits seem to be somewhat below average, but Radiant Splendor + Enlightment still looks like a very strong combination. This combination was noticeably less successfull (6/12 times) for heros with our traits when the Chaos deity was involved. The sample size is small, and the effect doesn't show when all heroes are considered, so this might be a statistical fluctuation; however it may be an idea to avoid further dealing with this deity, just in case it is giving inferior versions of traits to heroes like me to spread chaos.

Provisional Decision: Go with Radiant Splendor + Enlightenment