D&D.Sci III: Mancer Matchups

 So you foolish mortals, you don't trust me enough to give your true names,
  your worried that I might talk you out of your soul if you allow me to talk to you,
  and you feel the need to put in place precautions to stop me sending you anything other than your marching
  orders. Despite this you do apparently trust me to take an important decision for you. If everything is
  as it appears this set of mortals is even more stupid than the usual lot.
  
  Looking at the the list it start of with the necromancers attacking the geomancers.
  The the pyromancers intevene breifly, before the necromacners start attacking the vitamancers
  Then the vitamancers and the pyromancers start attacking one another
  Then things gets pretty random
  Then the necromancers attack ther cryomancers.
  Then there are 607 groups of 5 battles where all of the mancers ignoring the electomancer are involved in once.
  
 The groups of 5 battles involve a lot of groups with attacks on both sides territory, which suggest either a
 very good spy network predicting one sides attacks, or someone is actively organising this. It also suggests the records are grouped in chronological order.
 
  The so called good mancers have won more battles than the other side, in particular once the group battles
  occur their lead over the other mancers appears to increase at a roughly constant rate. Granted some battles
  can be more important than others, but it does rather suggest that this plane is in rather more danger of
  being taken over by this lot than the other lot. You may call yourselves good, but anyone can call 
  themselves that.
 
  Assuming the probabilities of each mancers winning a battle only depend on the mancers involved and the
  location, and not any other battles suggest that if I want to maximise their chances of winning 5 battles
  I should tell them to:
  Cryomancer COUNTER  v Pyromancer A 100.0 % of WIN Min data points 18
  Vitamancer A COUNTER  v Pyromancer B 71.42857142857143 % of WIN Min data points 18
  Geomancer A DEFEND  v Necromancer A 100.0 % of WIN Min data points 18
  Vitamancer B COUNTER  v Necromancer B 65.21739130434783 % of WIN Min data points 18
  Geomancer B DEFEND  v Necromancer C 100.0 % of WIN Min data points 18
  P(All Win) = 46.58385093167702
  P(4 Win) = 71.42857142857143
 
  And if I was to sabotage there efforts by maximising there chances of losing 5 battles:
  Vitamancer A DEFEND  v Pyromancer A 11.888111888111888 % of WIN Min data points 19
  Cryomancer DEFEND  v Pyromancer B 12.5 % of WIN Min data points 19
  Vitamancer B COUNTER  v Necromancer A 0.0 % of WIN Min data points 19
  Geomancer A COUNTER  v Necromancer B 28.000000000000004 % of WIN Min data points 19
  Geomancer B COUNTER  v Necromancer C 4.166666666666666 % of WIN Min data points 19
  P(All Lose) = 53.197552447552454
  P(4 Lose) = 73.88548951048952
  
 Is winning all 5 battles more important than a bigger chance of winning 4 battles . This bunch of incompetents don't say. In any case it makes no difference if I was to help them, and only a small difference if I wasn't.
  
  Confining the analysis just to the groups of 5 yields slightly different results:
  Vitamancer B COUNTER  v Pyromancer A 89.1891891891892 % of WIN Min data points 18
  Cryomancer COUNTER  v Pyromancer B 85.29411764705883 % of WIN Min data points 18
  Geomancer A DEFEND  v Necromancer A 100.0 % of WIN Min data points 18
  Vitamancer A DEFEND  v Necromancer B 76.92307692307693 % of WIN Min data points 18
  Geomancer B DEFEND  v Necromancer C 100.0 % of WIN Min data points 18
  P(All Win) = 58.51779381191147
  P(4 Win) = 76.0731319554849
  
  Vitamancer A DEFEND  v Pyromancer A 9.090909090909092 % of WIN Min data points 12
  Cryomancer DEFEND  v Pyromancer B 12.5 % of WIN Min data points 12
  Vitamancer B COUNTER  v Necromancer A 0.0 % of WIN Min data points 12
  Geomancer A COUNTER  v Necromancer B 31.57894736842105 % of WIN Min data points 12
  Geomancer B COUNTER  v Necromancer C 4.166666666666666 % of WIN Min data points 12
  P(All Lose) = 52.15809409888358
  P(4 Lose) = 76.23106060606062
  
As the situation qualitatively changed when the groups of battles started to occur these are the ones to use
if I want to help or hinder them? But do I? On the available data there is no obvious way to know which outcome would best server my interests. If they are  distrustfull enough to take all these precautions they may be expecting me to give them what seems to be the worst possible outcome, and they are trying to trick me by changing the labels so that it is actually the best.
  
 These mind games are what demons should be playing on mortals not the other way around. They are really out of line by putting me in this position! So I won't give them any advice at all. Whatever they were planning 
 it is highly unlikely that they would go to the trouble of summoning a demon in the hope that it would
 ignore them. And if I ever comes across them again in more normal circumstances I will be sure to teach them a lesson. Now back to some good old fashioning demoning...

order matters; these lines aren't randomized, nor are they sorted in any obvious way

a reasonable (but not enormous) number of results for every combination of (attacker,defender,territory) that can occur in our fights.

look for patterns in the data in the hope of figuring out general rules; there's enough noise that we shouldn't be very confident about any patterns we find, so we might as well just use the data.

for each possible matchup I propose that we use Laplace's rule of succession to estimate our winning probability should it happen; that is, if when A fights B in territory C we have so far seen A win a times and B win b times, we suppose that the probability of a win for A is (a+1)/(a+1+b+1). This corresponds to having a uniform prior on the winning probability. (Note in particular that when A has won every time we do not assume that this means A is literally unbeatable.)

that different fights have independent outcomes; we have no obvious way to tell whether that's so, still less to identify what specific correlations there might be, and on the face of it independence seems like a reasonable assumption in any case.

3840 different things we could do (120 permutations of the mages, times 32 choices of whether to attack or defend in each fight). I am assuming here that we have to fight all the battles, and that we can't send more than one mage to a single battle. (If we decide to be helpful, then given that "as many wins as possible" is our summoner's goal it seems clear that we should use all our mages, and we have no information about what happens if we use two at once, or what happens if an opponent has to attempt an attack immediately after successfully fighting off a counterattack.) So we can model all of them, see what distribution of #wins results from each, and then advise our summoner accordingly.

what we should actually say -- because we might decide to be helpful, or to screw our summoner over in revenge for disturbing our rest, or to do some random trollish thing. I don't really feel competent to legislate the utility functions of demons (maybe we love Evil and hate Good! maybe we love it when people do things that are bad for them! maybe we don't care about the outcome of these fights and should "reward" our summoner for being naive, in the hope of screwing him over more vigorously on a later occasion! etc., etc., etc.), so I'll offer some specific possibilities for specific goals we might have.

• If we decide to be helpful by maximizing the expected number of wins for the "good" guys, they should send C,VB,GA,GB,VA respectively to the five upcoming fights; the cryomancer and geomancers should attack and the vitamancers defend. This way they get 4.24 wins on average. This is also the strategy that maximizes the chance of winning all five fights (40.9%), and the strategy that minimizes the chance of losing all five (5x10^-6).
• If we decide to be malicious by minimizing the expected number of wins for the "good" guys, they should send VA,C,VB,GA,GB respectively to the five upcoming fights; the first two should attack, the others should defend. This way they get 0.69 wins on average. This is also the strategy that maximizes the chance of losing all five fights (46.4%). It isn't the one that minimizes the chance of winning all five, which is obtained by swapping the two vitamancers and swapping the two geomancers; this makes the win-all-five probability 8x10^-6. (Of course the strategies where they almost never win all five, like the strategies where they almost never lose all five, are made that way by having at least one fight where we've never seen the wrong mage win, and the exact probability we end up with is strongly influenced by e.g. my choice to use Laplace's rule of succession rather than something else.
• If we decide to maximize our influence on the outcome by minimizing the variance of the number of wins for the "good" guys, they should send C,GB,VB,GA,VA respectively, the first three attacking and the other two defending. This way there's an 84% chance that they win exactly one fight. This also minimizes the entropy of the outcome (sum of - p log p), if "outcome" means "number of wins".
• If we decide to maximize chaos by maximizing the variance of the number of wins for the "good" guys, they should send VB,VA,GB,C,GA to attack, attack, defend, defend, attack. This also maximizes the entropy of the outcome.

there are all sorts of interesting patterns in how the record of past wins and losses was generated; as explained above, I haven't thought it worthwhile looking for them because I don't think we can have enough evidence for favour such patterns over just using the results we have for outcomes given all the available information. I suppose it might turn out e.g. that territory makes no difference at all to the probabilities, in which case maybe we would get slightly better results by aggregating in the obvious way, but it's not obvious that that's so from a quick look at the data and I therefore don't think it would be safe to assume it's the case. (I mention this possibility because from the very quick and partial look I just took at the data, it's also not 100% obvious that it's false, though I see some things that look like evidence against.)

models where e.g. there's some territory-independent relationship between any two mages (and then a per-territory correction; more precisely I'd probably want something like log odds = a f1(attacker,defender) + b f2(attacker,territory) + c f3(defender,territory) + d f4(attacker,defender,territory); obviously we could always take a=b=c=0, but we'd put some sort of prior on the values of a,b,c,d, etc., etc., etc.; it all gets rather complicated), and try to estimate how strong that relationship is, which would give us the ability to use more information besides the results in matchups that correspond exactly to the ones we're interested in -- at a cost in worse results if it turns out that the model is a bad match for "reality". I wouldn't be terribly confident that this would actually do better than the naive approach I've taken above, but it would be interesting to play around with models of this sort and see how much if at all the results depend on what assumptions we make. Before doing any of that I'd obviously want to slice and dice the raw data in various ways in search of "obvious" patterns. And all this would take at least a couple of hours of extra work, and frankly I'm comfortable with the approach taken above which I expect to do reasonably well given any underlying reality that generates results that look broadly like the ones we have.