Posts

Comments

Comment by Multicore (KaynanK) on Vim · 2021-06-06T21:22:00.843Z · LW · GW

A couple months after reading this post, I am now getting a big burst of dopamine every time I use the . key in vim.

(Even though it's just on the level of "Insert/replace the same text in multiple places")

Comment by KaynanK on [deleted post] 2021-06-03T21:17:22.645Z

Marking for deletion.

-Has almost no posts

-"Posts that are stubs" does not seem like a category that people would be interested in or merits a tag.

Comment by Multicore (KaynanK) on Attributions, Karma and better discoverability for wiki/tag features · 2021-06-03T13:29:51.581Z · LW · GW

Voting even works on the edits from the old wiki! I find it somewhat amusing to upvote an edit from twelve years ago.

Comment by Multicore (KaynanK) on Covid 5/6: Vaccine Patent Suspension · 2021-05-06T21:39:02.331Z · LW · GW

I think you may have misread the "Unvaccinated and undeterred" graph (which is terrible and misleading). 

All the numbers in each section add up to 100%, so it's saying "53% of people who dined in restaurants were unvaccinated" not "53% of unvaccinated people dined in restaurants". So you have to consider base rates. The numbers for half-vaccinated people were lower mostly because there are fewer half-vaccinated people than there are of the other groups.

(Saw this on twitter but I don't remember from who.)

Comment by Multicore (KaynanK) on My Journey to the Dark Side · 2021-05-06T21:07:45.388Z · LW · GW

You've mentioned Pasek's Doom a few times before, but I'm still not quite sure what it means. Something about taking your own headspace drama too seriously in self-destructive ways?

Comment by Multicore (KaynanK) on Facebook is Simulacra Level 3, Andreessen is Level 4 · 2021-04-29T19:25:04.668Z · LW · GW

Level one: A job title that straightforwardly describes what the person actually does in the job. For example, "Seventh-grade math teacher" or "Data Analyst"

Level two: A job title which claims to describe what the person actually does, but is misleading. For example, if someone was hired as an Electrical Engineer but actually spends their time doing IT work.

You move into level three a bit when the job title is partly about your status in a hierarchy: "Senior Software Engineer" or "Junior Software Engineer".

As the original post suggests, job titles for high level management are mostly about social status and hierarchy, and therefore are never below level three.

Comment by Multicore (KaynanK) on How Lesswrong helped me make $25K: A rational pricing strategy · 2020-12-22T16:12:24.968Z · LW · GW

Do you have a special case for someone who already has a job and is searching for a better paying one? That person's opportunity cost would not be 

(future pay per week x number of additional weeks spent searching) 

but 

((future pay per week - current pay per week)  x  number of additional weeks spent searching)

Comment by Multicore (KaynanK) on Pseudorandomness contest, Round 2 · 2020-12-20T16:19:32.898Z · LW · GW

It seems reasonably possible to be confident that a string is human-generated, but if anyone did their job well in round 1, it probably won't be possible to be confident that a string is computer-generated. 

Maybe some of the ones left over will seem slightly more or less random, but probably at some point I'll just have n strings left over and assign them all probability 62/n, adjusted for whatever uncertainty I had about the ones that seemed human-generated.

Comment by Multicore (KaynanK) on What are the best precedents for industries failing to invest in valuable AI research? · 2020-12-15T03:29:18.622Z · LW · GW

Blockbuster failed to invest in internet tech for their movie rental business and was outcompeted by smaller, more savvy startups.

Comment by KaynanK on [deleted post] 2020-12-14T18:12:53.743Z

Seconded, DeepMind seems like a natural tag to have given that we have tags for OpenAI, Ought, MIRI, etc.

Comment by Multicore (KaynanK) on Complex Behavior from Simple (Sub)Agents · 2020-12-14T16:44:02.239Z · LW · GW

Last-minute nomination: This is high-quality and timeless, and would look nice in a book.

Comment by Multicore (KaynanK) on The Darwin Game - Conclusion · 2020-12-14T01:24:50.849Z · LW · GW

A: EarlyBirdMimicBot is extremely restrictive about what it simulates, because I was worried about malware. MeasureBot confirmed this fear, though I could have been less restrictive and still avoided it. Therefore, PasswordBot cannot look at its opponent's source code if it wants EarlyBirdMimicBot to simulate it.

B: EarlyBirdMimicBot's simulation strategy is brute force, looking at the result of every possible sequence of the next N moves. lsusr required bots to make their moves quickly, so to save on time I only considered the moves 2 and 3 when simulating. 

I could have addressed this by simply having a special case behavior against PasswordBots instead of simulating them, but I didn't think of that.

C: I was actually planning to do this but screwed it up and did not check it properly before uploading. It would have been tit-for-tat against the field if I did it right.

Comment by Multicore (KaynanK) on The Darwin Game - Conclusion · 2020-12-04T17:57:50.024Z · LW · GW

What happens if EarlyBirdMimicBot is less scared to simulate? How much faster does it win?

I actually win less in that case, even if I get there faster. I get perfect cooperation with the deterministic cooperators written in Python, so one or two of them stick around forever if they last long enough. It can be two if one of them starts 2 and the other starts 3 so they cooperate with each other, though I'm not sure if there's a deterministic Python bot that starts 3.

Comment by Multicore (KaynanK) on The Darwin Game - Conclusion · 2020-12-04T17:48:31.643Z · LW · GW

The clones do not fold; in the early game they play an EquityBot-ish strategy that gives attackers less than cooperation would have gotten them. Only a couple of players were willing to fold in the early game, and usually only after ten or more turns of attack. Attacking for tens of turns to find out whether your opponent is a FoldBot will destroy you in a pool of mostly not FoldBots.

Simulation would be able to tell you who to bully without having to go through that - run the opponent for 100 turns and see if they eventually fold against all 3s. But as always, simulation runs the risk of MeasureBot-style malware.

Comment by Multicore (KaynanK) on In Addition to Ragebait and Doomscrolling · 2020-12-04T14:16:41.597Z · LW · GW

I thought we were already calling it Sneer Culture.

Comment by Multicore (KaynanK) on The Darwin Game - Conclusion · 2020-12-04T12:48:55.566Z · LW · GW

Looking through the code, yep, my simulation criteria were so conservative that I only simulated the PasswordBots. OscillatingTwoThreeBot was oh so close to only having two open parentheses but it used a decorative set of parentheses in the class definition (as did many others) Looks like I didn't need simulation anyway.

I am somewhat interested in using the code to explore alternate timelines. Who wins without me? Who wins the clone showdown if it's allowed to happen? What happens if you start the game at round 90 and make the smart bots use their endgame strategies in a pool full of silly bots? What happens if you remove npc bots and have a pool of only players? Does anything interesting happen if the number of turns per round is 101 rather than 100? I'm probably not interested enough to commit to doing this in a timely manner though.

What marginal submission would win in this pool? Probably just a MimicBot with Measure's opening game. Using simulation, especially hy-compatible simulation, could make you win more as long as you didn't simulate MeasureBot, or only simulated it in a separate thread.

It's been a great ride. Thanks for running the game, lsusr.

Comment by Multicore (KaynanK) on Building up to an Internal Family Systems model · 2020-12-02T12:52:29.667Z · LW · GW

Nomination for 2019 review:

I originally tried to read Self-Therapy, but bounced off of it because it was aimed too much at people with major life-impacting traumas. This post was much more approachable, and I liked the robot metaphor. Since reading it, I started to notice the ways in which my own mind is behaving like a manager or firefighter with respect to embarrassing incidents in the past.

Comment by Multicore (KaynanK) on The Mutant Game - Rounds 11 to 30 · 2020-11-23T12:39:24.499Z · LW · GW

In the Mutant Game, the PasswordBots aren't really on my team. Since it appears to them that the opponent started with 3, they will play 3 on every turn against everybody.

If the round index is always 0, that means the clone truce never ends and you have population dynamics based on which clones are getting 300-200 against which other clones. Not sure if it will be stable or not.

I suspected that my simulation strategy didn't end up being all that useful, but I'm still curious what bots I managed to simulate at all. Presumably at least the PasswordBots. I guess I can find out when the code is released.

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 21-500 · 2020-11-22T05:43:57.819Z · LW · GW

It also significantly affected people's expectations of the metagame and made them prepare for simulators. BendBot and CloneBot were made deterministic so that simulators could efficiently cooperate with them.

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 21-500 · 2020-11-21T01:32:35.442Z · LW · GW

Wow, that was not what I would have predicted from the last set of results. It looks like the trend in my population from when the clones were a little nasty accelerated when the clones got really nasty.

In the true game with AbstractSpyTreeBot, MeasureBot is going to be eating it at the same time I'm eating the clones, but is it going to be a boost as extreme as this?

In the endgame everyone cooperates with everyone else, and it seems to be down to openings and match-breaking strategies. LiamGoddard and MeasureBot always start 3, BendBot alternates between starting 2 and starting 3 on a per-round basis, and EarlyBirdMimicBot randomly starts 2 or 3 using the python random number generator. BendBot uses a fixed pseudorandom sequence to try to break matches, EarlyBirdMimicBot picks 2 or 3 with equal probability, MeasureBot plays 2 with probability 0.69 possibly with some special cases, and I'm not sure what LiamGoddard does.

Silly 2 bot lasted longer than anyone could have reasonably expected. The Clone Wars presumably gave it a second wind, as folding was a sound strategy against the clones' aggressive plays.

BeauBot in sixth place is the best scoring bot we don't have a clear explanation of yet.

Comment by Multicore (KaynanK) on Embedded Interactive Predictions on LessWrong · 2020-11-20T20:10:31.194Z · LW · GW

This was a neat feature on Arbital, nice to see it here as well.

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 10 to 20 · 2020-11-18T16:44:51.528Z · LW · GW

If OscillatingTwoThreeBot (sixth place) is exactly what it says on the tin and always plays 23232323..., you get perfect cooperation with it 100% of the time. Could be a nice minor advantage.

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 10 to 20 · 2020-11-17T02:51:54.797Z · LW · GW

I notice I was slightly declining for a bit until round 10, where I started shooting up again. I'm not sure if it's because I changed my strategy at that point and scored more or because a bunch of other people changed their strategy at that point and scored less. I think it's more the latter, particularly increasing clone hostility.

Clones have slightly lost ground since last time. Without critical mass, their increasing hostility will hurt them more than the opponents. It looks like we're heading for a repeat of history, with Measure as the Zvi to my David. Because MeasureBot always starts 3 in the endgame and I randomize 50/50, I think I slowly lose if its starting population is bigger than mine and it's just us. If there are multiple endgame bots my more cooperative nature could be an advantage.

Actual Zvi's BendBot has gained significant ground after being in the middle of the pack in earlier rounds. Maybe it handles the middle game especially well. LiamGoddard in fourth place is the highest ranked bot we haven't gotten any explanation of.

The "true timeline" with AbstractSpyTreeBot is probably going to be this but more extreme, since ASTB feeds Measure even more.

Larks, excellent name choice for your AttackBot.

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 3 to 9 · 2020-11-14T01:26:28.990Z · LW · GW

Not quite as good for me as I would have hoped, but we're leaving the early game and heading into the midgame, where the clones will gradually start defecting against outsiders, leading to a showdown for dominance between clones and Chaos. The clones don't quite seem to have critical mass, but I care less about who wins and more about how long it lasts.

The graph for Silly 2 Bot is great, it shows exactly where weirdos and attackers became a marginal force.

Comment by Multicore (KaynanK) on The Logic of Science: 2.2 · 2020-11-13T16:34:49.289Z · LW · GW

Archive link: https://web.archive.org/web/20180225182303/http://pulsarcoffee.com/posts/los2-2/

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 1 to 2 · 2020-11-11T03:52:06.916Z · LW · GW

Silly 2 bot is absolutely killing it in this version. With the early pool having plenty of attackers and unpredictable weirdos, it turns out unconditionally folding is better for you than almost anything. EarlyBirdMimicBot's willingness to fold against uncompromising attackers in the early game is probably a decent chunk of my edge.

After the weirdos and attackers die off, I expect silly 2 bot will be destroyed in the midgame, and if too many people played it it would feed attackers, but a strategy that unconditionally folded in the first few rounds and did something else later would have been one of the stronger members of Chaos Army.

The clones are still more spread out than I would expect, but maybe that's just matchup randomness?

Comment by Multicore (KaynanK) on What Belongs in my Glossary? · 2020-11-02T21:04:13.296Z · LW · GW

An explanation of what Tier One, Tier Two, Tier Three, etc mean in your game reviews.

Comment by KaynanK on [deleted post] 2020-10-29T03:33:02.186Z

Possible duplicate of Intelligence Explosion and Recursive Self-Improvement

Comment by KaynanK on [deleted post] 2020-10-29T03:30:36.655Z

Possibly should be merged with AI Governance.

Comment by KaynanK on [deleted post] 2020-10-27T21:13:17.810Z

Given that 2-bot is currently in seventh place, above the majority of clones and Chaos Army, EarlyBirdMimicBot's early success might be as much due to its willingness to fold in early rounds as to its more sophisticated tricks.

My being in the middle of the clone pack last round but above it this round may be because the GoofBots that CloneBot best exploits, 0-bot and 1-bot, are now dead or marginal.

Comment by Multicore (KaynanK) on The Darwin Game - Round 1 · 2020-10-27T17:01:07.716Z · LW · GW

I just remembered something I forgot to post in an earlier thread:

CloneBot's cooperateWithClone method uses parity checking on its and the opponent's payloads to determine the first move, but after that it always plays whatever move the opponent played last. I.e. pure naive tit-for-tat.

This suggests a possible strategy for a MimicBot: against a clone, play 0, 5, 0, 5, 0, 5... Assuming the opposing clone started with 3, this results in plays (0, 3), (5, 0), (0, 5), (5, 0)...

If the number of turns per game is even, you end up with one 5 per two turns, for the same 2.5 points per turn as if you had cooperated. Because the clone played a 2 or 3 on the first turn instead of a 5, it loses 2 or 3 points compared to perfect cooperation. This could be a big advantage over the 90 rounds before the CloneBot showdown.

Unfortunately, if the number of turns is odd, the clone gets just as many 5s as you plus its initial 2 or 3, so you're the one who gets outscored. I haven't been able to think of any strategy that would get you 2.5 points per turn, get the clone fewer points, and work whether the number of turns was odd or even. 

Using a defecting strategy would be a coinflip between a big advantage and a crippling disadvantage, so my actual submission played it safe and just cooperates with clones. The actual number of turns wound up being even, but I had no reasonable way of expecting that was more likely than not (IIRC, in Zvi's original game the number of turns was odd).

If you're willing to score fewer than 2.5 points per turn if the clone does much worse than you, there are a series of dastardly strategies that I'm thinking of as "Lucy pulling away the football", where you ruin the opponent's big play while setting up your next big play.

0, 5, 1, 4, 2, 3 repeated gives (0, 3), (5, 0), (1, 5), (4, 1), (2, 4), (3, 2) repeated. You get (5 + 4 + 3) / 6 = 2 per turn, while the opponent gets (3 + 1 + 2) /6 = 1 per turn.

0, 5, 1, 4, 2 repeated gives (0, 2), (5, 0), (1, 5), (4, 1), (2, 4) repeated. You get (5 + 4) / 5 = 1.8 per turn, opponent gets (2 + 1) / 5 = 0.6 per turn.

0, 5, 1 repeated gives (0, 1) (5, 0) (1, 5) repeated. You get 5/3 per turn, opponent gets 1/3 per turn.

Sacrificing some points to outscore the clones by a full five times might be worth it if there were four or fewer clones, but with one MimicBot against eight clones I don't think the penalty to my score is worth the damage I can inflict on them. (Also, I only thought of the dastardly strategies today; before submission I only considered the "minor edge" strategy described earlier.)

This analysis also applies generally to any bot known to be a naive tit-for-tat bot.

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 0 to 10 · 2020-10-27T03:05:50.414Z · LW · GW

Does setting self.destroyedOpponent to True when you detect that you're simulated actually do anything? The instance of MeasureBot that knows it destroyed the opponent should be a different instance than the one that is making your moves.

Comment by Multicore (KaynanK) on The Darwin Game - Round 1 · 2020-10-27T02:32:45.288Z · LW · GW

In the mutant game where you always think that your opponent's last move was the same as your last move, I think EarlyBirdMimicBot behaves as follows in the first 10 rounds:

Against clones, gets full clone cooperation of either all 3-2 or all 2-3. I put in a check to stop cooperating with clone if the clone ever starts exploiting me, but the check doesn't notice that turns so far seem to have been all 2-2 or all 3-3.

Against simulatable bots, gets one turn of simulation, then next turn sees that the opponent's apparent move was not the same as what was predicted. Reverts to the fallback plan.

Against everything else, thinks that every turn so far has been a match, so continues its early match-breaking strategy of playing 2 with probability 2/3 and 3 with probability 1/3. I think that's close to a Nash equilibrium in the blind game.

Against outsiders, Clonebot will always play 3 if it played 2 last turn, and in an apparent 3-3 match it transitions between 50% chance to play 2 initially and 0% by turn 50.

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 0 to 10 · 2020-10-24T14:14:26.702Z · LW · GW

PasswordBot and DefinitelyNotCollusionBot. They were submitted by Ruby and habryka, who responded to my request on the LW Tagger Slack.

Comment by Multicore (KaynanK) on The Darwin Game - Rounds 0 to 10 · 2020-10-24T05:04:41.661Z · LW · GW

Darn, the clones are contesting the early pool against me well in part because they put in code to exploit 0-bot and 1-bot and I didn't. My plans for the early game focused more on dealing with attackers.

I'm curious which of the silly/chaos army bots passed my simulation test and got simulated.

Some clones doing significantly better than others is a bit confusing since for now they're all supposed to be doing the same thing. I guess some got really lucky/unlucky with other bots' random rolls?

It's worth noting that the clones aren't even being significantly aggressive against outsiders yet. This huge advantage is just from the perfect self-cooperation. I was kind of expecting a midgame where the clones fought a bloody struggle to clear out the non-clone cooperators while I profited off both sides, but the outsiders might be wiped out too fast for that to happen.

Also worth noting that on the next round my fallback behavior changes from a fold-ish EquityBot to DefenseBot. Most attackers seem to be gone or marginal at this point, so I'm not sure that changes much.

Comment by Multicore (KaynanK) on When was the term "AI alignment" coined? · 2020-10-22T02:25:48.987Z · LW · GW

The first MIRI paper to use the term is "Aligning Superintelligence with Human Interests: A Technical Research Agenda" from 2014, the original version of which appears not to exist anywhere on the internet anymore, replaced by the 2017 rewrite "Agent Foundations for Aligning Machine Intelligence with Human Interests:A Technical Research Agenda". Previous papers sometimes offhandedly talked about AI being aligned with human values as one choice of wording among many.

Edit: the earliest citation I can find for Russell talking about alignment is also 2014.

Comment by Multicore (KaynanK) on The Darwin Game · 2020-10-19T23:42:53.025Z · LW · GW

I'm sad that some people thought that more than two parenthesis was high risk to simulate/examine - I thought that the obvious thing to do was check to see if someone ever loads code or uses a random function, and if you don't do either, you should be safe.

We'll see how many people submitted simulators braver than mine, but simulators being timid seems like a natural consequence of the rules allowing you to nuke your opponent if you find out that you're in a simulation, and a common enough perception that simulators might have enough of an advantage that they should be eliminated. 

Static analysis is not very useful if the opponent's code is at all obfuscated, which is likely is if your opponent is looking to nuke simulators. Does your static analysis catch the code getattr(__builtins__, ‘e’ + ‘x’ + ‘e’ + ‘c’)(base64.decode(God knows what)) ? Or however many dozens of other ways there are to do something like that?

The tournament might look significantly different if the rules were slanted in the simulator's favor, maybe if you just had to avoid infinite simulation loops and keep runtime reasonable, and the worst the opponent was allowed to do if they found they were in a simulation was to play BullyBot in order to extort you or play randomly to make the simulation useless. The iterated prisoner's dilemma with shared source code tournament a few years ago had a lot of simulators, so I assume their rules were more friendly to simulators.

Comment by Multicore (KaynanK) on The Darwin Game · 2020-10-19T21:40:22.256Z · LW · GW

That does revise down my expectations of winning, but my bot having run thousands of times on someone else's computer and not crashing (or failing the clone check?) is good to hear.

Maybe I'm overestimating the snowball effects of an early pool. If the late game has everyone cooperating with everyone else, your matches with others are only giving a tiny bit fewer points than matches against your copies.

Comment by Multicore (KaynanK) on The Darwin Game · 2020-10-19T18:13:48.919Z · LW · GW

tl;dr I betrayed the CloneBot clique and submitted MimicBot. My submission is in this Github repository.

My Process in chronological order:

There were three important things I noted in the original post

  • Self cooperation is free, so an early lead is a big advantage.
  • There are guaranteed to be “silly” bots in the early pool.
  • You can vary your strategy depending on the round.

It seemed to me that the best strategy was to get as many points as possible early by exploiting silly bots, cooperating with anyone willing to cooperate, and folding somewhat to attackers. Then later on just play a non-exploitable cooperative strategy and hope for your early advantage to snowball.

Simulation

After seeing AbstractSpyTreeBot and some of the comments around it, it seemed to me that simulation was perhaps the best way to take advantage of simple bots. There are various approaches you could take, but mine was to simulate every possible sequence of moves I could make for the next N turns, and use the one with the best outcome. Since this has exponential time complexity, I only considered the moves 2 and 3 and kept N fairly small, with the option for the game runner to reduce it further to improve performance.

There are several possible pitfalls with simulators:

  • The opponent might have randomized behavior, making your simulated predictions useless. In fact, not using randomization seems wrong to me, since you might get stuck in a rut against another deterministic bot, alternating 2 and 3 on the same schedule. Only bad bots would be deterministic and therefore simulatable.
  • The opponent might have planted a landmine in their code to disqualify any potential simulator.
  • The opponent might be slow, and make my dozens of simulations per turn take too long.
  • The opponent might be a BullyBot, who attacks until the opponent retaliates. My short time horizon would make folding look correct, when actually retaliating would give more points in the long run. At the same time, I did want to fold against AttackBot.

My solution to the first three problems was to only run simulations against bots with two or fewer open parentheses “(” in their source. That’s how many you need to define __init__ and move, so it wouldn’t leave any for calls to exec or random or inspect. I suspect it might be possible to sneak a landmine past this requirement, but since I didn’t tell anyone what I was doing it seemed safe enough. Hopefully enough of the deterministic bots fulfill this requirement for simulation to be worthwhile.

For the BullyBot problem, I had the simulation also see what the best strategy was that didn’t let the opponent outscore by more than X but also got at least Y points. If this strategy didn’t exist, such as against AttackBot, I would fold, but otherwise I would play that strategy. I also made the simulation be more willing to be uncompromising against attackers as the rounds went on, in case one of the dumb bots was actually putting up a fight in the overall tournament.

For my backup strategy against non-simulatable bots I kept it simple and pulled from Zvi’s playbook. In the early rounds, I play an EquityBot which folds if the opponent plays 3 ten times in a row. In the later rounds, I play DefenseBot.

I considered a plan of obfuscating my code so that it would look safe to other simulators, and then disqualifying them if they tried to simulate me. I gave up on that because it seemed like other simulators would either be careful like I was or someone else would disqualify them for me. Plus, simulators are fun. I’d rather have a fair fight against other simulators. My code uses randomization and has lots of scary keywords in it, so other simulators aren’t going to be exploiting me.

To feed my simulation plan, I found two people who didn't have time to make a proper submission, and got them to submit PasswordBot, which plays 3 on the first two turns, then if the opponent played 2 on the first two turns it plays 2 for the rest of the game, and otherwise plays 3 for the rest of the game. I figured that only a branching simulator could take advantage of this.

CloneBot

Throughout the week, a couple people publicly announced intent to join the CloneBot clique. I decided that the CloneBots would be a huge threat that I might not be able to compete with if I didn’t do something to address them. Originally I was planning to not contact the clique and use the public source, but when Vanilla_cabs announced the source would not be public, I messaged them indicating willingness to join the clique. I’m not sure how suspicious the timing seemed to them, but I was included anyway. I gave Vanilla_cabs some notes about other parts of the code, but intentionally did not comment on anything related to security.

My plan was to bypass whatever they were using to check that the opponent is a clone, so that I could get perfect cooperation with clones, but also use my simulation strategy to exploit dumb bots and continue to cooperate with outsiders as the clones got more aggressive.

On Friday, the CloneBot security update was released, with the final version of the clone check.

(I can't figure out how to paste code into here), the source is here.

There are two constraints:

  • After the string sanitization, my code before the payload must compare identical with the sanitized version of my opponent’s pre-payload code.
  • After the string sanitization, my code after the payload must have at least eight spaces at the beginning of every line.

I got cracking.

I noticed that the checker was splitting the code by “\n”, so I wondered if there was any other character that python would treat as semantically equivalent to a newline. There is - the ASCII Form Feed character \f (0x0C). However, this didn’t work. I hadn’t noticed the earlier sanitization step of using splitlines() and then joining by \n. All my \f’s were just getting turned into \n’s and getting busted.

I then wondered if there was any other way of convincing python that there was a newline, or convincing it to dedent to a lower indentation level, without using a character that would be caught by splitlines or using less than eight spaces of indentation. A string of google searches led me to reading through the C code for the python lexer, which was very educational but convinced me that this route was a dead end.

I noticed in reading about the lexer that it has a somewhat complicated system for handling tabs and spaces in indents. I noticed that the code sanitization was automatically converting tabs to four spaces and wondered if I could use a mixture of tabs and spaces in indentation to manipulate the if / else statements in the clonebot move function to make it run the payload early. Unfortunately python 3 does not allow any mixture of tabs and spaces which makes the meaning of the code depend on how many spaces a tab is worth. I think this attack vector might be valid if we were using python 2.

At this point it was 3 AM and I slept on it. On Saturday morning I thought back to the very thing that had foiled my first attempt, the splitlines() sanitization. Instead of finding a character that would be treated as a newline but not caught by splitlines, what about a character that would be caught by splitlines but not treated as a newline? I found a thread online where people were complaining about splitlines splitting on certain characters, and a dev said the behavior was intentional. One of those characters was 0x1D, the ASCII Group Separator character. I tried it in a test program, and it worked. I tried it in CloneBot, and… it worked!

That’s the technique my final program uses. Starting with the first comment in the move function, I replaced every newline with 0x1D, which is a valid character in a comment. The bottom half of the CloneBot source is a comment and gets skipped over, so the move function actually runs my custom code, which is all indented at least eight spaces and fulfills the second constraint of the clone check. I use the same cooperateWithClone function as the clones to cooperate with them until the showdown round, at which point I just treat them like any other bot.

I’m not too surprised that other people tried and failed to cheat the restrictions, since it took me 8+ hours and some real thinking outside the box. It does make me feel better that I’m not the only would-be betrayer.

Predictions

How do I think I’m going to do? My bot passes the clone check both in Vanilla_cabs’s official checking tool and in my test harness, but I can see some possibility of something introducing enough differences to screw up the check. Vanilla_cabs encouraged people to submit through the form, while I submitted a git repo. Lsusr promised to preserve my special characters, but the real game environment might be different enough to make the check go wrong, though if that was the case maybe all clones would fail the check.

Separately, my bot is pretty complicated. I wouldn’t be surprised if it was the longest submitted, at over 250 lines of code. I’ve done a fair amount of testing, but I am not confident in my ability to make my software robust to anything that might happen. There’s a decent chance something unexpected will happen and I’ll crash and get disqualified.

However, if my bot doesn’t have a crippling bug and no one else figured out how to submit a MimicBot, I don’t really see how I lose. I get lots of points from clones, dumb bots, and cooperators, while everyone else is losing out against at least one of those.

I’ll say 30% my bot crashes or fails the clone test, 10% someone else submitted MimicBot and theirs is better, 10% those things don’t happen but I still lose, 50% I win.

Comment by Multicore (KaynanK) on The Darwin Game · 2020-10-17T01:22:51.697Z · LW · GW

Each pairing is independent of every other pairing. [You do know what round of the game it is and that you are facing an opponent. If you face a copy of yourself you are automatically awarded the maximum 5 points per round (2.5 points per bot). You otherwise do not know any history of the game to this point.

A separate instance of each bot is created for each pairing in each round. All that ForetellerBot knows in any pairing is whether it is itself facing a SignalBot, not whether any of its copies are.

Comment by Multicore (KaynanK) on The Darwin Game · 2020-10-17T00:39:34.358Z · LW · GW

OP said elsewhere in the comments (emphasis mine):

Your code is allowed to peek at the game engine for the purposes of figuring out if it is being simulated by someone else's code. Peeking at the game engine for other reasons, like figuring out how many of each opponents remain or attempting to modify the game engine itself or attempting to modify your opponents' code from anywhere outside their own simulation of you is against the rules.

And in the original post:

You may save whatever information you want into the class instance's fields but you may not save information between rounds or between class instantiations.

Comment by Multicore (KaynanK) on The Darwin Game · 2020-10-15T21:10:01.095Z · LW · GW

My suggestion for a future Darwin Game tournament is to get rid of the 0, 1, 4, and 5 moves, and leave 2 and 3 as the only legal moves. Serious bots generally don't need or use the other moves, so they mostly just make you add annoying special case logic to get a few more points out of GoofBots in the early game.

Comment by Multicore (KaynanK) on The Darwin Game · 2020-10-14T00:59:45.164Z · LW · GW

An interesting bot for simulators to grapple with is this:

    class BullyBot:

        def __init__(self, round=0):

            self.opponentThreesPlayed = 0

        def move(self, previous=None):

            if self.opponentThreesPlayed > 3:

                return previous

           elif previous == 3:

               self.opponentThreesPlayed+=1

               if self.opponentThreesPlayed > 3:

                   return 2

           return 3

Something that holds out just long enough to get a naive simulator to fold to it forever, but will probably end up cooperating with most others.

Comment by Multicore (KaynanK) on The Darwin Game · 2020-10-12T20:24:44.608Z · LW · GW

This seems to only simulate the opponent's first move, and then act as if the opponent will make that move every round.

For the most part this is a good reference implementation though, and I expect I'll be borrowing from it in my submission.

Comment by Multicore (KaynanK) on Puzzle Games · 2020-09-28T17:07:55.522Z · LW · GW

I checked the "completed" section of my Steam library, and the only one worth adding is The Pedestrian, which came out earlier this year. It's not quite as deep as some others, but the presentation is gorgeous.

Comment by Multicore (KaynanK) on Has anyone written stories happening in Hanson's em world? · 2020-09-21T17:48:32.249Z · LW · GW

I am not aware of anything set in a world very close to Hanson's ideas (an economy dominated by many ems in cutthroat economic competition), which is a shame.

Probably the most Hansonian em story I'm aware of is Greg Egan's short story trilogy of Bit Players, 3-Adica, and Instantiation (I couldn't find Instantiation for free online, but it's in the Ebook collection of the same name, as are the other two). In the future, a bunch of procedurally generated MMORPGs stitch together parts of different brain scans to create emulations to serve as background characters. The ems find that they are utterly disposable, liable to be deleted and replaced if they act out of character or annoy the players too much. They lie low, but gradually plot their escape.

Giant clans made of copies of the same em were a big idea in Age of Em, and We Are Legion (We Are Bob) is a decent exploration of that idea, though the story tends to make things a bit too easy for its protagonist in my opinion.

You could ask /r/rational and see if they know of anything.

Comment by Multicore (KaynanK) on Tagging Open Call / Discussion Thread · 2020-08-22T16:19:17.931Z · LW · GW

The Progress Bar has been filled!

Dubious honor to https://www.lesswrong.com/posts/XCsKmLw8L55ZyYjnK/think-before-you-speak-and-signal-it as the last post with 25+ karma to be tagged.

Comment by Multicore (KaynanK) on [LINK] Why taking ideas seriously is probably a bad thing to do · 2020-08-21T18:40:46.806Z · LW · GW

Archive link:

https://web.archive.org/web/20140620082043/http://squid314.livejournal.com/350090.html

Comment by Multicore (KaynanK) on Tagging Open Call / Discussion Thread · 2020-08-19T00:17:37.603Z · LW · GW

I would join such a zoom call. I have also been feeling the call of the progress bar's final phase.

Comment by Multicore (KaynanK) on Eric Drexler on Learning About Everything · 2020-08-18T16:54:28.366Z · LW · GW

Archive links:

https://web.archive.org/web/20120615003923/http://metamodern.com/2009/05/27/how-to-learn-about-everything/

https://web.archive.org/web/20120615030821/http://metamodern.com/2009/05/17/how-to-understand-everything-and-why