In the message-from-the-future scenario, a lot of information is gained from the choice of information sent in addition to the pure fact. They learn that we didn't send back the alchemical formula for gold, or the cure for cancer nor warn of an upcoming disaster.

"How many bits" isn't a very well defined question. Give it 14k words to persuade you to let it out, and it might succeed. Give it 140k chances to predict "rain or no rain, in this location and time?" and it has no chance. The problem is that if it works well for that, you'll probably want to start using it for more... and more..

It's this path that's the concerning thing. So long as we're sufficiently worried about having our minds changed, we're really good at ignoring good arguments and deluding ourselves into missing the fact that the arguments really ought to be compelling according to our own ways of reasoning. When people get their mind changed (against their best interests or not) it tends to be because their guards were not up, or were not up in the right places, or were not up high enough.

The question, so far as I can tell, is whether you recognize that you ought to be scared away from going further before or after you reach the point of no return where the AI has enough influence over you that it can convince you to give it power faster than you can ensure the power to be safe. Drugs make a useful analogy here. How much heroin can you take before you get addicted? Well, it depends on how terrified you are of addiction and how competent you are at these things. "Do you get hooked after one dose?" isn't really the right question if quitting after the first dose is so easy that you let yourself take more. If you recognized the threat as significant enough, it's possible to get shaken out of things quite far down the road (the whole "hitting rock bottom" thing sometimes provides enough impetus to scare people sober).

Superhuman AI is a new type of threat that's likely very easy to underestimate (perhaps to an extent that is also easy to underestimate), but I don't think the idea of "Give it carefully chosen ways to influence the world, and then stop and reassess" is doomed to failure regardless of the level of care.

It seems pretty clear to me that if you let it give you enough bits, interpreted in a flexible enough way, to solve any problem you couldn't solve by yourself, you're taking a lot more risk than you'd be taking by playing tic-tac-toe with it, which you have already said you wouldn't be willing to do. So as far as I can see you will never be able to "use" it without at least taking what you consider to be an unacceptable chance of letting it out.

In fact, I think that you probably couldn't get a lot of use out of it without actually, truly, fully letting it out. Nor, depending on a bunch of things, would it be a very nice thing to do even if you could.

So if you believed that there was no safe way to let it out, then why would you ever build it in the first place?

I'm not sure considering how to restrict interaction with super-AI is an effective way to address its potential risks, even if some restrictions might work (and it is not at all clear that such restrictions are possible). Humans tend not to leave capability on the table where there's competitive advantage to be had so it's predictable that even in a world that starts with AIs in secure boxes there will be a race toward less security to extract more value.

I could extract a lot of value with an oracle that could only answer yes/no (or option 1/2) questions. In the limit of the operator's patience, one bit at a time is unlimited bits.

I wonder what the bounds are on the AI getting you to accept more bits?

Why would you play tie-tac-toe?

1. No reason to do this.

2. No reason for you to be the person that does this. Have a program do it.

If you answered "No" to both of those questions, then congratulations!  The safety threshold for our AI is at least 1 bit!

If an expert looks at and uses the bit, then immediately incinerates the box, this works.

Suppose one of those answers goes on to influence all sorts of parts of the world, by chaos theory. Leading to a break in by burglars who let the AI out in one of the worlds. 

Basically the chance of the AI being let out is nearly doubled in the worst case. If this chance was already not too tiny, (say a powerful faction planned to seize the AI and do something stupid with it.) this could be bad.

Or we could let it out on the grounds that >50% likely to be friendly is better than whatever else we might make.

'c=spd of light. y=1/sqrt(1-v^2/c^2).  all lengths scale down by y' is 65 characters long, or 520 bits.

So we basically need a "Twitter for AI". :D

best way to use a sharply limited number of bits of information from a probably-Friendly superhuman AI

I figure we'd want it to answer restricted (essentially multiple-choice) questions.  Come to think of it, if necessary we could play games of 20 Questions with it.

If the number of bits we can get is restricted, then we'd want to get from it the most valuable-but-safe bits possible.  Very valuable bits that we don't have would fall into two categories: (a) those we know are valuable, but haven't acquired because it's very expensive, and (b) those we don't know are valuable.  Finding (b) involves more free-form questions that could be dangerous.  But (a) seems relatively simple and easy to extract value from, for certain values of "a superhuman AI" that include "able to simulate relevant systems".  For example:

"Given these descriptions of the SARS-CoV-2 virus and the human immune system, are you able to make decent predictions on how well humans' immune systems would respond to the virus and vaccines?  Respond Y/N."

"Y"

"Here are 10 proposed vaccine formulations our researchers came up with.  If we use a two-dose regimen with the doses separated by up to 12 weeks, then what is the best combination of vaccine and dose interval for minimizing chance of death from a SARS-CoV-2 infection on a random day up to 1 year after the first dose?  Your answer should be of the form "x y", where x is the index of the vaccine (from 1 to 10) and y is the dose interval in weeks (1 to 12)."

"6 8"

One would follow this up with an actual clinical trial before giving the vaccine to everyone, of course, and maybe also try out the humans' best guess if you're suspicious of the machine.  But it seems clear that, by these steps, you'd have a lot of likely upside if the machine is good, and a relatively small downside (wasted effort and delaying the normal solution) if the machine is treacherous.

Generally, if the machine can tell you the results of expensive experiments, and you can verify them much more cheaply than doing the entire set of experiments yourself, then that's a good use case.  (This is essentially the real-world analogue of "nondeterministic polynomial" problems from computing.)

Eliezer has taken some pains to argue that we cannot even talk to the AI:... And he's provided some compelling arguments that this is the case.

Any chance you have links to those arguments?  I know that is his argument and I agree with his intuition, but I've never seen anything more fleshed out than that.

We give it only a very short window of time to make its move, so that it can't manipulate move timings.

You've already lost.

1ms is $2^{134}$ Planck times. 3 bits in a millisecond is "actually" $2^{134}\text{choose}3$ possibilities, or ~$2^{137}$ possibilities. This is ~137 bits of information.

Now the question becomes 'are any of those ~$2^{137}$ possibilities dangerous'... which we could answer if we knew the exact laws of physics, but we don't. Even if we make a Newtonian approximation: Newtonian mechanics being chaotic, it's entirely possible that one of said possibilities e.g. causes an asteroid on a chaotic tumble to hit the Earth in a couple thousand years.

'c=spd of light. y=1/sqrt(1-v^2/c^2).  all lengths scale down by y' is 65 characters long, or 520 bits.  

This a) uses a terrible encoding (8 bits / character) and b) is nowhere near as compressed as it could be even given the encoding.

Information content of human text is "only" ~9.8 bits / word^[1]. A ballpark guess then is that said 137 bits can encode ~14 words worth of information.

Is there a series of 14 words that can encode the essence of e.g. Relativity? I would not be surprised.

1. ^{^}

   https://www.sciencedirect.com/science/article/pii/S0019995864903262 

Allow it to display info on a screen. Set up a simple poleroid camera that takes a phone every X seconds.

Ask the question, take physical photos of the screen remotely.

View the photos.

Large transmission of information in analog format.

In any reasonable scenario, those communicating with the AI in a box will not be the people empowered to let it out.  Ideally, those with the capability to let the AI out would be entirely isolated from those communicating with the AI and would not be able to access the conversations with the AI. 

I would also note that restricting the number of bits (a) just makes things go more slowly and (b) doesn't work very well in the competitive real world where the other guys are less restrictive.

Ultimately, the dangers of the AI in a box aren't that it can manipulate any human to let it out but that:

(i) it's really unclear how good our boxing skills are; and 

(ii) human beings have different risk reward functions and it is entirely possible that humans will convince themselves to let the AI out of the box even without any manipulation either as a result of perceived additional benefit, competitive pressure or sympathy for the AI.

You kind of assumed away (i), but part of (i) is setting things up as outlined in my first paragraph which points to the fact that even if our boxing skills were good enough, over time we will come to rely on less sophisticated and capable organizations to do the boxing which doesn't seem like it will end well.