Supposing you inherited an AI project...

Can someone please let me know why this is the most down-voted I have ever been since de-lurking on this site? I'm not whining, I genuinely want to know what intellectual standards I'm not meeting or what social rules I'm violating by posting this.

My goal in posting this was to identify possible dangling units within the friendly AI concept.

Why don't you read what's been said on this site and elsewhere about Holden Karnofsky's Tool AI, since that is what you are apparently trying to describe.

It gives remarkably accurate answers for well-formulated questions. Where there is a theoretical limit to the accuracy of an answer to a particular type of question, its answer usually comes close to that limit, plus or minus some tiny rounding error.

Note: This is rather long. Sorry about that, if it's too long, but I was making notes on a comparable question recently, and you did say to be specific.

Well, the last time I was thinking of features in an AI, was one that could ask questions that could potentially cleave answer space for any poorly formulated request(Because as a well known problem, there is far more processing time and programmer effort spent on poorly formulated requests then well formulated ones) and show a video demonstration of some likely behaviors prior to your answering assuming what it considered the most plausible potential answers.

1: Incredibly simplistic example: you request the AI: "Make me a Line graph of these numbers."

The AI responds: "Do you want a Line Graph, with number set one as the X axis, which would look like this, or do you want a Line Graph with number set two as the X axis, which would look like this, or are you thinking of something else?"

Without that feature, a program might take that and just go "ERROR: Insufficient parameters." Or, worse, they'll give you the WRONG line graph, and they won't tell you. Both are annoying.

2: More complicated request example: you request the AI: "Bring a drink inside the fridge to me."

The AI may respond: "Do you want me to bring an entire two liter bottle of soda, which would look like this, or do you want me to bring a can of soda, which would look like this, or do you want me to drag the entire fridge to your position, which would look like this, or are you thinking of something else?"

This is where the video demonstration comes in handy. Relative to you, the AI doesn't have common sense. It doesn't necessarily know that you probably don't want it to drag the entire fridge to your position to the same level of confidence you do. It just puts together some plausible answers and lets you pick one.

(Note: There is nothing particularly special about video, other than that it shows that the AI shouldn't start actually doing things until you've narrowed the request down, and that some requests have mistakes too complicated to be picked up by text only answers.)

Of course, in the previous case, the AI was mostly on the money. It's also possible that the AI doesn't have the slightest idea what you're are talking about in a dangerous way.

3: You request the AI: "Subdue the crazed man inside the building of innocents."

The AI may respond: "Do you want me to level the building with TNT, which would look like this, or do you want me to gas the building with Chemicals, which would look like this, or do you want me to concoct an Contagious All purpose, Anti-Human Virus and infect him, which would look like this, or are you thinking of something else?"

Whoops. You forgot to tell the AI to subdue the crazed man WITHOUT hurting the innocents. Thankfully it clearly demonstrated what plans it had put together before it implemented any of them, because that would've been bad. You may have also not noted that you wanted the man subdued non-lethally.

This also helps (not prevents entirely, of course) guard against the AI just doing something completely inexplicable relative to you.

4: "What's 2+2?"

"Is the answer 4, like this, or do you want me to build a solid sphere of Computronium advancing at lightspeed and continue calculating, like this, or are you thinking of something else?"

If you wanted the AI to be better, you could network it to a cloud database so it could learn "In 1000 cases When people ask for a drink inside the fridge, they never mean drag the fridge to them, I should stop giving that as a likely answer, and add 'Would you like me to pour your drink in a glass? which seems to come up quite a bit as an answer selected from Something else."

Since danger goes up as capability does, I'd probably want to train a dumber AI on a great deal of 1 and 2 before ever getting anywhere near 3, let alone something as existential as 4.

Of course, if the AI is substantially good enough at ANY request, it can respond to a request of:

5: "Increase your computing power."

With "Do you want me to increase my computing power by building more cores like this, or do you want me to increase my computing power by switching out my existing cores with this design that I came up, like this, or are you thinking of something else?"

In which case you have a Self-Improving AI that, after a great deal of training, gets what humanity wants when they tell it things, so it can be enlisted to save the world and make everyone live happily forever.

Except, that what this DIDN'T do, is it didn't actually follow the procedure that I laid out above, and give an alternate option under the assumption that this isn't what you asked for(and it might be very dangerous), and possibly ask if you just wanted something else entirely.

I want to note that those steps should be here, but have mainly been cut for space constraints, except as rough mentions.

This sounds like almost nothing, reminds me of the person who wrote a command-line interpreter and language interpretation/synth library, leaving only the problem of figuring out how to write code to come up with intelligent responses to questions. Frankly, what is described here this sounds like something that guy in my robotics class could write in a week in Python. In fact, this sounds suspiciously like an assignment for an elementary programming class for non-computer-science majors that I took.

(This assumes that A.1 and A.2 are NOT provided!)

You still need to write some drivers and interpretation scripts, on top of which go basic perception, on top of which go basic human thoughts, on top of which go culture and morality, on top of which go CEV or whatever other major human morality system you want to use to make the AI friendly.

It also... sounds like this thing doesn't even search the hypothesis space. Which makes it much safer and much, much less useful.

Edit: Actually, I realize that this is more substantial, and wish to apologize for condescension. But the OP still sounds like the job is not being sliced even slightly in the middle, and like it would take a lot of time, work, and additional stuff to make something simple and useless like a chatterbox.