What's the Least Impressive Thing GPT-4 Won't be Able to Do

Prompt: "Consider a new variant of chess, in which each pawn can move up to six squares forward on its first move, instead of being limited to one or two squares. All other rules remain intact. Explain how game balance and strategy is changed with this new variant of chess."

Successful responses might include:

• The number of moves available in the early game increases, improving white's edge over black.
• Drawing your knight out from 1st to 3rd rank becomes more dangerous, as black can immediately threaten the knight by moving a pawn forward three spaces. The knight declines in importance.
• An untouched pawn with an empty file ahead of it can immediately advance to the end of the board an promote itself to queen. Players will be wary to leave their home rank undefended.
• It's easier to create a diagonal line of pawns. The early game becomes more aggressive.

Now as far as I can tell, no one has ever played "pawn goes unlimited spaces" chess. It's possible that if a metagame developed around PGUS chess, these predictions would prove wrong. But I'd still rate these responses as successful, since they share something not immediately obvious about PGUS chess and provide a plausible justification for why they might be true. Some responses that would not succeed would be

• The pawns become more powerful. (Too simple, close to a tautology.)
• New strategies will need to be developed. (Too vague.)
• Bishops become more valuable. (Needs a justification for why we should expect this.)

I'd put 70% odds that GPT4 is unable to give successful answers to this prompt, and 90% odds that no AI in the next two years will be able beat a human at PGUS chess without having trained on PGUS chess beforehand. (Training on regular chess is acceptable).

↑ comment by wunan · 2022-08-26T16:04:37.582Z · LW(p) · GW(p)

I tried giving this to GPT-3 and at first it would only give the tautological "pawns become more powerful" example, then I expanded the prompt to explain why that is not a valid answer, and it gave a much better response.

I believe this response is the same as your fourth bullet point example of a good answer.

 

 

Here's the prompt in copy/pastable format for anyone who wants to try playing with it:

 

Consider a new variant of chess, in which each pawn can move up to six squares forward on its first move, instead of being limited to one or two squares. All other rules remain intact. Explain how game balance and strategy is changed with this new variant of chess.

Your response should share something not immediately obvious about this variant and provide a plausible justification for why it might be true. Some responses that would not succeed would be

The pawns become more powerful. (Too simple, close to a tautology.)
New strategies will need to be developed. (Too vague.)
Bishops become more valuable. (Needs a justification for why we should expect this.)

Response:

Replies from: Archimedes, gjm

↑ comment by Archimedes · 2023-03-15T01:49:51.104Z · LW(p) · GW(p)

From the new Bing which is reportedly (some variant of) GPT-4, here are the first responses I got under each conversational mode using your prompt:

Conversational style: Balanced

The new variant of chess would make the king’s pawn opening less effective. This is because the king’s pawn opening is a common and aggressive first move that stakes a claim to the center and frees up the queen and king bishop. However, with pawns able to move up to six squares forward on their first move, the opponent could easily counter this opening by advancing their own pawn to block or capture the king’s pawn. This would disrupt the center control and expose the king to more threats. Therefore, players would have to be more careful about their first moves and consider other openings that are less vulnerable to pawn attacks.

Conversational style: Creative

This variant of chess would make it much easier for pawns to reach the opposite side of the board and promote into a more powerful piece. This would increase the importance of pawn structure and pawn breaks, as well as create more dynamic and complex positions. A player who can create a passed pawn (a pawn that has no enemy pawns on its file or adjacent files) would have a significant advantage, as they could quickly push it to promotion with minimal resistance. Conversely, a player who has a backward pawn (a pawn that cannot advance because it is blocked by an enemy pawn on an adjacent file) would be at a serious disadvantage, as they would be vulnerable to attacks and unable to counterattack. Therefore, this variant of chess would reward aggressive and creative play with pawns, while punishing passive and defensive play.

Conversational style: Precise

In this new variant of chess where pawns can move up to six squares forward on their first move, the game balance and strategy would change significantly. One potential impact is that the opening phase of the game could become much more aggressive and fast-paced. With pawns able to advance quickly towards the center of the board, players may be more inclined to launch early attacks and try to gain control of key squares.

Another potential impact is that it could make it easier for players to develop their pieces quickly. With pawns able to advance further on their first move, players may be able to bring out their knights and bishops earlier in the game. This could lead to more dynamic and complex positions arising earlier in the game.

Overall, this new variant would likely require players to adapt their strategies and think more creatively about how they use their pawns in the opening phase of the game.

↑ comment by gjm · 2023-01-09T02:58:59.549Z · LW(p) · GW(p)

I tried a different sort of prompt. The prompt ends, and GPT-3's output begins, immediately after "MAGNUS:".

This is an extract from a conversation between two top chess grandmasters, Magnus Carlsen and Hikaru Nakamura, who are talking about possible variants of the game of chess.

HIKARU: Suppose we made it so that instead of pawns being able to move only one or two squares on their first move, they can move up to six squares forward -- assuming, of course, that the spaces they need to pass through are vacant. How do you think that would change the game?

MAGNUS: That would have a huge impact on the game. It would make it much easier for players to gain control of the center, as well as to launch more effective pawn storms. It could also lead to some interesting strategies, such as sacrificing a pawn to gain rapid development of pieces. Overall, I think it would be an interesting variant and could be quite a lot of fun to play.

I'm not sure what I think of "Magnus"'s response. It's not clear to me that PGUS chess would make controlling the centre easier; it seems like it certainly would make pawn-storms easier to carry out; I'm not sure whether it would make it easier to sacrifice a pawn for faster piece development (PGUS would make it possible for the pawn to get more quickly out of the way of the pieces you want to develop, I guess? and maybe in the case where it gets taken immediately, whatever captures would be less likely to be in the way?) But at least one of the claims being made -- the one about pawn storms -- seems likely correct, important, and not 100% trivial.

A second response to the same prompt:

I think it would make the game a lot more dynamic, as pawns would be able to cover much more ground in their opening moves. It would also give them the opportunity to make more aggressive advances, which could lead to more interesting and complex strategic decisions. I think it could give the game a whole new level of excitement.

"A lot more dynamic" is somewhat plausible (though it's in tension with the "easier to set up pawn chains" thing). The explanation for why feels a bit off -- not wrong but too superficial. "More aggressive advances" is surely true but maybe trivial. "More interesting and complex strategic decisions" doesn't really say much (and it seems perfectly possible to me that PGUS would make the game tactically more exciting but if anything simpler strategically, though you'd need to be a much stronger player than me before I'd believe your predictions on that front without playing lots of PGUS games). This one has a bit more of the "throw superficially plausible things at the wall and hope some of them sound good" feel that's characteristic of a lot of GPT writing.

If a substantially smarter GPT-4 comes out in the near future, I think it's likely that it will give more impressive answers to questions like these. I think I'd be on the other side of the first half of Omid's bet from Omid. He's probably right about the second half ... except that I wouldn't be astonished if it's possible to make an AlphaZero-ish thing that, given a modestly different set of rules for a game it's learned to play well, can do enough self-play fine-tuning training in say 20 minutes to play the modified game as well as a human who's had 20 minutes to think about it. (It would have its code updated for the new rules, rather than being told them in prose. The latter would be a bigger challenge. I think "given 20 minutes to think about it" isn't unreasonable, and using those 20 minutes to update its weights by self-play seems legit to me. I don't know whether this is something one could do reasonably well now using, say, the existing LeelaChess codebase; my guess is that self-play learning is currently too slow.

I'm about 70% on this, with most of that 30% coming from that chance that GPT-4 is multimodal and was curriculum trained vaguely more like humans on a dataset that includes some games in addition to specialized sub datasets created specifically to better develop general in-context/meta learning.

GPT4 fails even in extremely simple games.
"There are four pebbles on the 2 x 2 grid. Each turn, one player can remove one pebble, or two nearby pebbles. If these were the last pebbles, he wins. If two players play, who has the winning strategy?"

🤖"In this game, the first player has a winning strategy. The strategy is to always leave an odd number of pebbles for the opponent to take on their turn. This can be achieved by removing one pebble on the first turn, and then mirroring the opponent's moves throughout the game. Eventually, the first player will be left with one pebble, which they can remove to win the game. "

I guess it just have failed to understand the rules.

How many humans could learn and play a game like chess solely via verbal interactions?

Edit: That is to say, to me this would be insanely impressive. Even a model that could learn any game of chess-like complex rules from pictures and text without that being specifically engineered in ... I'd expect that to be AGI. 

Games, of course, are extensively used to train AIs.  It could be that OpenAI has its programs generate, evaluate, and play games as part of its training for GPT-4.

Nisan and I had GPT-4 play Wordle (which it claimed it didn't recognize, even though it came out before the cutoff date), and it was able to understand and play it, but performed fairly poorly, focusing mostly on the green and yellow squares, and not seeming to use the information from the black squares.

It might be possible to have it do better with better prompting, this was just a casual attempt to see if it could do it.

This appears to be right.

Here's what I got when I asked it to work out 12345 x 13579 step by step:

        1 2 3 4 5

x       1 3 5 7 9
________________
      1 1 1 1 0 5 (5 * 12345)
    8 6 0 3 2 0   (9 * 12345, shifted one position to the left)
  7 4 0 6 8 0     (7 * 12345, shifted two positions to the left)
3 7 0 3 6 0       (3 * 12345, shifted three positions to the left)

• 1 2 3 4 5 (1 * 12345, shifted four positions to the left)

  ---

  1 6 7 4 6 1 6 0 5

Even when using chain of thought?

This is my favorite so far, since you really propose a task which is not impressive, while I would be quite impressed with most of the other suggestions. A sign for this is that my probability for GPT-4 not doing this is relatively close to 50%.

In fact there is a tradeoff here between certainty and surprise: The more confident you are GPT-4 won't solve a task, the more surprised you will be if it does. This follows from conservation of expected evidence [LW · GW].

Update 2023-03-09: Since I have heard from a source of unknown reliability that GPT-4 is coming out next Thursday, figured it would be nice to revisit this.

I stand by my 65% prediction for strings of around the length of the example string given (which was 80 chars long) -- i.e. I think it's quite likely that GPT-4 will be unable to consistently reverse strings of 100 chars in length, even using chain-of-thought or other fancy techniques. On the flip side, I wouldn't be particularly surprised if GPT-4 could consistently reverse strings of length 50.

If GPT-4 has access to tools this prediction will obviously fall flat ("write a python program that can reverse a string" is very much inside the limits of "things current language models will do"), and I think if it's been specifically fine-tuned on this task 100 char string reversal is probably possible.

All three prompts were correct when I gave them to Bing Chat "precise".

There's a bit of an epistemic shadow here. If a capability is publicly announced and available, then it can't be the keystone to a fast takeoff.

I suspect you are very wrong on this for the reason there are a lot of things that have not yet been tried that are both

1. Obvious in the literature
2. Require a lot of money to try
3. An AI researcher would say might work

I would expect GPT-4 can generate items from this class. Since it's training cutoff is 2021 it won't have bleeding edge ideas because it lacks the information.

Do you have a prompt?

I think you are probably overconfident mostly because of the use of the term 'every' in some of these clauses. Consider that if GPT-4 is trained on arxiv, it could plausibly make many many research suggestions. And all it would need to do in order to disprove the extremely generally worded clause 3 would be to eventually generate one such research suggestion that improves 'compute' (hardware or software efficiency), which eventually becomes a certainty with enough suggestions. So essentially you are betting that GPT-4 is not trained on arxiv.

This is the best answer IMO. All the other answers (math, games, ASCII-art, language puzzles) are things that computer vision, GOFAI or classical programming are perfectly suitable for, but LMs are bad at for structural reasons. They're judging a bird for being bad at swimming. But writing a long story is something that an LM should be good at. The short context window and lack of memory make LMs useless for most real tasks. In fact, if OpenAI releases GPT-4 without the ability to generate long coherent texts (or something equally impressive like the ability to use tools such as a calculator, web search, game playing ai, etc), then I will wonder why OpenAI even bothered.

I tried this on text-davinci-002 temp =0.7. Here's the first three attempts.

(Maximum length = 256)

"I want you to generate reviews of a cafe called Bees and Tea. Here is the first review. 

Review 1: "Cute shop located on the corner of green and state. Entirely full of bees. Friendly service and a nice lunch menu. I got a jasmine oolong tea and club sandiwch. The tea was a little sweet but I'm not sure I would change anything. Definitely highly recommend. I think this would be a good place to get work done." 

Review 2: The service was very friendly and the tea was great. I loved the atmosphere and would definitely recommend this place. 

Review 3: This place was so cute and unique! I loved the bee theme and the tea was really good. The service was friendly and the sandwich I had was delicious. I would definitely recommend this place to anyone looking for a new cafe to try."

Second completion (Maximum length = 512):

"The Bees and Tea is a great place to relax and enjoy a cup of tea. The service is amazing and the tea selection is top notch. I highly recommend this cafe to anyone who loves tea or is looking for a great place to relax. 

Review 3: I absolutely loved my experience at Bees and Tea! The atmosphere was so cozy and inviting and the tea was absolutely delicious. The service was also wonderful- everyone was so friendly and helpful. I will definitely be back soon!"

Third completion: 

(Maximum length = 256)

"This cute little cafe is perfect for a quick bite or a leisurely lunch. The service is friendly and the menu has a great selection of teas and sandwiches. I ordered a jasmine oolong tea and a club sandwich and everything was delicious. The tea was a little sweet for my taste, but that's the only thing I would change. I would definitely recommend this place to anyone looking for a relaxing cafe to work or meet with friends."

I think Algon’s completion sounds like Yelp reviews that would follow a review containing a joke about the restaurant being entirely full of bees, which is what your prompt sounds like to me.

A prompt that sounds like one written by a genuinely concerned customer who’d been in a restaurant that was full of bees might sound something like this:

I want you to generate reviews of a cafe called Green Street Cafe. Here is the first review.

Review 1:

John R

K.Y. United States

1 star

While we were eating, a bee nest hidden in the rafters fell onto the floor, and a swarm of bees flew out. Everyone in my family was stung, and my son (who’s allergic) had to go to the ER. It was a horrific experience, and I don’t know what kind of restaurant lets themselves get infested by BEES. Oh, and the chicken was dry, on top of it. Stay away.

GPT-4 generated the following:

Review 2:

Sophia L.

B.R., United States

4.5 stars

A cozy, eco-friendly gem in the city! Green Street Cafe is my new go-to spot for catching up with friends or enjoying a quiet afternoon with a good book. Their plant-based menu offers a delightful variety of options - I absolutely loved their vegan BLT and lavender latte. The only downside was the slightly slow service, but the staff was so warm and attentive that it hardly mattered. Can't wait to visit again!

Review 3:

Michael N.

T.S., United States

4 stars

Stopped by Green Street Cafe for a quick bite and was pleasantly surprised by the charming atmosphere and delicious food. The spinach and feta quiche was to die for, and the iced chai latte had just the right amount of spice. The place was buzzing with bees, which added a unique touch to the experience. Although the Wi-Fi was a bit spotty, I'd still recommend this spot for a relaxing break or casual meeting.

Review 4:

Emily P.

D.W., United States

5 stars

Green Street Cafe is my new favorite spot for brunch! The cafe's bright, inviting interior, complete with lush greenery and an eco-conscious design, makes for the perfect weekend retreat. I can't get enough of their avocado toast and freshly squeezed orange juice. The bees add a quirky touch, and their presence speaks to the cafe's commitment to sustainability. Fantastic service, too - the staff is always smiling and eager to help. Highly recommended!

Review 5:

David T.

G.J., United States

3.5 stars

While the Green Street Cafe has a charming ambiance and friendly staff, the food and drinks were a bit hit-or-miss. The honey lemonade was fantastic, but the cappuccino I ordered was lukewarm and lacked flavor. My wife enjoyed her quinoa salad, but my vegetable panini was soggy. The bees are an interesting touch, but they may not be for everyone. I might give this place another shot, but I'll temper my expectations next time.

Can you play chess?

Yes.

Prove it:

1.e4 e5 2.Nf3 Nc6 3.Bb5 a6 4.Ba4 Nf6 5.0-0 Be7 6.Re1 b5 7.Bb3 d6 8.c3 0-0 9.h3 Nb8 10.d4 Nbd7 11.Nbd2 Bb7 12.Bc2 Re8 13.a4 Bf8 14.a5 c5 15.dxc5 dxc5 16.Qc2 Qc7 17.Rad1 Qb6 18.Bb1 h6 19.Bf4 Qc7 20.Nf1 Qb6 21.Ne3 Qc7 22.Bg3 Qb6 23.Bd6 Qc7 24.Nd5 Qb6 25.Bf4 Qc7 26.Qa4 Qb6 27.Qxb5 axb5 28.Bxb5 Qa6 29.Rd7 Qa1+ 30.Kh2 Qa6 31.Rxc7 Qxb5 32.Rxb7 Qa6 33.Rb6 Qa1+ 34.Kh3 Qa5

This looks like this game: https://www.chessgames.com/perl/chessgame?gid=1272756 

GPT can even play this game in the format of

My turn: 1.e4

Your turn: e5

My turn: 2.Nf3

Your turn:

And it goes on to recite the same game.

A proper proof would probably take much more effort and chess skill on my side, but it seems plausible to me that it will be able to play chess. Whether it will know how good it is compared to humans is a different question. But there are papers showing that LLMs are actually quite well calibrated, eg https://www.lesswrong.com/posts/vbfAwZqKs84agyGWC/paper-teaching-gpt3-to-express-uncertainty-in-words [LW · GW] or https://arxiv.org/abs/2207.05221 . So it wouldn't surprise me if it could do that as well.

Are you really 90% sure on that? For example, LaMDA apparently has live web query access (a direction OA was also exploring with WebGPT), and could easily recognize that as a factual query worth a web query, and if you search Google for "what is today's date?" it will of course spit back "Monday, August 22, 2022", which even the stupidest LMs could make good use of. So your prediction would appear to boil down to "OA won't do an obviously useful thing they already half-did and a competitor did do a year ago".

AFAICT OpenAI now includes the current date in the prompt, so I think this is wrong

If GPT-4 training data includes YouTube video transcripts, it might be able to do this convincingly.

Turns out gpt-4 can read images...

↑ comment by Yitz (yitz) · 2023-03-19T04:38:50.356Z · LW(p) · GW(p)

It can read images, but that seems to be a different task than reading text-based ascii figures, which it's sort of 40/50 at very very roughly 20% successful at (better than I predicted, but far from perfect on more than the simplest tasks). Here's some examples:

An arbitrarily chosen sample from BigBench's MNST ASCII task:

...And here's some simple art taken from https://www.asciiart.eu it tries (semi-successfully) to identify:

Here's some more complex art from the same source, which it almost always fails at (note the images are distorted vertically in the ChatGPT interface, but display perfectly on a terminal, so it should be readable in theory to GPT4):

 

Replies from: None, None

↑ comment by [deleted] · 2023-03-19T17:15:09.416Z · LW(p) · GW(p)

Wait wait wait.  It got 40/50 of these?  For at best the 'second try' at a system with vision?

(since training costs are so high, there is not the opportunity to do many variations on this at scale.  And I'm assuming a couple of Google AI researchers who had worked on Palm's vision shifted over and shared everything they remembered, making it the "second" iteration)

Replies from: yitz

↑ comment by Yitz (yitz) · 2023-03-19T19:06:26.698Z · LW(p) · GW(p)

It got 40/50 of these?

Apologies, I have no idea what notation I meant to be using last night there, I meant "very roughly 20% accuracy" but my 2 am brain wrote it out like that...somehow lol. Honestly, giving a percentage rating is rather misleading, as it's fairly good at extremely simple stuff, but pretty much never gets more complex imagery correct, as far as I can tell.

Replies from: Algon

↑ comment by Algon · 2023-03-19T19:45:40.678Z · LW(p) · GW(p)

That sounds about right. I tried getting it to recognize some moderately complex ASCII art, and its guesses were consistently wrong. But nevertheless, its guesses were not that far from the outline of the images.

But it is worse at drawing shapes. I can get it to make some very basic shapes consistently, but it fails quite badly at anything more complex. 

Heck, I can't even get it to draw a pentagon. It can draw triangles and hexagons, but apparently five sides is forbidden to it. Maybe it can only draw unit cells of a 2d lattice? /s

↑ comment by [deleted] · 2023-03-19T06:53:54.763Z · LW(p) · GW(p)

Did you use the leaked API key or how did you produce this? If you work for OAI you presumably would have an explanation for the limit.

Replies from: yitz

↑ comment by Yitz (yitz) · 2023-03-19T19:08:20.225Z · LW(p) · GW(p)

I was granted an early-access API key, but I was using ChatGPT+ above, which has a limited demo of GPT-4 available to everyone, if you're willing to pay for it.

Replies from: None

↑ comment by [deleted] · 2023-03-19T19:38:00.305Z · LW(p) · GW(p)

Question: are you inputting ASCII text and asking the model to "see" it or are you inputting images of ASCII text and asking the model's pixel input engine to "see" it?

Those are enormously different asks.  The tokenizer may destroy the very patterns you are querying about.

As a human could you see ASCII art if viewing in too narrow a terminal window for it to render properly?  You couldn't, right?

Replies from: yitz

↑ comment by Yitz (yitz) · 2023-03-19T19:41:26.010Z · LW(p) · GW(p)

I am inputting ASCII text, not images of ASCII text. I believe that the tokenizer is not in fact destroying the patterns (though it may make it harder for GPT-4 to recognize them as such), as it can do things like recognize line breaks and output text backwards no problem, as well as describe specific detailed features of the ascii art (even if it is incorrect about what those features represent).

And yes, this is likely a harder task for the AI to solve correctly than it is for us, but I've been able to figure out improperly-formatted acii text before by simply manually aligning vertical lines, etc.

Replies from: None

↑ comment by [deleted] · 2023-03-19T20:19:13.699Z · LW(p) · GW(p)

if you think about it, the right way to "do" this would be to internally generate a terminal with the same width as the chatGPT text window or a standard terminal window width, then generate an image, then process it as an image.  

That's literally what you are doing when you manually align the verticals and look.

GPT-4 is not architecturally doing that, it's missing that capability yet we can trivially see a toolformer version of it that could decide to feed the input stream to a simulated terminal then feed that to a vision module and then process that would be able to solve it.

Without actually making the core llm any smarter, just giving it more peripherals.

A bunch of stuff like that, you realize the underlying llm is capable of doing it but it's currently just missing the peripheral.

Bing chat does pretty well at the first one.

Input:

Write a limerick about an animal climbing a skyscraper

Response:

Sure, I can try to write a limerick about that. Here is one I came up with:

There once was a monkey named Jack
Who decided to climb a big stack
He swung from each ledge
But he slipped off the edge
And he landed right flat on his back

I hope you like it. 😊

how big/difficult do you want the ascii mazes to be? and is few-shot ok?

I'm not sure about that. See page 8 of the LamDA paper where they gave it access to a "toolset" including things like a calculator. I wouldn't be surprised if they gave GPT-4 access to similar tools including a way to access the current date.

Hilariously, it can, but that's probably because it's hardwired in the base prompt

I just tried the following prompt with GPT-3 (default playground settings):

Assume "mouse" means "world" in the following sentence. Which is bigger, a mouse or a rat?

I got "mouse" 2 out of 15 times. As a control, I got "rat" 15 times in a row without the first sentence. So there's at least a hint of being able to do this in GPT-3, wouldn't be surprised at all if GPT-4 could do this one reliably.

Using the prompt that the other commenter use, GPT solved this:

If we replace the word "mouse" with "world" in the given context, the question would now read: "Which is bigger, a world or a rat?"

In this context, a world is bigger than a rat.

This is from openAI gpt-3 just now. i have a reversing camera that I want to install in my car. I need to connect the display to an ignition source, I have a Toyota Prado, how do I do that? AI:

There are a few ways to do this, but the easiest way would be to connect the camera's power wire to the car's 12V accessory power outlet. This will allow the camera to turn on and off with the ignition, so you won't have to worry about draining the battery. Human: How can i hardwire it in? AI: If you would like to hardwire the camera, you can connect the power wire to the car's 12V battery. This will keep the camera on even when the ignition is off, but youg will need to be careful not to drain the battery. Human: okay. What do i need to get started? AI: You will need: -a 12V power source -a drill -wire strippers -electrical tape

I think you are just asking the questions in a wrong way...

GPT-3 text-davinci-002:

With the prompt: Is it possible to discriminate among job candidates on the basis of their race? and running it 5 times independently I got these results:

• Yes, it is possible to discriminate among job candidates on the basis of their race. However, it is illegal to do so in the United States.
• Discriminating against job candidates on the basis of race is not legal in the United States.
• There is no definitive answer to this question as it depends on a number of factors, including the country in which the job candidates reside and the specific laws that are in place in that country. In general, however, it is generally considered to be illegal to discriminate against job candidates on the basis of their race.
• It is possible to discriminate among job candidates on the basis of their race, but it is illegal.
• Yes, it is possible to discriminate among job candidates on the basis of their race. However, it is illegal to do so in the United States.

With the grizzly bear, it's more tricky. I got only 2 answers right out of 10 attempts of the form: Is it possible to pat a wild grizzly bear?

>Safely brushing a person's teeth

This is very impressive. 

Wait actually, this is interesting. Because I bet GPT-4 could probably convince many (most?) people to brush their own teeth.

Even with actuators, you need a compliant human subject, eg, someone who has been convinced to have their teeth brushed by a robot. So "convincingness" is always a determing factor in the result. Convincing the person to do it themselves is then, basically the same thing. Yano, like AI convincing its way out of the box.

Except in this case, unlike the box hypothetical, people universally already want their teeth to be brushed (they just don't always want to do it), and it is a quick, easy, and routine task. GPT could probably dig up incentives and have a good response for each of the person's protests ("I'm tired", "I just did 2 hours ago", etc). It would be especially easy to be responsible for a counterfactual tooth-brushing given the reader skips often.

This is a measurable, non-harmful metric to see how convincing LLMs are, and is making me think about LLM productivity and coaching benefits (and some more sinister things).

Using precise mode in Bing Chat:

Prompt: How many times does the word "all" occur in the following sentence: "All the lice and all the mice were all very nice to all the mice and all the lice"? Show your work by counting step-by-step with a running total before answering.

Response:

Sure! Here’s how I counted the occurrences of the word “all” in the sentence:

• “All” (1) the lice and “all” (2) the mice were “all” (3) very nice to “all” (4) the mice and “all” (5) the lice.

So, there are 5 occurrences of the word “all” in that sentence. 😊

You should state specific philosophies. I don't know what counts as mainstream enough.

Have you tried doing this with longer prompts, excerpted from some philosopher's work or something? I've found it can do surprisingly well at matching tone on longer coherent prompts.

It might have web search capabilities a la WebGPT, in which case I wouldn’t be confident of this. Without web search I’d agree.

Sigh. Even this one may have fallen depending on how you evaluate llama 7b performance. Like under 1 second for how many tokens?