Comment by Tirumal on Pausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky · 2023-03-30T13:44:47.732Z · LW · GW

Sigh, meanwhile GPT4 can’t arrange months in alphabetical order based on their 4th letter: GPT4 Prompt:

“ Arrange the months in alphabetical order based on their 4th letter

To arrange the months based on the 4th letter in their names, please see the following list:

April (A_p_r_il)

September (S_e_p_tember)

December (D_e_c_ember)

February (F_e_b_ruary)

January (J_a_n_uary)

June (J_u_n_e)

July (J_u_l_y)

November (N_o_v_ember)

August (A_u_g_ust)

October (O_c_t_ober)

May (M_a_y_)

March (M_a_r_ch)

This list starts with April, which has "r" as its fourth letter, and ends with March, which has "r" as its fourth letter as well.

“ When you start probing adversarially you can find many simple tasks that GPT4 sucks at, so let’s maybe just calm down.

Comment by Tirumal on What's the Least Impressive Thing GPT-4 Won't be Able to Do · 2023-03-19T23:44:43.186Z · LW · GW

I found one, GPT4 struggles with forming given a set of letters, for example asking it to form a word from the letters of “Croatia” , gave me the answer Carotid which is clearly wrong “aortic” is one solution. Seven letter word from “solution”, it gives me outlins (not a word)