Is your job replaceable by GPT-4? (as of March 2023)

post by Bezzi · 2023-03-23T22:16:53.502Z · LW · GW · 6 comments

Contents

6 comments

Disclaimer: I don't have access to GPT-4

There's a lot of discussion these days about technological unemployment due to LLMs. For example, someone could observe that GPT-4 scored better than average at the US Medical License Examination, but human doctors are obviously not going to be obsolete in the short term, if only because GPT-4 cannot legally issue a medical prescription[1]. 

I am not going to make predictions here; the aim of this post is to make ascertainments. Namely: can we write a list of jobs that could be fully replaced by GPT-4 literally tomorrow? 

Let me make clear what I'm not asking for: I'm not asking for hypotetical scenarios where a dedicated workflow including GPT-4 can be set up and tailored in some way in order to replace a human worker. For now I'm only interested in the you're-instantly-fired version, where your boss could just use plain GPT-4 (instead of you) without having to do anything more than writing the prompt.

I've obviously asked a LLM (Alpaca) about this, and its answer was:

I fully agree about customer service representatives, whose stereotype is a shitty job where you just throw nice-sounding bullshit at angry customers (and LLMs are really really good at generating nice-sounding bullshit with infinite patience). 

For the other items in the list, I don't think that we are near to the point where the job could be totally replaced by GPT. Google Translate has existed for many years, but the world is still full of human translators, and I've yet to see a printed book admittedly translated by a machine (probably because every time I've tried to translate something more complicated than a single sentence with Google Translate, the output had to be slightly edited... do we have evidence that GPT-4 can be a much better translator?).

"Writers" is a bit too broad. But I would agree that the subcategory of copywriters is probably fully replaceable already.

Other suggestions?

  1. ^

    If a human is formally holding the position for legal reasons while GPT does all the real work, does it still count as technological unemployment? For the sake of this discussion I would resolve this as "no".

6 comments

Comments sorted by top scores.

comment by Viliam · 2023-03-24T09:47:19.650Z · LW(p) · GW(p)

Google Translate has existed for many years, but the world is still full of human translators

I think this is because the translations by Google Translate are... obviously machine-translated, and often wrong. Try Deepl instead!

Sorry if this sounds like an advertisement, but I often translate texts from English to Slovak (usually just to put them on a blog or a social network). When I used Google Translate, my process was: "translate the text myself, use Google Translate when I find a word I am not sure about". When I found Deepl, my process became: "just put everything in Deepl, and then read both the original and the translation, side by side, and fix the mistakes". In both cases I do what is the faster way; it's just that with Google Translate the mistakes were too frequent.

This may depend on the language. Slovak has grammatical genders, and in nontrivial situations, Google Translate basically flips a coin, while Deepl seemingly can figure out which noun in a previous paragraph some pronoun in a later paragraph refers to. Also, Deepl is better at obscure words (archaisms, jargon) in situations where Google just gives up, but instead of admitting that it has no idea, it suggests to simply use the English word in the Slovak text. Also, try to translate an Italian opera written 100 or 200 years ago, Google Translate just writes random words, Deepl gets it right.

And of course, the next version of Google Translate may fix all the problems I have mentioned. Maybe it already happened, dunno.

Replies from: Bezzi
↑ comment by Bezzi · 2023-03-24T22:54:15.734Z · LW(p) · GW(p)

just put everything in Deepl, and then read both the original and the translation, side by side, and fix the mistakes

Well, actually this is also my typical workflow with Google Translate (I was aware of the existence of Deepl, I just mentioned Google Translate because it's more widely known). Maybe it got better in the last two years?

try to translate an Italian opera written 100 or 200 years ago, Google Translate just writes random words, Deepl gets it right.

I just tried with the most obscure scene I know of (Scena 13 from The Night Bell), which was deliberately written to be full of near-incomprehensible words even for its time. The results from Deepl and Google Translate seem pretty similar to me; in several cases Deepl is the one who gets it wrong (even at the very start: "mi dovete una ricetta come un fulmine spicciar" is correctly translated as "you owe me a recipe like a flash of lightning" by Google Translate, while Deepl writes "you owe me a recipe like lightning spicciar"). 

Anyway, this was precisely my point: these systems can significantly speed up the work of human translators, but not completely replace them (yet), because some form of proofreading is still needed, and for now the only way to proofread the translation is to have a human who knows both languages.

Replies from: Viliam
↑ comment by Viliam · 2023-03-25T21:03:46.707Z · LW(p) · GW(p)

My example was a version of Romeo and Juliet:

Eccomi in lieta vesta...
Eccomi adorna...
come vittima all'ara
Oh! almen potessi qual vittima
cader dell'ara al piede!
O nuziali tede, aborrite
cosi, cosi fatali,
siate, ah! siate per me
faci ferali.
Ardo... una vampa, un foco
tutta mi strugge.

Un refrigerio
ai venti io chiedo invano
Ove sei tu, Romeo?
in qual terra t'aggiri?
Dove, dove, inviarti,
dove i miei sospiri?

Google Translate:

Here I am in happy dress...
Here I am adorned...
as a victim in the arena
Oh! at least I could as a victim
fall of the altar at the foot!
O German nuptials, abhor you so,
so fatal,
be, ah! be for me
feral faci.
I burn... a blaze, a fire
everything torments me.

A refreshment
to the winds I ask in vain
Where are you, Romeo?
in which land are you wandering?
Where, where, to send you,
where my sighs?

Deepl:

Here I am in happy attire...
Here I am adorned...
As a victim at the altar
Oh! at least I could what victim
Fall of the altar at the foot!
O nuptial tede, abhorred
So, so fatal,
Be, ah! be for me
Faci ferali.
I burn, a blaze, a fire
All cringes at me.

A refreshment
To the winds I ask in vain
Where art thou, Romeo?
In what land wanderest thou?
Where, where, send thee,
where my sighs?

A human translator:

Here I am in a happy attire
Here I am adorned…
like a victim for the altar.
Oh, if I could at least fall a as a victim
at the foot of the altar!
O nuptial torches, abhorred, so fatal, ah,
be my funeral torches.
I am burning… 
a flame, a fire consumes me entirely.

In vain I beg the winds to refresh me.
Where are you, Romeo?
In what parts are you roaming?
Where, oh, where can I
send my sighs to you?

Let's compare...

"vittima all'ara" -- GT "a victim in the arena" wrong; D "a victim for the altar" correct

"almen potessi qual vittima cader dell'ara al piede" -- GT "at least I could as a victim fall of the altar at the foot" correct; D "at least I could what victim fall of the altar at the foot" almost ok

(this is the part where GT now does significantly better than one year ago, when it translated "ara" as a parrot, so the victim was falling at parrot's feet. D already understood that it was an altar.)

"nuziali tede" -- GT "German nuptials" wrong; D "nuptial tede" wrong

"faci ferali" -- GT "feral faci" at least it partially tried, but got it wrong anyway; D "faci ferali" just gave up

"un foco tutta mi strugge" -- GT "a fire everything torments me" almost ok; D "a fire all cringes at me" wrong

...well, now GT is narrowly a winner.

comment by trevor (TrevorWiesinger) · 2023-03-24T00:05:44.663Z · LW(p) · GW(p)

The threat model here is that if any of those jobs are automated, then everyone who is currently employed will have to compete with very large numbers of Administrative assistants, Accounting clerks, Web designers, Graphic designers, and Writers, who are extremely desperate for positions, and the few of them who succeed will be the ones willing to work substantially longer and harder than most of the people currently employed.

With Web designers, Graphic designers, and Writers, that also means a substantial risk that a large people applying for EA/AI safety jobs will be very desperate, which would mean that many of them would instinctively get very good at intelligently goodharting to get at the jobs, due to broader culture shifts in a dramatically more competitive work environment.

Replies from: Bezzi
↑ comment by Bezzi · 2023-03-24T23:09:16.467Z · LW(p) · GW(p)

I don't question that a web designer armed with GPT-4 could make the work of a small team of web designers. But in this scenario there will still be human web designers, and the resulting technological unemployment would not be fundamentally different from the older forms of technological unemployment where old jobs shrink and new jobs (AI sheperds?) appear. What I am questioning is the scenario where a lot of jobs suddenly disappear altogether rather than being augmented with LLMs.

comment by Hailey Collet (hailey-collet) · 2023-03-24T07:59:28.613Z · LW(p) · GW(p)

In the short term, job loss will happen through compression of teams more than anything ... Some of seniors taking on junior work, e.g. if you have a development team of multiple senior & junior engineers, I could see some juniors getting canned with how things are now. But the restriction of exactly as it is right now is pretty severe. You don't have to train new models to vastly increase its real world capabilities.