Slopworld 2035: The dangers of mediocre AI

The particular example given (architecture) doesn't really make sense IMO:

If you're building something and want to save money on architecture, the current solution is to take a boilerplate design, tweak it to the current plot and local building code, and build it. Either AI can do that tweaking well, in which case great, or it can't in which case the architectural costs are fairly low compared to the total cost of the building that it really makes no sense to cut corners there.

If you're doing something bespoke, you'll want something that works, and if the architecture firm you use is cheaper than other bespoke firms, but the final product is subpar, you'll sue. Or at the very least not use them again.

Now this is a nitpick, but I think it might largely apply to the wider point: when you dig into the details it doesn't really make sense.

The big picture is plausible but one major error you make is assuming "academics" will be a solid bastion of opposition. My understanding is that academics are often some of the first ones to fall (like when teachers struggle with students who use ChatGPT to cheat on homework), and many of the academic complaints about AI are just as slop-y as what the AI produces.

I saw someone who was worried that AI was gonna cause real economic trouble soon by replacing travel agents. But the advent of the internet made travel agents completely unnecessary, and it still only wiped out half the travel agent jobs. The number of travel agents has stayed roughly the same since 2008!

This reminds me of Patrick McKenzie's tweet thread: 

Technology-driven widespread unemployment ("the robots will take all the jobs") is, like wizards who fly spaceships, a fun premise for science fiction but difficult to find examples for in economic history. (The best example I know is for horses.)

I understand people who are extremely concerned by it, but I think you need a theory for it which successfully predicts bank teller employment trends over the last 50 years prior to justifying the number of column inches it gets.

"Haha Patrick you're not going to tell me bank teller employment is up since we made a machine literally called Automated Teller Machine are you." Bessen, 2015:

"Wait why did that happen?" Short version: the ATM makes each bank branch need less tellers to operate at a desired level of service, but the growth of the economy (and the tech-driven decline in cost of bank branch OpEx) caused branch count to outgrow decline in tellers/branch. 

The UK government also heavily used AI chatbots to generate diagrams and citations for a report on the impact of AI on the labour market, some of which were hallucinated.

This link is broken.

I think these sort of concerns will manifest in the near future, but it'll be confusing because AI's competence will continue to be unevenly distributed and unintuitive. I expect some AI systems will be superhuman, such as automated vehicles and some AI diagnosticians, and that incompetent AIs will gain unwarranted trust by association while the competent AIs get unwarranted distrust by association. Sometimes trusting AI will save lives, other times it will cost them.