LessWrong 2.0 Reader

I tend to follow the linguist, McWhorter, on historical trends in languages over time, in believing (controversially!) that undisrupted languages become weirder over time, and only gains learnability through pragmatic pressures, as in trading, slavery, conquest, etc which can increase the number of a language's second language learners (who edit for easy of learning as they learn).

A huge number of phonemes? Probably its some language in the mountains with little tourism, trade, or conquest for the last 8,000 years. Every verb conjugates irregularly? Likely to be found in the middle of a desert. And so on.

The normal, undisrupted, pattern is for every generation to make mistakes and play around, decorating the language with entropic silliness, and accidentally causing future children to "only really learn to speak fully properly" at older and older and older ages... until around 11 or 12 or 13 or 14 puberty strikes, and kids stop diligently learning any random bullshit the older people say based on trust. English competency arrives around age 8 because English is a toy language created by waves and waves and waves of trade, conquest, and cultural admixture. We have a lot of room to get much weirder and stay within traditional human bounds.

((That is, we have a lot of room for English, left alone, to mutate, IF this broader theory is correct. It might not be. 

A way to test the larger theory would be to anthropologically construct a way of predicting from first principles when puberty tends to start in human subpopulations (because we have strong suggestions that diet and social patterns can change it), then reconstruct the predicted value of puberty onset over historical timescales, then correlate that to modern relatively easily measured "age until language mastery" for many modern languages. 

That would confirm most of the theory. The other thing you'd need to track is the percentage of speakers who are speaking after learning any given language as a second language. High rates of this should simplify a tongue and cut against the other process that adds complexity by default.))

To show how weird English is: English is the only proto indo european language that doesn't think the moon is female ("la luna") and spoons are male (“der Löffel”). I mean... maybe not those genders specifically in every language. But some gender in each language.

I just looked up Gujurati, which is also descended from Proto-Indo-European and moon (chandri ("ચંદ્રા")) is feminine and ladle (chamcho ("ચમચો")) is masculine... but teaspoon (chamchi ("ચમચી")) is feminine(!)... so... yeah... that one retained gender and also has gender/semantic conflation! :-)

Except in English. The moon is a rock in English, not a girl. And a spoon is a tool, not a boy. Because English is a weird rare toy language (practically a creole, implying that it was a pidgin for many), that doesn't force people to memorize reams of playful historical bullshit, in order to "sound like they speak it properly" :-)

"English" traces all the way back to a language (with gendered declined nouns and verb conjugation) spoken by Eurasian Charioteers in 7000BC or whatever and at each step most of the changes were all just "part of the stream of invective".

...

Regarding word count specifically... 

Something you find over and over and over again in language is agglutinating grammar where entire sentences are just. One. Word. But not like that... rather: Asinglebigwordcanbeusedtocommunicate oneideafromamongavastarray.

These languages are also often irregular! (6) Like the language was already agglutinative 1000 years ago, (9) and then people spent the next ten centuries making it more pronounceable, and punny, and fun??? (16)

Taakkua uqausiit taimaigajunngimmijuttauq! (3) Suurlu uqausiq katinngaliriilaurmat 1000 arraaguit aniguqsimaliqtut, (6) ammalu inuit atulaurmata qulinik arraagunik tukiqarniqsauliqtittinasuaqtutik, ammalu quvianarniqsauliqtuni??? (8)

> These words are not normal either! (6) Like language was already coherent 1000 years ago, (8) and people spent the last decade trying to make it more sensible, and cool??? (14)

The above paragraph round trips through "Google's understanding of Inuktut", which (I think?) is a simplified language arising from systematizing and averaging out dialects starting from relatively normally complex languages like Inuktitut... and basically all of those polar languages are agglutinative, and have been at least for centuries.

I brought that one paragraph to English to suggest roughly how much was lost by Google's translation.

The parenthetic numbers show "words per clause" through the process: 

6-->3-->5!   9-->6-->8,   16-->9-->14???

So here's my (half silly) proposal: maybe English experienced catastrophic simplifications between ~600AD and ~1500AD and then became preternaturally frozen once it was captured in text by the rise of printing, literacy, industrialization, and so on. The starting point itself was relatively unnatural, I think.

So then, in recent history, maybe what we're seeing is just a looooong and slooooow motion trend (that'll take a millennium or three to complete at this rate (unless we abandon literacy or something, and free the language from the strictures of printing and mass education?)) where English is still slowly trying to become an agglutinative language with irregular morphology?

Like (here's the deep crazy idea:) like maybe every language wants to ultimately be after >200 generations of accumulated youthful ignorance, cryptogenic wordplay, lazy mouths, and no writing?

For example: I just made up the word "cryptogenic" to be "having a genesis in a desire to be hard to understand" (which I considered myself to have a right to do, since english has a productive morphology) but when I looked up other skilled speakers have deployed it in other ways... Oxford thinks it means "(of a disease) of obscure or uncertain origin" and most of the usages are for "diseases not yet subjectively diagnosed by the doctor during the course of treatment (rather than diseases whose etiology is a known mystery to standard medical science)". It gets used like "Knowing the cause of a cryptogenic stroke can help prevent recurrent stroke" (source is the metadata summary of this webpage).

Whereas I'm claiming that many words are cryptogenic in the sense that they started out, like "skibidi", within youth culture because kids liked that grownups didn't know what it means. If "skibidi" catches on, and gains an intergeneratioanlly re-usable meaning (maybe related to being scared in a fun way? or yet-another-adjective like hep? or whatever?) then it will have been partly possible because kids liked having their own words that "parents just don't understand".

This is hard for English, because it is written. And because many second language speakers learn English every year.

But one thing that English can do (despite enormous pressures to be learnable and written in a stable way) is boil itself down to stock phrases for entire sentences. Later, these stock phrases could eventually agglutinate into single words, maybe, or at least they might if global civilization and travel and communication collapses in a way that leaves literally any humans alive, but trapped in tiny local regions with low literacy for many generations... which is a very specific and unlikely possible future. (Prolly we either get wildly richer and become transhuman or else just all end up dead to predatory posthumans.)

Thanks for this post. I would argue that part of an explanation here could also be economic: modernity brings specialization and a move from the artisan economy of objects as uncommon, expensive, multipurpose, and with a narrow user base (illuminated manuscripts, decorative furniture) to a more utilitarian and targeted economy. Early artisans need to compete for a small number of rich clients by being the most impressive, artistic, etc., whereas more modern suppliers follow more traditional laws of supply and demand and track more costs (cost-effectiveness, readability and reader's time vs. beauty and remarkableness). And consumers similarly can decouple their needs: art as separate from furniture and architecture, poetry and drama as separate from information and literature. I think another aspect of this shift, that I'm sad we've lost, is the old multipurpose scientific/philosophical treatises with illustrations or poems (my favorite being de Rerum Natura, though you could argue that Nietzsche and Wagner tried to revive this with their attempts at Gesamtkunstwerke).

I agree, and am also confused with the idea that LLMs will be able to bootstrap something more intelligent.

My day job is a technical writer. I also do a bit of DevOps stuff. This combo ought to be the most LLM-able of all, yet I frequently find myself giving up on trying to tease out an answer from an LLM. And I'm far from the edge of my field!

So how exactly do people on the edge of their field make better use of LLMs, and expect to make qualitative improvements?

Feels like it'll have to be humans to make algorithmic improvements, at least up until a point.

When something is true, I desire to believe it’s true. When something is false, I desire to believe it’s false. This is the proper epistemics. If your epistemic goals are different, then they’re different. But “If the accused is in power, increase the probability estimate” is not how good epistemics are achieved.

Tangent here, just occurred to me while writing. The correct adjustment might be in the other direction: there are way more accusations against people in power, so part of the problem when considering them is: how do you keep your False Discovery Rate low? Like, if your neighbor is accused of a crime, he probably did it. But top politicians are accused of crimes every week, and many of those aren’t real, or aren’t criminal. And most or all False Discovery Rate adjustments lower the estimated probability of each instance. (Tangent over).

I think you may have a case about how one’s decision theory should adjust based on power and risk. Something like “I think there’s a 15% chance this is true, but if it were, it would be really bad, so 15% is high enough that I think we should investigate”.  But taking that decision theory thought process, and using it to speak as if the 15% thing has a greater-than-50% probability, for example, isn’t correct.

I agree with the majority of this, especially the part where we are in a race to increase our own intelligence before we destroy ourselves with AI or something else.

But I think there's something missing in your analysis.

"Genes are the piano; the microbiome is the pianist."

It’s time to admit that genes are not the blueprint for life (Feb 2024) https://www.nature.com/articles/d41586-024-00327-x - I see it's closed access now; here are some quotes.

More on the microbiome & genetics: https://humanmicrobiome.info/genetics/

Examples of how important the gut microbiome, and the parents' health, are for human development: https://humanmicrobiome.info/maternity/

I have the exact same goal as you, and I'm working on it from the microbiome perspective. See:

• https://forum.humanmicrobiome.info/threads/the-fda-and-fmt-regulation-part-2-jul-2024-humanmicrobes-org-i-met-wit.520/#post-1370
• https://forum.humanmicrobiome.info/threads/michael-harrop-q-a-nov-2023.200/ 

Thanks for the trendlines - they help us understand when AI can automate years of work!

Like you said, the choice of tasks can heavily change the trendline

Our estimate of the length of tasks that an agent can complete depends on methodological choices like the tasks used and the humans whose performance is measured. However, we’re fairly confident that the overall trend is roughly correct, at around 1-4 doublings per year. If the measured trend from the past 6 years continues for 2-4 more years, generalist autonomous agents will be capable of performing a wide range of week-long tasks.

I believe SWE-bench is the best benchmark to control for variables like the choice of task and how the agentic system is built, so I'm leaning more towards the doubling time of 70 days.

For a large scale / complex app, it takes around 1 year of development (though this is not a completely fair estimate since it doesn't take into account the number of man-hours), but going with this estimate and doubling in SWE-bench, it takes around 13 doublings from the beginning of 2025 or June 2027 to automate production of entire apps / complex websites.

Another big factor that this trendline and other trendlines don't take into account is the amount of AI acceleration. If AI automates a large portion of the work, the time to double would shorten as AI gets better, I'd be interested to see how that would affect this model.

Maybe the recent tariff blowup is actually just a misunderstanding due to bad terminology, and all we need to do is popularize some better terms or definitions. We're pretty good at that around here, right?

Here's my proposal: flip the definitions of "trade surplus" and "trade deficit." This might cause a bit of confusion at first, and a lot of existing textbooks will need updating, but I believe these new definitions capture economic reality more accurately, and will promote clearer thinking and maybe even better policy from certain influential decision-makers, once widely adopted.

New definitions:

• Trade surplus: Country A has a bilateral "trade surplus" with Country B if Country A imports more tangible goods (cars, steel, electronics, etc.) from Country B than it exports back. In other words, Country A ends up with more real, physical items. Country B, meanwhile, ends up with more than it started with of something much less important: fiat currency (flimsy paper money) or 1s and 0s in a digital ledger (probably not even on a blockchain!).

  If you extrapolate this indefinitely in a vacuum, Country A eventually accumulates all of Country B's tangible goods, while Country B is left with a big pile of paper. Sounds like a pretty sweet deal for Country A if you ask me.

  It's OK if not everyone follows this explanation or believes it - they can tell it's the good one because it has "surplus" in the name. Surely everyone wants a surplus.

• Trade deficit: Conversely, Country A has a "trade deficit" if it exports more tangible resources than it imports, and thus ends up with less goods on net. In return, it only receives worthless fiat currency from some country trying to hoard actual stuff for their own people. Terrible deal!

  Again, if you don't totally follow, that's OK, just pay attention to the word "deficit". Everyone knows that deficits are bad and should be avoided.

Under the new definitions, it becomes clear that merely returning to the previous status quo of a few days ago, where the US only "wins" the trade war by several hundred billion dollars, is insufficient for the truly ambitious statesman. Instead, the US government should aggressively mint more fiat currency in order to purchase foreign goods, magnifying our trade surplus and ensuring that in the long run the United States becomes the owner of all tangible global wealth.

Addressing second order concerns: if we're worried about a collapse in our ability to manufacture key strategic goods at home during a crisis, we can set aside part of the resulting increased surplus to subsidize domestic production in those areas. Some of the extra goods we're suddenly importing will probably be pretty useful in getting some new factories of our own off the ground. (But of course we shouldn't turn around and export any of that domestic production to other countries! That would only deplete our trade surplus.)

Probably do a screen recording to scale later with AI

Slides for my talk

Thanks.

I'm still not quite understanding what you're thinking though.

For other objects, like physical ones, quantifiers have to be used. Like "at least one" or "the" (the latter only presupposes there is exactly one object satisfying some predicate). E.g. "the cat in the garden". Perhaps there is no cat in the garden or there are several. So it (the cat) cannot be logically represented with a constant.

"the" supposes there's exactly one canonical choice for what object in the context is indicated by the predicate. When you say "the cat" there's basically always a specific cat from context you're talking about. "The cat is in the garden" is different from "There's exactly one cat in the garden".

Maybe "Superman" is actually two people with the same dress, or he doesn't exist, being the result of a hallucination. This case can be easily solved by treating those names as predicates.

• The woman believes the superhero can fly.
• The superhero is the colleague.

I mean there has to be some possibility for revising your world model if you notice that there are actually 2 objects for something where you previously thought there's only one.

I agree that "Superman" and "the superhero" denote the same object(assuming you're in the right context for "the superhero").

(And yeah to some extent names also depend a bit on context. E.g. if you have 2 friends with the same name.)

You can say "{(the fact that) there's an apple on the table} causes {(the fact that) I see an apple}"

But that's not primitive in terms of predicate logic, because here "the" in "the table" means "this" which is not a primitive constant. You don't mean any table in the world, but a specific one, which you can identify in the way I explained in my previous comment.

Yeah I didn't mean this as formal statement. formal would be:

{exists x: apple(x) AND location(x, on=Table342)} CAUSES {exists x: apple(x) AND see(SelfPerson, x)}