LessWrong 2.0 Reader

To me "generally avoid processed foods" would be kinda like saying "generally avoid breathing in gasses/particulates that are different from typical earth atmosphere near sea level".

People have been breathing a lot of smoke in the last million years or so, so one might think that we would have evolved to tolerate it, but it's still really bad for us. Though there are certainly lots of ways to go wrong deviating from what we are adapted to, our current unnatural environment is far better for our life expectancy than the natural one. As pointed out in other comments, some food processing can be better for us.

You might be interested in reading this [LW · GW]. I think you are reasoning in an incorrect framing. 

What if you build your school-as-social-service, and then one day find that the kids are selling drugs to each other inside the school?

Or simply that the kids are constantly interfering with each other so much that the minority who want to follow their interests can't?

Any theory of school that doesn't mention discipline is a theory of dry water. You say "for 100 children, we could have 2 nurses, 5 counselors, 5 social workers, 8 adult supervisors". What powers and duties would your 1-supervisor-per-12-kids have? Can they remove disruptive kids from rooms? From the building entirely? Give detentions?

I really can't get the point from the "3. Solve or Reduce Aversions", specifically: 
> Meanwhile, un-endorsed aversions should be targeted with exposure therapy or CoZE.

As I can see, here we should get rid of bad aversions. But the rest part of the text sounds like we should... reinforce them?..
> To apply exposure therapy, build a path of incremental steps towards the aversion

An example that's probably not very large is the discovery of DNA as the inheritance particle.

I had great fun reading Watson's scientific-literary fiction Double Helix. Watson and Crick are very clear that competitors were hot on their heels, a matter of months perhaps.

I sometimes had this feeling from Conway's work, in particular, combinatorial game theory and surreal numbers to me feel closer to mathematical invention than mathematical discovery. This kind of things are also often "leaf nodes" on the tree of knowledge, not leading to many followup discoveries, so you could say their counterfactual impact is low for that reason.

In engineering, the best example I know is vulcanization of rubber. It has had a huge impact on today's world, but Goodyear developed it by working alone for decades, when nobody else was looking in that direction.

Feymann's path integral formulation can't be that counterfactually large. It's mathematically equivalent to Schwingers formulation and done several years earlier by Tomonaga.

Penicillin. Gemini tells me that the antibiotic effects of mold had been noted 30 years earlier, but nobody investigated it as a medicine in all that time.

Gemini is telling you a popular urban legend-level understanding of what happened. The creation of Penicillin as a random event, "by mistake", has at most tangential touch with reality. But it is a great story, so it spread like wildfire. 

In most cases when we read "nobody investigated" it actually means "nobody succeeded yet, so they weren't in a hurry to make it known", which isn't very informative point of data. No one ever succeeds, until they do. And in this case it's not even that - antibiotic properties of some molds were known and applied for centuries before that (well, obviously, before the theory of germs they weren't known as "antibiotic", just that they helped...), the great work of Fleming and later scientists was about finding the particularly effective type of mold and extracting the exact effective chemical as well as finding a way to produce that at scale.

So I keep seeing takes about how to tell if LLMs are "really exhibiting goal-directed behavior" like a human or whether they are instead "just predicting the next token". And, to me at least, this feels like a confused sort of question that misunderstands what humans are doing when they exhibit goal-directed behavior.

Concrete example. Let's say we notice that Jim has just pushed the turn signal lever on the side of his steering wheel. Why did Jim do this?

The goal-directed-behavior story is as follows:

• Jim pushed the turn signal lever because he wanted to alert surrounding drivers that he was moving right by one lane
• Jim wanted to alert drivers that he was moving one lane right because he wanted to move his car one lane to the right.
• Jim wanted to move his car one lane to the right in order to accomplish the goal of taking the next freeway offramp
• Jim wanted to take the next freeway offramp because that was part of the most efficient route from his home to his workplace
• Jim wanted to go to his workplace because his workplace pays him money
• Jim wants money because money can be exchanged for goods and services
• Jim wants goods and services because they get him things he terminally values like mates and food

But there's an alternative story:

• When in the context of "I am a middle-class adult", the thing to do is "have a job".
• When in the context of "having a job", "showing up for work" is the expected behavior.
• When in the context where the current task is "show up for work", the expected behavior is "drive from home to work along the usual route"
• When in the context of "I am at my work's exit and I am going to work now", the expected action is "get into the exit lane".
• When in the context of "I'm trying to move one lane to the right", the action "turn on the right turn signal" is expected.

I think this latter framework captures some parts of human behavior that the goal-directed-behavior framework misses out on. For example, let's say the following happens

1. Jim is going to see his good friend Bob on a Saturday morning
2. Jim gets on the freeway - the same freeway, in fact, that he takes to work every weekday morning
3. Jim gets into the exit lane for his work, even though Bob's house is still many exits away
4. Jim finds himself pulling onto the street his workplace is on
5. Jim mutters "whoops, autopilot" under his breath, pulls a u turn at the next light, and gets back on the freeway towards Bob's house

This sequence of actions is pretty nonsensical from a goal-directed-behavior perspective, but is perfectly sensible if Jim's behavior here is driven by contextual heuristics like "when it's morning and I'm next to my work's freeway offramp, I get off the freeway".

Note that I'm not saying "humans never exhibit goal-directed behavior".

Instead, I'm saying that "take a goal, and come up with a plan to achieve that goal, and execute that plan" is, itself, just one of the many contextually-activated behaviors humans exhibit.

I see no particular reason that an LLM couldn't learn to figure out when it's in a context like "the current context appears to be in the execute-the-next-step-of-the-plan stage of such-and-such goal-directed-behavior task", and produce the appropriate output token for that context.

Pasteur had (also highly "counterfactual") help I think! Ignaz Semmelweis worked in this maternity ward where the women & babies kept dying.  The hospital had opened up some investigations over the years as to the cause of death but kept closing them with garbage explanations. He went somewhere else for a while and when he got back he noticed that the death numbers were down in his absence. Then he noticed his hands smelled like death after one of his routine autopsies and he was about to go plunge them in some poor mother! He had washed them but just with regular soap. If he put some bleach in the washwater then his hands didn't stink. He connected the dots. He had killed hundreds of mothers & babies but wrote a book about it anyway and thereby popularized disinfection (and strongly suggested the root cause of disease).

Probably the main reason that germ theory took so long to work out is that the people with the right evidence were too guilty and ashamed to share it.