LessWrong 2.0 Reader
View: New · Old · Top← previous page (newer posts) · next page (older posts) →
← previous page (newer posts) · next page (older posts) →
I think much of this is quite unreasonable (and some very unreasonable- you "don't like that I spoke as spoke as an authority on her life" because I wondered if what she observed was truly attributable to a causal effect?!), but I don't see the value in going over it, especially as others have made the points I would make about your tone and framing elsewhere. I continue to find your contributions on this topic a little more combative and "soldier mindset" than is ideal, but clearly you strongly disagree. (Although it's tempting to suggest that your admittedly "angry" and "unfair" reply, several times longer than your eventual response to the object level question, is evidence for the prosecution, not to mention that it somewhat calls into question your primary 'it's too much work' defence for ignoring substantive criticisms such as Chen's in the first place.) I don't see the point in continuing to argue about whose team is more rational, in any case; all I wanted was your response to Chen's objections to help inform my new dietary choices (something which, again, you concluded was worth dozens of hours and tens of thousands of dollars when multiplied by six, a matter of months ago).
With all that in mind, I have a few follow-up questions to your object-level response, but I will understand if you choose to ignore them, given that you don't seem to enjoy or value the interaction and I'm finding it lower value than I'd hoped, myself.
daniel-kokotajlo on Modern Transformers are AGI, and Human-LevelCurrent AIs suck at agency skills. Put a bunch of them in AutoGPT scaffolds and give them each their own computer and access to the internet and contact info for each other and let them run autonomously for weeks and... well I'm curious to find out what will happen, I expect it to be entertaining but not impressive or useful. Whereas, as you say, randomly sampled humans would form societies and fnd jobs etc.
This is the common thread behind all your examples Hjalmar. Once we teach our AIs agency (i.e. once they have lots of training-experience operating autonomously in pursuit of goals in sufficiently diverse/challenging environments that they generalize rather than overfit to their environment) then they'll be AGI imo. And also takeoff will begin, takeover will become a real possibility, etc. Off to the races.
Sure, the topics in this piece are dealt with superficially and the discussions are not especially thought-provoking; when compared to the amazing creative works that people on this site produce, it is low-mediocre. But Claude writes more coherently than a number of published authors and most of the general public.
justinpombrio on The Cognitive-Theoretic Model of the Universe: A Partial Summary and Reviewtldr; a spot check calls bullshit on this.
I know a bunch about formal languages (PhD in programming languages), so I did a spot check on the "grammar" described on page 45. It's described as a "generative grammar", though instead of words (sequences of symbols) it produces "L_O spacial relationships". Since he uses these phrases to describe his "grammar", and they have their standard meaning because he listed their standard definition earlier in the section, he is pretty clearly claiming to be making something akin to a formal grammar.
My spot check is then: is the thing defined here more-or-less a grammar, in the following sense?
If you don't have a thing plus a way to derive stuff from that thing, you don't have anything resembling a grammar.
My spot check says:
I'd categorize this section as "not even wrong"; it isn't doing anything formal enough to have a mistake in it.
Another fishy aspect of this section is how he makes a point of various things coinciding, and how that's very different from the standard definitions. But it's compatible with the standard definitions! E.g. the alphabet of a language is typically a finite set of symbols that have no additional structure, but there's no reason you couldn't define a language whose symbols were e.g. grammars over that very language. The definition of a language just says that its symbols form a set. (Perhaps you'd run into issues with making the sets well-ordered, but if so he's running headlong into the same issues.)
I'm really not seeing any value in this guy's writing. Could someone who got something out of it share a couple specific insights that got from it?
celarix on Vernor Vinge, who coined the term "Technological Singularity", dies at 79Noted, thank you. This does raise my confidence in Alcor.
scrollop on Many people lack basic scientific knowledgeTrying to determine what your point here is.
Since "intelligence" (thorny definition) could be said to be distributed in a normal distribution (if it's even possible to plot "intelligence" (let's say, "intellectual" or, dear me, "IQ")), and those people below the median would be less likely to be well educated (making assumptions here), then it would be expected that a significant proportion of every population will have people that won't answer these questions as well educated people with higher intelligence (to me these are obvious points, though maybe I'm missing something here).
I think a lot of people in high school (and otherwise) had little motivation to learn, so their knowledge may seem painfully inadequate compared to that readers of forums such as this. But, then, if you ask these people what practical use is knowledge of genetics or whether a star revolves around a plant of vice versa, their answer would probably not surprise you.
But, then, perhaps they could gut a fish in 30 seconds or fix the carburetor on your car as the zombies are closing in (faster than you* or I would whilst we're frantically speeding up the youtube video or sending the photo to chatgpt), so pluses and minuses (I won't judge in which direction the pluses and minsues tally for any person at any particular time in any particular universe).
On the other hand: Idiocracy (2006)
They took it down real quick for some reason.
g-w1 on From the outside, American schooling is weirdI'd be interested in what a steelman of "have teachers arbitrarily grade the kids then use that to decide life outcomes" could be?
The best argument I have thought of is that America loves liberty and hates centralized control. They want to give individual states, districts, schools, teachers the most power they can have as that is a central part of America's philosophy. Also anecdotally, some teachers have said that they hate standardized tests because they have to teach to it. And I hate being taught to for the test (like APs for example). It's much more interesting where the teacher is teaching something they find interesting and enjoy (and thus can choose to assess on).
However, this probably does not outweigh the downsides and is probably a bad approach overall.
tailcalled on tailcalled's ShortformI have a concept that I expect to take off in reinforcement learning. I don't have time to test it right now, though hopefully I'd find time later. Until then, I want to put it out here, either as inspiration for others, or as a "called it"/prediction, or as a way to hear critique/about similar projects others might have made:
Reinforcement learning is currently trying to do stuff like learning to model the sum of their future rewards, e.g. expectations using V, A and Q functions for many algorithm, or the entire probability distribution in algorithms like DreamerV3.
Mechanistically, the reason these methods work is that they stitch together experience from different trajectories. So e.g. if one trajectory goes A -> B -> C and earns a reward at the end, it learns that states A and B and C are valuable. If another trajectory goes D -> A -> E -> F and gets punished at the end, it learns that E and F are low-value but D and A are high-value because its experience from the first trajectory shows that it could've just gone D -> A -> B -> C instead.
But what if it learns of a path E -> B? Or a shortcut A -> C? Or a path F -> G that gives a huge amount of reward? Because these techniques work by chaining the reward backwards step-by-step, it seems like this would be hard to learn well. Like the Bellman equation will still be approximately satisfied, for instance.
Ok, so that's the problem, but how could it be fixed? Speculation time:
You want to learn an embedding of the opportunities you have in a given state (or for a given state-action), rather than just its potential rewards. Rewards are too sparse of a signal.
More formally, let's say instead of the Q function, we consider what I would call the Hope function: which given a state-action pair (s, a), gives you a distribution over states it expects to visit, weighted by the rewards it will get. This can still be phrased using the Bellman equation:
Hope(s, a) = rs' + f Hope(s', a')
Where s' is the resulting state that experience has shown comes after s when doing a, f is the discounting factor, and a' is the optimal action in s'.
Because the Hope function is multidimensional, the learning signal is much richer, and one should therefore maybe expects its internal activations to be richer and more flexible in the face of new experience.
Here's another thing to notice: let's say for the policy, we use the Hope function as a target to feed into a decision transformer. We now have a natural parameterization for the policy, based on which Hope it pursues.
In particular, we could define another function, maybe called the Result function, which in addition to s and a takes a target distribution w as a parameter, subject to the Bellman equation:
Result(s, a, w) = rs' + f Result(s', a', (w-rs')/f)
Where a' is the action recommended by the decision transformer when asked to achieve (w-rs')/f from state s'.
This Result function ought to be invariant under many changes in policy, which should make it more stable to learn, boosting capabilities. Furthermore it seems like a win for interpretability and alignment as it gives greater feedback on how the AI intends to earn rewards, and better ability to control those rewards.
An obvious challenge with this proposal is that states are really latent variables and also too complex to learn distributions over. While this is true, that seems like an orthogonal problem to solve.
Also this mindset seems to pave way for other approaches, e.g. you could maybe have a Halfway function that factors an ambitious hope into smaller ones or something. Though it's a bit tricky because one needs to distinguish correlation and causation.
teatieandhat on Politics are not serious by defaultInteresting, and very well written. Because you have access to particularly funny examples, you show very well how much politics is an empty status game.
I should probably point out that five years ago, I was a high school student in France, felt more or less the way you do, and went on to study political science at college (I don’t even need to say which college I’m talking about, do I?). It is a deep truth that politics is very unserious for most people, and that is perhaps most true for first-year political science students (or, god forbid, the sort of people who teach them introductory political science classes). I studied political science precisely because I agreed with the sentiment you describe here, and expected something a little more serious.
I definitely did not get it. The average political science undergraduate is very much like your friends—not least because they’re actually the same people a year older—and, while many professors are great, some are scarcely better than their students.
You gave your funny sad stories, here’s one of mine (carefully selected to be the most egregious I’ve seen, but 100% true): first year sociology class, taught by respected specialist of Jewish life in Soviet-era Poland. Me, really curious about why sociology doesn’t dialogue more with some apparently contradictory results in social psychology. I try my best to ask "how does sociology react to that kind of stuff, even though it’s a completely different discipline and all?" in the least offensive way I can.
Teacher’s face suddenly turns dark blue, she jumps off her chair, yelling "THIS IS SCIENCE! THIS IS SCIENTIFIC SCIENCE!". It takes me a few seconds to gather that she’s not blaming psychology for being science. Her brain registered something which kinda sounded like an attack against her discipline, and she’s defending the science-ness of her job. And not, certainly, doing anything like answering my question. In fact, she’s running around the room ("science! Science!"), and has forgotten about me entirely. After five or ten minutes, she eventually goes back to her chair, visibly exhausted ("well… where was I? Ah yes…") and resumes the class.
But the reason I’m writing this comment is exactly because I don’t want you to start seeing the whole lot of them as a bunch of crazies (as I myself did…). It’s really true that everyone who doesn’t end up working in politics, and even most of those who do, when they’re young, treat it as a deeply unserious status game (but, given what LW has to say about politics, I’d be really surprised if it was worse in France than in the US, or basically anywhere else?). It is also true that wanting to work on politics and decision-making doesn’t come with a specific knowledge of rationality. So, yeah, most people who think about politics do so in a very irrational way, because politics is a status game (not to mention being the mind-killer [LW · GW]). But if you think that this is not a strong enough description and that the ones you know are really more crazy than that, I think the difference is because they’re high-schoolers :-) It does get a little bit better with age, but you might miss that if you brand them as crazies and forget to change your mind when most of them have grown enough to be a little less crazy :-)