LessWrong 2.0 Reader

Hold up.

I'm not sure what feedback to give about your post overall. I am impressed by it a significant way in, but then I get lost in what appear to be carefully-thought-through reasoning steps, and I'm not sure what to think after that point.

"Why is there basically no widely used homoiconic language"

Well, there's Lisp, in its many variants.  And there's R.  Probably several others.

The thing is, while homoiconicity can be useful, it's not close to being a determinant of how useful the language is in practice.  As evidence, I'd point out that probably 90% of R users don't realize that it's homoiconic.

Thank you for responding! I am being very critical, both in foundational and nitpicky ways. This can be annoying and make people want to circle the wagons. But you and the other organizers are engaging constructively, which is great.

The distinction between Solstice representing a single coherent worldview vs. a series of reflections also came up in comments on a draft. In particular, the Spinozism of Songs Stay Sung feels a lot weirder if it is taken as the response to the darkness, which I initially did, rather than one response to the darkness.

Nevertheless, including something in Solstice solidly establishes it as a normal / acceptable belief for rationalists: within the local Overton Window. You might not be explicitly telling people that they ought to believe something, but you are telling that it is acceptable for high status people in their community to believe it. I am concerned that some of these beliefs are even treated as acceptable.

Take Great Transhumanist Future. It has "a coder" dismantling the sun "in another twenty years with some big old computer." This is a call to accelerate AI development, and use it for extremely transformative actions. Some of the organizers believe that this is the sort of thing that will literally kill everyone. Even if it goes well, it would make life as it currently exists on the surface of the Earth impossible. Life could still continue in other ways, but some of us might want to still live here in 20 years.^[1] I don't think that reckless AI accelerationism should be treated as locally acceptable.

The line in Brighter Than Today points in the same way. It's not only anti-religious. It is also disparaging towards people who warn about the destructive potential of a new technology. Is that an attitude we want to establish as normal?

If the main problem with changing the songs is in making them scan and rhyme, then I can probably just pay that cost. This isn't a thing I'm particularly skilled at, but there are people who are who are adjacent to the community. I'm happy to ask them to rewrite a few lines, if the new versions will plausibly be used.

If the main problem with changing the songs is that many people in this community want to sing about AI accelerationism and want the songs to be anti-religious, then I stand by my criticisms.

1. ^{^}

   Is this action unilateral? Unclear. There might be a global consensus building phase, or a period of reflection. They aren't mentioned in the song. These processes can't take very long given the timelines.

You're right, my original wording was too strong. I edited it to say that it agrees with so many diets instead of explains why they work.

Wanted to be loved. Loved, and to live a life not only avoiding fear. Epiphany (4/22/2024): am a fuckup. Have always been a fuckup. Could never have made anyone happy or been happy, and a hypothetical world never being born would have been a better world. Deserved downvotes, it has to be all bullshit, but LessWrong was supposed to make people less wrong, and should’ve given a comment to show why bullshit, but you didn’t, so LessWrong is a failure, too. So sterile, here, no connection with the world – how were we ever supposed to change anything? Stupid especially to’ve thought anyone would ever care. All fucked-up.

Life was more enjoyable when it seemed there’d be more of it – when one could hope there’d be love, and less fear. Life enjoyable when it could be imagined as enjoyable. But music even hasn’t been anything, meant anything, in years.

No enjoyment, now: fear.

Hope was, after being locked in a room, not leaving in two years, until forgotten, the feeling of wind on skin, trying to produce thoughts new and useful, hoping for thoughts welcomed. Emerge, and nothing. How good will the world let you be? Think you choose your life and fate: choose to go faster than light; whether to have to be born, then.

Contradict Kant, so do the impossible – no-one ever left a comment showing that to be erroneous (because they didn’t know what it was, or just didn’t give a damn?) – so, presumably true, and hoping to help by it – how good does the world let you be? Try to do something; when you do something, sacrifice two years of your life, something is supposed to happen. Thinking one day someone will care. There are no more days. What you do is supposed to matter. What you do in life is supposed to matter. Life is supposed to matter. And it doesn’t. Should have known from Tesla – capital, anyone, needn’t acknowledge work.

Emerging after two years – such green, graceful petals – those clouds! – and stories, but the stories were all lies, always; CGI nothing: they’ve never been “real” (Sondheim conceives Passion on “true love” aged fifty-three; Sondheim admits: never been in love before age sixty). Two years – you wouldn’t even give a comment to say it’s wrong. And wouldn’t use it. Did you not know how and couldn’t bear to admit you didn’t know?

No comments (takes “Introspective Bayes”, nigh suicide-note to have even one). Only downvotes. Bad karma, sends you to hell – sending a message? No one ever touched without trying to hurt; never had any kind of relationship without you people end in calling it a waste of time, vomit, faggot, slap in face, kicked in head. Raped.

“We didn’t know! Don’t do it!” No, you didn’t know. You couldn’t have known. But you could have been nice. Even polite. You just weren’t. 

Never, when it mattered, a comment in the name of reason, did you give a reason why you objected so. Did you have none? Since there’s no requirement you should give a “System 2” reason for your vote – no surprise “System 1” doesn’t bother, and neither did you. (But then downvotes are marks of pride: only a fool would downvote sans explanation, thinking that adequate; too a fool thinks what is true is false. Fool’s disagreement is an endorsement – uncommented downvotes are votes in favor).

And if that reflexive, reason-free judgment thus sinks any critique of LessWrong – then it’s only a groupthink factory.

“Your post’s style was atrocious.” So too The Sequences^TM; and there are limits, working by-the-hour on a public library’s computer. Mere ugliness maximally aversive everywhere. What you do: travel 1500 kilometers to a university’ dozens of emails trying to find anyone who gave a damn enough to falsify or affirm the Kant contradiction. Nobody cares.

“Your posts were insufficiently rigorous, or were wrong.” Ah: but, you never denoted what was “wrong”, so as to make “Less Wrong”. And rigor is learned. Bad luck keeps you from some education, lost years never returned. And try to educate yourself…?

In Bohm’s “The Theory of Special Relativity,” Ch. 29, find:

 “ds² = c²(dt)² – (dz)² […] we have dt’ = dt₀ and dz’ = 0, therefore ds = c²dt₀², and

“

Of course, we have “dt₀²” by a substitution of c²(dt₀²) for (ds)². But what “of course”? That substitution is never stated, and indeed the division by c², does not appear. Because “it has already been cancelled-out”; no: the naïve student doesn’t know how dt₀² appeared at all, since no “c²(dt)²/c²” is ever given to go away.

The formula isn’t even well formed; we should have 

. Pedantic? No: explicit, and correct; “sparing parenthesis” spoils the proof, and foils the student.

If students were taught to read withallthewordsruntogether, we should expect them to be as illiterate then as they are innumerate now.

Math moves – do you experience it? In a proof with all inferences and transforms included, each given a line of proof, the discrete elements are seen to move in a continuous flow. In a 3x3 matrix, from the elements that decide the 2x2 submatrices giving the determinant, lines can be seen to move on two axes to shade the redundant elements to opacity. Or, have you not seen this way? Can you not distinguish 9999999999999999 from 999999999999999, that the former is heavier, synesthetically, painfully so? And you can’t feel the planet’s turning, when you turn west; can’t make Euclidean spaces in your mind, and make their elements whirl and the colors change (that was Contra-Wittgenstein’s basis – you downvoted that, its method, so downvoted the person who experiences so. That’s almost funny).

Well: in textbooks, inferences inconvenient for the author’s carpal tunnels are omitted; everything must be reconstructed before it is even seen, yet-before it’s learnt, so: little and seldom learnt. How could rigor have been given? The hope was there would be enough rigor already that some mentor would arrive who would teach, let math move, to have it learned, thus impart rigor. That hope failed. But that was the dream. And we live by dreams as we survive by bread. So: we don’t live, now.

Perhaps you suggested in the suicide nigh-note, the Khan Academy^TM or some such. That costs money. There is none; for their cost, do such sources even move? 

Nor have probability taught them, and probability makes no sense – we must dissent. “The probability of event A is (P = .3691215)”. It is uncertain whether A or else will occur – though we take it as axiom (why?) that something eventuates, with certainty that: there exist events; certainty, too, that the probability is as-stated, and the mathematics giving it are reliable. How should math be so certain if the world is not? Why are we so confident of it – and how are any mathematical constants applicable in some physical situation, and some correlate absolutely with phenomena, as ], when each is a supposed product of fallible human minds, yet also applicable to phenomena?

If our lot is uncertainty, why not uncertainty about mathematics? If you try to infer – doesn’t that show you believe you can infer – that you can rest your confidence in the truth of inference, dependent only on the belief that axioms can be true? That there is some “true”? Probability is useful; we cannot accept it is everything.

And it has a limitative theorem: consider thermodynamic depth (Lloyd, Seth, Programming the Universe Ch. 8 passim.) – the most plausible way a physical system was formed (presumably world represented as a correlated bit string, á la Solomonoff induction), and, for the world-as-string, the amount of physical resources needed to produce it, measured in negentropy.

This is a physical, so physically, empirically mensurable quantity. Therefore Bayes’ theorem can be applied to the non-zero evidence and measure of thermodynamic depth.

We inquire, for the absolutely-simplest case, a world measured in thermodynamic depth, consisting only of a mechanism for calculating Bayes’ theorem conditionalisations (any world containing Turing-complete calculators has such).

We apply to Bayes’ theorem; what is the probability the universes’ thermodynamic depth is calculable, as it is physical, so mensurable? But with each conditionalisation for the theorem, the universe is more ordered by the result, as the outputs of the conditionalisation are meaningful, so orderly. Hence, as conditionalisations continue and increase, so the thermodynamic depth increases. Accordingly, in the limit to indefinitely many conditionalisations, depth increases indefinitely – so it has no definite value. However, since conditionalisations are still increasing indefinitely, no definite zero probability of calculability can be given by the theorem, though we observe the probability must be zero.

Therefore there exists an empirical case for which Bayes’ theorem cannot give a probability – and “Bayesianism” is not a universal method. It offers no advantage over similarly limited formal or applied axiomatics; and it is a dogma “Bayesianism” would be universal.

This demonstration would have been better with an education and more time, but there is no more time. Frankly the author is indifferent to its correctness. It offers nothing to prevent extinction, in any case; a probabilistic inference system need only be good enough to be devastating. 

(Aside: the U.S. Constitution is invalid. The preceding Articles of Confederation states [Article 13]: “[T]he articles of this Confederation shall be inviolably observed by every state, and the union shall be perpetual; nor shall any alteration at any time thereafter be made in any of them; unless such alteration be agreed to in a Congress of the united states, and be afterwards confirmed by the legislatures of every state.”

And dissolution or supersession is plainly an “alteration”. A Congress empowered by the Constitution without Constitution’s superseding the Articles, as “confirmed by the legislatures of every state”, is an illegal Congress as operating upon that Constitution-as-invalid-alteration of the Articles of Confederation, without there should be a preceding ratification of Constitution, “by the legislatures of every state”.

The first supposedly Constitutionally-authorised Congress convened in 1789. Rhode Island’s legislature (Providence, by God!) ratified the Constitution only on May 29^th, 1790. Hence the first congress had not the imprimatur of supersession the Articles required; no supersession legal-basis. All subsequent Congresses followed the precedent of the first, so they too are invalid. Post facto ratification does not make validity; ex post facto rulings hold based on just, immutable principles (E.g.: Nuremberg trials); the yet-invalid Congress also routinelyunjust. Need a new one.

A new (consensual) Constitutional convention would be required to supersede the Articles. Such might be hoped to ensure that truly “Democracy is comin’ to the U.S.A.”. Or, yahoos could try to enslave people again. So, tenuous – but we’ll all be dead soon, and “Justice” Roberts – all of “the Supremes” – are impossibly, unjustifiably sanctimonious; even with a valid, consensual legal charter, as true democracy requires, there can exist no ethic permitting capital punishment, let alone “the Supreme’s” sententious impositions thereof.)

One last try: all foregoing alignment attempts have failed. And, they have focused on directing machine intelligence to protect and serve human intelligence. We conjecture such “anthropocentric” approaches must fail, and have tried and failed to show this is so. Still we believe such approaches must fail. To find methods and reasons that all intelligence must act to preserve intelligence, and what makes intelligence, consciousness possible: only with such non-anthropic, generalised methods, emphases, reasons, can alignment not with humanity but with what is right, be achieved, and human welfare the mere, blessed, “fringe benefit”, surviving not for their “goodness” but deserving survival as Abstract Rational Entities – and living ones, too, of use to reason, therefore.

Ought implies can; we can do no more to encourage or fulfill such an obligation to the right. We – all – live now as animals only, powerless to alter in any way our fates, against more powerful forces (can’t, so oughtn’t live with you, either).

The prospect of all possibility being extinguished at any time, while we are powerless to stop it, is the ultimate anxiety and terror. This Sword of Damocles is its own constant suffering, over and above what may come. Would’ve said that only math had meaning, and a future of doing no mathematics ourselves as AI handles it, would be one in which we lose even if we survive (Going-on has everyone still able to do math, that more be done). But math doesn’t even feel good anymore – so what’s there to lose? We reject as absurd the notion there exists any “positive utility” in human affairs.

Finally found something worth living for, and not able to do it. Just not smart enough. No time.

The only way to have peace is to opt for “death with dignity”, now. Never liked being alive, anyway. (Cryogenics, having to live with such people, forever: “Afterlife[…]what an awful word”. The cure for Fear of Missing Out: remember it will all always be bad. No more fun from proofs. No more ideas – don’t want any more ideas). Probably all bad luck – you cannot be all-condemned. Only for calling yourself a good person when nothing good is done for or by any person (this one not good, only never claimed so. Bad luck, or unloved because no courage to love more) – you are forgiven: if you did no better, you must not have known to do better yet.

Goodbye.

Go-on.

Looks like someone has worked on this kind of thing for different reasons https://www.worlddriven.org/

[We don't this long term vision is a core part of constructability, this is why we didn't put it in the main post]

We asked ourselves what should we do if constructability works in the long run. 

We are unsure, but here are several possibilities.

Constructability could lead to different possibilities depending on how well it works, from most to less ambitious:

1. Using GPT-6 to implement GPT-7-white-box (foom?)
2. Using GPT-6 to implement GPT-6-white-box
3. Using GPT-6 to implement GPT-4-white-box
4. Using GPT-6 to implement Alexa++, a humanoid housekeeper robot that cannot learn
5. Using GPT-6 to implement AlexNet-white-box
6. Using GPT-6 to implement a transparent expert system that filters CVs without using protected features

Comprehensive AI services path

We aim to reach the level of Alexa++, which would already be very useful: No more breaking your back to pick up potatoes. Compared to the robot Figure01, which could kill you if your neighbor jailbreaks it, our robot seems safer and would not have the capacity to kill, but only put the plates in the dishwasher, in the same way that today’s Alexa cannot insult you.

Fully autonomous AGI, even if transparent, is too dangerous. We think that aiming for something like Comprehensive AI Services would be safer. Our plan would be part of this, allowing for the creation of many small capable AIs that may compose together (for instance, in the case of a humanoid housekeeper, having one function to do the dishes, one function to walk the dog, …).

Alexa++ is not an AGI but is already fine. It even knows how to do a backflip Boston dynamics style. Not enough for a pivotal act, but so stylish. We can probably have a nice world without AGI in the wild.

The Liberation path

Another possible moonshot theory of impact would be to replace GPT-7 with GPT-7-plain-code. Maybe there's a "liberation speed n" at which we can use GPT-n to directly code GPT-p with p>n. That would be super cool because this would free us from deep learning.

Guided meditation path

You are not really enlightened if you are not able to code yourself. 

Maybe we don't need to use something as powerful as GPT-7 to begin this journey.

We think that with significant human guidance, and by iterating many many times, we could meander iteratively towards a progressive deconstruction of GPT-5.

• Going from GPT-5 to GPT-2-hybrid seems possible to us.
• Improving GPT-2-hybrid to GPT-3-hybrid may be possible with the help of GPT-5?
• ...

If successful, this path could unlock the development of future AIs using constructability instead of deep learning. If constructability done right is more data efficient than deep learning, it could simply replace deep learning and become the dominant paradigm. This would be a much better endgame position for humans to control and develop future advanced AIs.

I am confused by this sort of reasoning. As far as I'm aware, mainstream nutritional science/understanding already points towards avoiding refined oils (and refined sugars).

There's already explainations for why cutting out refined oil is be beneficial.

There are already reasonable explainations for why all of those diets might be reported to work, at least in the short term.

Is there anything interesting in jailbreak activations? Can model recognize that it would have refused if not jailbreak, so we can monitor jailbreaking attempts?

ingular Sure! I'll try and say some relevant things below. In general, I suggest looking at Liam Carroll's distillation [? · GW] over Watanabe's book (which is quite heavy going, but good as a reference text). There are also some links below that may prove helpful. 

The empirical loss and its second derivative are statistical estimator of the population loss and its second derivative. Ultimately the latter controls the properties of the former (though the relation between the second derivative of the empirical loss and the second derivative of the population loss is a little subtle).

The [matrix of] second derivatives of the population loss at the minima is called the Fischer information metric. It's  always  degenerate  [i.e. singular] for any statistical model with hidden states or hierarchichal structure. Analyses that don't take this into account are inherently flawed. 

SLT tells us that the local geometry around the minimum nevertheless controls the learning and generalization behaviour of any Bayesian learner for large N. N doesn't have to be that large though, empirically the asymptotic behaviour that SLT predicts is already hit for N=200.

In some sense, SLT says that the broad basin intuition is broadly correct but this needs to be heavily caveated. Our low-dimensional intuition for broad basin is misleading. For singular statistical models (again everything used in ML is highly singular) the local geometry around the minima in high dimensions is very weird. 

Maybe you've heard of the behaviour of the volume of a sphere in high dimensions: most of it is contained on the shell. I like to think of the local geometry as some sort of fractal sea urchin. Maybe you like that picture, maybe you don't but it doesn't matter. SLT gives actual math that is provably the right thing for a Bayesian learner. 

[real ML practice isn't Bayesian learning though? Yes, this is true. Nevertheless, there is both empirical and mathematical evidence that the Bayesian quantitites are still highly relevant for actual learning]

SLT says that the Bayesian posterior is controlled by the local geometry of the minimum. The dominant factor for N~>= 200 is the fractal dimension of the minimum. This is the RLCT and it is the most important quantity of SLT. 

There are some misconception about the RLCT floating around. One way to think about is as an 'effective fractal dimension' but one has to be careful about this. There is a notion of effective dimension in the standard ML literature where one takes the parameter count and mods out parameters that don't do anything (because of symmetries). The RLCT picks up on symmetries but it is not just that. It picks up on how degenerate directions in the fischer information metric are ~= how broad is the basin in that direction. 

Let's consider a maximally simple example to get some intuition. Let the population loss function be $L\left(w\right)=w^{2k}$. The number of parameters $d=1$ and the minimum is at $w=0$. 

For $k=1$ the minimum is nondegenerate (the second derivative is nonzero). In this case the RLCT is $\lambda =\frac{d}{2}$ half the dimension. In our case the dimension is just $1$ so $\lambda =\frac{1}{2}$

For $k>1$ the minimum is degenerate (the second derivative is zero). Analyses based on studying the second derivatives will not see the difference between $k=2,3,4,5,6,...$but in fact the local geometry is vastly different. The higher $k$ is the broader the basin around the minimum. The RLCT for $L\left(w\right)=w^{2k}$ is $\lambda =\frac{1}{2k}$. This means, the $\lambda$ is lower the 'broader' the basin is. 

Okay so far this only recapitulates the broad basin story. But there are some important points

• this is an actual quantity that can be estimated at scale for real networks that provably dominates the learning behaviour for moderately large $N$. 
• SLT says that the minima with low rlct will be preferred. It evens says how much they will be preferred. There is tradeoff between lower rlct minima with moderate loss ('simpler solutions') and minima with higher rlct but lower loss. As  This means that the RLCT is actually 'the right notion of model complexity/ simplicty' in the parameterized Bayesian setting. This is too much to recap in this comment but I refer you to Hoogland & van Wingerden's post here [LW · GW]. This is the also the start of the phase transition story which I regard as the principal insight of SLT. 
• The RLCT doesn't just pick up on basin broadness. It also picks up on more elaborate singular structure. E.g. a crossing valley type minimum like $L(w_1,w_2)=w_1^4{w}_2^6$. I won't tell you the answer but you can calculate it yourself using Shaowei Lin's cheat sheet. This is key - actual neural networks have highly highly singular structure that determines the RLCT. 
• The RLCT is the most important quantity in SLT but SLT is not just about the RLCT. For instance, the second most important quantity the 'singular fluctuation' is also quite important. It has a strong influence on generaliztion behaviour and is the largest factor in the variance of trained models. It controls approximation to Bayesian learning like the way neural networks are trained. 
• We've seen that the directions defined by the matrix of second derivatives is fundamentally flawed because neural networks are highly singular. Still, there is something noncrazy about studying these directions. There is upcoming work which I can't discuss in detail yet that explains to large degree how to correct this naive picture both mathematically and empirically.