Posts
Comments
Another definition along the same vein:
Trauma is overgeneralization of emotional learning.
A real life use for smart contracts 😆
However, this would not address the underlying pattern of alignment failing to generalize.
Is there proof that this is an overall pattern? It would make sense that models are willing to do things they're not willing to talk about, but that doesn't mean there's a general pattern that e.g. they wouldn't be willing to talk about things, and wouldn't be willing to do them, but WOULD be willing to some secret third option.
I don't remember them having the actual stats, not watching it again though. I wonder if they published those elsewhere
They replicated it within the video itself?
Enjoyed this video by Veritasium with data showing how Politics is the Mind Killer
I'll send out to you round 2 when I've narrowed things done. Right now I'm looking for gut check system 1 decisions, and if you have trouble doing tahat I'd recommend waiting.
Want to help me out?
Vote on the book cover for my new book!
It'll be up for a couple of days. The contest website only gives me a few days before I have to pick finalists.
https://form.jotform.com/243066790448060
IME you can usually see in someone's face or body when they have a big release, just from the release of tension.
But I think it's harder to distinguish this from other hypotheses I've heard like "negative emotions are stored in the tissues" or "muscular tension is a way of stabilizing intentions."
Oh yes, if you're going on people's words, it's obviously not much better, but the whole point of vibing is that it's not about the words. Your aesthetics, vibes, the things you care about will be communicated non-verbally.
I predict you would enjoy the free-association game better if you cultivated the skill of vibing more.
Yes, this is an excellent point I didn't get across in the past above.
Yes, if people were using Wikipedia in the way they are using the LLMs.
In practice that doesn't happen though, people cite Wikipedia for facts but are using LLMs for judgement calls.
Of course a random person is biased. Some people will will have more authority than others, and we'll trust them more, and argument screens off authority.
What I don't want people to do is give chatGPT or Claude authority. Give it to the wisest people you know not Claude.
What they're saying is I got a semi-objective answer fast.
Exactly. Please stop saying this. It's not semi-objective. The trend of casually treating LLMs as an arbiter of truth leads to moral decay.
I doubt the orga got much of their own bias into the RLHF/RLAIF process
This is obviously untrue, orgs spend lots of effort making sure their AI doesn't say things that would give them bad press for example.
I desperately want people to stop using "I asked Claude or ChatGPT" as a stand-in for "I got an objective third party to review"
LLMs are not objective. They are trained on the internet which has specific sets of cultural, religious, ideological biases, and then further trained via RL to be biased in a way that a specific for-profit entity wanted them to be.
I happened to log on at that time and thought someone had launched a nuke
So far I’m seeing data that’s strongly in favor of it being easy for me to facilitate rapid growth for many people in this space. But am I missing something here? If you have any ideas please let me know in the comments.
My take:
You can facilitate rapid growth in these areas.
I don't think you're particularly unique in this regard. There are several people who I know (myself included) who can create these sorts of rapid changes on a semi-consistent basis. You named a few as reviewers. There are far more coaches/therapists who are ineffective, but lots of highly effective practitioners who can create rapid change using experiential methods.
@PJ_Eby @Kaj_Sotala @Damon Sasi all come to mind as people on LW who can do this. Having worked with many coaches and therapists, I assure you that many others also have the skill.
Right now I think you're overestimating just how consistent what you do is, and the results focus you're taking is likely creating other negative effects in the psyche that will have to be cleaned up later. It will also mean that if you don't get to the issue in the first session, it will be harder and harder for your work to have an impact over time.
But in general the approach you're taking can and will create rapid results in some people that haven't seen results before.
I've really been enjoying Charlie Anderson's YouTube channel for this reason, trying to find the absolute best way to make pizza.
https://youtube.com/@charlieandersoncooking?si=uhpLcNDyE7jLbTMY
It seems like the obvious thing to do with a model like o1 trained on reasoning through problems would be to train it to write code that helps it solve reasoning problems.
Perhaps the idea was to not give it this crutch so it could learn those reasoning skills without the help of code.
But it seems like from the examples that while its great at high level reasoning and figuring out where it went wrong, it still struggles with basic things like counting, which, if it had the instinct to write code in those areas which it's likely to get tripped up, would be easily solved.
Sorta surprised that this got so many up votes with the clickbaity title, which goes against norms around here
Otherwise th content seems good
I'm not talking about 10 year time horizons no
we know that's not what US executives were thinking because they don't think that long-term due to the incentives they face
The story of "they're doing something that's bad in the short term, but good in the long term, but only accidentally they're actually trying to do something good in the short term but failing" seems suspicious.
I know that the CEOs I know do plan in the long term.
I also know that the many of the worlds most famous consumer brands (Apple, Amazon. Tesla) have valuations that only make sense because people trust the CEOs to prioritize the long term and those future earnings are priced in.
And I also know that if you look at the spending budget of many of the top consumer tech companies, and the amount spent on longterm R&D and moon shots, it sure looks like they are spending on the long term.
I don't think that's the sophisticated argument for switching your in house app to the cloud. There's a recognition that because it's more efficient for developers, more and more talent will learn to use and infrastructure will be built on top of cloud solutions.
Which means your organization risks being bottlenecked on talent and infrastructure if you fall too far behind the adoption curve.
When magazines talked about, say, "microservices" or "the cloud" being the future, it actually made them happen. There are enough executives that are gullible or just want to be given something to talk about and work on that it established an environment where everyone wanted to get "microservices" or whatever on their resume for future job requirements, and it was self-sustaining.
Is the claim here that cloud computing and microservice architectures are less efficient and a mistake?
median rationalist at roughly MENSA level. This still feels wrong to me: if they’re so smart, where are the nobel laureates? The famous physicists? And why does arguing on Lesswrong make me feel like banging my head against the wall?
I think you'd have to consider both Scott Aaronson and Taylor Cowen to be rationalist adjacent, and both considered intellectual heavyweights
Dustin Moskovitz EA adjacent, again considered a heavyweight, but applied to business rather than academia
Then there's the second point, but unfortunately I haven't seen any evidence that someone being smart makes them pleasant to argue with (the contrary in fact)
The whole first part of the article is how this is wrong, due to the gaming of notable sources
One way that think about "forces beyond yourself" is pointing to what it feels like to operate from a right-hemisphere dominant mode, as defined by Ian McGilcrist.
The language is deliberately designed to evoke that mode - so while I'll get more specific here, know that to experience the thing I'm talking about you need to let go of the mind that wants this type of explanation in order to experience what I'm talking about.
When I'm talking about "Higher Forces" I'm talking about states of being that feel like something is moving through you - you're not a head controlling a body but rather you're first connecting to, then channeling, then becoming part of a larger universal force.
In my coaching work, I like to use Phil Stutz's idea of "Higher forces" like Infinite Love, Forward Motion, Self-Expression, etc, as they're particularly suited for the modern Western Mind.
Here's how Stutz defines the higher force of Self-Expression on his website:
"The Higher Force You’re Invoking: Self-Expression The force of Self-Expression allows us to reveal ourselves in a truthful, genuine way—without caring about others' approval. It speaks through us with unusual clarity and authority, but it also expresses itself nonverbally, like when an athlete is "in the zone." In adults, this force gets buried in the Shadow. Inner Authority, by connecting you to the Shadow, enables you to resurrect the force and have it flow through you."
Of course, religions also have names for these type of special states, calling them Muses, Jhanas, Direct Connection to God.
All of these states (while I can and do teach techniques, steps, and systems to invoke them) ultimately can only be accessed through surrender to the moment, faith in what's there, and letting go of a need for knowing.
It's precisely when handing your life to forces beyond yourself (not Gods, thats just handing your life over to someone else) that you can avoid giving your life over to others/society.
Souls is metaphorical of course, not some essential unchanging part of yourself - just a thing that actually matters, that moves you
In the early 2000s, we all thought the next productivity system would save us. If we could just follow Tim Ferriss's system and achieve a four-hour workweek, or adopt David Allen's "Getting Things Done" (GTD) methodology, everything would be better. We believed the grind would end.
In retrospect, this was our generation's first attempt at addressing the growing sacredness deficit disorder that was, and still is, ravaging our souls. It was a good distraction for a time—a psyop that convinced us that with the perfect productivity system, we could design the perfect lifestyle and achieve perfection.
However, the edges started to fray when put into action. Location-independent digital nomads turned out to be just as lonely as everyone else. The hyper-productive GTD enthusiasts still burned out.
For me, this era truly ended when Merlin Mann, the author of popular GTD innovations like the "hipster PDA" and "inbox zero," failed to publish his book. He had all the tools in the world and knew all the systems. But when it mattered—when it came to building something from his soul that would stand the test of time—it didn't make a difference.
Merlin wrote a beautiful essay about this failure called "Cranking" (https://43folders.com/2011/04/22/cranking). He mused on the sterile, machine-like crank that would move his father's bed when he could no longer walk. He compared this to the sterile, machine-like systems he used to get himself to write, not knowing what he was writing or why, just turning the crank.
No amount of cranking could reconnect him to the sacred. No system or steps could ensure that the book he was writing would touch your soul, or his. So instead of sending his book draft to the editor, he sent the essay.
Reading that essay did something to me, and I think it marked a shift that many others who grew up in the "productivity systems" era experienced. It's a shift that many caught up in the current crop of "protocols" from the likes of Andrew Huberman and Bryan Johnson will go through in the next few years—a realization that the sacred can't be reached through a set of steps, systems, lists, or protocols.
At best, those systems can point towards something that must be surrendered to in mystery and faith. No amount of cranking will ever get you there, and no productivity system will save you. Only through complete devotion or complete surrender to forces beyond yourself will you find it.
I just realized that this then brings the problem of "oh, but what's the meta-meta strategy i use), but I think there's just an element of taste to this.
One thing to note - Brainstorming itself is a meta-strategy that may or may not be the best approach at certain points in the problem, to generate meta-strategic approaches.
Brainstorming for me has a particular flavor - it's helpful when I have a lot of ideas but don't know where to start, or when it feels like my mind just needs the starter cord pulled a few times.
Other times, I get a lot more out of a taking a walk and let my mind wander around the problem, not specifically listing out lanes of attack, but sort of holding the intention that one may show up as I think in a free associative way and walk.
Other times it's helpful for me to have a conversation with a friend, especially one who I can see has the right mind-shape to frame this sort of problem.
Other times it's helpful to specifically look through the list of meta-strategies I have, wandering around my Roam and seeing how different mental models and frameworks can frame the problem.
I guess what I'm saying is, it's helpful to separate the move of "oh, it's time to figure out what meta-strategy I can use" from "oh, it's time to brainstorm"
For those who disagreed, I'd love to be linked to convincing arguments to the contrary!
I've heard several people who should know (Musk, Ascher) make detailed cases that seem right, and haven't heard any convincing arguments to the contrary.
But once they break the data-wall, competitors are presumably gonna copy their method.
Is the assumption here that corporate espionage is efficient enough in the AI space that inventing entirely novel methods of training doesn't give much of a competitive advantage?
i don't think the constraint is that energy is too expensive? i think we just literally don't have enough of it concentrated in one place
but i have no idea actually
Zuck and Musk point to energy as a quickly approaching deep learning bottleneck over and above compute.
This to me seems like it could slow takeoff substantially and effectively create a wall for a long time.
Best arguments against this?
Paul Ekmans software is decent. When I used it (before it was a SaaS, just a cd) it just basicallyflashed an expression for a moment then went back to neutral pic. After some training it did help to identify micro expressions in people
People talk about unconditional love and conditional love. Maybe I’m out of the loop regarding the great loves going on around me, but my guess is that love is extremely rarely unconditional. Or at least if it is, then it is either very broadly applied or somewhat confused or strange: if you love me unconditionally, presumably you love everything else as well, since it is only conditions that separate me from the worms.
Yes. this is my experience of cultivating unconditional love, it loves everything without target. I doesn't feel confused or strange, just like I am love, and my experience e.g. cultivating it in coaching is that people like being in the present of such love.
It's also very helpful for people to experience conditional love! In particular of the type "I've looked at you, truly seen you, and loved you for that."
IME both of these loves feel pure and powerful from both sides, and neither of them are related to being attached, being pulled towards or pushed away from people.
It feels like maybe we're using the word love very differently?
Both causal.app and getguesstimate.com have pretty good monte carlo uis
IME there is a real effect where nicotine acts as a gateway drug to tobacco or vaping
in general this whole post seems to make this mistake of saying 'a common second order effect of this thing is doing it in a way that will get you addicted - so don't do that' which is just such an obvious failure mode that to call it a chesterton fence is generous
The question is - how far can we get with in-context learning. If we filled Gemini's 10 million tokens with Sudoku rules and examples, showing where it went wrong each time, would it generalize? I'm not sure but I think it's possible
It seems likely to me that you could create a prompt that would have a transformer do this.
i like coase's work on transaction costs as an explanation here
coase is an unusually clear thinker and writer, and i recommend reading through some of his papers
i just don't see the buddha making any reference to nervous systems or mammalians when he talks about suffering(not even some sort of pali equivalent that points to the materialist understanding at the time)
? TBC I think the claims about suffering in Buddhism are claims about how our mammalian nervous systems happen to be wired and ways you can improve it.
This seems like quite a western modern take on buddhism
it feels hard to read the original buddha this way
Compare, the world will be exactly as it has been in the past, with the world will always be exactly as it is in this moment
it's true, but I don't think there's anything fundamental preventing the same sort of proliferation and advances in open source LLMs that we've seen in stable diffusion (aside from the fact that LLMs aren't as useful for porn). that it has been relatively tame so far doesn't change the basic pattern of how open source effects the growth of technology
yeah, it's much less likely now
it doesn't seem like that's the case to me - but even if it were the case, isn't that moving the goal posts of the original post?
I don't think time-to-AGI got shortened at all.