LessWrong 2.0 Reader

At job, what works for me is making notes. For each task, I start a new page in a note-making software, and put there everything related to the task: link to the Jira ticket, short description, people to contact about analysis and testing, links to relevant resources, etc. Sometimes I also write an outline like "first I will do this, then this". Then I start working on the task, adding more information as it emerges: things that people told me, things I found in the source code, links to the commits and pull requests I made, etc.

The reason is that interruptions are frequent (both planned and unplanned) and seems like I can't do much about it, but the thing I can do is make it easier to recover after the interruption. This way I can make a use of a short block of time, by reading about the planned next step in my notes, doing it, and adding a note about the result.

Unfortunately, the same strategy does not work for me in my private life. I am not sure why, but I have a few suspicions. In private life I have to play both the role of the manager (decide what to do) and the individual contributor (actually do it); my current version of the system works okay for the latter but not for the former. The difficult part is to make myself continue working on the interrupted project, when there are so many alternatives.

Without interruptions, this is automatic. It is difficult for me to start working on something, but once I do, I can easily get obsessed, and could continue working on the same thing for days. That is how I accomplished some things when I was single and childless; I knew that the right time for projects was weekends, especially the ones that had a holiday on Friday or Monday. I could work on something for 4 days in a row, only taking breaks for food and sleep. But now that I have kids, I simply don't get that amount of uninterrupted time, ever.

Interruptions at work are not just difficult for me, but also very unpleasant. It feels like getting hurt in some mental way; having my autonomy violated. Forcing myself to start doing something when I am not in the mood, it hurts. Finally getting in the mood as I am doing it, and then being forced to stop, it hurts again. To have an interruption looming ahead of me means to expect to get hurt soon... that is, if I actually start working on the project. The unpleasant feelings accumulate and result in aversion against the task they are associated with. The more I get interrupted trying on work on a certain task, the more I hate the task. At work, I usually don't have a choice, and I have to finish the task anyway. In private, this makes me abandon projects, or procrastinate on them a lot.

Not all interruptions have the same effect. Taking a break to eat, sleep, exercise, or take a walk is okay. Those are simple activities, so I can continue thinking about the project in the background. The bad kind of interruption is when I need to think hard about something different, when I need to solve a different problem.

There's an entire field of psychology, yes, but most men are still confused by women saying "it's fine" when they are clearly annoyed. Another thing is women dressing up because they want attention from specific men. Dressing up in a sexy manner is not a free ticket for any man to harass them, but socially inept men will say "they were asking for it" because the whole concept of selection and stardards doesn't occur to them in that context. And have you read Niccolò Machiavelli's "The Prince"? It predates psychology, but it is psychology, and it's no worse than modern books on office politics and such, as far as I can tell. Some things just aren't improving over time.

I think majority of people aren't aware of psychology and various fields under it. Ethics and decision theory kind of give a lot of clarity into such decisions when you analyse the payoff matrix. I haven't The prince but have read excerpts from it in self-improvement related diaspora, I am not denying the value which such literature gives us, I just think we should move on by learning from it and developing on top in light of newer methods. 

Beside I am more of a moral anti-realist so lol. I don't think there is universally compelling arguments [LW · GW]for these ethical things, but people with enough common psychological and culture grounds can cooperate.  [LW · GW]

Modern hard sciences like mathematics are too inhuman (autistic people are worse at socializing because they're more logical and objective). 

Well it depends on your definition of inhuman, my_inhuman =/= your_inhuman value is a two place function [LW · GW], my peers when I was in high school found at least one of the hard sciences fun. Like them I find hard sciences pretty cool [LW · GW] to learn about for fulfilling my other goals. 

And modern soft sciences are frankly pathetic 

Agreed. Some fields under psychology as pathetic. But the fields like cognitive biases etc are not. 

and that it hasn't failed you much in the social parts?

Well astrology has clearly failed me, my mom often had these luddite-adjacent ideas about what I am meant to do in life because her entire source of ethics was astrology. Astrology in career advice is like rolling a dice and assigning the all the well known professions to a number rather than actual life satisfaction or value fulfillment. 

"The rules of the brain are different than those of math, if you treat the brain like it's supposed to be rational, you will always find it to be malfunctioning for reasons that you don't understand"

I would strongly disagree on the front of intelligence [? · GW] . More Rational as in cognitive algorithms which tend to lead to systematic optimality in this case truth seeking/achieving goals is indeed possible and pretty much is a part of growth.

I would weakly disagree on the front of Internal family subsystems [? · GW] (with the internal double crux [? · GW] special case being extremely useful) and other introspective reductionist methods where you   break  [? · GW]  down  [? · GW]  your  [? · GW] emotional responses and process into parts and understand [? · GW] what you like/dislike and the various attempts to bridge the two. On this front there are plethora [? · GW]of competing theories due to easy problem of consciousness [LW · GW] and trying to understand experience functionally.   

And for brain not working as I want to be when I model other parts of this brain, I find it being emotionally engaged in things which aren't optimal for some of my goals and it isn't contradictory with rationality to acknowledge or deal with these feelings. 

I was praising goggins because he's more of the type who is willing to fight himself and in more than half of the introspective models that without acknowledgement is bordering on self-harm. I find his strategy to be intuitively much better lol. 

Where I would agree is that if you don't understand something then your theory is probably wrong. There are not confusing facts only models which are confused by facts. 

Too many geniuses have failed at living good lives for me to believe that intelligence is enough. 

I think growth is important, I like to think of it in intelligence being compute power and growth and learning being more of changing algorithms. Besides there is a good amount of coorelations with IQ you might want to look into, I think this area is very contentious (got a system1 response to check for the social norms due to past bans lol) , but we're on lesswrong, so you can continue. 

This might be why I have the courage to criticize science on LW in the first place.

You're welcome, maybe you should read sequence highlights to get introduced with LW's POV to understand other people's positions here. 

Why don't LLM's ask clarifying questions?

Caveat: I know little to nothing about the architecture of such things, please take this as naive user feedback if you wish, or you could ignore it.

Just now I was asking the Meta AI chatbot how to do an 'L-Cut' using the Android Instagram app. It hallucinated for quite a few messages instructions how to 'overlap' two video tracks when editing a reel before it finally admitted that no such ability in fact exists in the Instagram App.

My grossly negligent mistake was assuming that a AI LLM with Meta Branding would have current or accurate knowledge of Meta properties and products. 

However, imagine that there are two versions of the Instagram App, one that has this feature and one that doesn't - why didn't it ask "Just to check, what kind of phone are you using?" (which, also, would allay fears of 'spying' and invasion of privacy) and then, and only then give me advice or apologetically inform me that such a feature is not available. (In other branded/commercial chatbots, wouldn't this be an opportunity for an up-sell?^[1])

Maybe I'm not prompting LLMs in the right way, but they almost never ask me questions. And certainly never clarification questions. They make an assumption, and produce a slab of text with a boilerplate "let me know if this helps?" tacked on the end. I assume this is a conscious choice and not a limitation of the technology.
To test my observation I entered the same prompt into three different LLMs:

I just asked Claude Haiku 3 "When was Rosberg's first Grand Prix?" to see if it assumes I mean Keke Rosberg, or his son, Nico Rosberg.Claude apologies for not having information on Nico Rosberg's first Grand Prix... and then goes on to state that "Formula One has had multiple drivers named Rosberg compete over the years. More details would be needed to determine the exact driver and their debut race."

What I find interesting, not one sentence in the answer takes the form of an interrogative!

Next, I asked GPT-4o mini the exact same question, to which it confidently assumes which Rosberg I mean: "Nico Rosberg's first Formula One Grand Prix was at the 2006 Bahrain Grand Prix, which took place on March 12, 2006. He raced for the Williams team."

Microsoft CoPilot makes the same assumption, with the traditional tacked on boilerplate question:
"Nico Rosberg made his Formula One debut at the 2006 Bahrain Grand Prix. He went on to have a successful career, culminating in winning the World Drivers' Championship in 2016. Is there anything else you'd like to know about his career or Formula One in general? [FI car Emoji, Smoke Emoji]"

Meta AI makes the same assumption, and gives a lot more unsolicited information, but it does mention his father:
"Nico Rosberg's first Grand Prix was the 2006 Bahrain Grand Prix. This marked the beginning of his Formula One career, which would eventually see him drive for Williams from 2006 to 2009 and then Mercedes from 2010 to 2016. During his time with Mercedes, Rosberg won the Formula One Driver's championship in 2016, a feat that made him and his father, Keke Rosberg, the second-father pairing to achieve this impressive milestone" 
Not that it is relevant, but Graham and Damon Hill were the first.

This is not an exhaustive sample by any means, but does confirm my naive observation that LLMs do not ask clarifying questions or use interrogatives in order to better shape their responses to my needs. 

1. ^{^}

   I imagine such a commercial hellscape would look a little like this:
   
   "I was just wondering why I wasn't earning any Ultrabonus points with my purchases"
   "Before we continue, could you tell me, do you have a Overcharge Co. Premium savings account, or a Overcharge Co. Platinum savings account?"
   "Uhh I think it is a Premium."
   "I'm so sorry. if you have a Overcharge Co. Platinum savings account then you will not be able to enjoy our Overcharge co. ultrabonus points loyalty system. However you may be suprised that for only a small increase in account fee, you too can enjoy the range of rewards and discounts offered with the Overcharge co. ultrabonus points loyalty system. Would you like to learn more?"

I agree with this analysis. I mean, I'm not certain further optimization will erode the interpretability of the generated CoT, its possible the fact its pretrained to use human natural language pushes it in a stable equilibrium, but I don't think so, there are ways the CoT can become less interpretable in a step-wise fashion.

But this is the way its going, seems inevitable to me. Just scaling up models and then training them on English language internet text, is clearly less efficient (from a "build AGI" perspective, and from a profit-perspective) than training them to do the specific tasks that the users of the technology want. So thats the way its going. 

And once you're training the models this way, the tether between human-understandable concepts and the CoT will be completely destroyed. If they stay together, it will just be because its kind of a stable initial condition.

 

I agree with your technical points, but I don't think that we could particularly expect the other path. Safety properties of LLMs seem to be desirable from extremely safety-pilled point of view, not from perspective of average capabilities researcher and RL seems to be The Answer to many learning problems.

In the late 80's, I was homeschooled, and studied caligraphy (as well as cursive); but I considered that more of a hobby than preparation for entering the workforce of 1000 years ago. 

I also learned a bit about DOS and BASIC, after being impressed with the fractal-generating program that the carpenter working on our house wrote, and demonstrated on our computer. 

I believe o1-type models that are trained to effectively reason out loud may actually be better for AI safety than the alternative. However, this is conditional on their CoTs being faithful, even after optimization by RL. I believe that scenario is entirely possible, though not necessarily something that will happen by default. See the case for CoT unfaithfulness is overstated [LW · GW], my response [LW · GW] with specific ideas for ensuring faithfulness, and Daniel Kokotajlo's doc along similar lines.

There seems to be quite a lot of low-hanging fruit here! I'm optimistic that highly-faithful CoTs can demonstrate enough momentum and short-term benefits to win out over uninterpretable neuralese, but it could easily go the other way. I think way more people should be working on methods to make CoT faithfulness robust to optimization (and how to minimize the safety tax of such methods).

If someone's only option for dealing with a hostile telepath is self-deception, and then you come in and punish them for using it, thou art a dick.

Like, do you think it helps the abused mothers I named if you punish them somehow for not acknowledging their partners' abuse? Does it even help the social circle around them?

If that's their only option, and the hostility in your telepathy is antisocial, then yes. In some cases though, people do have other options and their self-deception is offensive, so hostile telepathy is pro-social. 

For example, it would probably help those mothers if the men knew to anticipate punishment for not acknowledging their abuse of their partners. I bet at least one of those abusive husbands/boyfriends will give his side of the story that's a bit more favorable than "I'm a bad guy, lol", and that it will start to fall apart when pressed. In those cases, he'll have to choose between admitting wrongdoing or playing dumb, and people often do their best to play really dumb. The self-deception there is a ploy to steal someone else's second box, so fuck that guy.

I think the right response is to ignore the "self" part of the deception and treat it like any other deception. If it's okay to lie to the Nazis about hiding Jews, then it's okay to deceive yourself into believing it too. If we're going to make it against the law to lie under oath, then making it legal so long as they lie to themselves too is only going to increase the antisocial deception.

Thanks for this perspective.

The therapy paradigm you describe here (going to a clinic to receive Spravato), is, as you point out, difficult and bureaucratic.

Through a regulatory loophole, there's another pathway where you can get ketamine sent to your house with less bureaucracy. https://www.mindbloom.com/ is the main provider I know of. They're very expensive, but in theory this could be done for cheap and maybe other providers are doing it, I don't know. If you have a cooperative psychiatrist, you can see if they know about this version and are willing to prescribe it.

As you point out, ketamine lasts a few weeks and then some people will crash back to their previous level of depression. If I am able to successfully treat a patient with ketamine, I usually recommend they continue it for six months, just like any other antidepressant. A cooperative doctor can do this by prescribing it to a cooperative compounding pharmacy. I don't know if Mindbloom or other companies provide this service by default. Obviously this is easier when you're doing the version in your house than if you have to go to a clinic each time.

I've written more of my thoughts about ketamine at https://lorienpsych.com/2021/11/02/ketamine/

I'm curious what your read of the history is, here? My impression is that most important paradigm-forming work so far has involved empirical feedback somehow, but often in ways exceedingly dissimilar from/illegible to prevailing scientific and engineering practice.

I have a hard time imagining scientists like e.g. Darwin, Carnot, or Shannon describing their work as depending much on "immediate feedback loops with present day" systems.

Thanks for the comment @Adam Scholl [LW · GW] and apologies for not addressing it sooner, it was on my list but then time flew. I think we're in qualitative agreement that non-paradigmatic research tends to have empirical feedback loops, and that the forms and methods of empirical engagement undergo qualitative changes in the formation of paradigms. I suspect we may have quantitative disagreements with how illegible these methods were to previous practitioners, but I don't expect that to be super cruxy.

The position which I would argue against is that the issue of empirical access to ASI necessitates long bouts of philosophical thinking prior to empirical engagement and theorization. The position which I would argue for is that there is significant (and depending on the crowd undervalued) benefit to be gained for conceptual innovation by having research communities which value quick and empirical feedback loops. I'm not an expert on either of these historical periods, but I would be surprised to hear that Carnot or Shannon did not meaningfully benefit from engaging with the practical industrial advancements of their day.

Giving my full models is out of scope for a comment and would take a sequence which I'll probably never write, but the 3 history and philosophy of science references which have had the greatest impact on my thinking around empiricism which I tend to point people towards would probably be Inventing Temperature, Exploratory Experiments, and Representing and Intervening.

So I'm curious whether you think PIBBSS would admit researchers like these into your program, were they around and pursuing similar strategies today?

In short I would say yes, because I don't believe the criteria listed above excludes the researchers which you called attention to. But independently of whether you buy into that claim, I would stress that different programs have different mechanisms of admission. The affiliateship as it's currently being run is designed for lower variance and is incidentally more tightly correlated with the research tastes of myself and the horizon scanning team [LW · GW] given that these are the folks providing the support for it. The summer fellowship is designed for higher variance and goes through a longer admission process involving a selection committee, with the final decisions falling on mentors.