LessWrong 2.0 Reader

Thanks, this is interesting.

My understanding is that cavities are formed because the very local pH on that particular sub-part of the tooth is below 5.5. IIUC teeth can't get cancer. Are you imagining Lumina colonies on the gums having this effect there, the Lumina colonies on the teeth affecting the general oral environment (which I think would require more calculation than just comparing to the hyper-local cavity environment) or am I misunderstanding something?

Could you explain more about being coercive towards subagents? I'm not sure I'm picking up exactly what you mean.

A (probably-fake-)framework I'm using is to imagine my mind being made up of subagents [? · GW] with cached heuristics about which actions are good and which aren't. They function in a sort-of-vetocracy—if any one subagent doesn't want to engage in an action, I don't do it. This can be overridden, but doing so carries the cost of the subagent "losing trust" in the rest of the system and next time putting up even more resistance (this is part of how ugh fields [LW · GW] develop).

The "right" way to solve this is to find some representation of the problem-space in which the subagent can see how its concerns are adressed or not relevant to the situation at hand. But sometimes there's not enough time or mental energy to do this, so the best available solution is to override the concern.

This seems right. One thing I would say is that kind of surprisingly it hasn't been the most aversive tasks where the app has made the biggest difference, it's the larger number of moderately aversive tasks. It makes expensive commitments cheap and cheap commitments even cheaper, and for me it has turned out that cheap commitments have made up most of the value.

Maybe for me the transaction costs are still a bit too high to be using commitment mechanisms, which means I should take a look at making this smoother.

Most typical skin cancer is basiloma - and it is rather benign - no metastases and can be removed without hospitalization. Many people get it.

Glad to be of help! I was almost put off by the overly rave reviews when I first tried it lol but now I can imagine myself writing one.

I haven't tried Focusmate, although I do do timed work sessions (and did before I used the app) and now use the app to enforce a certain amount of time per day, so this achieves a similar effect.

I registered on forfeit and see that it has pomodoro forfeits in the app, so using it for focusmate may be redundant.

It might be redundant, although one thing that is really great about it is that because screenshots are the universal interface you can keep using the apps you already use and just use Forfeit to submit evidence. This is why I still use Habitica even though I could equally put the same tasks in Forfeit directly, there is just no particular reason to do so.

Also, if you don't mind me asking, what is the commitment you most often fail? Is it time spent on twitter and youtube?

I've looked through at the ones I have failed and it looks like "reply to X person" and "publish blog post" are the categories that stand out. For replying to people I deliberately set the penalty low to begin with (and increase it if I fail the first time) so this is by design. For publishing blog posts, there aren't that many instances but I have failed >50% of them, here I was generally underestimating the effort it takes.

I actually rarely fail the twitter one because the penalty is quite high (and I block it via Freedom for the rest of the day if I'm getting close to the limit).

Could you explain more about being coercive towards subagents? I'm not sure I'm picking up exactly what you mean.

Some thoughts related to what I am interpreting you as meaning: I have found that my problems have shifted from not being able to follow through on things, to being almost too able to follow through on things, and thus getting entrenched in a pre-committed plan when I should change it (for instance repeating a time consuming daily habit that isn't doing much for me).

I see this as a good problem to have. When I was less able to execute on things my plans were just as bad, I just never found out because I never got as far as completing them. The problem of planning is still hard though, and isn't solved by the app, although I have found that it helps a lot [LW · GW].

I think these apps work best if really aversive tasks are set to a estimated-bare-minimum commitment, which in reality is probably the median amount of what one can get done.

This seems right. One thing I would say is that kind of surprisingly it hasn't been the most aversive tasks where the app has made the biggest difference, it's the larger number of moderately aversive tasks. It makes expensive commitments cheap and cheap commitments even cheaper, and for me it has turned out that cheap commitments have made up most of the value.

It has made a difference to the most aversive ones, I just don't think the category of extremely aversive things makes up that much of the value in my life.

This is possibly a way in which it's better than Beeminder, which I understand is oriented towards at least medium term goals (where you make several increments of progress along the yellow brick road). By number most of the things I have put in Forfeit have been one-off things that would be too small to be worth Beeminding (unless you could group them into a higher level of abstraction).

Another constraint is from computational complexity; should we treat things that are not polynomial-time computable as basically unknowable? Humans certainly can't solve NP-complete problems efficiently.

Generalized chess is EXPTIME-complete and while chess "exact solution" may be unavailable, we are pretty good at constructing chess engines.

Copyright seems to behave significantly different from the other categories. This is likely due to it being out of distribution from the subversive fine-tuning dataset used.

It would be nice to have a conclusion for the graph below this text. Like Mixtral refuses x% less than Llama 2 by default, and can be brought down to y%, compared to z% for Llama

For example, a researcher I've been talking to, when asked what they would need to update, answered, "An AI takes control of a data center." This would be probably too late.

Very much to avoid, but I'm skeptical it 'would be probably too late' (especially if I assume humans are aware of the data center takeover); see e.g. from What if self-exfiltration succeeds?:

'Most likely the model won’t be able to compete on making more capable LLMs, so its capabilities will become stale over time and thus it will lose relative influence. Competing on the state of the art of LLMs is quite hard: the model would need to get access to a sufficiently large number of GPUs and it would need to have world-class machine learning skills. It would also mean that recursive self-improvement is already possible and could be done by the original model owner (as long as they have sufficient alignment techniques). The model could try fine-tuning itself to be smarter, but it’s not clear how to do this and the model would need to worry about currently unsolved alignment problems.'

A 10-year global pause would allow for a lot of person-years-equivalents of automated AI safety R&D. E.g. from Some thoughts on automating alignment research [LW · GW] (under some assumptions mentioned in the post): 'each month of lead that the leader started out with would correspond to 15,000 human researchers working for 15 months.' And for different assumptions the numbers could be [much] larger still: 'For a model trained with 1000x the compute, over the course of 4 rather than 12 months, you could 100x as many models in parallel.[9] You’d have 1.5 million researchers working for 15 months.'

This would probably obsolete all previous AI safety R&D.

Of course, this assumes you'd be able to use automated AI safety R&D safely and productively. I'm relatively optimistic that a world which would be willing to enforce a 10-year global pause would also invest enough in e.g. a mix of control and superalignment to do this.

There's also an entire literature of variations of [e.g. sparse or disentangled] autoencoders and different losses and priors that it might be worth looking at and that I suspect SAE interp people have barely explored; some of it literally decades-old. E.g. as a potential starting point https://lilianweng.github.io/posts/2018-08-12-vae/ and the citation trails to and from e.g. k-sparse autoencoders.