LessWrong 2.0 Reader

On your categories:

As simulator theory [? · GW] makes clear, a base model is a random generator, per query, of members of your category 2. I view instruction & safety training that to generate a pretty consistent member of category 1, or 3 as inherently hard — especially 1, since it's a larger change. My guess would thus be that the personality of Claude 3.5 is closer to your category 3 than 1 (modulo philosophical questions about whether there is any meaningful difference, e.g. for ethical purposes, between "actually having" an emotion versus just successfully simulating the output of the same token stream as a person who has an emotion).

On your off topic comment:

I'm inclined to agree: as technology improves, the amount of havoc that one, or small group of, bad actors can commit increases, so it becomes both more necessary to keep almost everyone happy enough almost all the time for them not to do that, and also to defend against the inevitable occasional exceptions. (In the unfinished SF novel whose research was how I first went down this AI alignment rabbithole, something along the lines you describe that was standard policy, except that the AIs doing it were superintelligent, and had the ability to turn their long-term-learning-from-experience off, and then back on again if they found something sufficiently alarming). But in my post I didn't want to get sidetracked by discussing something that in herently contentious, so I basically skipped the issue, with the small aside you picked up on

seems like big step change in its ability to reliably do hard tasks like this without any advanced scaffolding or prompting to make it work. 

Keep in mind that o1 is utilising advanced scaffolding to facilitate Chain-Of-Thought reasoning, but it is just hidden from the user. 

This inspired me to give it the sestina prompt from the Sandman ("a sestina about silence, using the key words dark, ragged, never, screaming, fire, kiss"). It came back with correct sestina form, except for an error in the envoi. The output even seemed like better poetry than I've gotten from LLMs in the past, although that's not saying much and it probably benefited a lot from the fact that the meter in the sestina is basically free.

I had a similar-but-different problem in getting it to fix the envoi, and its last response sounded almost frustrated. It gave an answer that relaxed one of the less agreed-upon constraints, and more or less claimed that that it wasn't possible to do better... so sort of like the throwing-up-the-hands that you got. Yet the repair it needed to do was pretty minor compared to what it had already achieved.

It actually felt to me like its problem in doing the repairs was that it was distracting itself. As the dialog went on, the context was getting cluttered up with all of its sycophantic apologies for mistakes and repetitive explanations and "summaries" of the rules and how its attempts did or did not meet them... and I got this kind of intuitive impression that that was interfering with actually solving the problem.

I was sure getting lost in all of its boilerplate, anyway.

https://chatgpt.com/share/66ef6afe-4130-8011-b7dd-89c3bc7c2c03

W.r.t. maintaining a to-do list, I've noticed that, for me, at this time in my life (the last year or so), seldomly referencing a to-do has aided me in more quickly getting my work finished (both at my job and with my projects). As a result of seldomly referencing (looking at / thinking about) the to-do list, my behavior surrounding task completion has changed: I now pause, think of a task, and then decide to do the task, knowing that while I am doing this task I am getting closer to my end, whereas previously I could not be doing any task with often thinking about the other tasks remaining on the to-do list. Of late, there is just me and the task at hand, with very rare planning sessions, usually on the order of every 2-4 weeks where I reference a to-do list.

Historically, I've maintained long to-do lists, sorted by urgency and importance. Oftentimes I would go back and forth between tasks, i.e. task-switching, within the same day. To counter this, I would maintain another to-do list (keeping the longer one away), where this one I'd keep before me during my tasks, with a single written task, which I would cross out once completed. Despite these methods (the "backlog" and the one-task-at-a-time list), I found that, at times, the existence of these to-do lists resulted in my mind frequently reciting their items, and this recitation was enough to pull me away from the task at hand.

I've not attempted to quantitatively measure my productivity, so I do not have metrics to coincide with the aforementioned change in productivity.

In the context of minimum wage.

I assume Abdullah has many options mean he has many job offers/alternatives to jobs.

What does it mean for Benjamin to have many options?

I continue to think that capabilities from in-context RL are and will be a rounding error compared to capabilities from training (and of course, compute expenditure in training has also increased quite a lot in the last two years).

I do think that test-time compute might matter a lot (e.g. o1), but I don't expect that things which look like in-context RL are an especially efficient way to make use of test-time compute.

It seems like if the SCP hypothesis is true, block characters should cause it to act strangely.

• If a trebuchet requires you to solve the double pendulum problem (a classic example of a chaotic system) in order to aim, it is not a competition-winning trebuchet.

Ah, this is not quite the takeaway- and getting the subtlety here right is important for larger conclusions. If a simulating a trebuchet requires solving the double pendulum problem over many error-doublings, it is not a competition-winning trebuchet. This is an important distinction. 

If you start with a simulator and a random assortment of pieces, and then start naively optimizing for pumpkin distance, you will quickly see the sort of design shown at 5:02 in the video, where the resulting machine is unphysical because its performance depends on coincidences that will go away in the face of tiny changes in initial conditions. This behaviour shows up with a variety of simulators and optimizers.

An expensive but probably effective solution is to perturb a design several times, simulate it several times, and stop simulation once the simulations diverge. 

An ineffective solution is to limit the time of the solution, as many efficient and real-world designs take a long time to fire, because they begin with the machine slowly falling away from an unstable equilibrium. 

The chaos-theory motivated cheap solution is to limit the number of rotations of bodies in the solution before terminating it, as experience shows error doublings tend to come from rotations in trebuchet-like chaotic systems. 

The solution I currently have implemented at jstreb.hgreer.com is to only allow the direction of the projectile to rotate once before firing (specifically, it is released if it is moving upwards and to the right at a velocity above a threshold) which is not elegant, but seems mostly effective. I want to move to the "perturb and simulate several times" approach in the future.

Yep! If I think about those 10 people, 5 are having, or I expect to have large impact on the future. As for ages, all the people I thought of except one were over 20. There was one 14yo who is just naturally super high G.