LessWrong 2.0 Reader

They know that they have to play the long game, and they want everyone to live to the simulation-breakout.

(apologies in advance for the wall of text, don't feel you need to respond, I wrote it out and then almost didn't post).

To clarify, I wouldn't expect stagnant or decreasing salaries to be the norm. I just wanted to say that there are circumstances where I expect this to be the case. Specifically, if I am an employee who is living paycheck to paycheck (which many do), then I can't afford any time unemployed.

As a result, if my employer is able to squeeze me in this situation, I might agree to a lower wage out of necessity.

The problem with your proposed system is that it essentially encourages employees to selectively squeeze themselves- if they're in a situation where they can't afford to lose their job, then this will lower what they ask at a negotiation, and what they receive, even if the employer is offering the same rmax to all employees. This has little to do with their relative skills as an employee and everything to do with their financial situation and responsibilities outside work.

Here's an example. I'm not sure why I wrote it, but here it is:

Brenda and Karl work at a gas station supermarket. They both work the same job, on the checkout area, with some shelf stocking as needed.

Brenda is a single mom with a 2 year old child who she is paying for childcare, and the rest of her earnings go on rent, food and fuel for her beat up car (it's a miracle it's still running). She works at the gas station 4 days a week, 9-7.

Karl takes on shifts 2 nights a week, it helps pay him through college and he enjoys the extra money. His parents give him enough that he could probably survive without the job entirely, and certainly a period of unemployment would not be a big problem for him.

Brenda and Karl both get paid $15/hr for their work, but they know that the new "payPlav system (TM)" is being introduced by management, and they have a pay negotiation coming up.

Management asks them to read the rules of the new system carefully submit their r-min. They say that if rmax < rmin, then the employee will stay on their existing salary.

Brenda sets her rmin at $15.50. She could do with a significant pay bump, but she doesn't want to lose out on the pay increase entirely, since she's only holding it together at $15.

Karl sets a bolder rmin of $16.50. He works hard at the job and thinks he deserves more, but it's not a big deal if he misses out and stays at $15

Management sets rmax at $17

Brenda gets $16.25

Karl gets $16.75

I don't think this is fair. It's a clear case where the system creates a situation where employees who care less and need the money less will be rewarded more.

Here's another scenario - same as the above, but management says that rmax<rmin will mean termination of the contract.

Brenda sets rmin at $13.50. She simply can't lose this job, it would ruin her.

Karl sets his rmin to $16

Management sets rmax at $15.50

It's more extreme, to be sure, and maybe a little unrealistic.

Many workers employed on zero-hours contracts end up in this situation - since the employer is able to lower wages with impunity and they don't have many other options, they get squeezed for profit. Sometimes unscrupulous employers do this selectively, based on which employees can least afford to stop working. This results in the most impoverished employees losing out.

Midwits are often very impressed with themselves for knowing a fancy economic rule like Ricardo's Law of Comparative Advantage!

Could we have less of this sort of thing, please? I know it's a crosspost from another site with less well-kept discussion norms, but I wouldn't want this to become a thing here as well, any more than it already has.

Yesterday was the official application deadline for leading a project at the next AISC. This means that we just got a whole host of project proposals. 

If you're interested in giving feedback and advise to our new research leads, let me know. If I trust your judgment, I'll onboard you as an AISC advisor.

Also, it's still possible to send us a late AISC project proposals. However we will prioritise people how applied in time when giving support and feedback. Further more, we'll prioritise less late applications over more late applications. 

Yudkowsky's point about trying to sell an Oreo for $77 is that a billionaire isn't automatically going to want to buy something off you if they don't care about it (and neither would an ASI).

I thought Yudkowsky's point was that the billionaire won't give you $77 for an Oreo because they could get an Oreo for less than $77 via other means. But people don't just have an Oreo to sell you. My point in that sentence was to bring up that workers routinely have things of value that they can sell for well over $77, even to billionaires. Similarly, I claim that Yudkowsky did not adequately show that humans won't have things of substantial value that they can sell to future AIs.

I'm not sure anyone is arguing that smart AIs would immediately turn violent unless it was in their strategic interest

The claim I am disputing is precisely that it will be in the strategic interest of unaligned AIs to turn violent and steal from agents that are less smart than them. In that sense, I am directly countering a claim that people in these discussions routinely make.

Depending on your devops pipeline (that is, how often do you reimage with DD, as opposed to updating in place, and what changes you make AFTER imaging), you might consider a few timesaving options:

1. small root partition, rest as a data partition.  This lets you image only the root with dd.  It's probably FASTER to image a 4GB partition of a 32GB SDCARD than to image a 4GB card directly.  
2. Go smaller than 4GB (by partition, you probably don't want to buy smaller cards).  RPiOsLite should install in under 1GB, and a very usable commandline system with space for updates in 2GB.   If you only have 300MB free in a 4GB storage, something is wrong.  It may not be worth fiddling with, though.
   For multiple copies of a master, it's common to have a large/fast setup that you install, update, tune, and prepare on, then prepare the image for distribution.  There are lots of options for that, the simplest is to clean and shrink the filesystem, and use that shrunken filesystem as the image to distribute.
3. use a USB "real" SSD, not an SDCARD.  Rpi4 and above can boot from it, earlier models need to boot from SDCARD, but can have root and all data on usb.  this can be MASSIVELY faster and somewhat more reliable than sdcard.
4. Probably not worth the effort, but PiLFS or Yocto make it possible to build a "custom application distribution" for your pi.  It will contain only the packages you need, and needs no update mechanism on the devices - to do an update, you rebuild and re-distribute.  It can be TINY and still quite functional.

Basically people tend to value stuff they perceive in the biophysical environment and stuff they learn about through the social environment.

So that reduces the complexity of the problem - it’s not a matter of designing a learning algorithm that both derives and comes to value human abstractions from observations of gas particles or whatever. That’s not what humans do either.

Okay then, why aren’t we star-maximizers or number-of-nation-states maximizers? Obviously it’s not just a matter of learning about the concept. The details of how we get values hooked up to an AGI’s motivations will depend on the particular AGI design but probably reward, prompting, scaffolding or the like.

If that’s your hope—then you should already be alarmed at trends

Would be nice for someone to quantify the trends. Otherwise it may as well be that trends point to easygoing enough and aligned enough future systems.

For some humans, the answer will be yes—they really would do zero things!

Nah, it's impossible for evolution to just randomly stumble upon such complicated and unnatural mind-design. Next you are going to say what, that some people are fine with being controlled?

Where an entity has never had the option to do a thing, we may not validly infer its lack of preference.

Aha, so if we do give the option to an entity and it doesn't always kills all humans, then we have evidence it cares, right?

If there is a technical refutation it should simplify back into a nontechnical refutation.

Wait, why prohibiting successors would stop OpenAI from declaring easygoing system a failure? Ah, right - because there is no technical analysis, just elements of one.

Potential for o1-like long horizon reasoning post-training compromises capability evals for open weights models. If it's not applied before/during evals, then evals will significantly underestimate capabilities of the system that can be built out of the open weights later [LW · GW], when the recipe is reproduced.

This likely will be relevant for Llama 4 (as the first 1e26+ FLOPs model with open weights), if they don't manage a good reproduction of o1-like post-training before release, and continue the policy of publishing in open weights if the evals are not too alarming.

There's a billion reasonable-seeming impact metrics, but the main challenge of counterfactual-based impact is always how you handle chaos. I'm pretty sure the solution is to go away from counterfactuals as they represent a pathologically computationalist form of agency [LW · GW], and instead learn the causal backbone [LW · GW].

If we view the core of life as increasing rather than decreasing entropy [LW · GW], then entropy-production may be a reasonable candidate for putting quantitative order to the causal backbone. But bounded agency is less about minimizing impact and more about propagating the free energy of the causal backbone into new entropy-producing channels.