LessWrong 2.0 Reader

I agree fake pictures are harder to threaten with. But consider that the deepfake method makes everyone a potential target, rather than only targeting the population who would fall for the relationship side of the scam.

There are other reasons I think it would be grimly effective, but I am not about to spell it out for team evil.

Thanks for the feedback! Quite helpful to get more context.
Quick responses:
1) Yes, we did intend for the hook to be eye-grabbing/mildly salesy, as this is part of our promotional material shared across different platforms, and we were hoping this would be effective at garnering the interest of talented individuals and encourage them to work on AIS. Though we didn't think it was dishonest/false, instead we designed it be short but effective. 
2)It was a sincere mistake that the post was made from an org account.
3) We missed gauging issues with the use of 'leading AI Safety organisations' , however, I think you are right. We could have been more cautious in how we framed it.
4) We have taken a note of stating the scope of our efforts and intend to factor this into consideration when designing our next outreach framing.

Thanks for your inputs!
 

Why are you so sure it's a computer simulation? How do you know it's not a drug trip? A fever dream? An unfathomable organism staring into some kind of (to it's particular phenomenology) plugging it's senses into a pseudo-random pattern generator from which is hallucinates or infers the experience of OP?

How could we falsify the simulation hypothesis?

I'm afraid I don't understand a lot of your assumptions. For example, why you think you being an example of any given superlative is somehow a falsifying observation of the reality - especially if other people/objects don't exist in uniform distributions. So it's not like a video game where every other NPC  exactly 10 HP, but through use of cheat-code you've got 1000. And even so, that data from within the 'simulation' as you call it is not proof of something 'without'. I think the only evidence of that would be if you find yourself in a situation like Daffy Duck, the walls of reality closing in on you - meeting your maker directly.

I also wonder, how much Kant or Plato have you read and did you do any research, even on SEP before you asked? I feel like anyone who has questions about the 'simulation' would be best served reading the philosophers who have written eloquently on the matter of how we come to represent the world and really formed the concepts and language we use.

Or you could read (Tractatus) Wittgenstein and dismiss all metaphysics all together as nonsense - literally: that which cannot be sensed and therefore musn't be spoken about.

My thought process goes like: on most weekdays I sure wish I could skip breakfast and/or lunch and only have one sit-down meal with my family in the evening. Time savings and convenience are the main concerns I suppose.

The first solution that came to mind was to try Soylent/Mealsquares/Huel for a month and cross my fingers, 50/50 it just goes well and solves the problem. I posted to see if there were any obvious considerations I was missing, or clear standout options to try first.

Pre-made frozen meals and protein bars are also plausibly acceptable meal replacement options. 

On a first pass frozen meals register as bulky and hard to store a month of at a time, and not something I'd bring to work. I've also never had an item I can imagine stomaching every day. 

Protein bars seem mostly fine, but my vibe check is that meal replacements are basically enlightened protein bars? Like, maybe the nutrition profile is better and they are packaged in sizes more suitable for full meals?

See my response to Daniel (https://www.lesswrong.com/posts/auGYErf5QqiTihTsJ/what-indicators-should-we-watch-to-disambiguate-agi?commentId=WRJMsp2bZCBp5egvr). In brief: I won't defend my vague characterization of "breakthroughs" nor my handwavy estimates of how how many are needed to reach AGI, how often they occur, and how the rate of breakthroughs might evolve. I would love to see someone attempt a more rigorous analysis along these lines (I don't feel particularly qualified to do so). I wouldn't expect that to result in a precise figure for the arrival of AGI, but I would hope for it to add to the conversation.

I posted this as a reply [and then posted it in the wrong subthread...reposting final version here] but I thought about it more, and on the Mets/Brewers odds thing, the "win %" from ESPN is something they've been doing for years.  It's all over espn.com.  It's not very smart, and doesn't have to be, because nothing is riding on it.

ESPNbet.com is licensed to use the ESPN name but is otherwise separate.  Since their posted odds are their business, I expect them to be a lot more...meticulous, at least.  The -160/+125 odds posted imply (after removing the house edge, aka de-vigging) a 58/42 probability split.  And that represents only a 4.4% hold from those probabilities, so it's not outrageous.  The standard -110/-110 on even money is a 4.8% hold.

This is my "slow scenario". Not sure whether it's clear that I meant the things I said here to lean pessimistic – I struggled with whether to clutter each scenario with a lot of "might" and "if things go quickly / slowly" and so forth.

In any case, you are absolutely correct that I am handwaving here, independent of whether I am attempting to wave in the general direction of my median prediction or something else. The same is true in other places, for instance when I argue that even in what I am dubbing a "fast scenario" AGI (as defined here) is at least four years away. Perhaps I should have added additional qualifiers in the handful of places where I mention specific calendar timelines.

What I am primarily hoping to contribute is a focus on specific(ish) qualitative changes that (I argue) will need to emerge in AI capabilities along the path to AGI. A lot of the discourse seems to treat capabilities as a scalar, one-dimensional variable, with the implication that we can project timelines by measuring the rate of increase in that variable. At this point I don't think that's the best framing, or at least not the only useful framing.

One hope I have is that others can step in and help construct better-grounded estimates on things I'm gesturing at, such as how many "breakthroughs" (a term I have notably not attempted to define) would be needed to reach AGI and how many we might expect per year. But I'd be satisfied if my only contribution would be that people start talking a bit less about benchmark scores and a bit more about the indicators I list toward the end of the post – or, even better, some improved set of indicators.

I think you're overindexing on the phrase "status quo", underindexing on "industry standard", and missing a lot of practical microstructure.

Lots of firms or teams across industry have attempted to "EG" implement multi-factor authentication or basic access control mechanisms or secure software development standards or red-team tests. Sony probably had some of that in some of its practices in some of its departments when North Korea 0wned them.

Google does not just "OR them together" and half-ass some of these things. It "ANDs together" reasonably high quality versions of everything. Then every year they anneal the culture a little bit more around small controlled probes of global adequacy.

..

Also, in reading that RAND document, I would like to report another "thonk!" sound!

..

Rand's author(s) seem to have entirely (like at a conceptual level) left out the possibility that AGI (during a training run or during QA with humans or whatever) would itself "become the attacker" and need to be defended against.

It is like they haven't even seen Ex Machina, or read A Fire Upon The Deep or Daemon.

You don't just have to keep bad guys OUT, you have to keep "the possible bad guy that was just created by a poorly understood daemon summoning process" IN, and that perspective doesn't appear anywhere in any of the RAND document that I can see.

No results when I ^f for [demon], [summon], [hypno], [subvert], [pervert], [escape].

(("Subvert" was used once, but it was in a basic bitch paragraph like this (bold in original):

Most access control systems are either software systems or have significant software components. In addition to the more specialized ways of undermining such systems described above, an attacker could undermine them by finding code vulnerabilities and then subverting their behavior (without actually dealing with their cryptographic or core functionality at all). A major category of code vulnerabilities that undermine access control systems on a regular basis are privilege escalation vulnerabilities.

Duh.

))

The best thing in that RAND document is [infected] which occurs twice <3

Page 54 (bold not in original):

PyTorch’s prerelease Linux packages, PyTorch-nightly, were compromised via a malicious binary on PyPi.122 The torchtriton dependency in the PyTorch-nightly build was replaced with a malicious package, leading to the extraction of sensitive data from infected systems.

Page 61 (bold not in original, footnote not a link in the original):

Agent.btz, the malware that infected the U.S. Department of Defense in 2008 and led to the creation of U.S. Cyber Command, came from a USB device that was found in a parking lot on a base in the Middle East.164

"Infection" is a really good metaphor and word for bringing extreme rigor to the tracing of actual causal processes, because it gets at the idea of "programs" (viral or microbial or fungal or whatever genomes (or genome products) spreading via arbitrary physical transmission methods). This is a prett darn correct frame! :-)

Ever since covid I've been beating the drum for BSL5 to handle scary diseases by assuming that the real danger is NOT that "a known disease brought into a lab to be studied there merely infects a scientist (or everyone inside the facility for that matter)", but that someone inside the BSL5 gets infected with something new that would never naturally evolve and then uses creativity to escape the BSL5 while infected and then that causes a pandemic.

Currently the BSL system only goes up to 4, and all incremental increases in caution and safety are related to the protection of the lab worker from what they are working with more and more and more, while assuming good faith by the site administrator, and the security guards at the exit gate (which may or may not be distinct from the entry gate) who might also get infected, and so on.

I want a Bio-SL5 standard to exist in biology, and I want all GoF research restricted by an international treaty that ensures that the GoF only happens inside BSL5 facilities.

Similarly, I also think humanity also needs an analogous "CompSci-SL5 standard" that imagines the entire computing facility (data center plus programmers plus janitors who have physical access and so on plus all the managers of everyone working inside the CSSL5 campus) is subverted by a nascent superintelligence and then used by that nascent superintelligence to try to subvert all of Earth.

There is no hint of anything at all like this as part of the threat modeling in the RAND report.

Also, if Google had such a thing back when I worked there, I didn't hear about it. (Then again, maybe the existence of it would have been kept secret?)

In a large universe [? · GW], you, and everyone else, exists both in and not in simulations. That is: The pattern you identify with exists in both basement reality (in many places) and also in simulations (in many places).

There is a question of what proportion of the you-patterns exist in basement reality, but it has a slightly different flavour, I think. It seems to trigger some deep evolved patterns (around fakeness?) less than the kind of existential fear that simulations with the naive conception of identity sometimes brings up.

But to answer that question: Maybe simulators tend to prefer "flat" simulations, where the entire system is simulated evenly to avoid divergence from the physical system it's trying to gather information about. Maybe your unique characteristic is the kind of thing that makes you more likely to be simulated in higher fidelity than the average human, and simulators prefer uneven simulations. Or maybe it's unusual but not particularly relevant for tactical simulations of what emerges from the intelligence explosion (which is probably where the majority of the simulation compute goes). 

But, either way, that update is probably pretty small compared to the background high rate of simulations of "humans around at the time of the singularity". Bostrom's paper covers the general argument for simulations generally outnumbering basement reality due to ancestor simulations: https://simulation-argument.com/simulation.pdf

However, even granting all of the background assumptions that go into this: Not all observers who are you live in a simulation. You exist in both types of places. Simulations don't reduce your weight in the basement reality, they can only give you more places which you exist.