LessWrong 2.0 Reader

Isn’t the answer that the low hanging fruit of explaining unexplained observations has been picked?

I will bite. 

First of all, I appreciate the effort of trying to communicate better and hammering down the neat borders of word and how they used across domains - especially for words that are often used interchangeably and carelessly.

TLDR: Sometimes posts just get unlucky! And your style is on the verbose side and I am still somewhat confused about your value props. 
 
It seems like your frustration is from a lack of responses - often, a lack of response might just be luck based and how the LW algorithm works (exponential time decay). Maybe you posted at a time that most forum users are asleep, and/or you are running against the headwind of literally one of the most popular article of all time. [LW · GW] Sometimes, you just get unlucky! Twice even!

However, I can also say that the content is not written very legibly. It took me a long time to get the actual punch line and understand what you really want - "Can someone technical say something about how you want these terms to be used?".

In addition, the post is written very verbosely. It takes a long time to get to the point, it is not clear about what it wants until the very end. It doesn't say how you are going to this, or that if it is even worth engaging with you because it is not clear how you'd help to delineate the word boundary from your position. I am still unsure about the exact value proposition on how better delineating these words would lead to reduced p(doom). 

Keep trying!

Alright, this is the second time now. What am I doing wrong, LessWrongers? :/.

Thanks for sharing this and for the examples layed out. I was not familiar with all of them, though many. but I did omit stating that I meant outside of fiction. My assumption is still relatively short timeframes of 5 to 15 years. Under those assumptions I dont necessarily see scenario 1 or 7 being more likely than scenario 8. 

Quick note. I see a show like Upload being a potential representation of a facet of these scenarios. For example scenarios 2 to 7 could all have widespread virtual realities for the common person or those who opt out willingly or otherwise from base biological reality.

A part of my underlying assumption is that there are organizations, be it government, private or otherwise that are likely far mor advanced in their brain computer interface tech (BCI), than they would disclose publicly. There is also net negative value at some point in advancing AGI and BCI publicly versus privately. The first to get there, wins far more power using it in secret. The fiat money system is so far beyond repair and traceability that this is perfectly plausible to execute.

As to plausability and assumptions, my proposed approach is.. work within 5 to 15 year time frame, where we have advanced AGI but not ASI in the first 5. Then it is feasible for example to argue that it has integrated itself across critical system, compromised legacy equipment and code, led to rapid advancement in lab wet work and understanding of conciousness, resulted in development of new materials, had us build it a factory for manufacturing, is held by a select group who exploit it, etc.

I almost want to draft up a spreadsheet if anybody would be interested to collab.. track possible scenarios, possible variables, and probabilities based on present realities and possible near term wowza factors.

I don't really understand how a local copy of the weights gives the terrorists more practical control over the software's alignment. I don't think it's easy to manually tweak weights for so specific a purpose. Maybe they just mean the API is doing a good job of blocking sketchy requests?

You can finetune models for any specific purpose: just provide a few datapoints and train. The more specific the purpose, the easier tweaking the weights is, not harder. (Surely, if nothing else,  you've seen all of the LoRAs and other things for finetuning image generation model to generate a specific character?) There is an extensive literature at this point on how it is trivial to strip away all of the friendly chatbot persona from released checkpoints, such as LLaMA, if you are able to access and modify the model slow weights and fast weights directly.

I unfortunately don't think this proves anything relevant. The example just shows that there was one question where the market was very uncertain. This neither tells us how certain the market is in general (that depends on its confidence on other policy questions), nor how good this particular estimate was (that, I would argue, depends on how far along the information chart [LW · GW] it was, which is not measurable -- but even putting my pet framework aside, it seems intuitively clear "it was 56% and then it happened" doesn't tell you how much information the market utilized).

The point is that even if voters did everything right and checked prediction markets as part of their decision making algorithm, it wouldn't help.

This depends on the first point, which again requires looking at a range of policy markets, not just one. And actually, I personally didn't expect Trump to do any tarrifs at all (was 100% wrong there), so for me, the market would have updated me significantly into the right direction.

Claude says its a gray area when I ask, since this isn’t asking for the journalist to make a general change to the story or present Ben or the subject in a particular light.

I expect they will thus not want to use my quotes

Yep, my impression is that it violates the journalist code to negotiate with sources for better access if you write specific things about them.

Update: I haven't figured out the answer yet, but I did get a nice frame on this question. Both are basically different levels of the attention elevation game. From when any information leaks out to when it grabs collective attention of entire civilisation.

At lowest levels you just need to ensure it's not obvious spam, in order to circulate further. AI filters or a small payment or proof of work is enough. At highest levels you need thousands of people making (non-gameable) upvotes or video proofs or large donations as a proof in order to circulate it further. Building resilient tech for both of these faces different challenges.

Lesswrong is clearly no longer the right forum for me to get engagement on topics of my interest. Seems mostly focussed on AI risk.

On which forums do people who grew up on the cypherpunks mailing list hang out today? Apart from cryptocurrency space.