LessWrong 2.0 Reader

E

Did you mean 'D' here? (2nd equation of the structured SSM)

You seem to make a strong assumption that consciousness emerges from matter. This is uncertain. The mind body problem is not solved.

MIRI Technical Governance Team is hiring, please apply and work with me!

We are looking to hire for the following roles:

• Technical Governance Researcher (2-4 hires)
• Writer (1 hire)

The roles are located in Berkeley, and we are ideally looking to hire people who can start ASAP. The team is currently Lisa Thiergart (team lead) and myself.

We will research and design technical aspects of regulation and policy that could lead to safer AI, focusing on methods that won’t break as we move towards smarter-than-human AI. We want to design policy that allows us to safely and objectively assess the risks from powerful AI, build consensus around the risks we face, and put in place measures to prevent catastrophic outcomes.

The team will likely work on:

• Limitations of current proposals such as RSPs
• Inputs into regulations, requests for comment by policy bodies (ex. NIST/US AISI, EU, UN)
• Researching and designing alternative Safety Standards, or amendments to existing proposals
• Communicating with and consulting for policymakers and governance organizations

If you have any questions, feel free to contact me on LW or at peter@intelligence.org 

Yeah. My point was, we can't even be sure which behavior-preserving optimizations (of the kind done by optimizing compilers, say) will preserve consciousness. It's worrying because these can happen innocuously, e.g. when your upload gets migrated to a newer CPU with fancier heuristics. And yeah, when self-modification comes into the picture, it gets even worse.

Oh, I hope so! But I would like to get the perspective of people outside our bubble.

If EA has a bad image, we are not the right people to speculate why. And if we don't know why, then we cannot fix it. Even if Paul Christiano can convince people that he is okay, it would be better if he didn't have to do this the next time. Maybe next time he (or some other person associated with EA) won't even get a chance to talk in person to people who oppose EA for some reason.

and also they get backed into a corner once they write the first name after which their prediction is that they will get close rather than admitting they don’t have a full solution

This is a contingent tuning issue though, not a fundamental limitation. Chatbots are not predictors, they make use of meaningful features that formed when the base model was learning to solve its prediction task. It should be possible to tune the same base model to notice that it apparently committed to something it can't carry out and so needs to pivot. Eliciting in-context awareness of errors might be easier than not hallucinating in the first place, let alone setting up more expensive and complicated scaffolding.

I think there might be many local improvements, but I'm pretty uncertain about important factors like elasticity of "demand" (for robbery) with respect to how much of a medication is available on demand.  i.e. how many fewer robberies do you get if you can get at most a single prescriptions' worth of some kind of controlled substance (and not necessarily any specific one), compared to "none" (the current situation) or "whatever the pharmacy has in stock" (not actually sure if this was the previous situation - maybe they had time delay safes for storing medication that wasn't filling a prescription, and just didn't store the filled prescriptions in the safes as well)?

Question: Who is this event celebrating?

I've been trying to think a bit about the boundary of this event. Naturally I love LessWrong, but it's not simply about LessWrong.

This is not comprehensive, but here are three subcultures that I admire and am hoping to celebrate at this event.

1. Aspiring Rationalists – people interested in understanding and improving their cognitive algorithms
2. Rational Fiction Readers & Writers – people who write fiction that illustrate intelligent characters and lawful worlds
3. Public Worldview Builders and/or Polymaths – people earnestly trying to directly understand the world and explain what they’ve learned with online write-ups

Below are a few notes about each of them and what I hope from bringing them together.

Aspiring Rationalists

I continue to believe the mission to understand our cognitive algorithms and improve them is a worthwhile one, and there's still lots of people working on advancing this, whether it's finding new theoretical underpinnings for decision-making [? · GW] or coming up with pragmatic heuristics for improving the truth-tracking nature of discussions [LW · GW] or building up models for how the world is trying to fight your ability to understand it [? · GW] or something quite different.

I'm hoping to bring folks together who are interested in this mission and give them the ability to get ideas from other people who have been working on this too.

Rational Fiction Readers & Writers

Near to the aspiring rationalists is Rational Fiction. For detailed pointers to it check the description in the r/rational subreddit sidebar or the TV Tropes Page, but broadly it really respects the characters' intelligence and the rules of how the fictional world works. It's the sort of story where a key plot point might involve doing a fermi estimate for how much steel is required to withstand a given magical blast, or where a key moment of dramatic tension involves the protagonist carefully reasoning about their situation and reaching a novel conclusion, one that in-principle the reader themselves could've deduced given the information that they had available.

This community is very active and large! r/rational has 25k subscribers, and the best works are extremely popular. At the time of publication Harry Potter and the Methods of Rationality had the most reviews (37k) of any work on FanFiction.net (now second most) and is the most followed HP-fic (22k), Mother of Learning has the most views (18M) on Royal Road, and Worth the Candle is the 3rd most read original fiction on Archive of Our Own (I started it a month ago and am now 220 chapters in, no spoilers please). There's so many more amazing works, I'd guess like 300-1,000 rational-fics (especially counting all of the Glow-Fics).

I think this often (not always) has a similar nature to the old hard sci-fi, where people try to set up rules of a world and really ask what would follow from these rules, and authors put in a lot of work to be very creative-and-lawful.

I’m hoping to bring people together to talk about stories and the ideas that go into them and come away with lots of new ideas for stories they could write.

Public Worldview Builders and/or Polymaths

(This bit is something Ray helped a lot in writing)

There’s a style of discourse that has felt pretty important to me. It used to be kinda The Central Thing going on on the internet. It’s become less so, with the rise of social media. That Thing is a bit hard to pin down, but it’s something like:  

“Longform, thoughtful writing, earnestly trying to figure stuff out.”  

LessOnline is a festival celebrating that kind of writing. Furthermore, I have a strong hunch that there’s a craft that everyone invited is involved with in some way — an art of paying attention to the information you get about the world, piecing things together into a coherent perspective, extracting strong arguments to show to others, writing in a way that’s engaging and clear, and leaving people with a little bit of a better understanding of the world, that altogether hopefully adds up to leave me and humanity with a much more accurate and more leveraged map of the world.

The idea here is to invite the part of the blogosphere that seems sort of like "kindred spirits" to LessWrong, approaching similar kinds of questions from different perspectives. I think one framing is "we're looking for people who are publicly building out cohesive worldview in public."

I’m hoping to bring such people together to share ideas they’ve been thinking about and knowledge they’ve learned, and also to share insights about the shared craft. 

An quick and incomplete list of other subcultures I'm excited about bringing together

Econ Blogosphere, Statistics Blogosphere, History Blogosphere, Finance Blogosphere, The Nerdy Cartoon-O-Sphere (e.g. XKCD, SMBC, Abstruse Goose, many more), Agent Foundations researchers, Genetic Enhancement researchers, more.

The point of this post is to say that we can use a formal protocol to create an interface leveraging the elements that make cooperation optimal. Those elements can be found, for exemple, in studies about crowd wisdom and bridging systems (pol.is, computational democracy etc.)

So "we" is large, and more or less direct, I say "we" because I am not alone to think this is a good idea, although the specific setting that I propose is more intimately bound to my thoughts. Some people are already engaged in things at least greatly overlapping with what I exposed, or interested to see where my plan is going

And the really funny bit is NIST deliberately subverted the standard so that an organization who knew the master key (probably NSA) could break the security of the system. And then, in actualt implementation, the master key was changed so that someone else could break into everyone's system  And, officially at least, we have no idea who that someone is. Probably Chinese government. Could be organized crime, though probably unlikely.

The movie Sneakers had this as its plots years ago.. US government puts a secret backdoor in everyone's computer system .. and, then, uh,, someone steals the key to that backdoor;

But anyway, yes, it is absolutely NISTs fault that they unintentionally gave the Chinese government backdoor access into US government computers.