LessWrong 2.0 Reader

If it's about the belief and not about whether it can be quantified at all, people will always have somewhere to retreat their belief to. At the most extreme, even with extremely good copies people will bottom out to the No Cloning theorem to argue that the original is different.

I don't have very much rigorous data about what proportion of NFTs are scams, only anecdata. However, my prior due to my extensive experience with the broader crypto space is that for people without the right memetic antibodies, it's extremely easy to get screwed over in one way or another by shady practices and the very aggressive crypto memesphere (Ponzis, P&Ds, exit scams, vaporware, extremely aggressive marketing, FUD/FOMO, cults, etc), and what I've heard about NFTs seems largely in line. I think the term I'm looking for is a bit broader than "scam", though I'm not sure what the best term is, and I'd be open to changing my post to use that term instead. Overall, I'm not very interested in litigating out exactly what % of crypto/NFTs are this way, because it's not what I'm most interested in.

Dan Dennet has an excellent section on a very similar subject in his book 'freedom evolves'. To use a computer science analogy true telepathy would be the ability for 2+ machines with different instruction set architectures being able to cross compile to code that is binary compatible with the other ISA and transmit the blob to the other machine. We have to serialise to a poorly defined standard and then read from the resulting file with with a library that is only a best guess at and implementation of the de facto spec.

Yes, sorry, you're right; edited.

I don't know I'd say that guy torched a lot of future employment opportunities when when he sabotaged his repos. Also obligatory: https://xkcd.com/2347/

Apologies but I'm unclear if you are characterising my post or my comment as "a bunch of buzzwords thrown together" could you clarify? The post's main thrust was to make the simpler case that the more of our cognition takes place on a medium which we don't have control over and which is subject to external interest the more concerned we have to be about trusting our cognition. The clearest and most extreme case of this is if your whole consciousness is running one someone else's hardware and software stack. However, I'll grant that I've not yet make the case in full that this is bottleneck in raising the sanity waterline, perhaps this warrants a follow-up post. In outline: the asymmetric power relationship, lack of accountability, transparency oversight or effective governance of the big proprietary tech platforms is undermining trust in our collective and indeed individual ability to discern the quality of information and this erosion of the epistemic commons is undermining our ability to reason effectively and converge on better models. In Aumann agreement terms, common priors are distorted by amplified availability heuristics in online bubbles, common knowledge is compromised by pseudo science, scientific cargo cults, framed in a way that is hard to distinguish from 'the real deal'. Also the 'honest seekers of truth' assumption is undermined by, bots, trolls, and agent provocateur mascaraing as real people acting on behalf of entities with specific agendas. You only fix this with better governance and I content free software is a major part of that better governance model.

I heard about the case when an artist has sold the copyright of all his works to another person (including the works owned by other people, like previous buyers.) Basically it means that the right owner could forbid all other owners to ever exhibit an even look at their own works. But in reality in the case I am speaking about it was about right to make t-shirts with prints. 

I also saw purchase agreements which deliberately state that the artwork is coming with all rights.

"will the first animals that take over the world be able to solve the Riemann hypothesis", and the answer is no because humans intelligence, while general, is still pointed more at civilisation-building-style tasks than mathematics.

Pardon the semantics, but I think the question you want to use here is "will the first animals that take over the world have already solved the Riemann hypothesis". IMO humans do have the ability ("can") to solve the Riemann hypothesis, and the point you're making is just about the ordering in which we've done things.

I agree that a computing resources to run code on would be a more complex proposition to make available to all my point is more that if you purchase compute you should be free to use it to perform whatever computations you wish and arbitrary barriers should not be erected to prevent you from using it in whatever way you see fit (cough Apple, cough Sony, cough cough).

Instead of "dumb" or "narrow" I'd say "having a strong comparative advantage in X (versus humans)". E.g. imagine watching evolution and asking "will the first animals that take over the world be able to solve have already solved the Riemann hypothesis", and the answer is no because humans intelligence, while general, is still pointed more at civilisation-building-style tasks than mathematics.

Similarly, I don't expect that any AI which can do a bunch of groundbreaking science to be "narrow" by our current standards, but I do hope that they have a strong comparative disadvantage at taking-over-world-style tasks, compared with doing-science-style tasks.

And that's related to agency, because what we mean by agency is not far off "having a comparative advantage in taking-over-world style tasks".

Now, I expect that at some point, this line of reasoning stops being useful, because your systems are general enough and agentic enough that, even if their comparative advantage isn't taking over the world, they can pretty easily do that anyway. But the question is whether this line of reasoning is still useful for the first systems which can do pivotal task X. Eliezer thinks no, because he considers intelligence and agency to be very strongly linked. I'm less sure, because humans have been evolved really hard to be agentic, so I'd be surprised if you couldn't beat us at a bunch of intellectual tasks while being much less agentic than us.

Side note: I meant "pattern-matching" as a gesture towards "the bit of general intelligence that doesn't require agency" (although in hindsight I can see how this is confusing, I've just made an edit on the ACX comment).

If you thought the answers in that thread backed you up:

It's a mixed bag. A lot of near term work is scientific, in that theories are proposed and experiments run to test them, but from what I can tell that work is also incredibly myopic and specific to the details of present day algorithms and whether any of it will generalize to systems further down the road is exceedingly unclear. 

...

A lot of the other work is pre-paradigmatic, as others have mentioned, but that doesn't make it pseudoscience. Falsifiability is the key to demarcation.

That summarizes a few answers.

I agree, I wouldn't consider AI alignment to be scientific either. How is it a "problem" though?