LessWrong 2.0 Reader

Wow, this part really resonated with me:

This is why "build a 10% better mousetrap" is a legitimate goal, but "build a 10% better web portal for artists" is not. The 10% improvement means nothing if the community accuses you of being a greedy selfish bastard who only cares about money and not about art, and they blacklist you and everyone who cooperates with you. And yes, if you understand how the game is played, the initiators of the backlash are those who profit from the existing system. But you can't say this out loud; it would only prove that you care about the money. So both sides will keep arguing complete bullshit, trying to get the confused people on their side. The important thing is to get confused high-status people on your side, because then the rest will follow. 

Certainly have noticed a similar dynamic where people take pride in & applaud blind faith in the project & founders, and those who profess to be in it for love of community & not money are rewarded socially. Very cult like behavior, which of course is perfect for the leaders & ideal for that startup's "customer base", so I definitely applaud the founders for doing their job exceptionally. 

Lately I've been fascinated by these group dynamics, and how power & influence lie among any given "scene". Joined 3 vastly different scenes lately, which have vastly different norms, nearly 0 overlap, and norms/status symbols that aren't even on the same axes- it's been fascinating on my end to constantly context switch & pick up on what the differences vs commonalities are in these totally different communities.

I bet there's many axioms or essays out there about navigating the rules of social scenes, but so far only one leaps to mind for me.

I maybe mistaken as to the availability of the lab grown meat to the general public. apparently they are no longer on sale

https://www.wired.com/story/upside-foods-good-meat-cultivated-lab-grown-sale-stopped-singapore-california-crenn/

seem like my information are out of date

I've had caps lock remapped to escape for a few years now, and I also remapped a bunch of symbol keys like parentheses to be easier to type when coding. On other people's computers it is slower for me type text with symbols or use vim, but I don't mind since all of my deeply focused work (when the mini-distraction of reaching for a difficult key is most costly) happens on my own computers.

I agree, it takes extra effort to make the AI behave like a team of experts.

Thank you :)

Good luck on sharing your ideas. If things aren't working out, try changing strategies. Maybe instead of giving people a 100 page paper, tell them the idea you think is "the best," and focus on that one idea. Add a little note at the end "by the way, if you want to see many other ideas from me, I have a 100 page paper here."

Maybe even think of different ideas.

I cannot tell you which way is better, just keep trying different things. I don't know what is right because I'm also having trouble sharing my ideas.

I would say that, barring strong evidence to the contrary, this should be assumed to be memorization.

I think that's useful! LLM's obviously encode a ton of useful algorithms and can chain them together reasonably well

But I've tried to get those bastards to do something slightly weird and they just totally self destruct.

But let's just drill down to demonstrable reality: if past SWE benchmarks were correct, these things should be able to do incredible amounts of work more or less autonomously and get all the LLM SWE replacements we've seen have stuck to highly simple, well documented takes that don't vary all that much. The benchmarks here have been meaningless from the start and without evidence we should assume increments on them is equally meaningless

The lying liar company run by liars that lie all the time probably lied here and we keep falling for it like Wiley Coyote

9 years since the last comment - I'm interested in how this argument interacts with GPT-4 class LLMs, and "scale is all you need".

Sure, LLMs are not evolved in the same way as biological systems, so the path towards smarter LLMs aren't fragile in the way brains are described in this article, where maybe the first augmentation works, but the second leads to psychosis.

But LLMs are trained on writing done by biological systems with intelligence that was evolved with constraints.

So what does this say about the ability to scale up training on this human data in an attempt to reach superhuman intelligence?

After a very very cursory google search I wasn't able to find any (except in some places in Singapore), I'd be interested if this is available at all in the US

You mentioned Mixture of Experts. That's interesting. I'm not an expert in this area. I speculate that in an architecture similar to MoE, when one expert is working, the others are idle. In this way, we don't need to run all the experts simultaneously, which indeed saves computation, but it doesn't save memory. However, if an expert is shared among different tasks, when it's not needed for one task, it can handle other tasks, so it can stay busy all the time.

The key point here is the independence of the experts, including what you mentioned, that each expert has an independent self-cognition. A possible bad scenario is that although there are many experts, they all passively follow the commands of a Leader AI. In this case, the AI team is essentially no different from a single superintelligence. Extra efforts are indeed needed to achieve this independence. Thank you for pointing this out!

Happy holidays, too!

I wonder how you react to naysayers who say things [LW(p) · GW(p)] like:

How about if you solve a ban on gain-of-function research first, and then move on to much harder problems like AGI? A victory on this relatively easy case would result in a lot of valuable gained experience, or, alternatively, allow foolish optimists to have their dangerous optimism broken over shorter time horizons.

There are 2 things to keep in mind:

1. It's only now that LLMs are reasonably competent in at least some hard problems, and at any rate, I expect RL to basically solve the domain, because of verifiability properties combined with quite a bit of training data.

2. We should wait a few years, as we have another scale-up that's coming up, and it will probably be quite a jump from current AI due to more compute:

https://www.lesswrong.com/posts/NXTkEiaLA4JdS5vSZ/?commentId=7KSdmzK3hgcxkzmPX [LW · GW]