What's the evidence that LLMs will scale up efficiently beyond GPT4? i.e. couldn't GPT5, etc., be very inefficient? 2023-11-24T15:22:01.624Z
When building an organization, there are lots of ways to prevent financial corruption of personnel. But what are the ways to prevent corruption via social status, political power, etc.? 2023-10-17T18:51:47.127Z
Is there a widely accepted metric for 'genuineness' in interpersonal communication? 2023-09-27T05:30:46.716Z
Credible, costly, pseudonymity 2023-04-24T13:35:12.718Z
Practical ways to actualize our beliefs into concrete bets over a longer time horizon? 2023-04-20T21:21:21.257Z
Analysis of GPT-4 competence in assessing complex legal language: Example of Bill C-11 of the Canadian Parliament. - Part 1 2023-04-02T00:01:11.133Z
Is ChatGPT (or other LLMs) more 'sentient'/'conscious/etc. then a baby without a brain? 2023-03-10T19:00:36.011Z
Google announces 'Bard' powered by LaMDA 2023-02-06T19:40:44.459Z
Peter Thiel's speech at Oxford Debating Union on technological stagnation, Nuclear weapons, COVID, Environment, Alignment, 'anti-anti anti-anti-classical liberalism', Bostrom, LW, etc. 2023-01-30T23:31:26.134Z
Bounded distrust or Bounded trust? 2022-10-15T16:41:22.085Z
Vehicle Platooning - a real world examination of the difficulties in coordination 2022-10-13T19:33:21.424Z
Self-defeating conspiracy theorists and their theories 2022-10-04T00:48:53.998Z
Do meta-memes and meta-antimemes exist? e.g. 'The map is not the territory' is also a map 2022-08-07T01:17:43.916Z
How does one recognize information and differentiate it from noise? 2022-08-03T03:57:35.432Z
Letter from leading Soviet Academicians to party and government leaders of the Soviet Union regarding signs of decline and structural problems of the economic-political system (1970) 2022-08-01T22:35:08.750Z
Some reflections on the LW community after several months of active engagement 2022-06-25T17:04:16.233Z
An argument against excessive consumption of coffee from 1674 2022-02-12T18:39:19.593Z
The Defeat of Reason? 2022-02-10T04:29:07.864Z
Everyone’s mired in the deepest confusion, some of the time? 2022-02-09T02:53:58.551Z
How could a friendly AI deal with humans trying to sabotage it? (like how present day internet trolls introduce such problems) 2021-11-27T22:07:15.960Z
What are the mutual benefits of AGI-human collaboration that would otherwise be unobtainable? 2021-11-17T03:09:40.733Z
What’s the likelihood of only sub exponential growth for AGI? 2021-11-13T22:46:25.277Z
Avoiding Negative Externalities - a theory with specific examples - Part 1 2021-11-12T04:09:32.462Z
M. Y. Zuo's Shortform 2021-11-07T01:42:46.261Z


Comment by M. Y. Zuo on How much should e-signatures have to cost a country? · 2023-11-22T17:22:30.305Z · LW · GW

Likely the provider of e-signatures would have to assume some sort of legal liability, hence costs scaling. Maybe even exponentially as they become a bigger and bigger target. But the intelligence and foresight of even the best legal team in the world plateaus.

Comment by M. Y. Zuo on OpenAI Staff (including Sutskever) Threaten to Quit Unless Board Resigns · 2023-11-21T02:58:28.037Z · LW · GW

So it's been falsified? Isn't that a pretty big deal against the source, or whoever purports the letter to be 100% genuine?

Comment by M. Y. Zuo on On Tapping Out · 2023-11-20T02:07:34.040Z · LW · GW

The shy folks can hide behind pseudonyms.

For the shy folks that for whatever reason must use their real name, well there are costs and benefits to using real name identifiers.

And in any case  almost  none of them will ever be so important, or attain a position of such significance, that whether they disengage or engage will move the needle, frankly. 

Maybe if hundreds of such folks simultaneously did so en masse, but anything below that will see replacement.

Whether via themselves creating a pseudonym identity after being embarrassed too many times, new folks joining the online commentating sphere, etc.

And for the small fraction that refuse to do that and quit forever, who will not accept a pseudonym, well there simply isn't a need for that many conversational foils, devils-advocates, agitators, mouth-pieces, prima-donnas, etc...

Even for a LW sized community, a few dozen is probably sufficient to satisfy all relevant interest groups. 

So I'm not convinced it's a big enough problem to be worth changing any paradigms. This applies to all online communities, not just LW, a half-heartedly supported and enforced rule change is usually worse then no change at all.

Comment by M. Y. Zuo on On Tapping Out · 2023-11-18T02:56:31.884Z · LW · GW

I'd love it if tapping out as a safe, no-shame-attached way of leaving a discussion became normal outside of rationalist circles.

It's still unclear why exactly this is a superior paradigm.

Can you list out the upsides and downsides so that passing readers can have a more complete picture?

Dagon makes a good point about discovering external truths. Which is not valued in martial arts, but highly highly valued elsewhere.

Even one novel discovery on par with gravitation or the Hubble constant or similar would be more than worth embarrassing millions upon millions of online commentators in my view.

Comment by M. Y. Zuo on 'Theories of Values' and 'Theories of Agents': confusions, musings and desiderata · 2023-11-15T23:31:12.325Z · LW · GW

Can you link to a source for a definition of 'enactive'? 

Comment by M. Y. Zuo on It's OK to be biased towards humans · 2023-11-13T20:50:11.684Z · LW · GW

By this token, everyone is neutral, no one is friendly, unless I am literally their top priority in the whole world, and they mine (that doesn't sound like simple "friendship"... more like some kind of eternal fated bond between soulmates).

Can you explain your reasoning here? How does a bias towards or against imply a 'top priority'?

Comment by M. Y. Zuo on It's OK to be biased towards humans · 2023-11-12T12:50:11.011Z · LW · GW

Yes, it's possible to strike deals, but that doesn't mean they will actually be 'friendly', at most 'neutral'. They may superficially give off the appearance of being 'friendly' but then again humans do that too all the time.

Comment by M. Y. Zuo on Thinking By The Clock · 2023-11-12T02:33:40.354Z · LW · GW

2 is an interesting strategy. But that 'up goer five' term is an example of what I'm referring to. 

If by happenstance the 'xkcd' link failed for whatever reason, the median passing reader will likely never bother to puzzle out the meaning, at least not without further elaboration.

I'll repeat Raemon's question.

It doesn't seem like he formed a question? In fact it appears as if he specifically refrained from doing so, even after I pointed it out. 

What's the term you think is commonly accepted to describe something I'm talking about with different words?

"up goer five" instead of "simplification", "keeping things simple", etc...

For your prior comment, I don't know what your describing to think of equivalents, hence why I was asking. Asking me doesn't seem to make sense?

You would be the only who could know for certain.

Comment by M. Y. Zuo on Who is Sam Bankman-Fried (SBF) really, and how could he have done what he did? - three theories and a lot of evidence · 2023-11-12T01:59:16.362Z · LW · GW

Is there a specific reason 'affective' was chosen instead of 'emotional' in the naming? 

Is it also a connotation issue?

Comment by M. Y. Zuo on Thinking By The Clock · 2023-11-12T01:30:25.465Z · LW · GW

Of course not? My prior comment was only two sentences, it would be incredibly unlikely for anyone to be able to guess the exact terms and words. 

Comment by M. Y. Zuo on Thinking By The Clock · 2023-11-12T01:15:20.935Z · LW · GW

Thanks for sharing. Did you forget to include the terms you are also confused about or the question you wanted to ask? 

I don't see them on my end.

Comment by M. Y. Zuo on Thinking By The Clock · 2023-11-12T00:25:56.232Z · LW · GW

This is a bit tangential but is there a reason you avoid using the commonly accepted scientific/biological terms and invent your own terminology instead?

The comment chain is difficult to understand for someone who just noticed it under 'Recent Discussions'.

Comment by M. Y. Zuo on It's OK to be biased towards humans · 2023-11-12T00:20:47.326Z · LW · GW

And if I met a race of peaceful, artful, friendly aliens, you can be assured that I would not suddenly turn into a Warhammer 40K Inquisitor whose only wish is to stomp the filthy xenos under his jackboot.

Why would the 'friendly aliens' be friendly if they know your biased against them to any extent?

Comment by M. Y. Zuo on Vote on Interesting Disagreements · 2023-11-10T14:31:37.479Z · LW · GW
  • Manifold's codebase is also shaky and would not survive Manifold-the-company dying right now.

Can you elaborate on this point? Why wouldn't the codebase be salvageable?

Comment by M. Y. Zuo on What is democracy for? · 2023-11-10T14:25:40.558Z · LW · GW

Why do you believe such a system could even exist in the first place?

Comment by M. Y. Zuo on On Overhangs and Technological Change · 2023-11-06T14:47:12.638Z · LW · GW

Could an alien observer have identified Genghis Khan's and the Mongol's future prospects when he was a teenager?

I'm not quite sure.

Comment by M. Y. Zuo on AI Safety is Dropping the Ball on Clown Attacks · 2023-11-05T16:42:10.255Z · LW · GW

I'd love it if people could try the basic precautions and see how harmless they are! Especially because they might be the minimum ask in order to avoid getting your brain and motivation/values hacked.

Wouldn't this  be useful only if one knows for certain their 'brain and motivation/values' are not already 'hacked' beforehand?

Otherwise it would just strengthen the pre-existing 'hacks'.

Comment by M. Y. Zuo on The 6D effect: When companies take risks, one email can be very powerful. · 2023-11-05T16:39:13.870Z · LW · GW
  1. If you work at a company doing potentially risky things, insist on discussing dangers through documented media. If you are retaliated against for documenting communication of risks, you will have grounds for legal recourse. 

Based on what legal reasoning? 

You put in hashtag 'not legal advice', but to say it so confidently suggests it's not just a random thought.

Comment by M. Y. Zuo on Integrity in AI Governance and Advocacy · 2023-11-05T01:21:08.909Z · LW · GW

On the meta level, for unfalsifiable claims (and even falsifiable claims that would take more effort to verify then a normal independent 3rd party adult could spend in say a month)  it doesn't really seem to matter whether the person pushing the claims has integrity, beyond a very low bar?

They just need to have enough integrity to pass the threshold of not being some maniac/murderer/ persistent troll/etc...

But otherwise, beyond that threshold, there doesn't seem to be a much of a downside in treating folks politely while always assuming there's some hidden motivations going on behind the scenes.

And for the falsifiable claims that have reasonable prospects for independent 3rd party verification, assigning credibility, trust, integrity, etc., based on the author's track record of such claims proving to be true is more than sufficient for discussions. And without regard for what hidden motivations might there be.

Maybe this is not sufficient for the community organization/emotional support/etc. side of things, though you'd be the better judge of that.

Comment by M. Y. Zuo on Doubt Certainty · 2023-11-04T14:12:29.095Z · LW · GW

Perhaps you are unfamiliar with how prosecution systems or the judiciary works in general? 

To elaborate, there are several stages along which the 'decisions' of the prosecutors have close to zero impact, if the case does get dropped before final judgement. Even if there is solid evidence of guilt available from the beginning. 

At least in common law countries.

But that also doesn't prevent a prosecutor from going all the way with an actually innocent person based on  a belief in being "good at making correct moral decisions" and deciding their fate. 

For example, if they have a dozen cases on their desk, half actually innocent, half actually guilty, there  simply is no 'must' there. They could decide to drop all of them, drop none, decide based on gut feel, etc... 

And the default is to do nothing and let the paperwork sit and collect dust until the next prosecutor comes in, who can also do the same thing, etc., until the case gets too old and is automatically closed.

Only a small fraction makes it into any court at all, and only a small fraction of those ever go all the way through. Sustained partially, or sometimes entirely, based on their certitude in their 'correct moral decisions'. 

Comment by M. Y. Zuo on Does davidad's uploading moonshot work? · 2023-11-03T18:32:58.160Z · LW · GW

That's a very interesting point,  'synaptic plasticity' is probably a critical difference. At least the recent results in LLMs suggest.

The author not considering, or even mentioning, it also suggests way more work and thought needs to be put into this.

Comment by M. Y. Zuo on Doubt Certainty · 2023-11-03T15:11:05.951Z · LW · GW

When a decision must be made, one cannot infer anything about the certitude with which it is made.

It seems your a bit confused here? Prosecutors in many countries have great leeway to pick and choose. And even after choosing to prosecute someone, at each step along the way they have nearly complete leeway to pick and choose whether to continue on to the next step until final judgement or just drop it one day.

There are very rarely cases where they 'must' make a decision on a particular individual, particularly in the US.


I don't know of any countries where the opposite is true, perhaps you know of one?

Comment by M. Y. Zuo on Doubt Certainty · 2023-11-03T14:15:25.207Z · LW · GW

I don't know what "good at making correct moral decisions" looks like

Maybe you don't know, but at least millions of people, in the judicial systems of nearly every country, claim they do. And these folks, publicly announce, that they have the power to prosecute and decide your fate the moment you step on their territory, partially or sometimes entirely based on their certitude in their 'correct moral decisions'. 

So I think JBlack was pointing out that it seems a bit odd that you could be unaware.

Comment by M. Y. Zuo on The other side of the tidal wave · 2023-11-03T14:09:44.019Z · LW · GW

(Not just literally "labor", but even things like "this person typically makes the jokes, because they are better at making jokes; and this person typically says something empathic; and this person typically makes the decision when the others hesitate...".)

Why would it be desirable to maintain this kind of 'division of labor' in an ideal future?

Comment by M. Y. Zuo on Snapshot of narratives and frames against regulating AI · 2023-11-03T10:45:09.582Z · LW · GW

Why is the amount of matter in a human brain relevant? 

Comment by M. Y. Zuo on Snapshot of narratives and frames against regulating AI · 2023-11-02T23:25:59.730Z · LW · GW

Did you misunderstand my question? 

How does the total mass of the Earth or 'signal speed 1000m/s instead of 100m/s' demonstrate how you know?

Comment by M. Y. Zuo on Snapshot of narratives and frames against regulating AI · 2023-11-02T16:35:14.858Z · LW · GW can spawn 100000 copies at 10x speed...

How do you know that's possible?

Comment by M. Y. Zuo on Snapshot of narratives and frames against regulating AI · 2023-11-02T12:31:29.069Z · LW · GW

How about “It’s not proven yet that vastly super-intelligent machines (i.e. >10x peak human intelligence) are even possible.” as a possible frame?

I can’t see a counterargument to it yet.

Comment by M. Y. Zuo on President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence · 2023-11-02T02:58:21.469Z · LW · GW

Yes it's clear these are your views,  Why do you believe so?

Comment by M. Y. Zuo on President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence · 2023-11-01T21:10:22.270Z · LW · GW

Edit: I do think that there is some future line, across which AI academic publishing would be unequivocally bad. I also think slowing down AI progress in general would be a good thing.

Okay, I guess my question still applies?

For example, it might be that letting it progress without restriction has more upsides then slowing it down.

Comment by M. Y. Zuo on [Linkpost] Mark Zuckerberg confronted about Meta's Llama 2 AI's ability to give users detailed guidance on making anthrax - Business Insider · 2023-11-01T02:11:45.225Z · LW · GW

"Power corrupts the few, while weakness corrupts the many."

  • Eric Hoffer
Comment by M. Y. Zuo on President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence · 2023-11-01T01:16:34.345Z · LW · GW

Why do you believe it's, on the whole, a 'mistake' instead of beneficial?

I can think of numerous benefits, especially in the long term.

e.g. drawing the serious attention of decision makers who might have otherwise believed it to be a bunch of hooey, and ignored the whole topic.

e.g. discouraging certain groups from trying to 'win' in a geopolitical contest, by rushing to create a 'super'-GPT, as they now know their margin of advantage is not so large anymore.

Comment by M. Y. Zuo on President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence · 2023-10-31T21:02:13.892Z · LW · GW

Do you know who would be deciding on orders like this one? Some specialized department in the USG, whatever judge that happens to hear the case, or something else?

Comment by M. Y. Zuo on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-31T20:58:42.236Z · LW · GW

Okay then. 

Comment by M. Y. Zuo on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-31T18:12:49.540Z · LW · GW

How does this relate to the discussion Max H and Roko were having? Or the question I asked of Max H?

Comment by M. Y. Zuo on President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence · 2023-10-31T15:49:49.432Z · LW · GW

(i)    substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons;

Wouldn't this include most, if not all, uncensored LLMs?

And thus any person/organization working on them?

Comment by M. Y. Zuo on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-31T13:03:23.726Z · LW · GW

You do realize setting up each supply chain alone took up well over 1% of total US GDP right?

Comment by M. Y. Zuo on Lying is Cowardice, not Strategy · 2023-10-29T00:31:47.860Z · LW · GW

Why does it not feel like a fair basis?

Maybe if they were deceived into thinking the editor was genuine and trustworthy, but otherwise if they knew they're working with someone untrustworthy , and they still choose to associate their names together publicly, then obviously it impacts their credibility.

Comment by M. Y. Zuo on We're Not Ready: thoughts on "pausing" and responsible scaling policies · 2023-10-28T03:37:31.855Z · LW · GW

I never said it had to be an officially sanctioned one, plenty of folks are capable of writing drafts, ideations, conjectures, etc...

If literally no one has ever published something along these lines, then that's probably the most promising avenue of investigation.

Comment by M. Y. Zuo on Value systematization: how values become coherent (and misaligned) · 2023-10-28T01:49:22.042Z · LW · GW

There's an even more fundamental problem in terms of 'hard to pin down concepts', namely what counts as a 'human' in the first place? 

Comment by M. Y. Zuo on Sanctuary for Humans · 2023-10-28T01:45:00.436Z · LW · GW

Seems a bit of a paradox as 'their share of the universe' is not a fixed quantity, nor did such a concept exist before humans, so how could the 'share' even have been decided on beforehand in order for the first 'individual humans' to have a say?

Comment by M. Y. Zuo on We're Not Ready: thoughts on "pausing" and responsible scaling policies · 2023-10-27T23:11:25.084Z · LW · GW

One reason I'm critical of the Anthropic RSP is that it does not make it clear under what conditions it would actually pause, or for how long, or under what safeguards it would determine it's OK to keep going.

Can you link an example of what you believe to be a well-worded RSP?

Comment by M. Y. Zuo on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-27T01:49:44.621Z · LW · GW

I doubt you, or any human being, would even want to live in a world where such coordination 'succeeded', since it would almost certainly be in the ruins of society wrecked by countless WMDs, flung by the warring parties until all were exhausted except the 'winners', who would probably not have long to live.

In that sense the possible futures where control of powerful AI 'succeeded' could be even worse then where it failed.

Comment by M. Y. Zuo on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-27T01:07:15.910Z · LW · GW

Even a purely human organization with kind of potential power would be controversial enough that probably at least a single digit percentage of adults would not accept it. Which is to say hundreds of millions of humans would likely consider it an enemy too.

And that's assuming it can even be done considering the level of global cooperation demonstrated in 2023.

Comment by M. Y. Zuo on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-27T00:54:43.039Z · LW · GW

3. doesn't seem like a viable option, since there's a decent chance it can disguise itself into appearing as less than superintelligent.

Comment by M. Y. Zuo on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-27T00:46:44.096Z · LW · GW

The challenge, and duty, of coordination to defeat Moloch goes beyond what we think of as governance. We need coordination between AI researchers, AI alignment researchers, forecasters, politicians, investors, CEOs.

The problem is that an entity with that kind of real world coordination capacity would practically need to be so strong that it would likely be more controversial, and face more backlash, then the rogue AGI(s) itself. 

At which point some fraction of humans would likely defect and cooperate with the AGI(s) in order to take it down. 

Comment by M. Y. Zuo on Responsible Scaling Policies Are Risk Management Done Wrong · 2023-10-27T00:38:46.223Z · LW · GW

c. linked to a post which says "here's the actual thing which is needed to make us safe". 

Has anyone made such a credible, detailed, and comprehensive list?

If not, what would it look like in your opinion?

Comment by M. Y. Zuo on Architects of Our Own Demise: We Should Stop Developing AI · 2023-10-26T19:27:59.948Z · LW · GW

In the Manhattan project, there was no disagreement between the physicists, the politicians / generals, and the actual laborers who built the bomb, on what they wanted the bomb to do. They were all aligned around trying to build an object that would create the most powerful explosion possible.

Where did you learn of this?

From what I know it was the opposite, there were so many disagreements, even just among the physicists, that they decided to duplicate nearly all effort to produce two different types of nuclear device designs, the gun type and the implosion type, simultaneously.

e.g.  both plutonium and uranium processing supply chains were set up at massive expense, and later environmental damage,  just in case one design didn't work.

Comment by M. Y. Zuo on Lying is Cowardice, not Strategy · 2023-10-26T19:21:48.428Z · LW · GW

Yes, but the author wasn't forced at gunpoint, presumably, to work with that particular editor. So then the question can be reframed as: why did the author choose to work with an editor that seems untrustworthy?

Comment by M. Y. Zuo on AI Safety is Dropping the Ball on Clown Attacks · 2023-10-23T13:05:11.822Z · LW · GW

To be frank trevor, you don't seem to have referenced or cited any of the extensive 20th century and prior literature on memetics, social theory, sociology, mass movements, human psychology in large groups, etc... 

Which is likely what the parent was referring to.

Although I have read nowhere close to all of it, I've read enough to not see any novel substantive arguments or semi-plausible proofs.

Most LW readers don't expect anything at the level of a formal mathematical or logical proof, but sketching out a defensible semi-plausible path to one would help a lot. Especially for a post of this length.

It also doesn't help that your taking for granted many things which are far from decided. For example, the claim: cryopreservation; everyone on earth happens to be doing it catastrophically wrong and its actually a super quick fix, less than a few days or even a few hours and your entire existence is substantially safer. 

Sounds very exaggerated because cryopreservation itself does not have that solid of a foundation as your implying here. 

Since no one has yet offered a physically plausible solution to restoring a cryopreserved human, even with unlimited amounts of computation and energy, with their neural structure and so on intact. (That fits within known thermodynamic boundaries.)

It's more of a 'there is a 1 in a billion chance some folks in the future will stumble on a miracle and will choose to work on me' or some variation. And people doing it anyways since even a tiny tiny chance is better than nothing in their books.