Posts

Approval Extraction Advertised as Production 2019-12-15T14:51:44.878Z · score: 84 (38 votes)
Judgment, Punishment, and the Information-Suppression Field 2019-11-06T17:44:38.908Z · score: 20 (13 votes)
Blatant lies are the best kind! 2019-07-03T20:45:56.948Z · score: 24 (14 votes)
Reason isn't magic 2019-06-18T04:04:58.390Z · score: 116 (40 votes)
Drowning children are rare 2019-05-28T19:27:12.548Z · score: 15 (46 votes)
A War of Ants and Grasshoppers 2019-05-22T05:57:37.236Z · score: 17 (5 votes)
Towards optimal play as Villager in a mixed game 2019-05-07T05:29:50.826Z · score: 46 (13 votes)
Hierarchy and wings 2019-05-06T18:39:43.607Z · score: 26 (11 votes)
Blame games 2019-05-06T02:38:12.868Z · score: 44 (10 votes)
Should Effective Altruism be at war with North Korea? 2019-05-05T01:50:15.218Z · score: 15 (13 votes)
Totalitarian ethical systems 2019-05-03T19:35:28.800Z · score: 36 (12 votes)
Authoritarian Empiricism 2019-05-03T19:34:18.549Z · score: 40 (13 votes)
Excerpts from a larger discussion about simulacra 2019-04-10T21:27:40.700Z · score: 51 (17 votes)
Blackmailers are privateers in the war on hypocrisy 2019-03-14T08:13:12.824Z · score: 24 (17 votes)
Moral differences in mediocristan 2018-09-26T20:39:25.017Z · score: 21 (8 votes)
Against the barbell strategy 2018-09-20T15:19:08.185Z · score: 21 (20 votes)
Interpretive Labor 2018-09-05T18:36:49.566Z · score: 28 (16 votes)
Zetetic explanation 2018-08-27T00:12:14.076Z · score: 79 (43 votes)
Model-building and scapegoating 2018-07-27T16:02:46.333Z · score: 23 (7 votes)
Culture, interpretive labor, and tidying one's room 2018-07-26T20:59:52.227Z · score: 29 (13 votes)
There is a war. 2018-05-24T06:44:36.197Z · score: 64 (27 votes)
Talents 2018-05-18T20:30:01.179Z · score: 48 (13 votes)
Oops Prize update 2018-04-20T09:10:00.873Z · score: 42 (9 votes)
Humans need places 2018-04-19T19:50:01.931Z · score: 113 (28 votes)
Kidneys, trade, sacredness, and space travel 2018-03-01T05:20:01.457Z · score: 51 (13 votes)
What strange and ancient things might we find beneath the ice? 2018-01-15T10:10:01.010Z · score: 32 (12 votes)
Explicit content 2017-12-02T00:00:00.946Z · score: 14 (8 votes)
Cash transfers are not necessarily wealth transfers 2017-12-01T10:10:01.038Z · score: 111 (43 votes)
Nightmare of the Perfectly Principled 2017-11-02T09:10:00.979Z · score: 32 (8 votes)
Poets are intelligence assets 2017-10-25T03:30:01.029Z · score: 26 (9 votes)
Seeding a productive culture: a working hypothesis 2017-10-18T09:10:00.882Z · score: 28 (9 votes)
Defense against discourse 2017-10-17T09:10:01.023Z · score: 64 (21 votes)
On the construction of beacons 2017-10-16T09:10:00.866Z · score: 58 (18 votes)
Sabbath hard and go home 2017-09-27T07:49:40.482Z · score: 87 (49 votes)
Why I am not a Quaker (even though it often seems as though I should be) 2017-09-26T07:00:28.116Z · score: 62 (32 votes)
Bad intent is a disposition, not a feeling 2017-05-01T01:28:58.345Z · score: 12 (15 votes)
Actors and scribes, words and deeds 2017-04-26T05:12:29.199Z · score: 26 (11 votes)
Effective altruism is self-recommending 2017-04-21T18:37:49.111Z · score: 72 (53 votes)
An OpenAI board seat is surprisingly expensive 2017-04-19T09:05:04.032Z · score: 5 (6 votes)
OpenAI makes humanity less safe 2017-04-03T19:07:51.773Z · score: 22 (22 votes)
Against responsibility 2017-03-31T21:12:12.718Z · score: 13 (12 votes)
Dominance, care, and social touch 2017-03-29T17:53:20.967Z · score: 3 (4 votes)
The D-Squared Digest One Minute MBA – Avoiding Projects Pursued By Morons 101 2017-03-19T18:48:55.856Z · score: 1 (2 votes)
Threat erosion 2017-03-15T23:32:30.000Z · score: 1 (2 votes)
Sufficiently sincere confirmation bias is indistinguishable from science 2017-03-15T13:19:05.357Z · score: 19 (19 votes)
Bindings and assurances 2017-03-13T17:06:53.672Z · score: 1 (2 votes)
Humble Charlie 2017-02-27T19:04:37.578Z · score: 2 (3 votes)
Against neglectedness considerations 2017-02-24T21:41:52.144Z · score: 1 (2 votes)
GiveWell and the problem of partial funding 2017-02-14T10:48:38.452Z · score: 2 (3 votes)
The humility argument for honesty 2017-02-05T17:26:41.469Z · score: 4 (5 votes)

Comments

Comment by benquo on Less Wrong Poetry Corner: Walter Raleigh's "The Lie" · 2020-01-05T05:26:04.612Z · score: 9 (4 votes) · LW · GW

I don't understand how the second paragraph follows from the first at all.

Comment by benquo on Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think · 2019-12-27T14:18:50.778Z · score: 6 (3 votes) · LW · GW

https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases

Comment by benquo on Maybe Lying Doesn't Exist · 2019-12-25T13:56:44.297Z · score: 2 (1 votes) · LW · GW

The tone and implications of comments along the lines of “this wording is going to cause a lot of people to believe specific proposition X, when it seems to me like what you would actually be willing to defend narrower proposition Y” is very different from that of “this wording is inappropriate because it is likely to upset people.”

An important feature of the first comment is that it actually does some work trying to clarify, and is an implicit ITT.

Comment by benquo on Maybe Lying Doesn't Exist · 2019-12-25T13:50:58.198Z · score: 2 (1 votes) · LW · GW

Someone deceiving with conscious intent can apply more compute / intelligence and other resources for optimizing and maintaining the lie, which means the deception can be much bigger and more consequential, thereby causing greater damage to others.

Unconscious deception is hard to distinguish from innocent mistakes.

Surely someone consciously intending to deceive can apply some of that extra compute to making it harder to distinguish their behavior from an innocent mistake.

Comment by benquo on Approval Extraction Advertised as Production · 2019-12-21T23:36:41.013Z · score: 4 (2 votes) · LW · GW
Altman's belief is explicitly that he does not know what the next company should look like, what area it should be in, and what problem it should solve, and so I think he mostly thinks that good people need to be very reactive to their environment

I expect this to be good at many things - he's probably not wrong that Napoleon would make a good YC founder by his standards - but I expect it's not the mindset that can develop substantive new tech. And sure enough, it seems like substantive new tech is mostly not being developed.

Comment by benquo on Approval Extraction Advertised as Production · 2019-12-18T16:08:20.365Z · score: 4 (3 votes) · LW · GW

No.

Comment by benquo on Approval Extraction Advertised as Production · 2019-12-17T15:13:19.338Z · score: 10 (7 votes) · LW · GW

I didn't use the word scam in the post. Cousin_it's comment was defending YC by saying, substantively, don't worry, it's just a pretend gatekeeper, so I tried to make that more explicit. How, precisely, does that not constitute being a scam?

Are we supposed to be at war with lions now? What's wrong with apex predators?

Comment by benquo on Approval Extraction Advertised as Production · 2019-12-16T12:44:15.701Z · score: 3 (3 votes) · LW · GW

Either it matters a lot for a good startup’s success, or it doesn’t.

If it does, then the gatekeeper narrative is true. If it doesn’t, then how exactly isn’t it a scam?

This is such an obvious point that I’m worried that I’m confused about what’s really going on in this conversation.

Comment by benquo on Approval Extraction Advertised as Production · 2019-12-16T04:11:32.862Z · score: 1 (12 votes) · LW · GW

Yeah, it would be pretty cool for PG to more overtly admit that YC is a scam. As it is, it's not actually harmless, since it does attract a lot of attention from people who think they're paying attention to a nonscam.

Comment by benquo on Approval Extraction Advertised as Production · 2019-12-16T03:53:11.017Z · score: 17 (10 votes) · LW · GW

Y Combinator could very easily (if not for the apparent emotional difficulties mentioned in Black Swan farming) have instead been organized to screen for founders making a credible effort to create a great product, instead of screening for generalized responsiveness to tests.

Comment by benquo on The Lesson To Unlearn · 2019-12-15T14:57:23.207Z · score: 6 (3 votes) · LW · GW

https://www.lesswrong.com/posts/ypXHnS3vWeDRmDsK9/approval-extraction-advertised-as-production

Comment by benquo on ialdabaoth is banned · 2019-12-15T14:15:05.774Z · score: 12 (4 votes) · LW · GW

OK but what's actually being done is a one-off ban of someone with multiple credible public rape allegations against them. The specific policy goal of developing better immune responses to epistemic corruption is just not relevant to that and I don't see how anyone on the mod team is doing anything best explained by an attempt to solve that problem.

Comment by benquo on The Lesson To Unlearn · 2019-12-09T19:05:56.157Z · score: 2 (1 votes) · LW · GW

Graham's implying that at least for the vast majority of person-test combinations, the spirit of passing tests is hacking them:

Even though I was a diligent student, almost all the work I did in school was aimed at getting a good grade on something.
To many people, it would seem strange that the preceding sentence has a "though" in it. Aren't I merely stating a tautology? Isn't that what a diligent student is, a straight-A student? That's how deeply the conflation of learning with grades has infused our culture.
[...]
If getting into college were merely a matter of having the quality of one's mind measured by admissions officers the way scientists measure the mass of an object, we could tell teenage kids "learn a lot" and leave it at that.

I don't see how, specifically, to distinguish this sort of thing from what Hotel Concierge is saying, unless you think Hotel Concierge is against trying at anything. As far as I can tell Hotel Concierge isn't saying you shouldn't try to be happy, or achieve outcomes you care about via delayed gratification, or be smart, or learn a lot, just that it's a problem when people are pushed to optimize for performing simulacra of those things.

Comment by benquo on The Lesson To Unlearn · 2019-12-09T18:34:57.544Z · score: 5 (3 votes) · LW · GW

Currently writing this up at more length, I invite anyone to remind me if I forget to post a link here within a week. The short version is that Graham is implying that his behavior is aligned with the values of his essay - people are likely to trust Y-Combinator to help them learn to do real things, since it's a Paul Graham creation and he wrote this essay - while in fact he is doing the opposite and set up the world's best, most prestigious school for Succeeding by Passing Tests.

Comment by benquo on The Lesson To Unlearn · 2019-12-09T16:29:57.772Z · score: 12 (10 votes) · LW · GW

It's not that this essay is wrong. It's just that it's a rehash of what Hotel Concierge covered better and in more depth in The Stanford Marshmallow Prison Experiment, and it's dangerously misleading advertising for the VC fund / cult Graham started, Y-Combinator, which is the current apex predator of the real-life Stanford Marshmallow Prison Experiment.

Comment by benquo on Drowning children are rare · 2019-12-08T05:48:58.791Z · score: 2 (1 votes) · LW · GW

It was originally marketed as a health tonic, but its apparent curative properties were due to the powerful stimulant and analgesic cocaine, not any health-enhancing ingredients. Later the cocaine was taken out (but the “Coca” in the name retained), so now it fools the subconscious into thinking it’s healthful with - on different timescales - mass media advertising, caffeine, and refined sugar.

It’s less overtly a scam now, in large part because it has the endowment necessary to manipulate impressions more subtly at scale.

Comment by benquo on Drowning children are rare · 2019-12-06T14:15:44.921Z · score: 13 (3 votes) · LW · GW

Maybe more specifically an ingroup that takes over a potentially real, profitable social niche, squeezes out everyone else, and uses the niche’s leverage to maximize rent extraction, is a gang.

Comment by benquo on Drowning children are rare · 2019-12-06T13:56:07.705Z · score: 4 (2 votes) · LW · GW

Coca-Cola produces something about as worthless as Theranos machines, substituting the experience of a thing for the thing itself, & is pretty blatant about it. The scams that “win” gerrymander our concept-boundaries to make it hard to see. Likewise Pepsi. JPMorgan Chase & Bank of America, in different ways, are scams structurally similar to Bernie Madoff but with a legitimate state subsidy to bail them out when they blow up. This is not an exhaustive list, just the first 4 that jumped out at me. Pharma is also mostly a scam these days, nearly all of the extant drugs that matter are already off-patent.

Also Facebook, but “scam” is less obviously the right category.

Comment by benquo on Drowning children are rare · 2019-12-03T16:35:10.500Z · score: 6 (3 votes) · LW · GW
I have a background expectation that the most blatant kinds of fraudulence will be caught.

Consider how long Theranos operated, its prestigious board of directors, and the fact that it managed to make a major sale to Walgreens before blowing up. Consider how prominent Three Cups of Tea was (promoted by a New York Times columnist), for how long, before it was exposed. Consider that official US government nutrition advice still reflects obviously distorted, politically motivated research from the early 20th Century. Consider that the MLM company Amway managed to bribe Harvard to get the right introductions to Chinese regulators. Scams can and do capture the official narrative and prosecute whistleblowers.

Consider that pretty much by definition we're not aware of the most successful scams.

Related: The Scams Are Winning

Comment by benquo on Drowning children are rare · 2019-12-03T16:20:47.646Z · score: 3 (2 votes) · LW · GW
Presumably you believe that point 2 holds, not just because of the GDP example, but because you've seen many, many examples (like health care, which you mention above). Or maybe because you have an analytical argument that the sort of thing that happens with GDP has to generalize to other credit allocation systems?

Both - it would be worrying to have an analytic argument but not notice lots of examples, and it would require much more investigation (and skepticism) if it were happening all the time for no apparent reason.

I tried to gesture at the gestalt of the argument in The Humility Argument for Honesty. Basically, all conflict between intelligent agents contains a large information component, so if we're fractally at war with each other, we should expect most info channels that aren't immediately life-support-critical to turn into disinformation, and we should expect this process to accelerate over time.

For examples, important search terms are "preference falsification" and "Gell-Mann amnesia".

I don't think I disagree with you on GiveDirectly, except that I suspect you aren't tracking some important ways your trust chain is likely to make correlated errors along the lines of assuming official statistics are correct. Quick check: what's your 90% confidence interval for global population, after Googling the official number, which is around 7.7 billion?

Comment by benquo on Drowning children are rare · 2019-12-03T04:40:46.480Z · score: 5 (3 votes) · LW · GW

Overall your wording seems pretty close.

Most white collar workers are executing a similar maneuver, except that instead of using force, they are corrupting the victim's ability to make sense of the situation.

I think it's actually a combination of this, and actual coordination to freeze out marginal gangs or things that aren't gangs, from access to the system. Venture capitalists, for example, will tend to fund people who feel like members of the right gang, use the right signifiers in the right ways, went to the right schools, etc. Everyone I've talked with about their experience pitching startups has reported that making judgments on the merits is at best highly noncentral behavior.

If enough of the economy is cartelized, and the cartels are taxing noncartels indirectly via the state, then it doesn't much matter whether the cartels apply force directly, though sometimes they still do.

So called "career capital" amounts to having more prestige, or otherwise be better at convincing people, and therefore being able to extort larger amounts.

It basically involves sending or learning how to send a costly signal of membership in a prestigious gang, including some mixture of job history, acculturation, and integrating socially into a network.

Comment by benquo on Drowning children are rare · 2019-12-03T04:31:18.250Z · score: 2 (1 votes) · LW · GW

This is something like a 9 - gets the overall structure of the argument right with some important caveats:

I'd make a slightly weaker a claim for 2 - that credit-allocation methods have to be presumed broken until established otherwise, and no adequate audit has entered common knowledge.

An important part of the reason for 3 is that, the larger the share of "knowledge work" that we think is mostly about creating disinformation, the more one should distrust any official representations one hasn't personally checked, when there's any profit or social incentive to make up such stories. Based on my sense of the character of the people I met while working at GiveWell, and the kind of scrutiny they said they applied to charities, I'd personally be surprised if GiveDirectly didn't actually exist, or simply pocketed the money. But it's not at all obvious to me that people without my privileged knowledge should be sure of that.

Comment by benquo on Talents · 2019-12-02T18:14:55.043Z · score: 3 (2 votes) · LW · GW

I agree that there are more and less capital-intensive types of work, but it might make sense for the excellent programmer to use some of their surplus to outsource nonprogramming tasks (e.g. order delivery, hire a cleaning service, pay for high-quality day care for their kids), to free up more time for the thing they have the greatest comparative advantage at.

Comment by benquo on Drowning children are rare · 2019-12-02T18:11:39.585Z · score: 2 (1 votes) · LW · GW
I basically already thought that lots of jobs are bullshit, but I might skim or listen to David Graeber's book to get more data.

It's not really about how many jobs are bullshit, so much as what it means to do a bullshit job. On Graeber's model, bullshit jobs are mostly about propping up the story that bullshit jobs are necessary for production. Moral Mazes might help clarify the mechanism, and what I mean about gangs - a lot of white-collar work involves a kind of participatory business theater, to prop up the ego claims of one's patron.

The more we think the white-collar world works this way, the more skeptical we should be of the literal truth of claims to be "working on" some problem or other using conventional structures.

Comment by benquo on Drowning children are rare · 2019-12-02T18:06:44.238Z · score: 6 (3 votes) · LW · GW
I read the all of "There Is a War", but I still don't get the claim, "GDP is a measurement of the level of coercion in a society." I'm going to keep working at it.

I think it's analytically pretty simple. GDP involves adding up all the "output" into a single metric. Output is measured based on others' willingness to pay. The more payments are motivated by violence rather than the production of something everyone is glad to have more of, the more GDP measures expropriation rather than production. There Is A War is mostly about working out the details & how this relates to macroeconomic ideas of "stimulus," "aggregate demand," etc, but if that analytic argument doesn't make sense to you, then that's the point we should be working out.

Comment by benquo on Talents · 2019-12-02T18:03:55.668Z · score: 6 (3 votes) · LW · GW

Pretty close - the main reason I wouldn't say it like that is that "created value" is a bit ambiguous here. The issue is really whether you can add up all the value created to get something like total value produced (a measure like GDP), which would be a desirable thing to increase in a pure loaves world, but definitely not in an arrows-and-shields world.

Comment by benquo on There is a war. · 2019-11-26T18:55:30.587Z · score: 3 (2 votes) · LW · GW

Declining life expectancy suggests a general increase in scarcity. If the processes around you are trying to make things more scarce for you rather than less, then you're in something more like a conflict relation than a trade relation to them, and delayed gratification is much less feasible in wartime or other emergencies where you're likely to die if you don't get something done NOW.

What are the competing explanations for high time preference?

Comment by benquo on There is a war. · 2019-11-26T15:33:20.847Z · score: 2 (1 votes) · LW · GW

IRR, discount rate, and effective time preference are really the same thing as expressed in related domains.

Comment by benquo on Is daily caffeine consumption beneficial to productivity? · 2019-11-26T15:07:00.737Z · score: 8 (5 votes) · LW · GW

This question seems to conflate the claim that caffeine works as a stimulant, and the claim that in a meaningful sense stimulants increase productivity. I suspect this is true for measured productivity, but not at all obviously true for net production.

(Related: Talents, Blitzed: Drugs in the Third Reich)

Comment by benquo on Drowning children are rare · 2019-11-26T11:31:18.175Z · score: 16 (4 votes) · LW · GW

helping by de­fault (they pay their land­lord, who buys goods, which pays man­u­fac­tur­ers, etc.)

The exact opposite - getting paid should imply something. The naive Econ 101 view is that it implies producing something of value. "Production" is generally measured in terms of what people are willing to pay for.

If getting paid has little to do with helping others on net , then our society’s official unit of account isn’t tracking production (Talents), GDP is a measurement of the level of coercion in a society (There Is a War), the bullshit jobs hypothesis is true, we can’t take job descriptions at face value, and CEA’s advice to build career capital just means join a powerful gang.

This undermines enough of the core operating assumptions EAs seem to be using that the right thing to do in that case is try to build better models of what's going on, not act based on what your own models imply is disinformation.

Comment by benquo on Judgment, Punishment, and the Information-Suppression Field · 2019-11-07T03:24:26.276Z · score: 5 (2 votes) · LW · GW

Seems to me like punishment might not accomplish what it claims to & might be harmful on net. I have a future post planned to explain that point more fully though.

Comment by benquo on Judgment, Punishment, and the Information-Suppression Field · 2019-11-07T03:13:18.740Z · score: 2 (1 votes) · LW · GW

Thanks! Unfortunately I guess signals crossed and you missed a last-minute correction I made - changing the arguably untrue and value-laden “increasingly nasty” to the more factual “occasionally genocidal.” I’ve restored the correction, but figured I’d note it here because it was the topic of a few comments.

Comment by Benquo on [deleted post] 2019-08-21T17:55:05.090Z
I don't feel like "the epistemics are failing" is the coarse-grained description I'd use, I think there's more details about which bits are going on and why (and which bits actually seem to be going quite excellently!), but I wanted to agree with feeling sad about this particular bit.

I expect it would be quite useful both here and more generally for you to expand on your model of this.

Comment by Benquo on [deleted post] 2019-08-21T16:43:14.202Z

I'd put you in a cluster with Lahwran and Said Achmiz on this, if that helps clarify the gestalt I'm trying to identify. By contrast, I'd say that the cluster Benito's pointing to - which I'd say mainly publicly includes me, Jessicata, Zvi Mowshowitz, and Romeo, though obviously also Michael Vassar - is organized around the idea that if you honestly apply loose gestalt thinking, it can very efficiently and accurately identify structures in the world - all of which can ultimately be reconciled - but that this kind of honesty necessarily involves reprogramming ourselves to not be such huge tools, and most people, well, haven't done that, so they end up having to pick sides between perception and logic.

Comment by Benquo on [deleted post] 2019-08-21T16:29:13.099Z

My impression is that for the majority of my audience, my efforts to show how everything adds up to normality are redundant, and mostly they're going by a vague feeling. Overall, it seems to me that there are people trying to do the kind of translational work Davis is asking for, but the community is not, as a whole, applying the sort of discernment that would demand such work. Whether or not this is the problem Davis is trying to identify, I'm worried enough about it that LessWrong has been getting less and less interesting to me as a community to engage with. You're probably by far the person most worth reaching who isn't already in my "faction," such as it is, and Davis is one of the few others who seem to be trying to make sense of things at all.

Comment by Benquo on [deleted post] 2019-08-21T16:25:55.150Z

The rhetorical structure of this post seems to imply a substantially different model of who your audience is, and what sort of appeals will work, than the model explicitly described. Since the question of which arguments will work on your intended audience is actually the whole point of your post, I think you should do an internal double crux on this issue, the results of which I expect will change your entire strategic sense of what's going on in the Rationalist community the value of schisms, etc. I'm happy to spend plenty of time in live conversation on this if you're interested and think seeking that sort of mutual understanding might be worth your time.

Explicitly, it seems like you're saying that the way in which woo ideas are being brought into the community discourse is objectionable, because people are simply adopting new fragmentary beliefs that on the surface blatantly contradict much of the other things they believe, without doing the careful work of translation, synthesis, and rigorous grounding of concepts. You're arguing that we should be alarmed by this trend, and engage in substantial retrenching. This is fundamentally an appeal to rational monism.

But rhetorically, you begin by offering a simple list of the names of things admitted into the conversation, and implicitly ask the reader to agree that this is objectionable before talking about method at all (and you don't go into enough detail on the type of skepticism and rigor you're suggesting for me to have a sense of whether I even agree.) The implied model here is that for most of your readers appeals to reason are futile, and you can at best get them to reject some ideas as foreign.

I think that the second model - the one you used to decide how to write the post - is a better representation of the current state of the Rationalist community than the first one. I don't see much value in preserving or restoring the "integrity" of such a community (constituting in practice the cluster of people vaguely attracted to the Sequences, HPMoR, EA, the related in-person Bay Area community, CFAR, and the MIRI narrative). I see a lot of value in a version of this post clearly targeted to the remnant better-described by the first model. It would be nice if we could communicate about this in a way that didn't hurt the feelings of the MOPs too badly, since they never wanted to hurt anyone.

Comment by benquo on "Rationalizing" and "Sitting Bolt Upright in Alarm." · 2019-08-20T21:46:57.600Z · score: 4 (3 votes) · LW · GW
it's not obvious to me that humans are friendly if you scale them up

It seems like any AI built by multiple humans coordinating is going to reflect the optimization target of the coordination process building it, so we had better figure out how to make this so.

Comment by benquo on Power Buys You Distance From The Crime · 2019-08-18T21:04:56.949Z · score: 9 (2 votes) · LW · GW

How is that relevant to the OP?

Comment by Benquo on [deleted post] 2019-08-14T22:21:21.443Z

I think my basic worry is that if there's not an active culture-setting drive against concern-trolling, then participating on this site will mean constant social pressure against this sort of thing. That means that if I try to do things like empathize with likely readers, take into account feedback, etc., I'll either gradually become less clear in the direction this kind of concern trolling wants, or oppositionally pick fights to counteract that, or stop paying attention to LessWrong, or put up crude mental defenses that make me a little more obtuse in the direction of Said. Or some combination of those. I don't think any of those are great options.

No one here since Eliezer seems to have had both the social power and the willingness to impose new - not quite standards, but social incentive gradients. The mod team has the power, I think.

Thanks for clarifying that you're firmly in favor of at least tolerating this kind of speech. That is somewhat reassuring. But the culture is also determined by which things the mods are willing to ban for being wrong for the culture, and the implicit, connotative messages in the way you talk about conflicts as they come up. The generator of this kind of behavior is what I'm trying to have an argument with, as it seems to be to be basically embracing the drift towards pressure against the kind of clarity-creation that creates discomfort for people connected to conventional power and money. I recognize that asking for an active push in the other direction is a difficult request, but LessWrong's mission is a difficult mission!

Comment by Benquo on [deleted post] 2019-08-14T22:11:44.583Z

You got it right, 1 was the suggested change I was most disappointed by, as the weakening of rhetorical force also took away a substantive claim that I actually meant to be making: that GiveWell wasn't actually doing utilitarian-consequentialist reasoning about opportunity cost, but was instead displaying a sort of stereotyped accumulation behavior. (I began Effective Altruism is Self-Recommending with a cute story about a toddler to try to gesture at a similar "this is stereotyped behavior, not necessarily a clever conscious scheme," but made other choices that interfered with that.)

4 turned out to be important too, since (as I later added a quote and link referencing) "unfairness" literally was a stated motivation for GiveWell - but Zack didn't know that at the time, and the draft didn't make that clear, so it was reasonable to suggest the change.

The other changes basically seemed innocuous.

Comment by benquo on Power Buys You Distance From The Crime · 2019-08-14T17:08:52.148Z · score: 6 (3 votes) · LW · GW

Who, specifically, is the enemy here, and what, specifically, is the evil thing they want?

It seems to me as though you’re describing motives as evil which I’d consider pretty relatable, so as far as I can tell, you’re calling me an enemy with evil motives. Are people like me (and Elizabeth’s cousin, and Elizabeth herself, both of whom are featured in examples) a special exception whom it’s nonsuspect to call evil, or is there some other reason why this is less suspect than the OP?

Comment by benquo on Power Buys You Distance From The Crime · 2019-08-12T06:42:48.464Z · score: 19 (8 votes) · LW · GW

If your concern is that this is evidence that the OP is wrong (since it has conflict-theoretic components, which are mindkillers), it seems important to establish that there are important false object-level claims, not just things that make such mistakes likely. If you can't do that, maybe change your mind about how much conflict theory introduces mistakes?

If you're just arguing that laying out such models are likely to have bad consequences for readers, this is an important risk to track, but it's also changing the subject from the question of whether the OP's models do a good job explaining the data.

Comment by benquo on Benito's Shortform Feed · 2019-08-07T17:33:45.506Z · score: 7 (3 votes) · LW · GW

I'm much more interested in finding out what your model is after having tried to take those considerations into account, than I am in a point-by-point response.

Comment by benquo on Power Buys You Distance From The Crime · 2019-08-07T17:27:01.315Z · score: 8 (4 votes) · LW · GW
  1. Talk is cheap, especially when claiming not to hold opinions widely considered blameworthy.
  2. Buchanan's academic career (and therefore ability to get our attention) can easily depend on racists' appetite for convenient arguments regardless of his personal preferences.
Comment by benquo on Power Buys You Distance From The Crime · 2019-08-07T17:23:04.963Z · score: 7 (4 votes) · LW · GW

This is an excellent analytical account of the underlying dynamics. It also VERY strongly resembles the series of blame-deflections described in Part II Chapter VII of Atlas Shrugged (the train-in-the-tunnel part), where this sort of information suppression ultimately backfires on the nominal beneficiary.

Comment by benquo on Power Buys You Distance From The Crime · 2019-08-05T02:09:27.434Z · score: 4 (2 votes) · LW · GW

I don't quite draw the line at denotative vs enactive speech - command languages which are not themselves contested would fit into neither "conflict theory" nor "mistake theory."

"War is the continuation of politics by other means" is a very different statement than its converse, that politics is a kind of war. Clausewitz is talking about states with specific, coherent policy goals, achieving those goals through military force, in a context where there's comparatively little pretext of a shared discourse. This is very different from the kind of situation described in Rao where a war is being fought in the domain of ostensibly "civilian" signal processing.

Comment by benquo on Power Buys You Distance From The Crime · 2019-08-04T17:26:01.929Z · score: 4 (2 votes) · LW · GW

Mechanism design is, to a large extent, a conflict theory, because it assumes conflicts of interest between different agents, and is determining what consequences should happen to different agents, e.g. in some cases “who we should be angry at” if that’s the best available implementation.

"Conflict theory" is specifically about the meaning of speech acts. This not the general question of conflicting interests. The question of conflict vs mistake theory is fundamentally, what are we doing when we talk? Are we fighting over the exact location of a contested border, or trying to refine our compression of information to better empower us to reason about things we care about?

Comment by benquo on Mistake Versus Conflict Theory of Against Billionaire Philanthropy · 2019-08-04T17:03:11.469Z · score: 2 (1 votes) · LW · GW

For me, a conflict theorist is someone who thinks the main driver of disagreement is self-interest rather than honest mistakes.

I don't see how to reconcile this with:

Conflict theorists treat politics as war. Different blocs with different interests are forever fighting to determine whether the State exists to enrich the Elites or to help the People.

Comment by benquo on Making Exceptions to General Rules · 2019-07-23T05:27:27.921Z · score: 21 (5 votes) · LW · GW

I think this schema could benefit from a distinction between rules for internal and external consumption. For external consumption there's some benefit to having (implied) policies for exceptions that are responsive to likely costs, both the internal costs of acting according to the rule in an emergency, and the external cost of having one's expectations violated.

But for internal consumption, it makes more sense to, as Said Achmiz points out, just change the rule to a better one that gets the right answer in this case (and all the prior ones). I think people are confused by this in large part because they learned, from authoritarian systems, to rule themselves as though they were setting rules for external consumption or accountability, instead of reasoning about and doing what they want.

This leads to a weird tension where the same person is sternly setting rules for themselves (and threatening to shame themselves for rule violations), and trying to wiggle out of those same rules as though they were demands by a malevolent authority.

Comment by benquo on Dialogue on Appeals to Consequences · 2019-07-22T01:02:13.897Z · score: 4 (2 votes) · LW · GW

This is maybe half or more of what Robin Hanson wrote about back when it was still all on overcomingbias.com