supposedlyfun's Shortform 2020-09-26T20:56:30.074Z


Comment by supposedlyfun on The Comprehension Curve · 2021-02-23T07:03:35.708Z · LW · GW

The part about habitual speed rings true to me. I am slowly working my way through the Bermúdez textbook on cognitive science and find that my brain "wants" to read at a customary pace, which is too fast for comprehension of a subject I'm relatively unfamiliar with (haven't touched a science textbook since high school).

Forcing myself to slow down gut-feels like going under the speed limit in the far-left lane on the interstate, like wasting time, even though I conscious-know it's approximately infinity times more important to understand the text than it is to be able to truthfully say that I did "reading" on all the words in it.

Comment by supposedlyfun on Overconfidence is Deceit · 2021-02-18T12:48:56.723Z · LW · GW

When a presidential candidate promises to do something that would require congressional approval but ends up unable to get it, how is that not an instance of apparent overconfidence? And neither president seems to have suffered electorally due to that particular failure. So they both seem like evidence for the claim that apparent overconfidence (within reason) isn't punished.

Comment by supposedlyfun on The map and territory of NFT art · 2021-02-16T13:28:40.611Z · LW · GW

This comment captures both the substance and the style of my reaction to the story.

Comment by supposedlyfun on Covid 1/7: The Fire of a Thousand Suns · 2021-01-08T00:48:35.672Z · LW · GW

You deserve to be paid for these covid updates. Are you getting paid? How does one pay you for this?  This must've taken at least 15 hours.

Comment by supposedlyfun on Give it a google · 2020-12-29T23:56:20.552Z · LW · GW

It could be me, too. I used to have to think about SEO when I was in charge of a previous employer's minimal online marketing/blogging, so plenty of this sensation could be confirmation bias (tendency to blame bad Google results on SEO rather than Google, whose product it is). But I think I can spot a page that was generated primarily through SEO--probably with a better accuracy rate than me differentiating between GPT-3 and Wittgenstein.

Comment by supposedlyfun on Give it a google · 2020-12-29T23:08:07.148Z · LW · GW

SEO has gotten to the point that it actually takes some skill to google things usefully.

So true and breaks my heart. I can feel the increased Googling difficulty, in my bones, over the last 18 months or so.  Tragedy of the commons strikes again.

When the Internet was young, I would go straight to for device reviews. When Google got good, I would go to Google.  Now, I go straight to for device reviews.

Comment by supposedlyfun on Moloch Hasn’t Won · 2020-12-29T01:22:49.842Z · LW · GW

I would be very interested in your proposed follow-up but don't have enough game theory to say whether the idea has obvious flaws.

Comment by supposedlyfun on TAI Safety Bibliographic Database · 2020-12-23T05:35:58.662Z · LW · GW

This is a helpful resource across the board, and I found the charts exceptionally informative.

Comment by supposedlyfun on Epistemic Warfare · 2020-12-20T13:02:44.436Z · LW · GW

Fair points.  Upon reflection, I would probably want to know in advance that the Dark Arts intervention was going to work before authorizing it, and we're not going to get that level of certainty short of an FAI anyway, so maybe it's a moot point.

Comment by supposedlyfun on What is the state of the ego depletion field? · 2020-12-20T12:59:54.766Z · LW · GW

I'm late to this party, but Baumeister's responses and comments in the recent journal article are horrifying.  

Having mentored several dozen budding researchers as graduate students and postdocs, I have seen ample evidence that people’s ability to achieve success in social psychology varies. My laboratory has been working on self-regulation and ego depletion for a couple decades. Most of my advisees have been able to produce such effects, though not always on the first try. A few of them have not been able to replicate the basic effect after several tries. These failures are not evenly distributed across the group. Rather, some people simply seem to lack whatever skills and talents are needed. Their failures do not mean that the theory is wrong.

wat ;(

Comment by supposedlyfun on Motive Ambiguity · 2020-12-15T22:11:07.686Z · LW · GW

Re your last claim, can you provide evidence other than the existence of the discourse?  If we're just comparing firsthand experience, mine has been the exact opposite of

For a majority of women that behavior isn't attractive.

Comment by supposedlyfun on Epistemic Warfare · 2020-12-11T05:28:23.827Z · LW · GW

Dark times are ahead.

When it comes to epistemic warfare, they are already here, confidence 85%.  (People on both sides of a given political divide would agree that they are already here, of course for different reasons.  Pro-choice and pro-life; Brexit and Remain; Republican and Democrat.)

when considerations of proportionality and mitigating collateral damage are applied

Do you have a more concrete model for when x units of censorship/lying are appropriate for y utils/hedons/whatever?  Not a trick question, although I doubt any two people could agree on such a model unless highly motivated ("you can't come out of the jury room until you agree").  The question may be important when it comes time to teach an AI how to model our utility function.  

My intuitive model would be "no censorship or lying is ever appropriate for less than n utils, and p units of censorship or lying are never appropriate for any number of utils short of a guaranteed FAI".  And then...a vast grayness in the middle.  n is fairly large; I can't think of any remotely feasible political goals in the U.S. that I'd endorse my representatives lying and censoring in order to accomplish.  

I'd endorse widespread lying and censorship to prevent/avoid/solve a handful of seemingly intractable and also highly dangerous Nash equilibria with irreversible results, like climate change.  We'd need to come up with some Schelling fences first, since you wouldn't want to just trust my judgment (I don't).

Comment by supposedlyfun on Human Genetic Engineering: Increasing Intelligence · 2020-12-06T09:53:41.636Z · LW · GW

This was all a really nice read, very informative, strong favorited. You did lose me in claiming a double exponent for economic growth resulting from increased intelligence. I don't have enough information to say you're right or wrong, but for such an extreme claim, I want to see more of your model. I also want to see more of your model because I just think it would be interesting!

Comment by supposedlyfun on Postmortem on my Comment Challenge · 2020-12-04T22:36:23.241Z · LW · GW

I liked this series of posts for its attempt to identify a community problem, develop a potential solution, and describe one attempt.  I would be interested in more details about the various failure modes because I suspect they are relevant to anyone who sets any kind of productivity/focus goal.

Comment by supposedlyfun on Small Habits Shape Identity: How I became someone who exercises · 2020-11-27T01:28:13.092Z · LW · GW

Great work on the side project you started! I lack the words to even describe it. You don't seem at all like the type of person who lies around reading Reddit all the time, and I should know.


Comment by supposedlyfun on Small Habits Shape Identity: How I became someone who exercises · 2020-11-27T01:25:19.512Z · LW · GW

GOT IT. Great prose. Thank you.

Comment by supposedlyfun on Notes on Endurance · 2020-11-27T00:14:14.361Z · LW · GW

if your difficulty enduring typically shows itself in the face of _________ circumstances, maybe it’s really _________ not endurance that you’re lacking

Iterating this for every conceivable adjective {f(adjective) = virtue} seems like a good way to figure out exactly what endurance is versus some other virtue.  If a given adjective doesn't spit out another virtue, maybe endurance wins by default.

Related, maybe endurance is an absolute value relative to the other virtues you do have.  We don't expect children to endure during a ten-mile hike if they've never hiked more than two miles, maybe because we model them as not having many virtues required for such a hike.  Maybe endurance could be modeled as

endurance = {units of effort required to complete a given task} - {"free" units of effort provided to you by other relevant virtues you already possess, such as patience (this is a long hike but will be worth it) and, idk, filial piety (my parents really want to finish this hike)} 

Comment by supposedlyfun on Nash Score for Voting Techniques · 2020-11-27T00:01:12.351Z · LW · GW

This voting method is, of course, patently absurd. Any one person can throw the whole election by putting "no confidence" above everything else. This could only be appropriate for very small group elections, and even then, it's questionable.

This is because a real democracy doesn't actually seek the voluntary participation of every single member...

This is great explaining throughout, but especially the block quote.  You communicated to me (as in, I had a click/eureka moment) the idea you were trying to teach using very close to the least number of words required.

Comment by supposedlyfun on Small Habits Shape Identity: How I became someone who exercises · 2020-11-26T23:56:59.979Z · LW · GW

You write a lot of good content, so I think it's likely that you are making a good point here.  However, I don't understand it at all and am finding this comment like trying to climb a smooth wall--I can't even get started.  Then again, I am relatively new to this community.  If you have time, I'd be thankful (get it?? I ate so much I don't want to move) if you could rewrite it with your definitions and assumptions made explicit. 

Comment by supposedlyfun on Success Buys Freedom · 2020-11-26T00:10:01.540Z · LW · GW

I appreciate your series and the first examples in this article.

The "Personal Experience" section doesn't seem like evidence for "success buys freedom."  Your examples seem like weak evidence in favor of alkjash's theory.  

You say, "If it weren't for my previous successes," having discussed failing to get into top universities, failing to be accepted by the Air Force, failing to try to get a date, failing [maybe the wrong verb?] to be able to afford a programming degree, failing to not live surrounded by drug dealers on a plywood slab, and failing several times to get hired at a reasonable rate.  (If a nearby you got into the Air Force, it would be very hard for zir to also run a startup from a cupboard in China.)  The only thing your description indicates you [conventionally] succeeded at before your startup was teaching yourself to code (which is a success, for sure).

Maybe it's a matter of framing?

Maybe as another article in the series, write the story of a nearby counterfactual you who went to the Air Force Academy and did five years as a lieutenant, then captain, in IT security (so server programming, similar to the low-paying job you had), then got out.  Of the skills and psychological traits that you have needed to be successful in the exact same startup, which ones would have been harmed by the Air Force?  Which ones would have been augmented?  (My prediction is that lbh jbhyq unir rdhvinyrag be uvture fxvyyf nsgre gur Nve Sbepr ohg gung fbzr bs lbhe "znirevpx" crefbanyvgl genvgf jbhyq unir orra erqhprq, ohg abg rabhtu gb xrrc lbh sebz fhpprrqvat va gur fgneghc svryq.)  

Comment by supposedlyfun on Pain is the unit of Effort · 2020-11-25T23:39:16.313Z · LW · GW

"We don't know how many counterfactual lsusr clones died or were permanently disabled after pushing too hard."

This is not only a clever and concise way of putting this thought, but putting it concisely and cleverly really helped to crystallize it in my brain, whereas before, it was amorphous.

Comment by supposedlyfun on AGI Predictions · 2020-11-22T10:11:34.788Z · LW · GW

I really appreciate the effort that went into collecting all of these questions, framing them clearly, and coding the clickable predictions.

Comment by supposedlyfun on Final Babble Challenge (for now): 100 ways to light a candle · 2020-11-21T00:15:31.072Z · LW · GW

XD even though I haven't owned a car that actually came with a cigarette lighter in about 12 years

Comment by supposedlyfun on Final Babble Challenge (for now): 100 ways to light a candle · 2020-11-13T06:02:50.616Z · LW · GW

You're my hero.

Comment by supposedlyfun on Final Babble Challenge (for now): 100 ways to light a candle · 2020-11-13T05:49:51.343Z · LW · GW

I'm using the markdown editor, so maybe my attempt at redacting will take this time.



Build Boy Scout-approved fire and stick candle in it

Flint and striker

Take it to California and use an existing wildfire

Put Crysis on Ultra, run the benchmark, and put the wick on my video card

Parabolic mirror

Find a smoker and hold it up to their cigarette

Buy a sketchy laptop, wait for the battery to catch on fire, use that

Use positive thinking

Wait for quantum uncertainty to lead to a state where it’s lit

Sneak onto an aircraft carrier and hold it up to the exhaust of a jet taking off

While on aircraft carrier, detonate a bomb and use that flame

While on aircraft carrier, borrow an M9 from a sailor and use propellant explosion

Make a potato battery using a giant potato, run it through highly resistant wire that gets really hot

Move a magnet near a wire so it gets really hot

Drill a hole in my car engine to reach combustion chamber

Hook spark plug up to car battery

Find a Tesla that just crashed and is on fire

Travel back in time to a Spanish Inquisition auto-da-fe and use that fire

Fly to the sun in a rocket and get close enough to light the candle

Use the rocket’s exhaust to light the candle

Use a toy rocket engine to light the candle and also launch a toy rocket

Call my genius engineer friend and use the craziest idea he comes up with

Learn some chemistry and come up with a really baroque way to make purple fire

Go back in time, learn how Greek Fire was made, use that

Go back in time, use any of these methods to light the candle, and presto--the candle is lit in the present

Shoot incendiary bullet at the candle

Throw a white phosphorous grenade at the candle

Light my dog’s fart on fire in the direction of the candle

Stationary bike hooked to a generator

Striker on gas stove

Attend dyno test of F1 engine; use red-hot headers to light candle

Go to Bonneville Salt Flats during rocket-car test, use rocket-car exhaust

World’s most badass capacitor generates spark

Arc welder

Oxy welder

Plasma torch

Magnifying lens

Pray to Prometheus

Turbocharge a record player, replace CD with flint, replace needle with striker

Fabricate evidence of WMD in dictatorship country, send in the military, use a burning building or tank to light candle

Tweet at Elon Musk that the candle needs to be lit in order to avoid AI X-risk

Tweet at Donald Trump that it’s sad that the candle isn’t lit

Stake out the Olympic Torch route in Japan

Put match on end of barbell and strike surface near Lasha Talakhadze during world-record snatch attempt

Order a kid’s chemistry set and start mixing until I find an exothermic reaction by accident

Go to an import tuner illegal street race, find the person with the biggest exhaust flame

Go to an Italian restaurant with the little red candle covers, remove the cover to access the flame within

At same Italian restaurant, throw candle into their pizza oven

Crash neighbor’s weekend barbecue and use grill flame

Funeral home incinerator

Get matches from an old MRE

Put match in sole of my track flats, sprint past striker and candle next to track

Put candle in 100% heat-insulating box, then kick the box until the kinetic energy converted into heat gets hot enough to light candle

Sneak into transformer substation and start swinging a bat around until something sparks

Borrow flame from hobo trashcan fire

Make hologram of signal fire; go to next station where they’ve lit an actual signal fire

Conceive a desire that the candle be lit; future AI has this as part of its utility function, sends a time traveler back to light the candle

Put striker surface near candle, teach my cat to scratch the striker rapidly after giving him cool metal claws

Climb natural gas well plume and use that fire

Visit set of anti-fracking documentary where someone has lit their contaminated drinking water on fire, and use that fire

Harness power of fracking-induced earthquakes in Oklahoma

Bathysphere to active underwater magma eruption

Hike to active magma eruption in Hawai’i

Overcharge the flash on my camera

EDM festival subwoofer vibrations

Road flare “borrowed” from police officer

Throw paper in a hot oven

Follow a smoker around to gas stations until they light up and something catches on fire. Administer first aid and call 911

Take a hammer to my cell phone until it starts sparking

Toss lithium in water

Don’t clean out lint trap in dryer; keep running dryer; use flame from burning dryer

Old-timey chemical camera flash, on steroids

Biceps curl machine that charges a capacitor

Get a PhD in oxidative chemistry and pick the absolute simplest possible reaction that will work

Pilot light

Go to church just before ceremony starts but after altar kids have lit the candles already

Visit friends in the country on burn day

Cigarette lighter in car

Purple flame is already burning outside Gozer the Gozerian’s lair

Disconnect Pornhub server from cooling system until it catches on fire

Take tour of coal-burning power plant, sneak into area with fire

Get a job as a short-order cook, use their grill

Attend Texas A&M; use their bonfire

Strip club in city of previous residence had fire sconces outside, wouldn’t even have to go in

Dumpster fire caused by 2020

Genetically engineer myself to be able to breathe fire like the dragons in Reign of Fire

Use hot coals at a firewalking seminar

Crash a holiday party; use their fireplace

Use police scanner to listen for location of fire caused on Thanksgiving by someone putting a turkey in a deep fryer without thawing it first; use existing fire

Go to Scientology center and offer to use E-Meter if they’ll light candle for me

Invading aliens’ blasters

Airstrike on candle

Orbital kinetic bombardment using rods made of striker; target flint near candle

Douse wick in oil with low flame point, then touch wick to fancy steak as it comes out of steakhouse’s 500-degree finishing oven

Breathe really, really, really hot breath on candle

Convince PhD optics student to give me access to the good lasers

Tour whiskey distillery that makes its own barrels, put candle in front of the fire they use to get the barrels ready

Snap really, really hard

Comment by supposedlyfun on Babble Challenge: 50 thoughts on stable, cooperative institutions · 2020-11-06T03:57:39.286Z · LW · GW

Thank you.

Comment by supposedlyfun on Babble Challenge: 50 thoughts on stable, cooperative institutions · 2020-11-05T23:12:59.104Z · LW · GW


  1. The desire to outcompete the USSR in social outcomes was driven by nation-level existential concerns about what would happen if communism had better social outcomes. (see also Marshall Plan)
  2. Anti-expert sentiment is hardly new in the USA.  (cf The Paranoid Style in American Politics). Facebook makes it far easier for anti-expert people to coordinate.
  3. Increased anti-expert coordination places evolutionesque pressure on bad arguments, leading to better versions of those arguments.
  4. Bothsidesism in various sectors weakens immuno-epistemic protections against stupid ideas
  5. Increased sorting and extremism between Rs and Ds makes people less willing to coordinate on public goods.  (“Republicans are more likely to [do something akin to Prisoner’s Dilemma defecting], so why bother trying to deal?”)
  6. Google, Amazon, and grocery stores are (in this sense) a different category of thing from public goods.  There is a profit motive pushing them forward in a laissez-faire capitalist system.
  7. The memetic success of capitalism makes it more socially acceptable to critique the USPS by saying that it doesn’t turn a profit.  (The military doesn’t turn a profit.  Or does it? Has anyone studied counterfactual models of gas prices where we didn’t go into Iraq?  Was $2.4MMM a good deal?)
  8. Two-forward one-back advances in the rights of historically marginalized groups (women, Black people, Latinx, Asianx) has made historically relatively better-off groups feel like progress is a zero-sum game, making them less willing to coordinate
  9. The bigger society gets, the more our savannah monkey brains are Just Not Cut Out for A World This Complicated
  10. Universities create a motive for scientific progress, but only insofar as the prestige leads to more students being willing to attend.  State-level spending on higher ed in my alma mater’s state goes down every year.
  11. Incentive: do research that is “cool” (more students) or likely to lead to financial support from private sector (new Koch Hall for applied economics or whatever).
  12. Some people are going to do what they want because they’re wired that way, and some of them will end up making a lot of money (Elon Musk).  This will carry pet projects forward.
  13. Museums, orchestras, operas, some theaters owe their start to the Kochs and Musks and Gateses of 100+ years ago, who took all their surplus money and put it into stuff they cared about (High art and not Low art).
  14. Presumably, being on a nonprofit entity’s board has always been a giant cluster headache due to the same infighting/social signaling/etc that people complain about now; maybe previous Americans were more willing to play that game in exchange for status because status was more important to them
  15. NYT examples are interesting.  I think they dovetail with the “objective journalism” trend which is a flash in the pan relative to the history of printed news. (compare London’s various dailies (at least as of 20 years ago) with people identifying themselves politically in part by stating which paper they read)
  16. Strongly suspect that objective journalism was an outgrowth of the Cold War, somehow. Maybe people looked at Pravda and Nazi newspapers and realized that part of winning was not having [handwaves] whatever Pravda is.
  17. Yes, US bridges are crumbling, etc. However, when a decent-sized bridge falls down, it gets rebuilt (right?).  So the problem is less coordination and more people being stupid about economics and risk.  Or maybe that stupid leads to failure to coordinate in terms of taxes.
  18. I was reading an LW post recently about how hard it is to scale up corporate culture from, say, a startup--the NYT US news desk is probably not THAT large.
  19. Not counting “fake news,” i.e. not considering people with judgments proudly based on irrationality and ignorance, is the NYT actually perceived less well than in years past? The only big hit I remember them taking in my adult life is cheerleading the Iraq war.
  20. Obviously, carving out those people is missing a major part of the story.  Does the conservative media environment (Fox News, Breitbart, Rush) create the “common knowledge” that allows coordination among people who want to see the country coordinate less? Cf Ben Pace’s “The Costly Coordination Mechanism of Common Knowledge”
  21. I want political scientists to break their backs publishing articles and studies about what happened in the 2020 election, and why everyone voted for who they voted for.
  22. Covid is another thing. Suppose Trump had said, “Do what Fauci says, the guy knows what’s up,” what percent of anti-maskers would have been pro-mask?  In other words, how much no-coordinating is driven by politics?
  23. What is it about masks in particular that triggered the “I refuse to coordinate unless and until it kills my family” meme?  At first, it was supposedly about the government mandate (rather than the requested act), but that has shifted to a focus on the act itself.
  24. Why didn’t we do contact tracing more, and why didn’t my mayor bang on and on about how important it was and why? I had to look really hard to find any info at all about contact tracing nationwide or in my state.  It didn’t need to be voted on, just have money thrown at it.
  25. Why didn’t people answer the phone when the contact tracers called?  Coordination problem?  Do local, state, and national health entities’ failures count as institutional failures if they are caused by people’s reaction to them, independent of what the entities actually did?
  26. Why were people okay with “herd immunity” as a strategy when sensible estimates indicated that would kill over a million people?  Is there an empathy gap or large number problem with coordinating on this issue?
  27. If Those People think the CDC is so crap, why don’t they want to reform it?  (Does anti-government sentiment just reflect the view that some coordination/defection problems are absolutely intractable? Is this defensible?)
  28. Harvard. My sense is not that it has lost face in general, just that people are arguing that its admissions criteria have been Goodharted. (not so much the argument that the criteria were wrong when they were generated…)
  29. Where is Moloch in all this?  Moloch, who generated examples of unions becoming corrupt, which were then used to justify suppressing unions rather than fixing them?
  30. My middle of the country view on California in particular is that the claims of widespread governmental failures there are driven by people who think the government is too liberal.
  31. That said, the gun laws seem to have been written by people who don’t know very much about guns.
  32. Surely the massive flows of dark money generated by Citizens United have made it harder for small groups to coordinate and for new institutions to develop--the signal:noise ratio surely has gotten smaller. (Data on this?)
  33. The 0-3 model seems right.  You just need gatekeepers and line-holders to make sure the average norm-adherence stays high enough.
  34. Note: NYT has always been held to some extent by the Ochs/Sulzberger family.  (Parallel: Fox News and Murdoch empire.)   Hypothesis: strong institutions under capitalism require patrons.
  35. Elon Musk is a patron, too.  Was there really market pressure to come up with rockets that could land on a platform floating in the ocean?
  36. This is connected to the LW (and elsewhere) concept of slack. Someone with resources can help NYT preserve its culture by partially shielding NYT from the market demands that there be no slack in its business model.
  37. I wonder: Are people more interested in money and less interested in status now?  Or maybe status is something you can get more easily now in places other than your job?  Or maybe job status means less when you feel like your job is the public-capitalist arena and you suspect the system is rotten or at least a problem?
  38. Loss of regard for the political parties, if it exists, must be a function of increased sorting and polarization.  (ie more people hate the parties because they oppose them strongly, versus people who lean in the party’s direction but are frustrated by the apparatus)
  39. It must be very hard to count the number of subculture-communities out there.  How do you even find them to begin a study?
  40. Why are the scholars of community so obsessed with bowling leagues?  (Why did so many people participate in them?  Or is this a bad just-so story based on the title of Bowling Alone?)
  41. A political party/presidential government takes a sledgehammer to some government agency, which leads the brightest people there to say “fuck this” and go to the private sector, which leads to the agency becoming less functional, which leads to…
  42. More people know about Dunning Kruger, coordination problems, etc now than before (I think…) Is knowledge of all humanity’s cognitive biases an infohazard relative to one’s ability to get motivated to work for the common good, to coordinate, to support a sometimes-frustrating but mostly valuable institution of which one is a member?
  43. I feel in my bones that anti-expertise is the X Factor here, but I can’t justify that empirically, and obviously anti-expertise has many other causes, so proving it was the X Factor would only get us closer to an answer.
  44. Maybe I only feel that way because I am an expert at some things.
  45. Functional institutions are a levee built against the eldritch horrors.
  46. Do functional institutions start out with an eldritch horror as a target, or do they struggle forward blindly and only sometimes find a target?
  47. [Just hit a wall. Hit a wall at #47 last time, too!] Ultra babble mode engaged: What role does mental health play in all this?  If everyone has crippling anxiety because they’re afraid of being evicted, is there a ceiling on how complex of an institution they can create?
  48. What drives people to become patrons? Guilt at earning so much money?  Maybe: Jeff Bezos’ charitable giving isn’t aimed at e-commerce as far as I know.
  49. A large, long-standing functional institution that needs to be insulated from the profit motive: can it even exist without some kind of patron?  Or will it inevitably be outcompeted otherwise?
  50. Obviously some people like to tear down functional institutions. What is wrong with them?:
Comment by supposedlyfun on Multiple Worlds, One Universal Wave Function · 2020-11-05T01:13:53.321Z · LW · GW

This added to my layperson's understanding of both MWI and quantum mechanics more generally.  

Immediately under the subhead "The Apparent Collapse of The Wave Function," what is a-sub-i in the initial state?

Comment by supposedlyfun on ChristianKl's Shortform · 2020-11-04T23:47:42.191Z · LW · GW

I endorse gjm's final comment 100%.  (I wrote a much longer response but eventually decided that it was just repeating what ze said.)

Comment by supposedlyfun on Notes on Honesty · 2020-11-03T15:34:17.309Z · LW · GW

Totally fair point. I agree that not all fictional heroes' possibly justifiable lies are subject to my "author writes the story to protect reader perception of hero" concept.  Maybe narrow my comment to "When using a fictional character's lies as a model for when lying is acceptable, one should be alert for situations where the author has built the fictional world in such a way that Lie X is maybe justifiable in the fictional world but would not be in nearby counterfactual worlds."

Comment by supposedlyfun on What is the right phrase for "theoretical evidence"? · 2020-11-02T02:09:40.985Z · LW · GW

Okay, thank you for engaging.  Those answers weren't clear to me from the parent piece.

Maybe I reacted strongly because my current prior on my own intuitions is something like "Your intuition is just savannah-monkey-brain cognitive shortcuts and biases layered over your weird life experiences".  One thing I've been thinking about lately is how often that prior is actually justified versus how often it's merely a useful heuristic (or a shortcut/bias? ha!) to remind me to shut up and Google/multiply.  

Comment by supposedlyfun on What is the right phrase for "theoretical evidence"? · 2020-11-02T01:37:23.600Z · LW · GW

Imagine that you are working on a product. A/B tests are showing that option A is better, but your instincts, based on your understanding of how the gears turn, suggest that B is better.

Imagining it now. "are showing" makes it sound like your A/B tests are still underway, in which case wait for the study to end (presumably you designed a good study with enough power that the end results would give you a useful answer on A vs. B).  But if the tests show A > B, why would you hold on to your B > A prior?  Or if you think the tests are only 50% conclusive, why would you not at least update the certainty or strength of your B > A prior?  

I think this is why Idan said, "Or, since they explicitly go against the empirical evidence, how about we just call it 'stubbornness'?"

Comment by supposedlyfun on Babble challenge: 50 consequences of intelligent ant colonies · 2020-10-30T02:27:33.709Z · LW · GW

Omega comes to you and says, "I just flipped a coin, and if it had come up heads and you had studied eusocial organisms in grad school..."

Comment by supposedlyfun on Babble challenge: 50 consequences of intelligent ant colonies · 2020-10-30T01:42:37.844Z · LW · GW


  1. PETA suddenly gains a larger following.
  2. AI researchers develop human-brain models based on ant signaling, which maybe is less complex than the human brain?
  3. Public intellectuals debate whether an ant colony is smarter than a human based on emphasizing different aspects of how the intelligence emerges.
  4. Philosophy departments at universities go from borderline irrelevant to somewhat less irrelevant; however, none of the thorny philosophy questions about intelligence/personhood become any easier.
  5. Charles Murray calculates that the average ant colony is smarter than the average Black person.
  6. The debate over whether Murray’s calculations are accurate generates, within five years, more published articles than The Bell Curve has generated to date.
  7. Rule 34 has a number of implications here.  First is that there will be CGI porn of someone having sex with an ant colony.
  8. The next step is that the teledildonics industry develops increasingly complex ways of allowing a human to have sex with an ant colony, witht the first methods totally stupid, but improving over time.
  9. The next step is that a human falls in love with an ant colony. This person writes a personal essay that The New Yorker publishes.
  10. Colonies can make a lot of money doing sex work.
  11. If language learning is on average equally available to human-level intelligence, ie there’s nothing unique about human intelligence vis a vis language learning (I don’t know enough), ant colonies learn to communicate with humans through (spelling out words with ant bodies? pheromones?
  12. A ten-year-old-equivalent ant colony works with a {handwave to insert computer genius} to come up with a relatively easy way for ant colonies to type.
  13. Once an ant colony can type, it is trivially easy for the colony to coordinate with other colonies.
  14. Ant colonies also start doing AI safety research.
  15. I’m very worried about conservative Christians in this scenario.  God gave you dominion over all the creatures, but an ant colony is as smart as you.  Do you throw out your ant poison? Or buy more and start murdering? (You wouldn’t consider it murder. Or would you?)
  16. The Catholic Church announces that ant colonies have souls.  Many consequences arise from this.
  17. The Southern Baptist Convention announces that ant colonies do NOT have souls.  Many consequences arise from this.
  18. Ants are indifferent except for instrumentalist reasons, because their brain-equivalent doesn’t have the specific part of the human brain responsible for the perception of a “higher power”.
  19. People who don’t think the colonies are human-equivalent say “How do we even know what it’s like for an ant colony? We don’t know if there’s something going on under the hood, maybe the intelligence is just emergent phenomena” and I say “Hah, you’ve fallen for a Socratic trap. How do you know there’s something going on under my hood?”  The ant colony situation does not assist philosophers with this question.  My interlocutor determines that I’m a philosophical zombie and tries to murder me.
  20. The colonies develop ant political parties; or
  21. The colonies’ intelligence is such a way that they are all unified on whatever question they consider; or
  22. The colonies are not unified but have an anty knack for solving coordination problems and out-coordinate humans; or
  23. The colonies are not unified, have anty knack, but end up supporting existing political parties because it’s the best coordination case for them at the time
  24. Ant queens have conferences where they get together and communicate by pheromone, much easier than doing it using ICQ
  25. The Great American Ant Novel is published and gets a starred review in Kirkus.
  26. There is massive variation in how various governments treat the issue.  Strongmen outgroup them even harder than they outgroup ethnic minorities.
  27. More liberalish states/governments give sentient colonies the franchise.  However, they impose intelligence tests that so happen to mean there aren’t enough voting colonies to affect human politics--at first
  28. Ants quickly realize that in order to survive, they need a deterrent military force.  They are small, and slow, but they are legion.  Russia supplies computer-operated drone and remote weapon systems to Ukrainian colonies, which are willing to lob a rocket at western Ukraine sometimes in exchange for top cover from Putin.
  29. There are anti-ant pogroms in parts of the U.S.A.  I’m not confident enough to predict where they are.
  30. Colonies develop the intellectual symptoms of severe anxiety due to feeling under constant threat.
  31. A colony writes a sci-fi novel about a human suddenly waking up one day and being as intelligent as an ant colony.
  32. Scarlett Johansson starts in a movie about a woman-looking ant colony that picks up men on the side of the road in Great Britain and then eats them.
  33. It becomes taboo to ask someone online if they are a colony or not.
  34. Corollary: sometimes you ask for nudes, and you get...a picture of a writhing mass of ants.
  35. Colony hipsters reject online communication with other ants and instead hitch rides with sympathetic humans to go hang out with a nearby or far-flung colony, like an exchange program.
  36. Every ant colony is surrounded by webcams broadcasting to Facebook Live so they can surveill people who may be about to poison them.
  37. Inconvenient colonies agree to be relocated in exchange for the reasonable value of the property they inhabit.
  38. If the president is a Democrat, there is created a cabinet-level post for anty stuff.
  39. The next Republican president does away with it (cf. Carter’s solar panels on the White House roof).
  40. Humanist political parties arise, but it’s not humanist like secular humanist but rather like human-supremacist, and that use of “humanist” outcompetes the “secular humanist” use into oblivion. At least 20% of the American population would support wiping out all ants capable of forming sentient colonies.
  41. Some states make it murder to destroy a sentient colony, but there’s a lot of jury nullification, and each side hires expert witnesses to say that the victim was or wasn’t intelligent enough to be sentient.
  42. A colony manages to kill an inept human assailant and is tried for murder.  Before the trial can proceed, someone sets it on fire and destroys it.  The prosecutor declines to bring charges.
  43. One colony develops a standup comedy routine that absolutely slays at rationalist conventions, primarily riffs involving the word “signaling”
  44. Colonies are difficult to do research on because they almost all determine that teaching humans how they think is too dangerous.  Humans need a big skull bottle you can make holes in to great effect if you have a drone rifle.  Anty brains are much harder to destroy.
  45. One colony tells the literal truth and participates in all studies in good faith, but the information on it gets lost in the noise.
  46. Dogs, on average, are found to be far more loyal to colony owners than human owners.
  47. Eventually, things get weird enough that the anty teledildonic industry comes up with a way to turn human-generated sex stuff into ant sensations.
  48. Colonies outperform humans at chess at a rate much higher than projected.  An emergent property of colony brains responsible for strategic thinking is suspected.
  49. Colonies coordinate on their own to wipe out fire ants--turns out nobody likes those guys.
  50. The U.S. elects its first colony president about 200 years after the colonies become intelligent. :::
Comment by supposedlyfun on Notes on Honesty · 2020-10-28T12:12:39.173Z · LW · GW

I'm not per se disagreeing with your sentiment about Odysseus, but it would be helpful if you could model it more explicitly than "lies like he tells are okay". I can see why SPOILERS not telling Polyphemus your real name is potentially acceptable dishonesty, but Kant would disagree, I guess.

The problem with setting a fictional hero as your standard for any component of morality is that the omniscient narrator will often have your back in edge cases. The best example to me is Ender's Game, SPOILERS in which he kills two people without intending to, but also intends to attack them in potentially deadly ways and does so; yet in the next chapter, the literal adults in the literal next room are there to tell the audience, "If Ender hadn't killed Bonzo, Bonzo would have killed him, and we then wouldn't win the war, so it's fine." To me, their belief is maybe true but definitely not justified on the evidence. Bullies don't kill people all the time. Card is heavily motivated to protect Ender's morality for various reasons and builds the story around him for that particular goal.

Comment by supposedlyfun on AGI safety from first principles: Alignment · 2020-10-19T02:11:14.176Z · LW · GW

"A robot arm with a purposely disabled gripper found a way to hit the box in a way that would force the gripper open"

The list of AI misbehavior in the blog you link to should be in the Sequences. I had no idea that we were programming things with this level of AI-ish sophistication in the 90s, much less that they were going so spectacularly wrong sometimes. Hello, Skynet.

This is a great series that is written in reasonably plain English with only necessary jargon, and I thank and salute you.

Comment by supposedlyfun on Brainstorming positive visions of AI · 2020-10-08T01:20:41.626Z · LW · GW

Using AI/ML to fix some edge problems with government would be so neat.  My optimistic scenario is that AI/ML gets common enough that nobody thinks it's weird, and then the state legislature of, say, California says, "How about we let an AI try to draw our Congressional districts?" and a nonpartisan committee comes up with a reasonably fair utility function, or maybe there's even a statewide ranked-choice vote on ten different values for the AI to consider, and the AI gives back a map that everyone agrees is not perfect for their side but pretty close to fair.  

It would almost be like seeing Rawls' veil of ignorance play out in real life, if everyone is comfortable enough with AI in general to not reject whatever it comes up with just because they aren't winning as much as they'd like to.  (It's possible that voters would be on board for this particular issue.  Cyborg government!  We're not that far off.)

Comment by supposedlyfun on Postmortem to Petrov Day, 2020 · 2020-10-08T01:01:45.088Z · LW · GW

I genuinely, sincerely appreciate that you took the time to make this all explicit, and I think you assumed a more-than-reasonable amount of good faith on my part given how lathered up I was and how hard it is to read tone on the Internet.

I think the space we are talking across is "without checking if this will make my beliefs more accurate."  Accuracy entails "what do I think is true" but also "how confident am I that it's true".  Persuasion entails that plus "will this persuasion strategy actually make their beliefs more accurate".  In hindsight, I should have communicated why I thought what I proposed would make people's beliefs about humanity more accurate.

However, the response to my comments made me less confident that the intervention would be effective at making those beliefs more accurate.  Plus, given the context, you had little reason to assume that my truth+confidence calculation was well-calibrated.    

There's also the question of whether the expected value of button-pressing exceeds the expected life-worsening, and how confident a potential button-presser is in their answer and the magnitude of the exceeding.  I do think that's a fair challenge to your final thought.

Thanks again.

Comment by supposedlyfun on Postmortem to Petrov Day, 2020 · 2020-10-04T20:18:21.476Z · LW · GW

I'm missing something fundamental here given that this is your view as organizer, so I'm going to go back to lurking for a while.

Comment by supposedlyfun on Postmortem to Petrov Day, 2020 · 2020-10-04T19:19:51.458Z · LW · GW

It sounds like we draw the line differently on the continuum between persuasion and brain hacking.  I'd like to hear more about why you think some or all parts of this are is  hacking so I can calibrate my "I'm probably not a sociopath" prior.   

Or maybe we are diverging on what things one can legitimately claim are the purposes of Petrov Day.   If the purpose of an in-person celebration is "Provide a space for community members to get together, break bread, and contemplate, and the button's raises the tension high enough that it feels like something is at stake but there's no serious risk that the button will be pressed," then I'm wrong, I'm being cruel, and some other forum is more appropriate for my Ruiner to raise zir concern.  But I don't get the sense that there's a community consensus on this.  

On the contrary, the fact that all attendees are supposed to leave in silence if the button is pressed suggests that the risk is meant to be real and to have consequences.

Comment by supposedlyfun on Postmortem to Petrov Day, 2020 · 2020-10-04T19:09:29.502Z · LW · GW

I understand and sympathize with the desire to know that people around you can hold that power without abusing it. I would also like to know that.  

But it's only ever a belief about the average behavior of the people in the community.  It should update when new information becomes available.  The button is a test of your belief.  Each decision made by each person to press or not to press the button is information that should feed your model of how probable it is that people can hold the power without abusing it.  A bunch of people staring at a red button with candles lit nearby can be an iterated Prisoner's Dilemma depending on the participants' utility functions.

If I decide not to Petrov-Ruin out of a desire to protect your belief that people can hold the power without abusing it, and I make that change because I care about you and your suffering as a fellow human and think your life will be much worse if my actions demolish that belief, then a successful Petrov Day is at risk of becoming another example of Goodhart's Law. 

I think, anyway. Sometimes my prose comes off as aggressive when I'm just trying to engage with thoughtful people.  I swear, on SlateStarCodex's review of Surfing Uncertainty, that I'm typing in good faith and could have my mind changed on these issues.

Comment by supposedlyfun on Postmortem to Petrov Day, 2020 · 2020-10-04T15:50:58.476Z · LW · GW

So in some ways this has been good for making the challenge level clear – this is a real coordination problem, that we can fail, and isn't overdetermined in either direction.

I think every single part of this exercise, and all of the responses to it, have been extremely informative.  

  1. The priors of hundreds of thoughtful people have been updated substantially.  
  2. Others learned that they cared about something (either the site or the Petrov Day button) more than they thought they did, and, being thoughtful, learned something about themselves by thinking about why they cared so much.  
  3. At least one person (chris) received an object lesson in security that I doubt ze will forget.  
  4. The rest of us had the opportunity to learn that lesson as well, although lessons not learned in blood are soon forgotten, and for a given individual, "losing the front page for one day" is far less blood than what it cost chris.   
  5. I updated my priors about myself.  If you'd asked me in 2019, after the successful no-press, whether I would have pressed it with codes, I probably would have said no.  However, seeing the amount of thoughtfulness generated by what went down in 2020, if I get codes in 2021, I will give serious thought to pressing the button because doing so:
    1. will, no matter the reason, instill even further caution and security-paranoia in AI researchers, which is stressful for them but beneficial for humanity.  
    2. causes the community to learn something about itself which can't be predicted in advance but is almost certainly accurate.
    3. shows that if it's something I would consider, it's something certain types of AI would consider, too, if we're not careful.  AI risk researchers are aware of that problem, but pressing the button gives one the opportunity to make them feel it in their guts.

I wonder if there is instrumental value in pre-committing to being a Petrov Day ruiner unless certain types of conditions are met.  "I am coming to the San Francisco Petrov Day 2022 celebration and will press the button five minutes before the end of the ceremony unless I first come to understand how we can solve the problem of whether an autonomous vehicle should swerve, and thereby kill the 18-year-old pregnant valedictorian, or not swerve, and thereby kill the 84-year-old semi-retired Medicin Sans Frontieres who hand-carves toys for kids at the orphanage."  

Comment by supposedlyfun on Postmortem to Petrov Day, 2020 · 2020-10-04T15:18:19.896Z · LW · GW

The Web is full of jokey "red buttons"

This point is important.  LW is the type of website I would expect to do some quirky applet that gives an object lesson in iterated game theory, or something, and as a person still trying to learn on LW's key concepts, I would be inclined to try to play.  Thus, the fact that chris pressed the button last year and entered fake codes is neutral evidence to me.  It's what I would have done if I were playing what I thought was an instructive game.

By contrast, Petrov believed (I assume correctly or at least justifiably) that all of the "buttons" available to him were real buttons that would cause the Soviets to launch at least one nuke and probably hundreds.

I don't have a good solution for the problem of "Convince a relative newcomer to the site, solely via the button, that the button is real and will do what it says it does".  For me, as such a newcomer, this is because HPMOR and SSC (my entrees into the community) have such a screwball sense of humor.  

Thus, the more times you try to say, "For real, clicking this button will actually cause the front page to go down for 24 hours," or "Lots of people are going to be annoyed by this," or "Clicking this button will tend to show the community that its xrisk calculations are not unreasonably high," or even "Petrov would be ashamed of you (epistemic status: confident)," the more it looks like a game, because most of the Internet is so seamless that multiple caveats come off as satire.

Maybe you could add links to the postmortems for Petrov Day 2019 and 2020 in the button?

Comment by supposedlyfun on MikkW's Shortform · 2020-10-03T14:57:14.421Z · LW · GW

A cute senior in my high school Physics class asked me to tutor her after school because she was having a hard time.  I can't overstate the ways in which this improved me as a young-geek-person, and I think she got better at doing physics, too.  Your proposal would tend to create more opportunities like that, I think, for cross-learning among students who are primarily book-intelligent and those who may be more social-intelligent.

Comment by supposedlyfun on supposedlyfun's Shortform · 2020-10-03T14:44:36.064Z · LW · GW

Does anyone have some good primary/canonical/especially insightful sources on the question of "Once we make a superintelligent AI, how do we get people to do what it says?"

I'm trying to hold the question to the question posed, rather than get into the weeds on "how would we know the AI's solutions were good" and "how do we know it's benign" and "evil AI in a box" as I know where to look for that information.  

So assume (if you will) all other problems with AI are solved and that the AI's solutions are perfect except that they are totally opaque.  "To fix global warming, inject 5.024 mol of boron into the ionosphere at the following GPS coordinates via a clone of Buzz Aldrin in a dirigible...".  And then maybe global warming would be solved, but Exxon's PR team spends $30 million on a campaign to convince people it was actually because we all used fewer plastic straws, because Exxon's baby AI is telling them that the superintelligence is about tell us to dismantle Exxon and execute its board of directors by burning at the stake.

Or give me some key words to google.  

Comment by supposedlyfun on supposedlyfun's Shortform · 2020-09-27T21:16:09.303Z · LW · GW

I figure that at some point in the next ~300 years, computers will become powerful enough to do the necessary math/modeling to figure this out based on advances in understanding genetics.

Comment by supposedlyfun on supposedlyfun's Shortform · 2020-09-26T20:56:30.386Z · LW · GW

I'm grateful for MIRI etc and their work on what is probably as world-endy as nuclear war was (and look at all the intellectual work that went into THAT).

The thing that's been eating me lately, almost certainly mainly triggered by the political situation in the U.S., is how to manage the transition from 2020 to what I suspect is the only way forward for the species--genetic editing to reduce or eliminate the genetically determined cognitive biases we inherited from the savannah.  My objectives for the transition would be

  1. Minimize death
  2. Minimize physical suffering
  3. Minimize mental/emotional suffering
  4. Maximize critical thinking
  5. Maximize sharing of economic resources

I'm extra concerned about tribalism/outgrouping and have been thinking a lot about the lunch-counter protestors in the U.S. practice/role-playing the inevitable taunts, slurs, and mild or worse physical violence they would receive at a sit-in, knowing that if they were anything less than absolute model minorities, their entire movement could be written off overnight. 

I'm only just starting to look into what research there might already be on such a broad topic, so if you see this, and you have literally any starting points whatsoever (beyond what's on this site's wiki and SlateStarCodex), say something.

Comment by supposedlyfun on Human Biases that Obscure AI Progress · 2020-09-25T18:43:31.879Z · LW · GW

This helped me to distill a lot of things that I already "knew" on a semi-verbal/intuitive level but had not put into words.  I expect it will help me discuss these concepts with people unfamiliar with the ins and outs of AI. Thank you for doing the mental work of synthesizing and then translating it into plain English with only the necessary jargon.  

Comment by supposedlyfun on Book Review: Working With Contracts · 2020-09-19T20:57:00.845Z · LW · GW

In most US jurisdictions, interpretation of a contract is a question of law (read: decided by the judge). However, some subparts of that inquiry are questions of fact (read: decided by the jury). By far the most common is to argue that some key term of the contract is ambiguous--that if we read the document as a whole, that key term has two mutually inconsistent meanings.

This opens the door to evidence about the parties' intent, which is a jury question. We lawyers are nothing if not sensitive to language, so we're very good at coming up with possible ambiguities. A judge confronted with a bona fide ambiguity, or even a borderline one, ought to let the case go to trial for the jury to resolve the intent question. (The "ought" comes from common law about when summary judgment is inappropriate.)

Another good one is in contracts for specialized goods and services with esoteric terms and requirements. You identify some inadequately defined word no jury is familiar with and hire an expert witness to opine that the word's meaning in the particular commercial context is x. Your opponent hires an expert to say it's y. Again, the appellate courts tell the trial courts that in this situation, a jury should resolve the "battle of the experts." P.S. you can find an expert to say just about anything for $5,000, $10,000 on the high side.

As you can imagine, it's much easier to litigate the interpretation of a contract than to draft one that can't really be litigated. Transactional attorneys tend to give litigators a hard time about this, and litigators give transactional attorneys a hard time about being terrified of court rooms.

Comment by supposedlyfun on Book Review: Working With Contracts · 2020-09-18T02:04:08.610Z · LW · GW

I'm a lawyer and have programmed very simple things in very simple programming languages, and I endorse both your Contracts 101 description, which is very easy to understand and also accurate, and your parallels with computer programming. Your focus on risk allocation is spot on. I promise to do something for you, you promise to pay me, and the other 50 pages are just risk allocation and rules on how to parse the risk allocation.

Re patches on top of patches: There is some truth here. For context, there are two reasons that contracts I work on end up looking like this. The first is that the patches are carve-outs for some very particular contingency the client is worried about, and because they're not lawyers, they don't understand the non-patchy things that address their concern, so they want something specific addressing their concern. The other side tends to agree because they, too, know that the patchy thing is superfluous. It's sort of like if you were a programmer who was really bad at math, and you wanted to print "hello world" if x was between 5 and 10 but it was EXTREMELY PERSONALLY IMPORTANT to you that "hello world" be printed where x=7, you would code

{if(5<x<10), PRINT "hello world"

if(x=7), PRINT "hello world"}

The second reason is cost. I could write a bespoke contract for every client's every need, but to save time==money, there are vast form books/online repositories for standard transactions, plus we ruthlessly plagiarize from each other--it really is the highest form of flattery. The patches are added to address the specific concerns or unique details of the particular deal. We then run a sanity check applying the patches to the rest of the agreement to make sure nothing weird happens.

Finally, I would characterize a "reasonableness" requirement as "we'll just ask a jury" rather than a "do what I mean" button. Because taking a case to trial is so flipping expensive, and putting in a reasonableness standard or "course of business" all but guarantees (due to common law) that you will have to go to trial if there's an unresolvable dispute, and juries are super unpredictable, you would almost never use "reasonableness" as a substitute for "do what I mean." You would keep hammering at a verbalization of "what I mean".

If the other side were insisting on a "reasonableness" standard for some action which we could define by reference to objective facts (like "200 widgets made out of ASTM A125-96 steel"), my mental model of them would update to "so likely to be planning to cheat my client that we should walk the deal".

The other use of "reasonable" you mention is more common--to give the party some freedom but not absolute discretion.

Thanks again for this excellent post. I feel hopeful when I see non-lawyers understanding law stuff, because it makes me feel hopeless for the future of the Rule of Law (TM) when some part of my job is so complex that no person without specialized training could hope to do it. The law should be for everyone.