Jackpot! An AI Vignette 2021-07-08T20:32:22.528Z
What price would you pay for the RadVac Vaccine and why? 2020-08-22T00:17:56.155Z
Solving Math Problems by Relay 2020-07-17T15:32:00.985Z
Post-mortem on the Center for Long-Term Cybersecurity forecasts 2020-01-09T19:38:33.289Z
2020's Prediction Thread 2019-12-30T23:18:46.991Z
[Part 1] Amplifying generalist research via forecasting – Models of impact and challenges 2019-12-19T15:50:33.412Z
[Part 2] Amplifying generalist research via forecasting – results from a preliminary exploration 2019-12-19T15:49:45.901Z
[Link] John Carmack working on AGI 2019-11-14T00:08:37.250Z
bgold's Shortform 2019-10-17T22:18:11.822Z
Running Effective Structured Forecasting Sessions 2019-09-06T21:30:25.829Z
How to write good AI forecasting questions + Question Database (Forecasting infrastructure, part 3) 2019-09-03T14:50:59.288Z
AI Forecasting Resolution Council (Forecasting infrastructure, part 2) 2019-08-29T17:35:26.962Z
AI Forecasting Dictionary (Forecasting infrastructure, part 1) 2019-08-08T16:10:51.516Z
Do bond yield curve inversions really indicate there is likely to be a recession? 2019-07-10T01:23:36.250Z
What is the best online community for questions about AI capabilities? 2019-05-31T15:38:11.678Z
What's the best approach to curating a newsfeed to maximize useful contrasting POV? 2019-04-26T17:29:30.806Z
How do S-Risk scenarios impact the decision to get cryonics? 2019-04-21T15:59:50.342Z


Comment by bgold on What price would you pay for the RadVac Vaccine and why? · 2021-02-06T02:04:28.847Z · LW · GW

hah yes - seeing that great post from johnwentsworth inspired me to review my own thinking on RadVac. Ultimately I placed a lower estimate on RadVac being effective - or at least effective enough to get me to change my quarantine behavior - such that the price wasn't worth it, but I think I get a rationality demerit for not investing more in the collaborative model building (and collaborative purchasing) part of the process.

Comment by bgold on What price would you pay for the RadVac Vaccine and why? · 2021-02-04T17:42:45.558Z · LW · GW

I'm sorry I didn't see this response until now - thank you for the detailed answer!

Comment by bgold on Player vs. Character: A Two-Level Model of Ethics · 2020-10-01T18:32:18.593Z · LW · GW

I'm guessing your concern feels similar to ones you've articulated in the past around... "heart"/"grounded" rationality, or a concern about "disabling pieces of the epistemic immune system". 

I'm curious if 8 mo's later you feel you can better speak to what you see as the crucial misunderstanding?

Comment by bgold on Thiel on Progress and Stagnation · 2020-08-18T22:43:28.785Z · LW · GW

Out of curiosity what's one of your more substantive disagreements with Thiel?

Comment by bgold on RT-LAMP is the right way to scale diagnostic testing for the coronavirus · 2020-08-04T21:27:14.828Z · LW · GW

I'd be quite interested in reading that guide!

Comment by bgold on Competition: Amplify Rohin’s Prediction on AGI researchers & Safety Concerns · 2020-08-01T02:37:02.705Z · LW · GW

Forecast - 25 mins

  • I thought it was more likely that in the short run there could be a preference cascade among top AGI researchers, and as others have mentioned due to the operationalization of top AGI researchers might be true already.
  • If this doesn't become a majority concern by 2050, I expect it will be because of another AI Winter, and I tried to have my distribution reflect that (a little hamfistedly).
Comment by bgold on Rereading Atlas Shrugged · 2020-07-29T03:11:22.616Z · LW · GW

Thanks for posting this. I recently reread the Fountainhead, which I similarly enjoyed and got more out of than did my teenage self - it was like a narrative, emotional portrayal of the ideals in Marc Andreessen's It's Time to Build essay.

I interpreted your section on The Conflict as the choice between voice and exit.

Comment by bgold on Solving Math Problems by Relay · 2020-07-18T00:07:49.706Z · LW · GW

The larger scientific question was related to Factored Cognition, and getting a sense of the difficulty of solving problems through this type of "collaborative crowdsourcing". The hope was running this experiment would lead to insights that could then inform the direction of future experiments, in the way that you might fingertip feel your way around an unknown space to get a handle on where to go next. For example if it turned out to be easy for groups to execute this type of problem solving, we might push ahead with competitions between teams to develop the best strategies for context-free problem solving.

In that regard it didn't turn out to be particularly informative, because it wasn't easy for the groups to solve the math problems, and it's unclear if that's because of the problems selected, the team compositions, the software, etc. So re: the larger scientific question I don't think there's much to conclude.

But personally I felt that by watching relay participants I gained a lot of UX intuitions around what type of software design and strategy design is necessary for factored strategies - what I broadly think of as problem solving strategies that rely upon decomposition - to work. Two that immediately come to mind:

  • Create software design patterns that allow the user to hide/reveal information in intuitive ways. It was difficult, when thrown into a huge problem doc with little context, to know where to focus. I wanted a way for the previous user to only show me the info I needed. For example, the way workflow-y / Roam Research bullet points allow you to hide unneeded details, and how if you click on a bullet point you're brought into an entirely new context.
  • When designing strategies try focusing on the return signature: When coming up with new strategies for solving relay problems, at first it was entirely free form. I as a user would jump in, try pushing the problem as far as I could, and leave haphazard notes in the doc. Over time we developed more complex shorthand and shared strategies for solving a problem. One heuristic I now use when developing strategies for problem solving that use decomposition is to prioritizing thinking about what each sub part of the strategy will return to the top caller. That clarifies the interface, simplifies what the person working on the sub strategy needs to do, and promotes composability.

These ideas are helpful because - I posit - we're faced with Relay Game like problems all the time. When I work on a project, leave it for a week, and come back, I think I'm engaging in a relay between past Ben, present Ben, and future Ben. Some of these ideas informed my design of templates for collaborative group forecasting.

Comment by bgold on Solving Math Problems by Relay · 2020-07-17T23:08:18.775Z · LW · GW

Thanks, rewrote and tried to clarify. In essence the researchers were testing transmission of "strategies" for using a tool, where an individual was limited in what they could transmit to the next user, akin to this relay experiment.

In fact they found that trying to convey causal theories could undermine the next person's performance; they speculate that it reduced experimentation prematurely.

Comment by bgold on bgold's Shortform · 2020-07-17T17:52:59.256Z · LW · GW

... my god...

Comment by bgold on ESRogs's Shortform · 2020-07-04T15:36:27.767Z · LW · GW

Thanks for posting this. Why did you invest in those three startups in particular? Was it the market, the founders, personal connections? And was it a systematic search for startups to invest in, or more of an "opportunity-arose" situation?

Comment by bgold on What are the best tools for recording predictions? · 2020-05-24T23:22:13.431Z · LW · GW

I know Ozzie has been thinking about this, because we were chatting about how to use an Alfred workflow to post to it. Which I think would be great!

Comment by bgold on What are the best tools for recording predictions? · 2020-05-24T23:00:46.050Z · LW · GW

I've spent a fair bit of time in the forecasting space playing w/ different tools, and I never found one that I could reliably use for personal prediction tracking.

Ultimately for me it comes down to:

1.) Friction: the predictions I'm most interested in tracking are "5-second-level" predictions - "do I think this person is right", "is the fact that I have a cough and am tired a sign that I'm getting sick" etc. - and I need to be able to jot that down quickly.

2.) "Routine": There are certain sites that are toothbrush sites, aka I use them everyday. I'm much more likely to adopt a digital habit if I can use one of those sites to fulfill the function.

So my current workflow for private predictions is to use a textexpander snippet w/ Roam.

- [[Predictions]]
- {percentage}%
- [[operationalized]]:
- [[{date}]]
- {{[[TODO]]}} [[outcome]]:

It doesn't have graphs, but I can get a pretty good sense of how calibrated I am, and if I want I could quickly export the markdown and evaluate it.

Of course I want to mention as another good site - if you want to distributions that's definitely the way to go.

Comment by bgold on How likely is it that US states or cities will prevent travel across their borders? · 2020-03-14T19:40:52.129Z · LW · GW

The commerce clause gives the federal government broad powers to regulate interstate commerce, and in particular the the U.S. Secretary of Health and Human Services can exercise it to institute quarantine.

Comment by bgold on bgold's Shortform · 2020-03-01T20:53:50.899Z · LW · GW

Depression as a concept doesn't make sense to me. Why on earth would it be fitness enhancing to have a state of withdrawal, retreat, collapse where a lack of energy prevents you from trying new things? I've brainstormed a number of explanations:

    • depression as chemical imbalance: a hardware level failure has occurred, maybe randomly maybe because of an "overload" of sensation
    • depression as signaling: withdrawal and retreat from the world indicates a credible signal that I need help
    • depression as retreat: the environment has become dangerous and bad and I should withdraw from it until it changes.

I'm partial to the explanation offered by the Predictive Processing Model, that depression is an extreme form of low confidence. As SSC write:

imagine the world’s most unsuccessful entrepreneur. Every company they make flounders and dies. Every stock they pick crashes the next day. Their vacations always get rained-out, their dates always end up with the other person leaving halfway through and sticking them with the bill.
What if your job is advising this guy? If they’re thinking of starting a new company, your advice is “Be really careful – you should know it’ll probably go badly”.
if sadness were a way of saying “Things are going pretty badly, maybe be less confidence and don’t start any new projects”, that would be useful...
Depression isn’t normal sadness. But if normal sadness lowers neural confidence a little, maybe depression is the pathological result of biological processes that lower neural confidence.

But I still don't understand why the behaviors we often see with depression - isolation, lack of energy - are 'longterm adaptive'. If a particular policy isn't working, I'd expect to see more energy going into experimentation.

[TK. Unfinished because I accidentally clicked submit and haven't finished editing the full comment]

Comment by bgold on How much delay do you generally have between having a good new idea and sharing that idea publicly online? · 2020-02-22T20:28:19.494Z · LW · GW

I rarely share ideas online (I'm working on that); when I do the ideas tend to be "small" observations or models, the type I can write out quickly and send. ~10mins - 1 day after I have it.

Comment by bgold on What is Success in an Immoral Maze? · 2020-01-10T18:24:26.309Z · LW · GW

I've heard that Talking Heads song dozens of times and have never watched the video. I was missing out!

Comment by bgold on bgold's Shortform · 2020-01-06T19:26:01.206Z · LW · GW

neat hadn't seen that thanks

Comment by bgold on What were the biggest discoveries / innovations in AI and ML? · 2020-01-06T19:25:11.754Z · LW · GW

NeurIPS best paper awards will likely contain good leads.

Comment by bgold on Circling as Cousin to Rationality · 2020-01-05T22:52:15.328Z · LW · GW

I expect understanding something more explicitly - such as yours and another persons boundaries - w/o some type of underlying concept of acceptance of that boundary can increase exploitability. I recently wrote a shortform post on the topic of legibility that describes some patterns I've noticed here.

I don't think on average Circling makes one more exploitable, but I expect it increases variance, making some people significantly more exploitable than they were before because previously invisible boundaries are now visible, and can thus be attacked (by others but more often by a different part of the same person).

And yeah it does seem similar to the valley of bad rationality; the valley of bad circling, where when you're in the valley you're focusing on a naive form of connection without discernment of the boundaries.

Comment by bgold on bgold's Shortform · 2020-01-05T22:16:52.528Z · LW · GW
  • Yes And is an improv technique where you keep the energy in a scene alive by going w/ the other persons suggestion and adding more to it. "A: Wow is that your pet monkey? B: Yes and he's also my doctor!"
  • Yes And is generative (creates a lot of output), as opposed to Hmm No which is critical (distills output)
  • A lot of the Sequences is Hmm No
  • It's not that Hmm No is wrong, it's that it cuts off future paths down the Yes And thought-stream.
  • If there's a critical error at the beginning of a thought that will undermine everything else then it makes sense to Hmm No (we don't want to spend a bunch of energy on something that will be fundamentally unsound). But if the later parts of the thought stream are not closely dependent on the beginning, or if it's only part of the stream that gets cut off, then you've lost a lot of potential value that could've been generated by the Yes And.
  • In conversation yes and is much more fun, which might be why the Sequences are important as a corrective (yeah look it's not fun to remember about biases, but they exist and you should model/include them)
  • Write drunk, edit sober. Yes And drunk, Hmm No in the morning.
Comment by bgold on [Part 1] Amplifying generalist research via forecasting – Models of impact and challenges · 2020-01-05T18:22:22.198Z · LW · GW

IMO the term "amplification" fits if the scheme results in a 1.) clear efficiency gain and 2.) it's scalable. This looks like (delivering equivalent results but at a lower cost OR providing better results for an equivalent cost. (cost == $$ & time)), AND (~ O(n) scaling costs).

For example if there was a group of people who could emulate [Researcher's] fact checking of 100 claims but do it at 10x speed, then that's an efficiency gain as we're doing the same work in less time. If we pump the number to 1000 claims and the fact checkers could still do it at 10x speed without additional overheard complexity, then it's also scalable. Contrast that with the standard method of hiring additional junior researchers to do the fact checking - I expect it to not be as scalable ("huh we've got all these employees now I guess we need an HR department and perf reviews and...:)

It does seem like a fuzzy distinction to me, and I am mildly concerned about overloading a term that already has an association w/ IDA.

Comment by bgold on ozziegooen's Shortform · 2020-01-02T22:04:43.428Z · LW · GW

Is there not a distillation phase in forecasting? One model of the forecasting process is person A builds up there model, distills a complicated question into a high information/highly compressed datum, which can then be used by others. In my mind its:

Model -> Distill - > "amplify" (not sure if that's actually the right word)

I prefer the term scalable instead of proliferation for "can this group do it cost-effectively" as it's a similar concept to that in CS.

Comment by bgold on bgold's Shortform · 2020-01-02T21:58:39.549Z · LW · GW

Thanks for including that link - seems right, and reminded me of Scott's old post Epistemic Learned Helplessness

The only difference between their presentation and mine is that I’m saying that for 99% of people, 99% of the time, taking ideas seriously is the wrong strategy

I kinda think this is true, and it's not clear to me from the outset whether you should "go down the path" of getting access to level 3 magic given the negatives.

Probably good heuristics are proceeding with caution when encountering new/out there ideas, remembering you always have the right to say no, finding trustworthy guides, etc.

Comment by bgold on bgold's Shortform · 2020-01-02T00:38:43.389Z · LW · GW
  • Why do I not always have conscious access to my inner parts? Why, when speaking with authority figures, might I have a sudden sense of blankness.
  • Recently I've been thinking about this reaction in the frame of 'legibility', ala Seeing like a State. State's would impose organizational structures on societies that were easy to see and control - they made the society more legible - to the actors who ran the state, but these organizational structure were bad for the people in the society.
    • For example, census data, standardized weights and measures, and uniform languages make it easier to tax and control the population. [Wikipedia]
  • I'm toying with applying this concept across the stack.
    • If you have an existing model of people being made up of parts [Kaj's articles], I think there's a similar thing happening. I notice I'm angry but can't quite tell why or get a conceptual handle on it - if it were fully legible and accessible to conscious mind, then it would be much easier to apply pressure and control that 'part', regardless if the control I am exerting is good. So instead, it remains illegible.
    • A level up, in a small group conversation, I notice I feel missed, like I'm not being heard in fullness, but someone else directly asks me about my model and I draw a blank, like I can't access this model or share it. If my model were legible, someone else would get more access to it and be able to control it/point out its flaws. That might be good or it might be bad, but if it's illegible it can't be "coerced"/"mistaken" by others.
    • One more level up, I initially went down this track of thinking for a few reasons, one of which was wondering why prediction forecasting systems are so hard to adopt within organizations. Operationalization of terms is difficult and it's hard to get a precise enough question that everyone can agree on, but it's very 'unfun' to have uncertain terms (people are much more likely to not predict then predict with huge uncertainty). I think the legibility concept comes into play - I am reluctant to put a term out that is part of my model of the world and attach real points/weight to it because now there's this "legible leverage point" on me.
      • I hold this pretty loosely, but there's something here that rings true and is similar to an observation Robin Hanson made around why people seem to trust human decision makers more than hard standards.
  • This concept of personal legibility seems associated with the concept of bucket errors, in that theoretically sharing a model and acting on the model are distinct actions, except I expect often legibility concerns are highly warranted (things might be out to get you)
Comment by bgold on 2020's Prediction Thread · 2019-12-30T23:20:59.297Z · LW · GW

I'd also encourage you to link your predictions to Foretold/Metaculus/other prediction aggregator questions, though only if you write your prediction in the thread as well to prevent link rot.

Comment by bgold on AlphaStar: Impressive for RL progress, not for AGI progress · 2019-11-13T18:12:28.958Z · LW · GW

I watched all of the Grandmaster level games. When playing against grandmasters the average win rate of AlphaStar across all three races was 55.25%

  • Protoss Win Rate: 78.57%
  • Terran Win Rate: 33.33%
  • Zerg Win Rate: 53.85%

Detailed match by match scoring

While I don't think that it is truly "superhuman", it is definitely competitive against top players.

Comment by bgold on bgold's Shortform · 2019-10-23T19:22:34.889Z · LW · GW

I remember seeing other claims/analysis of this but don't remember where

Comment by bgold on bgold's Shortform · 2019-10-21T20:17:07.533Z · LW · GW

Is the clearest "win" of a LW meme the rise of the term "virtue signaling"? On the one hand I'm impressed w/ how dominant it has become in the discourse, on the other... maybe our comparative advantage is creating really sharp symmetric weapons...

Comment by bgold on bgold's Shortform · 2019-10-17T22:18:11.972Z · LW · GW

I have a cold, which reminded me that I want fashionable face masks to catch on so that I can wear them all the time in cold-and-flu season without accruing weirdness points.

Comment by bgold on Daniel Kokotajlo's Shortform · 2019-10-14T19:18:57.220Z · LW · GW

I'm interested, and I'd suggest using for this

Comment by bgold on Hazard's Shortform Feed · 2019-10-14T17:51:56.607Z · LW · GW

I'd like to see someone in this community write an extension / refinement of it to further {need-good-color-name}pill people into the LW memes that the "higher mind" is not fundamentally better than the "animal mind"

Comment by bgold on Daniel Kokotajlo's Shortform · 2019-10-14T17:47:21.312Z · LW · GW

I'd agree w/ the point that giving subordinates plans and the freedom to execute them as best as they can tends to work out better, but that seems to be strongly dependent on other context, in particular the field they're working in (ex. software engineering vs. civil engineering vs. military engineering), cultural norms (ex. is this a place where agile engineering norms have taken hold?), and reward distributions (ex. does experimenting by individuals hold the potential for big rewards, or are all rewards likely to be distributed in a normal fashion such that we don't expect to find outliers).

My general model is in certain fields humans look more tool shaped and in others more agent shaped. For example an Uber driver when they're executing instructions from the central command and control algo doesn't require as much of the planning, world modeling behavior. One way this could apply to AI is that sub-agents of an agent AI would be tools.

Comment by bgold on Honoring Petrov Day on LessWrong, in 2019 · 2019-09-26T16:33:04.737Z · LW · GW

so shiny. It's like, it's begging to be pressed.

Comment by bgold on Conversation on forecasting with Vaniver and Ozzie Gooen · 2019-08-11T16:52:29.913Z · LW · GW

From a 2 min brainstorm of "info products" I'd expect to be action guiding:

  • Metrics and dashboards reflecting the current state of the organization.
  • Vision statements ("what do we as an organization do and thus what things should we consider as part of our strategy")
  • Trusted advisors
  • Market forces (e.g. price's of goods)

One concrete example is from when I worked in a business intelligence role. What executives wanted was extremely trustworthy reliable data sources to track business performance over time. In a software environment (e.g. all the analytic companies constantly posting to Hacker News) that's trivial, but in a non-software environment that's very hard. It was very action-guiding to be able to see if your last initiative worked, because if it did you could put a lot more money into it and scale it up.

Comment by bgold on Conversation on forecasting with Vaniver and Ozzie Gooen · 2019-08-09T21:09:29.327Z · LW · GW

This seems true that there's a lot of way to utilize forecasts. In general forecasting tends to have an implicit and unstated connection to the decision making process - I think that has to do w/ the nature of operationalization ("a forecast needs to be on a very specific thing") and because much of the popular literature on forecasting has come from business literature (e.g. How to Measure Anything).

That being said I think action-guidingness is still the correct bar to meet for evaluating the effect it has on the EA community. I would bite the bullet and say blogs should also be held to this standard, as should research literature. An important question for an EA blog - say, LW :) - is what positive decisions it's creating (yes there are many other good things about having a central hub, but if the quality of intellectual content is part of it that should be trackable).

If in aggregate many forecasts can produce the same type of guidance or better as many good blog posts, that would be really positive.

Comment by bgold on Quotes from Moral Mazes · 2019-05-30T20:26:15.707Z · LW · GW

This is great, I also had struggled reading Moral Mazes and I appreciate the selected quotes.

For a more readable, modern treatment of the subject I strongly recommend Power: Why Some People Have It - And Others Don't. The author draws heavily from Moral Mazes as well as other case studies.

Comment by bgold on What is a reasonable outside view for the fate of social movements? · 2019-01-04T00:58:55.922Z · LW · GW

Off the cuff:

  • Temperance movement in the United States
  • Much of the radical left movement from the 60s to the 70s (ex. Students for a Democratic Society -> Weatherman)
  • Georgism
  • The Shakers

Another useful line of inquiry might be factoring out what success for a social movement looks like, find social movements that "succeeded", and see what happened to the social movements they were competing against.

Comment by bgold on Oops on Commodity Prices · 2018-06-12T21:27:04.172Z · LW · GW

+1 for noting mistake and for noting the importance of being bold, and asking questions and sharing models even when you're uncertain.

Your use of the Epistemic status tag - which I think /u/gwern pioneered? - seems good for balancing the value of sharing models while preventing polluting the "idea space" with potentially misleading/untrue things.