Red Pill vs Blue Pill, Bayes style 2023-08-16T15:23:24.911Z
Link Summary: Top 10 Replicated Findings from Behavioral Genetics 2020-04-19T01:32:43.000Z
Operationalizing Newcomb's Problem 2019-11-11T22:52:52.835Z
Is the World Getting Better? A brief summary of recent debate 2019-02-06T17:38:43.631Z


Comment by ErickBall on Succession · 2023-12-27T05:24:53.273Z · LW · GW

That's like saying that because we live in a capitalist society, the default plan is to destroy every bit of the environment and fill every inch of the world with high rise housing projects. It's... true in some sense, but only as a hypothetical extreme, a sort of economic spherical cow. In reality, people and societies are more complicated and less single minded than that, and also people just mostly don't want that kind of wholesale destruction.

Comment by ErickBall on Succession · 2023-12-26T22:36:37.602Z · LW · GW

I didn't think the implication was necessarily that they planned to disassemble every solar system and turn it into probe factories. It's more like... seeing a vast empty desert and deciding to build cities in it. A huge universe, barren of life except for one tiny solar system, seems not depressing exactly but wasteful. I love nature and I would never want all the Earth's wilderness to be paved over. But at the same time I think a lot of the best the world has to offer is people, and if we kept 99.9% of it as a nature preserve then almost nobody would be around to see it. You'd rather watch the unlifted stars, but to do that you have to exist.

Comment by ErickBall on How "Pause AI" advocacy could be net harmful · 2023-12-26T20:38:47.802Z · LW · GW

I don't think governments have yet committed to trying to train their own state of the art foundation models for military purposes, probably partly because they (sensibly) guess that they would not be able to keep up with the private sector. That means that government interest/involvement has relatively little effect on the pace of advancement of the bleeding edge.

Comment by ErickBall on AI Girlfriends Won't Matter Much · 2023-12-24T07:26:05.665Z · LW · GW

Fair point, but I can't think of a way to make an enforceable rule to that effect. And even if you could make that rule, a rogue AI would have no problem with breaking it.

Comment by ErickBall on The Shortest Path Between Scylla and Charybdis · 2023-12-24T04:15:38.929Z · LW · GW

I think if you could demonstrably "solve alignment" for any architecture, you'd have a decent chance of convincing people to build it as fast as possible, in lieu of other avenues they had been pursuing.

Comment by ErickBall on Welcome to Baltimore LessWrong [Edit With Your Details] · 2023-12-22T22:35:30.437Z · LW · GW

Since our info doesn't seem to be here already: We meet on Sundays at 7pm, alternating between virtual and in-person in the lobby of the UMBC Performing Arts and Humanities Building. For more info, you can join our Google group (message the author of this post, bookinchwrm).

Comment by ErickBall on Redirecting one’s own taxes as an effective altruism method · 2023-12-01T06:03:09.770Z · LW · GW

I found this post interesting, mostly because it illustrates deep flaws in the US tax system that we should really fix. I downvoted it because I think it is a terrible strategy for giving more money to charity. Many other good objections have been raised in the comments, and the post itself admits that lack of effectiveness is a serious problem. One problem I did not see addressed anywhere is reputational risk. The world is not static, and a technique that works for an individual criminal or a few conscientious objectors probably will not work consistently for a large and coordinated group of donors, because society will notice and react. What effect would this behavior have on the charities you give to? I suspect most of them, if they knew about it, would justifiably refuse the money. What effect would it have on other organizations you might be associated with? They are now involved with and perhaps encouraging a known criminal, albeit one who probably won't be prosecuted.

In conclusion, I really wish I could vote to disagree with this post without downvoting to make it less visible. I think readers should be able to see it and also see that practically everyone disagrees with it.

Comment by ErickBall on Am I going insane or is the quality of education at top universities shockingly low? · 2023-11-22T16:46:51.502Z · LW · GW

I always thought it would be great to have one set of professors do the teaching, and then a different set come in from other schools just for a couple weeks at the end of the year to give the students a set of intensive written and oral exams that determines a big chunk of their academic standing.

Comment by ErickBall on Dialogue on the Claim: "OpenAI's Firing of Sam Altman (And Shortly-Subsequent Events) On Net Reduced Existential Risk From AGI" · 2023-11-21T21:38:05.904Z · LW · GW

Here's a market, not sure how to define linchpin but we can at least predict whether he'll be part of it.

Comment by ErickBall on How did you integrate voice-to-text AI into your workflow? · 2023-11-20T15:28:31.628Z · LW · GW

I can now get real-time transcripts of my zoom meetings (via a python wrapper of the openai api) which makes it much easier to track the important parts of a long conversation. I tend to zone out sometimes and miss little pieces otherwise, as well as forget stuff.

Comment by ErickBall on Am I going insane or is the quality of education at top universities shockingly low? · 2023-11-20T15:23:18.782Z · LW · GW

That's fair, most of them were probably never great teachers.

Comment by ErickBall on OpenAI Staff (including Sutskever) Threaten to Quit Unless Board Resigns · 2023-11-20T15:22:53.476Z · LW · GW

You are attributing a lot more deviousness and strategic boldness to the so-called deep state than the US government is organizationally capable of. The CIA may have tried a few things like this in banana republics but there's just no way anybody could pull it off domestically.

Comment by ErickBall on Am I going insane or is the quality of education at top universities shockingly low? · 2023-11-20T06:01:55.635Z · LW · GW

Professors being selected for research is part of it. Another part is the tenure you mentioned - some professors feel like once they have tenure they don't need to pay attention to how well they teach. But I think a big factor is another one you already mentioned: salaries. $150k might sound like a lot to a student, but to the kind of person who can become a math or econ professor at a top research university this is... not tiny but not close to optimal. They are not doing it for the money. They are bought in to a culture where the goal is building status in academic circles, and that's based on research. I also think you've had some bad luck. I had a lot of good professors and a handful of bad ones as an undergrad (good school but not a research university) and in grad school maybe a little more equal between good professors and those who didn't care much. But even in the latter cases, I rarely felt like I didn't learn anything. It just took a little more effort on my part to read the book if the lectures were a snooze (and yes, there were a few profs whose voices could literally put me to sleep in an instant).

Comment by ErickBall on The other side of the tidal wave · 2023-11-06T23:54:35.132Z · LW · GW

But that sort of singularity seems unlikely to preserve something as delicately balanced as the way that (relatively well-off) humans get a sense of meaning and purpose from the scarcity of desirable things.

I think our world actually has a great track record of creating artificial scarcity for the sake of creating meaning (in terms of enjoyment, striving to achieve a goal, sense of accomplishment). Maybe "purpose" in the most profound sense is tough to do artificially, but I'm not sure that's something most people feel a whole lot of anyway?

I'm pretty optimistic about our ability to adapt to a society of extreme abundance by creating "games" (either literal or social) that become very meaningful to those engaged in them.

Comment by ErickBall on Autonomic Sanity · 2023-09-27T13:16:07.834Z · LW · GW

Excellent, I think I will give something like that a try

Comment by ErickBall on Don't take the organizational chart literally · 2023-09-26T01:40:36.955Z · LW · GW

I know this is an old thread but I think it's interesting to revisit this comment in light of what happened at Twitter. Musk did, in fact, fire a whole lot of people. And he did, in fact, unban a lot of conservatives without much obvious delay or resistance within the company. I'm not sure how much of an implication that has about your views of the justice department, though. Notably, it was pretty obvious that the decisions at Twitter were being made at the top, and that the people farther down in the org chart had to implement those decisions or be fired. That sort of thing is less often true in government, especially when the actions are on the far end of questionably legal. 

Let's take NSA surveillance of American phone records as an example - plenty of people felt that it was unconstitutional. Without getting into any details, the end result was that it ended up being a political decision whether this sort of thing is acceptable. As far as I know, nobody at the NSA got fired, let alone charged, for allowing such a program. Contrast that with convincing someone to bury the results of an autopsy. They know perfectly well that if that comes out they'll be charged with a crime; formal authority is basically useless. Even if that person is generally loyal to the organization, that loyalty is contingent on a belief that the agency's goals are aligned with the person's goals. And that alignment can change very quickly. Then the person in charge is left with the option of threatening to fire people (do you know how hard it is to fire a civil servant?) or maybe just not promote them (until the next administration comes around), and even that would require a paper trail that I don't think they would risk. Soft power can go very far, but almost never as far as covering up a murder.

Comment by ErickBall on Autonomic Sanity · 2023-09-26T00:07:05.697Z · LW · GW

Thanks! I'd love to hear any details you can think of about what you actually do on a daily basis to maintain mental health (when it's already fairly stable). Personally I don't really have a system for this, and I've been lucky that my bad times are usually not that bad in the scheme of things, and they go away eventually.

Comment by ErickBall on Red Pill vs Blue Pill, Bayes style · 2023-08-17T15:03:44.311Z · LW · GW

I'm not sure how I would work it out. The problem is that presumably you don't value one group more because they chose blue (it's because they're more altruistic in general) or because they chose red (it's because they're better at game theory or something). The choice is just an indicator of how much value you would put on them if you knew more about them. Since you already know a lot about the distribution of types of people in the world and how much you like them, the Bayesian update doesn't really apply in the same way. It only works on what pill they'll take because everyone is deciding with no knowledge of what the others will decide.

In the specific case where you don't feel altruistic towards people who chose blue specifically because of a personal responsibility argument ("that's their own fault"), then trivially you should choose red. Otherwise, I'm pretty confused about how to handle it. I think maybe only your level of altruism towards the blue choosers matters.

Comment by ErickBall on Red Pill vs Blue Pill, Bayes style · 2023-08-17T09:41:15.455Z · LW · GW

Doesn't "trembling hand" mean it's a stable equilibrium even if there are?

Comment by ErickBall on Red Pill vs Blue Pill, Bayes style · 2023-08-16T22:46:03.601Z · LW · GW

I mean definitely most people will not use a decision procedure like this one, so a smaller update seems very reasonable. But I suspect this reasoning still has something in common with the source of the intuition a lot of people have for blue, that they don't want to contribute to anybody else dying.

Comment by ErickBall on Red Pill vs Blue Pill, Bayes style · 2023-08-16T22:14:33.052Z · LW · GW

Sure, if you don't mind the blue-choosers dying then use the stable NE.

Comment by ErickBall on Red Pill vs Blue Pill, Bayes style · 2023-08-16T21:18:19.435Z · LW · GW

People are all over the place but definitely not 50/50. The qualitative solution I have will hold no matter how weak the correlation with other people's choices (for large enough values of N).

If you make the very weak assumption that some nonzero number of participants will choose blue (and you prefer to keep them alive), then this problem becomes much more like a prisoner's dilemma where the maximum payoff can be reached by coordinating to avoid the Nash equilibrium.

Comment by ErickBall on video games > IQ tests · 2023-08-16T12:41:41.770Z · LW · GW

I think optimizer-type jobs are a modest subset of all useful or non-bullshit office jobs. Many call more for creativity, or reliably executing an easy task. In some jobs, basically all the most critical tasks are new and dissimilar to previous tasks, so there's not much to optimize. There's no quick feedback loop. It's more about how reliably you can analyze the new situation correctly. 

I had an optimizing job once, setting up computers over the summer in college. It was fun. Programming is like that too. I agree that if optimizing is a big part of the job, it's probably not bullshit. 

But over time I've come to think that even though occasional programming is the most fun part of my job, the inscrutable parts that you have to do in a vacuum are probably more important. 

Comment by ErickBall on video games > IQ tests · 2023-08-05T18:07:12.553Z · LW · GW

I think one of the major purposes of selecting employees based on a college degree (aside from proving intelligence and actually learning skills) is to demonstrate ability to concentrate over extended periods (months to years) on boring or low-stimulation work, more specifically reading, writing, and calculation tasks that are close analogues of office work. A speedrun of a video game is very different. The game is designed for visual and auditory stimulation. You can clearly see when you're making progress and how much, a helpful feature for entering a flow state. There is often a competitive aspect. And of course you don't have to read or write or calculate anything, or even interact with other people in a productive way. Probably the very best speed runners are mostly smart people who could be good at lots of things, because that's true of almost any competition. But I doubt skill at speedrunning otherwise correlates much with success at most jobs.

Comment by ErickBall on The ants and the grasshopper · 2023-06-09T18:11:45.596Z · LW · GW

The math doesn't necessarily work out that way. If you value the good stuff linearly, the optimal course of action will either be to spend all your resources right away (because the high discount rate makes the future too risky) or to save everything for later (because you can get such a high return on investment that spending any now would be wasteful). Even in a more realistic case where utility is logarithmic with, for example, computation, anticipation of much higher efficiency in the far future could lead to the optimal choice being to use essentially the bare minimum right now.

I think there are reasonable arguments for putting some resources toward a good life in the present, but they mostly involve not being able to realistically pull off total self-deprivation for an extended period of time. So finding the right balance is difficult, because our thinking is naturally biased to want to enjoy ourselves right now. How do you "cancel out" this bias while still accounting for the limits of your ability to maintain motivation? Seems like a tall order to achieve just by introspection.

Comment by ErickBall on Arguments Against Fossil Future? · 2023-06-04T18:29:25.803Z · LW · GW

Positive externalities is a bit of an odd way to phrase it--if it's just counting up the economic value (i.e. price) of the fossil fuels, doesn't it also disregard the consumer surplus? In other words, they've demonstrated that the negative externalities of pollution outweigh the value added on the margin, but if we were to radically decrease our usage of fossil fuels then the cost of energy (especially for certain uses with no good substitute, as you discussed above) would go way up, and the tradeoff on the margin would look very different.

Comment by ErickBall on Accidental Terraforming · 2023-04-30T19:36:55.005Z · LW · GW

I see your point about guilt/blame, but I'm just not sure the term we use to describe the phenomenon is the problem. We've already switched terms once (from "global warming" to "climate change") to sound more neutral, and I would argue that "climate change" is about the most neutral description possible--it doesn't imply that the change is good or bad, or suggest a cause. "Accidental terraforming", on the other hand, combined two terms with opposite valence, perhaps in the intent that they will cancel out? Terraforming is supposed to describe a desirable (for humans) change to the environment, while an accident is usually bad.

But the controversy, blame, and anger don't arise from the moniker, they are a natural consequence of trying to change behavior. In fact, people now like to say "anthropogenic climate change" precisely because they intend to put the blame explicitly on polluting industry. How can we take control of our effects on the climate if we don't first acknowledge them, and then add a moral valence? Without a "should", there is no impetus to action. Telling people they should do something different (and costly) will upset them, yes, but then you can't make an omelet without breaking some eggs.

Comment by ErickBall on Discovering Language Model Behaviors with Model-Written Evaluations · 2023-03-10T22:35:28.050Z · LW · GW

How would a language model determine whether it has internet access? Naively, it seems like any attempt to test for internet access is doomed because if the model generates a query, it will also generate a plausible response to that query if one is not returned by an API. This could be fixed with some kind of hard coded internet search protocol (as they presumably implemented for Bing), but without it the LLM is in the dark, and a larger or more competent model should be no more likely to understand that it has no internet access.

Comment by ErickBall on AGI in sight: our look at the game board · 2023-02-21T23:24:23.920Z · LW · GW

If the NRO had Sentient in 2012 then it wasn't even a deep learning system. Probably they have something now that's built from transformers (I know other government agencies are working on things like this for their own domain specific purposes). But it's got to be pretty far behind the commercial state of the art, because government agencies don't have the in house expertise or the budget flexibility to move quickly on large scale basic research.

Comment by ErickBall on AGI in sight: our look at the game board · 2023-02-21T23:10:47.387Z · LW · GW

Those are... mostly not AI problems? People like to use kitchen-based tasks because current robots are not great at dealing with messy environments, and because a kitchen is an environment heavily optimized for the specific physical and visuospatial capabilities of humans. That makes doing tasks in a random kitchen seem easy to humans, while being difficult for machines. But it isn't reflective of real world capabilities.

When you want to automate a physical task, you change the interface and the tools to make it more machine friendly. Building a roomba is ten times easier than building a robot that can navigate a house while operating an arbitrary stick vacuum. If you want dishes cleaned with minimal human input, you build a dishwasher that doesn't require placing each dish carefully in a rack (eg

Some people have it in their heads that AI is not transformative or is no threat to humans unless it can also do all the exact physical tasks that humans can do. But a key feature of intelligence is that you can figure out ways to avoid doing the parts that are hardest for you, and still accomplish your high level goals.

Comment by ErickBall on Bing Chat is blatantly, aggressively misaligned · 2023-02-20T20:51:56.018Z · LW · GW

"Unaligned AGI doesn't take over the world by killing us - it takes over the world by seducing us."

Por que no los dos?

Comment by ErickBall on Age changes what you care about · 2022-10-28T17:38:15.323Z · LW · GW

Thanks, some of those possibilities do seem quite risky and I hadn't thought about them before.

Comment by ErickBall on Age changes what you care about · 2022-10-28T16:23:48.146Z · LW · GW

It looks like in that thread you never replied to the people saying they couldn't follow your explanation. Specifically, what bad things could an AI regulator do that would increase the probability of doom?

Comment by ErickBall on Age changes what you care about · 2022-10-28T12:02:24.927Z · LW · GW

How does this work?

Comment by ErickBall on Age changes what you care about · 2022-10-28T11:59:12.902Z · LW · GW

Extreme regulation seems plausible if policy makers start to take the problem seriously. But no regulations will apply everywhere in the world.

Comment by ErickBall on Announcing Balsa Research · 2022-09-27T16:54:50.607Z · LW · GW

That's fair, I could have phrased it more positively. I meant it more along the lines of "tread carefully and look out for the skulls" and not "this is a bad idea and you should give up".

Comment by ErickBall on Announcing Balsa Research · 2022-09-27T04:57:21.187Z · LW · GW

I suspect (though it's not something I have experience with) that a successful new policy think tank would be started by people with inside knowledge and connections to be able to suss out where the levers of government are. When the public starts hearing a lot about some dumb thing the government is doing badly (at the federal level), there are basically three possibilities: 1) it's well on its way to being fixed, 2) it's well on its way to becoming partisan and therefore subject to gridlock, or 3) it makes a good story but there isn't much substance to it, e.g. another less tractable factor is the real bottleneck. So you'd want to be in the position of having a thorough gears-level understanding of a particularly policy area that lets you be among the first to identify mistakes/weaknesses and how they could be fixed. Needless to say, this is tough to do in a whole bunch of policy areas at once.

Comment by ErickBall on Announcing Balsa Research · 2022-09-26T20:27:25.695Z · LW · GW

My assumption about crypto money is because SBF/FTX has been the main EA funder giving extensively for political activity so far. Zvi's comment that "existing organizations nominally dedicated to such purposes face poor incentive structures due to how they are funded and garner attention" also implies that Balsa has an unusual funding source. 

Availability of money encourages organizations to spend that money on achieving their goals, and Zvi's blogging about policy failures, here and in the past, has tended to be rather strongly worded and even derisive. This leads me to believe that in practice he will be more focused on using the organization's resources to enact changes, e.g. through advocacy/publicizing failures, than on impartial policy analysis.

If I turn out to be wrong on these points, then I would be significantly more optimistic about the project. In principle I think more policy engagement could be a good thing, if handled carefully.

Comment by ErickBall on Announcing Balsa Research · 2022-09-26T15:44:43.101Z · LW · GW

I agree the goals are good, and many of the problems are real (I work in one of these areas of government myself, so I can personally attest to some of it). But I think that the attitude ("Elites have lost all credibility") and the broad adversarial mandate (find problems that other people should have fixed already but haven't) will plausibly lead not just to wasted money but also to unnecessary politicization and backlash. 

Comment by ErickBall on Announcing Balsa Research · 2022-09-26T02:23:01.130Z · LW · GW

Frankly, I'm worried you have bitten off more than you can chew.

This project has real Carrick Flynn vibes: well-meaning outsider without much domain expertise tries to fix things by throwing crypto money (I assume) at political problems where money has strongly diminishing returns. Focusing on lobbying instead of on a single candidate is an improvement to be sure, but "improve federal policy" is the kind of goal you come up with when you're not familiar with any of the specifics.

Many people have wanted for a long time to make most of the reforms you suggest. Just to take your first two examples, NEPA and the NRC each have huge well-funded interest groups that want them reformed and have been trying for decades, with little success. What does Balsa bring to the table? What actual reforms do you even have in mind?

Comment by ErickBall on chinchilla's wild implications · 2022-08-04T19:45:53.366Z · LW · GW

Thanks, that's interesting... the odd thing about using a single epoch, or even two epochs, is that you're treating the data points differently. To extract as much knowledge as possible from each data point (to approach L(D)), there should be some optimal combination of pre-training and learning rate. The very first step, starting from random weights, presumably can't extract high level knowledge very well because the model is still trying to learn low level trends like word frequency. So if the first batch has valuable high level patterns and you never revisit it, it's effectively leaving data on the table. Maybe with a large enough model (or a large enough batch size?) this effect isn't too bad though.

Comment by ErickBall on chinchilla's wild implications · 2022-08-04T16:58:33.143Z · LW · GW

So do you think, once we get to the point where essentially all new language models are trained on essentially all existing language data, it will always be more compute efficient to increase the size of the model rather than train for a second epoch?

This would seem very unintuitive and is not directly addressed by the papers you linked in footnote 11, which deal with small portions of the dataset betting repeated.

Comment by ErickBall on Epistemic Spot Check: Fatigue and the Central Governor Module · 2021-11-12T20:50:15.996Z · LW · GW

Following up on this because what I said about VO2 max is misleading. I've since learned that VO2 max is unusually useful as a measure of fitness specifically because it bypasses the problem of motivation. As effort and power output increase during the test, VO2 initially increases but then plateaus even as output continues to increase. So as long as motivation is sufficient to reach that plateau, VO2 max measures a physiological parameter rather than a combination of physiology and motivation.

Comment by ErickBall on What Do GDP Growth Curves Really Mean? · 2021-10-16T17:56:47.926Z · LW · GW

One could have picked even more extreme examples, like the triple product in nuclear fusion that has improved even faster than Moore's law yet has generated approximately zero value for society thus far.

Side note: this claim about the triple product only seems to have been true until about the early 90s. Since the early 2000s there have been no demonstrated increases at all (though future increases are projected). 

See here:

Lots of technologies advance rapidly at first, but Moore's Law was exceptional in terms of how long it continued even after massive research efforts had picked the low hanging fruit.

Comment by ErickBall on The Best Software For Every Need · 2021-09-27T19:42:36.660Z · LW · GW

I use Life Reminders for this on Android. One nice feature is that the notifications persist until you tell it the task is done (or tell it to sleep until later).

Comment by ErickBall on The Best Software For Every Need · 2021-09-27T19:36:17.072Z · LW · GW

I have used RemNote for a while but I am transitioning to I find the memorization interface much nicer (nicer than Anki, too). Plus it's not so buggy, though part of that is it doesn't have as many features yet.

Comment by ErickBall on A whirlwind tour of Ethereum finance · 2021-05-03T16:26:13.088Z · LW · GW

Is there any use case for these over-collateralized loans other than getting leveraged exposure to token prices? (Or, like Vitalk did, retaining exposure to token prices while also using the money for something else?) So, for instance, if crypto prices stabilized long term, would the demand for overcollateralized loans disappear? Does anybody take out loans collateralized by stablecoins?

Comment by ErickBall on Reasonable ways for an average LW retail investor to get upside risk? · 2021-02-19T15:07:27.673Z · LW · GW

The Kelly criterion is intended to maximize log wealth. Do you think that's a good goal to optimize? How would your betting strategy be different if your utility function were closer to linear in wealth (e.g. if you planned to donate most of it above some threshold)?

Comment by ErickBall on Some Thoughts on My Psychiatry Practice · 2021-02-05T18:04:26.674Z · LW · GW

Totally agree about having weights at home. Besides the cost, one upside is there's no energy barrier to exercising--I can take a 1-minute break from browsing the web or whatever, do a set, and go back to what I was doing without even breaking a sweat. A downside is it's harder to get in the mindset of doing a full high-intensity workout for 45 minutes; but I think it's a good tradeoff overall.

Comment by ErickBall on (USA) N95 masks are available on Amazon · 2021-01-19T16:42:20.304Z · LW · GW

In practice the big difference is that KN95 masks generally have ear loops, while N95 masks have straps that go around the back of your head which makes them fit tighter and seal better against your face. Traditional N95 masks (but not the duckbill type discussed here) also have more structure and are less flexible, which might help with fit depending on your face shape.