Posts

AlphaGeometry: An Olympiad-level AI system for geometry 2024-01-17T17:17:30.913Z
The Dictatorship Problem 2023-06-11T02:45:59.388Z
Wealth as a source of technological stagnation? 2022-07-07T05:46:01.663Z
Failing to fix a dangerous intersection 2022-06-30T18:09:18.565Z
AI Training Should Allow Opt-Out 2022-06-23T01:33:17.801Z
Humans are very reliable agents 2022-06-16T22:02:10.892Z
alyssavance's Shortform 2022-06-04T06:20:04.731Z
A Quick Note on AI Scaling Asymptotes 2022-05-25T02:55:45.994Z
Visible Homelessness in SF: A Quick Breakdown of Causes 2022-05-25T01:40:43.768Z
Is AI Progress Impossible To Predict? 2022-05-15T18:30:12.103Z
Almost everyone should be less afraid of lawsuits 2021-11-27T02:06:52.176Z
What Is Optimal Philanthropy? 2012-07-12T00:17:00.016Z
Advice On Getting A Software Job 2012-07-09T18:52:42.271Z
Negative and Positive Selection 2012-07-06T01:34:25.968Z
New York Less Wrong: Expansion Plans 2012-07-01T01:20:09.534Z
Meetup : A Game of Nomic 2012-06-29T00:49:38.820Z
What Would You Like To Read? A Quick Poll 2012-06-21T00:38:09.212Z
Why Academic Papers Are A Terrible Discussion Forum 2012-06-20T18:15:57.377Z
Ideas for rationalist meetup topics 2012-01-12T05:04:03.356Z
Quantified Health Prize Deadline Extended 2012-01-05T09:28:09.184Z
New Haven / Southern Connecticut Meetup, Wednesday Apr. 27th 6 PM 2011-04-25T04:00:07.525Z
Levels of Action 2011-04-14T00:18:46.695Z
Vassar talk in New Haven, Sunday 2/27 2011-02-27T02:49:28.919Z
Vassar talk in New Haven, Sunday 2/27 2011-02-26T20:57:27.652Z
Science: Do It Yourself 2011-02-13T04:47:46.252Z
Rally to Restore Rationality 2010-10-18T18:41:33.876Z
Call for Volunteers 2010-10-02T17:43:04.690Z
Reminder: Weekly LW meetings in NYC 2010-09-12T20:44:50.435Z
MIRI Call for Volunteers 2010-07-12T23:49:57.918Z
Singularity Summit 2010 on Aug. 14-15 in San Francisco 2010-06-02T06:01:26.276Z
Proposed New Features for Less Wrong 2010-04-27T01:10:39.138Z
Announcing the Less Wrong Sub-Reddit 2010-04-02T01:17:44.603Z
The Graviton as Aether 2010-03-04T22:13:59.800Z
On the Power of Intelligence and Rationality 2009-12-23T10:49:44.496Z
Test Your Calibration! 2009-11-11T22:03:38.439Z
Arrow's Theorem is a Lie 2009-10-24T20:46:07.942Z

Comments

Comment by alyssavance on The Dictatorship Problem · 2023-06-11T17:58:44.357Z · LW · GW

FWIW I'm not convinced by the article on Haley, having bad conservative policies != being an anti-democratic nut job who wants to rig elections and put all your opponents in jail. She's super unlikely to win, though.

Comment by alyssavance on The Dictatorship Problem · 2023-06-11T17:48:21.570Z · LW · GW

Most highly educated people lean left, but there really are just very few Stalinists. I quoted a poll above showing that just 8% of Americans would support an AOC hard-left party, and actual Stalinists are a small fraction of that. There's no developed country where tankies get a serious fraction of the vote. See Contrapoints for why communist revolutionaries are super unlikely to take power: https://youtu.be/t3Vah8sUFgI

There are many crazy professors of various stripes, but universities aren't states. They can't shoot you, can't throw you in jail, can't seize your house or business, and are ultimately dependent on government funding to even exist.

Comment by alyssavance on The Dictatorship Problem · 2023-06-11T15:38:59.796Z · LW · GW

The current constitution isn't that old (although 65 years is still longer than most democracies), but with brief interruptions, France has been a democracy for around 150 years, which is far longer than most countries can claim. 

Comment by alyssavance on The Dictatorship Problem · 2023-06-11T15:15:17.797Z · LW · GW

Thanks for the response! Here are some comments:

 - India, Turkey, and Hungary are widely referred to as "hybrid regimes" (https://en.wikipedia.org/wiki/Hybrid_regime), in which opposition still exists and there are still elections, but the state interferes with elections so as to virtually guarantee victory. In Turkey's case, there have been many elections, but Erdogan always wins through a combination of mass arrests, media censorship, and sending his most popular opponent to prison for "insulting public officials" (https://www.bbc.com/news/world-europe-63977555). In India's case, Modi is no doubt very popular, but elections are likewise hardly fair when the main opponent is disqualified and sent to prison for "defamation" (insulting Modi). Rather than being voted out, hybrid regimes usually transition to full dictatorships, as has happened in (eg.) Russia, Belarus, Venezuela, Nicaragua, Iran, etc.

 - Of course nothing is certain, but France's president is very powerful, and this article discusses in detail how Le Pen could manipulate the system to get a legislative supermajority and virtually unlimited power if elected: https://foreignpolicy.com/2022/04/20/france-election-le-pen-macron-constitution-separtism-law-state-of-emergency-referendum/

 - I made a map here of roughly how much electoral support the far right has in each country worldwide (https://twitter.com/alyssamvance/status/1656882958903418880). However, it is tricky to forecast based on this, because far-right parties can appear from nothing and gain a wide support base very quickly (as happened with eg. Chile's Republican Party).

Comment by alyssavance on Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover · 2022-07-19T18:29:52.697Z · LW · GW

I'm surprised by how strong the disagreement is here. Even if what we most need right now is theoretical/pre-paradigmatic, that seems likely to change as AI develops and people reach consensus on more things; compare eg. the work done on optics pre-1800 to all the work done post-1800. Or the work done on computer science pre-1970 vs. post-1970. Curious if people who disagree could explain more - is the disagreement about what stage the field is in/what the field needs right now in 2022, or the more general claim that most future work will be empirical?

Comment by alyssavance on It’s Probably Not Lithium · 2022-06-30T23:13:03.794Z · LW · GW

I think saying "we" here dramatically over-indexes on personal observation. I'd bet that most overweight Americans have not only eaten untasty food for an extended period (say, longer than a month); and those that have, found that it sucked and stopped doing it. Only eating untasty food really sucks! For comparison, everyone knows that smoking is awful for your health, it's expensive, leaves bad odors, and so on. And I'd bet that most smokers would find "never smoke again" easier and more pleasant (in the long run) than "never eat tasty food again". Yet, the vast majority of smokers continue smoking:

https://news.gallup.com/poll/156833/one-five-adults-smoke-tied-time-low.aspx

Comment by alyssavance on A Quick List of Some Problems in AI Alignment As A Field · 2022-06-30T21:55:29.174Z · LW · GW

https://transformer-circuits.pub/ seems impressive to me!

Comment by alyssavance on A Quick List of Some Problems in AI Alignment As A Field · 2022-06-23T20:38:16.947Z · LW · GW

There are now quite a lot of AI alignment research organizations, of widely varying quality. I'd name the two leading ones right now as Redwood and Anthropic, not MIRI (which is in something of a rut technically). Here's a big review of the different orgs by Larks:

https://www.lesswrong.com/posts/C4tR3BEpuWviT7Sje/2021-ai-alignment-literature-review-and-charity-comparison

Comment by alyssavance on Nonprofit Boards are Weird · 2022-06-23T17:54:07.142Z · LW · GW

Great post. I'm reminded of instructions from the 1944 CIA (OSS) sabotage manual:

"When possible, refer all matters to committees, for “further study and consideration.” Attempt to make the committee as large as possible — never less than five."

Comment by alyssavance on Let's See You Write That Corrigibility Tag · 2022-06-22T07:42:42.015Z · LW · GW

Eliezer's writeup on corrigibility has now been published (the posts below by "Iarwain", embedded within his new story Mad Investor Chaos). Although, you might not want to look at it if you're still writing your own version and don't want to be anchored by his ideas.

https://www.projectlawful.com/posts/6075?page=21

Comment by alyssavance on The inordinately slow spread of good AGI conversations in ML · 2022-06-22T07:39:31.477Z · LW · GW

Would be curious to hear more about what kinds of discussion you think are net negative - clearly some types of discussion between some people are positive.

Comment by alyssavance on Let's See You Write That Corrigibility Tag · 2022-06-21T07:19:03.548Z · LW · GW

Thanks for writing this! I think it's a great list; it's orthogonal to some other lists, which I think also have important stuff this doesn't include, but in this case orthogonality is super valuable because that way you're less likely for all lists to miss something. 

Comment by alyssavance on Pivotal outcomes and pivotal processes · 2022-06-21T03:11:08.451Z · LW · GW

This is an awesome comment, I think it would be great to make it a top-level post. There's a Facebook group called "Information Security in Effective Altruism" that might also be interested

Comment by alyssavance on Humans are very reliable agents · 2022-06-19T22:02:17.498Z · LW · GW

I hadn't seen that, great paper!

Comment by alyssavance on Where I agree and disagree with Eliezer · 2022-06-19T20:37:42.171Z · LW · GW

Fantastic post! I agree with most of it, but I notice that Eliezer's post has a strong tone of "this is really actually important, the modal scenario is that we literally all die, people aren't taking this seriously and I need more help". More measured or academic writing, even when it agrees in principle, doesn't have the same tone or feeling of urgency. This has good effects (shaking people awake) and bad effects (panic/despair), but it's a critical difference and my guess is the effects are net positive right now.

Comment by alyssavance on Humans are very reliable agents · 2022-06-17T21:28:15.622Z · LW · GW

I edited the MNIST bit to clarify, but a big point here is that there are tasks where 99.9% is "pretty much 100%" and tasks where it's really really not (eg. operating heavy machinery); and right now, most models, datasets, systems and evaluation metrics are designed around the first scenario, rather than the second.

Intentional murder seems analogous to misalignment, not error. If you count random suicides as bugs, you get a big numerator but an even bigger denominator; the overall US suicide rate is ~1:7,000 per year, and that includes lots of people who have awful chronic health problems. If you assume a 1:20,000 random suicide rate and that 40% of people can kill themselves in a minute (roughly, the US gun ownership rate), then the rate of not doing it per decision is ~20,000 * 60 * 16 * 365 * 0.4 = 1:3,000,000,000, or ~99.99999997%.

You say "yet again", but random pilot suicides are incredibly rare! Wikipedia counts eight on commercial flights in the last fifty years, out of a billion or so total flights, and some of those cases are ambiguous and it's not clear what happened: https://en.wikipedia.org/wiki/Suicide_by_pilot

Comment by alyssavance on alyssavance's Shortform · 2022-06-04T06:20:04.963Z · LW · GW

Crazy idea: LessWrong and EA have been really successful in forming student groups at elite universities. But in the US, elite university admissions select on some cool traits (eg. IQ, conscientiousness), don't select on others, and anti-select on some (eg. selection against non-conformists). To find capable people who didn't get into an elite school, what if someone offered moderate cash bounties (say, $1,000-$5,000 range) to anyone who could solve some hard problem (eg. an IMO gold medal problem, or something like https://microcorruption.com/), without already being a "recognized expert" (say, under age 25, not at an elite school, not already working in the field, etc.). This would be similar to the old "Quixey Challenge" (https://venturebeat.com/2012/06/09/quixey-gamifies-job-hunting-in-an-entirely-new-way/), where coders were offered $100 to find a bug in one minute, but at a slightly broader scale and for different skills. This would select from a broader range of people, and could be adapted to whatever types of skills are most in demand, eg. hacking challenges for trying to recruit computer security people (https://forum.effectivealtruism.org/posts/ZJiCfwTy5dC4CoxqA/information-security-careers-for-gcr-reduction). 

Comment by alyssavance on Intergenerational trauma impeding cooperative existential safety efforts · 2022-06-04T05:50:21.983Z · LW · GW

I think it's true, and really important, that the salience of AI risk will increase as the technology advances. People will take it more seriously, which they haven't before; I see that all the time in random personal conversations. But being more concerned about a problem doesn't imply the ability to solve it. It won't increase your base intelligence stats, or suddenly give your group new abilities or plans that it didn't have last month. I'll elide the details because it's a political debate, but just last week, I saw a study that whenever one problem got lots of media attention, the "solutions" people tried wound up making the problem worse the next year. High salience is an important tool, but nowhere near sufficient, and can even be outright counterproductive.

Comment by alyssavance on Visible Homelessness in SF: A Quick Breakdown of Causes · 2022-05-26T22:12:35.220Z · LW · GW

On the state level, the correlation between urbanization and homelessness is small (R^2 = 0.13) and disappears to zero when you control for housing costs, while the reverse is not true (R^2 of the residual = 0.56). States like New Jersey, Rhode Island, Maryland, Illinois, Florida, Connecticut, Texas, and Pennsylvania are among the most urbanized but have relatively low homelessness rates, while Alaska, Vermont, and Maine have higher homelessness despite being very rural. There's also, like, an obvious mechanism where expensive housing causes homelessness (poor people can't afford rent).

The correlation between homelessness and black population in a state is actually slightly negative (R^2 = 0.09). Mississippi, Louisiana and Alabama are some of the blackest states and have the lowest homelessness rates in the US; Hawaii, Oregon, Alaska and Vermont have some of the highest despite being <5% black. 

Data from 538's Urbanization Index: https://fivethirtyeight.com/features/how-urban-or-rural-is-your-state-and-what-does-that-mean-for-the-2020-election/

Comment by alyssavance on A Quick Note on AI Scaling Asymptotes · 2022-05-25T14:14:39.684Z · LW · GW

Do you have links handy?

Comment by alyssavance on A Quick Note on AI Scaling Asymptotes · 2022-05-25T04:09:33.398Z · LW · GW

Thanks, I hadn't seen that! Added it to the post

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-24T05:15:53.144Z · LW · GW

See my response to Gwern: https://www.lesswrong.com/posts/G993PFTwqqdQv4eTg/is-ai-progress-impossible-to-predict?commentId=MhnGnBvJjgJ5vi5Mb

In particular, extremely noisy data does not explain the results here, unless I've totally missed something. If the data is super noisy, the correlation should be negative, not zero, due to regression-to-mean effects (as indeed we saw for the smallest Gopher models, which are presumably so tiny that performance is essentially random). 

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-24T05:14:15.454Z · LW · GW

See my response to Gwern: https://www.lesswrong.com/posts/G993PFTwqqdQv4eTg/is-ai-progress-impossible-to-predict?commentId=MhnGnBvJjgJ5vi5Mb

Sorry, I'm not sure I understood everything here; but if the issue were that task performance "saturated" around 100% and then couldn't improve anymore, we should get different results when we graph logit(performance) instead of raw performance. I didn't see that anywhere. 

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-16T21:25:57.725Z · LW · GW

See my reply to Gwern: https://www.lesswrong.com/posts/G993PFTwqqdQv4eTg/is-ai-progress-impossible-to-predict?commentId=MhnGnBvJjgJ5vi5Mb

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-16T21:12:48.962Z · LW · GW

I re-ran the Gopher MMLU and Big-Bench data as logits rather than raw percentages, the correlation is still zero:

https://i.imgur.com/mSeJoZM.png

(Logit performances for the 400M model and 7B model were highly significantly different, p = 6*10^-7 in single factor ANOVA.)

In the case of MMLU, because random performance is 25% rather than 0%, I tried subtracting 14% (the lowest score of any model on any task) before running the logit, to try to reduce noise from floor effects; the correlation was still zero. The highest score of any model on any task was 96%, few were above 90% and averages were in the 25%-75% range, so I don't think ceiling effects are currently significant here.

If the performance of any given model on any given task were super noisy, you should expect negative correlation, not zero, because of reversion-to-mean effects. Eg., here is a simulation I ran with n = 60 simulated tasks, with different values for the ratio between "how much variance is there in task scalability?" and "how much noise is there for the performance of a given model on a given task?":

https://i.imgur.com/1I71IO0.png

If there is a lot of noise, the correlation is negative; it's pretty unusual to get exactly zero. (Code: https://gist.github.com/rationalism/b8925017700605b339b8f8439283d670)

The way questions are chunked is pretty non-arbitrary, in that questions within a task are much more similar to each other than random questions? Eg., here are two questions from one random BIG-bench task and two from a second task:

      "input": "Each time you play your guitar, you are playing an instrument.",
     "target_scores": { "causal": 0, "correlative": 0, "neutral": 1 }
     "input": "Looking into a bright light makes my eyes water.",
     "target_scores": { "causal": 1, "correlative": 0, "neutral": 0 }
     
     Q: (1 + 1 + 1 + 1) =
     A: 4
     Q: ((2 * 2) + (3 * 1)) =
     A: 7

Comment by alyssavance on Long COVID risk: How to maintain an up to date risk assessment so we can go back to normal life? · 2022-05-15T21:59:57.500Z · LW · GW

I dug into this a little, and right now I think serious, long-term illness from COVID is pretty unlikely. There are lots of studies on this, but in addition to all the usual reasons why studies are unreliable, it's hard to avoid reporting bias when you're analyzing subjective symptoms. (If you catch COVID, you might be more primed to notice fatigue, asthma, muscle pain, etc., that you already had or would have gotten anyway. Random, unexplainable minor medical problems are ridiculously common.)

Some worry that COVID will permanently disable millions of people, leaving them unable to work. This doesn't seem to be happening, disability claims are down from 2019 and haven't tracked infection rates:

https://www.ssa.gov/oact/STATS/dibStat.html

Disability insurance rates (per $ of coverage) went down in 2021; this makes me think that odds of serious long-term COVID disability must be <1%. Insurers would raise rates if there were a flood of new claims, especially given adverse selection:

https://www.meetbreeze.com/disability-insurance/cost-of-long-term-disability-insurance-report/

I think the press is just lying about how common long COVID is. Eg. this article describes a "flood" of COVID disability claims. But it admits there are only 23,000 claims - that's a tiny fraction! That's ~fewer people than have ever been hit by lightning:

https://www.washingtonpost.com/business/2022/03/08/long-covid-disability-benefits/

This article says "large numbers" of people are filing claims. This isn't true! Filing for COVID-related disability is <1% of all claims, and <0.01% of US adults. Overall claim numbers are down vs. 2019:

https://www.nbcnews.com/investigations/got-long-covid-cost-dearly-rcna17942

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-15T21:22:53.610Z · LW · GW

I tried converting the Gopher figures to logits, still got effectively zero correlation. I can't figure out how to embed an image here, but here's the link:

https://imgur.com/3tg397q

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-15T21:07:32.725Z · LW · GW

Thanks! Is the important thing there log-error, though, or just that if the absolute performance difference between models is small enough, then different task performance between the two is noise (as in parallel runs of the same model) and you do wind up reverting to the mean?

I can't get the image to display, but here's an example of how you get a negative correlation if your runs are random draws from the same Gaussian:

https://i.imgur.com/xhtIX8F.png

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-15T19:35:48.501Z · LW · GW

Good to ask, but I'm not sure what it would be. The code is just a linear regression I did in a spreadsheet, and eyeballing the data points, it doesn't look like there are any patterns that a regression is missing. I tried it several different ways (comparing to different smaller models, comparing to averages of smaller models, excluding extreme values, etc.) and the correlation was always zero. Here's the raw data:

https://docs.google.com/spreadsheets/d/1Y_00UcsYZeOwRuwXWD5_nQWAJp4A0aNoySW0EOhnp0Y/edit?usp=sharing

It's hard to know if there is some critical bug in all the results being reported in the Gopher, Chinchilla, and PaLM papers, since we don't have access to the models, but it would surprise me if no one caught that across multiple independent teams. 

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-15T19:29:05.452Z · LW · GW

OK, here's a Google sheet I just threw together: https://docs.google.com/spreadsheets/d/1Y_00UcsYZeOwRuwXWD5_nQWAJp4A0aNoySW0EOhnp0Y/edit?usp=sharing

Comment by alyssavance on Is AI Progress Impossible To Predict? · 2022-05-15T19:03:52.702Z · LW · GW

I just got it from the papers and ran a linear regression, using pdftables.com to convert from PDF to Excel. I used pages 68 and 79 in the Gopher paper:

https://arxiv.org/pdf/2112.11446.pdf

Page 35 in the Chinchilla paper:

https://arxiv.org/pdf/2203.15556.pdf

Pages 79 and 80 in the PaLM paper:

https://arxiv.org/pdf/2203.15556.pdf

Comment by alyssavance on Clarifying the palatability theory of obesity · 2022-02-11T00:31:22.078Z · LW · GW

These maps don't adjust for socioeconomic status, which has a huge correlation with obesity and health in general. West Virginia, Kentucky, and the Black Belt of the South are some of the poorest areas, while Colorado is one of the richest and best-educated. 

http://proximityone.com/graphics/mhi_stcty_17b.gif

https://www.washingtonpost.com/blogs/govbeat/files/2014/04/tumblr_n4jrdrOOC41rasnq9o1_1280.jpg

Comment by alyssavance on Zoe Curzi's Experience with Leverage Research · 2021-10-14T06:51:35.947Z · LW · GW

I have them, but I'm generally hesitant to share emails as they normally aren't considered public. I'd appreciate any arguments on this, pro or con

Comment by alyssavance on Zoe Curzi's Experience with Leverage Research · 2021-10-14T03:52:09.228Z · LW · GW

EDIT: This comment described a bunch of emails between me and Leverage that I think would be relevant here, but I misremembered something about the thread (it was from 2017) and I'm not sure if I should post the full text so people can get the most accurate info (see below discussion), so I've deleted it for now. My apologies for the confusion

Comment by alyssavance on [deleted post] 2020-02-17T22:50:05.680Z

Note: I have deleted a long comment that I didn't feel like arguing with. I reserve the right to do this for future comments. Thank you.

Comment by alyssavance on CFAR’s new focus, and AI Safety · 2016-12-03T02:02:03.752Z · LW · GW

This is just a guess, but I think CFAR and the CFAR-sphere would be more effective if they focused more on hypothesis generation (or "imagination", although that term is very broad). Eg., a year or so ago, a friend of mine in the Thiel-sphere proposed starting a new country by hauling nuclear power plants to Antarctica, and then just putting heaters on the ground to melt all the ice. As it happens, I think this is a stupid idea (hot air rises, so the newly heated air would just blow away, pulling in more cold air from the surroundings). But it is an idea, and the same person came up with (and implemented) a profitable business plan six months or so later. I can imagine HPJEV coming up with that idea, or Elon Musk, or von Neumann, or Google X; I don't think most people in the CFAR-sphere would, it's just not the kind of thing I think they've focused on practicing.

Comment by alyssavance on On the importance of Less Wrong, or another single conversational locus · 2016-11-30T01:12:31.584Z · LW · GW

Seconding Anna and Satvik

Comment by alyssavance on On the importance of Less Wrong, or another single conversational locus · 2016-11-27T22:32:02.905Z · LW · GW

Was including tech support under "admin/moderation" - obviously, ability to eg. IP ban people is important (along with access to the code and the database generally). Sorry for any confusion.

Comment by alyssavance on On the importance of Less Wrong, or another single conversational locus · 2016-11-27T17:37:29.503Z · LW · GW

If the money is there, why not just pay a freelancer via Gigster or Toptal?

Comment by alyssavance on On the importance of Less Wrong, or another single conversational locus · 2016-11-27T10:39:26.176Z · LW · GW

I appreciate the effort, and I agree with most of the points made, but I think resurrect-LW projects are probably doomed unless we can get a proactive, responsive admin/moderation team. Nick Tarleton talked about this a bit last year:

"A tangential note on third-party technical contributions to LW (if that's a thing you care about): the uncertainty about whether changes will be accepted, uncertainty about and lack of visibility into how that decision is made or even who makes it, and lack of a known process for making pull requests or getting feedback on ideas are incredibly anti-motivating." (http://lesswrong.com/lw/n0l/lesswrong_20/cy8e)

That's obviously problematic, but I think it goes way beyond just contributing code. As far as I know, right now, there's no one person with both the technical and moral authority to:

  • set the rules that all participants have to abide by, and enforce them
  • decide principles for what's on-topic and what's off-topic
  • receive reports of trolls, and warn or ban them
  • respond to complaints about the site not working well
  • decide what the site features should be, and implement the high-priority ones

Pretty much any successful subreddit, even smallish ones, will have a team of admins who handle this stuff, and who can be trusted to look at things that pop up within a day or so (at least collectively). The highest intellectual-quality subreddit I know of, /r/AskHistorians, has extremely active and rigorous moderation, to the extent that a majority of comments are often deleted. Since we aren't on Reddit itself, I don't think we need to go quite that far, but there has to be something in place.

Comment by alyssavance on Why CFAR's Mission? · 2015-12-31T13:11:21.240Z · LW · GW

I mostly agree with the post, but I think it'd be very helpful to add specific examples of epistemic problems that CFAR students have solved, both "practice" problems and "real" problems. Eg., we know that math skills are trainable. If Bob learns to do math, along the way he'll solve lots of specific math problems, like "x^2 + 3x - 2 = 0, solve for x". When he's built up some skill, he'll start helping professors solve real math problems, ones where the answers aren't known yet. Eventually, if he's dedicated enough, Bob might solve really important problems and become a math professor himself.

Training epistemic skills (or "world-modeling skills", "reaching true beliefs skills", "sanity skills", etc.) should go the same way. At the beginning, a student solves practice epistemic problems, like the ones Tetlock uses in the Good Judgement Project. When they get skilled enough, they can start trying to solve real epistemic problems. Eventually, after enough practice, they might have big new insights about the global economy, and make billions at a global macro fund (or some such, lots of possibilities of course).

To use another analogy, suppose Carol teaches people how to build bridges. Carol knows a lot about why bridges are important, what the parts of a bridge are, why iron bridges are stronger than wood bridges, and so on. But we'd also expect that Carol's students have built models of bridges with sticks and stuff, and (ideally) that some students became civil engineers and built real bridges. Similarly, if one teaches how to model the world and find truth, it's very good to have examples of specific models built and truths found - both "practice" ones (that are already known, or not that important) and ideally "real" ones (important and haven't been discovered before).

Comment by alyssavance on Why CFAR? The view from 2015 · 2015-12-18T03:19:17.768Z · LW · GW

Hey! Thanks for writing all of this up. A few questions, in no particular order:

  • The CFAR fundraiser page says that CFAR "search[es] through hundreds of hours of potential curricula, and test[s] them on smart, caring, motivated individuals to find the techniques that people actually end up finding useful in the weeks, months and years after our workshops." Could you give a few examples of curricula that worked well, and curricula that worked less well? What kind of testing methodology was used to evaluate the results, and in what ways is that methodology better (or worse) than methods used by academic psychologists?

  • One can imagine a scale for the effectiveness of training programs. Say, 0 points is a program where you play Minesweeper all day; and 100 points is a program that could take randomly chosen people, and make them as skilled as Einstein, Bismarck, or von Neumann. Where would CFAR rank its workshops on this scale, and how much improvement does CFAR feel like there has been from year to year? Where on this scale would CFAR place other training programs, such as MIT grad school, Landmark Forum, or popular self-help/productivity books like Getting Things Done or How to Win Friends and Influence People? (One could also choose different scale endpoints, if mine are too suboptimal.)

  • While discussing goals for 2015, you note that "We created a metric for strategic usefulness, solidly hitting the first goal; we started tracking that metric, solidly hitting the second goal." What does the metric for strategic usefulness look like, and how has CFAR's score on the metric changed from 2012 through now? What would a failure scenario (ie. where CFAR did not achieve this goal) have looked like, and how likely do you think that failure scenario was?

  • CFAR places a lot of emphasis on "epistemic rationality", or the process of discovering truth. What important truths have been discovered by CFAR staff or alumni, which would probably not have been discovered without CFAR, and which were not previously known by any of the staff/alumni (or by popular media outlets)? (If the truths discovered are sensitive, I can post a GPG public key, although I think it would be better to openly publish them if that's practical.)

  • You say that "As our understanding of the art grew, it became clear to us that “figure out true things”, “be effective”, and “do-gooding” weren’t separate things per se, but aspects of a core thing." Could you be more specific about what this caches out to in concrete terms; ie. what the world would look like if this were true, and what the world would look like if this were false? How strong is the empirical evidence that we live in the first world, and not the second? Historically, adjusted for things we probably can't change (like eg. IQ and genetics), how strong have the correlations been between truth-seeking people like Einstein, effective people like Deng Xiaoping, and do-gooding people like Norman Borlaug?

  • How many CFAR alumni have been accepted into Y Combinator, either as part of a for-profit or a non-profit team, after attending a CFAR workshop?

Comment by alyssavance on A Proposed Adjustment to the Astronomical Waste Argument · 2013-05-27T20:45:49.350Z · LW · GW

Religions partially involve values and I think values are a plausible area for path-dependence.

Please explain the influence that, eg., the theological writings of Peter Abelard, described as "the keenest thinker and boldest theologian of the 12th Century", had on modern-day values that might reasonably have been predictable in advance during his time. And that was only eight hundred years ago, only ten human lifetimes. We're talking about timescales of thousands or millions or billions of current human lifetimes.

Conceivably, the genetic code, base ten math, ASCII, English language and units, Java, or the Windows operating system might last for trillions of years.

This claim is prima facie preposterous, and Robin presents no arguments for it. Indeed, it is so farcically absurd that it substantially lowers my prior on the accuracy of all his statements, and it lowers my prior on your statements that you would present it with no evidence except a blunt appeal to authority. To see why, consider, eg., this set of claims about standards lasting two thousand years (a tiny fraction of a comparative eyeblink), and why even that is highly questionable. Or this essay about programming languages a mere hundred years from now, assuming no x-risk and no strong-AI and no nanotech.

For specific examples of changes that I believe could have very broad impact and lead to small, unpredictable positive trajectory changes, I would offer political advocacy of various kinds (immigration liberalization seems promising to me right now), spreading effective altruism, and supporting meta-research.

Do you have any numbers on those? Bostrom's calculations obviously aren't exact, but we can usually get key numbers (eg. # of lives that can be saved with X amount of human/social capital, dedicated to Y x-risk reduction strategy) pinned down to within an order of magnitude or two. You haven't specified any numbers at all for the size of "small, unpredictable positive trajectory changes" in comparison to x-risk, or the cost-effectiveness of different strategies for pursuing them. Indeed, it is unclear how one could come up with such numbers even in theory, since the mechanisms behind such changes causing long-run improved outcomes remain unspecified. Making today's society a nicer place to live is likely worthwhile for all kinds of reasons, but expecting it to have direct influence on the future of a billion years seems absurd. Ancient Minoans from merely 3,500 years ago apparently lived very nicely, by the standards of their day. What predictable impacts did this have on us?

Furthermore, pointing to "political advocacy" as the first thing on the to-do list seems highly suspicious as a signal of bad reasoning somewhere, sorta like learning that your new business partner has offices only in Nigeria. Humans are biased to make everything seem like it's about modern-day politics, even when it's obviously irrelevant, and Cthulhu knows it would be difficult finding any predictable effects of eg. Old Kingdom Egypt dynastic struggles on life now. Political advocacy is also very unlikely to be a low-hanging-fruit area, as huge amounts of human and social capital already go into it, and so the effect of a marginal contribution by any of us is tiny.

Comment by alyssavance on A Proposed Adjustment to the Astronomical Waste Argument · 2013-05-27T19:11:48.645Z · LW · GW

The main reason to focus on existential risk generally, and human extinction in particular, is that anything else about posthuman society can be modified by the posthumans (who will be far smarter and more knowledgeable than us) if desired, while extinction can obviously never be undone. For example, any modification to the English language, the American political system, the New York Subway or the Islamic religion will almost certainly be moot in five thousand years, just as changes to Old Kingdom Egypt are moot to us now.

The only exception would be if the changes to post-human society are self-reinforcing, like a tyrannical constitution which is enforced by unbeatable strong nanotech for eternity. However, by Bostrom's definition, such a self-reinforcing black hole would be an existential risk.

Are there any examples of changes to post-human society which a) cannot ever be altered by that society, even when alteration is a good idea, b) represent a significant utility loss, even compared to total extinction, c) are not themselves total or near-total extinction (and are thus not existential risks), and d) we have an ability to predictably effect at least on par with our ability to predictably prevent x-risk? I can't think of any, and this post doesn't provide any examples.

Comment by alyssavance on Why AI may not foom · 2013-03-23T23:58:57.816Z · LW · GW

A handful of the many, many problems here:

  • It would be trivial for even a Watson-level AI, specialized to the task, to hack into pretty much every existing computer system; almost all software is full of holes and is routinely hacked by bacterium-complexity viruses

  • "The world's AI researchers" aren't remotely close to a single entity working towards a single goal; a human (appropriately trained) is much more like that than Apple, which is much more like than than the US government, which is much more like that than a nebulous cluster of people who sometimes kinda know each other

  • Human abilities and AI abilities are not "equivalent", even if their medians are the same AIs will be much stronger in some areas (eg. arithmetic, to pick an obvious one); AIs have no particular need for our level of visual modeling or face recognition, but will have other strengths, both obvious and not

  • There is already a huge body of literature, formal and informal, on when humans use System 1 vs. System 2 reasoning

  • A huge amount of progress has been made in compilers, in terms of designing languages that implement powerful features in reasonable amounts of computing time; just try taking any modern Python or Ruby or C++ program and porting it to Altair BASIC

  • Large sections of the economy are already being monopolized by AI (Google is the most obvious example)

I'm not going to bother going farther, as in previous conversations you haven't updated your position at all (http://lesswrong.com/lw/i9/the_importance_of_saying_oops/) regardless of how much evidence I've given you.

Comment by alyssavance on MetaMed: Evidence-Based Healthcare · 2013-03-05T20:59:33.547Z · LW · GW

Clients are free to publish whatever they like, but we are very strict about patient confidentiality, and do not release any patient information without express written consent.

Comment by alyssavance on What Would You Like To Read? A Quick Poll · 2012-06-21T02:04:04.715Z · LW · GW

I would agree if I were going to spend a lot of hours on this, but I unfortunately don't have that kind of time.

Comment by alyssavance on What Would You Like To Read? A Quick Poll · 2012-06-21T02:02:27.664Z · LW · GW

What would you propose as an alternative? LW (to my knowledge) doesn't support polls natively, and using an external site would hugely cut response rate.

Comment by alyssavance on What Would You Like To Read? A Quick Poll · 2012-06-21T00:39:46.256Z · LW · GW

Vote up this comment if you would be most likely to read a post on Less Wrong or another friendly blog.

Comment by alyssavance on What Would You Like To Read? A Quick Poll · 2012-06-21T00:39:27.088Z · LW · GW

Vote up this comment if you would be most likely to read a book chapter, available both on Kindle and in physical book form.