Open AI: Can we rule out near-term AGI?

2018-11-09T12:16:18.821Z · score: 12 (5 votes)
Comment by shardphoenix on No Really, Why Aren't Rationalists Winning? · 2018-11-05T09:36:06.381Z · score: 15 (8 votes) · LW · GW

The winningest rationalist I know of is Dominic Cummings, who was the lead strategist behind the Brexit pro-leave movement. While the majority of LWers may not agree with his goals, he did seem to be effective, and he frequently makes references to rationalist concepts (including IIRC some references to the work of Eliezer Yudkowsky) on his blog: https://dominiccummings.com/

Open AI co-founder on AGI

2018-09-16T10:18:02.603Z · score: 32 (15 votes)
Comment by shardphoenix on What To Do If Nuclear War Seems Imminent · 2018-09-14T10:01:29.015Z · score: 2 (4 votes) · LW · GW

NZ is such a cliche for nuclear escape by now that I wonder if it might be targeted in a full-on nuclear war just to get the escaping rich westerners.

Comment by shardphoenix on Who Wants The Job? · 2018-07-23T10:42:16.357Z · score: 2 (1 votes) · LW · GW

> I've never heard of anyone doing this directly. Has anyone else?

There' s a Brazilian job website that requires users to pay, though I think it's on a subscription basis.

Melbourne Social Meetup July

2018-07-01T06:29:16.877Z · score: 6 (1 votes)
Comment by shardphoenix on Duncan Sabien: "In Defense of Punch Bug" · 2018-05-05T14:02:30.690Z · score: 12 (3 votes) · LW · GW
I should note, for my readers in less sensitive and explicit cultures: I think that the fact that Harley was able to surface and articulate these beliefs is a good thing, on balance.

Really? These anecdotes sound really awful and exhausting to me. They're going out of their way to start status-threatening conflicts over minor inconveniences.

edit: Reading further it seems OP essentially agrees that this behaviour is unhealthy.

Comment by shardphoenix on Good News for Immunostimulants · 2018-04-19T10:01:11.010Z · score: 4 (2 votes) · LW · GW
is about to present its results from two trials.

Might have been better to wait for the result?

Comment by shardphoenix on Rationalist Lent is over · 2018-03-31T05:46:31.837Z · score: 12 (3 votes) · LW · GW

I was semi-abstaining from Reddit (outside work) and Twitter (altogether). I didn't miss Twitter much at all. I did end up going on Reddit a bit for specific hobby/game-related things, but I mostly managed to avoid politics, memes, and other addictive things. Overall I think it was somewhat helpful in terms of avoiding stuff that distracted me from my real problems via anxious-consumption, and I had fewer bouts of political anger. Not sure how I'll take it moving forward - I'll probably stay off Twitter at least. Will have to think about how I want to approach Reddit.

Comment by shardphoenix on God Help Us, Let’s Try To Understand Friston On Free Energy · 2018-03-05T09:19:36.284Z · score: 3 (1 votes) · LW · GW

Reading this I was wondering when Scott suddenly got way more confident with math. Turns out the quotes are messed up in this version. Only the first paragraph of each long block-quote is quoted properly.

Comment by shardphoenix on Links with underscores · 2018-02-24T23:23:38.054Z · score: 3 (1 votes) · LW · GW

Thanks, I didn't know that.

Comment by shardphoenix on Mythic Mode · 2018-02-24T13:30:18.483Z · score: 7 (2 votes) · LW · GW

Meditations on Moloch was creative and effective but ultimately "just" a restatement of well-known game theory. This post is a lot more speculative and anecdotal.

Comment by shardphoenix on Mythic Mode · 2018-02-24T13:23:28.573Z · score: 29 (7 votes) · LW · GW

Two different possible failure modes (if continuing far enough down this path) are along the lines of trying to cure one's cancer with magic instead of medicine, and/or fatalism/"inshallah".

A milder possible failure case is simply spending too much time on this stuff, if it feels fun and effective even when isn't working.

(I didn't downvote you by the way)

Links with underscores

2018-02-24T11:32:48.752Z · score: 2 (1 votes)
Comment by shardphoenix on Mythic Mode · 2018-02-24T11:24:17.470Z · score: 33 (9 votes) · LW · GW

I'm pretty skeptical of your narrative around flying to NY to see the Buddhists. It sounds a bit too much like "The Secret" and other forms of popular wishful thinking to me. I understand that you're trying to sandbox this reasoning to "mythic mode", but the way you write about it in this post (while presumably not in mythic mode) makes it seem like the sandbox might be a bit leaky.

Comment by shardphoenix on Kenshō · 2018-02-17T17:28:56.217Z · score: 6 (2 votes) · LW · GW

I'm a very occasional meditator but I think I mostly get what you mean by Looking. But it's 4am and I'm commenting more to report that something weird happened as result of this post. I read this earlier today, then later via Scott's review read this page about A&P events: http://integrateddaniel.info/the-arising-and-passing-away/

Then tonight in bed I started having some strange sensations. I started seeing some of those vibrations, maybe. This part was only visual and could have been imaginary. Then I started getting a strong sense of "inadequateness" of all sensation which was quite disturbing. Again I might be been tricking myself somehow - though it kept going even when I was up and out of bed. This went on and off for a while. Then when I was in a hypnogogic state (and seeing vibrations), I started to experience strong flow of "energy"* flowing up my upper spine into the back of my head, while my upper body was tense and shaking, which lasted a few seconds. Unlike the previous stuff this was an undeniably real thing. During this time the visual that popped into my head was something like a webcomic featuring crudely drawn anthropomorphic toasters(!) with no legible dialogue, which gave the whole thing an almost comical sense of un-profundity. The sensation was perhaps somewhat enjoyable but not exactly euphoric or anything. This seems like the basic physiological description of an "A&P" event, but as mentioned, no accompanying euphoria or sense of understanding the universe or whatever.

Also, while I don't recall having had this happen before, the "A&P" bit didn't seem entirely unfamiliar and I wouldn't be shocked to learn I'd experienced something similar in childhood. But I wouldn't be surprised if not either.

Anyway I'm a little disturbed by this whole course of events (feels like I was somehow hypnotized into having strong mental sensations) and hope there won't be any bad outcomes. Advice welcome.

* That's a vague way to describe it but it wasn't really like burning or electricity or anything, just some sort of "rush" of... something.

Comment by shardphoenix on Rationalist Lent · 2018-02-15T08:33:04.176Z · score: 7 (2 votes) · LW · GW

Going to do the same.

edit: Might still look at Reddit on my phone at work sometimes when taking a break, but not at home.

Comment by shardphoenix on Arbital postmortem · 2018-02-01T13:35:37.170Z · score: 13 (4 votes) · LW · GW

When I contributed a little in the early days I think it was driven by desire to promote the things I was interested in (mostly video games this case). I'd guess the same goes for historians etc writing the more serious articles. It helped that standards for things like NPOV and citations were a lot lower back then - I don't think the project would have gotten far if everything had to be cited up the wazoo like it is now. (Now that Wikipedia is established as being important people are willing to put a bit more work in).

Although I'm not autistic as such I do suspect there's a connection between the above and the stereotypical autistic desire to excessively talk about one's topic of obsessive interest.

Comment by shardphoenix on A LessWrong Crypto Autopsy · 2018-01-29T10:40:47.418Z · score: 5 (2 votes) · LW · GW

At the time I took the survey I would have answered that I made no money out of cryptocurrency. Since then, I found out that some free cryptocurrency I had forgotten about, which was worth less than $10 at the time, is now worth several thousand. Given that I went to the (small) effort of noticing and signing up for the giveaway, but only did the long-term-hold strategy by accident, I'll give myself a D+. I'm also too lazy to sell it for now so hopefully there isn't a hard crash.

Comment by shardphoenix on Demon Threads · 2018-01-11T07:06:51.503Z · score: 1 (1 votes) · LW · GW

Agreed, and I think this is why "This will be downvoted but.../This is an unpopular opinion but..." can actually work, because they let you broadcast your opinion without others worrying that it will become common knowledge.

Comment by shardphoenix on The Darwin Results · 2017-11-27T00:32:35.292Z · score: 2 (2 votes) · LW · GW

I guess the issue is that it wasn't as obvious to me as to you why claims like the following are true:

> It could have ended in MimicBots and CliqueBots winning together and dividing the pool. This could happen even if their numbers declined slightly early on, if they survived long enough while creating sufficient growth of FoldBot.

(and I was too lazy to step through the reasoning and hoping for a greater degree of hand-holding). But I've been pretty mentally exhausted lately so this may not be representative of the typical reader.

Comment by shardphoenix on The Darwin Results · 2017-11-26T00:54:44.968Z · score: 3 (3 votes) · LW · GW

If you're going to continue tersely talking about X-bots and Y-bots, I don't think I (and I image others) will be able to fully follow unless I've gone through some worked examples, or thought much harder than I normally would about a blog post. But at present it's not yet clear that it's worth doing "homework" to understand this.

Comment by ShardPhoenix on [deleted post] 2017-10-21T08:14:52.764Z

Is there an advantage to doing so?

Comment by shardphoenix on Ten small life improvements · 2017-09-03T06:10:03.602Z · score: 1 (1 votes) · LW · GW

You can also move windows between monitors with Win + Shift + Left/Right.

Comment by shardphoenix on Civil resistance and the 3.5% rule · 2017-02-03T11:56:42.475Z · score: 1 (1 votes) · LW · GW

This describes the case where there's an establishment and a resistance. What happens when there's two establishments and two resistances, each opposing the other? I think this is the more relevant scenario for your interest.

Comment by shardphoenix on How often do you check this forum? · 2017-02-02T06:10:06.643Z · score: 2 (2 votes) · LW · GW

I used to come here daily or more, now more like once a week, as the amount of interesting discussion has gradually declined.

Comment by shardphoenix on Open thread, Oct. 24 - Oct. 30, 2016 · 2016-10-28T23:54:20.923Z · score: 1 (1 votes) · LW · GW

Wouldn't most of the energy go into mechanical deformation of the comet and the sun, rather than EM radiation?

Comment by shardphoenix on June 2016 Media Thread · 2016-06-05T00:54:06.226Z · score: 0 (0 votes) · LW · GW

I've only played about 12 hours, but the Total War: Warhammer crossover is very enjoyable so far. It's the first Total War game I've played since Medieval 1, but it still feels familiar enough while introducing interesting new elements and atmosphere via the Warhammer fantasy setting. On the downside, some of the UI elements are a bit fiddly or obscure, and there are a lot of features to take in when you're first getting started.

For those unfamiliar:

Warhammer = Pretty standard late-medieval fantasy setting with humans, dwarfs, elves (not in this first release), undead, etc, but with more grimdark/heavy metal, and a touch of steampunk.

Total War = Strategy games where you alternate between building and moving armies around on a turn-based strategic map, and playing real-time tactical battles that are unusually realistic and slow-paced compared to the typical hyperactive RTS.

Comment by shardphoenix on June 2016 Media Thread · 2016-06-05T00:50:00.396Z · score: 0 (0 votes) · LW · GW

Death Parade

I watched the first episode of this, but found it too torture-porn-y. Is it all like that?

Comment by shardphoenix on The AI in Mary's room · 2016-05-29T13:36:54.116Z · score: 0 (2 votes) · LW · GW

I'm arguing that there's no contradiction and that this inability is just a limit of humans/organic brains, not a fundamental fact about pain or information.

Comment by shardphoenix on The AI in Mary's room · 2016-05-29T11:15:02.177Z · score: 0 (2 votes) · LW · GW

A way to rephrase the question is, "is there any sequence of sensory inputs other than the stimulation of red cones by red light that will cause Mary to have comparable memories re: the color red as someone who has had their red cones stimulated at some point". It's possible that the answer is no, which says something interesting about the API of the human machine, but doesn't seem necessarily fundamental to the concept of knowledge.

Comment by shardphoenix on The AI in Mary's room · 2016-05-29T01:33:55.509Z · score: 0 (0 votes) · LW · GW

Since you'd know it was a false memory, it doesn't necessarily seem to be a problem, at least if you really need to know what red is like for some reason.

Comment by shardphoenix on The AI in Mary's room · 2016-05-29T01:29:27.250Z · score: 1 (1 votes) · LW · GW

non physical facts

What is a "non physical fact"? The experience of red seems to be physically encoded in the brain like anything else. It does seem clear that some knowledge exists which can't be transmitted from human to human via means of language, at least not in the same way that 2+2=4 can. However, this is just a limitation of the human design that doesn't necessarily apply to eg AIs (which depending on design may be able to transmit and integrate snippets of their internal code and data), and I don't think this thought experiment proves anything beyond that.

Comment by shardphoenix on The AI in Mary's room · 2016-05-26T07:06:35.588Z · score: 6 (6 votes) · LW · GW

Consider a situation where Mary is so dexterous that she is able to perform fine-grained brain surgery on herself. In that case, she could look at what an example of a brain that has seen red looks like, and manually copy any relevant differences into her own brain. In that case, while she still never would have actually seen red through her eyes, it seems like she would know what it is like to see red as well as anyone else.

I think this demonstrates that the Mary's room though experiment is about the limitations of human senses/means of learning, and that the apparent sense of mystery it has comes mainly from the vagueness of what it means to "know all about" something. (Not saying it was a useless idea - it can be quite valuable to be forced to break down some vague or ambiguous idea that we usually take for granted).

Comment by shardphoenix on Open Thread May 16 - May 22, 2016 · 2016-05-16T03:46:13.701Z · score: 3 (3 votes) · LW · GW

FODMAP sensitivity is one source of digestive troubles. This causes sensitivity to garlic, onion, pears, apples, beans, among other things. If you seem sensitive to those things you can get a hydrogen breath test to test it further.

Comment by shardphoenix on 2016 LessWrong Diaspora Survey Analysis: Part One (Meta and Demographics) · 2016-05-14T13:58:47.293Z · score: 4 (4 votes) · LW · GW

Asian (East Asian): -0.600% 80 3.300%
Asian (Indian subcontinent): +0.300% 60 2.500%

Something I've been curious about for a while is the low proportion of Asian and Indian people in the LWsphere compared to STEM communties in general. Any ideas?

Comment by shardphoenix on April 2016 Media Thread · 2016-04-02T10:43:47.435Z · score: 3 (3 votes) · LW · GW

I think it speaks to the unfortunately(?) low status of videogames in current society that this thread has so many specific sections but none for games. (Not that I have any recommendations to make right this minute). (Obviously we can just put them under "other", but it still seems odd to me. Surely games are more popular than podcasts?)

Comment by shardphoenix on April 2016 Media Thread · 2016-04-02T10:37:18.680Z · score: 2 (2 votes) · LW · GW

For anyone who's been interested but who prefers completed works, Homestuck is finally coming to an end on the 13th. If you start reading now you'll probably catch up sometime after it's done. If you haven't heard of it, it's a long and increasingly epic comedy/action/sci-fi/drama web-comic/multimedia-extravaganza with an interesting setting and a huge cast of characters, with a fanbase ranging from hordes of teenage girls to Eliezer Yudkowsky. Definitely not for everyone though. Helps if you can enjoy irreverent humour.

edit: If you do start reading, it may be helpful to know that for the first 3.5 acts Homestuck was designed to resemble an adventure game, and the commands on the bottom of the page (linking to the next page) were submitted by readers, with the resulting panels being created in response to the commands. This leads to a fairly high level of silliness early on. Later it gets a bit more focused.

Comment by shardphoenix on Consider having sparse insides · 2016-04-01T01:49:23.132Z · score: 7 (7 votes) · LW · GW

Paul Graham has an essay on this: http://www.paulgraham.com/identity.html

Comment by shardphoenix on AlphaGo versus Lee Sedol · 2016-03-10T00:15:16.427Z · score: 5 (5 votes) · LW · GW

One difference is that you can't get feedback as fast when dealing with human judgement rather than win/lose in a game (where AlphaGo can play millions of games against itself).

Comment by shardphoenix on March 2016 Media Thread · 2016-03-02T08:43:39.177Z · score: 1 (1 votes) · LW · GW

This season's ERASED (Boku dake ga Inai Machi) is a good mystery/thriller/time-travel-lite with cinematic presentation and a strong emotional resonance (I've teared up several times so far). You'll probably like it if you liked the thriller aspects of Stein's;Gate, but it lacks truly original ideas or deep philosophical meaning so far.

For a more niche appeal this season, Shouwa Genroku Rakugo Shinjuu has a grown-up, literary sort of take on Rakugo (traditional Japanese storytelling/comedy) in the 40's and 70's, with a focus on friendship and rivalry.

Comment by shardphoenix on Open Thread Feb 22 - Feb 28, 2016 · 2016-02-26T02:14:30.199Z · score: 1 (1 votes) · LW · GW

Very interesting insight. It does feel like it solves the problem in some way, and yet in a quantum version as specified, it seems there must be a 1000_year_old_Villiam out there going "huh, I guess I was wrong back on Less Wrong that one time..." Can we really say he doesn't count, even if his measure is small?

Comment by shardphoenix on The ethics of eating meat · 2016-02-19T03:59:30.603Z · score: 3 (3 votes) · LW · GW

trying to make sure we don't repeat this mistake

I disagree that there is a fact of the matter to be mistaken about here (rather than just some consensus opinion that may change over time).

Comment by shardphoenix on [Link] AlphaGo: Mastering the ancient game of Go with Machine Learning · 2016-01-28T23:01:59.768Z · score: 5 (5 votes) · LW · GW

Doesn't a similar criticism apply to ML researchers who claim not to fear AI? (i.e. it would be inconvenient for them if it became widely thought that ML research was dangerous).

Comment by shardphoenix on January 2016 Media Thread · 2016-01-03T01:21:34.870Z · score: 4 (4 votes) · LW · GW

One Punch Man is a super-hero satire staring Saitama, who is dissatisfied with his heroing because he beats all villains with one punch, thus not having any challenging battles. Features great action, animation and direction, and decent humour. No great depth besides some light satire of Japanese society, but a lot of fun.

Comment by shardphoenix on Open thread, Nov. 23 - Nov. 29, 2015 · 2015-11-23T23:16:36.631Z · score: 6 (6 votes) · LW · GW

What is the optimal amount of attention to pay to political news? I've been trying to cut down to reduce stress over things I can't control, but ignoring it entirely seems a little dangerous. For an extreme example, consider the Jews in Nazi Germany - I'd imagine those who kept an eye on what was going on were more likely to leave the country before the Holocaust. Of course something that bad is unlikely, but it seems like it could still be important to be aware of impactful new laws that are passed - eg anti-privacy laws, or internet piracy now much more heavily punishable, etc.

So what's the best way to keep up on things that might have an impact on one's life, without getting caught up in the back-and-forth of day-to-day politics?

Comment by shardphoenix on November 2015 Media Thread · 2015-11-07T09:56:01.712Z · score: 1 (1 votes) · LW · GW

There's Owari, Zoku-Owari, Kizu (3 movies), then the author has also announced some new books to be written. So who knows...

Comment by shardphoenix on November 2015 Media Thread · 2015-11-02T04:01:05.035Z · score: 1 (1 votes) · LW · GW

the Kanbaru arc which seems to be the final arc

That's the final arc chronologically, but there are still a number of unresolved/unrevealed things that will be/are being dealt with in the ongoing season, Owarimonogatari, which is very good so far.

I find the character Nadeko Sengoku impossible to understand

Her issue was that she felt forced to act overly cute to please other people, and eventually got sick of it and lashed out.

Comment by shardphoenix on Open thread, Sep. 28 - Oct. 4, 2015 · 2015-09-29T09:30:30.481Z · score: 4 (4 votes) · LW · GW

Do you ever get an anarchist, a communist and a neoreactionary turning up to the same meeting? If so, does it cause problems?

I've been part of some arguments between libertarians and socialists. They got moderately heated but not severely so. Rationality-wise they seemed better than I've experienced in other communities, but still pretty far from a cool-headed ideal. To be fair I've also had some somewhat heated arguments over more abstract philosophical issues, though with few hard feelings.

Comment by shardphoenix on [LINK] Deep Learning Machine Teaches Itself Chess in 72 Hours · 2015-09-15T23:47:54.780Z · score: 1 (1 votes) · LW · GW

Or is does the trick lie in the "stronger than a naive implementation of depth-limited search", and is there some reason why we expect depth-limited search to have sophisticated implementations, but do not expect this for probablistic search?

Something like that I think. The paper suggests that optimizations applied to depth-based search techniques in more sophisticated engines are already effectively like an approximation of probability-based search.

Comment by shardphoenix on [LINK] Deep Learning Machine Teaches Itself Chess in 72 Hours · 2015-09-15T23:31:19.433Z · score: 2 (2 votes) · LW · GW

In the paper they start with just material balance - then via the learning process, their score on the evaluation test goes from "worse than all hand-written chess engines" to "better than all except the very best one" (and the best one, while more hand-crafted, also uses some ML/statistical tuning of numeric params, and has had a lot more effort put into it).

The reason why the NN solution currently doesn't do as well in real games is because it's slower to evaluate and therefore can't brute-force as far.

Comment by shardphoenix on [LINK] Deep Learning Machine Teaches Itself Chess in 72 Hours · 2015-09-14T23:34:48.396Z · score: 9 (9 votes) · LW · GW

Although it's not better than existing solutions, it's a cool example of how good results can be achieved in a relatively automatic way - by contrast, the evaluation functions of the best chess engines have been carefully engineered and fine-tuned over many years, at least sometimes with assistance from people who are themselves master-level chess players. On the other hand this neural network approach took a relatively short time and could have been applied by someone with little chess skill.

edit: Reading the actual paper, it does sound like a certain amount of iteration, and expertise on the author's part, was still required.

edit2: BTW, the paper is very clear and well written. I'd recommend giving it a read if you're interested in the subject matter.

Comment by shardphoenix on Stupid Questions September 2015 · 2015-09-05T10:41:10.054Z · score: 4 (4 votes) · LW · GW

I've never been in this situation, but one question I'm confused about is how both sides arrive at an appropriate-sized bribe. What's the stop them taking everything you have?

Comment by shardphoenix on Stupid Questions September 2015 · 2015-09-03T08:09:01.838Z · score: 12 (14 votes) · LW · GW

Not sure about the sexbots, but I predict that in my lifetime (ie maybe another 40 years) knowing how to drive will become about as relevant for most people as knowing how to ride a horse.

I do think it's still a bit too soon to be thinking about buying a driverless car though.

An attempt at a short no-prerequisite test for programming inclination

2013-06-29T23:36:06.042Z · score: 4 (7 votes)

Main section vs. discussion section

2012-03-06T05:37:31.090Z · score: 39 (40 votes)

Suggestions for a presentation on FAI?

2011-02-11T06:09:27.227Z · score: 2 (5 votes)