Comment by redlizard on Thoughts on Ben Garfinkel's "How sure are we about this AI stuff?" · 2019-02-07T07:19:53.836Z · score: 4 (3 votes) · LW · GW

General rationality question that should not be taken to reflect any particular opinion of mine on the topic at hand:

At what point should "we can't find any knowledgeable critics offering meaningful criticism against <position>" be interpreted as substantial evidence in favor of <position>, and prompt one to update accordingly?

Comment by redlizard on Good arguments against "cultural appropriation" · 2019-01-05T06:17:56.538Z · score: 4 (2 votes) · LW · GW
Having lost this signaling tool, we are that much poorer.

Are we? Signaling value is both a blessing and a curse, and my impression is that it is generally zero-sum. Personally, I consider myself *richer* when a mundane activity or lifestyle choice loses its signaling association, for it means I am now less restricted in applying it.

Comment by redlizard on Fixed Point Exercises · 2018-11-18T08:11:45.216Z · score: 10 (7 votes) · LW · GW

At the time of writing, for the two spoilers in the main post, hovering over either will reveal both. Is that intentional? It does not seem desirable.

Comment by redlizard on The funnel of human experience · 2018-10-19T04:12:40.104Z · score: 2 (2 votes) · LW · GW

I think there is about a three orders of magnitude difference between the difficulties of "inventing calculus where there was none before" and "learning calculus from a textbook explanation carefully laid out in the optimal order, with each component polished over the centuries to the easiest possible explanation, with all the barriers to understanding carefully paved over to construct the smoothest explanatory trajectory possible".

(Yes, "three orders of magnitude" is an actual attempt to estimate something, insofar as that is at all meaningful for an unquantified gut instinct; it's not just something I said for rhetoric effect.)

Comment by redlizard on "Now here's why I'm punching you..." · 2018-10-17T00:51:08.562Z · score: 7 (3 votes) · LW · GW

I think it will be next to impossible to set up a community norm around this issue for all communities save those with a superhuman level of general honesty. For if there is a norm like this in place, Alice always has a strong incentive to pretend that she is punching based on some generally accepted theory, and that the only thing that needs arguing is the application of this theory to Bob (point 2). Even when there is in fact a new piece of theory ready to be factored out of Alice's argument, it is in Alice's interest to pass this off as being a straightforward application of some more general principle rather than anything new, and she will almost certainly be able to convincingly pass it off as such.

As soon as there is a community norm around building your new punching theories separately from any actual punches, anyone who can argue that their justification for punching Bob doesn't need any new theory, that is, that the justification follows trivially from accepted ideas, will have a that much stronger position. Thus, only the most scrupulous of punchers will ever actually implement step (1), and the norm will collapse.

Comment by redlizard on Fundamentals of Formalisation level 1: Basic Logic · 2018-06-19T00:45:35.894Z · score: 24 (6 votes) · LW · GW

Why oh way does this system make it so needlessly inconvenient to partake in these courses?

I just want to read the lecture material on the topics that interest me, and possibly do some of the exercises. Why do I need to create an account for that using a phony email, subscribe to courses, take tests with limited numbers of retries, and all of that? I am not aiming to get a formal diploma here, and I don't think you plan on awarding me any. So why can't I just... browse the lectures in a stateless web 1.0 fashion?

Comment by redlizard on In Defense of Ambiguous Problems · 2018-06-19T00:28:52.895Z · score: 5 (2 votes) · LW · GW

This. The phrasing "if you are on Vulcan, then you are on the mountain" *sounds* like it should be orthogonal to, and therefore gives no new information on and cannot affect the probability of, your being on Vulcan.

This is quite false, as can be shown easily by the statement "if you are on Vulcan, then false". But it is a line of reasoning I can see being tempting.

Comment by redlizard on Three types of "should" · 2018-06-02T07:46:56.925Z · score: 14 (3 votes) · LW · GW

To me, this form of "epistemic should" doesn't feel like a responsibility-dodge at all. To me, it carries a very specific meaning of a particular warning: "my abstract understanding predicts that X will happen, but there are a thousand and one possible gotchas that could render that abstract understanding inapplicable, and I have no specific concrete experience with this particular case, so I attach low confidence to this prediction; caveat emptor". It is not a shoving off of responsibility, so much as a marker of low confidence, and a warning to everyone not to put their weight down on this prediction.

Of course, if you make such a claim and then proceed to DO put your weight down on the low-confidence prediction without a very explicit decision to gamble in this way, then you really are shoving responsibility under the carpet. But that is not how I have experienced this term being used, either by me or by those around me.

Comment by redlizard on The simple picture on AI safety · 2018-05-28T19:54:24.168Z · score: 16 (6 votes) · LW · GW

I agree that a distillation of a complex problem statement to a simple technical problem represents real understanding and progress, and is valuable thereby. But I don't think your summary of the first half of the AI safety problem is one of these.

The central difficulty that stops this from being a "mere" engineering problem is that we don't know what "safe" is to mean in practice; that is, we don't understand in detail *what properties we would desire a solution to satisfy*. From an engineering perspective, that marks the difference between a hard problem, and a confused (and, usually, confusing) problem.

When people were first trying to build an airplane, they could write down a simple property that would characterize a solution to the problem they were solving: (thing) is heavier than air yet manages to stay out of contact with the ground, for, let say at least minutes at a time. Of course this was never the be-all end-all of what they were trying to accomplish, but this was the central hard problem a solution of which they expected to be able to build on incrementally into the unknown direction of Progress.

I can say the same for, for example, the "intelligence" part of the AI safety problem. Using Eliezer Yudkowsky's optimization framework, I think I have a decent idea of what properties I would want a system to have when I say I want to build an "intelligence". That understanding may or may not be the final word on the topic for all time, but at least it is a distillation that can function as a "mere" engineering problem, a solution for which I can recognize as such and which we can then improve on.

But for the "safe" part of the problem, I don't have a good idea about what properties I want the system to achieve at all. I have a lot of complex intuitions on the problem, including simple-ish ideas that seem to be an important part of it and some insight of what is definitely *not* what I want, but I can't distill this down to a technical requirement that I could push towards. If you were to just hand me a candidate safe AI on a platter, I don't think I could recognize it for what it is; I could definitely reject *some* failed attempts, but I could not tell whether your candidate solution is actually correct or whether it has a flaw I did not see yet. Unless your solution comes with a mighty lecture series explaining exactly why your solution is what I actually want, it will not count as a "solution". Which makes the "safe" part of your summary, in my mind, neither really substantive understanding, nor a technical engineering problem yet.

Comment by redlizard on Slack · 2017-12-16T19:56:47.174Z · score: 9 (3 votes) · LW · GW

I think what we are looking at here is Moloch eating all slack out of the system. I think that is a summary of about 75% of what Moloch does.

Comment by redlizard on More Dakka · 2017-12-03T18:22:13.961Z · score: 6 (3 votes) · LW · GW
In these cases, I do not think such explanations are enough.
Eliezer gives the model of researchers looking for citations plus grant givers looking for prestige, as the explanation for why his SAD treatment wasn’t tested. I don’t buy it. Story doesn’t make sense.

On my model, the lack of exploitability is what allowed the failure to happen, whereas your theory on reasons why people do not try more dakka may be what caused the failure to happen.

If the problem were exploitable in the Eliezer-sense, the market would bulldoze straight through the roadblocks you describe. The fact that the problem is not exploitable allows your roadblocks to have the power they empirically have.

If more light worked, you’d get a lot of citations, for not much cost or effort. If you’re writing a grant, this costs little money and could help many people. It’s less prestigious to up the dosage than be original, but it’s still a big prestige win.

I don't think this is true, empirically. Being the first person to think of a new treatment with a proof of concept is prestigious. Working out all the details and engineering it into something practical is much less so.

Comment by redlizard on Intrinsic properties and Eliezer's metaethics · 2017-09-10T17:28:56.928Z · score: 0 (0 votes) · LW · GW

My interpretation of this thesis immediately remind me of Eliezer's post on locality and compactness of specifications, among others.

Under this framework, my analysis is that triangle-ness has a specification that is both compact and local; whereas L-opening-ness has a specification that is compact and nonlocal ("opens L"), and a specification that is local but noncompact (a full specification of what shapes it is and is not allowed to have), but no specification that is both local and compact. In other words, there is a short specification which refers to something external (L) and a long lookup-table specification that talks only about the key.

I think this is the sensible way of caching out your notion of intrinsicness. (Intrinsicity? Intrinsicitude?)

In this interpretation, I don't think human morality should be judged as non-intrinsic. The shortest local specification of our values is not particularly compact; but neither can you resort to a nonlocal specification to find something more compact. "Whatever makes me happy" is not the sum of my morality, nor do I think you will find a simple specification along those lines that refers to the inner workings of humans. In other words, the specification of human morality is not something you can easily shorten by referring to humans.

There is an important sense in which goodness is more like being ∆-shaped than it is like being L-opening. Namely, goodness of a state of affairs is something that I can assess myself from outside a simulation of that state. That is of course only true because human brains have complex brainware for judging morality, and do not have complex brainware for judging L-opening. For that reason, I don't think "can humans simulate this in a local way" is a measure that is particularly relevant.

Comment by redlizard on Rationality Quotes Thread December 2015 · 2015-12-02T19:08:40.164Z · score: -1 (9 votes) · LW · GW

Consensus tends to be dominated by those who will not shift their purported beliefs in the face of evidence and rational argument.

Jim

Comment by redlizard on Stupid Questions May 2015 · 2015-05-05T23:34:04.906Z · score: 2 (2 votes) · LW · GW

I have tried exactly this with basic topology, and it took me bloody ages to get anywhere despite considerable experience with coq. It was a fun and interesting exercise in both the foundations of the topic I was studying and coq, but it was by no means the most efficient way to learn the subject matter.

Comment by redlizard on Principles of Disagreement · 2015-04-01T16:29:49.108Z · score: 0 (0 votes) · LW · GW

My take on it:

You judge an odds ratio of 15:85 for the money having been yours versus it having been Nick's, which presumably decomposes into a maximum entropy prior (1:1) multiplied by whatever evidence you have for believing it's not yours (15:85). Similarly, Nick has a 80:20 odds ratio that decomposes into the same 1:1 prior plus 80:20 evidence.

In that case, the combined estimate would be the combination of both odds ratios applied to the shared prior, yielding a 1:1 15:85 80:20 = 12:17 ratio for the money being yours versus it being Nicks. Thus, you deserve 12/29 of it, and Nick deserves the remaining 17/29.

Comment by redlizard on Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113 · 2015-03-03T00:09:41.636Z · score: 1 (1 votes) · LW · GW

So it's nonstandard clever wordplay. Voldemort will still anticipate a nontrivial probability of Harry managing undetected clever wordplay. Which means it only has a real chance of working when threatening something that Voldemort can't test immediately.

Comment by redlizard on Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113 · 2015-03-01T05:07:33.925Z · score: 2 (2 votes) · LW · GW

I don't think this is likely, if only because of the unsatisfyingness. However:

And the messages would come out in riddles, and only someone who heard the prophecy in the seer's original voice would hear all the meaning that was in the riddle. There was no possible way that Millicent could just give out a prophecy any time she wanted, about school bullies, and then remember it, and if she had it would've come out as 'the skeleton is the key' and not 'Susan Bones has to be there'. (Ch.77)

Some foreshadowing on the idea of ominous-sounding prophecy terms actually referring to people's names.

Beneath the moonlight glints a tiny fragment of silver, a fraction of a line... (black robes, falling) ...blood spills out in litres, and someone screams a word.

"blood spills out in litres" meshes well with "TEAR APART".

Comment by redlizard on 2014 Survey Results · 2015-01-04T07:01:17.037Z · score: 16 (16 votes) · LW · GW

MIRI Mission/MIRI Effectiveness .395 (1331)

This result sets off my halo effect alarm.

Comment by redlizard on 2014 Less Wrong Census/Survey · 2014-10-26T19:11:24.891Z · score: 38 (38 votes) · LW · GW

I took the survey. No scanner available, alas.

Comment by redlizard on What math is essential to the art of rationality? · 2014-10-18T13:26:11.426Z · score: 0 (0 votes) · LW · GW

Seconded. P versus NP is the most important piece of the basic math of computer science, and a basic notion of algorithms is a bonus. The related broader theory which nonetheless still counts as basic math is algorithmic complexity and the notion of computability.

Comment by redlizard on Harry Potter and the Methods of Rationality discussion thread, July 2014, chapter 102 · 2014-07-26T21:17:16.240Z · score: 10 (10 votes) · LW · GW

I've always modeled it as a physiological "mana capacity" aspect akin to muscle mass -- something that grows both naturally as a developing body matures, and as a result of exercise.

Comment by redlizard on Against utility functions · 2014-06-20T04:37:18.432Z · score: 3 (3 votes) · LW · GW

Certainly, though I should note that there is no original work in the following; I'm just rephrasing standard stuff. I particularly like Eliezer's explanation about it.

Assume that there is a set of things-that-could-happen, "outcomes", say "you win $10" and "you win $100". Assume that you have a preference over those outcomes; say, you prefer winning $100 over winning $10. What's more, assume that you have a preference over probability distributions over outcomes: say, you prefer a 90% chance of winning $100 and a 10% chance of winning $10 over a 80% chance of winning $100 and a 20% change of winning $10, which in turn you prefer over 70%/30% chances, etc.

A utility function is a function f from outcomes to the real numbers; for an outcome O, f(O) is called the utility of O. A utility function induces a preference ordering in which probability-distribution-over-outcomes A is preferred over B if and only if the sum of the utilities of the outcomes in A, scaled by their respective probabilities, is larger than the same for B.

Now assume that you have a preference ordering over probability distributions over outcomes that is "consistent", that is, such that it satisfies a collection of axioms that we generally like reasonable such orderings to have, such as transitivity (details here). Then the von Neumann-Morgenstern theorem says that there exists a utility function f such that the induced preference ordering of f equals your preference ordering.

Thus, if some agent has a set of preferences that is consistent -- which, basically, means the preferences scale with probability in the way one would expect -- we know that those preferences must be induced by some utility function. And that is a strong claim, because a priori, preference orderings over probability distributions over outcomes have a great many more degrees of freedom than utility functions do. The fact that a given preference ordering is induced by a utility function disallows a great many possible forms that ordering might have, allowing you to infer particular preferences from other preferences in a way that would not be possible with preference orderings in general. (Compare this LW article for another example of the degrees-of-freedom thing.) This is the mathematical structure I referred to above.

Comment by redlizard on Against utility functions · 2014-06-19T22:19:01.129Z · score: 2 (2 votes) · LW · GW

It's full of hidden assumptions that are constantly violated in practice, e.g. that an agent can know probabilities to arbitrary precision, can know utilities to arbitrary precision, can compute utilities in time to make decisions, makes a single plan at the beginning of time about how they'll behave for eternity (or else you need to take into account factors like how the agent should behave in order to acquire more information in the future and that just isn't modeled by the setup of vNM at all), etc.

Those are not assumptions of the von Neumann-Morgenstern theorem, nor of the concept of utility functions itself. Those are assumptions of an intelligent agent implemented by measuring its potential actions against an explicitly constructed representation of its utility function.

I get the impression that you're conflating the mathematical structure that is a utility function on the one hand, and representations thereof as a technique for ethical reasoning on the other hand. The former can be valid even if the latter is misleading.

Comment by redlizard on Against utility functions · 2014-06-19T08:13:23.638Z · score: 12 (14 votes) · LW · GW

It's more than a metaphor; a utility function is the structure any consistent preference ordering that respects probability must have. It may or may not be a useful conceptual tool for practical human ethical reasoning, but "just a metaphor" is too strong a judgment.

Comment by redlizard on The Power of Noise · 2014-06-18T08:56:50.593Z · score: 1 (1 votes) · LW · GW

A more involved post about those Bad Confused Thoughts and the deep Bayesian issue underlying it would be really interesting, when and if you ever have time for it.

Comment by redlizard on Some alternatives to “Friendly AI” · 2014-06-17T05:57:12.283Z · score: 5 (5 votes) · LW · GW

Upvoted for the simple reason that this is probably the first article I've EVER seen with a title of the form 'discussion about ' which is in fact about the quoted term, rather than the concept it refers to.

Comment by redlizard on Come up with better Turing Tests · 2014-06-10T16:52:29.597Z · score: 8 (8 votes) · LW · GW

As a point of interest, I want to note that behaving like an illiterate immature moron is a common tactic for (usually banned) video game automation bots when faced with a moderator who is onto you, for exactly the same reason used here -- if you act like someone who just can't communicate effectively, it's really hard for others to reliably distinguish between you and a genuine foreign 13-year-old who barely speaks English.

Comment by redlizard on Can noise have power? · 2014-05-25T19:22:09.878Z · score: 2 (2 votes) · LW · GW

"Worst case analysis" is a standard term of art in computer science, that shows up as early as second-semester programming, and Eliezer will be better understood if he uses the standard term in the standard way.

Actually, in the context of randomized algorithms, I've always seen the term "worst case running time" refer to Oscar's case 6, and "worst-case expected running time" -- often somewhat misleadingly simplified to "expected running time" -- refer to Oscar's case 2.

A computer scientist would not describe the "omega" case as random -- if the input is correlated with the random number source in a way that is detectable by the algorithm, they're by definition not random.

A system that reliably behaves like the omega case is clearly not random. However, a random system such as case 2 may still occasionally behave like omega, with probability epsilon, and it is not at all unreasonable or uncommon to require your algorithm to work efficiently even in those rare cases. Thus, one might optimize a random system by modelling it as an omega system, and demanding that it works well enough even in that context.

Comment by redlizard on [LINK] Prisoner's Dilemma? Not So Much · 2014-05-21T06:29:12.059Z · score: 4 (6 votes) · LW · GW

Group selectionism alert. The "we are optimized for effectively playing the iterated prisoner's dilemma" argument, AKA "people will remember you being a jackass", sounds much more plausible.

Comment by redlizard on Rationality Quotes May 2014 · 2014-05-15T02:58:04.627Z · score: 18 (18 votes) · LW · GW

Even with measurements in hand, old habits are hard to shake. It’s easy to fall in love with numbers that seem to agree with you. It’s just as easy to grope for reasons to write off numbers that violate your expectations. Those are both bad, common biases. Don’t just look for evidence to confirm your theory. Test for things your theory predicts should never happen. If the theory is correct, it should easily survive the evidential crossfire of positive and negative tests. If it’s not you’ll find out that much quicker. Being wrong efficiently is what science is all about.

-- Carlos Bueno, Mature Optimization, pg. 14. Emphasis mine.

Comment by redlizard on The Fallacy of Gray · 2014-05-01T18:09:45.858Z · score: 3 (3 votes) · LW · GW

I already knew it, but this post made me understand it.

Comment by redlizard on 2013 Survey Results · 2014-01-19T04:38:44.990Z · score: 9 (9 votes) · LW · GW

Passphrase: eponymous haha_nice_try_CHEATER

Well played :)

Comment by redlizard on Tell Culture · 2014-01-18T23:50:24.940Z · score: 7 (7 votes) · LW · GW

Trust -- the quintessential element of your so-called "tell culture" -- and vulnerability are two sides of the same coin.

That's true in general. In network security circles, a trusted party is one with the explicit ability to compromise you, and that's really the operational meaning of the term in any context.

Comment by redlizard on How do you tell proto-science from pseudo-science? · 2013-12-03T21:24:52.902Z · score: 0 (0 votes) · LW · GW

My own definition - proto-science is something put forward by someone who knows the scientific orthodoxy in the field, suggesting that some idea might be true. Pseudo-science is something put forward by someone who doesn't know the scientific orthodoxy, asserting that something is true.

This seems like an excellent heuristic to me (and probably one of the key heuristics people actually use for making the distinction), not not valid as an actual definition. For example, Sir Roger Penrose's quantum consciousness is something I would classify as pseudoscience without a second thought, despite the fact that Penrose as a physicist should know and understand the orthodoxy of physics perfectly well.

Comment by redlizard on 2013 Less Wrong Census/Survey · 2013-11-27T03:14:45.375Z · score: 2 (2 votes) · LW · GW

Taking the survey IS posting something insightful.

Comment by redlizard on 2013 Less Wrong Census/Survey · 2013-11-23T22:17:00.856Z · score: 24 (24 votes) · LW · GW

Taken to completion.

The Cryonics Status question really needs an "other" answer. There are more possible statuses one can be in than the ones given; in particular there are more possible "I'd want to, but..." answers.

Comment by redlizard on Harry Potter and the Methods of Rationality discussion thread, part 26, chapter 97 · 2013-08-23T16:00:13.910Z · score: 3 (3 votes) · LW · GW

To figure out a strange plot, look at what happens, then ask who benefits. Except that Dumbledore didn't plan on you trying to save Granger at her trial, he tried to stop you from doing that. What would've happened if Granger had gone to Azkaban? House Malfoy and House Potter would've hated each other forever. Of all the suspects, the only one who wants that is Dumbledore. So it fits. It all fits. The one who really committed the murder is - Albus Dumbledore!

I think if you use this line of reasoning and then allow yourself to dismiss arbitrary parts of it as "not part of the plan", you can make a convincing argument for almost anything. For that reason, I consider the entire theory suspect.

Comment by redlizard on The Amanda Knox Test: How an Hour on the Internet Beats a Year in the Courtroom · 2013-08-17T00:32:40.096Z · score: 1 (1 votes) · LW · GW

In particular, I wish I had been more explicit about the central probability-theoretic point: the fact that the evidence against Guede screens off Kercher's death as evidence against Knox and Sollecito.

I think this insight warrants a great amount of emphasis. The fact that Kercher's death is screened off by some factor unrelated to Knox and Sollecito means that the question of whether the given evidence against Knox and Sollecito is sufficient to judge them co-conspirators is equivalent to the question of whether the given evidence against them would have been sufficient to judge them murder-conspirators in the absence of a body. And I don't think anyone believes THAT is the case.

Comment by redlizard on Harry Potter and the Methods of Rationality discussion thread, part 25, chapter 96 · 2013-08-12T01:23:22.089Z · score: 0 (2 votes) · LW · GW

A piece of evidence in favour of this idea is that Harry, in spite of Dumbledore's warnings, has tried to interpret the prophecy and arrived at almost exactly the canon interpretation on his first try. With dramatic convention regarding the interpretation of prophecies demanding that Harry's interpretation is completely wrong, this lends credibility to the Dark Lord Death hypothesis.

Comment by redlizard on Harry Potter and the Methods of Rationality discussion thread, part 25, chapter 96 · 2013-08-10T17:55:32.928Z · score: 1 (1 votes) · LW · GW

In canon, the assignment of eligible hearers to prophecies is done by Minesty workers. Specifically, the judgment that "the one with the power to vanquish the Dark Lord" refers to Harry, and thus that Harry should have access to the prophecy, was made some time after the recording of the prophecy, by a human. On the assumption that things work the same in the rational-verse, the fact that Lily and James could hear the prophecy isn't evidence of anything other than the interpretation of the Minestry worker who handled the case.

Comment by redlizard on Harry Potter and the Methods of Rationality discussion thread, part 23, chapter 94 · 2013-07-11T03:16:14.845Z · score: 2 (2 votes) · LW · GW

Prediction: by the end of the story, Harry will somehow have managed to dispell the Interdict of Merlin. Given his opinions on muggle society that gains power with each generation, versus wizard society that loses power with each generation, there is no way Harry is going to let that stand. Given that it was brought about by magic, presumably it can be cancelled by magic, and Harry will find a way.

Comment by redlizard on Harry Potter and the Methods of Rationality discussion thread, part 22, chapter 93 · 2013-07-11T02:00:58.865Z · score: 2 (2 votes) · LW · GW

If a text can have Unfortunate Implications even if there was no alternative way to tell the story and the story is legitimate, then I don't understand this concept of Unfortunate Implications and I think it oughtn't to be called "Unfortunate Implications". Because there is no implication of anything.

That sounds a lot like Conservation of Expected Evidence to me, by analogy if not quite literally.

Comment by redlizard on Why one-box? · 2013-07-04T19:26:06.213Z · score: 1 (1 votes) · LW · GW

Okay, those with a two-boxing agent type don't win but the two-boxer isn't talking about agent types. They're talking about decisions. So they are interested in what aspects of the agent's winning can be attributed to their decision and they say that we can attribute the agent's winning to their decision if this is caused by their decision. This strikes me as quite a reasonable way to apportion the credit for various parts of the winning.

Do I understand it correctly that you're trying to evaluate the merits of a decision (to two-box) in isolation of the decision procedure that produced it? Because that's simply incoherent if the payoffs of the decision depend on your decision procedure.

Comment by redlizard on Tiling Agents for Self-Modifying AI (OPFAI #2) · 2013-07-02T15:47:04.605Z · score: 0 (0 votes) · LW · GW

I really think the "You're just as likely to get results in the opposite direction" argument is on the priors overstated for most forms of research. Does Scott think that work we do today is just as likely to decrease our understanding of P/NP as increase it?

My own interpretation of Scott's words here is that it's unclear whether your research is actually helping in the "get Friendly AI before some idiot creates a powerful Unfriendly one" challenge. Fundamental progress in AI in general could just as easily benefit the fool trying to build a AGI without too much concern for Friendliness, as it could benefit you. Thus, whether fundamental research helps out avoiding the UFAI catastrophy is unclear.

Comment by redlizard on No, Really, I've Deceived Myself · 2013-05-22T15:51:03.279Z · score: 1 (1 votes) · LW · GW

I highly doubt that that expectation is due to hidden belief in gods. It sounds more like an overly strong generalization from "it all adds up to normality" to me.

In other words, you can expect the future to turn out alright without any agents actively making it so based purely on inductive bias.

Comment by redlizard on No Safe Defense, Not Even Science · 2013-02-21T22:59:49.309Z · score: 3 (3 votes) · LW · GW

No one begins to truly search for the Way until their parents have failed them, their gods are dead, and their tools have shattered in their hand.

Nihil supernum.