Comment by tag on Internalizing Existentialism · 2019-11-19T12:57:22.461Z · score: 1 (1 votes) · LW · GW

We tend to say things are good or bad when really what we mean is that we like them or we don’t.

Complete subjectivism doesn't follow from anti-realism. In particular, people get punished and rewarded for do immoral and moral things, so there is a need for the society that implements punishments and rewards to agree what right and wrong are.

Comment by tag on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-15T10:02:54.128Z · score: -7 (2 votes) · LW · GW

I don't think the QM sequence would tell you anything you don't know.

The story is that EY became super convinced by Deutch's Fabric or Reality, and never really looked into any approaches other than the ones DD mentioned.

Comment by tag on Creationism and Many-Worlds · 2019-11-15T09:36:10.132Z · score: 2 (2 votes) · LW · GW

You don't need MWI to bump up the chances of lfe forming somewhere, just a large enough universe. Anthropics tells you that you should expect to find yourself on the one planet with life, if there is just one planet with life, in the same way it tells you that you should expect to find yourself in the one branch containing life under MWI.

Comment by tag on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-14T13:57:54.993Z · score: 1 (1 votes) · LW · GW

The most modern Copenhagenish position would be QBism which asserts “whenever I learn something new by means of a measurement, I update”.

There's no doubt a story as to why QBism didn't become the official LessWrong position.

Comment by tag on Insights from the randomness/ignorance model are genuine · 2019-11-14T10:05:02.748Z · score: 1 (1 votes) · LW · GW

the set of integers, each element is strictly more complex than the previous one

Not by algorithmic complexity. The integer consisting of a million 3s in a row is quite compressible.

Comment by tag on [Link] Is the Orthogonality Thesis Defensible? (Qualia Computing) · 2019-11-14T09:34:28.857Z · score: 2 (2 votes) · LW · GW

Metaphysical claims like OI are typically argued on the basis of abduction, ie as a better explanation for existing data.

I say typically, because I am not sure that the claims in question work that way.

Comment by tag on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-13T14:05:01.194Z · score: 1 (1 votes) · LW · GW

But you still don't have an apriori guarantee that a computable model will succeed--that doesn't follow from the claim that the human mind operated within computable limits. You could be facing evidence that all computable models must fail, in which case you should adopt a negative belief about physical/naturalism, even if you don't adopt a positive belief in some supernatural model.

Comment by tag on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-12T13:23:31.980Z · score: 1 (1 votes) · LW · GW

Since the topic is metaphysics , and metaphysics is about what reality really is, the relevant question is whether the world contains hypercomputation.

Comment by tag on Realism about rationality · 2019-11-11T11:54:39.892Z · score: 1 (1 votes) · LW · GW

Physics is not the ter­ri­tory, physics is (quite ex­plic­itly) the mod­els we have of the ter­ri­tory.

People tend to use the word physics in both the map and the territory sense.

We can imag­ine, to a point, that we live in a uni­verse which con­tains hy­per­com­put­ers, but since our own brain is not a hy­per­com­puter, we can never fully test such a the­ory.

That would follow if testing a theory consisted of solely running a simulation in your head, but that is not how physics, the science, works. If the universe was hypercomputational, that would manifest as failures of computatable physics. Note that you only need to run computable physics to generate predictions that are then falsified.

This IMO is the most fun­du­men­tal sig­nifi­cance of the Church-Tur­ing the­sis: since we only per­ceive the world through the lens of our own mind, then from our sub­jec­tive point of view, the world only con­tains com­putable pro­cesses.

If true, that is a form of neo-Kantian idealism. Is that what you really wanted to say?

Comment by tag on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-11T10:19:43.737Z · score: 1 (1 votes) · LW · GW

Algorithmic information theory and AI theory show a clear path towards formalizing metaphysics.

If you define metaphysics as being concerned with deciding between natural and supernatural explanations, the techniques we currently have that are based algorithmic complexity aren't doing a great job.

The problem is that our standard notions of computational limits is based on physical limitations - - the topic of hypercomputation deals, with the computation that might be possible absent physical limits - - so there is a question begging assumption of physicalism built in.

Comment by tag on I would like to try double crux. · 2019-11-10T21:13:55.834Z · score: 1 (1 votes) · LW · GW

if the universe is sufficiently large, whatever pattern in matter constitutes “you” will have multiple identical instances. There is no fact of the matter as to which “one” you are. The patterns are identical, so you are all of them. When you make a choice, you choose for all of them, because they are identical, they have no ability to be different.

Copies of you that arise purely from the size of the universe will have the same counterfacutal or funcitonal behaviour, that is they will do the same thing under the same circumstances...but they will not, in general, do the same thing because they are not in the same circumstances. (There is also the issue that being in different circumstances and making different decisions will feed back into your personality and alter it).

Comment by tag on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-08T09:23:41.525Z · score: 1 (1 votes) · LW · GW

I don't think the argument is strong whatever you call it.

Comment by tag on Randomness vs. Ignorance · 2019-11-08T09:16:45.839Z · score: 1 (1 votes) · LW · GW

It could be argued that it’s all ignorance. The die will roll the way that physics demands, based on the velocity, roll, pitch, yaw of the die, and the surface properties of the felt.

Assuming physics is deterministic, which is not known to be the case.

You can, if you like, believe that there is unknowability at the quantum level, but even that doesn’t imply true randomness, just ignorance of which branch you’ll find your perceptive trail following

Assuming MWI is the correct interpretation of QM, which is also not known to he the case.

Comment by tag on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-07T12:10:30.324Z · score: 2 (2 votes) · LW · GW

The testability criteria are not only intended to demarcate science, but also meaning.

Intended by Popper? What you are saying sounds much more like logical positivism.

Comment by tag on Probability Space & Aumann Agreement · 2019-11-07T11:03:26.656Z · score: 1 (1 votes) · LW · GW

Interesting that the problems with Aumann's theorem were pointed out ten years ago, but belief in it continues to be prevalent.

Comment by tag on Probability Space & Aumann Agreement · 2019-11-07T10:54:25.327Z · score: 1 (1 votes) · LW · GW

[...] re­quires each party to as­sume the other is com­pletely ra­tio­nal and hon­est. [/..] But as­sum­ing that the other party is in fact to­tally ra­tio­nal is just silly.

Assuming honesty is pretty problematical, too. In real-world disputes, participants are likely to disagree about what constitutes evidence ("the Bible says.."), aren't rational, and suspect each others honesty.

Comment by tag on A simple sketch of how realism became unpopular · 2019-11-01T11:29:34.616Z · score: 1 (1 votes) · LW · GW

The reason I’m focusing on this is that I think some of the phrasings you chose in trying to summarize Kant (and translate or steelman his views) are sliding between the three different claims I described above:

[1] “We can’t know things about ultimate reality without relying on initially unjustified knowledge/priors/cognitive machinery.” [2] “We can’t know things about ultimate reality.” [3] “(We can know that) ultimate reality is wildly different from reality-as-we-conceive-of-it.”

With regard to Kant's claims, there is no "sliding" because he asserts all 3 of those, and provides separate arguments for them. In particular, [3] is supported by the Antinomies, which purport to show that spatiality is not a property of objective reality. He is not deriving everything from a single argument, any other than Berkeley is.

Comment by tag on Aella on Rationality and the Void · 2019-11-01T07:24:06.249Z · score: 18 (9 votes) · LW · GW

You can’t be misled about the sense of having an insight.

But you can be misled about *actually having * an insight, since your insight module is as capable of misfiring as anything else.

Comment by tag on Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More · 2019-10-31T11:56:23.787Z · score: 1 (1 votes) · LW · GW

I downvoted TAG’s comment because I found it confusing/misleading.

You could have asked for clarification. The point is that Yudkowsky's early movement was disjoint from actual AI research, and during that period a bunch of dogmas and approaches became solidified, which a lot of AI researchers (Russell is an exception) find incomprehensible or misguided. In other words, you can disapprove of amateur AI safety without dismissing AI safety wholesale.

Comment by tag on A simple sketch of how realism became unpopular · 2019-10-31T11:47:49.992Z · score: 1 (1 votes) · LW · GW

Aumanns theorem assumes there is no ambiguity or disagreement about what counts as evidence or information in the first place. In the real world, that is the major barrier to achieving agreement.

Comment by tag on Is requires ought · 2019-10-31T11:31:04.943Z · score: 3 (2 votes) · LW · GW

I guess I’m just still not sure what you expect the oughts to be doing.

I was assuming that the point was that "oughts" and "ises" aren't completely disjoint, as a crude understanding of the "is-ought divide" might suggest.

I think one of the reasons people are poking holes or bringing up non-”ought”-compliant agents is that we expect humans to sometimes be non-compliant too. This goes back to my question of whether every agent has some oughts, or whether every (sufficiently smart/rational/etc) agent would be impacted by every ought. If you give me a big list of oughts, I’ll give you a big list of ways humans violate them.

If you assume something like moral realism, so that there is some list of "oughts" that are categorical, so that they don't relate to specific kinds of agents or specific situations, then it is likely that humans are violating most of them.

But moral realism is hard to justify.

On the other hand, given the premises that

  1. moral norms are just one kind of norm
  2. norms are always ways of performing a function or achieving an end

then you can come up with a constructivist metaethics that avoids the pitfalls of nihilism, relativism and realism. (I think. No idea if that is what Jesicata is saying).

Comment by tag on Is requires ought · 2019-10-31T11:24:09.198Z · score: 1 (1 votes) · LW · GW

If by “ought” claims you mean things we assign truth values that aren’t derivable from is-statements, then I agree that humans require such beliefs to function. Maybe we could describe choice of a universal Turing machine as such a belief for a Solomonoff inductor.

If by “ought” statements you mean the universally compelling truths of moral realism, then no, it seems straightforward to produce counterexample thinkers that would not be be compelled. As far as I can tell, the things you’re talking about don’t even set a specific course of action for the thing believing them, they have no necessary function beyond the epistemic.

There's a third way of thinking where norms are just rules for achieving a certain kind of result optimally or at least reliably.

I think there’s some dangerous reasoning here around the idea of “why.” If I believe that a plate is on the table, I don’t need to know anything at all about my visual cortex to believe that. The explanation is not a part of the belief, nor is it inseparably attached, nor is it necessary for having the belief, it’s a human thing that we call an explanation in light of fulfilling a human desire for a story about what is being explained.

Nonetheless, your visual cortex must do certain things reliably for you to be able to perceive,

Comment by tag on Is requires ought · 2019-10-31T11:07:53.497Z · score: 1 (1 votes) · LW · GW

Why should they have something to do with ethics?

Comment by tag on Is requires ought · 2019-10-30T11:58:30.693Z · score: 1 (1 votes) · LW · GW

There's a parallel with realism versus instrumentalism, in that both are downstream of value judgements. If you value metaphysical truth , instrumentalism is wrong-for you, because it can't deliver it. And if you don't value metaphysical truth, realism is wrong-for-you becae it is an unnecessary complication.

Comment by tag on Is requires ought · 2019-10-30T11:07:37.099Z · score: 1 (1 votes) · LW · GW

The sections "Social Systems" and "Nondualist Epistemology" seem to be trying to establish that the norms of rationality are ethical norms, and I don't see any need for that. Simple arguments show that there are non-ethical norms, such as the norms relating to playing games, so an epistemological norm can just be another kind of non-ethical norm.

Comment by tag on Is requires ought · 2019-10-30T10:56:20.975Z · score: 1 (1 votes) · LW · GW

I doubt that cats have much notion of “ought” corresponding to the human notion.

A system can be interpreted as following norms from a "stance" perspective. For instance, a kettle ought to switch itself off when the water reaches boiling point. Following norms is not the same as having reflexive awareness of norms.

I doubt that cats have much notion of “ought” corresponding to the human notion.


Comment by tag on Is requires ought · 2019-10-30T10:52:59.870Z · score: 1 (1 votes) · LW · GW

The first line doesn't make it clear whether the kind of "ought" you have in mind is a moral "ought". This seems to be one of the main sources of confusion.

Comment by tag on Is requires ought · 2019-10-30T10:28:54.987Z · score: 1 (1 votes) · LW · GW

Compelled action is setting the bar much to high as well. We don't expect humans to do the right thing on the basis of rational persuasion alone, we also have punishments and rewards.

Comment by tag on What's your big idea? · 2019-10-30T10:25:29.492Z · score: 1 (1 votes) · LW · GW

The market can't solve (high school) education because education is mostly public.

Comment by tag on I would like to try double crux. · 2019-10-30T10:23:06.628Z · score: 1 (1 votes) · LW · GW

Typo for MWI

Comment by tag on A simple sketch of how realism became unpopular · 2019-10-29T19:19:07.864Z · score: 1 (1 votes) · LW · GW

I claim that there’s an important equivocation at work in the idealist tradition between “backtracking” or finding a more “raw” or ultimate version of a thing, and “describe a thing without describing it”.

I can't see why equivocation is helpful. If you want to ascertain the existence of some kind of noumena, you need to distinguish the thing you can do -- come up with a theory of the causes of your perceptions as external physical things -- from the thing you can't do -- get outside the map entirely.

We can’t conceive of an electron without conceiving of it” makes it sound trivial, whereas the way of speaking that phrases things almost as though there were some object in the world (Kant’s ‘noumena’) that transcends our conceptual frameworks and outstrips our every attempt to describe it, makes it sound novel and important and substantive.

The significant-sounding claim does indeed follow from the trivial sounding one. That makes it a good argument. Good arguments should draw non-obvious conclusions from well-founded premises.

Comment by tag on A simple sketch of how realism became unpopular · 2019-10-29T17:59:19.256Z · score: 1 (1 votes) · LW · GW

The same “that sounds silly” heuristic that helps you reject Berkeley’s argument (when it’s fringe and ‘wears its absurdity on its sleeve’) helps you accept 19th-century idealists’ versions of the argument (when it’s respectable and framed as the modern/scientific/practical/educated/consensus view on the issue).

Well, maybe that is right. You haven't really argued against sophisticated idealism, or noticed that scientific realism has dumb forms.

Comment by tag on What's your big idea? · 2019-10-29T15:54:48.145Z · score: 1 (1 votes) · LW · GW

Because people not knowing geography could be a disadvantage to employERs as well as employees. A minimal education system could be below the economic optimum.

Comment by tag on Is requires ought · 2019-10-29T13:54:44.583Z · score: 3 (2 votes) · LW · GW

Suppose I write a programme that spits out syntactically correct, but otherwise random statements. Is that a "theorem prover"? No, because it is not following any norms of reasoning. A theorem prover will follow norms programmed into it, or it is not a theorem prover. Of course it is not reflexively aware that is is following norms, any more than it is aware it is making claims. And it is not as if the ability of humans to follow norms is not at least partially "programmed" from the outside.

Comment by tag on What's your big idea? · 2019-10-29T13:43:21.231Z · score: 1 (1 votes) · LW · GW

Just teach what’s universally useful.

Why? It's not obvious that that is better than teaching a bit of everything. For instance, if 10% of jobs need a little bit of geography, then having only candidates who know nothing about geography is going to be a disadvantage to those employers.

Comment by tag on Is requires ought · 2019-10-29T13:20:50.498Z · score: 3 (2 votes) · LW · GW

If by “ought” statements you mean the universally compelling truths of moral realism,

"universally compelling" is setting the bar extremely high. To set it a bit more reasonably: there are moral facts if there is evidence or argument a rational agent would agree with.

Comment by tag on I would like to try double crux. · 2019-10-29T10:26:26.349Z · score: 0 (2 votes) · LW · GW

I think you are missing some things that are quite basic: essentially no one believes in things like the Mathematical Universe on the basis of empiricism or induction. Instead, Occams razor is the major factor.

Note that by things like MUH include MWI. It is straightforwardly impossible to prove MWI or any other interpretation on the basis of evidence, because they make the same predictions. So the argument given for MWI is in terms of simplicity and consilience.

Not many people here reject all reasoning of that type. Many reject it selectively.

The simplicity criterion means MUH is preferable to CUH, since CUH has an additional constraint.

Comment by tag on I would like to try double crux. · 2019-10-29T09:54:32.625Z · score: 1 (1 votes) · LW · GW

This point is very important: The theory must make predictions to be knowledge—if your theory is equally good at explaining anything (like the witch), then you have zero knowledge, because it fails to constrain anticipation

"Laws of nature do not hold 100%" is a prediction. That's why atheists feel it necessary to argue against miracles.

Comment by tag on I would like to try double crux. · 2019-10-29T09:33:57.666Z · score: 1 (1 votes) · LW · GW

This point is very important: The theory must make predictions to be knowledge—if your theory is equally good at explaining anything (like the witch), then you have zero knowledge, because it fails to constrain anticipation.

If you apply that consistently, you get instrumentalism. Most people here aren't instrumentalists, and do care about theories that dont constrain experience., such as [MWI], MUH and the simulation hypothesis. If you are going to reject metaphysics, you should reject all of it.

Comment by tag on What are some unpopular (non-normative) opinions that you hold? · 2019-10-28T13:26:30.300Z · score: 2 (2 votes) · LW · GW

The experts say that +2C will lead to increases in drought, flooding and cyclones affecting 100s of millions. Are they wrong?

Comment by tag on A simple sketch of how realism became unpopular · 2019-10-27T20:02:21.590Z · score: 1 (1 votes) · LW · GW

In particular, here’s Wood on why Kant is sometimes saying ‘we can’t know about the world outside our heads’, not just ‘we can’t have knowledge without relying on some conceptual framework or way of perceiving’ (p. 64):

Kant often distinguishes appearances from things in themselves through locutions like the following: “What the objects may be in themselves would still never be known through the most enlightened cognition of their appearance, which alone is given to us” (KrV A43/B60). “Objects in themselves are not known to us at all, and what we call external objects are nothing other than mere representations of our sensibility, whose form is space, but whose true correlate, i.e. the thing in itself, is not and cannot be cognized through them” (KrV A30/B45).

I dont think the quoted passage supports your claim at all. If the object "in itself" is the object outside of any conceptual or perceptual framework, then K. Is just that all knowledge occurs within a framework, and knowledge outside a framework is impossible.

I could, however, see how Kant could be read as denying any knowledge of external objects, under circumstances where no particular meaning is assigned to "in itself".

Comment by tag on George's Shortform · 2019-10-27T18:28:23.809Z · score: 1 (3 votes) · LW · GW

That's a very weak man form of redistribution. If you redistribute into services, such as public health and education, you avoid the inflationary problem, and since both are labour intensive, you can create jobs.

Comment by tag on A simple sketch of how realism became unpopular · 2019-10-27T18:03:14.467Z · score: 1 (1 votes) · LW · GW

We can’t know things about ultimate reality without relying on initially unjustified knowledge/priors/cognitive machinery.”

I think that is actually two related problems.

1a. A general problem of founding knowledge.. the Epistemic Circle or Munchausen Trilemma...which applies to everything, even maths.

1b. The specifically Kantian problem, which is limited to knowledge of the external world.

This sliding is probably the real thing we have Kant to thank for, and the thing that’s made anti-realist strains so slippery and hard to root out;

I think it is pretty unfair to blame the whole problem on Kant. He is a much less clear writer than Berkely, but then he is expressing much more complex ideas. Also, a lot of the sliding is surely cause by non-philosophers summarising complex ideas into one line slogans.

Comment by tag on A simple sketch of how realism became unpopular · 2019-10-27T17:25:08.093Z · score: 1 (1 votes) · LW · GW

Physicalism does contradict the claim “there is no way to backtrack to a raw form”, if this is taken to mean we should be agnostic about whether things are (really, truly, mind-independently) physical

"Mind independent" and "physical" don't mean quite the same thing, and there are multiple ways of cashing out "physical".

I assert that the “raw form” of an electron, insofar as physics is accurate, is just straightforwardly and correctly described by physics;

What "raw" means is "absent any conceptual framework". Making that substitution, your statement is a clear contradiction,along the lines of "We have a way of knowing what an electron is outside of any conceptual framework, and that way is to use the conceptual framework known as physics".

But even given the correctness of the tautologous version of Kant, a version of physicalism can still be asserted as, for instance, "ideal physical science is the correct framework".

If by a ‘thing in itself’ we mean a thing standing outside any relation to our cognitive powers, then of course it seems impossible for us to know such things; perhaps it is even self-contradictory to suppose that we could so much as think of them.

Well, indeed. The question is: what follows from this truth being merely tautologous? One thing that follows is the falsehood of strong physicalism. (I believe that a solution to the hard problem of consciousness follows that in turn, but that is a topic for another day).

Comment by tag on I would like to try double crux. · 2019-10-27T11:21:00.708Z · score: 1 (1 votes) · LW · GW

You can’t prove a halting oracle exists inductively

How is that relevant? It is perfectly possible for a mathematical universe to be a form of Platonic realism.

The CUH would not have a halting oracle.

Which implies that the MUH might.

Comment by tag on Occam's Razor May Be Sufficient to Infer the Preferences of Irrational Agents: A reply to Armstrong & Mindermann · 2019-10-27T11:06:25.592Z · score: 1 (1 votes) · LW · GW

After the experiment has been performed, the counterfactual is now actual, but it was a counterfactual beforehand. Even if you take the view that everything is determined, experiments are still exploring logical counterfactuals. On the other hand, if you assume holism, then you can't explore counterfactuals with experiments because you can't construct a complete state of the universe.

Comment by tag on A simple sketch of how realism became unpopular · 2019-10-26T13:54:46.166Z · score: 3 (2 votes) · LW · GW

I'd like to put forward a lukewarm defence of Kant as a friend of science.

Kant is on board with the idea that common sense observations like "refrigerators exist", or a car has four wheels" are true... enough.. In context. He calls it "emprical realism". He summarises his system as "empirical realism and transcendental idealism". Not as "you can't know anything".

Empirical realism means perceptions are based on information entering the mind via the sense organs. Kant is in agreement with basic scientific realism on this point. He also emphasised the realist side of his philosophy by adding a "Refutation of Idealism" to the second version of the Critique of pure Reason.

Transendental idealism means all specific human perceptions are moulded by the general form of human perception and there is no way to backtrack to a raw form. We can concede that humans and bats perceive the world differently.. although Kant goes further than most.

The kind of knowledge he says you can't have is knowledge of the thing in itself, which in modern terms would mean something like knowledge that is not relative to some conceptual framework or way of perceiving. Physicalism doesn't refute that in the least, because it is explicitly based on using physical science as its framework.

Why does Kant include Transcendental Idealism and not go for 100% realism? Because he is trying to rescue science from some philosophical criticisms! Berkely and other idealists attacked the occult, invisible nature of Newtonian gravity and absolute space. It seemed an embarassment for empirical science to be based on invisible things. Additionally, Hume attacked the cause-effect relation as indiscernable from mere temporal succession

Transcendental Idealism is a stepping stone to the central point of Kants system, the argument that space and time and causality and substance are categories that the human mind uses to organise its sense data, not external realities. He described this radical view with a term borrowed from science: a Copernican Revolution.

Kant was impressed by Newton's physics (and lectured on astronomy himself). He believed that nothing less than a necessary connection between cause and effect would do justice to Newton's clockwork determinism. His contemporaries were happy to accept that logical implications were necessary, but Hume had argued that fire did not imply smoke.

To meet this difficulty, Kant restructures the traditional logical/empirical distinction into a fourfold grid of analytical/synthetic times apriori/a posteriori, and places causality in the "synthetic apriori" quadrant. By any account, something is necessary if it true in all possible cases. Kant tries to restore the necessity of causality by arguing that cause and affect (and the other categories of transcendental idealism) must hold on all possible cases of human perception.

So the idealistic aspects of Kants are there to support the science of his day against sceptical attacks, and are accompanied by realistic elements.

Whether his strange and ingenious scheme is still necessary is very open to debate. Neither physical determinism nor logical necessity are as important as they were; our notion of empiricism includes a strong element of abduction, or influence to the best explanation; we end to have multiple theories rather than different paradigms; and we have become comfortable with invisible entities.

Nonetheless, some of Kant's ideas remain open question, for instance the ontological status of time.

Comment by tag on A simple sketch of how realism became unpopular · 2019-10-26T13:39:46.371Z · score: 1 (1 votes) · LW · GW

Wood thinks Kant was just really confused and fuzzy about his own view, and didn’t have a consistent model here (pp. 63-71).

Maybe confused, maybe nuanced, but certainly not backing some clear but extreme view.

Comment by tag on What's your big idea? · 2019-10-26T13:33:33.153Z · score: 3 (2 votes) · LW · GW

Schools teach too much, not too little

It's hard not to, when you don't know what people are going to end up doing. If you know that the son of the blacksmith is going to be a blacksmith, the problem gets much simpler.

Comment by tag on Artificial general intelligence is here, and it's useless · 2019-10-26T11:50:23.638Z · score: 0 (2 votes) · LW · GW

Think of “possible” here as meaning “possible to formalize”

Why not say “possible to formalize”, if that is what is meant?

It is tempting to associate intelligence with optimisation, but there is a problem. Optimising for one thing is pretty automatically optimising against other things, but AI theorists need a concept of general intelligence -- optimising accross the board.

AIXI, as a theoretical concept. is a general optimiser, so if AIXI is possible, general optimisation is possible. But AIXI isn't possible in the relevant sense. You can't build it out of atoms.

If AIXI, is possible, the "Artificial general optimisation" would make sense. Since it is not possible, the use of optimisation to clarify the meaning of "intelligence" leaves AGI=AGO as contradictory concept, like a square circle.

An agent (humans included) is “intelligent” only to the degree that it approximates AIXI.

Which brings in the the other problem with AIXI. Humans can model themselves, which is doing something AIXI cannot.