Call for new SIAI Visiting Fellows, on a rolling basis

I don't feel like having this discussion in public, but Anna's invitation is framed in broad enough terms that I'll be getting in touch.

Thanks for the heads up. Oddly enough, it was sitting in the spam filter on my SIAI account (without making it through forwarding to my gmail account, where I was checking the spam filter). Yours was the only message caught in the SIAI spam filter, out of 19 who emailed so far in response to this post.

Did you have special reason to expect to be caught in a spam filter?

I wonder how long I can expect to wait before receiving a response. I sent my email on Wednesday, by the way.

Yup. The logic at the time went something like, "I want something that will be reasonably fast and scale to lots of multiple processors and runs in a tight sandbox and has been thoroughly debugged with enterprise-scale muscle behind it, and which above all is not C++, and in a few years (note: HAH!) when we start coding, Java will probably be it." There were lots of better-designed languages out there but they didn't have the promise of enterprise-scale muscle behind their implementation of things like parallelism.

Also at that time, I was thinking in terms of a much larger eventual codebase, and was much more desperate to use something that wasn't C++. Today I would say that if you can write AI at all, you can write the code parts in C, because AI is not a coding problem.

Mostly in that era there weren't any good choices, so far as I knew then. Ben Goertzel, who was trying to scale a large AI codebase, was working in a mix of C/C++ and a custom language running on top of C/C++ (I forget which), which I think he had transitioned either out of Java or something else, because nothing else was fast enough or handled parallelism correctly. Lisp, he said at that time, would have been way too slow.

I can't understand how it could ever have been part of his thinking. (Java was even worse years ago!)

Anna, and in general the Vassarian lineage, are more effective cooperators than I am. The people who I have the ability to cooperate with, form a much more restricted set than those who they can cooperate with.

It does kinda give the impression that a) donors and b) programmers are all that SIAI has a use for, though.

I once had that impression too, almost certainly in part from SYWTBASAIP.

It does kinda give the impression that a) donors and b) programmers are all that SIAI has a use for, though.

FWIW, I and almost everyone outside SIAI who I've discussed it with have had this misconception; in my case, SYWTBASAIP did support it.

I've set up a SIAI account on LibraryThing, for a bunch of reasons even though I've not heard back from MBlume.

http://www.librarything.com/catalog/siai

The heuristic "it's easier to seek forgiveness than permission" seemed to apply, the upvotes on the comments below indicate interest, I wanted to separate my stuff from SIAI's but still have a Web 2.0-ish way to handle it, and information wants to be free.

If this was a mistake on my part, it's easily corrected.

Re anonym's comment, maybe you might like to set up a SIAI/LW LibraryThing account. I'll gladly donate the $25 to make it a lifetime account.

I like how the Wii games are included

Thanks !

If it's an option, please make the spreadsheet sortable. It would be much easier to browse if it were sorted by (location, creator), so all math books would be together, and books by the same author on the same topic would be together.

Thanks for making this available though. I enjoyed browsing and already bought one. You might consider putting Amazon links in there with an affiliate tag for SIAI.

I'm using (at Andrew Hay's kind suggestion) Delicious Library 2 (sadly only available on Macintosh, but I happen to be an Apple fanboy) which integrates with my webcam to do all my barcode scanning for me.

Upvoted. I don't understand what difference is meant.

Eliezer, it makes me nervous when my behavior or reasoning differs from the vast majority of human beings. Surely that's a reasonable concern?

On this planet? No. On this planet, I think you're better off just worrying about the object-level state of the evidence. Your visceral nervousness has nothing to do with Aumann. It is conformity.

Knowing that people are crazy and the world is mad helps a bit, but not too much because people who are even crazier than average probably explain their disagreements with the world in exactly this way.

What do you care what people who are crazier than average do? You already have enough information to know you're not one of them. You care what these people do, not because you really truly seriously think you might be one of them, but because of the gut-level, bone-deep fear of losing status by seeming to affiliate with a low-prestige group by saying something that sounds similar to what they say. You may be reluctant to admit that you know perfectly well you're not in this group, because that also sounds like something this low-prestige group would say; but in real life, you have enough info, you know you have enough info, and the thought has not seriously crossed your mind in a good long while, whatever your dutiful doubts of your foregone conclusion.

Seriously, just make the break, clean snap, over and done.

So, I'm inclined to try to find more detailed explanations of the differences. Is there any reason you can think of why that might be unproductive, or otherwise a bad idea?

Occam's Imaginary Razor. Spending lots of time on the meta-level explaining away what other people think is bad for your mental health.

I typically class these types of questions with other similar ones:

What are the odds that a strategy of approximately continuous insanity, interrupted by clear thinking, is a better evolutionary adaptation than continuous sanity, interrupted by short bursts of madness? That the first, in practical, real-world terms, causes me to lead a more moral or satisfying life? Or even, that the computational resources that my brain provides to me as black boxes, can only be accessed at anywhere near peak capacity when I am functioning in a state of madness?

Is it easier to be sane, emulating insanity when required to, or to be insane, emulating sanity when required to?

Given that we're sentient products of evolution, shouldn't we expect a lot of variation in our thinking?

Finding solutions to real world problems often involve searching through a space of possibilities that is too big and too complex to search systematically and exhaustively. Evolution optimizes searches in this context by using a random search with many trials: inherent variation among zillions of modular components. I hypothesize that we individually think in non-rational ways so that as a population we search through state space for solutions in a more random way.

Observing the world for 32-odd years, it appears to me that each human being is randomly imprinted with a way of thinking and a set of ideas to obsess about. (Einstein had a cluster of ideas that were extremely useful for 20th century physics, most people's obsessions aren't historically significant.)

Interesting. I wonder if you've misinterpreted me or if there's actually someone competent out there? Quick example if possible?

Ancient philosophy would look very different if the Greek philosophers had been making stuff up at random. Plato and Aristotle followed cognitive strategies, strategies that they (1) could communicate and (2) felt constrained to follow. For these reasons, I don't think that those philosophers could be characterized in general as "making stuff up".

Of course, they followed different strategies respectively, and they often couldn't communicate their feelings of constraint to one another. And of course their strategies often just didn't work.

Your own answer is also pretty relevant to FAI. Because anything that confuses you can turn out to contain the black box surprise from hell.

Until you know, you don't know if you need to know, you don't know how much you need to know, and you don't know the penalty for not knowing.

Thanks.

I'll try and explain a bit more why I am not very interested in probabilities and DTs. I am interested in how decisions are made, but I am far more interested in how an agent gets to have a certain model in the first place (before it is converted into an action). With a finite agent there are questions such as why have model X rather than Y. Which I think impinges on the question on what topics we should discuss. I'd view most people not having a low probability that DTs are important, but simply not storing a probability for that preposition at all. They have never explored it so have no evidence either way.

The model of the world you have can dominate the DT, in determining the action taken. And in the end that is what we care about, the action taken in response to the input and history.

I also think that DT with its fixed model ignores the possibility of communication between the bit running through the model and picking an action and the bit that creates the model. For example if I see a very good contest/offer I might think it too good to be true, and look for more information to alter my model and find the catch before taking the offer up.

It would be nice if we could get both groups (international olympiads and Google) reading relevant articles, and thinking about rationality and existential risk. Any thoughts here, alexflint or others?

Thinking about this point is leading me to conclude that Google is substantially more likely than SIAI to develop a General AI before anyone else. Gintelligence anyone?

Goertzel, Voss, and similar folks are not working on the FAI problem. They're working on the AGI problem. Contrary to what Goertzel, Voss, and similar folks find most convenient to believe, these two problems are not on the same planet or even in the same galaxy.

I shall also be quite surprised if Goertzel's or Voss's project yields AGI. Code is easy. Code that is actually generally intelligent is hard. Step One is knowing which code to write. It's futile to go on to Step Two until finishing Step One. If anyone tries to tell you otherwise, bear in mind that the advice to rush ahead and write code has told quite a lot of people that they don't in fact know which code to write, but has not actually produced anyone who does know which code to write. I know I can't sit down and write an FAI at this time; I don't need to spend five years writing code in order to collapse my pride.

The arc of Less Wrong read start to finish should be sufficient for an intelligent person to discard existing AGI projects - once your "mysterious answer to mysterious question" detector is initialized and switched on, and so on - so I consider my work of explanation in that area to be pretty much done. Anything left is public relations, taking an existing explanation and making it more persuasive.

. . . Java is slower then C++ because of all the overheads of running the code. . . .

A fast programming language is the last thing we need. Literally--when you're trying to create a Friendly AI, compiling it and optimizing it and stuff is probably the very last step.

(Yes, I did try to phrase the latter half of that in such a way to make the former half seem true, for the sake of rhetoric.)

If that's the case why does Ben Goertzel have a company working on AGI the very problem your trying to solve? Why does he actually have design and some portions implemented and you do not have any portions implemented?

He is solving a wrong problem (i.e. he is working towards destroying the world), but that's completely tangential.

Java is slower then C++ because of all the overheads of running the code.

Those damnable overheads. Assembly language FTW!

Just an aside for some posters above this post who have been talking about Java as the superior choice to C++ what planet do you come from? Java is slower then C++ because of all the overheads of running the code.

A world in which a segfault in an FAI could end it.

Oh, hell yeah. Anna's side can recruit them, no problem. And I'm certainly not saying that no one who works at these organizations could make the cut for the Final Programmers. Just that you can't hire Final Programmers at random from anywhere, not even Google.

He is at Berkeley working under Stuart Russell (of AI: A Modern Approach, among other things).

Flight is based on some pretty simple principles, intelligence not necessarily so.

I think the analogy still maps even if this is true. We can't build useful AIs until we really understand intelligence. This holds no matter how complicated intelligence ends up being.

If intelligence turns out to be fundamentally complex then emulating a physical brain might be the easiest way to create AI.

First, nothing is "fundamentally complex." (See the reductionism sequence.) Second, brain emulation won't work for FAI because humans are not stable goal systems over long periods of time.

Re: "Flight is based on some pretty simple principles, intelligence not necessarily so. If intelligence turns out to be fundamentally complex then emulating a physical brain might be the easiest way to create AI."

Hmm. Are there many more genes expressed in brains than in wings? IIRC, it's about equal.

Intelligence is defined in terms of response to a variable environment - so you just use an environment with a wide range of different problems in it.

In practice, enough of us retain online library access through our former universities that we can reach the articles we need reasonably easily. Almost everything is online.

If this ceases to be the case, we'll probably buy library privileges through Stanford, San Jose State, or another nearby university.

"We increasingly have a useful, though informal, career network for folks interested in earning and donating"

I wonder if people will soon start applying to this SIAI program just as a means to get into the Silicon Valley job market?

Not that it would be a negative thing (great instead!), if they anyway are useful while in the program and perhaps even value the network they receive enough to donate to SIAI simply for keeping it...

If someone is an "edge case", would it ever make a difference that they could cover part of the costs (normally covered by SIAI) with their own funds?

Yes, it might, although covering one's own costs is no guarantee (since space and our attention is limited), and unusually large expenses (e.g., airfare from Australia, or being really attached to having one's partner with one) is not a reason not to apply.

If someone's serious about reducing risks and wishes to cover their own expenses, though, coming for a short visit (1 to 7 days, say) is almost certainly an option. We have a guest room and can help people hash out conceptual confusions (you don't need to agree with us; just to care about evidence and accuracy) and think through strategy for careers and for reducing existential risks.