Why are we not starting to map human values? 2014-01-22T14:03:45.990Z
Definition of AI Friendliness 2013-09-11T14:55:58.368Z


Comment by djm on The AI in Mary's room · 2016-05-25T03:28:30.665Z · LW · GW

Interesting thought experiment. Do we know an AI would enter a different mental state though?

I am finding it difficult to imagine the difference between software "knowing all about" and "seeing red"

Comment by djm on Crazy Ideas Thread - October 2015 · 2015-10-09T05:32:27.337Z · LW · GW

Anyone have an AI (narrow or AGI) that can learn and respond to quizzes?

There is a new Kaggle competition out which you may be interested in

Comment by djm on AI: requirements for pernicious policies · 2015-07-17T16:28:57.925Z · LW · GW

whether the AI can test these policies. Even if the AI can find pernicious policies that rank high on its objective function, it will never implement them unless it can ascertain this fact

I don't know if the AI should be taking responsibility for testing its own policies, especially in the initial stages. We should have a range of tests that humans apply that the formative AI runs on each iteration so that we can see how it is progressing.

Comment by djm on Examples of AI's behaving badly · 2015-07-16T23:48:16.127Z · LW · GW

This is the fundamental problem that is being researched - the top layer of abstraction would be that difficult to define one called "Be Friendly".

Instead of friendly AI maybe we should look at "dont be an asshole" AI (DBAAAI) - this may be simpler to test and monitor.

Comment by djm on Examples of AI's behaving badly · 2015-07-16T14:42:20.888Z · LW · GW

These are all task specific problem definition issues that occurred while fine tuning algorithms (but yes they do show how things could get out of hand)

Humans already do this very well, for example tax loopholes that are exploited but are not in the 'spirit of the law'.

The ideal (but incredibly difficult) solution would be for AI's to have multiple layers of abstraction, where each decision gets passed up and is then evaluated as "is this really what they wanted", or "am I just gaming the system".

Comment by djm on Open Thread, Jun. 8 - Jun. 14, 2015 · 2015-06-13T05:08:30.409Z · LW · GW

Actually I think it is me not seeing them. Some do have the date at the top header, like

But most don't, nor in the footer or at the end of the paper.

I realise now I was looking in the wrong spot - papers like this have the date in the bottom left of the first page. Checking other PDF's shows the same thing, so I assume that is one of the standards?

Comment by djm on Open Thread, Jun. 8 - Jun. 14, 2015 · 2015-06-12T04:23:37.676Z · LW · GW

Why is the date or year of publication usually missing from PDF versions of research publications?

Is this a convention, perhaps specific to certain fields? I find it frustrating at times and am curious as to the reason behind it.

Comment by djm on Learning Optimization · 2015-04-28T04:05:23.492Z · LW · GW

In terms of books, there are a couple of recommended lists:

I dont have any recommendations for books, but the course "" is very good. Even if it has finished you can join and watch / download the lectures.

Comment by djm on Un-optimised vs anti-optimised · 2015-04-17T04:55:54.118Z · LW · GW

Would minimising the number of CPU cycles work as a lazy incentive.

This assumes that lesser CPU cycles will produce an outcome that is satisified rather than optimised, though in our current state of understanding any optimisation routines take a lot more computing effort than 'rough enough' solutions.

Perhaps getting the AGI's to go Green will kill two birds with one stone.

Comment by djm on LessWrong experience on Alcohol · 2015-04-17T04:44:12.175Z · LW · GW

I love the taste of bitter flavors - beer, coffee, olives, etc but not strong alcohol.

I also love spicy foods / curries - for those with low tolerance of bitter flavours, do you avoid these foods as well?

Comment by djm on Lessons from each HPMOR chapter in one line [link] · 2015-04-10T02:04:04.500Z · LW · GW

Thanks for this - I am try to be strict with my time at the moment so have not yet read HPMOR, so a nice succinct overview is useful.

I am a little curious about chapter 97 and getting a lawyer though.

Comment by djm on [link] Thoughts on defining human preferences · 2015-04-01T01:34:22.107Z · LW · GW

My personal hypothesis is that all human preferences are transient preferences, produced as flashes of positive or negative affect towards some mental concept.

I like that. People do change preferences, a lot - there was that [not very accurate] quote saying (US Centric) along the lines of "If you under 25 and vote republic, you have no heart" "If you over 25 and vote liberal you have no brains"

The most difficult part of this is that people have ingrained beliefs and preferences that will make them unhappy if the other side is picked - rational or not, we cant pick preferences that make all people happy.

For example

Group1 prefers more taxes for social services and hates social injustice

Group2 hates higher taxes

So for Group1 to be happy the Tax_rate should be between 0.2 -> 0.4

but this makes Group2 unhappy as their preferred tax rate is between 0.05 -> 0.15

There is no value in tax_rate that makes all groups happy.

Even if this were solved with infiniate social services and zero tax rates the million other disparate preferences - whether rational or not would cause bigger issues (religeous | athiest, vi | emacs, etc)

Comment by djm on Models as definitions · 2015-03-27T12:18:31.179Z · LW · GW

Even without an AI, the current trend may well have a world where there is a blurring of real Football matches and simulations.

Certainly you can’t keep an AI safe by using such a model of football

I used to think that a detailed ontological mapping could provide a solution to keeping AI's safe but have slowly realized that it probably isn't likely to work overall. It would be interesting to test this though for small specifically defined domains (like a game of football) - it could work, or at least it would interesting to make a toy experiment to see how it could fail.

Comment by djm on Intelligence modules · 2015-03-24T00:14:39.340Z · LW · GW

It would make sense to have a modular approach - do you have any suggestions on the types of modules the AI might consist of (excluding the engineering type ones like NLP, Machine vision, etc).

The type of modules an AI would consist of would depend on how it is actually implemented.

Comment by djm on PredictIt, a prediction market out of New Zealand, now in beta. · 2015-03-16T12:41:10.984Z · LW · GW

I tried to sign up but it fails with "not available in my location" error.

I am from Australia - is that a true error message or a glitch?

Comment by djm on Detecting agents and subagents · 2015-03-11T11:15:35.926Z · LW · GW

This in an interesting article, though necessarily abstract - how can we take this to implement an actual AI detector?

This could be the combination of research in the areas of antivirus software, network detection and intrusion, stock market agents along with some sort of intelligence honeypot (ie some construed event / data that exists for a split second that only an AI could detect and act on)

almost all agents are "enemies" to resource gathering agents, as they use up precious resources

If this were a friendly AI, shouldn't the utility function of the gathering agent take sharing / don't be greedy into account. Though as AI development advances we can't be sure that all governments / corporations / random researchers will put the necessary effort into making sure their intelligent agent is friendly.

Comment by djm on Assessors that are hard to seduce · 2015-03-10T12:30:37.260Z · LW · GW

That would likely work for initial versions of an AI, but I still cant help feeling that this is just tampering with the signal and that an advanced AI would detect this.

Would it not question the purpose of the utility function around detecting thermodynamic miracles - how would this work with its utility function to detect tampering or false data.

If I saw a miracle, I would [hope] my thinking would follow the logic below

a) it must be a trick/publicity stunt done with special effects b) I am having some sort of dream / mental breakdown / psychotic episode c) some other explanation I don't of

I don't think an intelligent agent would or should jump to "it's a miracle", and I would be concerned of its response if/when it does realise that it has been tricked all along.

Comment by djm on Human Capital Contracts · 2015-03-10T11:19:15.990Z · LW · GW

I like this idea, but it could be subject to abuse - a brilliant engineering student may get a large upfront finance then suddenly change course and decide to be part time artist.

Comment by djm on Assessors that are hard to seduce · 2015-03-10T11:02:46.427Z · LW · GW

I agree that useful behavior could come of this - religion has always been a very effective control mechanism.

The main criticism of religion is that it's untrue, and that a dedicated observer will realise this

Unfortunately, it would be a challenging problem to maintain this control over an increasingly intelligent AI.

Comment by djm on AI-created pseudo-deontology · 2015-02-13T05:09:39.551Z · LW · GW

I think the idea of having additional agents B (and C) to act as a form of control is definitely worth pursuing, though I am not clear how it would be implemented.

Is 'w' just random noise added to the max value of u?

If so, would this just act as a limiter and eventually it would find a result close to the original max utility anyway once the random noise falls close to zero?

Comment by djm on I played as a Gatekeeper and came pretty close to losing in a couple of occasions. Logs and a brief recap inside. · 2015-02-09T02:48:15.979Z · LW · GW

Stranger: You should be boxed.

Great response! That was interesting - felt a bit disturbing at times (not by the AI but by the human)

Nicely played on both sides.

Comment by djm on Slides online from "The Future of AI: Opportunities and Challenges" · 2015-01-16T14:03:59.161Z · LW · GW

Thanks for posting - looks like it was an impressive turnout. I would love to see some comments on any highlights / thoughts / conclusions that came about from that.

Comment by djm on Stupid Questions January 2015 · 2015-01-08T23:40:34.190Z · LW · GW

I dont think it is a problem to have few connections. Everyone starts with zero connections at the start, and many (if not most?) people only use it occasionally.

I think it will become more valuable in the future for employment, as it does provide a fairly easy 'living resume', that potential employers can see; so make your work history well written and polished (as it, treat it exactly like a resume).

Comment by djm on Why do you really believe what you believe regarding controversial subjects? · 2015-01-04T23:15:42.680Z · LW · GW

I think there are several levels to this question:

  1. Any issue based on Politics or Religion will have experts on both sides arguing for their case based on their own personal choice and biases - these are more often than not, arguments and discussions rooted entirely on opinion.

  2. For controversial issues in Science and Economics I will try to determine if there are any biases based on point 1. In some cases this becomes visible based on the political leanings of the Scientists and / or corporate sponsorship of the study.

  3. In the Maths world there doesn't seem to be any controversial issues over the results (which is why I kind of like maths a lot!), however there are many issues on which approach to take and how useful a result is. For those sorts of predictive issues I don't usually have a strong opinion

So in general, it comes down to what the biases are. Everyone has an opinion, and these are (or ought to be) less relevant the deeper you get into the Science and Maths.

Comment by djm on Open thread, Dec. 29, 2014 - Jan 04, 2015 · 2014-12-30T13:39:29.652Z · LW · GW

Instead of using a name like a_lurker and asking for PM's I would suggest going trying to be a little more public - your goal should be to display to potential employers that you can code. This is actually harder than it sounds, as programmers (especially self taught) are more likely to be introverted and don't like marketing themselves. [from my personal perspective]

Some suggestions would be:

  1. Pick some sort of professional sounding name for yourself (doesn't have to be a business name) that you want to be known as - better if it is rare on google. You will use this name to promote your knowledge and collaboration on many websites

  2. Register a domain - even if it is a .info with a simple About you page, saying you are looking for work and your resume. This website should be in your email signature and plugged on other site below.

  3. Start a github account (learn git first) and publish something - anything that you think was good code [as long as it isn't the answer to any of your course assignments]

  4. Answer questions on Stackoverflow with your professional name - and ask questions. Don't spam it, but don't be afraid to ask stupid questions.

  5. Get a linkedin account and grow your network there

Happy to answer any PM's you have, but you should think about promoting yourself if you want work.



Comment by djm on How many words do we have and how many distinct concepts do we have? · 2014-12-18T02:27:37.596Z · LW · GW

This is not an unreasonable question to ask, and I believe you are referring to an Ontology. OpenCyc is probably the best one of many that does this, though it has its own criticisms.

There are many other ontologies and I did a short summary when I was trying to find an answer the same question myself.

Comment by djm on [LINK] Steven Hawking warns of the dangers of AI · 2014-12-04T07:39:02.328Z · LW · GW

There has certainly been increased general media coverage lately , and MIRI was mentioned in the Financial Times recently.

Comment by djm on MIRI Research Guide · 2014-11-10T02:31:17.306Z · LW · GW

Thanks for this - I particularly like the suggested ordering.

Is this designed to replaced the course list , or is the course list more for future tasks in that it has some actual programming subjects?

Comment by djm on In the grim darkness of the far future there is only war continued by other means · 2014-10-22T09:49:33.023Z · LW · GW

Very interesting article. The level of PVP could be related to sports in terms that the top players (Olympics) are the best of the best but most of us are happy playing weekend sports or even watching it.

Comment by djm on Can AIXI be trained to do anything a human can? · 2014-10-21T13:02:15.014Z · LW · GW

Therefore, they cannot identify "that computer running the code" with "me", and would cheerfully destroy themselves in the pursuit of their goals/reward.

I am curious as to why an AIXI like entity would need to model itself (and all its possible calculations) in order to differentiate the code it is running with the external universe.

The human in charge of a reward channel could work for initial versions, but once its intelligence grew wouldn't it know what was happening (like the box AI example - not likely to work in the long term).

Comment by djm on Open thread, Sept. 29 - Oct.5, 2014 · 2014-10-01T09:18:48.598Z · LW · GW

My general rule of thumb is

if customer.dictates('specific language')
        use('specific language') # usually Java / PLSQL / .Net
except error('too slow')

I love the simplicity and power of Python and will use it to prototype proof of concepts (not so much GUI work - would use HTML or .NET for that). For me, Python really makes programming a lot of fun again and though it is slower, I haven't yet had the need to shell out to C code though I expect I will soon.

Comment by djm on How realistic would AI-engineered chatbots be? · 2014-09-11T23:35:32.540Z · LW · GW

How would you feel about talking to a realistic AI-engineered chatbot?

Comment by djm on Another type of intelligence explosion · 2014-08-22T00:03:13.231Z · LW · GW

Maybe a form of unit testing could be useful? Create a simple and not so simple test for a range of domains and get all AI's to run them periodically.

By default the narrow AI's would fail even the simple tests in other domains, but we would be able to monitor if / as it learns other domains.

Comment by djm on The metaphor/myth of general intelligence · 2014-08-20T03:30:49.213Z · LW · GW

Automatic construction of general interfaces would be tricky to say the least. It would surely have to depend on why agentA needs to interface with agentB in the first place - for general data transfer (location , status, random data) it would be fine, but unless both agents had the understanding of each others internal models/goals/thought processes it seems unlikely that they would benefit from a transfer except at an aggregate level

Comment by djm on What should a friendly AI do, in this situation? · 2014-08-09T03:37:35.135Z · LW · GW

Good question. You may think it would be a better overall outcome to show the manipulative one to shock the programmers into breaking the law to (possibly) halt the other AI, but then it is no longer an FAI if it does this.

Training an FAI should be kept free from any real world 'disaster scenario' that it may think it needs more power to solve, because the risk it itself becomes an UFAI is amplified for many reasons (false information for one)

Comment by djm on LINK-Robot apocalypse taken (somewhat) seriously · 2014-08-05T04:10:31.745Z · LW · GW

I remember less than a year ago thinking (and posting) the same thing. Until you actually think about it, it does appear from the outside, to be a relatively easily solvable problem.

I still believe It will have to be done, one way or another, though I have no idea how it will be done.

Comment by djm on Jokes Thread · 2014-07-24T12:41:12.415Z · LW · GW

Q: Why did the AI researcher die?

A: They were giving a live AI demo and while handing out papers, said "Damn, there are never enough paperclips - I wish I would never run out"

Comment by djm on Open thread, July 21-27, 2014 · 2014-07-23T04:30:42.409Z · LW · GW

Good idea - it is also worth having a scan of the complete list of commands every few years. I did this after a "friend" spent half a day writing a C program to do something that was already an inbuild command - live and learn!

Comment by djm on Open thread, July 21-27, 2014 · 2014-07-22T06:00:56.281Z · LW · GW

The ones I will probably take next are:

Game Theory - good course according to reviews

Introduction to Logic - I have done a similar course on this but need more practice in this area.

and Exploring Neural Data - no idea what this one is like, but seems interesting. You get to play with some data in Python

Comment by djm on Confused as to usefulness of 'consciousness' as a concept · 2014-07-11T15:06:49.487Z · LW · GW

Interesting post - while I don't have any real answers I have to disagree with this point:

"Why do you think your computer is not conscious? It probably has more of a conscious experience than, say, a flatworm or sea urchin. (As byrnema notes, conscious does not necessarily imply self-aware here.)"

A computer is no more conscious than a rock rolling down a hill - we program it by putting sticks in the rocks way to guide to a different path. We have managed to make some impressive things using lots of rocks and sticks, but there is not a lot more to it than that in terms of consciousness.

Comment by djm on Tools want to become agents · 2014-07-05T14:26:19.591Z · LW · GW

Can you list the 6 working AGI projects - I'd be interested but I suspect we are talking about different things.

Comment by djm on Tools want to become agents · 2014-07-04T15:31:33.799Z · LW · GW

I'm curious - who are the AI and AGI designers- seeing one hasn't been publicly built yet. Or is this other researchers in the AGI field. If you are looking for feedback from a technical though not academic, I'd be very interested in assisting.

Comment by djm on Open thread, 23-29 June 2014 · 2014-06-23T14:40:15.009Z · LW · GW

Good point, I should learn it anyway. But in terms of learning and solving problems, do you work them out using LaTex or do you use pen and paper / whiteboard?

Comment by djm on Open thread, 23-29 June 2014 · 2014-06-23T14:36:13.911Z · LW · GW

That technique would be beyond me at this stage - I have done courses in Calculus, Linear Algebra and Logic and can finally understand most of the syntax and flow in research papers, though I still don't feel at all competent, seeing that I feel I could not recreate the proofs they come up with.

I think that is my issue - I can read lots of maths, I can do the exercises but I am not sure how to go about 'doing something real' - ask me to write any software you can think of and I can do that, but I feel I am missing some fundamental point / learning in maths. Or maybe I am over thinking this, and haven't had a concrete problem to play with.

Comment by djm on Open thread, 23-29 June 2014 · 2014-06-23T11:09:17.028Z · LW · GW

What are some ways to effectively practice and apply Maths learning?

I've been doing a lot of learning and found that practicing on paper is generally easier than Latex or a cobbled syntax in electronic documents, but would like to know if I just really need to bite the bullet and do it this way (Latex or similar).

Once I have (what I believe) an understanding of the problem types, I will generally write code to do it for me as doing this makes it even clearer in my head. Problem is though, once I do the code, I generally don't practice on paper anymore and I am not sure if this is going to be hindrance in understanding more complex topics.

My end goal is to be able to read and understand the maths in any AI focused research paper, and then be able to do some maths which isn't just practice examples but I am not sure on how to get to that last step.

Comment by djm on How do you take notes? · 2014-06-23T10:15:35.339Z · LW · GW

What are your strategies of taking notes for new information?

Start by reading random articles -> wiki -> papers / books Decide how complex it is likely to be for me and how far deep I want to learn the topic (e.g. how useful is this to me for really learn this)

Do you take any notes on paper? If so do you scan them or otherwise digilatize them?

Used to do notes on paper in bound books, a book per area (ideas, programs, building, etc) but never digitized them. For the last few years I use Google docs - like evernote, easy to search, write and read anywhere.

I still do Maths problems of paper as it seems quicker to scribble and draw diagrams / charts - would love to know a digital option though that worked as easily as pen.

Do you have specific strategies for deciding which information to write down?

New document for each topic to learn , or application of knowledge. e.g. document "Learn maths" - progress of maths courses , "AI applications" = how to actually build something.

The usual structure of a document is

*Overview - what is it, what do I want to learn

*Progress - what do I need to do, current tasks, upcoming tasks

*Application of Knowledge - ways to apply the knowledge - for me, usually software / website. I feel I need to actually apply new knowledge otherwise it doesn't stick after a few years. This section often has links to other project documents

*Notes - sub headings as needed, usually a 2-3 sentence summary of the area, links and the actual content contains things I didn't know / should learn / not obvious

*Log of Events - dated entry of when I read, learn or apply the knowledge

How do you write notes to capture all important information?

I don't - I only write down things I don't know, paragraphs usually with links to appropriate sources for deeper reminders. I find actually typing in my own words helps best for me to learn.

Do you tag your notes?

clear use of of document titles, standard sections, make sure right keywords are in the document (anywhere) and Google makes it very easy to get to the right document

If you use Evernote, or a similar system how private are your notes? Would you allow friends to read in them? Your spouse?

All notes are private by default, but some code I write as part of applying the learning is public e.g. - github. My spouse isn't at all interested in Maths or AI but would happily share if she wanted to read them

Comment by djm on Come up with better Turing Tests · 2014-06-11T04:36:16.979Z · LW · GW

You could simply ask it Implement a plan to maximize the number of paperclips produced.

If the answer involves consuming all resources in the Universe, then we can assume it is an AI

If the answer is reasonable and balanced, then it is either a person or a Friendly AI, in which case it doesn't matter

Comment by djm on Finding LessWrongers on LinkedIn · 2014-05-14T04:28:41.037Z · LW · GW

Great idea - I have followed the groups and MIRI on LinkedIn, and already follow several other AI related groups. It is worth looking at Google+ as well - I have found this to have a higher quality of discussions than LinkedIn / Facebook

Comment by djm on Calling all MIRI supporters for unique May 6 giving opportunity! · 2014-05-07T13:54:09.844Z · LW · GW

Yes that must have been a glitch - I tried at the start to donate but it didn't work. 4 hours later when I tried it was fine, and again at the end.

Comment by djm on Open Thread, May 5 - 11, 2014 · 2014-05-06T00:22:55.332Z · LW · GW

Anyone else doing the course Functional Programming Principles in Scala ? It started last week, but still should be time to join and get the first assignment done.