Open and Welcome Thread – July 2021

Hey guys I’m Sam. You may have read my post on the obesity epidemic. I studied econ, although I’d say I’m really only qualified to assert any expertise at all when it comes to financial markets. I’ve been on reddit for what, 10 years and have thus stumbled on here from time to time.

From me, you’ll probably see a lot of trying to figure out just what in the world is going on in a complex system, and then throwing my hands in the air. Also, you may see me going to logical extremes (“how about we model the universe as one atom going in a straight line the void”) both of these are habits from econ that die hard.

I have a particular focus on insightful and novel ideas (sometimes, i even have them), and I have a strong tendency to prefer what is simple and elegant as an explanation for nearly anything. You’ll probably find that i have a very detached style; this gets me in trouble in real life but this is the only place where it seems to go over well.

I need some advice.

A little context: I'm a CSE undergraduate who'll graduate next July.  I think AI Safety is what I should be working on.  There are, as far as I've seen, no opportunities for that in my country.  I don't know what path to go down in the immediate future.

Ideally, I'd begin working in Safety directly next year.  But I don't know how likely that is, given I don't have a Master's degree or a PhD; and MIRI's scaling back on new hires, as I understand it (I thought about interning after I graduate, but I’m not sure if they’ll take interns next year).

I plan to apply to Master's programs anyway, but those are also a long shot - the tuition fees are even steeper when converted to other currencies, so I don't want to apply to programs that aren't worth it (it's possible my qualifications are sufficient for some of the ones I'll apply to, but I have little context to tell).

I could work in software for a couple years, trying to do independent research in that time, to switch over after.  This is complicated both by that independent research is shockingly difficult when you can't bounce ideas off of someone who gets it (I don't really know anyone who does at the level where this is viable), and that I'll need to spend a non-insignificant amount of time and effort now to get a decent chance of getting a great job (I really don't want to work at a job I both don't believe in and that doesn't build career capital), time that could be better spent, especially if my chances aren't good even in the end (again, I speak from my expectations given limited context).

I've been thinking seriously about this for a couple weeks from several angles (I tried to hold off on this until I had enough qualifications to make credible predictions about my chances, but now I think the bar for credible is much higher than I'd expected), and came to some answers, but also decided I needed to ask the opinion of someone who gets my motivations, and hopefully has better context for any of this than I do.  Both about the future, and the present.

Some additional info about what I think I'm good at, relative to an undergrad level, if that helps: I have a couple years of experience with frontend systems for web and mobile (although I've recently been told I should work on improving my code structure, since I learned it all on my own and have worked primarily on my own projects).  I understand ML theory (DL slightly more) to an extent (I have a preprint on cGAN image processing I’m trying to figure out how to publish, since my university really doesn’t help with this stuff, I welcome any advice on that too).  I also have some amount of experience tinkering with their code; while I doubt it reaches the level of familiarity even a new industry ML developer would have, I'm fairly confident that I could get there without much trouble (could be wrong, correct me if this isn't your experience).

I typically try to avoid making posts of this sort, but this is kinda sorta important to me, and I feel comfortable trusting the people here to help me a little in making the right call here.  So thanks for that.  And thanks in advance for any suggestions.

There is a situation I've thought of in which functional decision theory, according to my understanding of it, does poorly. I might just be making some sort of mistake, but I tried to be pretty careful when reading Eliezer's paper on functional decision theory, and it still seems to be a problem. I'm interested in what others think of this.

The situation is just like Newcomb's problem, except that player is a superintelligent AI who is aware of the exact mental makeup of the predictor (the one considering placing things in the boxes) and can infer the predictor's choices with what is effectively certainty. The predictor is just a regular, non-superintelligent creature that knows the superintelligence uses function decision theory and is superintelligent.

In this situation, there doesn't seem to be a logical connection between what the superintelligent outputs and the its prediction of what the predictor does. I mean, the superintelligence can exactly infer what the predictor does, without referencing its own action, so it doesn't seem like the superintelligence knowing what it itself does would really by informative.

So, suppose the superintelligence predicts the predictor puts money in both boxes, and would believe this prediction no matter what the superintelligence decides to do. In this situation, I don't see any reason for it to not take two boxes.

And the predictor reasons, "Since the AI can predict my exact output with effective certainty, eliminating a logical connection between its choice and the content of the box, the AI has no reason not to just take two boxes if they contain money. So, I predict the AI would two-box, so I won't put money in either box."

And then the superintelligent AI gets $0.

I don't think that's what a superintelligence is supposed to get. And if the AI didn't have the knowledge and power to predict the predictor's output exactly, then a logical connection could have be preserved and the AI could potentially get money. But I don't think more knowledge and more intelligence are supposed to make agents do worse.

"Can You Ever Be Too Smart for Your Own Good? Comparing Linear and Nonlinear Effects of Cognitive Ability on Life Outcomes", by Matt I. Brown, Jonathan Wai, Christopher F. Chabris, 2021 Mar 8. (Open preprint.)

In a word, no.

"We found no support for any downside to higher ability and no evidence for a threshold beyond which greater scores cease to be beneficial. Thus, greater cognitive ability is generally advantageous—and virtually never detrimental."

I have being reading content from LW sporadically for the last several years; only recently, though, did I find myself visiting here several times per day, and have made an account given my heightened presence. 

From what I can tell, I am in a fairly similar position to Jozdien, and am also looking for some advice.

I am graduating with a B.A. in Neuroscience and Mathematics this January. My current desire is to find remote work (this is important to me) that involves one or more of: [machine learning, mathematics, statistics, global priorities research]. 

In spirit of the post The topic is not the content [LW · GW], I would like to spend my time (the order is arbitrary) doing at least some of the following: discussing research with highly motivated individuals, conducting research on machine learning theory, specifically relating to NN efficiency and learnability, writing literature reviews on cause areas, developing computational models and creating web-scraped datasets to measure the extent of a problem or the efficacy of potential solution, and recommending courses-of-action (based on my assessments generated from the previous listed entity). 

Generally, my skill set and current desires lead me to believe that I will find advancing the capabilities of machine learning systems, quantifying and defining problem afflicting humans, and synthesizing research literature to inform action, all fulfilling, and that I will be effective in working on these things done as well. My first question: How should I proceed with satisfying my desires, i.e. what steps should I take to determine whether I enjoy machine learning research more than global priorities research, or vice versa? 

It is my plan to attend graduate school for one of [machine learning, optimization, computer science] at some point in life (my estimate is around the age of 27-30), but I would first like to experiment with working at an EA affiliated organization (global priorities research) or in industry doing machine learning research. I am aware that it is difficult to get a decent research position without a Master's or PhD, but I believe it is still worth trying for. I have worked on research projects in computational neuroscience/chemistry for one company and three different professors at my school, but none of these projects turned into publications. This summer, I am at a research internship and am about to submit my research on ensemble learning for splice site prediction for review in the journal Bioinformatics - I am 70% confident that this work will get published, with me as the first author. Additionally, my advisor said he'd be willing to work with me to publish a dataset of 5000 fossils image I've taken of various fossils from my collection. While this work is not in machine learning theory, it increases my capacity for being hired and is helping me refine my competence as a researcher / scientist.  

Several weeks ago, I applied to Open Philanthropy's Research Fellow position, which is a line of work I would love doing and would likely be effective at. They will contact me with updates on or before August 4th, and I anticipate that I will not be given the several follow-up test assignments OpenPhil uses to evaluate its candidates, provided that their current Research Fellows have more advanced degrees and more experience with the social sciences than I do. I have not yet applied to any organizations whose focus is machine learning, but will likely begin doing so during this coming November. This brings me to my final questions: What can I do to increase my capacity for being hired by an organization whose focus is global priorities research? Also, which organizations or institutions might be a good fit for both my skills in computational modeling and machine learning and my desire to conduct global priorities research?

Any other advice is welcome, especially advice of the form "You can better prioritize / evaluate your desires by doing [x]", "You seem to have [x] problem in your style of thought / reasoning, which may be assuaged by reading [y] and then thinking about [z]", or "You should look into work on [x], you might like it given your desire to optimize/measure/model things". Thank you, live well. 

I wonder if Shortform posts are replacing this, in terms of use:

If it’s worth saying, but not worth its own post, here's a place to put it.

Does anyone know an article that expands on the idea of separating teaching institutions and assessment? I wrote a short expansion (750~ words) myself, and will probably publish it soon, but I'd like to read articles by other people too if they exist.