Posts
Comments
Did you and GPT4 only output the moves, or did you also output the board state after each turn?
Unfortunately without speaker labels the YouTube transcript is less useful unless you're listening while reading.
Is there a transcript anywhere?
Another similar result was that AlphaFold was trained on its own high-confidence predictions for protein sequences with unknown structures:
The AlphaFold architecture is able to train to high accuracy using only supervised learning on PDB data, but we are able to enhance accuracy (Fig. 4a) using an approach similar to noisy student self-distillation35. In this procedure, we use a trained network to predict the structure of around 350,000 diverse sequences from Uniclust3036 and make a new dataset of predicted structures filtered to a high-confidence subset. We then train the same architecture again from scratch using a mixture of PDB data and this new dataset of predicted structures as the training data, in which the various training data augmentations such as cropping and MSA subsampling make it challenging for the network to recapitulate the previously predicted structures. This self-distillation procedure makes effective use of the unlabelled sequence data and considerably improves the accuracy of the resulting network.
I'm also dealing with chronic illness and can relate to everything you listed. I've been thinking that a discord server specifically for people with chronic illness in the rationality community might be helpful to make it easier for us to share notes and help each other. There are different discord servers for various conditions unaffiliated with the rationality community, but they tend to not have great epistemic standards and generally have a different approach than what I'm looking for. Do you have any interest in a discord server?
By forward do you mean sooner (shorter timelines) or later (longer, slower timelines)?
I tried giving this to GPT-3 and at first it would only give the tautological "pawns become more powerful" example, then I expanded the prompt to explain why that is not a valid answer, and it gave a much better response.
I believe this response is the same as your fourth bullet point example of a good answer.
Here's the prompt in copy/pastable format for anyone who wants to try playing with it:
Consider a new variant of chess, in which each pawn can move up to six squares forward on its first move, instead of being limited to one or two squares. All other rules remain intact. Explain how game balance and strategy is changed with this new variant of chess.
Your response should share something not immediately obvious about this variant and provide a plausible justification for why it might be true. Some responses that would not succeed would be
The pawns become more powerful. (Too simple, close to a tautology.)
New strategies will need to be developed. (Too vague.)
Bishops become more valuable. (Needs a justification for why we should expect this.)Response:
Agreed that it would be insanely impressive. It would probably convince me that a fast takeoff is very likely coming within the next 5 years. Yet I can't really say I'm more than 90% confident that GPT-4 won't be able to do it. Maybe 95%.
I'm not sure about that. See page 8 of the LamDA paper where they gave it access to a "toolset" including things like a calculator. I wouldn't be surprised if they gave GPT-4 access to similar tools including a way to access the current date.
There are probably less impressive things than this that it won't be able to do, but here's one prediction in which I am about 90% confident:
If you invent or find a board game of similar complexity to chess that GPT-4 has never seen before and explain the rules using only text (and, if GPT-4 is multimodal, also images), GPT-4 will not be able to perform as well at the game as an average human who has never seen the game before and is learning it for the first time in the same way. I say "perform as well as" rather than "beat" because, depending on the specific rules of the game, I expect GPT-4 will likely not output legal moves.
This prediction applies to whatever OpenAI reveals with the name GPT-4 or that is clearly the GPT-3 successor regardless of size, assuming it's revealed in the next two years and is not trained on specifically that task (e.g. by generating tens of thousands of unique synthetic board game examples with synthetic dialogues where they're explained and played over text).
I ran into another similar example in the last section of this post.
The hat is another example of prompt engineering for humans.
As another example, I remember somebody (maybe EY or Gwern?) suggest that prompting people with "how would somebody smarter than you solve this problem?" can actually be effective at soliciting better solutions than just asking them to solve it directly.
I don't remember where I saw this so if anybody has a link feel free to share.
This sounds similar to John Sarno -- are you familiar with it and do you know whether the approaches are substantially different?
This is deeply perverse, but three "risk factors" over which you may have direct, immediate control are smoking (possibly of marijuana as well as tobacco) your level of vaccination, and your amount of physical activity. It is not medical advice to take up smoking, avoid vaccination/booster shots, or sit around all day, in order to qualify for Paxlovid should you get sick. But I do worry that some people might go to such lengths due to this policy.
I can't imagine why anybody would take up smoking or stop exercising in order to get Paxlovid instead of just falsely telling their doctor that they smoke/don't exercise. Your doctor is not going to check. You can also easily claim to be gay or bisexual.
I'm not sure about the timing of when the edits in your post were made, but if you want feedback about your planned contact with Demis Hassabis I think you should make a new post about it -- most people who would comment on it may have missed it because they only saw the original unedited post about Tao, which had already received feedback.
I also think that, for the same reason that you chose to let someone else contact Tao instead of you, it may be better to let someone else contact Hassabis (or find someone else to contact him).
Sci-hub lets you get around paywalls for pretty much all academic articles.
Did your previous experiences with VR involve something where your in-game movement wasn't one-to-one with your actual movement (e.g. where you could move your character by pushing forward on an analog stick, rather than by walking)? It's pretty rare for VR games with one-to-one movement (like Beat Saber and TotF) to cause motion sickness, so if your previous sickness was in a non-one-to-one game it may be worth giving VR another shot with a more comfortable game.
Thanks for writing! I don't see an actual answer to the question asked in the beginning -- "Given the ongoing history of continually increasing compute power, what is the maximum compute power that might be available to train ML models in the coming years?" Did I miss it?
And would a doctor's note make any difference towards them allowing you to wear something like a versaflo or other PAPR?
Does anybody know what is the best mask that they'll allow you to wear on an airplane? Has anyone worn a P100 with the exhale valve covered, or do they not allow that?
hopefully realize it's a bad idea have a morality that allows this
To expand on this: https://www.nickbostrom.com/papers/unilateralist.pdf
What do you mean by "immune erosion"? Is this different than "immune evasion" and "immune escape"? I can't find any explanation on google -- is this a standard term?
What is meant by "immune erosive"? Is this different than "immune evasive"? I can't find any explanation on google -- is this a standard term?
If it's a normal distribution, what's the standard deviation?
For software development, rewriting the code from scratch is typically a bad idea. It may be helpful to see how well the arguments in that article apply to your domain.
Context for anyone who's not aware:
Nerd sniping is a slang term that describes a particularly interesting problem that is presented to a nerd, often a physicist, tech geek or mathematician. The nerd stops all activity to devote attention to solving the problem, often at his or her own peril
If MIRI hasn't already, it seems to me like it'd be a good idea to try reaching out. It also seems worth being at least a little bit strategic about it as opposed to, say, a cold email.
+1 especially to this -- surely MIRI or a similar x-risk org could attain a warm introduction with potential top researchers through their network from someone who is willing to vouch for them.
On one hand, meditation -- when done without all the baggage, hypothetically -- seems like a useful tool. On the other hand, it simply invites all that baggage, because that is in the books, in the practicing communities, etc.
I think meditation should be treated similarly to psychedelics -- even for meditators who don't think of it in terms of anything supernatural, it can still have very large and unpredictable effects on the mind. The more extreme the style of meditation (e.g. silent retreats), the more likely this sort of thing is.
Any subgroups heavily using meditation seem likely to have the same problems as the ones Eliezer identified for psychedelics/woo/supernaturalism.
Possible small correction: GPT-2 to GPT-3 was 16 months, not 6. The GPT-2 paper was published in February 2019 and the GPT-3 paper was published in June 2020.
I can't tell from the descriptions, but it seems like these programs have been run before -- is that right? Are there any reviews or other writeups about participants' experiences anywhere?
That would make a good monthly open thread.
If compute is the main bottleneck to AI progress, then one goalpost to watch for is when AI is able to significantly increase the pace of chip design and manufacturing. After writing the above, I searched for work being done in this area and found this article. If these approaches can actually speed up certain steps in this process from taking weeks to just taking a few days, will that increase the pace of Moore's law? Or is Moore's law mainly bottlenecked by problems that will be particularly hard to apply AI to?
Do you have some examples? I've noticed that rationalists tend to ascribe good faith to outside criticisms too often, to the extent that obviously bad-faith criticisms are treated as invitations for discussions. For example, there was an article about SSC in the New Yorker that came out after Scott deleted SSC but before the NYT article. Many rationalists failed to recognize the New Yorker article as a hit piece which I believe it clearly was, even more clearly now that the NYT article has come out.
Yeah, my main takeaway from that question was that a change in the slope of of the abilities graph was what would convince him of an imminent fast takeoff. Presumably the x axis of the graph is either time (i.e. the date) or compute, but I'm not sure what he'd put on the Y axis and there wasn't enough time to ask a followup question.
Even without having a higher IQ than a peak human, an AGI that merely ran 1000x faster would be transformative.
Of the bottlenecks I listed above, I am going to mostly ignore talent. IMO, talented people aren't the bottleneck right now, and the other problems we have are more interesting.
Can you clarify what you mean by this? I see two main possibilities for what you might mean:
- There are many talented people who want to work on AI alignment, but are doing something else instead.
- There are many talented people working on AI alignment, but they're not very productive.
If you mean the first one, I think it would be worth it to survey people who are interested in AI alignment but are currently doing something else -- ask each of them, why aren't they working on AI alignment? Have they ever applied for a grant or job in the area? If not, why not? Is money a big concern, such that if it were more freely available they'd start working on AI alignment independently? Or is it that they'd want to join an existing org, but open positions are too scarce?
The movie Downsizing is about this.
Psychedelics, maybe.
What's the advantage of taking over an existing corporation rather than creating a new organization?
What are some examples of makers who gained wealth/influence/status by having a huge negative impact on the world?
What I mean is that they haven't really considered it. As I'm sure you're aware, your mind does not work like most people's. When most people consider the question of whether they'd like to die someday, they're not really thinking about it as if it were a real option. Even if they give detailed, logically coherent explanations for why they'd like to die someday, they haven't considered it in in near mode.
I am very confident of this -- once they have the option, they will not choose to die. Right now they see it as just an abstract philosophical conversation, so they'll just say whatever sounds nice to them. For a variety of reasons, "I'd like to die someday; being immortal doesn't appeal to me" sounds wise to a lot of people. But once they have the actual option to not die? Who's going to choose to die? Only people who are suicidally depressed or who are in very extreme cults will choose that.
They'll almost definitely change their minds once we have good treatments for aging.
Those graphs all show the percentage share of the different variants, but more important would be the actual growth rate. Is the delta variant growing, or is it just shrinking less quickly than the others?
Why is that?
Can you taboo the words "weird" and "wrong" in that comment?
Precommitment for removal and optionality for adding.
There's a discord for Crypto+Rationalists you may be interested in if you're not already aware: https://discord.gg/3ZCxUt8qYw
To any high schoolers reading this: If I could send just one of the items from the above list back to myself in high school, it would be "lift weights." Starting Strength is a good intro.
I have a potential category of questions that could fit on Metaculus and work as an "AGI fire alarm." The questions are of the format "After an AI system achieves task x, how many years will it take for world output to double?"