Posts

What are we predicting for Neuralink event? 2019-07-12T19:33:57.759Z
LW Dev question: FB-style tagging? 2019-06-20T19:19:45.807Z
Using rationality to debug Machine Learning 2018-04-10T20:03:44.357Z
Bayesian statistics as epistemic advantage 2017-07-25T17:07:49.660Z
MILA gets a grant for AI safety research 2017-07-21T15:34:55.493Z
Learning from Human Preferences - from OpenAI (including Christiano, Amodei & Legg) 2017-06-13T15:52:00.294Z
NIPS 2015 2015-12-07T20:31:11.779Z
Velocity of behavioral evolution 2014-12-19T17:34:36.217Z
What Peter Thiel thinks about AI risk 2014-12-11T21:22:27.167Z
Cognitive distortions of founders 2014-12-11T03:19:58.802Z
FAI PR tracking well [link] 2014-08-15T21:23:48.622Z
Hawking/Russell/Tegmark/Wilczek on dangers of Superintelligent Machines [link] 2014-04-21T16:55:56.240Z
Gunshot victims to be suspended between life and death [link] 2014-03-27T16:33:49.413Z
Huffington Post article on DeepMind-requested AI ethics board, links back to LW [link] 2014-01-30T01:20:10.579Z
H+ review of James Miller's Singularity Rising [link] 2014-01-17T02:16:08.126Z
PSA for LW futurists/academics 2013-10-31T15:54:39.861Z
Nudging around the world - [link] 2013-09-05T15:52:05.786Z
Course - Saving Millions of lives at a time [link] 2013-08-12T16:30:29.192Z
Proposal: periodic repost of the Best Learning resources 2013-08-10T16:21:19.138Z
RIP Doug Engelbart 2013-07-03T19:19:59.616Z
NES-game playing AI [video link and AI-boxing-related comment] 2013-04-12T13:11:36.365Z
Eliezer's YU lecture on FAI and MOR [link] 2013-03-07T16:09:54.710Z
Interesting discussion of concentration and productivity [link] 2013-02-06T13:58:35.082Z
CFAR’s Inaugural Fundraising Drive 2012-12-18T01:19:00.272Z
Popular media coverage of Singularity Summit -the Verge [link] 2012-10-23T03:19:03.676Z
Judea Pearl's Turing Award Lecture video now online 2012-09-10T15:18:43.251Z
Luke' AMA gets a plug @ Wired [link] 2012-08-26T04:15:49.745Z
Opinions on Boltzmann brain arguments constraining modern multiverse theories 2012-08-10T18:05:52.417Z
Russian plan for immortality [link] 2012-08-01T20:49:41.319Z
SIAI May report 2012-06-16T14:01:37.488Z
Arrison, Vassar and de Grey speaking at Peter Thiel's startup class [link] 2012-06-16T13:58:35.578Z
Sebastian Thrun AMA on reddit [link] 2012-06-14T02:53:15.258Z
MIT is working on industrial robots that (attempt to) learn what humans want from them [link] 2012-06-13T15:42:58.015Z
Peter Thiel's AGI discussion in his startups class @ Stanford [link] 2012-06-07T12:27:48.075Z
Audio interview with Judea Pearl [link] 2012-05-10T12:47:07.615Z
"Big Surprise" - the famous atheists are actually Bayesians [link] 2012-04-08T16:11:30.166Z
Singularity-oriented Sci-Fi collection to be published [link] 2012-02-29T16:39:40.714Z
Model Thinking class [link] 2012-02-20T18:27:51.909Z
Top 5 regrets of the dying [link] 2012-02-04T17:56:22.156Z
Bill Gates asks HS students "What are most important choices the world faces"? 2012-01-11T20:34:11.272Z
Nick Bostrom TED talk on world's biggest problems 2012-01-06T18:52:09.991Z
Roger Williams (Author of Metamorphosis of Prime Intellect) on Singularity 2012-01-06T05:09:36.511Z
Future of Moral Machines - New York Times [link] 2011-12-26T14:44:01.763Z
SingInst bloomberg coverage [link] 2011-12-19T19:31:41.651Z
For those in the Stanford AI-class 2011-12-10T22:54:38.294Z
Good interview with Kahneman [link] 2011-12-02T15:04:24.480Z
FAI-relevant XKCD 2011-11-22T13:28:44.070Z
Three more classes coming from Stanford of interest here 2011-11-17T17:57:42.468Z
Human augmented with CBT algorithms games Jeopardy [link] 2011-11-17T03:22:25.901Z
Biointelligence Explosion 2011-11-07T14:05:08.985Z

Comments

Comment by Dr_Manhattan on Douglas Hofstadter changes his mind on Deep Learning & AI risk (June 2023)? · 2023-07-03T17:38:08.052Z · LW · GW

He was only a de facto mysterian: thought mind is so complicated that it may as well be mysterious (but ofc he believed it's ultimately just physics). This position is updateable, and he clearly updated.

Comment by Dr_Manhattan on Bing Chat is blatantly, aggressively misaligned · 2023-02-16T15:48:01.302Z · LW · GW

A net saying "I'm thinking about ways to kill you" does not necessarily imply anything whatsoever about the net actually planning to kill you

 

Since these nets are optimized for consistency (as it makes textual output more likely), wouldn't outputting text that is consistent with this "thought" be likely? E.g. convincing the user to kill themselves, maybe giving them a reason (by searching the web)? 

Comment by Dr_Manhattan on How it feels to have your mind hacked by an AI · 2023-01-13T22:06:50.572Z · LW · GW

I've been wishing for someone to write AI-singularity parallel of Bardbury's Martian Chronicles (which are pretty much independent sample/ simulations of how living on Mars could go)

Comment by Dr_Manhattan on Replacing Karma with Good Heart Tokens (Worth $1!) · 2022-04-01T13:04:26.381Z · LW · GW
Comment by Dr_Manhattan on Mental nonsense: my anti-insomnia trick · 2022-03-29T18:19:43.094Z · LW · GW

Sharing a personal weird trick why not. I like falling asleep to light TV (via iPad). I watch short shows that a) I like and don't think are boring b) I have seen before. Usually 10 minutes into a 20 min show I'm ready (Futurama is my favorite for this + my meme game is much improved by this)

Comment by Dr_Manhattan on I left Russia on March 8 · 2022-03-13T18:38:59.221Z · LW · GW

Was thinking about you! Glad you made it out. Feel free to DM if I can be of assistance

Comment by Dr_Manhattan on (briefly) RaDVaC and SMTM, two things we should be doing · 2022-01-12T20:42:46.457Z · LW · GW

MIRI is bottlenecked more on ideas worth pursuing and people who can pursue them, than on funding

Ideas come from (new) people, and you mentioned seed planting which should contribute to having such people in 4-6 years, seems like still a worthy thing to do for AGI if anything is worth doing for any cause at all (given your short timelines). If you agree what's the bottleneck for that effort?

Comment by Dr_Manhattan on Visible Thoughts Project and Bounty Announcement · 2021-12-02T14:45:48.500Z · LW · GW

Related work: 
Show Your Work: Scratchpads for Intermediate Computation with Language Models
https://arxiv.org/abs/2112.00114

(from very surface-level perusal) Prompting the model resulted in 
1) Model outputting intermediate thinking "steps"

2) Capability gain

Comment by Dr_Manhattan on Study Guide · 2021-11-17T14:30:47.607Z · LW · GW

Koller & Friedman

 

They primarily & extensively statistical graphical models, not causality (but have a chapter on it)

Comment by Dr_Manhattan on My experience at and around MIRI and CFAR (inspired by Zoe Curzi's writeup of experiences at Leverage) · 2021-10-19T00:31:15.765Z · LW · GW

Since comments get occluded you should refer to an edit/update somewhere at the top if you want it to be seen by those who already read your original comment.

Comment by Dr_Manhattan on My experience at and around MIRI and CFAR (inspired by Zoe Curzi's writeup of experiences at Leverage) · 2021-10-18T10:33:29.964Z · LW · GW

Yes, I would! Any pointers? 
(to avoid miscommunication I'm reading this to say that people are more likely to build UFAI because of traumatizing environment vs. normal reasons Eli mentioned)

Comment by Dr_Manhattan on Precognition · 2021-07-01T10:38:38.418Z · LW · GW

Double-masking

 

Wait, you can't get N95 or KN95 there? 

Comment by Dr_Manhattan on Covid 8/6: The Case of the Missing Data · 2020-08-06T21:48:57.695Z · LW · GW

thought this was interesting

https://twitter.com/HealthAllegheny/status/1291466180386062340?s=20

Comment by Dr_Manhattan on Be impatient · 2020-07-28T19:18:23.490Z · LW · GW

Sounds like vaguely-good conclusions from my pattern-matching experience but very poorly argued, with much overloading of "impatience" and many cherry picked examples. Surprisingly bad quality from you. Also, "patience" is a great virtue, context matters a lot.

Comment by Dr_Manhattan on Credibility of the CDC on SARS-CoV-2 · 2020-03-09T14:59:54.730Z · LW · GW

Ok. To clarify, one of them is to blame. Maybe it's not the CDC. History will tell.

Comment by Dr_Manhattan on Credibility of the CDC on SARS-CoV-2 · 2020-03-09T14:09:19.561Z · LW · GW
Most obviously, blaming the CDC for the FDA and HHS not allowing 3rd party detection kits is somewhere between false and misleading.

Please support this claim. It seems obvious that they shat the bed (don't know which agency, let god sort them out for now, history and FOIA requests will sort them out in the future). It seems obvious from reading the news that many many local and commercial labs would have been ready with capacity a lot sooner than they are if FDA/CDC/HHS conglomerate got out of the way sooner.

It's quite plausible that this is due to Trump pressure, history will sort this out, but my estimation of guilt will likely just move from "weasel" to "weak for not resisting", and the facts remain the same

Comment by Dr_Manhattan on What are the long-term outcomes of a catastrophic pandemic? · 2020-03-03T12:37:19.890Z · LW · GW
  • Significant political regime disruption in some places, Iran specifically (probably for the better there)
  • Depending on how much Chinese government fudges with the current numbers they will appear to a) have bungled at first b) actually efficiently handled the problem at an enormous scale and US will appear to do things in the opposite order. Which leads me to...
  • Trump is weakened and Bloomberg's position is going to look very strong: some actual history of crisis management, strong-man people feel comfortable with in times of uncertainty, plus he "owns" one of coutry's top med schools
  • Massive disruption in service sector as people cut down on non-necessities...
Comment by Dr_Manhattan on Coronavirus: Justified Practical Advice Thread · 2020-03-01T22:03:05.603Z · LW · GW

This is useful in case you have facing a choice of riding it out at home and going to a hospital with high probability of getting infected if you're not already. E.g. if you have fever chances are still high you're just experiencing regular flu, and should not go to the hospital, but if your oxigen starts dropping into the danger zone you need to go.

Comment by Dr_Manhattan on Draft: Models of Risks of Delivery Under Coronavirus · 2020-02-29T21:10:56.109Z · LW · GW

Potential method of coping: disinfecting room. Unpack the stuff in protective gear, then after unpacked blast it with UV light?

Comment by Dr_Manhattan on Coronavirus: Justified Practical Advice Thread · 2020-02-29T15:24:33.879Z · LW · GW
But it's possible we could even isolate there in the in-law unit.

I thought this was the pretty clear cut answer before you wrote it. Totally endorse. Wear masks on the flight if possible. Ask your parents to stock up or start sending prep packages there (Amazon, Costco delivers)

Comment by Dr_Manhattan on Quarantine Preparations · 2020-02-26T12:22:44.070Z · LW · GW
The first question for me is are people starving in Wuhan due to the outbreak?

Answer is no, as of now, though food situation is uncomfortable. (my wife has relatives there she's in contact with). Trucks come to apartment complexes and people pick up.

I'm not sure the analogy translates well to US though. For better or worse us people are less organized. Also large % population live in suburbs where such deliveries are not feasible.

OTOH we have an excellent general delivery system in Amazon, UPS etc.

I'm slightly worried.

Comment by Dr_Manhattan on Making Sense of Coronavirus Stats · 2020-02-23T18:08:55.222Z · LW · GW

There's some but not a lot of interest in this topic on LW; I have a mailing list with primarily rationalist types on the topic; PM me email address to be added

Comment by Dr_Manhattan on Making Sense of Coronavirus Stats · 2020-02-21T13:40:17.167Z · LW · GW

Data always says something unless it's randomly generated. At the very least Chinese data provides lower bounds on some things. You can get somewhat better estimates if you model their incentives (though the lying will greatly increase the uncertainty and complexity of any model)

Comment by Dr_Manhattan on Rationalist prepper thread · 2020-01-30T13:42:52.951Z · LW · GW

There's the opposite effect: the early diagnosed cases tend to concentrate on the more obviously serious ones (more likely to die).

https://twitter.com/robertwiblin/status/1222577306515976192

Comment by Dr_Manhattan on Rationalist prepper thread · 2020-01-28T18:14:46.858Z · LW · GW
It is also good to invest in improving ones’ immune system by health food, vitamins, light therapy, as it is our best protection of the virus. Evacuation into a cold county house would weaken the immune system.

How much can one "improve" one's immune system by these methods in a short time? Is there any data to back this up?

In general agree with the rest. In worst-case scenario ability to self-isolate for a while ("bug in" in prepper lingo) seems worthwhile.

Comment by Dr_Manhattan on CFAR Participant Handbook now available to all · 2020-01-03T17:13:40.146Z · LW · GW

What do you use to listen to pdfs?

Comment by Dr_Manhattan on Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think · 2019-12-27T13:29:47.640Z · LW · GW

On the one hand this post does a great job of connecting to previous work, leaving breadcrumbs and shortening the inferential distance. On the other hand what is this at the end?

But one thing I'm pretty sure won't help much is clever logic puzzles about implausibly sophisticated Nazis.

I have no idea what this is talking about.

Comment by Dr_Manhattan on T-Shaped Organizations · 2019-12-18T13:39:23.196Z · LW · GW
[edit] It also seems like this is the sort of thing that marketing pretty strongly encourages misrepresentation of. "All children are above-average," in that the restaurant wants to present itself as serving healthy, cheap, tasty food, while also paying its employees well and having good returns for its investors. But several of those variables are in direct tension with each other, and there's not great language for speaking publicly about the tradeoffs you're making. 

Couple of reasons spring to mind:

  • Marketing leverages the Halo Effect
  • Not emphasizing being above average on a particular dimension is a chink in the armor against competitors (who can establish beach head by claiming to be superior in that dimension)

Comment by Dr_Manhattan on Approval Extraction Advertised as Production · 2019-12-16T20:09:49.582Z · LW · GW

The way I understand the objection it that YC promotes "building great products", which attracts (a lot of) certain kinds of founders, but in fact YC is optimizing for something else (primarily described in Black Swan Farming, confirmed by other sources). I believe they are quite value-additive to the companies they accept, but attract more founders than if they were "honest about their optimization function", where some founders could have been better off engaging with other VCs on possibly better terms.

Comment by Dr_Manhattan on Rule Thinkers In, Not Out · 2019-12-13T17:11:04.824Z · LW · GW
>> Fiddly puttering with something that fascinates you is the source of most genuine productivity. (Anything from hardware tinkering, to messing about with cost spreadsheets until you find an efficiency, to writing poetry until it “comes out right”.) Sometimes the best work of this kind doesn’t look grandiose or prestigious at the time you’re doing it.
Hmm, I use to spend quite a bit of time fiddling with assembly language implementations of encryption code to try to squeeze out a few more percent of speed. Pretty sure that is not as productive as more "grandiose" or "prestigious" activities like thinking about philosophy or AI safety, at least for me... I think overall I'm more afraid that someone who could be doing productive "grandiose" work chooses not to in favor of "fiddly puttering", than the reverse.

I suspect this might be a subtler point?

http://paulgraham.com/genius.html

suggests really valuable contributions are more bottlenecked on obsession rather than being good at directing attention in a "valuable" direction

For example, for the very ambitious, the bus ticket theory suggests that the way to do great work is to relax a little. Instead of gritting your teeth and diligently pursuing what all your peers agree is the most promising line of research, maybe you should try doing something just for fun. And if you're stuck, that may be the vector along which to break out.
Comment by Dr_Manhattan on Rule Thinkers In, Not Out · 2019-12-13T17:03:53.876Z · LW · GW

I think in part these could be "lessons relevant to Sarah", a sort of a philosophical therapy that can't be completely taken out of context. Which is why some of these might seem of low relevance or obvious.

Comment by Dr_Manhattan on Rule Thinkers In, Not Out · 2019-12-13T16:55:59.912Z · LW · GW
Fiddly puttering with something that fascinates you is the source of most genuine productivity. (Anything from hardware tinkering, to messing about with cost spreadsheets until you find an efficiency, to writing poetry until it "comes out right".) Sometimes the best work of this kind doesn't look grandiose or prestigious at the time you're doing it.

http://paulgraham.com/genius.html seems to be promoting a similar idea

Comment by Dr_Manhattan on Bayesian examination · 2019-12-11T13:25:47.813Z · LW · GW

I'll claim LW priority for pointing to the idea (but not to elaborating it in a post) https://www.lesswrong.com/posts/isSMDR8rMr5pTzJK5/example-of-poor-decision-making-under-pressure-from-game?commentId=YzvDcA357NboxD2fE :)

Comment by Dr_Manhattan on Bayesian examination · 2019-12-09T22:24:52.982Z · LW · GW

Ross https://web.stanford.edu/~shachter/ uses something like this to score answers to (appropriately) Decision Analysis homework questions. (Don't remember the exact rule, but the intent was the same)

Comment by Dr_Manhattan on Ways that China is surpassing the US · 2019-11-04T13:28:27.435Z · LW · GW
What does it imply for things like AI governance and global coordination on x-risks?

I've read the article a while ago, and vaguely concluded there should be some implications here (but largely uncertain about the direction or magnitude, being a non-expert). Interested to hear what people think (esp. people who concentrate on policy)

Comment by Dr_Manhattan on Unstriving · 2019-08-22T12:14:31.776Z · LW · GW
How about we let go of success, but keep doing challenging stuff anyway, just for the fun of it?

This sort of feels like Feynman's attitude, despite him being extremely successful.

Comment by Dr_Manhattan on GPT-2: 6-Month Follow-Up · 2019-08-21T12:41:46.516Z · LW · GW

Also notable: NVIDIA trained a half-order-of-magnitude larger model https://nv-adlr.github.io/MegatronLM?utm_campaign=NLP%20News&utm_medium=email&utm_source=Revue%20newsletter

Comment by Dr_Manhattan on What are we predicting for Neuralink event? · 2019-07-17T19:27:57.727Z · LW · GW

Seems way off from the actual release; any post-mortem?

Comment by Dr_Manhattan on Integrity and accountability are core parts of rationality · 2019-07-16T14:55:07.325Z · LW · GW

A decent solution to the "who should you be accountable to", from the wisdom of the ancients (shows thought on many of the considerations mentioned)

When in doubt, remember Warren Buffett’s rule of thumb: “… I want employees to ask themselves whether they are willing to have any contemplated act appear the next day on the front page of their local paper—to be read by their spouses, children and friends—with the reporting done by an informed and critical reporter.”
Comment by Dr_Manhattan on What are we predicting for Neuralink event? · 2019-07-16T14:17:51.917Z · LW · GW

(might contain spoilers/private info) https://www.technologyreview.com/s/613961/elon-musks-brain-interface-company-is-promising-big-news-heres-what-it-could-be/

Comment by Dr_Manhattan on Open Thread July 2019 · 2019-07-10T15:59:46.855Z · LW · GW

I think the Hypothesis is not about Open Threads specifically

Comment by Dr_Manhattan on What would be the signs of AI manhattan projects starting? Should a website be made watching for these signs? · 2019-07-03T20:55:52.786Z · LW · GW

Tracking employment/location and publishing/conference attendance records of researchers will probably be good source data for this.

Comment by Dr_Manhattan on What was the official story for many top physicists congregating in Los Alamos during the Manhattan Project? · 2019-07-03T20:53:47.200Z · LW · GW

I think it was easier in that era; AFAIK they used conventional secrecy methods (project names, locations, misdirection, need to know, obfuscation) to pull it off. Feynman's "Surely you're joking" and Rhodes "making the atomic bomb" are good sources for some examples (and otherwise recommended)

Comment by Dr_Manhattan on Contest: $1,000 for good questions to ask to an Oracle AI · 2019-07-01T21:27:06.741Z · LW · GW

Small typo:

Hence it has no motivation to manipulate[d] humans through its answer.
Comment by Dr_Manhattan on What is the evidence for productivity benefits of weightlifting? · 2019-06-27T16:34:19.929Z · LW · GW

IN MICE

Comment by Dr_Manhattan on Jordan Peterson on AI-FOOM · 2019-06-27T12:34:12.255Z · LW · GW

I somewhat overlooked this line and yes, it's a nod in the right direction

Comment by Dr_Manhattan on Jordan Peterson on AI-FOOM · 2019-06-26T19:19:06.026Z · LW · GW

Based on the transcript this does not sound like a FOOM discussion (as in rapid self-improvement) other than mentioning "group learning" by autonomous cars, which is maybe somewhat related. Also the pregnancy ad story is much more about pattern recognition with lots of data than any serious AI.

Basically JP is, in this area, a complete layman (unlike Gates, Musk, or, from the other side, Pinker) whose opinion counts for little and not talking about FOOM anyway.

Comment by Dr_Manhattan on Being the (Pareto) Best in the World · 2019-06-25T12:32:24.217Z · LW · GW

Is this much different from Scott Adams' advice https://dilbertblog.typepad.com/the_dilbert_blog/2007/07/career-advice.html

of


if you want something extraordinary, you have two paths:
1. Become the best at one specific thing.
2. Become very good (top 25%) at two or more things.

?


Comment by Dr_Manhattan on Reason isn't magic · 2019-06-20T19:10:45.161Z · LW · GW

Do you mean Zvi's "Change is bad"?

Comment by Dr_Manhattan on Reason isn't magic · 2019-06-19T14:34:39.633Z · LW · GW
Second, we're not actually comparing reason to tradition - we're comparing changing things to not changing things. Change, as we know, is bad.

Request for clarification: isn't "reasonable solution" always a "change" when compared to preexisting tradition?