I'll pay at least $75 for this comment. If nothing else, alerting me to RaDVaC's funding gap is clearly worth that much. I think it offered some interesting considerations beyond that. E.g. the search term polyethylene glycol seems useful, though I haven't looked into it much at all and definitely don't have strong models of that domain.
(I also think the fact that this comment bundled together a lot of different arguments and considerations caused the karma to take a downward hit.)
I'm excited about mechanism design in this space. Like, if you have a prediction market (or forecasting question with a good aggregation algorithm), you can sort of selectively throw out pieces of information, and then reward people based on how much those pieces moved the market. (And yes, there are of course lots of goodhart-y failure modes to iron out to make it work.)
In this case I'm not going to be quite so formal. I don't have that strong of an initial view, so it might often be more of rewarding "provided a very useful write-up" than "provide a compelling counterargument to a thoroughly considered belief".
I think this post strikes a really cool balance between discussing some foundational questions about the notion of agency and its importance, as well as posing a concrete puzzle that caused some interesting comments.
For me, Life is a domain that makes it natural to have reductionist intuitions. Compared to say neural networks, I find there are fewer biological metaphors or higher-level abstractions where you might sneak in mysterious answers that purport to solve the deeper questions. I'll consider this post next time I want to introduce someone to some core alignment questions on the back of a napkin, in a shape that makes it more accessible to start toying with the problem without immediatley being led astray. (Though this is made somewhat harder by the technicalities mentioned in the post, and Paul's concerns about whether Life is similar enough to our physics to be super helpful for poking around).
One important nuance, though, is that some of your intense work can be investing in things that decrease the likelihood of getting stuck in a bad attractor.
That way, you have shot at jumping to high-output equilibria that you can actually sustain.
From personal experience, I needed at least 4 different things to go right at the same time before I could start doing 60-80h weeks that didn't burn me out:
using a Freewrite
building a custom GTD system in Roam that used the API to tailor it very heavily to my preferences
using the Breaktimer app for mac (I set uncancellable 1-min breaks every 45 min)
having a set time each Sunday for doing life admin tasks
Each of these work to counteract overwhelm and burnout in their own way. For example, the regular Breaktimers (3) prevent me from ending up in spirals of working endless, almost addictive sessions without eating or drinking. Admin Sundays (4) mean I get time for improtant tasks that aren't work, that can otherwise blow up and become the straw that break a camel's back ("I've worked 70h this week, I'm exhausted, and now I need to deal with the massively schlepy implications of forgetting to pay my taxes?") Stream-of-consciousness writing on the Freewrite (1) allows me to notice unhappy parts of myself, and neglected needs.
Also, each of them provide a space for doing meta-cognition. Crucially, that means that if one of them fails, the others are likely to catch that and help me take corrective action. For example, during the frequent breaks (3) I might notice I feel scattered because I haven't been using my Roam GTD system, because it feels too cumbersome. So I'll use task capture to send that task to my admin Sundays (4), which I'll use to code up a more frictionless GTD system. The following week, I might notice the breaktimers not working properly. But now I'm able to use the actually nice GTD system to add a task to look into and fix the whatever the breaktimer bug is.
Thus, together, these 4 help create a self-sustaining equilibrium.
Curated. I enjoyed how this post was a little journey of deconfusion from the inside. It went through some of the actual cognitive motions one might make when trying to understand economics. (Or, rather, when trying become less confused about questions like "Why does everyone's lives today seem so much better than people I read about in history books?" or "How is it that the guy at Papa John's down the street can spend a few days making pizza, and then go to the store... and return with a little all-in-one pocket camera-computer-telephone-thing more powerful than devices that used to help send astronauts into space!?" And so forth.)
The questions sprinkled throughout, and the exercise at the end, also kept me curious and engaged. It's the kind of post that doesn't just hand down an insight, but conveys some of the skill required for generating similar insights.
(I lived in this house) The estimate was largely driven by fear of long covid + a much higher value per hour of time, which also factored in altruistic benefits from housemate's work that aren't captured by the market price of their salary.
There were also about 8 of us, and we didn't assume everyone would get it conditional on infection (household attack rates are much lower than that, and you might have time to react and quarantine). We assumed maybe like 2-3 others.
I totally expect we would have paid $84,600 to prevent a random one of us getting covid -- and it would've even looked like a pretty cheap deal compared to getting it!
Comment by jacobjacob on [deleted post]
This model makes explicit something I’ve had intuitions about for a while (though I wasn’t able to crystallise them nearly as perspicaciously or usefully as UnexpectedValues). Beyond the examples given in the post, I'm reminded of Zvi’s discussion of control systems in his covid series, and also am curious about how this model might apply to valuing cryptocurrencies, which I think display some of the same dynamics.
The post is also very well-written. It has the wonderful flavour of a friend explaining something to you by a whiteboard, building up a compelling story almost from first principles with clear diagrams. I find this really triggers my curiosity -- I want to go out and survey housemates to pin down the social behavior curves around me; go up to the whiteboard and sketch some new graphs and figure out what they imply, and so forth.
I had nudging cached in my memory as, more or less, a UX movement.
Want to increase charity donation at your company? Make it opt-out, rather than opt-in. Want to increase completion rates of your survey? Make it shorter.
And so forth.
So I was surprised by Jacob Falkovich claiming that nudgerism caused the elaborate psychological theorising used to inform covid policy. Many such policies mostly seemed to be about oddly specific, second-order claims. Like, in the case of expected resistance to challenge trials, or vaccine hesitancy. Those arguments venture more heavily into psychoanalysing people; rather than appealing to simple behavioural economics and basic UX.
(My cached memory of the nudge movement might be too narrow, though)
Habryka, is the reasoning that politicians have a real incentive to accurately predict public response -- because it entirely determines whether they remain in power -- whereas behavioral scientists have a much weaker incentive, compared to the dominant incentive of publishing significant results?
I haven't looked at the links, but making problem lists like this seems really cool. I'm glad they tried it, and then followed up.
I'm curious whether you know anything about why they tried it?
Hamming's original lecture talks about how most scientists he had lunch with sort of flinched away from their field's Hamming problems. He asked why they weren't working on them. It's implied that the conversation usually didn't go down very well, and the next day he had to eat lunch with someone else.
Why were things different for the Accounts of Chemical Research people? Unusual amounts of curiosity, courage, accident, or something else?
Comment by jacobjacob on [deleted post]
There is an argument that the use of willpower is undesirable.
I went down the neoantigen rabbithole, and it was quite interesting.
I liked this talk on "Developing Personalized Neoantigen-Based Cancer Vaccines".
It seems a core part of their methodology is using machine learning to predict which peptides will elicit a T-cell response, based on sequencing the patient's tumour. (Discussed starting from around 11 minutes in.)
They use this algorithm, which seems to be a neural network with a single hidden layer just ~60 neurons wide, and some amount of handcrafting of input features (based on papers from 2003 and 2009). I wonder what one could accomplish with more modern tools (though I haven't yet read the papers deeply enough to have a model of how big of a bottleneck this is to creating an effective treatment, and how much room for improvement there is).
I'm updating fairly hard on the four radvac team members who found antibodies using custom-built ELISA assays (rather than commercial tests). I wasn't super compelled by arguments that those might be false positives, but I do find it important that we don't know the denominator off how many of them took that test.
It maybe moved my probability from 17% to 45% that it would work for me (so still less optimistic than Wentworth!)
Though I think even a 5% chance of it working would make the original question worth asking. As they say: huge if true :)
(Also, the more competent version of me who solved it in a month would need to be competent on many other dimensions as well, not just knowing about peptide vaccines. Thinking about it, just the peptide delivery time could be longer than a month, as could the vaccine booster schedule. I do think there are worlds where it's actually a month, but I'll update the question to say "a few")
This actually flies against my sense that Bell Labs was able to build the transistor because of their resources and build-up of particular knowledge and expertise they had after 20-years. Possibly their ideas were just getting spread around via their external contacts, or actually, solid-state physics was taking off generally.
Woah, this was striking to me. It seems like pretty big evidence against Bell Labs actually having a secret sauce of enabling intellectual progress. I would have to look into it more, though. (Also the update is tempered by the fact that another argument for Bell Labs' greatness is the sheer number of inventions, like UNIX, satellites, lasers, information theory, and other stuff.)
Well, this post was just crying out for some embedded predictions! So here we go:
Thanks johnswentworth for help with some of the operationalisations!
I included many different ones, as I think it is often good try to triangulate high stakes questions via different operationalisations. This reduces some some "edge-case noise" stemming from answering vague questions in overly specific ways.
Yep, this is indeed a reason proper scoring rules don't remain proper if 1) you only have a small sample size of questions, and 2) utility of winning is not linear in the points you obtain (for example, if you really care about being in the top 3, much more than any particular amount of points).
Some people have debated whether it was happening in the Good Judgement tournaments. If so, that might explain why extremizing algorithms improved performance. (Though I recall not being convinced that it was actually happening there). When Metaculus ran its crypto competition a few years ago they also did some analysis to check if this phenomenon was present, yet they couldn't detect it.
And in doing so, I feel proud to assume the role of Patron Saint of LessWrong Challenges, and All Those Who Test Their Art Against the Territory.
Some reasons I'm excited about this post:
1) Challenges help make LessWrong more grounded, and build better feedback loops for actually testing our rationality. I wrote more about this in my curation notice for The Darwin Game challenge, and wrote about it in the various posts of my own Babble Challenge sequence.
2) It was competently executed and analysed. There were nice control groups used; the choice of scoring rule was thought through (as well as including what would've been the results of other scoring rules); the data was analysed in a bunch of different ways which managed to be both comprehensive while at the same time maintaining my curiosity and being very readable.
Furthermore, I can imagine versions of this challenge that would either feel butchered, in such a way that I felt like I didn't learn anything from reading the results, or needlessly long and pedantic, in such a way that getting the insight wouldn't have been worth the trek for most people. Not so with this one. Excellent job, UnexpectedValues.
3) I want to celebrate the efforts of the participants, some of whom devised and implemented some wonderful strategies. The turtle graphic fingerprints, gzip checks, mean-deviation scatter, and many others were really neat. Kudos to all who joined, and especially the winners, Jenny, Reed, Eric, Scy, William, Ben, Simon, Adam and Viktor!
I would love to see more activities like these on LessWrong. If you want to run one and would like help with marketing, funding for prizes, or just general feedback -- do send me a message!
Congratulations on your first LessWrong post! :) (Well, almost first)
As a piece of feedback, I will note that I found the "Rosenberg's crux" section pretty hard to read, because it was quite dense.
I feel like if I would've have read the original letter exchange, I could then have turned to this post, and gone "a-ha!" In other words, it felt like a useful summary, but didn't give me the original generators/models, such that I could pass the intellectual Turing test of what Dennett and Rosenberg actually believe.
By comparison, I think the section on the "cryptographer's constraint" was clearer; since it was more focused on elaborating on a particular principle and why it was important, along with considering some concrete examples more in depth.
The forecasters were only quite loosely selected for "some forecasting experience". Some of them I know are very able forecasters, others are people much less experienced, and who I don't think are affiliated that much with the rationality or effective altruism communities.
I have a beginning draft of a survey for the Secret of Our Success. I hoped I could finish it up yesterday, but instead I had work on shipping the LessWrong Books. Will see if I can get it out later this week.
Have at least one 2h conversation about a particular post, and write up a review after, almost regardless of how I feel the conversation went
Didn't happen and didn't really come close.
My main post-mortem is that I had multiple calendar reminders about the commitment, but for all of them I postponed them into the future. Until it was the last weekend and I was out of time. I should've spent more meta-cognition during some of them, thinking about how much time I would need to complete the tasks on time.
Author here: I think this post could use a bunch of improvements. It spends a bunch of time on tangential things (e.g. the discussion of Inadequacy and why this doesn't come through in textbooks, spending a while initially setting up a view to then tear down).
But really what would be nice is to have it do a much better job at delivering the core insight. This is currently just done in two bullets + one exercise for the reader.
Even more important would be to include JenniferRM's comment which adds a core mechanism (something like "cultural learning").
Overall, though, I still stand by the importance of the underlying concept; and think it's a crucial part of the toolkit required to apply economic thinking in practice.
Formulations are basically just lifted from the post verbatim, so the response might be some evidence that it would be good to rework the post a bit before people vote on it.
I thought a bit about how to turn Katja's core claim into a poll question, but didn't come up with any great ideas. Suggestions welcome.
As for whether the claims are true or not --
The "broken parts" argument is one counter-argument.
But another is that it matters a lot what learning algorithm you use. Someone doing deliberate practice (in a field where that's possible) will vastly outperform someone who just does "guessing and checking", or who Goodharts very hard on short-term metrics.
Maybe you'd class that under "background knowledge"? Or maybe the claim is that, modulo broken parts, motivation, and background knowledge, different people can meta-learn the same effective learning strategies?
I made some prediction questions for this, and as of January 9th, there interestingly seems to be some disagreement with the author on these.
Would definitely be curious for some discussion between Matthew and some of the people with low-ish predictions. Or perhaps for Matthew to clarify the argument made on these points, and see if that changes people's minds.