Posts
Comments
Ahh. One is uncertain which world they’re in. This feels like it could address it neatly. Thanks!
Strong-downvoted.
>She’s all over the EA and AI-related subreddits /r/singularity, /r/artificial, /r/ArtificialIntelligence, /r/ChatGPT, /r/OpenAI, /r/Futurology
In other words, everywhere but here. Since that’s the case, it would be better to take your fight to those places. Ms. Woods’s only post on Less Wrong in the past year was a short notice about o3 safety testing sign-ups, which was unobjectionable.
>I don’t like the vibes.
I was thinking the same thing. This post badly, badly clashes with the vibe of Less Wrong. I think you should delete it, and repost to a site in which catty takedowns are part of the vibe. Less Wrong is not the place for it.
I expect our intuitions about objective randomness would clash quite violently! My own intuition revolts at even the phrase itself :)
I looked into it a bit, and understand it to mean that one must accept one of:
- Physical uncertainty exists
- Non-locality exists
Quantum mechanics is wrong
is that breakdown correct?
Been thinking about your answer here, and still can’t decide if I should view this as solving the conundrum, or just renaming it. If that makes sense?
Do weights of quantum configuration, though they may not be probabilities, similar enough in concept to still imply that physical, irreducible uncertainty exists?
I’ve phrased this badly (part of why it took me so long to actually write it) but maybe you see the question I’m waving at?
Hm - reading Ben’s linked comment, it seems to me that the thrust is that negative probabilities must be admitted. But I don’t understand how that is related to the map vs. territory / probability-in-the-mind-or-physical distinction?
Like, “one must modify the relevant functions to allow negative probabilities” seems consistent with “probability is in the mind”, since functions are a part of the map, but it seems you consider it a counterexample! So I find myself confused.
>In other words, you don’t need reality to be i.i.d.; you simply need to structure your beliefs in a way that allows an “as if” i.i.d. interpretation.
I think I view exchangeability vs. iid slightly differently. In my view, the “independence” part of iid is just way too strong, and is not required in most of the places people scatter the acronym “iid”.
For example, say you are catching fish in a lake, and you know only bass and carp live in the lake, and that there are a ton of fish in it, but not how many of each, and you’re trying to estimate the proportion of carp as you catch fish.
When I catch a carp, the probability that my next catch is a carp goes up. So my probability is dependent on my previous catches - that’s why I can learn things about the proportion! If they were indeed independent, then I couldn’t learn anything. But happily, the correct requirement is not independence, but exchangeability, so I can still update my beliefs as I see more fish.
However, I may just be confused about “iid” as classicists use it, since I never properly learned classic statistics. Interested in what you think about the difference between the two in this example.
Thanks!
Thanks for putting this together!
I have a vague memory of a post saying that taking zinc early, while virus was replicating in the upper respiratory tract, was much more important than taking it later, because later it would have spread all over the body and thus the zinc can’t get to it, or something like this. So I tend to take a couple early on then stop. But it sounds like you don’t consider that difference important.
Is it your current (Not asking you to do more research!) impression that it’s useful to take zinc throughout the illness?
The post is an advertisement, without other content. I think a post of that type should only be on the site if it comes with some meat - an excerpt, at least. (And even then I’m not sure). The reader can’t even look up or read the book yet if he wanted to!
(There is a quote of the thesis of the book, but the text is stuff I’ve been rereading for years now. It feels like someone is always telling me liberalism is under threat recently.)
Interesting! The current Sonnet 3.5 agrees (for equivalent concentrations), for the same reason you've described, and I was about to update the essay with a correction, but then 4o argued that 1. formaldehyde is metabolized much more quickly, so has little time to do damage or build up, and 2. that it considers formic acid's inhibition of a critical enzyme (cytochrome c oxidase) in the mitochondrial electron transport chain to be pretty bad.
Or maybe a better summary of 4o's argument is "In equivalent concentrations, formaldehyde is worse, but the differences in rapidity of metabolization mean formic acid builds up more and causes more damage in real-life scenarios."
So I've linked your comment in the relevant section, sort of waving my hands and succumbing to both-sides-ism. Interested in what you think about the rapidity-of-metabolization argument.
Guesses: people see it as too 101 of a question; people think it’s too controversial / has been done to death many years ago; one guy with a lot of karma hates the whole concept and strong-downvoted it
I think the 101 idea is most likely. But I don’t think it’s a bad question, so I’ve upvoted it.
Years ago, a coworker and I were on a project with a guy we both thought was a total dummy, and worse, a dummy who talked all the time in meetings. We rarely expressed our opinion on this guy openly to each other - me and the coworker didn’t know each other well enough to be comfortable talking a lot of trash - but once, when discussing him privately after yet another useless meeting, my coworker drew in breath, sighed, looked at me, and said: “I’m sure he’s a great father.” We both laughed, and I still remember this as one of the most cutting insults I’ve heard.
Cheers!
I’d guess that weekend dips come from office workers, since they rarely work on weekends, but students often do homework on weekends.
If OP were advocating banning normal parties, in favor of only having cancellable parties, I would agree with this comment.
Appreciate it! Cheers.
A good post, of interest to all across the political spectrum, marred by the mistake at the end to become explicitly politically opinionated and say bad things about those who voted differently than OP.
The integral was incorrect! Fixed now, thanks! Also added the (f * g)(x) to the equality for those who find that notation better (I've just discovered that GPT-4o prefers it too). Cheers!
Yes, I’m not so sure either about the stockfish-pawns point.
In Michael Redmond’s AlphaGo vs AlphaGo series on YouTube, he often finds the winning AI carelessly loses points in the endgame. It might have a lead of 1.5 or 2.5 points, 20 moves before the game ends; but by the time the game ends, has played enough suboptimal moves to make itself win by 0.5 - the smallest possible margin.
It never causes itself to lose with these lazy moves; only reduces its margin of victory. Redmond theorizes, and I agree, that this is because the objective is to win, not maximize point differential, and at such a late stage of the game, its victory is certain regardless.
This is still a little strange - the suboptimal moves do not sacrifice points to reduce variance, so it’s not like it’s raising p(win). But it just doesn’t care either way; a win is a win.
There are Go AI that are trained with the objective of maximizing point difference. I am told they are quite vicious, in a way that AlphaGo isn’t. But the most famous Go AI in our timeline turned out to be the more chill variant.
Quip about souls feels unnecessary and somehow grates on me. Something about putting an athiesm zinger into the tag for cooking… feels off.
Would you be willing to share your ethnicity? Even as simple as “Asian / not Asian”?
I do think it has some of that feeling to me, yeah. I had to re-read the entire thing 3 or 4 times to understand what it meant. My best guesses as to why:
I felt whiplashed on transitions like “be motivated towards what's good and true. This is exactly what Marc Gafni is trying to do with Cosmo-Erotic Humanism”, since I don’t know him or that type of Humanism, but the sentence structure suggests to me that I am expected to know these. A possible rewrite could perhaps be “There are two projects I know of that aim to create a belief system that works with, instead of against, technology. The first is Marc Gafni; he calls his ‘Cosmo-Erotic Humanism’…”
There are some places I feel a colon would be better than a comma. Though I’m not sure how important these are, it would help slow down the pace of the writing:
“increasingly let go of faith in higher powers as a tenet of our central religion: secular humanism.” “But this is crumbling: the cold philosophy”
While minor punctuation differences like this are usually not too important, the way you wrote gives me a sense of, like, too much happening too fast: “wow, this is a ton of information delivered extremely quickly, and I don’t know what appolonian means, I don’t know who Gafni is, or what dataism is…” So maybe slowing down the pace with stronger punctuation like colons is more important than it would otherwise be?
Also, phrases like “our central religion is secular humanism” and “mystical true wise core” read as very Woo. I can see where both are coming from, but I’ve read a lot of Woo, but I think many readers would bounce off these phrases. They can still be communicated, but perhaps something like “in place of religion, many have turned to Secular Humanism. Secular humanism says that X, Y, Z, but has no concept of a higher power. That means the core motivation that…”
(To be honest I’ve forgotten what secular humanism is, so this was another phrase that added to my feeling of everything moving too fast, and me being lost).
There are some typos too.
So maybe I’d advise making the overall piece of writing slower, by giving more set-up each time you introduce a term readers are likely to be unfamiliar with. On the other hand, that’s a hassle, and probably annoying to do in every note, if you write on this topic often. But it’s the best I’ve got!
I read this book in 2020, and the way this post serves as a refresher and different look at it is great.
I think there might be some mistakes in the log-odds section?
The orcs example starts:
We now want to consider the hypothesis that we were attacked by orcs, the prior odds are 10:1
Then there is a 1/3 wall-destruction rate, so orcs should be more likely in the posterior, but the post says:
There were 20 destroyed walls and 37 intact walls… corresponding to 1:20 odds that the orcs did it.
We started at 10:1 (likely that it’s orcs?), then saw evidence suggesting orcs, and ended up with a posterior quite against orcs. Which doesn’t seem right. I was thinking maybe “10:1” for the prior should be “1:10”, but even then, going from 1:10 in the prior to 1:20 in the posterior, when orcs are evidenced, doesn’t work either.
All that said, I just woke up, so it’s possible I’m all wrong!
In Korea every convenience store sells “hangover preventative”, “hangover cure drink”, with pop idols on the label. Then you come back to America and the instant you say “hangover preventative”, people look at you crazy, like no such thing could possibly exist or help. I wonder how we got this way!
Thanks for your review! I've updated the post to make the medications warning be in italicized bold, in the third paragraph of the post, and included the nutrient warning more explicitly as well.
Thank you!
“(although itiots might still fall for the "I'm an idiot like you" persona such as Donald Trump, Tucker Carlson, and particularly Alex Jones).”
This line is too current-culture-war for LessWrong. I began to argue with it in this comment, before deleting what I wrote, and limiting myself to this.
It changed to be much more swipe-focused. It’s been 5 years since I used it, but even in 2018, I remember being surprised at how much it had changed. Apparently now even open messaging is gone, and you need to have someone Like you before you can message them, though I haven’t actually checked this.
Yes, agree - I've looked into non-identical distributions in previous posts, and found that identicality isn't important, but I haven't looked at non-independence at all. I agree dependent chains, like the books example, is an open question!
Love this! Definitely belongs on LessWrong. High-quality sci-fi, that relates to social dynamics? Very relevant! I’ve been away from the site for a while, tiring of the content, but am glad I scrolled and saw this today.
Enjoyed this! Very well written. The two arrow graphs, where the second has everything squished down to the bottom, are especially charming
I don’t think the problem is this big if you’re trying to control one specific model. Given an RLHF’d model, equipped with a specific system prompt (e.g. helpless harmless assistant), you have either one or a small number of luigis, and therefore around the same amount of waluigis - right?
Good question!
Hmm! I’m not sure about this. The patient in the linked paper received hemodialysis (which, I think, manually takes the methanol out) before his body could get around to metabolizing it into formaldehyde and formic acid. For someone who doesn’t receive hemodialysis, I think the methanol would still have to be metabolized at some point, even if when that happens is much delayed? In which case the same toxic effects of formaldehyde and formic acid would hit, just much later.
Thanks!
What form/brand/dose do you take the ketone esters in?
I think the poker example is OK, and paragraphs like
“The second decision point was when the flop was dealt and you faced a bet. This time you decided to fold. Maybe that wasn't the best play though. Maybe you should have called. Maybe you should have raised. Again, the goal of hand review is to figure this out.”
made sense to me. But the terminology in the dialogue was very tough: button, Rainbow, LAGgy, bdfs, AX, nut flush, nitty - I understood none of these. (I’ve played poker now and then, but never studied it). So keeping the example but translating it a bit further to more widely-used language (if possible) might be good.
Very interesting! I work in health insurance and we try to encourage vaginal delivery and discourage C-sections; the other side you present here is a surprise. Good stuff.
Very very good. The full power of science fiction - taking the concept of the redaction machines and finding this many interesting consequences of them, and fitting them into the story - really good
You don’t need to call tails to explore whether tails is possible, though - the information gain of a coin flip is the same whether you call heads or tails
I’m not the asker, but I think I get where they’re coming from. For a long time, linear and logistic regression were the king & queen of modeling. Then the introduction of non-linear functions like random forest and gradient boosters made us far more able to fit difficult data. So the original question has me wondering if there’s a similar possible gain in going from linearity to non-linearity in interpretability algorithms.
I agree with the encouragement to look harder for a sooner TMJ appointment. ADHD testing has similar waits now - looking in May, I was told everyone was booked up till September. But I lucked out, and the first testing doctor I talked to had just had some people cancel appointments, and nobody on his waitlist was responding, so I ended up seeing him a week later, in June, instead of in September. So there are opportunities for luck like this around. And this is without me looking out of state.
I like the trigger point idea. OP should note too that there are injection treatments for trigger points: https://www.webmd.com/pain-management/guide/trigger-point-injection
I just quit caffeine a month ago after years of daily dependence on it, and I feel better than I did on it. I now limit myself to 100mg a week. The dependence had a consistent moderate negative affect on my life, so I’d recommend people be very careful to avoid dependence.
Sure - there are plenty of cases where a pair of interactions isn’t interesting. In the image net context, probably you’ll care more about screening-off behavior at more abstract levels.
For example, maybe you find that, in your trained network, a hidden representation that seems to correspond to “trunk” isn’t very predictive of the class “tree”. And that one that looks like “leaves” is predictive of “tree”. It’d be useful to know if the reason “trunk” isn’t predictive is that “leaves” screens it off. (This could happen if all the tree trunks in your training images come with leaves in the frame).
Of course, the causality parts of the above analysis don’t address the “how should you assign labels in the first place” problem that the post is most focused on! I’m just saying both the ML parts and the causality parts work well in concert, and are not opposing methods.
This post does a sort of head-to-head comparison of causal models and deep nets. But I view the relationship between them differently - they’re better together! The causal framework gives us the notion of “screening off”, which is missing from the ML/deep learning framework. Screening-off turns out to be useful in analyzing feature importance.
A workflow that 1) uses a complex modern gradient booster or deep net to fit the data, then 2) uses causal math to interpret the features - which are most important, which screen off which - is really nice. [This workflow requires fitting multiple models, on different sets of variables, so it’s not just fit a single model in step 1), analyze it in step 2), done].
Causal math lacks the ability to auto-fit complex functions, and ML-without-causality lacks the ability to measure things like “which variables screen off which”. Causality tools, paired with modern feature-importance measures like SHAP values, help us interpret black-box models.
From my personal experience, I would have them take up one or two competitive arts.
Timing yourself to improve your personal best at, say, running, does not count. Running on a track against, or in a longer race against, a small handful of people that you can potentially beat does count, although I would lean toward recommending something where you have to deal with the counter-moves of your opponent. Boxing or kickboxing training that includes sparring against others counts; doing boxing training at a gym that does not do sparring doesn’t count. Playing chess or Go counts. Basketball, hockey, soccer, etc, all count. Playing online competitive video games technically counts under this definition, but I’m excluding it; those mostly make me feel bad.
Having an opponent that will challenge you with counter-moves, and do their best to get one over on you, but who you can beat if you train and try hard enough, has no substitute. Winning against someone who has put everything into the fight gives confidence that you can apply all over the place. Plus it feels great.
My experience: I’ve spent the past two years running and lifting. These mean I look great physically, and am healthy, and get the exercise endorphins and stuff. But they didn’t meet the competitive need! I’ve recently gotten back into boxing at a sparring gym. The competitive aspect of being in the ring, trying to best the other guy, is something I’ve really been missing. It also directs my training at a real concrete purpose, instead of the colder “increase the weight / increase running speed” metric-tracking approach to those forms of exercise.
I also play Go, and it used to serve this purpose well in my life.
(This competitiveness stuff might be more important for men than it is for women - I’m not sure. I’d definitely give this advice to a man, and I’d give it as a ‘maybe’ to a woman. Of course, women can get a lot of value from competition; I’m just not sure if the lack of it would gnaw at them the way it was gnawing at me.)
I have pretty bad energy-level problems and have been looking for more things to try to fix them. I’d always thought quitting caffeine would make it so I could attain my current energy levels without caffeine; it never occurred to me that energy levels after quitting could be higher. So this is very interesting. Thanks for sharing.
That’s great! Happy to hear it - thanks for reporting back, especially in such detail.