Posts
Comments
Curated. Since this post has been published, there's been a couple of times I've heard the whisper of "More Dakka ... do it again". I've booked additional appointments with potential lawyers and tailors, called around for more quotes, and bought spares of trousers I like.
I haven't explored most of the bullets in this post, but I think posts that (a) move concepts closer to next actions, (b) flesh out concepts in a bunch of different ways and explore them from different angles and (c) give examples are undersupplied. This post does all of those!
I think Romoeo is thinking of checking a bunch of mediators of risk (like aldehyde levels) as well as of function (like whether the organism stays colonised)
Whoa! The meetup to beat
Ohhhhh!
But @henry was involved in the explainer, so it's nice for him to get the comment notifications and the karma.
This is a straightforward consequence of the good regulator theorem
IIUC, the good regulator theorem doesn't say anything about how the model of the system should be represented in the activations of the residual stream. I think the potentially surprising part is that the model is recoverable with a linear probe.
(I would agree-react but I can't actually make it)
It seems unlikely that different hastily cobbled-together programs would have the same bug.
Is this true? My sense is that in, for example, Advent of Code problems, different people often write the same bug into their program.
"Crucial to our disagreement" is 8 syllables to "cruxy"'s 2.
"Dispositive" is quite American, but has a more similar meaning to "cruxy" than plain "crucial". "Conclusive" or "decisive" are also in the neighbourhood, though these are both feel like they're about something more objective and less about what decides the issue relative to the speaker's map.
D&D.Sci forces the reader to think harder than anything else on this website
D&D.Sci smoothly entices me towards thinking hard. There's lots of thinking hard that can be done when reading a good essay, but the default is always to read on (cf Feynman on reading papers) and often I just do that while skipping the thinking hard.
Curated! This kicked off a wonderful series of fun data science challenges. I'm impressed that it's still going after over 3 years, and that other people have joined in with running them, especially @aphyer who has an entry running right now (go play it!).
Thank you, @abstractapplic for making these. I don't think I've ever submitted a solution, but I often like playing around with them a little (nowadays I just make inquiries with ChatGPT). I particularly like
That it nuanced my understanding of the supremacy of neural networks and when "just throw a neural net" at it might work or might not.
Here's to another 3.4 years!
Maybe "counterfactually robust" is an OK phrase?
I am sad to see you getting so downvoted. I am glad you are bringing this perspective up in the comments.
I like comments about other users' experiences for similar reasons why I like OP. I think maybe the ideal such comment would identify itself more clearly as an experience report, but I'd rather have the report than not.
What you probably mean is "completely unexpected", "surprising" or something similar
I think it means the more specific "a discovery that if it counterfactually hadn't happened, wouldn't have happened another way for a long time". I think this is roughly the "counterfactual" in "counterfactual impact", but I agree not the more widespread one.
It would be great to have a single word for this that was clearer.
Enovid is also adding NO to the body, whereas humming is pulling it from the sinuses, right? (based on a quick skim of the paper).
I found a consumer FeNO-measuring device for €550. I might be interested in contributing to a replication
(No, "you need huge profits to solve alignment" isn't a good excuse — we had nowhere near exhausted the alignment research that can be done without huge profits.)
This seems insufficiently argued; the existence of any alignment research that can be done without huge profits is not enough to establish that you don't need huge profits to solve alignment (particularly when considering things like how long timelines are even absent your intervention).
To be clear, I agree that OpenAI are doing evil by creating AI hype.
Is there anything particularly quantum about this effect?
Using the simulator frame, one might think there's space to tweak:
- The basic physical laws
- The fundamental constants
- The "PRNG" (in an Everettian picture this looks kind of weird because its more like throwing out parts of the wavefunction to save on computation; reminds me a little of mangled worlds)
Perhaps the idea is that tweaking 1 & 2 results in worlds less interesting to the simulator?
I'm not seeing any active rate limits. Do you know when you observed it? It's certainly the case that an automatic rate limit could have kicked in and then, as voting changed, been removed.
Good question! From the Wiki-Tag FAQ:
A good heuristic is that tag ought to have three high-quality posts, preferably written by two or more authors.
I believe all tags have to be approved. If I were going through the morning moderation queue, I wouldn't approve an empty tag.
I was trying to figure out why you believed something that seemed silly to me! I think it barely occurred to me that it's a joke.
The main subcultures that I can think of where this applies are communities based around solving some problem:
- Weight loss, especially if based around a particular diet
- Dealing with a particular mental health problem
- Trying to solve a particular problem in the world (e.g. explaining some mystery or finding the identity of some criminal)
Any favourite examples?
I think my big problem with complexity science (having bounced off it a couple of times, never having engaged with it productively) is that though some of the questions seem quite interesting, none of the answers or methods seem to have much to say.
Which is exacerbated by a tendency to imply they have answers (or at least something that is clearly going to lead to an answer)
I would like to read it! Satire is sometimes helpful for me to get a perspective shift
To answer, for now, just one piece of this post:
We're currently experimenting with a rule that flags users who've received several downvotes from "senior" users (I believe 5 downvotes from users with above 1,000 karma) on comments that are already net-negative (I believe that were posted in the last year).
We're currently in the manual review phase, so users are being flagged and then users are having the rate limit applied if it seems reasonable. For what it's worth, I don't think this rule has an amazing track record so far, but all the cases in the "rate limit wave" were reviewed by me and Habryka and he decided to apply a limit in those cases.
(We applied some rate limit in 60% of the cases of users who got flagged by the rule).
People who get manually rate-limited don't have an explanation visible when trying to comment (unlike users who are limited by an automatic rule, I think).
We have explained this to users that reached out (in fact this answer is adapted from one such conversation), but I do think we plausibly should have set up infrastructure to explain these new rate limits.
Hello and welcome to the site! I'm glad you're saying hello despite having been too shy :-)
Do let us know in this thread or in the intercom in the bottom right if you run into any problems.
Curated.
Using Bayes-type epistemology is a core LessWrong topic, and I think this represents a bunch of progress on that front (whether the results are already real-world-ready or just real-world-inspired). I have only engaged with small parts of the thesis, but those parts seem pretty exciting; so far, I particularly like knowing about quasi-arithmetic pooling. It feels like I've become less confused about something that I didn't know I was confused about — the connection between the character of the proper scoring rule and the right ways to aggregate those probabilities.
I also appreciate Eric's work making blogposts explaining more of his thoughts in a friendly way. Hope to see a few more distillations come out of this thesis!
Much sweat and some tears were spent on trying to get something like that working, but the Shoggoths are fickle
Thanks so much for the fix!
(I assume you mean the story with him and the SS soldier; I think a couple of people got confused and thought you were referring to the fact Kahneman had died)
From a message I wrote to a friend once that seems a little relevant
[H]ow should you act when you’re inside someone’s OODA loop? I was thinking about how like Wikipedia/tab explosions are sort of inside my ooda loop. But sometimes I can be more of an active reader who is navigating the concepts being exposed to me as I choose, and the process becomes like a magic genie or butler who is doing interpretative labour and conjuring up new scenes following my fickle interest.
So it seems like one thing that the person with the smaller loop can do is interpretative labour, and spend the faster cycles on self-legibilising.
Yep, the question is definitely about how far it transfers.
increase the agent's expected future value
I wonder if there's a loopiness here is which breaks the setup (the expectation I'm guessing is relative to the prediction markets probabilities? Though it seems like the market is over sensory experiences but the values are over world states in general, so maybe I'm missing something). But it seems like if I take an action and move the market at the same time, I might be able to extract a bunch of extra money and acquire outsize control.
- Bidding to control the agent's actions for the next N timesteps
This seems like it's wasteful relative to contributing to a pool that bids on action A (or short-term policy P). I guess coordination is hard if you're just contributing to the pool though, and all connects to the merging process you describe.
I mean when I journal I come up with little exercises to improve areas of my life. I imagine that people in your cohort might do similarly, and given that they signed up to improve their IQ, that might include things adjacent to the tasks of the IQ test.
And I don't think general meditation should count as training, but specific meditations could (e.g. if you are training doing mental visualisations and the task involves mental rotations).
I'm not trying to say that there are definitely cross-training effects, just that these seem like the kinds of thing which are somewhat more likely (than, say, supplements) to create fairly narrow improvements close to the test.
And I can make people think “out of the box” (e.g. via specific games, specific “supplements”, specific meditations)
And prod people to think about how they can improve in whatever areas they want (e.g. via journaling, talking, and meditating)
Ah, these two have made me more concerned about training effects: especially the games, but also the meditations and journaling.
It seems pretty plausible certain games could basically train the same skills as the IQ test.
I think this a real problem (tho I think it's more fundamental than your hypothesis would suggest; we could check commenting behaviour in the 2000s as a comparison).
We have some explorations underway addressing related issues (like maybe the frontpage should be more recommender-y and show you good old posts, while the All Posts page is used for people who care a lot about recency). I don't think we've concretely considered stuff that would show you good old posts with new comments, but that might well be worth exploring.
My quick guess is that people don't agree about what constitutes a (relevant) flaw. (And there are lots of irrelevant flaws so you can't just check for the existence of any flaws at all).
I think if people could agree, the authorial incentives would follow. I'm fairly sympathetic to the idea that readers aren't incentivised to correctly agree on what consitutes a flaw.
I notice that the formalised owings have professionals dedicated (in part) to making sure people line up their behaviour with the formalised owings. I wonder if other social credit formalizations would also need this
I recognised the allusion but also disliked the title.
Sometimes running to stand still is the right thing to do
It's nice when good stuff piles up into even more good stuff, but sometimes it doesn't:
- Sometimes people are worried that they will habituate to caffeine and lose any benefit from taking it.
- Most efforts to lose weight are only temporarily successful (unless using medicine or surgery).
- The hedonic treadmill model claims it’s hard to become durably happier.
- Productivity hacks tend to stop working.
These things are like Alice’s red queen’s race: always running to stay in the same place. But I think there’s a pretty big difference between running that keeps you exactly where you would have been if you hadn’t bothered, and running that either moves you a little way and then stops, or running that stops you moving in one direction.
I’m not sure what we should call such things, but one idea is hamster wheels for things that make no difference, bungee runs for things that let you move in a direction a bit but you have to keep running to stay there, and backwards escalators for things where you’re fighting to stay in the same place rather than moving in a direction (named for the grand international pastime of running down rising escalators).
I don't know which kind of thing is most common, but I like being able to ask which dynamic is at play. For example, I wonder if weight loss efforts are often more like backwards escalators than hamster wheels. People tend to get fatter as they get older. Maybe people who are trying (but failing) to lose weight are gaining weight more slowly than similar people who aren’t trying to do so?
Or my guess is that most people will have more energy than baseline if they take caffeine every day, even though any given dose will have less of an effect than taking the same amount of caffeine while being caffeine-naive, so they've bungee ran (done a bungee run?) a little way forward and that's as far as they'll go.
I am currently considering whether productivity hacks, which I’ve sworn off, are worth doing even though they only last for a little while. The extra, but finite, productivity could be worth it. (I think this would count as another bungee run).
I'd be interested to hear examples that fit within or break this taxonomy.
I sometimes like things being said in a long way. Mostly that's just because it helps me stew on the ideas and look at them from different angles. But also, specifically, I liked the engagement with a bunch of epistemological intuitions and figuring out what can be recovered from them. I like in particular connecting the "trend continues" trend to the redoubtable "electron will weight the same tomorrow" intuition.
(I realise you didn't claim there was nothing else in the dialogue, just not enough to justify the length)
How many people read your post is probably meaningful to you, and karma affects that a lot.
I say this because I certainly care about how many people read which posts, so it's kind of sad when karma doesn't track value in the post (though of course brevity and ease of reading are also important and valuable).
- -- Some beliefs about AI Scaling Labs that I'm redacting on LW --
Is the reason for the redaction also private?
Hm, I'm not seeing the issue. Could you screenshot it?
There are separate random number generators for most things in Slay the Spire.
Specifically, this piece of code is executed at the start of a run:
public static void generateSeeds() { logger.info("Generating seeds: " + Settings.seed); monsterRng = new Random(Settings.seed); eventRng = new Random(Settings.seed); merchantRng = new Random(Settings.seed); cardRng = new Random(Settings.seed); treasureRng = new Random(Settings.seed); relicRng = new Random(Settings.seed); potionRng = new Random(Settings.seed); // The following rngs are actually re-initialized each floor: monsterHpRng = new Random(Settings.seed); aiRng = new Random(Settings.seed); shuffleRng = new Random(Settings.seed); cardRandomRng = new Random(Settings.seed); miscRng = new Random(Settings.seed); }
From https://forgottenarbiter.github.io/Correlated-Randomness/ (which does point out there is some unfortunate correlated randomness, though I think there are mods that fix that)
Dungeon Crawl Stone Soup
I had previously guessed air movement made me feel better because my body expected air movement (i.e. some kind of biophilic effect). But this explanation seems more likely in retrospect! I'm not quite sure how to run the calculation using the diffusivity coefficient to spot check this, though.