Posts
Comments
I have been using raindrop.io for my bookmarks for seven years or so and it is pretty good. Comments all have permalinks as you know.
Nobody has a deal where they'll pay you to not take an offer from an AI lab right? I realize that would be weird incentives, just curious.
Kinda sucks how it is easy to have infinity rules or zero rules but really hard to have a reasonable amount of rules. It reminds me of how I check my email — either every 5 seconds or every 5 weeks.
Could you say more about where the whole sequence is going / what motivated it? I am curious.
I believe I did explain/decompose the underlying mechanism
A simple way to see this: even a relatively crappy factory will make more goods in a month than it & its employees consume in a month. Likewise for farms, mines, fishing vessels, etc etc.
I could also have mentioned that it's relatively easy for two people to make three.
If someone prints money for themselves, they'll devalue their currency, but they won't be making factories less productive.
Intel makes more stuff than they use, no technical progress required.
I should've said "a dollar's worth of stuff can produce 1.03 dollar's worth of stuff". That would have been more clear.
Easy to say when you're already known by almost everyone in your world, have total career security, and have a full-sized family! I've never really done teaser links, but I can see why anyone would. You're more likely to gain some reputation or a job or a spouse if the reader goes to your website and sees your name there at the top.
Also, in terms of value to the reader: my life has changed in a big way because of a blog post I read two times that I can think of, but never from Twitter, despite spending more time reading Twitter than blogs by now. When I see something important on Twitter, I usually bookmark it and forget about it; when I see something important in a blog post, I often act. This is my own fault, but I suspect it's a common experience. Infinite scroll certainly doesn't help.
No that should be one of the fastest and most cachable queries
I appreciate this post very much! What a great question you have found. Some old thoughts I had on this topic came back to me.
Personally I think of wealth as neither extractive nor hyper-efficient. I think of it as blind dumb compounding growth. A dollar has a lifespan of 1 year and bears 1.07 offspring in that time. A simple way to see this: even a relatively crappy factory will make more goods in a month than it & its employees consume in a month. Likewise for farms, mines, fishing vessels, etc etc. If nothing major goes wrong, money just grows, like a pile of rabbits if the air were made of carrots. Sometimes there's lots of opportunity to be clever and bump up a couple productivity notches and go rags to riches. Other times there's no low fruit on the technological/economic tech tree and you can only get richer quick by stealing/scamming/pillaging. But the core dynamic is that it's relatively easy for anyone to make more than they use, once they have 1/.07=14x their annual spending available as liquid cash (or tied up in decent investments).
This is all because we live in a universe that clearly wants things to happen. We have the greatest endowment ever, a whole damn star, and others nearby! Nuclear energy sitting in the ground if you need cash fast. It's better than ZIRP, better than a stimulus check! You don't even have to find a trade partner; you can use it directly to do anything you want, no questions asked. No credit check or collateral. No minimum hours per week; in fact, nothing is asked in exchange at all; you couldn't give anything back even if you tried — usable energy seems to only go down — but we start with so much that it won't matter for a trillion years.
This notion hasn't made me rich and it's not much use for my day-to-day decisions, but I think it does describe the reality I see. My parent's friends are all rich (compared to young people) from a lifetime of saving, even the dumb ones. Half the crappy stocks I owned doubled simply because the companies stayed in business; if energy is free then "stay in business" is all you need to do once you reach $10M in annual profits or so. If an individual holds more than ~$100M, they'll make a good profit indiscriminately investing in every single startup and land offering that shows up in their face: with compounding growth and diminishing losses, the losses just don't really matter. This seems to me to essentially be the fundamental law of our universe, at least until some pine tree is tall enough to block out everyone's light at once. (That pine tree may only be 1-4 years away now.)
Wrt swimsuit+umbrella, I feel similarly to Dave Orr. I don't see why eg the matter of AI Safety vs Global Health has to be settled. Effective altruism is more coherent IMO than most social movements and even some governments (eg I know someone who is paid by the US federal govt to help immigrants avoid ICE). I don't know if coherence much beyond the current level can be achieved without just falling into blind ideology. Even a very thoughtful and rational collective decision making process might be worse than the current chaotic one, due to missed opportunities. You disagree?
I am glad to see somebody make the point properly. It's a weird state of affairs. We know the models can implement PoCs for CVEs better than most coders. We know the models can persuade people pretty effectively. Obviously the models can spread and change very easily. It's also easy for a rogue deployment to hide because datacenter GPUs draw 70W idle and update scripts constantly use tons of bandwidth. There's just no urgency to any of it.
How do you measure results?
If you wanted to take this idea to an absurd level, you could install a dropped ceiling made partially of furnace filters, and a grid of fans above it. Maybe have the outer perimeter of fans blowing up and the inner area blowing down, to try to get one large convection through the entire room.
How do you figure out the optimal filter thickness? If you hypothetically had a very weak fan then it wouldn't push much air through even furnace filters. If you had a magic constant air flow source then you would want the thickest filter possible.
I guess I am just wondering if you could use something better-looking and cheaper, like semi-transparent paper with lights behind it or a washable sheet/tapestry.
Have you heard of Big Ass Fans? It's a company that makes what you would expect. Do you think your ceiling fan filter could work with a 30ft fan?
What is the current popular (or ideally wise) wisdom wrt publishing demos of scary/spooky AI capabilities? I've heard the argument that moderately scary demos drive capability development into secrecy. Maybe it's just all in the details of who you show what when and what you say. But has someone written a good post about this question?
Einstein started doing research a few years before he actually had his miracle year. If he started at 26, he might have never found anything. He went to physics school at 17 or 18. You can't go to "AI safety school" at that age, but if you have funding then you can start learning on your own. It's harder to learn than (eg) learning to code, but not impossibly hard.
I am not opposed to funding 25 or 30 or 35 or 40 year olds, but I expect that the most successful people got started in their field (or a very similar one) as a teenager. I wouldn't expect funding an 18-year-old to pay off in less than 4 years. Sorry for being unclear on this in original post.
I don't have a witty, insightful, neutral-sounding way to say this. The grantmakers should let the money flow. There are thousands of talented young safety researchers with decent ideas and exceptional minds, but they probably can't prove it to you. They only need one thing and it is money.
They will be 10x less productive in a big nonprofit and they certainly won't find the next big breakthrough there.
(Meanwhile, there are becoming much better ways to make money that don't involve any good deeds at all.)
My friends were a good deal sharper and more motivated at 18 than now at 25. None of them had any chance at getting grants back then, but they have an ok shot now. At 35, their resumes will be much better and their minds much duller. And it will be too late to shape AGI at all.
I can't find a good LW voice for this point but I feel this is incredibly important. Managers will find all the big nonprofits and eat their gooey centers and leave behind empty husks. They will do this quickly, within a couple years of each nonprofit being founded. The founders themselves will not be spared. Look how the writing of Altman or Demis changed over the years.
The funding situation needs to change very much and very quickly. If a man has an idea just give him money and don't ask questions. (No, I don't mean me.)
Wasted opportunity to guarantee this post keeps getting holywar comments for the next hundred years.
This is pretty inspiring to me. Thank you for sharing.
The other day I was trying to think of information leaks that a competent conspiracy couldn't prevent, regarding this. I just thought of one small one: people will sometimes randomly die or have their homes raided. If the slavery is common, then sometimes the slaves will be discovered during these events. Even if the escapees wanted to silence the story out of shame, cops would probably gossip to the press.
So you can probably tally such events, crunch the numbers, and get a decent conspiracy-resistant estimate.
As a layman, I have not seen much unrealistic hype. I think the hype-level is just about right.
You should not bury such a good post in a shortform
Maybe it should be a game that everyone can play
Yeah I just wanted to check that nobody is giving away money before I go do the exact opposite thing I've been doing. I might try to tidy something up and post it first
I do think I could put a good team together and make decent contributions quickly
I can only find capabilities jobs right now. I would be interested in starting a tiny applied research org or something. How hard is it to get funding for that? I don't have a strong relevant public record, but I did quite a lot of work at METR and elsewhere.
I wonder if anybody has tried to quantify how much it's worth to be a swing voter. I imagine if you are the government contractor up for renewal then it's worth quite a lot, but I wonder how much of the money/benefits the average Joe sees.
I don't know much about swing state benefits except that Milwaukee, Wisconsin got their lead pipes replaced by the fed and the workers were required to be local and they say they were paid quite well https://youtube.com/watch?v=4VpwgG0P8VU
Aw man we used the same word for different things again
Your examples fit the definition quite well. Apparently this is in the dictionary now. https://www.merriam-webster.com/dictionary/gaslighting
Regarding this
Such as the moms in the abusive partners example above: each one could acknowledge her self-deception once it was safe for her abusive partner to know too. She got enough power (financial or social) to protect herself and her child, making the telepathic scan no longer a dire threat.
I would add that most abusive people don't really like crushing their loved ones and it is sometimes easy to get them to stop, eg by having a peer of the abuser get a private word with the two parties separately. I think it is common for there to be simple miscommunication/misunderstanding — the abuser does not typically actually benefit from the accusative situation.
Why haven't abuser & abusee already talked and figured this out? Well there is some force field where you can't have a normal conversation with someone who is hitting you (or you are hitting) about the hitting. Although I don't know how to put it in your terms here from this post.
What gaslighting goes on in math class?
I am impressed with how far you thought this through. Amend the constitution, including the constitution amendment section
The opposing states in the coalition will simply declare war against the defectors. It's surely worth keeping your own army to keep being a swing bloc.
I want this to be a board game
I don't know if this would be good for the country, but it would certainly be good political entertainment.
Doesn't everybody always code in a strong time-discount? I have never seen code without it.
The o1 calculation is correct! https://math.stackexchange.com/a/1264753
.5 * (1 - sqrt(1.5e11^2 - 6.4e6^2)/1.5e11) = 4.55e-10
I am surprised. I have seen it mix up million and billion when calculating how many nukes the solar energy that hits earth is equivalent to.
Of course the sun is not nearly a point but whatever.
No good science without some good fun.
Wrong link? Looks like this is it https://arxiv.org/abs/2409.06927
Here is my understanding. Is this right?
Incredible!! I am going to try this myself. I will let you know how it goes.
honesty vector tuning showed a real advantage over honesty token tuning, comparable to honesty vector steering at the best layer and multiplier:
Is this backwards? I'm having a bit of trouble following your terms. Seems like this post is terribly underrated -- maybe others also got confused? Basically, you only need 4 terms, yes?
* base model
* steered model
* activation-tuned model
* token cross-entropy trained model
I think I was reading half the plots backwards or something. Anyway I bet if you reposted with clearer terms/plots then you'd get some good followup work and a lot of general engagement.
Hey!!! Thanks for replying. But did you or anyone you know consider chemical cisgenderization? Or any mention of such in the forums? I would it expect it to be a much stronger effect than eg joining the military. Although I hear it is common for men in the military to take steroids, so maybe there would be some samples there.... I imagine taking cis hormones is not an attractive idea, because if you dislike the result then you're worse off than you started.
(Oh and we were still together then. LK has child now, not sure how that affects the equation.)
Thank you! Seems like this bot works quite well for this task
I have used a number of discourse forums and they just feel bad/wrong but I cannot explain why. I would also vote for more of an old-fashioned php BB with a nice theme. Those are always great, even though all my intuitions tell me they seem like they should suck. Shows how little I know.
Eg https://github.com/phpbb/phpbb
Also has styles: https://www.phpbb.com/customise/db/styles/board_styles-12?sid=6245508b90fd3410be19888406fae215
Basically I'm repeating what Said said
If you have a clear metric to judge candidates on (eg engagement on a linkedin ad) then you might be able to do a super effective and quick performance-based hiring method. Shameless plug: https://www.lesswrong.com/posts/3AZkXwcCJZc5CAFQN/how-to-hire-somebody-better-than-yourself
Good luck!
Thanks for the cached explanation, this is similar to what I thought before a few days ago. But now I'm thinking that an older-but-still-youthful mouse would be better at avoiding predators and could be just as fertile, if mice were long lived. So the food & shelter might be "better spent" on them, in terms of total expected descendants. This would only leave the disease explanation, yes?
Where has the "rights of the living vs rights of the unborn" debate already been had? In the context of longevity. (Presuming that at some point an exponentially increasing population consumes its cubically increasing resources.)
Hey thanks much for sharing new info with me. What a nice comment to read. I was sure someone would come by and be pissed and mean as hell, but folks have been engaging in quite good faith.
but I'm more reserved
I think this might point at the central problem with my evidence. People vary in how publicly they live their lives by orders of magnitude. It could be that only 1% of math geniuses are trans women but they post / get views on Twitter 100x more. Or a similar thing in high school and the workplace. Math professors tend to live quiet lives...
Anyway, unfortunately I think this post might be kinda too toxic/hurtful for the average reader to be worthwhile overall (although nobody has mentioned that to me) and I'll probably move it to a pastebin or something.
I think the basic question (whether hormones are fucking or helping your brain long-term) is quite important and deserves a better treatment. I might try to do that eventually.
Any ideas?