Posts
Comments
Just edited the post because I think the way it was phrased kind of exaggerated the difficulties we've been having applying the newer models. 3.7 was better, as I mentioned to Daniel, just underwhelming and not as big a leap as either 3.6 or certainly 3.5.
If you plot a line, does it plateau or does it get to professional human level (i.e. reliably doing all the things you are trying to get it to do as well as a professional human would)?
It plateaus before professional human level, both in a macro sense (comparing what ZeroPath can do vs. human pentesters) and in a micro sense (comparing the individual tasks ZeroPath does when it's analyzing code). At least, the errors the models make are not ones I would expect a professional to make; I haven't actually hired a bunch of pentesters and asked them to do the same tasks we expect of the language models and made the diff. One thing our tool has over people is breadth, but that's because we can parallelize inspection of different pieces and not because the models are doing tasks better than humans.
What about 4.5? Is it as good as 3.7 Sonnet but you don't use it for cost reasons? Or is it actually worse?
We have not yet tried 4.5 as it's so expensive that we would not be able to deploy it, even for limited sections.
We use different models for different tasks for cost reasons. The primary workhorse model today is 3.7 sonnet, whose improvement over 3.6 sonnet was smaller than 3.6's improvement over 3.5 sonnet. When taking the job of this workhorse model, o3-mini and the rest of the recent o-series models were strictly worse than 3.6.
I haven't read the METR paper in full, but from the examples given I'm worried the tests might be biased in favor of an agent with no capacity for long term memory, or at least not hitting the thresholds where context limitations become a problem:
For instance, task #3 here is at the limit of current AI capabilities (takes an hour). But it's also something that could plausibly be done with very little context; if the AI just puts all of the example files in its context window it might be able to write the rest of the decoder from scratch. It might not even need to have the example files in memory while it's debugging its project against the test cases.
Whereas a task to fix a bug in a large software project, while it might take an engineer associated with that project "an hour" to finish, requires stretching the limits of the amount of information it can fit inside a context window, or recall beyond what we seem to be capable of doing today.
There was a type of guy circa 2021 that basically said that gpt-3 etc. was cool, but we should be cautious about assuming everything was going to change, because the context limitation was a key bottleneck that might never be overcome. That guy's take was briefly "discredited" in subsequent years when LLM companies increased context lengths to 100k, 200k tokens.
I think that was premature. The context limitations (in particular the lack of an equivalent to human long term memory) are the key deficit of current LLMs and we haven't really seen much improvement at all.
If AI executives really are as bullish as they say they are on progress, then why are they willing to raise money anywhere in the ballpark of current valuations?
The story is that they need the capital to build the models that they think will do that.
Moral intuitions are odd. The current government's gutting of the AI safety summit is upsetting, but somehow less upsetting to my hindbrain than its order to drop the corruption charges against a mayor. I guess the AI safety thing is worse in practice but less shocking in terms of abstract conduct violations.
It helps, but this could be solved with increased affection for your children specifically, so I don't think it's the actual motivation for the trait.
The core is probably several things, but note that this bias is also part of a larger package of traits that makes someone less disagreeable. I'm guessing that the same selection effects that made men more disagreeable than women are also probably partly responsible for this gender difference.
I suspect that the psychopath's theory of mind is not "other people are generally nicer than me", but "other people are generally stupid, or too weak to risk fighting with me".
That is true, and it is indeed a bias, but it doesn't change the fact that their assessment of whether others are going to hurt them seems basically well calibrated. The anecdata that needs to be explained is why nice people do not seem to be able to tell when others are going to take advantage of them, but mean people do. The posts' offered reason is that generous impressions of others are advantageous for trust-building.
Mr. Portman probably believed that some children forgot to pay for the chocolate bars, because he was aware that different people have different memory skills.
This was the explanation he offered, yeah.
This post is about a suspected cognitive bias and why I think it came to be. It's not trying to justify any behavior, as far as I can tell, unless you think the sentiment "people are pretty awful" justifies bad behavior in of itself.
The game theory is mostly an extended metaphor rather than a serious model. Humans are complicated.
Elon already has all of the money in the world. I think he and his employs are ideologically driven, and as far as I can tell they're making sensible decisions given their stated goals of reducing unnecessary spend/sprawl. I seriously doubt they're going to use this access to either raid the treasury or turn it into a personal fiefdom. It's possible that in their haste they're introducing security risks, but I also think the tendency of media outlets and their sources will be to exaggerate those security risks. I'd be happy to start a prediction market about this if a regular feels very differently.
If Trump himself was spearheading this effort I would be more worried.
Anthropic has a bug bounty for jailbreaks: https://hackerone.com/constitutional-classifiers?type=team
If you can figure out how to get the model to give detailed answers to a set of certain questions, you get a 10k prize. If you can find a universal jailbreak for all the questions, you get 20k.
Yeah, one possible answer is "don't do anything weird, ever". That is the safe way, on average. No one will bother writing a story about you, because no one would bother reading it.
You laugh, but I really think a group norm of "think for yourself, question the outside world, don't be afraid to be weird" is part of the reason why all of these groups exist. Doing those things is ultimately a luxury for the well-adjusted and intelligent. If you tell people over and over to question social norms some of those people will turn out to be crazy and conclude crime and violence is acceptable.
I don't know if there's anything to do about that, but it is a thing.
So, to be clear, everyone you can think of has been mentioned in previous articles or alerts about Zizians so far? Because I have only been on the periphery of rationalist events for the last several years, but in 2023 I can remember sending this[1] post about rationalist crazies into the San Antonio LW groupchat. A trans woman named Chase Carter, who doesn't generally attend our meetups, began to argue with me that Ziz (who gets mentioned in the article as an example) was subject to a "disinformation campaign" by rationalists, her goals were actually extremely admirable, and her worst failure was a strategic one in not realizing how few people were like her in the world. At the next meetup we agreed to talk about it further, and she attended (I think for the first time) to explain a very sympathetic background of Ziz's history and ideas. This was after the alert post but years before any of the recent events.
I have no idea if Chase actually self-identifies as a "Zizian" or is at all dangerous and haven't spoken to her in a year and a half. I just mention her as an example; I haven't heard her name brought up anywhere and I really wouldn't expect to know any of these people to begin with on priors.
- ^
Misremembered that I sent the alert post into the chat, but actually it was the Habryka post about rationalist crazies.
I know you're not endorsing the quoted claim, but just to make this extra explicit: running terrorist organizations is illegal, so this is the type of thing you would also say if Ziz was leading a terrorist organization, and you didn't want to see her arrested.
Why did 2 killings happen within the span of one week?
According to law enforcement the two people involved in the shootout received weapons and munitions from Jamie Zajko, and one of them also applied for a marriage certificate with the person who killed Curtis Lind. Additionally I think it's also safe to say from all of their preparations that they were preparing to commit violent acts.
So my best guess is that:
- Teresa Youngblut and/or Felix Bauckholt were co-conspirators with the other people committing violent crimes
- They were preparing to commit further violent crimes
- They were worried that they might be arrested
- They made an agreement with each other to shoot it out with law enforcement in the event someone tried to arrest them
- If the press/law enforcement isn't lying, they were stopped on the road by a border patrol officer that was checking up on a visa, they thought were about to be taken in for something more serious, and *Teresa pulled a gun
The border patrol officer seems like a hero. Whether he meant it or not, he died to save the lives of several other people.
I think an accident that caused a million deaths would do it.
I think this post is quite good, and gives a heuristic important to modeling the world. If you skipped it because of title + author, you probably have the wrong impression of its contents and should give it a skim. Its main problem is what's left unsaid.
Some people in the comments reply to it that other people self-deceive, yes, but you should assume good faith. I say - why not assume the truth, and then do what's prosocial anyways?
You're probably right, I don't actually know many/haven't actually interacted personally with many trans people. But also, I'm not really talking about the Zizians in particular here, or the possibility of getting physically harmed? It just seems like being trans is like taking LSD, in that it makes a person ex-ante much more likely to be someone who I've heard of having a notoriously bizarre mental breakdown that resulted in negative consequences for the people they've associated themselves with.
"Assumed to be dangerous" is overstated, but I do think trans people as a group are a lot crazier on average, and I sort of avoid them personally.
It also seems very plausible to me, unfortunately, that a community level "keep your distance from trans people" rule would have been net positive starting from 2008. Not just because of Ziz; trans people in general have this recurring pattern of involving themselves in the community, then deciding years later that the community is the source of their mental health problems and that they should dedicate themselves to airing imaginary grievances about it in public (or in this case committing violent crimes).
be "born a woman deep down"
start a violent gang
This is the craziest shit I have ever read on LessWrong, and I am mildly surprised at how little it is talked about. I get that it's very close to home for a lot of people, and that it's probably not relevant to either rationality as a discipline or the far future. But like, multiple unsolved murders by someone involved in the community is something that I would feel compelled to write about, if I didn't get the vague impression that it'd be defecting in some way.
Most of the time when people publicly debate "textualism" vs. "intentionalism" it smacks to me of a bunch of sophistry to achieve the policy objectives of the textualist. Even if you tried to interpret English statements like computer code, which seems like a really poor way to govern, the argument that gets put forth by the guy who wants to extend the interstate commerce clause to growing weed or whatever is almost always ridiculous on its own merits.
The 14th amendment debate is unique, though, in that the letter of the amendment goes one way, and the stated interpretation of the guy who authored the amendment actually does seem to go the exact opposite way. The amendment reads:
All persons born or naturalized in the United States, and subject to the jurisdiction thereof, are citizens of the United States and of the State wherein they reside
Which is pretty airtight. Virtually everyone inside the borders of the United States is "subject to the jurisdiction" of the United States, certainly people here on visa, with the possible exception of people with diplomatic immunity. And yet:
So it seems like in this case the textualism vs. intentionalism debate is actually possibly important.
What’s reality? I don’t know. When my bird was looking at my computer monitor I thought, ‘That bird has no idea what he’s looking at.’ And yet what does the bird do? Does he panic? No, he can’t really panic, he just does the best he can. Is he able to live in a world where he’s so ignorant? Well, he doesn’t really have a choice. The bird is okay even though he doesn’t understand the world. You’re that bird looking at the monitor, and you’re thinking to yourself, ‘I can figure this out.’ Maybe you have some bird ideas. Maybe that’s the best you can do
Sarcasm is when we make statements we don't mean, expecting the other person to infer from context that we meant the opposite. It's a way of pointing out how unlikely it would be for you to mean what you said, by saying it.
There are two ways to evoke sarcasm; first by making your statement unlikely in context, and second by using "sarcasm voice", i.e. picking tones and verbiage that explicitly signal sarcasm. The sarcasm that people consider grating is usually the kind that relies on the second category of signals, rather than the first. It becomes more funny when the joker is able to say something almost-but-not-quite plausible in a completely deadpan manner. Compare:
- "Oh boyyyy, I bet you think you're the SMARTEST person in the WHOLE world." (Wild, abrupt shifts in pitch)
- "You must have a really deep soul, Jeff." (Face inexpressive)
As a corollary, sarcasm often works more smoothly when it's between people who already know each other, not only because it's less likely to be offensive, but also because they're starting with a strong prior about what their counterparties are likely to say in normal conversation.
This all seems really clearly true.
Just reproduced it; all I have to do is subscribe to a bunch of people and this happens and the site becomes unusable:
The image didn't upload but it's a picture of my browser saying that the web page's javascript is using a ton of resources and I can force-stop it if I wish
I am a little confused as to why Israel does not have the hostages yet. My understanding was that Israel has essentially taken control of Gaza and decimated the Hamas leadership. Who are they even "negotiating" with to secure their release? Why can't the IDF just kidnap and waterboard that person to get the location of the remaining prisoners? Does the person with the authority to make a deal also not know? Are there clandestine cells of Hamas personnel hiding in a basement somewhere waiting for some "signal" from a third party to give up the Israelis?
Somehow I just found out about this. Although this happened within a few minutes of me trying to use it:
I watched Fall Guy last month because I noticed it had 5 stars on Zvi's letterboxd. Can confirm it's an amazing movie.
Suppose Sam Bankman-Fried is imprisoned for 25 years. After that time, he will be a decent, law-abiding member of society, who is safe to release from prison.
I voted 75% because taken literally I think in 25 years AI will be so advanced that he won't have much of an ability to impact the world at all 🤓
(Otherwise 40%)
That's a lot more than I expected!
I'll note that when people are doing something in near mode, like running a restaurant, they rarely feel it necessary engage in lofty symbolism. You instead get this sort of thing much more often in internet political discussions.
I am skeptical of there being legitimate reasons for talking in "symbolic speak" about the real world. I think one reason people do this is so they can cause in listeners emotional reactions that are appropriate for their "myth" but not appropriate for what's actually true. This is a peculiar way of misleading people, often including one's self.
Another reason is so that people can talk about things without having to take definite stances on what they mean. This ambiguity often just amounts to merely refusing to choose between several plausible truth conditions for their statements, but there's something emotionally attractive about that to some people. This also seems not legitimate...
A reason that is legitimate is to use a metaphor to help someone grasp something by pointing out similarities to things they are already familiar with. This is sometimes done in science education. But the metaphors are discarded as misleading simplifications once understanding progresses.
Of course, a totally valid reason for talking in story is to entertain, as we do in fiction. But in that context everyone knows we are just engaged in entertainment and not talking about the real world."
- Sean Last, during a conversation about the Jordan Peterson/Richard Dawkins religion discussion, which happened a few months back
They have "galaxy brains", but applying those galaxy brains strategically well towards your goals is also an aspect of intelligence. Additionally, those "galaxy brains" may be ineffective because of issues with alignment towards the company, whereas in a startup often you can get 10x or 100x more out of fewer employees because they have equity and understand that failure is existential for them. Demis may be smart, but he made a major strategic error if his goal was to lead in the AGI race, and despite the fact that the did he is still running DeepMind, which suggests an alignment/incentive issue with regards to Google's short term objectives.
On the contrary, massive stellar objects post-AGI will be less useful to you than a house is today, as far as your selfish personal preferences are concerned. Consider the difference in your quality of life living in a nice house vs. skimping and saving 50% and living in a cheap apartment so you can save money. Next, consider the difference in your quality of life owning your own planet (replete with superintelligent servants) vs. owning merely half a planet. What can you do with a whole planet that you can't do with half a planet? Not that much.
It matters if it means I can live twice as long, because I can purchase more negentropy with which to maintain whatever lifestyle I have.
#1 and #2 are serious concerns, but there's not really much I can do about them anyways. #3 doesn't make any sense to me.
You'll probably be able to buy planets post-AGI for the price of houses today
Right, and that seems like OP's point? Because I can do this, I shouldn't spend money on consumption goods today and in fact should gather as much money as I can now? Certainly massive stellar objects post-AGI will be more useful to me than a house is pre-agi?
As to this:
By contrast, *very few people are spending money to influence AGI development right now. If you want future beings to have certain inalienable rights, or if you want the galaxies to be used in such-and-such a way, you can lobby AGI companies right now to change their spec/constitution/RLHF, and to make commitments about what values they'll instill, etc.
I guess I just don't really believe I have much control over that at all. Further, I can specifically invest in things likely to be important parts of the AGI production function, like semiconductors, etc.
If you're so smart why aren't you married?
In many/most police departments, the coroner is the person with final say about whether or not a death was a murder. Like other unaccountable government bureaucrats, coroners can be pretty bad at their jobs, if for no other reason than that there is no one to stop them from being incompentent.
You might wonder why it's not the detective's job to decide whether to open an investigation. And often it's because the detective is graded on the percentage of cases they're able to solve. If detectives were given the responsibility of determining which cases of theirs merited investigation, they might try to avoid investigating cases that seemed more difficult.
Still, the coroner's job is hard, and they will naturally want to keep good rapport with the people they're working with every day. So if there are odd or bizarre features making a death suspicious - a broken window, a recent threat - those circumstances can fail to make it to their report, and what was once an obvious homicide can vanish into thin air.
I find it suspicious that a lot of the criticisms I read online of Indian-Americans (nepotism, obsequiousness, "dual loyalty", scheming) are very similar to the criticisms I hear of Jews.
I don't emphasize this because I care more about humanity's survival than the next decades sucking really hard for me and everyone I love.
I'm flabbergasted by this degree/kind of altruism. I respect you for it, but I literally cannot bring myself to care about "humanity"'s survival if it means the permanent impoverishment, enslavement or starvation of everybody I love. That future is simply not much better on my lights than everyone including the gpu-controllers meeting a similar fate. In fact I think my instincts are to hate that outcome more, because it's unjust.
But how do LW futurists not expect catastrophic job loss that destroys the global economy?
Slight correction: catastrophic job loss would destroy the ability of the non-landed, working public to paritcipate in and extract value from the global economy. The global economy itself would be fine. I agree this is a natural conclusion; I guess people were hoping to get 10 or 15 more years out of their natural gifts.
I think if I got asked randomly at an AI conference if I knew what AGI was I would probably say no, just to see what the questioner was going to tell me.
Saying "I have no intention to kill myself, and I suspect that I might be murdered" is not enough.
Frankly I do think this would work in many jurisdictions. It didn't work for John McAfee because he has a history of crazy remarks, it sounds like the sort of thing he'd do to save face/generate intrigue if he actually did plan on killing himself, and McAfee made no specific accusations. But if you really thought Sam Altman's head of security was going to murder you, you'd probably change their personal risk calculus dramatically by saying that repeatedly on the internet. Just make sure you also contact police specifically with what you know, so that the threat is legible to them as an institution.
If someone wants to murder you, they can. If you ever walk outside, you can't avoid being shot by a sniper.
If the person or people trying to murder you is omnicompetent, then it's hard. If they're regular people, then there are at least lots of temporary measures you can take that would make it more difficult. You can fly to a random state or country and check into a motel without telling anybody where you are. Or you could find a bunch of friends and stay in a basement somewhere. Mobsters used to call doing that sort of thing for a time before a threat had receded "going to ground".
Wearing a camera that is streaming to a cloud 24/7, and your friends can publish the video in case of your death... seems a bit too much. (Also, it wouldn't protect you e.g. against being poisoned. But I think this is not a typical way how whistleblowers die.) Is there something simpler?
If you move to New York or London, your every move outside of a private home or apartment will already be recorded. Then place a security camera in your house.
Postdiction: Modern "cancel culture" was mostly a consequence of new communication systems (social media, etc.) rather than a consequence of "naturally" shifting attitudes or politics.
I have a draft that has wasted away for ages. I will probably post something this month though. Very busy with work.
The original comment you wrote appeared to be a response to "AI China hawks" like Leopold Aschenbrenner. Those people do accept the AI-is-extremely-powerful premise, and are arguing for an arms race based on that premise. I don't think whether normies can feel the AGI is very relevant to their position, because one of their big goals is to make sure Xi is never in a position to run the world, and completing a Manhattan Project for AI would probably prevent that regardless (even if it kills us).
If you're trying to argue instead that the Manhattan Project won't happen, then I'm mostly ambivalent. But I'll remark that that argument feels a lot more shaky in 2024 than in 2020, when Trump's daughter is literally retweeting Leopold's manifesto.