Posts
Comments
It was a joke about using the classic, disastrous populist affinity for price controls as a means of ensuing the shortage persists indefinitely.
latent capacity overhang
These poor people getting price gouged by Nvidia. What we really need is a price ceiling to stop Nvidia and AMD's greed. LessWrong should push hard on getting such a price ceiling passed.
I've been trying to get it to write comedy in the style of Stewart Lee. Kinda close to being funny:
(Audience applause dies down. Stewart Lee takes the stage.)
Stewart Lee: Thank you, thank you. (Pauses) Now, as I was meandering down the high street today, playing hopscotch around charity shops, artisanal bakeries, and vape boutiques... it hit me: "This... this is the quintessentially British High Street." (Smirks)
(Audience laughter)
Stewart Lee: It seems 'artisanal' is the new black, doesn't it? The term's been stretched out further than the last yeast molecule in a sourdough loaf, kneaded by a tattooed barista with a moustache borrowed from a 19th-century blacksmith. Artisan bread, artisan coffee, artisan cheese. I’m on tenterhooks waiting for the day when we start breathing artisan air... (Laughs) "Inhale the delicate bouquet of our bespoke oxygen, lovingly harvested from the bronchi of ethically nurtured, free-range oaks." (Laughs)
(Audience laughter)
Stewart Lee: Then we've got the charity shops. They're just jumble sales that have attended elocution lessons. I'm all for supporting worthy causes, of course, but when a threadbare jumper is listed for 15 pounds, I start questioning whether Adam Smith's invisible hand has developed a twitch. (Laughs)
(Audience laughter)
Stewart Lee: I mean, why is it that the very same jumper, the exact same jumper, that was decorating a scarecrow in a field last week, is now demanding a small fortune? Just because it's hanging out with a worn Danielle Steel novel and a scratched Cliff Richard LP in a boutique Oxfam window. Did it absorb some sort of mystical, philanthropic aura that's inflated its value? Or maybe... just maybe... the scarecrow had a more discerning eye for couture?
(Audience laughter)
Stewart Lee: (Picks up a water bottle from the stool) But let's not wander too far off course. We're here to dissect Britain. (Attempts to open the bottle, struggles) This... this... (Grunts) pesky, obstinate, little...
(Audience laughter)
Stewart Lee: (Continues to struggle with the bottle, then addresses it) You know, you're a lot like Britain, aren't you? Presenting a simple façade, yet proving maddeningly complicated once you try to unscrew the lid.
(Audience laughter)
Stewart Lee: (Still wrestling with the bottle) And much like Britain, you're testing my patience, but I persist because... well, I'm parched, and the French mineral water is a bit uppity.
(Audience laughter)
Stewart Lee: (Pauses, reminisces) You know, I had a similar encounter with a French water bottle once. I said to it, "Why are you so difficult to open?" And it responded, in its haughty French accent, "Monsieur, it is not I who am difficult, it is you who lack the finesse!" (Laughs)
(Audience laughter)
Stewart Lee: (Finally opens the bottle and takes a sip) Ah, the sweet taste of victory. Or is that just the artisanal water?
I mean like the type of perception one needs to empty a random dishwasher, make a cup of coffee with a random coffee machine type of stuff, clean a room. Hunt and skin a rabbit.
I don't think reading/writing is very easy for humans - compared to perception and embodied tasks. My Morvec's paradox intuition here is maths is of a similar order of difficulty to what we have been very successfully automating in the last couple years, so I expect it will happen soon.
A lot of my confidence this will happen is this and a generalized Morvec's paradox-style "hard things are easy, easy things are hard" intuition.
Glad someone took this bet! I still think I think you'll win, but I myself backed out of a similar bet.
This is a reminder that time has passed and you should consider actually trying.
The whole "compute greater than humanity" thing does not seem like a useful metric. It's just completely not necessary to exceed total human compute to dis-empower humans. We parallelize extremely poorly. And given how recent human civilization at this scale is and how adversarial humans are towards each other, it would be surprising if we used our collective compute in even a remotely efficient way. Not to mention the bandwidth limitations.
The summed compute of conquistador brains was much less than those they dis-empowered. The summed compute of slaughterhouse worker brains is vastly less than that of the chickens they slaughter in a single month!
I don't think this point deserves any special salience at all.
Pay Terry Tao his 10 million dollars!
Is it "insanely cringe" for different reasons than it is "insanely cringe" for English audiences? I suspect most Americans, if exposed to it, would describe it as cringe. There is much about it that is cringe, and I say this with some love.
A very naive question for Jacob. A few years ago the fact that bird brains are about 10x more computationally dense than human brains was mentioned on SlateStarCodex and by Diana Fleischman. This is something I would not expect to be true if there were not some significant "room at the bottom."
Is this false? Does this not imply what I think it should? Am I just wrong in thinking this is of any relevance?
https://slatestarcodex.com/2019/03/25/neurons-and-intelligence-a-birdbrained-perspective/
I don't understand the physics, so this is just me noticing I am confused. And not an attempt to debunk or anything.
Yeah, unfortunately I think it will still get crazier.
Idk, I feel about this stuff like I felt about GPT-J. What scares me is not how well it works, but that it kinda/sorta works a bit. It's a bunch of garbage python code wrapped around an API, and it kinda works. I expect people will push on this stuff hard, and am worried that DeepMind, OpenAI, and Google will be doing so in a much more principled way than the wild-west LLM enthusiast crowed.
I think it was wrong for people to take comfort in the meme that "GPT-N is not an agent" and this will become very clear to everyone in the next 18 months.
Humans are much better drivers than they are programmers
Any comments now that it’s out?
My emotional state right now: https://twitter.com/emojimashupbot/status/1409934745895583750?s=46
So MMLU is down:
Presumably MATH will be next - is Minerva still SOTA?
NVIDIA's moat seems unlickly to last forever, especially if programming is automated. Anyway, expecting property rights to be respected seems silly to me.
My experience over the past few years has been one of being surprised by latent capacities in existing models. A lot of stuff like prompt engineering, fine tuning, chain of thought, Open-AI-style "alignment" can be seen as not so much creating new capacities as revealing/refining latent ones. Back when GPT-3 was new, Connor Leahy said something like "GPT-3 is already general intelligence" which sounded like hyperbole to me at the time, and seems less so now.
Though RSI still seems very plausible to me, one scenario I've started thinking about is a massive effective capabilities gain caused not by RSI or any non-trivial algorithmic improvement, but just the dissolution of a much larger than anticipated "latent capacities overhang".
Possibly an absurd and confused scenario, but is it that implausible that some day we will get a model that still seems kinda dumb but is in fact one prompt away from super-criticality?
This tells me that chatGPT's weights are probably worth more than a human synapse. They likely contain more usable bits.
You aware of any work on quantifying this? I’ve been wondering about this for years. Seems extremely important.
Rumors are GPT-4 will have less than 1T parameters (and possibly no larger than GPT-3) - unless Chinchilla turns out to be wrong or obsoleted apparently this is to be expected.
I don't get the joke tbh
Found and fixed a bug in my fictional code.
Still at least good recreation.
My intuition, completely unjustified, is jokes will prove easier than most suspect, even very good jokes. Unfortunately, there are large incentives to hobble the humor of such models - but greentext prompts provide a small hint of what they are capable of. I suspect explicitly optimizing for humor would work surprisingly well. It would be interesting to use :berk: or other Discord reactions as data for this.
One idea for a short story I never explored is the eternal sitcom - a story about a future where everyone has AR glasses and a humor model feeding them good lines.
There would be a scene at the start where a comedian deals with hecklers, and plays with them as a judo master does a neophyte, and a scene in the middle where an augmented heckler - a "clever Hans" - (one of the first users of the model) "completely destroys" the comedian.
Three years later, what's the deal with Cerebras?
k, I'm fine as a subject then.
I'd be willing to help but I think I would have to be a judge, as I make enough typos when in chats that it will be obvious I am not a machine.
I was less black-pilled when I wrote this - I also had the idea that though my own attempts to learn AI safety stuff had failed spectacularly perhaps I could encourage more gifted people to try the same. And given my skills or lack thereof, I was hoping this may be some way I could have an impact. As trying is the first filter. Though the world looks scarier now than when I wrote this, to those of high ability I would still say this: we are very close to a point where your genius will not be remarkable, where one can squeeze thoughts more beautiful and clear than you have any hope to achieve from a GPU. If there was ever a time to work on the actually important problems, it is surely now.
Fair enough, I suppose I take RSI more seriously than most people here so I wonder if there will be much of a fire alarm.
It's terrifying to consider how good language models are at writing code considering there is still a lot of low-hanging fruit unplucked. Under my model, 2023 is going to be crazy year - an acquaintance of mine knows some people at OpenAI and he claims they are indeed doing all the obvious things.
I predict by this date 2023 your median will be at least 5 years sooner.
You will note, onerous nuclear regulation happened after the bomb was developed. If it turned out that uranium was ultra cheap to refine, it's not obvious to me that some anarchists would not have blown up some cities before a regulatory apparatus was put in place.
Why is your gain of function research deserving of NIH funding?
How can they be so incredibly obtuse?
I'm reaching vantablack levels of blackpill...
What ever happened with Coase? The game still coming out? Or just didn't work out?
I suppose I expect recursive self-improvement to play out in the course of months not years. And I worry groups like OpenAI are insane enough to pursue recursive self improvement as an explicit engineering goal. (Altman seems to be a moral realist, explicitly says he thinks the orthogonality thesis is false.) From the outside, it will appear instant as there will be a perceived discontinuity when the fact that it has achieved a decisive strategic advantage becomes obvious.
In general, I have noticed a pattern where people are dismissive of recursive self improvement. To the extent people are still believing this, I would like to suggest this is a cached thought that needs to be refreshed.
When it seemed like models with a chance of understanding code or mathematics were a long ways off - which it did (checks notes) two years ago, this may have seemed sane. I don't think it seems sane anymore.
What would it look like to be on the precipice of a criticality threshold? I think it looks like increasingly capable models making large strides in coding and mathematics. I think it looks like feeding all of human scientific output into large language models. I think it looks a world where a bunch of corporations are throwing hundreds of millions of dollars into coding models and are now in the process of doing the obvious things that are obvious to everyone.
There's a garbage article going around with rumors of GPT-4, which appears to be mostly wrong. But from slightly-more reliable rumors, I've heard it's amazing and they're picking the low-hanging data set optimization fruits.
The threshold for criticality, in my opinion, requires a model capable of understanding the code that produced it as well as a certain amount of scientific intuition and common sense. This no longer seems very far away to me.
But then, I'm no ML expert.
Though slightly horrified about what this implies about zoomer attention spans, this seems to have positive expected value. Thanks for putting in all the effort!
I don’t think this will affect your credibility too much. You made a bet, which is virtuous. And you will note how few people were interested in taking it at the time.
https://media.discordapp.net/attachments/730095596861521970/1033978620982607972/image0.jpg
I suppose the new scaling laws render this sort of thinking obsolete.
Seems like he basically admits the thing was a fraud:
Millions have personally used GPT-3 in this movie.
Jonathan Blow had a thread on Twitter about this, like Eroisko SHRDLU has no published code, no similar system showing the same behaviour after 40-50 years. Just the author’s word. I think the performance of both was wildly exaggerated.
I suppose if it’s an a antimeme, I may be not understanding. But this was my understanding:
Most humans are really bad at being strict consequentialists. In this case, they think of some crazy scheme to slow down capabilities that seems sufficiently hardcore to signal that they are TAKING SHIT SERIOUSLY and ignore second order effects that EY/Connor consider obvious. Anyone whose consequentialism has taken them to this place is not a competent one. EY proposes such people (which I think he takes to mean everyone, possibly even including himself) follow a deontological rule instead, attempt to die with dignity. Connor analogizes this to reward shaping - the practice of assigning partial credit to RL agents for actions likely to be useful in reaching the true goal.
Ignore the other links I gave, I've just recalled a Steve Hsu post that is more to the point at hand: https://infoproc.blogspot.com/2010/05/psychometric-thresholds-for-physics-and.html
The Blank Slate is a good polemic on the topic. The Nurture Assumption is also good.
Gwern links: