Posts

Virtually Rational - VRChat Meetup 2024-01-28T05:52:36.934Z
Global LessWrong/AC10 Meetup on VRChat 2024-01-24T05:44:26.587Z
What Software Should Exist? 2024-01-19T21:43:50.112Z
The Drowning Child 2023-10-22T16:39:53.016Z
Taboo Truth 2023-07-08T23:23:27.942Z
Beauty and the Beast 2022-06-11T18:59:58.412Z
The Liar and the Scold 2022-01-20T20:31:14.765Z
The Maker of MIND 2021-11-20T16:28:56.327Z
On Raising Awareness 2021-11-17T17:12:36.843Z
The Best Software For Every Need 2021-09-10T02:40:13.731Z
(2009) Shane Legg - Funding safe AGI 2021-07-17T16:46:24.713Z
What would it look like if it looked like AGI was very near? 2021-07-12T15:22:35.321Z
Going Out With Dignity 2021-07-09T22:07:59.169Z
Irrational Modesty 2021-06-20T19:38:25.320Z
AI-Based Code Generation Using GPT-J-6B 2021-06-16T15:05:26.381Z
A Breakdown of AI Chip Companies 2021-06-14T19:25:46.720Z
Parameter vs Synapse? 2021-03-11T15:30:59.745Z
Thoughts on the Repugnant Conclusion 2021-03-07T19:00:37.056Z

Comments

Comment by Tomás B. (Bjartur Tómas) on Partial value takeover without world takeover · 2024-04-05T14:57:47.611Z · LW · GW

I was in the middle of writing a frustrated reply to Matthew's comment when I realized he isn't making very strong claims. I don't think he's claiming your scenario is not possible. Just that not all power seeking is socially destructive, and this is true just because most power seeking is only partially effective. Presumably he agrees that in the limit of perfect power acquisition most power seeking would indeed be socially destructive. 

Comment by Tomás B. (Bjartur Tómas) on LessWrong's (first) album: I Have Been A Good Bing · 2024-04-03T14:35:34.080Z · LW · GW

Every toaster a Mozart, every microwave a Newton, every waffle iron a Picasso.

Comment by Tomás B. (Bjartur Tómas) on LessWrong's (first) album: I Have Been A Good Bing · 2024-04-01T15:10:09.102Z · LW · GW

Was this all done through Suno? You guys are much better at prompting it than I am.

Comment by Tomás B. (Bjartur Tómas) on On Devin · 2024-03-22T15:35:27.442Z · LW · GW

The bet is with a friend and I will let him judge.

I agree that providing an api to God is a completely mad strategy and we should probably expect less legibility going forward. Still, we have no shortage of ridiculously smart people acting completely mad.

Comment by Tomás B. (Bjartur Tómas) on On Devin · 2024-03-18T15:08:44.339Z · LW · GW

This seems to be as good of a place as any to post my unjustified predictions on this topic, the second of which I have a bet outstanding on at even odds.

  1. Devin will turn out to be just a bunch of GPT-3.5/4 calls and a pile of prompts/heuristics/scaffolding so disgusting and unprincipled only a team of geniuses could have created it.
  2. Someone will create an agent that gets 80%+ on SWE-Bench within six months. 

I am not sure if 1. being true or false is good news. Both suggest we should update towards large jumps in coding ability very soon.

Regarding RSI, my intuition has always been that automating AI research will likely be easier than automating the development and maintenance of a large app like, say, Photoshop, So I don't expect fire alarms like "non-gimmicky top 10 app on AppStore was developed entirely autonomously" before doom.

Comment by Tomás B. (Bjartur Tómas) on OpenAI wants to raise 5-7 trillion · 2024-03-11T05:05:46.832Z · LW · GW
Comment by Tomás B. (Bjartur Tómas) on The Last Laugh: Exploring the Role of Humor as a Benchmark for Large Language Models · 2024-03-10T02:25:31.501Z · LW · GW

After spending several hours trying to get Gemini, GPT-4 and Claude 3 to make original jokes - I now think I may be wrong about this. Still could be RLHF, but it does seem like an intelligence issue. @janus are the base models capable of making original jokes?

Comment by Tomás B. (Bjartur Tómas) on Increasing IQ is trivial · 2024-03-04T22:36:46.600Z · LW · GW

Looks to me he's training on the test set tbh. His ambition to get an IQ of 195 is admirable though. 

Comment by Tomás B. (Bjartur Tómas) on Increasing IQ is trivial · 2024-03-02T23:36:06.768Z · LW · GW

I very much doubt this will work. I am also annoyed you don't share your methods. If you can provide me with a procedure that raises my IQ by 20 points in a manner that convinces me this is a real increase in g, I will give you one hundred thousand dollars.

Comment by Tomás B. (Bjartur Tómas) on Supposing the 1bit LLM paper pans out · 2024-03-02T17:30:20.534Z · LW · GW

@Veedrac suppose this pans out and custom hardware is made for such networks.  How much faster/larger/cheaper will this be?

Comment by Tomás B. (Bjartur Tómas) on Supposing the 1bit LLM paper pans out · 2024-02-29T23:39:16.388Z · LW · GW

This is applied to training. It’s not a quantization method.

Comment by Tomás B. (Bjartur Tómas) on China-AI forecasts · 2024-02-25T20:01:43.009Z · LW · GW

Hsu on China's huge human capital advantage:

Returning to Summers' calculation, and boldly extrapolating the normal distribution to the far tail (not necessarily reliable, but let's follow Larry a bit further), the fraction of NE Asians at +4SD (relative to the OECD avg) is about 1 in 4k, whereas the fraction of Europeans at +4SD is 1 in 33k. So the relative representation is about 8 to 1. (This assumed the same SD=90 for both populations. The Finnish numbers might be similar, although it depends crucially on whether you use the smaller SD=80.) Are these results plausible? Have a look at the pictures here of the last dozen or so US Mathematical Olympiad teams (the US Asian population percentage is about 3 percent; the most recent team seems to be about half Asians). The IMO results from 2007 are here. Of the top 15 countries, half are East Asian (including tiny Hong Kong, which outperformed Germany, India and the UK).

Incidentally, again assuming a normal distribution, there are only about 10k people in the US who perform at +4SD (and a similar number in Europe), so this is quite a select population (roughly, the top few hundred high school seniors each year in the US). If you extrapolate the NE Asian numbers to the 1.3 billion population of China you get something like 300k individuals at this level, which is pretty overwhelming.

If you think AGI will not come in the next 5-10 years then I think there is a very good chance it comes from China. Also, China certainly has the engineering talent to surpass the rest of the world in basically everything, including fabs. Personally, I expect AGI in the new few years though.  

Comment by Tomás B. (Bjartur Tómas) on What is the research speed multiplier of the most advanced current LLMs? · 2024-02-21T15:23:17.427Z · LW · GW

GPT-5 with a context window that can fit entire code bases is going to be very scary. Particularly if you think, as I do, that agency is going to start to work soon. I really do think at least "weak recursive self improvement" of the form of automating AI research/training loops is on the table relatively soon. 

Comment by Tomás B. (Bjartur Tómas) on The Last Laugh: Exploring the Role of Humor as a Benchmark for Large Language Models · 2024-02-12T20:28:38.094Z · LW · GW

I would like to register a prediction. I believe a GPT-4-level model that has been RLHFd for humour will be super-human or near superhuman at humour.  At least in the 99th percentile of professional comedians. My intuition is humour is much easier than people think, and current models fail at it mostly because the forms of RLHF existing models use pushed them into humourlessness . 

Comment by Tomás B. (Bjartur Tómas) on OpenAI wants to raise 5-7 trillion · 2024-02-09T17:29:22.956Z · LW · GW

Conditional on this being true, he must be very certain we are close to median human performance, like on the order of one to three years.  I don't think this amount of capital can be efficiently expended in the chips industry unless human capital is far less important than it once was. And it will not be profitable, basically, unless he thinks Winning is on the table in the very near term.

Comment by Tomás B. (Bjartur Tómas) on OpenAI wants to raise 5-7 trillion · 2024-02-09T16:49:25.684Z · LW · GW

I feel 5 trillion must be a misprint. This is like several years worth of American tax revenues. Conditional on this being true I would take this as significant evidence that what they have internally is unbelievably good. Perhaps even an AI with super-persuasion!

It is such a ridiculous figure, I suspect it must be off by at least an OOM. 

Comment by Tomás B. (Bjartur Tómas) on Why I take short timelines seriously · 2024-01-30T15:38:05.624Z · LW · GW

My take on self-driving taking forever is driving is near AGI complete. Humans drive roughly a million miles between fatal accidents; it would not be particularly surprising if in these million miles (where you are interacting with intelligent agents) you inevitably encounter near AGI-complete problems. Indeed, as the surviving self-driving companies are all moving to end-to-end approaches, self-driving research is begining to resemble AGI research more and more.

Comment by Tomás B. (Bjartur Tómas) on What should a non-genius do in the face of rapid progress in GAI to ensure a decent life? · 2024-01-02T01:23:15.522Z · LW · GW

I bought index funds. I would say it has the advantage of being robust to AGI not happening, but with birth rates as they are I am not so sure that's true! If we survive, Hanson's economic growth calculations predict the economy will start doubling every few months. Provided the stock market can capture some of this, I guess learning how to live on very little (you really want to avoid burning your capital in this future, so should live as modestly as possible both so you can acquire capital and so you can use as little as possible until the market prices in such insane growth) and putting everything in index funds should be fine with even modest amounts of invested capital.  However, I doubt property rights will be respected.

Comment by Tomás B. (Bjartur Tómas) on Can a stupid person become intelligent? · 2023-11-09T05:49:44.889Z · LW · GW

Any evidence for it working? Seriously doubt.

Comment by Tomás B. (Bjartur Tómas) on Can a stupid person become intelligent? · 2023-11-09T05:45:37.465Z · LW · GW

Nope. Sadly. And if there were, your intellect would not be impressive for such tools would reach fixation.

If it’s any consolation, all the brilliant people able to make many multiples of your salary due to being born with a better brain - while almost to a man being incredibly smug about it - will soon be losing intellectual death matches with toaster ovens.

Comment by Tomás B. (Bjartur Tómas) on Stuxnet, not Skynet: Humanity's disempowerment by AI · 2023-11-05T13:44:24.016Z · LW · GW

https://youtu.be/OGxdgNJ_lZM

Comment by Tomás B. (Bjartur Tómas) on My AI Predictions 2023 - 2026 · 2023-10-16T16:44:14.470Z · LW · GW

And OpenAI has explicitly said this is what they want to do! Their Superalignment strat looks suspiciously like "gunning for RSI".

Comment by Tomás B. (Bjartur Tómas) on My AI Predictions 2023 - 2026 · 2023-10-16T16:41:50.076Z · LW · GW

It does seem to me a little silly to give competitors API access to your brain. If one has enough of a lead, one can just capture your competitors markets. 

Comment by Tomás B. (Bjartur Tómas) on Intelligence Enhancement (Monthly Thread) 13 Oct 2023 · 2023-10-16T16:36:28.758Z · LW · GW

I think I may be almost crazy enough to volunteer for such a procedure, ha, should you convince me. 

Comment by Tomás B. (Bjartur Tómas) on Intelligence Enhancement (Monthly Thread) 13 Oct 2023 · 2023-10-16T16:32:51.669Z · LW · GW

This comment was assuming causal variants are known, which I admit is a big gimme. More of a first-principles type eye-balling.   

Comment by Tomás B. (Bjartur Tómas) on Intelligence Enhancement (Monthly Thread) 13 Oct 2023 · 2023-10-16T16:17:02.494Z · LW · GW

So I made this comment awhile back, though I admit being ignorant on how good modern somatic gene therapy is:

I think somatic gene therapy, while technically possible in principal, is extremely unpromising for intelligence augmentation. Creating a super-genius is almost trivial with germ-line engineering. Provided we know enough causal variants, one needs to only make a low-hundreds number of edits to one cell to make someone smarter than any human that has ever lived. With somatic gene therapy you would almost certainly have to alter billions of cells to get anywhere.

Am I just wrong here? Is somatic gene therapy really robust and error-free enough to safely edit billions of cells? 

Comment by Tomás B. (Bjartur Tómas) on My AI Predictions 2023 - 2026 · 2023-10-16T15:52:10.127Z · LW · GW

This is a very good, and very scary point - another thing that could provide, at least the appearance of, a discontinuity. One symptom of this this scenario would be a widespread, false belief that "open source" models are SOTA.

Might be good to brainstorm other symptoms to prime ourselves to recognize when we are in this scenario. Complete hiring-freezes/massive layoffs at the firms in question, aggressive expansion into previously-unrelated markets, etc. 

Comment by Tomás B. (Bjartur Tómas) on AGI is easier than robotaxis · 2023-08-14T16:18:24.162Z · LW · GW

One argument I've had for self-driving being hard is: humans drive many millions of miles before they get in fatal accidents. In this long tail, would it be that surprising if there were AGI complete problems within it? My understanding is Waymo and Cruise both use teleoperation in these cases. And one could imagine automating this, a God advising the ant in your analogy. But still, at that point you're just doing AGI research. 

Comment by Tomás B. (Bjartur Tómas) on What Does LessWrong/EA Think of Human Intelligence Augmentation as of mid-2023? · 2023-07-08T16:35:04.340Z · LW · GW

I think somatic gene therapy, while technically possible in principal, is extremely unpromising for intelligence augmentation. Creating a super-genius is almost trivial with germ-line engineering. Provided we know enough causal variants, one needs to only make a low-hundreds number of edits to one cell to make someone smarter than any human that has ever lived. With somatic gene therapy you would almost certainly have to alter billions of cells to get anywhere. 

Networking humans is interesting but we have nowhere close to the bandwidth needed now. As a rough guess lets suppose we need similar bandwidth to the corpus callosum, neuralink is ~5 OOMs off.

I suspect human intelligence enhancement will not progress much in the next 5 years, not counting human/ML hybrid systems.

Comment by Tomás B. (Bjartur Tómas) on [Linkpost] Introducing Superalignment · 2023-07-06T20:04:57.229Z · LW · GW

Is this your position: there is no acceptable reason to deliberately optimize for s-risky things like sadism. And doing so to red-team s-risk detection is obviously madness. But possibly red-teaming conventional misalignment which would simply kill everyone if the absolute worst happened and is the default anyway maybe makes some sense?

Comment by Tomás B. (Bjartur Tómas) on [Linkpost] Introducing Superalignment · 2023-07-06T14:30:43.368Z · LW · GW

I do sometimes worry that humanity is dumb enough to create “gain of function for s-risks” and call it alignment research.

Comment by Bjartur Tómas on [deleted post] 2023-05-29T20:29:47.370Z

It was a joke about using the classic, disastrous populist affinity for price controls as a means of ensuing the shortage persists indefinitely. 

Comment by Tomás B. (Bjartur Tómas) on Gemini will bring the next big timeline update · 2023-05-29T19:36:55.246Z · LW · GW

latent capacity overhang

Comment by Bjartur Tómas on [deleted post] 2023-05-29T19:26:34.121Z

These poor people getting price gouged by Nvidia. What we really need is a price ceiling to stop Nvidia and AMD's greed. LessWrong should push hard on getting  such a price ceiling passed.

Comment by Tomás B. (Bjartur Tómas) on Malthusian Competition (not as bad as it seems) · 2023-05-25T15:44:48.153Z · LW · GW
Comment by Tomás B. (Bjartur Tómas) on I bet $500 on AI winning the IMO gold medal by 2026 · 2023-05-13T16:13:18.640Z · LW · GW

I mean like the type of perception one needs to empty a random dishwasher, make a cup of coffee with a random coffee machine type of stuff, clean a room. Hunt and skin a rabbit. 

Comment by Tomás B. (Bjartur Tómas) on I bet $500 on AI winning the IMO gold medal by 2026 · 2023-05-12T15:30:02.690Z · LW · GW

I don't think reading/writing is very easy for humans - compared to perception and embodied tasks. My Morvec's paradox intuition here is maths is of a similar order of difficulty to what we have been very successfully automating in the last couple years, so I expect it will happen soon.

Comment by Tomás B. (Bjartur Tómas) on I bet $500 on AI winning the IMO gold medal by 2026 · 2023-05-11T20:26:38.181Z · LW · GW

A lot of my confidence this will happen is this and a generalized Morvec's paradox-style "hard things are easy, easy things are hard" intuition. 

Comment by Tomás B. (Bjartur Tómas) on I bet $500 on AI winning the IMO gold medal by 2026 · 2023-05-11T20:23:37.628Z · LW · GW

Glad someone took this bet! I still think I think you'll win, but I myself backed out of a similar bet. 

Comment by Tomás B. (Bjartur Tómas) on How do AI timelines affect how you live your life? · 2023-05-07T23:55:10.606Z · LW · GW

This is a reminder that time has passed and you should consider actually trying.

Comment by Tomás B. (Bjartur Tómas) on $250 prize for checking Jake Cannell's Brain Efficiency · 2023-04-28T15:55:21.525Z · LW · GW

The whole "compute greater than humanity" thing does not seem like a useful metric. It's just completely not necessary to exceed total human compute to dis-empower humans.  We parallelize extremely poorly. And given how recent human civilization at this scale is and how adversarial humans are towards each other,  it would be surprising if we used our collective compute in even a remotely efficient way. Not to mention the bandwidth limitations.  

The summed compute of conquistador brains was much less than those they dis-empowered. The summed compute of slaughterhouse worker brains is vastly less than that of the chickens they slaughter in a single month! 

I don't think this point deserves any special salience at all. 

Comment by Tomás B. (Bjartur Tómas) on My Assessment of the Chinese AI Safety Community · 2023-04-25T16:40:52.701Z · LW · GW

Pay Terry Tao his 10 million dollars!

Comment by Tomás B. (Bjartur Tómas) on My Assessment of the Chinese AI Safety Community · 2023-04-25T16:38:56.282Z · LW · GW

Is it "insanely cringe" for different reasons than it is "insanely cringe" for English audiences? I suspect most Americans, if exposed to it, would describe it as cringe. There is much about it that is cringe, and I say this with some love. 

Comment by Tomás B. (Bjartur Tómas) on Contra Yudkowsky on AI Doom · 2023-04-24T14:40:06.566Z · LW · GW

A very naive question for Jacob. A few years ago the fact that bird brains are about 10x more computationally dense than human brains was mentioned on SlateStarCodex and by Diana Fleischman. This is something I would not expect to be true if there were not some significant "room at the bottom." 

Is this false? Does this not imply what I think it should? Am I just wrong in thinking this is of any relevance? 

https://slatestarcodex.com/2019/03/25/neurons-and-intelligence-a-birdbrained-perspective/

I don't understand the physics, so this is just me noticing I am confused. And not an attempt to debunk or anything. 

Comment by Tomás B. (Bjartur Tómas) on Updating my AI timelines · 2023-04-11T16:03:40.177Z · LW · GW

Yeah, unfortunately I think it will still get crazier. 

Comment by Tomás B. (Bjartur Tómas) on Agentized LLMs will change the alignment landscape · 2023-04-10T16:01:58.942Z · LW · GW

Idk, I feel about this stuff like I felt about GPT-J. What scares me is not how well it works, but that it kinda/sorta works a bit. It's a bunch of garbage python code wrapped around an API, and it kinda works. I expect people will push on this stuff hard, and am worried that DeepMind, OpenAI, and Google will be doing so in a much more principled way than the wild-west LLM enthusiast crowed. 

I think it was wrong for people to take comfort in the meme that  "GPT-N is not an agent" and this will become very clear to everyone in the next 18 months. 

Comment by Tomás B. (Bjartur Tómas) on When will computer programming become an unskilled job (if ever)? · 2023-03-18T16:45:53.292Z · LW · GW

Humans are much better drivers than they are programmers

Comment by Bjartur Tómas on [deleted post] 2023-03-17T05:37:56.595Z

Any comments now that it’s out?

Comment by Tomás B. (Bjartur Tómas) on Conceding a short timelines bet early · 2023-03-16T22:46:55.354Z · LW · GW

My emotional state right now: https://twitter.com/emojimashupbot/status/1409934745895583750?s=46

Comment by Tomás B. (Bjartur Tómas) on A concrete bet offer to those with short AGI timelines · 2023-03-16T15:26:47.090Z · LW · GW

So MMLU is down:

Presumably MATH will be next - is Minerva still SOTA?