LessWrong 2.0 Reader

I think it seems like a fine possibility in principle, actually; sorry to have given the wrong impression! It's not my central hope, since strategy-stealing seems like it should make many human-augmentations "available" to AI systems as well. This is notably not true for things involving, e.g. BCIs or superbabies.

College students are wasting their time getting an education from evil institutions. Where did you go to college? Are you just defending horrendous institutions just to make your 'education' look better than it is? You are gaslighting me into believing that I deserve violence because you are an evil person.

That article is sloppily written enough to say "Early testers report that the AI [i.e. o3 and/or o4-mini] can generate original research ideas in fields like nuclear fusion, drug discovery, and materials science; tasks usually reserved for PhD-level experts" linking, as a citation, to OpenAI's January release announcement of o3-mini.

Thanks Tenoke, that's useful feedback. The inline links have become less obvious since we made the body text brighter, so we'll need to look at ways of making them stand out more.

Will Brown: it's simple, really. GPT-4.1 is o3 without reasoning ... o1 is 4o with reasoning ... and o4 is GPT-4.5 with reasoning.

Price and knowledge cutoff for o3 strongly suggest it's indeed GPT-4.1 with reasoning. And so again we don't get to see the touted scaling of reasoning models, since the base model got upgraded instead of remaining unchanged. (I'm getting the impression that GPT-4.5 with reasoning is going to be called "GPT-5" rather than "o4", similarly to how Gemini 2.5 Pro is plausibly Gemini 2.0 Pro with reasoning.)

In any case, the fact that o3 is not GPT-4.5 with reasoning means that there is still no word on what GPT-4.5 with reasoning is capable of. For Anthropic, Sonnet 3.7 with reasoning is analogous to o1 (it's built on the base model of the older Sonnet 3.5, similarly to how o1 is built on the base model of GPT-4o). Internally, they probably already have a reasoning model for some larger Opus model (analogous to GPT-4.5) and for a newer Sonnet (analogous to GPT-4.1) with a newer base model different from that of Sonnet 3.5.

This also makes it less plausible that Gemini 2.5 Pro is based on a GPT-4.5 scale model (even though TPUs might've been able to make its price/speed possible even if it was), so there might be a Gemini 2.0 Ultra internally after all, at least as a base model. One of the new algorithmic secrets disclosed in Gemma 3 was that pretraining knowledge distillation works even when the teacher model is much larger (rather than modestly larger) than the student model, it just needs to be trained for enough tokens for this to become an advantage rather than a disadvantage (Figure 8), something that for example Llama 3.2 from Sep 2024 still wasn't taking advantage of. This makes it useful to train the largest possible compute optimal base model regardless of whether its better quality justifies its inference cost, merely to make the smaller overtrained base models better by pretraining them from the large model logits with knowledge distillation instead of from raw tokens.

Your grievance with your former employer seems to me to have little relevance to how would-be college students should plan to spend their time, and even if it had, you haven't shared enough detail for people to judge your report as accurate (assuming this is in fact the case).

That person appears to be quite a fan of the map :)

Extrapolating current trends provides weak evidence that the AGI will end up being too expensive to use properly, since even the o3 and o4-mini models are rumored to become accessible at a price which is already comparable with the cost of hiring a human expert, and the rise to AGI could require a severe increase of compute-related cost. 

1. ^{^}

   Unfortunately, the quality of officially-same-level experts varies from country to country. For instance, the DMCS of SPBU provides a course on Lie theory for undergraduate students, while in Stanford Lie theory is a graduate-level course. 

2. ^{^}

   Here I assume that each model in the series o1-o3-o4-o5-o6 is the same number of times more capable than the previous one.  If subsequent training of more capable models ends up being slowed down by compute deficiency or even World War III, then this will obviously impact both the METR doubling law and the times when costly models appear, but not the order in which AI becomes too expensive to use and capable of replacing workers.

3. ^{^}

   Exponential increase in time spent between the models' appearance does not ensure that o5 and o6 are released later than in the other scenario [LW · GW].

No, you're right! It is just a policy/ AI Safety advocacy argument, but one that does change minds and shape decisions. I guess it's not as visible as it should be. Still, glad you brought this up!

I think you're asking the VLLM to do too much in a single call. 

I was trying to get the VLLM to extract GD&T data from the NIST standardized renders of ASME Y14 tolerancing standards (eg. screenshotting 1 page at a time from https://www.nist.gov/system/files/documents/noindex/2022/04/06/nist_ftc_06_asme1_rd.pdf ). 

Asking "List all GD&T and their matching element IDs" would totally fail, but would start to be reasonably close when I only asked for 5-8 specific element ids at a time, and asking for the data of a single element id at a time with few shot examples would be ~90% accurate.

While right now you have to build the scaffold for them to perform well, eventually an agent could realize it needs to build it's own scaffold or the model would be able to effectively use all parts of the context window.

I'd be curious how well the models do if you wrote out a 7 step list, and then asked about each step in isolation, and then asked a model to summarize the results of 7 separate calls.