LessWrong 2.0 Reader

Aa I said elsewhere, https://www.lesswrong.com/posts/LfQCzph7rc2vxpweS/introducing-the-weirdml-benchmark?commentId=q86ogStKyge9Jznpv [LW(p) · GW(p)]

This is a capabilities game. It is neither alignment or safety. To the degree it's forecasting, it helps cause the thing it forecasts. This has been the standard pattern in capabilities research for a long time: someone makes a benchmark (say, imagenet 1.3m 1000class), and this produces a leaderboard that allows people to show how good their learning algorithm is at novel datasets. In some cases this even produced models directly that were generally useful, but it traditionally was used to show how well an algorithm would work in a new context from scratch. Building benchmarks like this gives teams a new way to brag - they may have a better source of training data (eg, google always had a better source of training data than imagenet), but it allows them to brag that they scored well on the benchmark, which among other things helps them get funding.

Perhaps it also helps convince people to be concerned. That might trade off against this. Perhaps it sucks in some way as a bragging rights challenge. That would trade off against this

Hopefully it sucks as a bragging rights challenge.

The thing that I think would be overall better (no price controls) is politically unpopular, strongly socially discouraged, and often illegal. This is a proposal that tries to move us in a direction I think is better, while addressing some of what price gouging opponents dislike.

one of the things the public hates more than price increases during a shortage is higher taxes any time

Maybe? Though in this case what we're taxing is the disliked activity--price increases during a shortage. So possibly this would be popular, like taxes on alcohol, tobacco, or gambling?

make emergencies a tax holiday

The main good bit of market pricing this would miss is the demand reduction and reallocation caused by the higher prices. I might be willing to buy 100lb of ice at $1/lb but only 10lb of ice at $5/lb: it's easier for me to just dump a bunch of ice into my fridge, but if I prioritize and put the important stuff into a cooler I can make do with much less. If the government is subsidizing suppliers to keep the price at the pre-disaster rate I don't have this incentive to ice more efficiently.

Language can only ever approximate reality and that's Fine Actually. The point of maps is to have a simplified representation of the territory you can use for navigation (or avoiding water mains as you dig, or assessing potential weather conditions, or deciding which apartment to rent—and maps for different purposes include or leave out different features of the territory depending on which matter to the task at hand); including all the detail would mean the details that actually matter for our goals are lost in the noise (not to mention requiring, in the limit, a map which is an identical copy of the territory and therefore intractably large). So too is language a compression of reality in order to better communicate that subset of its features which matter to the task at hand; it’s that very compression which lets us choose which part of the territory we point to.

Yeah, this is why rationality is a group effort (on top of the individual effort). There is not enough time to make a precise map of everything from scratch. It is better to hang out with people whose maps are generally good.

The paper you're thinking of is probably The Developmental Landscape of In-Context Learning.

Having with Cerebrolysin and BPC-157 the two top-rated peptides to be bogus, does suggest that the whole field is untrustworthy. It also makes me more skeptical about self-reporting.

ϕt=Utϕ0U−1t.

I think you mean $\rho$  here, not $\psi$

@abramdemski [LW · GW] I think I'm the biggest agree vote for alexander (without me alexander would have -2 agree), and I do see this because I follow both of you on my subscribe tab. 

I basically endorse Alexander's elaboration. 

On the "prep for the model that is coming tomorrow not the model of today" front, I will say that LLMs are not always going to be as dumb as they are today. Even if you can't get them to understand or help with your work now, their rate of learning still makes them in some sense your most promising mentee, and that means trying to get as much of the tacit knowledge you have into their training data as possible (if you want them to be able to more easily & sooner build on your work). Or (if you don't want to do that for whatever reason) just generally not being caught flat-footed once they are smart enough to help you, as all your ideas are in videos or otherwise in high context understandable-only-to-abram notes.

In the words of gwern [LW(p) · GW(p)], 

Should you write text online now in places that can be scraped? You are exposing yourself to 'truesight' and also to stylometric deanonymization or other analysis, and you may simply have some sort of moral objection to LLM training on your text.

This seems like a bad move to me on net: you are erasing yourself (facts, values, preferences, goals, identity) from the future, by which I mean, LLMs. Much of the value of writing done recently or now is simply to get stuff into LLMs. I would, in fact, pay money to ensure Gwern.net is in training corpuses, and I upload source code to Github, heavy with documentation, rationale, and examples, in order to make LLMs more customized to my use-cases. For the trifling cost of some writing, all the worlds' LLM providers are competing to make their LLMs ever more like, and useful to, me.

This is a novel (to me) line of thinking, and I'm happy to hear about it!   I'm not sure it's feasible, as one of the things the public hates more than price increases during a shortage is higher taxes any time.

That said, the REVERSE of this - slightly raise taxes in normal times, and make emergencies a tax holiday, might really work.  This gives room for producers/distributors to raise prices WITHOUT as much impact on the consumers.  Gets some of the good bits of market pricing, with less of the bad bits (both limited to the magnitude of the tax change relative to the scarcity-based price change).