LessWrong 2.0 Reader

I think one underused trick for training LLMs is to explicitly "edit" them. That is, suppose they generate some text X in response to prompt Y, and it has some error or is missing something. In that case you can create a text X' that fixes this problem, and do a gradient update to increase log P(X'|Y)/P(X|Y).

For example, if we generate virtual comments in the style of certain LW users, one could either let those users browse the virtual comments that have been created in their style and correct them, or one could let the people who receive the virtual comments edit them to remove misunderstanding or similar.

Read your comment, donated 10$.

I was going to email but I assume others will want to know also so I'll just ask here. What is the best way to donate an amount big enough that it's stupid to pay a Stripe fee, e.g. $10k? Do you accept donations of appreciated assets like stock or cryptocurrency?

Yes, absolutely – I've always been a rationalist (or at least attempted to be). I read a lot of the new atheists' work early on, and have been involved in various rationalist-type communities on the internets. I really ought to be more involved in the community and seek to make the School of Thought more involved too. Thanks for making this post!

I've gotten enormous value out of LW and its derived communities during my life, at least some of which is attributable to the LW2.0 revival and its effects on those communities. More recently, since moving to the Bay, I've been very excited by a lot of the in-person events that Lighthaven has helped facilitate. Also, LessWrong is doing so many things right as a website and source-of-content that no one else does (karma-gated RSS feeds! separate upvote and agree-vote! built-in LaTeX support!) and even if I had no connection to the other parts of its mission I'd want to support the existence of excellently-done products. (Of course there's also the altruistic case for impact on how-well-the-future-goes, which I find compelling on its own merits.) Have donated $5k for now, but I might increase that when thinking more seriously about end-of-year donations.

(Conflict of interest notice: two of my housemates work at Lightcone Infrastructure and I would be personally sad and slightly logistically inconvenienced if they lost their jobs. I don't think this is a big contributor to my donation.)

Sent a tenner, keep up the excellent work!

Since I couldn't find it quickly on the web, GPT o1 estimated that the labour hours per ton kilometer of trucking is about 100 times as much as ships, and rail is just about the same as ships (I would have thought rail would have been at least a few times higher than ships). So based on the historic US and current Europe, maybe water transport in the US would increase an order of magnitude if the Jones act were repealed. As Zvi points out, even though the US ship manufacturing jobs would be lost, there probably would be an increase overall shipping employment because of repairing ships and staffing ships. So let's say staffing the ships is 3 times as much as the current employment of manufacturing and staffing very few ships. I suspect that most of the lost inland transportation due to the shift from shipping would be rail, but even if 10% of it were trucking, that would mean the loss of jobs in trucking would be 10 times as much as the staffing of the additional ships, and 30 times as much as the employment constructing and staffing the current ships.^[1] So if you had to compensate 30 times as many people, it would be much more difficult. Now it is true that the majority of total cost of rail and shipping is energy (it's about even between energy and labour for trucking), so the large overall economic savings of moving to shipping should be sufficient to compensate all those truckers, but it's just not nearly the slamdunk that it appeared to be when only looking at marine employment.

1. ^{^}

   Technically not all the freight that would be moved by ships is currently moved by truck/rail/pipeline because a smaller amount gets transported because of the higher cost. But using Zvi's example of $0.63 per barrel increase, since it is 42 gallons per barrel, that's 1.5 cents/gallon, or ~0.5% of the total cost, which wouldn't change quantity demanded very much, well within other uncertainties of this analysis.

As someone who isn't really in a position to donate much at all, and who feels rather silly about the small amount I could possibly give, and what a tiny drop that is compared the bucket this post is sketching...

I uh ... sat down and did some simple math. If everyone who ever votes (>12M) donates $10 then you'd have >$120 million covered. If we follow bullshit statistics of internet activity, where it's said 99% of all content is generated by 1% of all people, then this heuristic would get us $1.2M from people paying this one time "subscription" fee. Now I also feel, based on intuition and ass-numbers, that LW folk have a better ratio than that, so let's multiply by 2 and then we could get a $2.4 million subscriber fee together from small donations.

Now on the pure power of typical mind ... I personally like people knowing when I do a nice thing - even a stupidly small thing.

So I'm commenting about it.

I find this embarrassing, and I'm working over the embarrassment to make it easier for other's to farm this nutrient too and just normalize it in case that helps with getting critical mass of small donations of the $10 variety.

Basically my point to readers is: 'Everyone' paying a one-time $10 subscription fee would solve the problem.

The trick is mostly to help each other generate the activation energy to do this thing. If it helps to post, high five, or wave about it, please do! Visibility of small donations may help activation energy and get critical mass! Group action is awesome. Using your natural reward centers about it is great! <3 Hi :D Wanna join? ^_

Well argued. I’m in. I’ve received ample surplus value over the years from LW. Less Online was a blast this year. Thank you for all the work you and your team do!

Julien Chaumond on X: "Qwen QwQ switching to Chinese when it needs to _really think_ about something, then switching back to English, is pretty cool @Alibaba_Qwen https://t.co/jpTIHWyXim" / X

This is extremely weird - no one actually writes like this in Chinese. "等一下" is far more common than "等待一下", which seems to mash the direct translation of the "wait" [等待] in "wait a moment" - 等待 is actually closer to "to wait". The use of “所以” instead of “因此” and other tics may also indicate the use of machine-translated COT from English during training. 

The funniest answer would be "COT as seen in English GPT4-o1 logs are correlated with generating quality COT. Chinese text is also correlated with highly rated COT. Therefore, using the grammar and structure of English GPT4 COT but with Chinese tokens elicits the best COT".