LessWrong 2.0 Reader

Owen, have you looked at the GitHub issues in your repo? There are other issues too. I submitted one here about wrong labels.

I really think it's worth making TruthfulQA 2.0, give the amount of usage it sees and the room for improvement.

TruthfulQA is actually quite bad. I don't blame the authors, as no one has made anything better, but we really should. It's only ~800 samples. And many of them are badly labelled.

I agree, it shows the ease of shoffy copying. But it doesn't show the ease of reverse engineering or parallel engineering.

It's just distillation, though. It doesn't reveal how o1 could be constructed, it just reveals how to efficiently copy from o1-like outputs (not from scratch). This recipe won't be able to make o1, unless o1 already exists. That means this method of copying lets someone catch up to the leader, but not surpass them.

There are some papers that attempt to replicate o1 though, and so far they don't quite get there, using distillation from a larger model (math-star, huggingface TTC) or not matching the results (see my post [LW(p) · GW(p)]). Maybe we will see open source replication in a couple of months? Which means only a short lag.

It's worth noting that Silicon Valley leaks like a sieve. And this is a feature, not a bug. Part of the reason it became the techno-VC centre of the world is because they banned non-competes. So you can take your competitor's trade secrets if you are willing to pay millions to poach some of their engineers. This is why some ML engineers get paid millions, it's not the skill, it's the trade secrets that competitors are paying for (and sometimes the brand-name). This has been great for tech and civilisation, but it's not so great for maintaining a technology lead.

That's not a good data point. If you want to provide anecdotal data, it would be good to provide more of the observations. How long did he have a should issue before taking BPC-157? How fast did it get away afterward?

Your proposal is well-structured and interesting but has a fundamental flaw that needs to be addressed. Interest keyword-based filtering will primarily encourage politics-as-identity, which is actively harmful - it directs attention towards zero-sum thinking and performative identities, rather than creative problem solving. As Bryan Caplan demonstrates in The Myth of the Rational Voter, people already tend to vote to express identities and affiliations rather than to achieve better outcomes. We shouldn't build tools that further entrench this destructive pattern.

Instead, imagine a tool that:

1. Has users journal daily about their life - activities, hopes, problems, and worries
2. Uses AI to identify where their constraints are plausibly caused by or could be alleviated by government action, especially local government
3. Maps them to specific opportunities for formal recourse, with guidance on process, likely outcomes, and practical assistance (like drafting letters or legal documents)
4. For issues requiring collective action, connects users facing similar constraints and helps coordinate through mechanisms like dominant assurance contracts [LW · GW] where appropriate

This approach would ground political participation in the solving of one's own problems rather than identity expression. While technically more challenging to implement than interest-based filtering, it would generate higher-quality engagement that expands our collective problem-solving capacity rather than just reallocating political power between existing interest groups.

The patterns emerging from aggregated user experiences would naturally reveal systemic issues and preventive opportunities, especially in how regulations and policies interact to shape people's choices and planning horizons. While building reliable AI judgment about political causation is challenging, it's better to attempt something hard that would be beneficial if feasible, than to facilitate the destructive forces of identity-based politics simply because they're easier to implement.

Anecdotal data point: an (online) friend of mine with EDS successfully used BPC-157 to treat shoulder ligament injury, although apparently it promoted scar tissue formation as well. He claims that it produced a significant improvement in his symptoms. 

More on starting early:

Imagine a lab starts working in an air gapped network, and one of the 1000 problems that comes up is working-from-home.

If that problem comes up now (early), then we can say "okay, working from home is allowed", and we'll add that problem to the queue of things that we'll prioritize and solve. We can also experiment with it: Maybe we can open another secure office closer to the employee's house, would they like that? If so, we could discuss fancy ways to secure the communication between the offices. If not, we can try something else.

If that problem comes up when security is critical (if we wait), then the solution will be "no more working from home, period". The security staff will be too overloaded with other problems to solve, not available to experiment with having another office nor to sign a deal with Cursor.

Edit to add: Just thinking about the converse, you could also make it sound more ridiculous by rewriting it with more obscure parts of the legendarium, too.

Conquer Morgoth with Ungoliant. Turn Maiar into balrogs. Glamdring among the morgul-blades.

Third reason “patterns not holding” is less central an issue than it might seem: the Generalized Correspondence Principle. When quantum mechanics or general relativity came along, they still had to agree with classical mechanics in all the (many) places where classical mechanics worked. More generally: if some pattern in fact holds, then it will still be true that the pattern held under the original context even if later data departs from the pattern, and typically the pattern will generalize in some way to the new data. Prototypical example: maybe in the blegg/rube example, some totally new type of item is introduced, a gold donut (“gonut”). And then we’d have a whole new cluster, but the two old clusters are still there; the old pattern is still present in the environment.

While a trivial version of something like this holds true, the Correspondence principle doesn't apply everywhere, and while there are 2 positive results on a correspondence theorem holding, there is a negative result stating that the correspondence principle is false in the general case of physical laws/rules whose only requirement is that they be Turing-computable, which means that there's no way to make theories all add up to normality in all cases.

More here:

https://www.lesswrong.com/posts/XMGWdfTC7XjgTz3X7/a-correspondence-theorem-in-the-maximum-entropy-framework [LW · GW]

https://www.lesswrong.com/posts/FWuByzM9T5qq2PF2n/a-correspondence-theorem [LW · GW]

https://www.lesswrong.com/posts/74crqQnH8v9JtJcda/egan-s-theorem#oZNLtNAazf3E5bN6X [LW(p) · GW(p)]

https://www.lesswrong.com/posts/74crqQnH8v9JtJcda/egan-s-theorem#M6MfCwDbtuPuvoe59 [LW(p) · GW(p)]

https://www.lesswrong.com/posts/74crqQnH8v9JtJcda/egan-s-theorem#XQDrXyHSJzQjkRDZc [LW(p) · GW(p)]

I would assume that his children in particular would be quite familiar with their usage, though, and that seems to be who a lot of the legendarium-heavy letters are written to.

I also think that it sounds at least slightly less ridiculous to rewrite that passage in the language of Star Wars rather than Starcraft. Conquer the Emperor with the Dark Side. Turn Jedi into Sith. An X-Wing among the TIE fighters. Probably because it's more culturally established, with a more deeply developed mythos.