LessWrong 2.0 Reader

This is pretty much the same thing, except breaking out the “economic engine” into two elements of “world needs it” and “you can get paid for it.”

There are things that are economic engines of things that world doesn't quite need (getting people addicted, rent seeking, threats of violence).

Here's what I'm currently using and how much I am paying:

• Superwhisper (or other new Speech-to-Text that leverage LLMs for rewriting) apps. Under $8.49 per month.
• Cursor Pro ($20-30/month). Switch to API credits when the slow responses take too long.
• Claude.ai Pro ($20/month). You could consider getting two accounts or a Team account to worry less about hitting the token limit.
• Chatgpt.com Pro account ($20/month). Again, can get a second account to have more o1-preview responses from the chat.
• Aider (~$10/month max in API credits if used with Cursor Pro).
• Google Colab Pro subscription ($9.99/month). You could get the Pro+ plan for $49.99/month.
• Google One 2TB AI Premium plan ($20/month). This comes with Gemini chat and other AI features. I also sign up to get the latest features earlier, like Notebook LM and Illuminate.
• v0 chat ($20/month). Used for creating Next.js websites quickly.
• jointakeoff.com ($22.99/month) for courses on using AI for development.
• I still have GitHub Copilot (along with Cursor's Copilot++) because I bought a long-term subscription.
• Grammarly ($12/month).
• Reader by ElevenLabs (Free, for now). Best quality TTS app out there right now.

Other things I'm considering paying for:

• Perplexity AI ($20/month).
• Other AI-focused courses that help me best use AI for productivity (web dev or coding in general).
• Suno AI ($8/month). I might want to make music with it.

Apps others may be willing to pay for:

• Warp, an LLM-enabled terminal ($20/month). I don't use the free version enough to upgrade to the paid version.

There are ways to optimize how much I'm paying to save a bit of cash for sure. But I'm currently paying roughly $168/month.

That said, I am also utilizing research credits from Anthropic, which could range from $500 to $2000 depending on the month. In addition, I'm working on an "alignment research assistant" which will leverage LLMs, agents, API calls to various websites, and more. If successful, I could see this project absorbing hundreds of thousands in inference costs.

Note: I am a technical alignment researcher who also works on trying to augment alignment researchers and eventually automate more and more of alignment research so I'm biasing myself to overspend on products in order to make sure I'm aware of the bleeding-edge setup.

To me, the heart of the matter is being trapped in local minima which involve negative sum competitions. Pyrrhic victory is sometimes involved, but sometimes not. An example of not a mutual loss would be if Actor A does come out ahead of where they started, but Actor B lost more than Actor A gained.

A rat race which devours slack, as Dagon mentions is another example. Although that doesn't seem as central to Moloch to me as Negative Sum competition.

Also, there's a related psychological aspect to this. When people get scared and insecure and hostile in their trades and competitions, they have a tendency to stop even searching for win-win solutions. Liv Boeree talks about Win-Win being the opposite of Moloch. https://www.youtube.com/channel/UC09fp6hZ2RHiUYwY8hNCirA

Maybe others are using it in secret but don't want to admit it for some reason? I can't find any mention of Anthropic having filed a patent on the idea, but maybe other companies would feel too much like it looked like they were second-rate imitators if they said they were copying Anthropic's idea?

Just speculating, I don't know. Sure seems like a useful idea to copy.

I think that's a great idea. I don't think the duplication of polls would be a problem, since it would be easy to filter aggregate results for user-created vs official polls.

A few questions:

• The literature review is very strange to me. Where is the section on certified robustness against epsilon-ball adversarial examples? The techniques used in that literature (e.g. interval propagation) are nearly identical to what you discuss here.
• Relatedly, what's the source of hope for these kinds of methods outperforming adversarial training? My sense from the certified defenses literature is that the estimates they produce are very weak, because of the problems with failing to model all the information in activations. (Note I'm not sure how weak the estimates actually are, since they usually report fraction of inputs which could be certified robust, rather than an estimate of the probability that a sampled input will cause a misclassification, which would be more analogous to your setting.)
• If your catastrophe detector involves a weak model running many many inferences, then it seems like the total number of layers is vastly larger than the number of layers in M, which seems like it will exacerbate the problems above by a lot. Any ideas for dealing with this? 
• What's your proposal for the distribution $P_0$ for Method 2 (independent linear features)?

This suggests that we must model the entire distribution of activations simultaneously, instead of modeling each individual layer.

• Why think this is a cost you can pay? Even if we ignore the existence of C and just focus on M, and we just require modeling the correlations between any pair of layers (which of course can be broken by higher-order correlations), that is still quadratic in the number of parameters of M and so has a cost similar to training M in the first place. In practice I would assume it is a much higher cost (not least because C is so much larger than M).

I partly disagree; steganography is only useful when it's possible for the outside / receiving system to detect and interpret the hidden messages, so if the messages are of a type that outside systems would identify, they can and should be detectable by the gating system as well. 

That said, I'd be very interested in looking at formal guarantees that the outputs are minimally complex in some computationally tractable sense, or something similar - it definitely seems like something that @davidad [LW · GW] would want to consider.

I really like that idea, and the clarity it provides, and have renamed the post to reflect it! (Sorryr this was so slow- I'm travelling.)

That seems fair!

@Ericf [LW · GW] you are right, I didn't calculate it like this. I´m going to adjust the formula to see if it changes the overall outcome of the essay or not.