LessWrong 2.0 Reader

Haskell is a beautiful language, but in my admittedly limited experience it's been quite hard to reason about memory usage in deployed software (which is important because programs run on physical hardware. No matter how beautiful your abstract machine, you will run into issues where the assumptions that abstraction makes don't match reality).

That's not to say more robust programming languages aren't possible. IMO rust is quite nice, and easily interoperable with a lot of existing code, which is probably a major factor in why it's seeing much higher adoption.

But to echo and build off what @ustice said earlier [LW(p) · GW(p)]:

The hard part of programming isn't writing a program that transforms simple inputs with fully known properties into simple outputs that are meet some known requirement. The hard parts are finding or creating a mostly-non-leaky abstraction that maps well onto your inputs, and determining what precise machine-interpretable rules produce outputs that look like the ones you want.

Most bugs I've seen come at the boundaries of the system, where it turns out that one of your assumptions about your inputs was wrong, or that one of your assumptions about how your outputs will be used was wrong.

I almost never see bugs like this

• My sort(list, comparison_fn) function fails to correctly sort the list"
• My graph traversal algorithm skips nodes it should have hit
• My pick_winning_poker_hand() function doesn't always recognize straights

Instead, I usually see stuff like

• My program assumes that when the server receives an order_received webhook, and then hits the server to fetch the order details from the vendor's API for the order identified in the webhook payload, the vendor's API will return the order details and not a 404 not found"
• My server returns nothing at all when fetching the user's bill for this month, because while the logic is correct (determine the amount due for each order and sum), this particular user had 350,000 individual orders this month so the endpoint takes >30 seconds, times out, and returns nothing.
• The program takes satellite images along with Metadata that includes the exact timesamp, which satellite took the picture, and how the satellite was angles. It identifies locations which match a specific feature, and spits out a latitude, longitude, label, and confidence score. However, when viewing the locations on a map, they appear to be 100-700 meters off, but only for points within the borders of China (because the programmer didn't know about GCJ-02)

Programming languages that help you write code that is "correct" mostly help prevent the first type of bug, not the second.

I feel *so* pedantic making this comment — please forgive me — but also:

CeraVe may have degraded in quality when they were purchased by L’Oréal and changed the source of the fatty alcohols in their formulation. Fatty alcohols that have been sourced from coconut are more likely to cause skin irritation than those that have been sourced from palm. Plus, retinoids can actually push these fatty alcohols deeper into the pores for the ultimate backfire effect. My source is u/WearingCoats on Reddit, who runs a dermatology practice and does product consulting for drugstore brands. She's another one of my favourite resources; I recommend running a search on her posts if there's something specific you're interested in learning.

In general, you don't want to use a moisturizer that is heavy or occlusive over retinoids. It could make the treatment more effective than intended, which might compromise your moisture barrier and contribute to irritated skin. Cetaphil may work better for you if you're going down that route. I personally own a variety of moisturizers, which I rotate depending on what else I'm using on my face, and the season.

Niacinamide decreases sebum production, which is great for acne, as well as giving your skin a “mattifying” look…. but it can also compound a drying effect. I personally apply a few drops in the morning before my moisturizer, but it's one of those things where if you go too hard, too fast, and throw too many things at your skin at once, you're likely to overwhelm your moisture barrier.

Vitamin C is a can of worms. There are a bunch of different compounds that are all called "vitamin C," including L-ascorbic acid (which can come in a powder form, or suspended in oils like squalene or silicone), magnesium or sodium ascorbyl phosphate, ascorbyl glucoside, 3-0 ethyl ascorbic acid, and ascorbyl palmitate. (I don’t have a fantastic memory; I grabbed these out of Skincare Decoded.) Water-based L-ascorbic acid serum in combination with vitamin E and ferulic acid is the classic, patented formula developed by SkinCeuticals, which has the most scientific data to back up its claims. However, it's also ridiculously expensive and the price keeps going up. I personally use Timeless' clone of this formula and buy a new bottle every couple of months or so, but there's no way to tell if it's truly effective short of getting into research equipment and controlled studies.

Some of the vitamin C products are better than others, and some are pretty much rubbish (hi hi, ascorbyl palmitate). L-ascorbic acid is notoriously unstable in water, which is why SkinCeuticals holds the holy grail with their patented formula to stabilize it. Sodium ascorbyl phosphate actually has some promising evidence as an acne treatment, and I really think that it should have been marketed as such instead of a less effective vitamin C alternative. There are also less-studied antioxidants you might want to check out, like resveratrol and EUK-134. It's important to pay attention to the concentrations that have actually been studied in clinical trials because many cosmetics lines underdose (or overdose!) their products and can't really claim that they have scientific evidence behind them. For example, The Ordinary's resveratrol formulation contains double the amount that has been studied, which might be irritating to some people's skin.

I'm skeptical about hyaluronic acid being the holy grail of humectants, compared to, say, good ol' glycerin, especially given the marketing hype around it. Here's a blog post comparing some of the research surrounding the two. Like @nebuchadnezzar [LW · GW]pointed out, molecular weight matters, and HA comes in a variety of different sizes so you don't quite know what you're getting in terms of penetration and effects. Also, humectants are hydrophilic, meaning, they grab water molecules around them and bind them to themselves, but the downside is that they're not discerning about where they're getting their water from, pulling water from your skin as well as drawing water from the air. If you’re in a dry environment, this can actually work in the opposite direction of what you want. One trick is to apply humectants over wet skin or in the shower, or mist your face with water (when I’m feeling purely indulgent, I use Mario Badescu’s rosewater spray).

@Vanessa Kosoy [LW · GW], it's hard for me to give a straight answer to your question because there are so many factors to consider in your skincare routine, which is why I think a primer on some of the basic science will lead you to being better equipped to pick the right products for you.

That ought to buy you a couple weeks, anyway. ;)

Any pinching concern with those straps?

Or is that sentence meant to indicate that an instance running after training might figure out how to hack the computer running it so it can actually change it's own weights?

I was thinking of a scenario where OpenAI deliberately gives it access to its own weights to see if it can self improve.

I agree that it would be more like to just speed up normal ML research.

Enjoyed this post! Quick question about obtaining the steering vectors:

Do you train them one at a time, possibly adding an additional orthogonality constraint between each train?

Under the current version of the interactive model, its median prediction is just two decades earlier than that from Cotra’s forecast

Just?

Question about the "rules of the game" you present.  Are you allowed to simply look at layer 0 transcoder features for the final 10 tokens - you could probably roughly estimate the input string from these features' top activators.  From you case study, it seems that you effectively look at layer 0 transcoder features for a few of the final tokens through a backwards search, but wonder if you can skip the search and simply look at transcoder features.  Thank you.

By "gag order" do you mean just as a matter of private agreement, or something heavier-handed, with e.g. potential criminal consequences?

I have trouble understanding the absolute silence we seem to be having. There seem to be very few leaks, and all of them are very mild-mannered and are failing to build any consensus narrative that challenges OA's press in the public sphere.

Are people not able to share info over Signal or otherwise tolerate some risk here? It doesn't add up to me if the risk is just some chance of OA trying to then sue you to bankruptcy, especially since I think a lot of us would offer support in that case, and the media wouldn't paint OA in a good light for it.

I am confused. (And I grateful to William for at least saying this much, given the climate!)

Maybe GPT-5 will be extremely good at interpretability, such that it can recursively self improve by rewriting its own weights.

I am by no means an expert on machine learning, but this sentence reads weird to me. 

I mean, it seems possible that a part of a NN develops some self-reinforcing feature which uses the gradient descent (or whatever is used in training) to go into a particular direction and take over the NN, like a human adrift on a raft in the ocean might decide to build a sail to make the raft go into a particular direction. 

Or is that sentence meant to indicate that an instance running after training might figure out how to hack the computer running it so it can actually change it's own weights?

Personally, I think that if GPT-5 is the point of no return, it is more likely that it is because it would be smart enough to actually help advance AI after it is trained. While improving semiconductors seems hard and would require a lot of work in the real world done with human cooperation, finding better NN architectures and training algorithms seems like something well in the realm of the possible, if not exactly plausible.

So if I had to guess how GPT-5 might doom humanity, I would say that in a few million instance-hours it figures out how to train LLMs of its own power for 1/100th of the cost, and this information becomes public. 

The budgets of institutions which might train NN probably follows some power law, so if training cutting edge LLMs becomes a hundred times cheaper, the number of institutions which could build cutting edge LLMs becomes many orders of magnitude higher -- unless the big players go full steam ahead towards a paperclip maximizer, of course. This likely mean that voluntary coordination (if that was ever on the table) becomes impossible. And setting up a worldwide authoritarian system to impose limits would also be both distasteful and difficult. 

I have been contemplating Connor Leahy's Cyborgism [LW · GW] and what it would mean for us to improve human workflows enough that aligning AGI looks less like:

Sisyphus attempting to roll a 20 tonne version of The One Ring To Rule Them All into the caldera of Mordor while blindfolded and occasionally having to bypass vertical slopes made out of impossibility proofs that have been discussed by only 3 total mathematicians ever in the history of our species - all before Sauron destroys the world after waking up from a restless nap of an unknown length.

I think this is what you meant by "make the ultimate <programming language/text editor/window manager/file system/virtual collaborative environment/interface to GPT/...>"

Intuitively, the level I'm picturing is:

A suite of tools that can be booted up from a single icon on the home screen of a computer which then allows anyone who has decent taste in software to create essentially any program they can imagine up to a level of polish that people can't poke holes in even if you give a million reviewers 10 years of free time.

Can something at this level be accomplished?

Well, what does coding look like currently? It seems to look like a bunch of people with dark circles under their eyes reading long strings of characters in something basically the equivalent to an advanced text editor, with a bunch of additional little windows of libraries and graphics and tools.

This is not as domain where human intelligence performs with as much ease as in other domains like spearfishing or bushcraft.

If you want to build Cyborgs, I am pretty sure where you start is by focusing on building software that isn't god-machines, throwing out the old book of tacit knowledge, and starting over with something that makes each step as intuitive as possible. You probably also focus way more on quality over quantity/speed.

So, plaintext instructions on what kind of software you want to build, or a code repository and a plaintext list of modifications? Like, start with an elevator pitch, see the raw/AI generated material, critique in a step-by-step organized fashion where debugging/feature analysis checklists are generated and scored on whether they included everything you would have thought of/stuff that is valid which you didn't think of.

I think the point in this post is valid, though a bit more in the realm of angsty shower thought rather than a ready-to-import heuristic for analysing the gap between competence-in-craft and unleased-power-levels.

There is a bit of a bootstrapping problem with Cyborgism. I don't think you get the One Programming Suite To Rule Them All by plugging in a bunch of different LLMs fine tuned to do one part of the process really well - then packaging the whole thing up and running it on 6 gaming GPUs. That is the level of super-software that seems in reach, and it just seems doomed to be full of really hard-to-perceive holes like a super high-dimensional block of swiss cheese.

Does that even get better if we squeeze the weights of LLMs to get LLeMon juice:

Python script that does useful parts of the cool smart stuff LLMs do without all the black box fuzzy embedding-spaces/vectors + filter the juice for pulp/seeds (flaws in the specific decoded algorithm that could cause errors via accidental or deliberate adversarial example) + sweeten it (make the results make sense/be understandable to humans)

... then plug a bunch of that LLeMonade into a programming suite such that the whole thing works with decently competent human programmer(s) to reliably make stuff that actually just works & alert people ahead of time of the exhaustive set of actual issues/edge cases/gaps in capability of a program?

This problem does seem difficult - and probably the whole endeavor just actually won't work well enough IRL, but it seems worth trying?

Like, what does it look like to throw everything and the kitchen sink at Alignment? It probably looks at least a little like the Apollo program, and if you're doing NASA stuff properly, then you end up making some revolutionary products for everyone else.

I think those products - the random externalities of a healthy Alignment field - look more like tools that work simply and reliably, rather than the giant messy LLM hairballs AI labs keep coughing up and dumping on people.

Maybe all of this helps flesh out and make more useful the flinchy heuristic of "consumer software works terribly -> ... -> AI destroys the future."

Alignment as a field goes out ahead of the giant rolling gooey hairball of spaghetti-code Doom - untangles it and weaves it into a beautiful textile - or we are all dead.