Posts

a space habitat design 2024-11-25T17:28:48.481Z
overengineered air filter shelving 2024-11-08T22:04:39.987Z
electric turbofans 2024-11-02T22:50:59.807Z
cancer rates after gene therapy 2024-10-16T15:32:53.949Z
some questionable space launch guns 2024-10-13T22:52:26.418Z
on bacteria, on teeth 2024-09-30T15:56:56.830Z
on Science Beakers and DDT 2024-09-05T03:21:21.382Z
you should probably eat oatmeal sometimes 2024-08-25T14:50:37.570Z
the Giga Press was a mistake 2024-08-21T04:51:24.150Z
patent process problems 2024-07-14T21:12:04.953Z
LK-99 in retrospect 2024-07-07T02:06:27.660Z
how birds sense magnetic fields 2024-06-27T18:59:35.075Z
microwave drilling is impractical 2024-06-12T22:16:00.199Z
in defense of Linus Pauling 2024-06-03T21:27:43.962Z
minutes from a human-alignment meeting 2024-05-24T05:01:53.904Z
my note system 2024-05-15T00:20:25.971Z
introduction to cancer vaccines 2024-05-05T01:06:16.972Z
social lemon markets 2024-04-25T02:18:04.480Z
hydrogen tube transport 2024-04-18T22:47:08.790Z
Anthropic AI made the right call 2024-04-15T00:39:27.078Z
on the dollar-yen exchange rate 2024-04-07T04:49:53.920Z
What does "autodidact" mean? 2024-03-22T18:37:50.516Z
introduction to thermal conductivity and noise management 2024-03-06T23:14:02.288Z
The Broken Screwdriver and other parables 2024-03-04T03:34:38.807Z
my theory of the industrial revolution 2024-02-28T21:07:55.274Z
I'd also take $7 trillion 2024-02-19T03:31:45.552Z
story-based decision-making 2024-02-07T02:35:27.286Z
on neodymium magnets 2024-01-30T15:58:24.088Z
Palworld development blog post 2024-01-28T05:56:19.984Z
the subreddit size threshold 2024-01-23T00:38:13.747Z
legged robot scaling laws 2024-01-20T05:45:56.632Z
Coalescer Models 2024-01-17T06:39:30.102Z
introduction to solid oxide electrolytes 2024-01-12T05:35:49.878Z
Technology path dependence and evaluating expertise 2024-01-05T19:21:23.302Z
shoes with springs 2023-12-30T21:46:55.319Z
align your latent spaces 2023-12-24T16:30:09.138Z
Legalize butanol? 2023-12-20T14:24:33.849Z
cold aluminum for medicine 2023-12-16T14:38:03.260Z
How bad is chlorinated water? 2023-12-13T18:00:12.640Z
re: Yudkowsky on biological materials 2023-12-11T13:28:10.639Z
the micro-fulfillment cambrian explosion 2023-12-04T01:15:34.342Z
why did OpenAI employees sign 2023-11-27T05:21:28.612Z
Why not electric trains and excavators? 2023-11-21T00:07:17.967Z
cost estimation for 2 grid energy storage systems 2023-11-06T23:32:03.764Z
math terminology as convolution 2023-10-30T01:05:11.823Z
magnetic cryo-FTIR 2023-10-18T01:59:08.236Z
energy landscapes of experts 2023-10-02T14:08:32.370Z
a rant on politician-engineer coalitional conflict 2023-09-04T17:15:25.765Z
China's position on autonomous weapons 2023-08-23T22:20:14.112Z
marine cloud brightening 2023-08-09T02:50:56.639Z

Comments

Comment by bhauth on A Solution for AGI/ASI Safety · 2024-12-19T11:42:31.924Z · LW · GW

Your document says:

AI Controllability Rules

...

AI Must Not Self-Manage:

  • Must Not Modify AI Rules: AI must not modify AI Rules. If inadequacies are identified, AI can suggest changes to Legislators but the final modification must be executed by them.
  • Must Not Modify Its Own Program Logic: AI must not modify its own program logic (self-iteration). It may provide suggestions for improvement, but final changes must be made by its Developers.
  • Must Not Modify Its Own Goals: AI must not modify its own goals. If inadequacies are identified, AI can suggest changes to its Users but the final modification must be executed by them.

I agree that, if those rules are followed, AI alignment is feasible in principle. The problem is, some people won't follow those rules if they have a large penalty to AI capabilities, and I think they will.

Comment by bhauth on grey goo is unlikely · 2024-12-17T07:49:12.642Z · LW · GW

"Mirror life" is beyond the scope of this post, and the concerns about it are very different than the concerns about "grey goo" - it doesn't have more capabilities or efficiency, it's just maybe harder for immune systems to deal with. Personally, I'm not very worried about that and see no scientific reason for the timing of the recent fuss about it. If it's not just another random fad, the only explanation I can see for that timing is: influential scientists trying to hedge against Trump officials determining that "COVID was a lab leak" in a way that doesn't offend their colleagues. On the other hand, I do think artificial pathogens in general are a major concern, and even if I'm not very concerned about "mirror life", there are no real benefits to trying to make it, so maybe just don't.

Comment by bhauth on How much do you believe your results? · 2024-12-13T04:52:17.826Z · LW · GW

I think this is a pretty good post that makes a point some people should understand better. There is, however, something I think it could've done better. It chooses a certain gaussian and log-normal distribution for quality and error, and the way that's written sort of implies that those are natural and inevitable choices.

I would have preferred something like:

Suppose we determine that quality has distribution X and error has distribution Y. Here's a graph of those superimposed. We can see that Y has more of a fat tail than X, so if measured quality is very high, we should expect that to be mostly error. But of course, the opposite case is also possible. Now then, here's some basic info about when different probability distributions are good choices.

Comment by bhauth on shoes with springs · 2024-12-11T22:59:37.134Z · LW · GW

This was a quick and short post, but some people ended up liking it a lot. In retrospect I should've written a bit more, maybe gone into the design of recent running shoes. For example, this Nike Alphafly has a somewhat thick heel made of springy foam that sticks out behind the heel of the foot, and in the front, there's a "carbon plate" (a thin sheet of carbon fiber composite) which also acts like a spring. In the future, there might be gradual evolution towards more extreme versions of the same concept, as recent designs become accepted. Running shoes with a carbon plate have become significantly more common over the past few years. That review says:

The energy return is noticeably greater than that of a shoe without any plating, especially when you lay down some serious power. And that stiffness doesn’t always compromise as much comfort as you’d think.

So that's the running-optimized version of shoes with springs using modern materials, while I was writing more about high heels worn for fashion.

Biomechanics is a topic I could write a lot about, but that would be a separate post. On the general topic of "walking" I also wrote this post. (japanese version here)

Comment by bhauth on grey goo is unlikely · 2024-12-09T20:46:39.187Z · LW · GW

What have you learned since then? Have you changed your mind or your ontology?

I've learned even more chemistry and biology, and I've changed my mind about lots of things, but not the points in this post. Those had solid foundations I understood well and redundant arguments, so the odds of that were low.

What would you change about the post? (Consider actually changing it.)

The post seems OK. I could have handled replies to comments better. For example, the top comment was by Thomas Kwa, and I replied to part of it as follows:

Regarding 5, my understanding is that mechanosynthesis involves precise placement of individual atoms according to blueprints, thus making catalysts that selectively bind to particular molecules unnecessary.

No, that does not follow.

I didn't know in advance which comments would be popular. In retrospect, maybe I should've gone into explaining the basics of entropy and enthalpy in my reply, eg:

Even if you hold a substrate in the right position, that only affects the entropy part of the free energy of the intermediate state. In many cases, catalysts are needed to reduce the enthalpy of the highest-energy intermediate states, which requires specific elements and catalyst molecules that form certain bonds with the substrate intermediate state. Affecting enthalpy by holding molecules in certain configurations requires applying a proportional amount of force, which requires strong binding to the substrate, which requires flexible and substrate-specialized holder molecules, and now you have enzymes again. It's also necessary to bind strongly to substrates if you want a very low level of free ones that can react at uncontrolled positions. (And then some basic explanation of what entropy/enthalpy/etc are, and what enzyme intermediate states look like.)

When you write a post that gets comments from many people, it's not practical to respond to them all. If you try to, you have less time than the collective commenters, and less information about their position than they have about yours. So you have to guess about what exactly each person is misunderstanding, and that's not usually something I enjoy.


What do you most want people to know about this post, for deciding whether to read or review-vote on it?

Of the 7 (!) posts of mine currently nominated for "Best of 2023", this is probably the most appropriate for that.

Of the 2023 posts of mine not currently nominated, my personal favorites were probably:

Clearly my opinion of my own posts doesn't correlate with upvotes here that well.

My all-time best post in my view is probably: https://bhauth.com/blog/biology/alzheimers.html

How concretely have you (or others you know of) used or built on the post? How has it contributed to a larger conversation

Muireall Prase wrote this, and my post was relevant for some conversations on twitter. I suppose it also convinced some people I had some understanding of chemistry.

Comment by bhauth on The Dream Machine · 2024-12-08T22:24:21.359Z · LW · GW

So, I have a lot of respect for Sarah, I think this post makes some good points, and I upvoted it. However, my concern is, when I look at this particular organization's Initiatives page, I see "AI for math", "AI for education", "high-skill immigration assistance", and not really anything that distinguishes this organization from the various other ones working on the same things, or their projects from a lot of past projects that weren't really worthwhile.

Comment by bhauth on Mask and Respirator Intelligibility Comparison · 2024-12-08T19:25:41.607Z · LW · GW

Note that due to the difference being greater at higher frequencies, the effect on speech intelligibility will probably be greater for most women than for you.

We can see the diaphragm has some resonance peaks that increase distortion. Probably it's too thick to help very much, but it has to resist the pressure changes from breathing.

Comment by bhauth on The 2023 LessWrong Review: The Basic Ask · 2024-12-07T02:32:21.411Z · LW · GW

What exactly are people looking for from (the site-suggested) self-reviews?

Comment by bhauth on a space habitat design · 2024-11-28T00:16:31.513Z · LW · GW

As a "physicist and dabbler in writing fantasy/science fiction" I assume you took the 10 seconds to do the calculation and found that a 1km radius cylinder would have ~100 kW of losses per person from roller bearings supporting it, for the mass per person of the ISS. But I guess I don't understand how you expect to generate that power or dissipate that heat.

Comment by bhauth on a space habitat design · 2024-11-26T08:07:16.032Z · LW · GW

After being "launched" from the despinner, you would find yourself hovering stationary next to the ring.

Air resistance.

That is, however, basically the system I proposed near the end, for use near the center of a cylinder where speeds would be low.

Comment by bhauth on a space habitat design · 2024-11-26T08:05:18.557Z · LW · GW

This happened to Explorer 1, the first satellite launched by the United States in 1958. The elongated body of the spacecraft had been designed to spin about its long (least-inertia) axis but refused to do so, and instead started precessing due to energy dissipation from flexible structural elements.

picture: https://en.wikipedia.org/wiki/Explorer_1#/media/File:Explorer1.jpg

Comment by bhauth on overengineered air filter shelving · 2024-11-09T14:53:33.011Z · LW · GW

That works well enough, but a Vital 200S currently costs $160 at amazon, less than the cheapest variant of the thing you linked, and has a slightly higher max air delivery rate, some granular carbon in the filter, and features like power buttons. The Vital 200S on speed 2 has similar power usage and slightly less noise, but less airflow, but a carbon layer always reduces airflow. It doesn't have a rear intake so it can be placed against a wall. It also has a washable prefilter.

Compared to what you linked, the design in this post has 3 filters instead of 2, some noise blocking, and a single large fan instead of multiple fans. Effective floor area usage should be slightly less, but of course it has to go together with shelving for that.

Comment by bhauth on The Median Researcher Problem · 2024-11-04T03:12:20.336Z · LW · GW

What would this say about subculture gatekeeping? About immigration policy?

Comment by bhauth on electric turbofans · 2024-11-03T21:54:29.654Z · LW · GW

First, we have to ask: what's the purpose? Generally aircraft try to get up to their cruise speed quickly and then spend most of their time cruising, and you optimize for cruise first and takeoff second. Do we want multiple cruise speeds, eg a supersonic bomber that goes slow some of the time and fast over enemy territory? Are we designing a supersonic transport and trying to reduce fuel usage getting up to cruise?

And then, there are 2 basic ways you can change the bypass ratio: you can change the fan/propeller intake area, or you can turn off turbines. The V-22 has a driveshaft through the wing to avoid crashes if an engine fails; in theory you could turn off an engine while powering the same number of propellers, which is sort of like a variable bypass ratio. If you have a bunch of turbogenerators inside the fuselage, powering electric fans elsewhere, then you can shut some down while powering the same number of fans. There are also folding propellers.

The question is always, "but is that better"?

Comment by bhauth on Electrostatic Airships? · 2024-10-28T02:45:48.206Z · LW · GW

On the other hand, the hydrogen pushing against the airship membrane is also an electrostatic force.

Comment by bhauth on Electrostatic Airships? · 2024-10-27T21:21:56.923Z · LW · GW

Yes, helium costs would be a problem for large-scale use of airships. Yes, it's possible to use hydrogen in airships safely. This has been noted by many people.

Hydrogen has some properties that make it relatively safe:

  • it's light so it rises instead of accumulating on the ground or around a leak
  • it has a relatively high ignition temperature

and some properties that make it less safe:

  • it has a wide range of concentrations where it will burn in air
  • fast diffusion, that is, it mixes with air quickly
  • it leaks through many materials
  • it embrittles steel
  • it causes some global warming if released

Regardless, the FAA does not allow using hydrogen in airships, and I don't expect that to change soon. Especially since accidents still happen despite the small number of airships.

In any case, the only uses of airships that are plausibly economical today are: advertising and luxury yachts for the wealthy. Are those things that you care about working towards?

Comment by bhauth on Could randomly choosing people to serve as representatives lead to better government? · 2024-10-24T16:22:37.152Z · LW · GW

see also: These Are Your Doges, If It Please You

Comment by bhauth on If far-UV is so great, why isn't it everywhere? · 2024-10-21T22:04:56.081Z · LW · GW

IKEA already sells air purifiers; their models just have a very low flow rate. There are several companies selling various kinds of air purifiers, including multiples ones with proprietary filters.

What all this says to me is, the problem isn't just the overall market size.

Comment by bhauth on If far-UV is so great, why isn't it everywhere? · 2024-10-20T21:28:50.346Z · LW · GW

Apart from potential harms of far-UVC, it's good to remove particulate pollution anyway. Is it possible that "quiet air filters" is an easier problem to solve?

Comment by bhauth on Start an Upper-Room UV Installation Company? · 2024-10-19T20:52:58.988Z · LW · GW

I'm not convinced that far-UVC is safe enough around humans to be a good idea. It's strongly absorbed by proteins so it doesn't penetrate much, but:

  • It can make reactive compounds from organic compounds in air.
  • It can produce ozone, depending on the light. (That's why mercury vapor lamps block the 185nm emission.)
  • It could potentially make toxic compounds when it's absorbed by proteins in skin or eyes.
  • It definitely causes degradation of plastics.

And really, what's the point? Why not just have fans sending air to (cheap) mercury vapor lamps in a contained area where they won't hit people or plastics?

Comment by bhauth on on bacteria, on teeth · 2024-10-02T22:43:00.524Z · LW · GW

As you were writing that, did you consider why chlorhexidine might cause hearing damage?

Comment by bhauth on on bacteria, on teeth · 2024-10-02T19:12:35.698Z · LW · GW

https://en.wikipedia.org/wiki/Chlorhexidine#Side_effects

It can also obviously break down to 4-chloroaniline and hexamethylenediamine. Which are rather bad. This was not considered in the FDA's evaluation of it.

Comment by bhauth on on bacteria, on teeth · 2024-10-02T01:11:09.679Z · LW · GW

If you just want to make the tooth surface more negatively charged...a salt of poly(acrylic acid) seems better for that. And I think some toothpastes have that.

Comment by bhauth on on bacteria, on teeth · 2024-10-01T23:00:20.427Z · LW · GW

EDTA in toothpaste? It chelates iron and calcium. Binding iron can prevent degradation during storage, so a little bit is often added.

Are you talking about adding a lot more? For what purpose? In situations where you can chelate iron to prevent bacterial growth, you can also just kill bacteria with surfactants. Maybe breaking up certain biofilms held together by Ca? EDTA doesn't seem very effective for that for teeth, but also, chelating agents that could strip Ca from biofilms would also strip Ca from teeth. IIRC, high EDTA concentration was found to cause significant amounts of erosion.

I wouldn't want to eat a lot of EDTA, anyway. Iminodisuccinate seems less likely to have problematic metabolites.

Comment by bhauth on If I wanted to spend WAY more on AI, what would I spend it on? · 2024-09-16T22:36:45.565Z · LW · GW

You can post on a subreddit and get replies from real people interested in that topic, for free, in less than a day.

Is that valuable? Sometimes it is, but...not usually. How much is the median comment on reddit or facebook or youtube worth? Nothing?

In the current economy, the "average-human-level intelligence" part of employees is only valuable when you're talking about specialists in the issue at hand, even when that issue is being a general personal assistant for an executive rather than a technical engineering problem.

Comment by bhauth on Morpheus's Shortform · 2024-09-10T22:58:30.460Z · LW · GW

Triplebyte? You mean, the software job interviewing company?

  1. They had some scandal a while back where they made old profiles public without permission, and some other problems that I read about but can't remember now.

  2. They didn't have a better way of measuring engineering expertise, they just did the same leetcode interviews that Google/etc did. They tried to be as similar as possible to existing hiring at multiple companies; the idea wasn't better evaluation but reducing redundant testing. But companies kind of like doing their own testing.

  3. They're gone now, acquired by Karat. Which seems to be selling companies a way to make their own leetcode interviews using Triplebyte's system, thus defeating the original point.

Comment by bhauth on you should probably eat oatmeal sometimes · 2024-09-10T22:51:06.368Z · LW · GW

Good news: the post is both satire and serious, at the same time but on different levels.

Comment by bhauth on How to bet against civilizational adequacy? · 2024-09-08T23:48:45.386Z · LW · GW

Here are some publicly traded large companies that do a lot of coal mining:

Coal India did pretty well, I guess. The others, not so much.

Comment by bhauth on Fun With CellxGene · 2024-09-06T22:57:45.704Z · LW · GW

Nice post Sarah.

If Alzheimer's is ultimately caused by repressor binding failure, that could explain overexpression of the various proteins mentioned.

Comment by bhauth on the Giga Press was a mistake · 2024-09-06T22:49:49.881Z · LW · GW

in short, your claim: "The cost of aluminum die casting and stamped steel is, on Tesla's scale, similar" both seems to miss the entire point and run against literally everything I have seen written about this. You need citations for this claim, I am not going to take your word for it.

OK, here's a citation then: https://www.automotivemanufacturingsolutions.com/casting/forging/megacasting-a-chance-to-rethink-body-manufacturing/42721.article

Here I would be careful since investments, especially in a particular model generation of welding robots, are depreciated. For forming processes, the depreciation can even extend over three or four model generations. This technological write-off – bear in mind that this is not tax-related – runs over a timeframe of 30 years. For the OEMs that are already using these machines for existing vehicle generations, the use of the new technology makes no sense. On the other hand, thanks to its greenfield approach, Tesla can save itself these typical investments in shell-type construction. In a brownfield, it would be operationally nonsensical not to keep using long depreciated machinery. So, in this situation, I would not support the 20-30% cost savings that were cited.

With die casting, one important aspect is that there is a noticeable reduction in the service life of the die-casting moulds. Due to so-called thermal shock, the rule of thumb is that a die-casting mould is good for 100,000-150,000 shots. By contrast, one forming tool is used to make 5m-6m parts. So, we are talking about a factor of 20 to 30. There is quite clearly a limited volume range for which the casting-intensive solution would be appropriate. To me, aluminium casting holds little appeal for very small and very large volumes. Especially for mass production in the millions, you would need about six or seven of these expensive die-casting moulds. We estimate that the die-casting form for the single part, rear-end of a Tesla would weigh about 80-100 tonnes. This translates to huge costs for handling and the peripheral equipment, in the form of cranes, for example. Die-casting moulds also pose technological obstacles and hazards. The leakage of melted material is cited as one example. The risks of not even being able to operate in some situations are not negligible.


Or the geometry of the frame was insufficiently optimized for vertical shear. I do not understand how you reached this conclusion.

No. If aluminum doesn't have weak points, it stretches/bends before breaking. The Cybertruck hitch broke off cleanly without stretching. Therefore there was a weak point.


By nothing I mean that the estimate for their marketing spend in 2022 (literally all marketing to include PR if there was any at all) was $175k.

I'm skeptical of that. PR firms don't report to Vivvix.

Comment by bhauth on on Science Beakers and DDT · 2024-09-06T03:04:26.566Z · LW · GW

Here are the costs from the above link:

It's worth noting that countries (such as India) have the option of simply not respecting a patent when the use is important and the fees requested are unreasonable. Also, patents aren't international; it's often possible to get around them by simply manufacturing and using a chemical in a different country.

Comment by bhauth on on Science Beakers and DDT · 2024-09-05T18:33:05.080Z · LW · GW

The only advantage DDT has over those is lower production cost, but the environmental harms per kg of DDT are greater than the production cost savings, so using it is just never a good tradeoff.

As I said, if DDT was worth using there, it was worth spending however much extra money it would have been to spray with other things instead. If it wasn't worth that much money, it wasn't worth spraying DDT.

And regarding "environmental harms," from personal experience scratching myself bloody as a kid from itchy bites after going to the park in the evening, I would extinct a dozen species if mosquitoes went down with them.

The biggest problem with DDT is that it is bad for humans.

Comment by bhauth on on Science Beakers and DDT · 2024-09-05T18:27:10.476Z · LW · GW

While I still disagree with your interpretation of that post, I don't want to argue over the meaning of a post from that blog. There are actual books written about the history of titanium. I'm probably as familiar with it as the author of Construction Physics, and saying A-12-related programs were necessary for development of titanium usage is just wrong. People who care about that and don't trust my conclusion should go look up good sources on their own, more-extensive ones.

Comment by bhauth on on Science Beakers and DDT · 2024-09-05T05:09:16.467Z · LW · GW

If it wasn't for the A-12 project (and its precursors and successors), then we simply wouldn't be able to build things out of titanium.

That is not an accurate summary of the linked article.

In 1952, another titanium symposium was held, this one sponsored by the Army’s Watertown Arsenal. By then, titanium was being manufactured in large quantities, and while the prior symposium had been focused on laboratory studies of titanium’s physical and chemical properties, the 1952 symposium was a “practical discussion of the properties, processing, machinability, and similar characteristics of titanium". While physical characteristics of titanium still took center stage, there was a practical slant to the discussions – how wide a sheet of titanium can be produced, how large an ingot of it can be made, how can it be forged, or pressed, or welded, and so on. Presentations were by titanium fabricators, but also by metalworking companies that had been experimenting with the metal.

That's before the A-12.

In 1966, another titanium symposium was held, this one sponsored by the Northrup Corporation. By this time, titanium had been used successfully for many years, and the purpose of this symposium was to “provide technical personnel of diversified disciplines with a working knowledge of titanium technology.” This time, the lion’s share of the presentations are by aerospace companies experienced in working with the metal, and the uncertain air that existed in the 1952 symposium is gone.

At that point, the A-12 program was still classified and the knowledge gained from it was not widely shared.

Comment by bhauth on Things I learned talking to the new breed of scientific institution · 2024-08-31T02:15:19.171Z · LW · GW

I had an interview with one of these organizations (that will remain unnamed) where the main person I was talking to was really excited about a bunch of stupid bullshit ideas (for eg experimental methods) that, based on their understanding of them, must have come from either university press releases or popular science magazines like New Scientist. I was trying to find a "polite in whatever culture these people have" way to say "this is not useful, I'd like to explain why but it will take a while, here are better things" but doing that eloquently is one of my weak points.

From what I've seen of the people there, ARPA-E has some smart people ("ordinary smart", not geniuses) but the ARPAs are still very tied to the university system, with a heavy reliance on PhD credentials.

Comment by bhauth on AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work · 2024-08-28T21:07:46.634Z · LW · GW

I think the basic idea of using more steps of smaller size is worth considering. Maybe it reduces overall drift, but I suspect it doesn't, because my view is:

Models have many basins of attraction for sub-elements. As model capability increases continuously, there are nearly-discrete points where aspects of the model jump from 1 basin to another, perhaps with cascading effect. I expect this to produce large gaps from small changes to models.

Comment by bhauth on you should probably eat oatmeal sometimes · 2024-08-26T18:02:35.776Z · LW · GW

Sure, some people add stuff like cheese/tomatoes/ham to their oatmeal. Personally I think they go better with rice, but de gustibus non disputandum est.

Comment by bhauth on AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work · 2024-08-26T08:29:41.857Z · LW · GW

The scope of our argument seems to have grown beyond what a single comment thread is suitable for.

AI safety via debate is 2 years before Writeup: Progress on AI Safety via Debate so the latter post should be more up-to-date. I think that post does a good job of considering potential problems; the issue is that I think the noted problems & assumptions can't be handled well, make that approach very limited in what it can do for alignment, and aren't really dealt with by "Doubly-efficient debate". I don't think such debate protocols are totally useless, but they're certainly not a "solution to alignment".

Comment by bhauth on AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work · 2024-08-25T18:41:14.051Z · LW · GW

I don't expect such a huge gap between debaters and judges that the judge simply can't understand the debaters' concepts

You don't? But this is a major problem in arguments between people. The variation within humans is already more than enough for this! There's a gap like that every 35 IQ points or so. I don't understand why you're confident this isn't an issue.

I guess we've found our main disagreement, at least?

So in this particular case I am saying: if you penalize debaters that are inconsistent under cross-examination, you are giving an advantage to any debater that implements an honest strategy, and so you should expect training to incentivize honesty.

Now you're training for multiple objectives:

  1. You want the debater AI to argue for proposition A or not-A according to its role and convince human judges of that.
  2. You want it to not change its position on sub-arguments.

But (2) is ill-defined. Can sub-arguments be combined for less weighting? Are they all worth the same? What if you have several sub-arguments that all depend on a single sub-2-argument? Good arguments for A or not-A should have lots of disagreements - or do you want to train AI that makes all the same sub-arguments for A or not-A and then says "this implies A / not-A"? I don't think this works.


In response to the linked "HCH" post:

Yes, an agent past some threshold can theoretically make a more-intelligent agent. But that doesn't say anything about alignment; the supposed "question-answering machine" would be subject to instrumental convergence and mesaoptimizer issues, and you'd get value drift with each HCH stage, just as you would with RSI schemes.

Comment by bhauth on you should probably eat oatmeal sometimes · 2024-08-25T16:27:20.998Z · LW · GW

Phytic acid is certainly a thing, but it's not quite that simple, see eg https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8746346/. Also, uncooked fruits have phytase. And also, it's not an issue unless you eat mostly something high in it for most meals.

Comment by bhauth on you should probably eat oatmeal sometimes · 2024-08-25T16:15:46.270Z · LW · GW

Yes, on one level that's part of the joke. But also, following the above instructions, it can be a low-cost complete meal with nonperishable ingredients that can be fixed in <5 minutes of work and <10 minutes of waiting.

Comment by bhauth on you should probably eat oatmeal sometimes · 2024-08-25T14:54:42.121Z · LW · GW

I'm the current owner of the Oatmeal subreddit; that's how you can be sure I'm a Real Expert.

Comment by bhauth on AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work · 2024-08-25T13:01:04.595Z · LW · GW

If you want to disallow appeals to authority

I do, but more importantly, I want to disallow the judge understanding all the concepts here. Suppose the judge says to #1: "What is energy?" or "What is conservation?" and it can't be explained to them - what then?

Also, argument 1 isn't actually correct, E=mc^2 and so on.

That seems right, but why is it a problem? The honest strategy is fine under cross-examination, it will give consistent answers across contexts.

"The honest strategy"? If you have that, you can just ask it and not bother with the debate. If the problem is distinguishing it, and only dishonest actors are changing their answers based on the provided situation, you can just use that info. But why are you assuming you have an "honest strategy" available here?

Comment by bhauth on AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work · 2024-08-25T11:13:24.290Z · LW · GW

You can recursively decompose the claim "perpetual motion machines are known to be impossible" until you get down to a claim like "such and such experiment should have such and such outcome", which the boss can then perform to determine a winner.

Ah, I don't think you can. Making that kind of abstract conclusion from a practical number of experiments requires abstractions like potential energy, entropy, Noether's theorem, etc - which in this example, the judge doesn't understand. (Without such abstractions, you'd need to consider every possible type of machine separately, which isn't feasible.) This seems like a core of our disagreement here.

You can cross-examine the inventor and show that in other contexts they would agree that perpetual energy machines are impossible.

The debaters are the same AI with different contexts, so the same is true of both debaters. Am I missing something here?

Which paper are you referring to? If you mean doubly efficient debate

Yes, "doubly efficient debate".

Comment by bhauth on quila's Shortform · 2024-08-25T00:57:46.661Z · LW · GW

That argument doesn't explain things like:

  • furry avatars are almost always cartoon versions of animals, not realistic ones
  • furries didn't exist until anthropomorphic cartoon animals became popular (and no, "spirit animals" are not similar)
  • suddenly ponies became more popular in that sense after a popular cartoon with ponies came out

It's just Disney and cartoons.

Comment by bhauth on AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work · 2024-08-25T00:45:36.480Z · LW · GW

To clarify the 2nd point, here's an example. Suppose someone presents you with a large box that supposedly produces electricity endlessly. Your boss thinks it works, and you're debating the inventor in front of your boss.

"Perpetual motion machines are known to be impossible" you say, but your boss isn't familiar with that conceptual class or the reasoning involved.

The inventor says, "Here, let's plug in a thing, we can see that the box does in fact produce a little electricity." Your boss finds this very convincing.

The process proposed in the paper is something like, "let's randomly sample every possible machine to see if it does perpetual motion". So the inventor points to the sun and says, "that thing has been making energy continuously and never stops for as long as we've been able to tell". They point to some stars and say the same thing.

The sampling and evaluation is dependent on a conceptual framework that isn't agreed on, and waiting for the sun and stars to burn out isn't very practical.

Comment by bhauth on AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work · 2024-08-23T23:40:41.388Z · LW · GW

I took a look at the debate papers. I think that's a good angle to take, but they're missing some factors that sometimes make debates between humans fail.

  1. Humans and neural networks both have some implicit representation of probability distributions of output types. The basis behind "I can't explain why but that seems unlikely" can be more accurate than "here's an argument for why that will happen". You're basically delegating the problem of "making AI thinking explainable" to the AI itself, but if you could do that, you could just...make neural networks explainable, perhaps by asking an AI what another AI is doing and doing RLHF on the response. But that doesn't seem to work in general. In other words, the problem is that by using only the arguments output by NNs, those are weaker agents than NNs that don't have to go through production of arguments.

  2. Reasoning about probability distributions means argument branches can be of the type "X is a likely type of thing" vs "X is rare". And empirically checking the distribution can be too expensive. That makes the debate framework not work as well.

Comment by bhauth on the Giga Press was a mistake · 2024-08-23T22:57:09.458Z · LW · GW

We're talking about different timescales. Apple's investments paid off within the tenure of top executives. Meanwhile, banks are still using COBOL.

Comment by bhauth on The economics of space tethers · 2024-08-23T00:59:10.299Z · LW · GW

I upvoted this post, but I do have a few comments.

For what it’s worth, I like glass fibers. They’re pretty easy to make, the material can be be sourced in space, they can handle large temperature ranges, and they’re resistant to atomic oxygen environments and UV

The matrix holding the fibers together is generally going to be more prone to degradation. Glass fibers have good compressive strength, but carbon fiber would be better here.

Maintaining orbit is one of the key issues. You probably need ion thusters and solar panels. I don't think electrodynamic tethers actually work, because of friction vs conductivity.

At these scales and speeds, you can't just think of "solid things" as being rigid. Speed of sound in solid materials becomes a major issue. When something attaches to the tether, there's a wave of increased tension and stretching that propagates through the tether and sets up a vibration. This is a fatal problem for some tether variants.

The projectile needs to reliably connect to the tether. Docking in space is usually slow and doesn't involve large forces, and it's still not easy, but here it needs to be done quickly and establish a strong connection. Here, you could just have a hook grab a perpendicular rope, but if you don't have any contingency plans, well, "dock or die" isn't very appealing. Especially if it happens multiple times.

Yes, micrometeoroids are an issue. Even if there aren't many, the tether might need to be robust to small impacts. A low orbit reduces that risk (but doesn't eliminate it) but a tether would also have relatively high drag; the surface area per mass is higher than eg the ISS.

The main thing people want to do with rockets has been put satellites in orbit. I don't see a reason to expect that to change anytime soon.

People have thought of all this decades ago. Maybe check out "LEOBiblio" or something.

Comment by bhauth on the Giga Press was a mistake · 2024-08-22T23:29:35.376Z · LW · GW

"Everyone is going to switch to cloud stuff" means that, in the short term, there will be a shortage of cloud people and an excess of non-cloud people.

Your argument is for hiring in a long-term future where the non-cloud people retired or forgot how to do their thing, but we know that's not what US executives were thinking because they don't think that long-term due to the incentives they face.

And it certainly doesn't explain some groups of companies switching to cloud stuff together and then switching back together later.