Posts
Comments
Reading your comment and then rereading mine, I think I've been doing a terrible job explaining myself. I am not generally in favor of central planning, and am generally in favor of permitting reform, more utility scale solar, fewer subsidies, removal of net metering, and introduction of real time electricity pricing.
What I haven't been commenting on is which things I think are going to happen whether I like it or not, which things I think would be good but only if we also remove the other distortions they currently counterbalance, and which I don't think are politically feasible regardless of what their practical impacts would be.
I think within a few years it will become clear to many farmers that agrivoltaics would be a net benefit to themselves, so long as policy doesn't stand in their way. There's a lot more buried in that caveat than I feel like going into here, though.
FWIW, I agree with that. But, while land is not scarce in the US, long distance transmission capacity is. There are definitely places where putting solar on roofs is cheaper, or at least faster and easier, than getting large amounts of utility scale solar permitted and building the transmission capacity to bring it to where the demand is.
And I don't just think agrivoltaics is cool. I think it dodges a lot of usually-bogus-but-impactful objections that so many large scale new construction projects get hit with.
Excellent comment, spells out a lot of thoughts I'd been dancing around for a while better than I had.
-- Avoid tying up capital in illiquid plans. One exception is housing since 'land on Holy Terra' still seems quite valuable in many scenarios.
This is the step I'm on. Just bought land after saving up for several years while being nomadic, planning on building a small house soon in such a way that I can quickly make myself minimally dependent on outside resources if I need to. In any AI scenario that respects property rights, this seems valuable to me.
-- Make whatever spiritual preparations you can, whatever spirituality means to you. If you are inclined to Buddhism meditate. Practice loving kindness. Go to church if you are Christian. Talk to your loved ones. Even if you are atheist you need to prepare your metaphorical spirit for what may come. Physical health may also be valuable here.
This I need to double down on. I've been more focused on trying to get others close to me on board with my expectations of impending weirdness, with moderate success.
The idea that AIs would go around killing everyone instead of just doing what we tell them to do seems like science fiction.
I've had this experience too. The part that baffles me about it is the seeming lack of awareness of the gap between "what we tell them to do" and "what we want them to do." This gap isn't sci-fi, it already exists in very clear ways (and should be very familiar to anyone who has ever written any code of any kind).
I have (non-AI-expert) colleagues that I've talked to about LLM use, where they dismiss the response to a prompt as nonsense, so I ask to see the chat logs, and I dig into it for 5 minutes. Then I inform them that actually, the LLM's answer is correct, you didn't ask what you thought you were asking, you missed an opportunity to learn something new, and also here's the three-times-as-long version of the prompt that gives enough context to actually do what you expected.
Edit to add: I'm also very much a fan of panko toasted in butter in a pan as a topping. Can be made in advance and left in a bowl next to the mac and cheese for everyone to use however much they want.
Yes, at conferences I've been to the discussion is increasingly not "How will we afford all the long term energy storage?" so much as "how much of a role will there be for long term energy storage?"
Personally I'm fairly confident that we'll eventually need at least 4-16 hrs energy storage in most places, and more in some grids, but I suspect that many places will be able to muddle and kludge their way through most multi-day storage needs with a bunch of other partial solutions that generate power or shift demand.
One of many cases where it's much easier to predict the long-term trajectory than the path to get there, and most people still don't.
I like to put the numbers in a form that less mathy folks seem to find intuitive. If you wanted to replace all of the world's current primary energy use with current solar panels, and had enough storage to make it all work, then the land area you'd need is approximately South Korea. Huge, but also not really that big. (Note: current global solar panel manufacturing capacity is enough to get us about a half to a third of the way there if we fully utilize it over the next 25 years).
In practice I think over the next handful of decades we're going to need 3-10x that much electricity, but even that doesn't really change the conclusion, just the path. But also, we can plausibly expect solar panel efficiencies and capacity factors to go up as we start moving towards better types of PV tech. For example, based on already demonstrated performance values, a 2 or 3 junction tandem bifacial perovskite solar panel (which no one currently manufactures at scale, and which seemed implausible to most people including me even two years ago) could get you close to double the current kWh/m2 we get from silicon, and the power would be spread much less unevenly throughout the day and year.
Context: right now gas peaker plants with ~10% utilization have LCOE of about 20 cents/kWh, about 3-5x most other energy sources. I think in the proposed scenario here we'd be more like 20-40% utilization, since we'd also get some use out of these systems overnight night and in winter.
If this became much more common and people had to pay such variable prices, we'd also be able to do a lot more load shifting to minimize the impact on overall energy costs (when to dry clothes and heat water, using phase change materials in HVAC, using thermal storage in industrial facilities' systems, etc.).
I agree with this reasoning.
I'd add that part of the answer is: as various other relevant technologies become cheaper, both the solar farm and nuclear plant operators (and/or their customers) are going to invest in some combination of batteries and electrolyzers (probably SOECs for nuclear to use some of the excess heat) and carbon capture equipment and other things in order to make other useful products (methanol, ammonia, fuels, other chemicals, steel, aluminum, etc.) using the excess capacity.
Personally I'm not a fan of the pasta texture of baked mac and cheese, but I've definitely sauced the cooked pasta, topped with cheese, and broiled it. That's fast, and you could spread it across multiple pans so it has more surface area. I suspect a blow torch could also work?
Our solar farms are not yet visible from space; we don't yet have patches of desert turning black.
Keep in mind that as best we have researched so far, agrivoltaics enable dual land use, and for some crops in some environments can increase crop yields and lower water consumption. It is not obvious that meeting most electricity demand with solar requires all that much new land, if we start to make effective use of both farmland and rooftops. From what I've read it looks like depending on a whole lot of factors you can get about 80% as much electricity as a dedicated solar farm and 80-160% as much crop yield as a regular farm at the same time. This seems to be true even for corn (5-10% decrease in yield) and pasture (increase in yield). When you consider that ~8% of the world's cropland is used for ethanol production (>30% for corn in the US), this suggests that a switch towards solar electric for vehicles and away from biofuels might plausibly keep land requirements for farming constant or even reduce the total.
I have absolutely no confidence in any government's capacity to actually structure markets, regulations, and incentives in a way that allows us to realize anything like an optimal food and energy system. But, if there were a food-and-energy-czar actually designing such a system, this problem has several foreseeable solutions. And as both solar panels and batteries continue to get cheaper and pressure to decarbonize increases, I think we'll stumble towards something like this anyway.
It is a lot of assumption and conjecture, that's true. But it is not all conjecture and assumptions. When comparative advantage applies despite one side having an absolute advantage, we know why it applies. We can point to which premises of the theory are load-bearing, and know what happens when we break those premises. We can point to examples within the range of scenarios that exist among humans, where it doesn't apply, without ever considering what other capabilities an ASI might have.
I will say I do think there's a bit of misdirection, not by you, but by a lot of the people who like to talk about comparative advantage in this context, to the point that I find it almost funny that it's the people questioning premises (like this post does) getting accused of making assumptions and conjectures. I've read a number of articles that start by talking about how comparative advantage normally means there's value in one agent's labor even when another has absolute advantage, which is of course true. Then they simply assume the necessary premises apply in the context of humans and ASI, without actually ever investigating that assumption, looking for limits and edge cases, or asking what actually happens if and when they don't hold. In other words, the articles I've read, aren't trying to figure out whether comparative advantage is likely to apply in this case. They're simply assuming it will, and that those questioning this assumption or asking about the probability and conditions of it holding don't understand the underlying theory.
For comparative advantage to apply, there are conditions. Breaking the conditions doesn't always break comparative advantage, of course, because none of them perfectly apply in real life ever, but they are the openings that allow it to sometimes not apply. Many of these are predictably broken more often when dealing with ASI, meaning there will be more examples where comparative advantage considerations do not control the outcome.
A) Perfect factor mobility within but none between countries.
B) Zero transportation costs.
Plausibly these two apply about as well to the ASI scenario as among humans? Although with labor as a factor, human skill and knowledge act as limiters in ways that just don't apply to ASI.
C) Constant returns to scale - untrue in general, but even small discrepancies would be much more significant if ASI typically operates at much larger o much more finely tuned scale than humans can.
D) No externalities - potentially very different in ASI scenario, since methods used for production will also be very different in many cases, and externalities will have very different impacts on ASI vs on humans.
E) Perfect information - theoretically impossible in ASI scenario, ASI will have better information and understanding thereof
F) Equivalent products that differ only in price - not true in general, quality varies by source, and ASI amplifies this gap.
For me, the relevant questions, given all this, are 1) Will comparative advantage still favor ASI hiring humans for any given tasks? 2) If so, will the wage at which ASI is better off choosing to pay humans be at or above subsistence? 3) If so, are there enough such scenarios to support the current human population? 4) Will 1-3 continue to hold in the long run? 5) Are we confident enough in 1-4 for these considerations to meaningfully affect our strategy in developing and deploying AI systems of various sorts?
I happily grant that (1) is likely. (2) is possible but I find it doubtful except in early transitional periods. (3)-(4) seem very, very implausible to me. (5) I don't know enough about to begin to think about concretely, which means I have to assume "no" to avoid doing very stupid things.
The traditional comparative advantage discussion also, as I understand it, does not account for entities that can readily duplicate themselves in order to perform more tasks in parallel, and does not account for the possibility of wildly different transaction costs between ASI and humans vs between ASI and its non-human robot bodies. Transaction costs in this scenario include monitoring, testing, reduced quality, longer lag times. It is possible that the value of using humans to do any task at all could actually be negative, not just low.
Human analogy: you need to make dinner using $77 worth of ingredients. A toddler offers to do it instead. At what price should you take the deal? When does the toddler have comparative advantage?
I'm also not a professional, so my version is also very incomplete, but at a bare minimum, exposure therapy needs to be done kinda the way building up tolerance for poisons or allergens is done - carefully, in risk-minimizing contexts, with support systems on standby if things go wrong.
To quote SSC, I think accurately despite being in a very different context:
Psychotherapists treat arachnophobia with exposure therapy, too. They expose people first to cute, little spiders behind a glass cage. Then bigger spiders. Then they take them out of the cage. Finally, in a carefully controlled environment with their very supportive therapist standing by, they make people experience their worst fear, like having a big tarantula crawl all over them. It usually works pretty well.
Finding an arachnophobic person, and throwing a bucket full of tarantulas at them while shouting “I’M HELPING! I’M HELPING!” works less well....
There are two problems with its approach. The first is that it avoids the carefully controlled, anxiety-minimizing setup of psychotherapy.
The second is that YOU DO NOT GIVE PSYCHOTHERAPY TO PEOPLE WITHOUT THEIR CONSENT.
If a person... doesn’t want psychotherapy, then even as a trained psychiatrist I am forbidden to override that decision unless they become an immediate danger to themselves or others.
Although, in another SSC post:
Exposure therapy can also be useful for panic attacks or specific phobias. This is where they expose you to the thing you’re scared of (or deliberately initiate a panic attack) and keep doing it until you stop being scared and start being bored. According to a bunch of studies it works neither better nor worse than cognitive-behvioral therapy for most things, but my unsupported impression has always been that it’s better at least for panic disorder. Cognitive-behavioral therapy seems clearly superior for social phobia.
“But exposure works!” people yell from across the street.
Do not cite the Deep Magic to me, Witch.
In some ways, that's my fully general response to being given Standard Advice(TM). The giver may not know the Deeper Magic.You may not either, and you may not have been there when it was written, but if it doesn't sound right to you, you can look before deciding whether to buy into it.
In this case, that may be the layperson's standard advice, but it's definitely not the professional's standard advice, which can't be condensed to a single short phrase. Five Words Are Not Enough, because "exposure" and "works" are both so underspecified.
Communication strategy is important. Who is the audience. When to speak vs. not. How much detail to use. Which points to make. What kind of perspective to speak from.
It's a set of skills many of us attracted to this community don't instinctively lean towards, but yeah, it's really valuable.
Nothing Machiavellian needed. Mostly, the same skills are what you need even when everyone involved is well aware of who is trying to communicate about what to whom, and why.
We still can find errors in every phishing message that goes out, but they’re getting cleaner.
Whether or not this is true today, it is a statement in which I put near-zero credence.
Harry had unthinkingly started to repeat back the standard proverb that there was no such thing as a perfect crime, before he actually thought about it for two-thirds of a second, remembered a wiser proverb, and shut his mouth in midsentence. If you did commit the perfect crime, nobody would ever find out - so how could anyone possibly know that there weren't perfect crimes? And as soon as you looked at it that way, you realized that perfect crimes probably got committed all the time, and the coroner marked it down as death by natural causes, or the newspaper reported that the shop had never been very profitable and had finally gone out of business...
And also, not entirely unrelated:
Do people really not get why ‘machines smarter than you are’ are not ‘just another technology’?
Assuming the question framing is mostly rhetorical, but if not: No, they really, really don't. That's not even really a problem with it being "technology." Have you ever tried explaining to someone that some human or human organization much smarter than them is using that intelligence to harm them in ways they aren't smart enough to be able to perceive? It being "technology" just makes it even harder for people to realize.
Of course it's weird. My dad, at 11, was allowed to ride his bike all around Queens, NY, with a friend. I, even at 17 in the early 2000s, was not allowed to ride my bike unsupervised in our own cul-de-sac in the suburbs, let alone to actually go anywhere. Sanity and sense have nothing to do with it.
This definitely happens too, but have you ever had to deal with customer service departments unwilling to think anything is something other than the most obvious? Or dealt with a hard to diagnose medical condition and had doctor after doctor keep insisting on going through the same useless-to-you diagnostic flow chart until you finally find one willing to actually think with you?
In some contexts, there's also the opposite problem of people seeing a hard to understand problem and insisting they need to investigate when that's not necessary for finding a solution. In analogy to the (yes, very possibly false) vanilla ice cream story, the man could have tried switching brands of vanilla, or walking to the aisle instead of buying from the endcap, or buying multiple pints of ice cream during normal grocery shopping to last the week instead of making a special trip, without ever bothering to investigate. Or, if you have symptoms that your doctor thinks come from an inoperable congenital defect, but the solution for the symptoms is the same medication whether you have the defect or not, then there's no value in finding the etiology, and no real reason for them to insist on expensive tests, but they often will anyway before treating, and pointing out this fact doesn't always help.
Traditionally, the y variable is another way of defining the output of a function. y=f(x)=ax+b, where a and b are numerical constants. You may not know the constants, but they are specific numbers. The x is a variable you can change to get different values of y=f(x). y=ax+b has two unknown constants, a parameter, and an output.
Concrete example: Say I am looking at a hill with a straight line sloped side. For every x feet I walk forward I move ax feet up. The a has a single numerical value defined by topography. The b has a single numerical value defined by the altitude at the base of the hill where x=0. The x can be anywhere from zero (a point on the perimeter of the base of the hill) to however far horizontally it is from there to the top of the hill. As x varies, y goes from b to b + the height/prominence of the hill.
Note: y isn't always intended to mean a function output. That depends on context. For example, say instead you have z=ax+by+c. This defines a plane that crosses the z axis at c, has slope a in the x direction, and has slope b in the y direction. It has two parameters (x and y) and three constants that generate the output, z=f(x,y)=ax+by+c
Parameters are also called the arguments of a function
Different concrete example from computer science: Say I want to write a program that multiplies a number by 4. I write code to take input from the user. I call the input x. I return x*4. y=4x. Or, instead, I could add a line in the code that defines a constant, a=4, and then I return ax. Why? Maybe I want to make it easy to see where to change a if I update the code in the future (I put a=4 at the top so I don't have to read to hunt for the place where I use the number 4). Maybe I'm writing code for a more complicated function that uses a multiple times and I want to be able to make updates while only changing it manually in one place, for readability and lower risk of making mistakes. Basically the constant a is defined in the code, while the argument x is defined by the user.
Congrats, and thanks for posting this. I agree there are a lot of people, here and otherwise, at or well below your 2022 level of mathematical skills. I think a lot of people here should get value out of posts like this, which remind the community of this fact. See also: https://xkcd.com/2501/. Below are two real world examples I'd like to share from my own life.
Around 2018, I had a conversation with a friend who had never encountered LW. I told them that, in practice, in terms of how most people make decisions, they aren't consequentialists. "This is predictably going to lead to outcomes other than what you want" is often just not a significant part of the thought process that determines actions. This was very eye-opening for this person.
Long before that, I once had a conversation with my mom, who as a kid had been in the top 5% of her high school class of almost a thousand students. She told me she didn't know whether a million or a billion was larger, or how far apart they were. For her, they were all just words for big numbers with no referents.
Completely agreed with the premise. I had something of the opposite experience: Truly excellent high school bio teacher, mediocre physics teacher, terrible chem teacher. I'd wanted to be a physicist since I was 5, but entered college wanting to be a biochem major.
That lasted all of one semester. Problem was, the intro courses for bio were full of (and geared towards) the hundreds of pre-med students who just wanted and needed to memorize the facts. Physics had a separate track for majors and non-majors. Chem had separate tracks for pre-med and physical science folks. Not bio, so I was stuck mostly re-learning things I already knew. Ironically, in my professional life two decades later, I don't deal with biology itself at all, but I make extensive use of the little bits of complex systems analysis and modeling I picked up in my bio, biochem, biophysics, and biostatistics classes. I just didn't need those at all in the classes themselves.
That ties into a part I find interesting. In society, we accord doctors very high status among professionals, but accord biologists comparatively low status among academics. Being a doctor is very hard and requires way more than just intelligence, I certainly couldn't do the job. But, I struggle to think of another field where we accord what are, effectively, highly skilled technicians more status than we do research scholars.
I've been known to say that, done right, biology, and even more so psychology, should be the most difficult sciences, and physics is simple. Most people really don't get what I mean, but the few that do are invariably interesting to talk to.
But it seems kind of obvious.
Then you should probably write the post, if you wanted to. It is emphatically not obvious to many other people.
In some ways, this would be better if you can get universal buy-in, since there wouldn't be a race for completion. There might be a race for alignment to particular subgroups? Which could be better or worse, depending.
Also, securing it against bringing insights and know-how back to a clandestine single-nation competitor seems like it would be very difficult. Like, if we had this kind of project being built, do I really believe there won't be spies telling underground data centers and teams of researchers in Moscow and Washington everything it learns? And that governments will consistently put more effort into the shared project than the secret one?
Back in the dot-com bubble, I had some contact with VCs who had a decent vision of how valuable nanotech could be (and also some lesser VCs who struggled to imagine it being worth a few billion). AFAICT, the best VCs rejected nanotech startups because nanotech was well over 5 years away. It's somewhat plausible that that kind of VC can be persuaded that nanotech now could be profitable with less than 10 years of effort. Those VCs could influence some other world leaders to think about contingency plans related to nanotech.
VCs have definitely been investing in various forms of nanotech, just not Drexlerian nanotech. They focused on much more specific aims like particular nanoparticles or structured thin films for particular purposes (nanowires and other forms, too, but less so). And those technically have had large benefits over the last few decades, while also catalyzing development of better design software and macro-scale production tools. Only once they work, we stop caring about whether they're nano. We stop advertising them that way. So for most people "nano" just becomes a buzzword of stuff that doesn't work, with no mentally available referent for "commercially successful nanotech." Ditto things like "smart materials" and "metamaterials."
Basically I think a big part of the problem is that the word nanotech has been so diluted from its roots by people with good but narrower goals wanting to look more ambitious and futuristic than they were, that we either need to find a way to reclaim it (in the minds of the target audience) or else acknowledge it's no longer a useful category word. My mental short handle for this is that, for similar reasons, we don't have a word for "centitech." Centitech is great: there's so many things you can do with objects that are between 1cm and 100cm in at least one dimension, including build humans, who can do everything else, even self-replicate! It's just not useful, as a word, for a class of technologies.
I have nowhere near the technical capability to have anything like a clear plan, and your response is basically what I expected. I was just curious. Seems like it could be another cheap "Who knows? Let's see what happens" thing to try, with little to lose when it doesn't help anyone with anything. Still, can we distinguish individuals in unlabeled recordings? Can we learn about meaning and grammar (or its equivalent) based in part on differences between languages and dialects?
At root my thought process amounted to: we have a technology that learns complex structures including languages from data without the benefit of the structural predispositions of human brains. If we could get a good enough corpus of data, it can also learn things other than human languages, and find approximate mappings to human languages. I assumed we wouldn't have such data in this case. That's as far as I got before I posted.
Edit to add: I think if any project like this succeeds, what you'll be teaching is not so much a language as a pidgin. Whether or not it can become a creole, whether we can learn to incorporate orca vocabulary and grammar to find a medium of communication both species can reproduce and interpret, those are much bigger questions and longer term projects.
Aside: Do we have enough recordings of orca calls to train an LLM on them?
I think the hypothesis of high orca intelligence is interesting and plausible. I am all in favor of low-cost low-risk long shot experiments to learn more about other forms of intelligence, whether they end up instrumentally useful to humans or not. Personally I'd be curious just to know whether this kind of attempt is even able to catch the interest of orcas, and if so, to what degree. Even if you just stuck to simple two and three word declarative sentences, it would be interesting to see if they intuit the categories of noun and verb, if they can learn the referent of a word from a picture or video without acoustic or olfactory or stereo visual data.
You mention languages that don't have articles, but keep in mind this is not rare. Most East Asian, Slavic, and Bantu languages don't; Latin and Sanskrit don't/didn't. Though, Mandarin (the only one of these I know even a little) does use demonstrative adjectives like "this one" and "that one," and makes extensive use of measure words for some of the same purposes as English uses articles. If it's not necessary among humans, it shouldn't be necessary for other mind types in general.
Another factor to consider is that some languages, like Mandarin, have no word modifications for case/number/gender/tense/perspective markers; these are done with extra words where necessarily and not at all otherwise. Probably a feature you want to have when trying to teach with few examples.
When looking at where to do this, consider that in many/most places, orcas are protected species with rules about approaching them and interfering with their natural behavior/environment.
Strong upvote - well laid out, clear explanation of your position and reasoning, I learned things.
Overall I think the lines of thought all make sense, but they seem to me to hinge entirely on your assigning a low probability to AI takeover scenarios, which you point out you have not modeled. I mean this in the sense that power concentration risks, as described, are only meaningful in scenarios where the power resides with the humans that create the AI, rather than the AI. Relatedly, the only way power concentration risks are lower in the non-centralization branch is if multiple projects yield AGI before any of them become particularly powerful, whereas this post assumes China would not be able to catch up to the hypothetical unified US project. I see the graphs showing a longer US lead time in the latter scenario, but I do not know if I agree the effect would be large enough to matter.
In other words, if instead you believed AI takeover scenarios were likely, or that the gap from human to superhuman level were small, then it wouldn't really matter how many projects there are that are close to AGI, only the quality of the one that gets there first. I don't want whoever-is-in-charge-at-the-DOD to be in control of the ultimate fate of humanity forever. I don't particularly want any private corporation to have that power either. I would, however, prefer that almost any human group be in such a position, than for humanity to unintentionally lose control of its future and be permanently disempowered or destroyed.
Of course, the terms AGI, human level, and superhuman level are abstractions and approximations anyway, I get that. I personally am not convinced there's much difference between human and superhuman, and think that by the time we get robust-human-quality-thinking, any AI will already by sufficiently superhuman in other areas that we'll be well past human level overall.
I'm someone who doesn't have children and doesn't plan to, but I agree with pretty much everything in this post. I do have reservations on how to go about justly raising someone's taxes now because of now-irreversible decisions they made not to have children in the past before the policy existed, but am all in favor of lump sums paid by future taxes, including higher taxes on me personally.
Is it sad I have higher expectations that society will solve the really hard technological problem of aging in the next couple of decades than the economically-should-be-straightforward problem of fertility gaps?
One thing I have struggled with is not having the cultural context to interpret or appreciate thinkers further in time and space from my own perspective. Who were they, what did they care about, why were they writing about these topics in particular, who were they writing for, what else was going on at the time, what concepts and metaphors and examples were or weren't readily accessible as starting points. Things like that. So when I started branching out more in reading philosophy, I also had to start reading more deep history. Which I enjoyed anyway, so I'm probably overlooking other good paths in favor of one I like, but it worked for me.
E.g. I first read Plato, Sun Tzu, and Musashi in middle and high school. I got a lot more out of Plato when I revisited after taking a single classics course in college. I got a lot more out of Sun Tzu after working for a few years with clients who needed to manage groups of people and large companies and coordinate diverse interests in pursuit of their goals. I got more out of Musashi after I started reading the Sequences and thought about what it meant to have a single goal that subsumes or outweighs all others.
How much impact, if any, would it have if more of our long and medium distance transmission lines eventually moved to HVDC lines?
I don't.
Every probability estimate I make is implicitly contingent on a whole host of things I can't fully list or don't bother to specify because it's not worth the overhead. This is one of them. Somewhere in my head, implicitly or explicitly, is a world model that includes the rest, including ones I'm not aware of. I do not know if the set of assumptions this implies is finite or not. I know false memories and hallucinations and tricks and so on exist, but unless I already have specific reason to expect such, I reason without keeping track. When I say P(Heads), I actually mean it as shorthand for P(Heads|all the model assumptions needed for "Heads" to make sense as a concept or event). When I find a reason to think one of my unstated model assumptions is wrong, I'm changing which set of conditional probabilities I'm even thinking about.
Over time, as I improve my world model and my level of understanding of my world model, I am more able to explicitly identify my underlying assumptions and choose to question them or question other things in light of them, when I deem it worth the effort, in order to get a little bit closer to a more general underlying probability distribution.
You mention microgrids, but what about energy storage systems? Geomagnetic storms typically last minutes ot hours, so if I have even a 4 hour battery in my home, with an inverter, I'm a lot less susceptible to grid failures. And if everyone does, the grid operators have more options for shutting down parts of the grid to prevent equipment failures without causing blackouts. If most large centralized generating facilities have energy storage, then temporarily shutting down parts of the grid does not need to involve much loss of energy production.
Good question. Curious to hear what the OP thinks, too.
Personally I'm not convinced that the results of a conscious process are actually "wants" in the sense the described here, until they become more deeply internalized. Like, obviously if I want ice cream it's partly because at some point I consciously chose to try it and (plausibly) wanted to try it. But I don't know that I can choose to want something, as opposed to putting myself in the position of choosing something with the hope or expectation that will come to want it.
The way I think about it, I can choose to try things, or do things. I can choose to want to want things, or want to like things. As I try and do the things I want to want and like, I may come to want them. I can use various techniques to make those subconscious changes faster, easier, or more likely. But I don't think I can choose to want things.
I do think this matters, because in the long run, choosing to not do or get the things you want, in favor of the things you consciously think you should want, or want to want, but don't, is not good for mental health.
Plus, butter is churned, so it is a few percent air by volume when solid.
Humans have been trying to parse out the hidden variables in the word "good" for millennia - plausibly since before the dawn of writing. We've made progress - a lot of it - but that doesn't stop the remaining problem from being very hard, because yes, the word is something like an abstraction of an approximation of a loose cloud of ideas. We can parse out and taboo variables 1-9, and come to agree on them, and still caught off guard by variable 184 when it interacts with variables 125 and 4437, each of which is irrelevant (or an unchanging background) to most people in most contexts to the point we've never bothered to consciously notice them.
You talked about thermometers. You're right, but consider that it took centuries to go from "We want to measure how hot or cold things are," to "here's the actual physical definition of temperature." Even still it's unintuitive, there's no single instrument that works to measure it in all cases, and it doesn't quite align with what every user of the word wants out of the concept. "Good" is a lot more complicated than "temperature."
In other words, yes, of course, let's keep doing this, more and better! But let's be honest about it being actually hard and complicated.
I find this intuitively reasonable and in alignment with my own perceptions. A pet peeve of mine has long been that people say "sentient" instead of "sapient" - at minimum since I first read The Color of Magic and really thought about the difference. We've never collectively had to consider minds that were more clearly sapient than sentient, and our word-categories aren't up to it.
I think it's going to be very difficult to disentangle the degree to which LLMs experience vs imitate the more felt aspects/definitions of consciousness. Not least because even humans sometimes get mixed up about this within ourselves.
In the right person, a gradual-onset mix of physicalized depression symptoms, anhedonia, and alexithymia can go a long way towards making you, in some sense, not fully conscious in a way that is invisible to almost everyone around you. Ask me how I know: There's a near decade-long period of my life where, when I reminisce about it, I sometimes say things like, "Yeah, I wish I'd been there."
Edit to add: I think almost every concept we use in life is part metaphor, part not, and the difference is one of degree and not kind. I was definitely surprised to learn this, or at least to learn how deep the rabbit hole goes.
Almost all human thinking is part metaphor.
Words have uses not meanings. Definitions are abstractions.
In otherwise, everything is (in part) a metaphor.
This is almost true. Fat is less dense than water, so a tablespoon of butter weighs something like 10% less than a half ounce. Not enough to matter in practice for most cooking. Your toast and your average chocolate chip cookie don't care. But, many approximations like this exist, and are collectively important enough that professionals use weight not volume in most recipes. And enough that the difference in fat content between butters (as low as 80% in the US but more often 85+% in European or otherwise "premium" butters) can matter in more sensitive recipes, like pie crust and drop biscuits. I used to add 1-2 Tbsp of shortening to my pie crust. I stopped when I switched to Kerrygold butter - no longer needed.
A classic example is that, at least in English, time is often described using distance metaphors
For me, I knew this was a metaphor, but until I took Mandarin in college I never realized that other languages/cultures/people used different spatial metaphors for time. Is the future in front of you, or behind? Are you moving towards it, or it towards you? This has some practical applications, since apparently even in English people have different intuitions about what it means to push a meeting or event up/out/back/ahead.
I think it would be more accurate to say we should have multiple levels of dangerous capability tests, which reflect different levels of increased danger.
For example, someone who has never pipetted before might struggle to measure microliters precisely or contaminate the tip when touching a bottle. Acquiring these skills often takes months of learning from experienced scientists — something terrorists can’t easily do.
This seems like a very thin line of defense. I first worked as an intern in a bio lab and learned how to pipette when I was in high school. Physical lack of a fume hood for sterile technique seems like a slightly bigger barrier, but even then, it's not all that hard to find people with radical beliefs who have science or engineering degrees, and many of those will be able to get access to basic lab facilities. How many people have ever worked in a wet lab for one month? In the proposed scenario, all of those people will have access to the knowledge and skills needed to make bioweapons within a year or two. How many of these people could be persuaded to teach a layperson the basic mechanical skills needed? Maybe pay them and claim they're teaching an introductory class for a job training program or something.
In other words:
you probably aren’t going to make perfect meringues the first time because everything about your kitchen — the humidity, the dimensions, and power of your oven, the exact timing of how long you whipped the egg whites — is a little bit different than the person who wrote the recipe
is true. But, it's not so reassuring if I want to make sure no one successfully makes meringues without permission, and they each can get at least a handful of attempts on average, and there are millions of people who've done other kinds of cooking before, and in addition to the recipe you're allowed to watch every video and read every article available for free online showing different aspects of the processes being demonstrated.
This is true. But ideally I don't think what we need is to be clever, except to the extent that it's a clever way to communicate with people so they understand why the current policies produce bad incentives and agree about changing them.
I think our collective HHS needs are less "clever policy ideas" and more "actively shoot ourselves in the foot slightly less often."
That's a good point about public discussions. It's not how I absorb information, but I can definitely see that.
I'm not sure where I'm proposing bureaucracy? The value is in making sure a conversation efficiently adds value for both parties, by not having to spend time rehashing things that are much faster absorbed in advance. This avoids the friction of needing to spend much of the time rehashing 101-level prerequisites. A very modest amount of groundwork beforehand maximizes the rate of insight in discussion.
I'm drawing in large part from personal experience. A significant part of my job is interviewing researchers, startup founders, investors, government officials, and assorted business people. Before I get on a call with these people, I look them (and their current and past employers, as needed) up on LinkedIn and Google Scholar and their own webpages. I briefly familiarize myself with what they've worked on and what they know and care about and how they think, as best I can anticipate, even if it's only for 15 minutes. And then when I get into a conversation, I adapt. I'm picking their brain to try and learn, so I try to adapt to their communication style and translate between their worldview and my own. If I go in with an idea of what questions I want answered, and those turn out to not be the important questions, or this turns out to be the wrong person to discuss it with, I change direction. Not doing this often leaves everyone involved frustrated at having wasted their time.
Also, should I be thinking of this as a debate? Because that's very different than a podcast or interview or discussion. These all have different goals. A podcast or interview is where I think the standard I am thinking of is most appropriate. If you want to have a deep discussion, it's insufficient, and you need to do more prep work or you'll never get into the meatiest parts of where you want to go. I do agree that if you're having a (public-facing) debate where the goal is to win, then sure, this is not strictly necessary. The history of e.g. "debates" in politics, or between creationists and biologists, shows that clearly. I'm not sure I'd consider that "meaningful" debate, though. Meaningful debates happen by seriously engaging with the other side's ideas, which requires understanding those ideas.
I can totally believe this. But, I also think that responsibly wearing the scientist hat entails prep work before engaging in a four hour public discussion with a domain expert in a field. At minimum that includes skimming the titles and ideally the abstracts/outlines of their key writings. Maybe ask Claude to summarize the highlights for you. If he'd done that he'd have figured out many of the answers to many of these questions on his own, or much faster during discussion. He's too smart not to.
Otherwise, you're not actually ready to have a meaningful scientific discussion with that person on that topic.
If I'm understanding you correctly, then I strongly disagree about what ethics and meta-ethics are for, as well as what "individual selfishness" means. The questions I care about flow from "What do I care about, and why?" and "How much do I think others should or will care about these things, and why?" Moral realism and amoral nihilism are far from the only options, and neither are ones I'm interested in accepting.
I'm not saying it improves decision making. I'm saying it's an argument for improving our decision making in general, if mundane decisions we wouldn't normally think are all that important have much larger and long-lasting consequences. Each mundane decision affects a large number of lives that parts of me will experience, in addition to the effects on others.