Posts
Comments
This was a great post, really appreciate the summary and analysis! And yeah, no one should have high certainty about nutritional questions this complicated.
For myself, I mostly eliminated these oils from my diet about 4 years ago, along with reducing industrially-processed food in general. Not 100%, I'm not a purist, but other than some occasional sunflower oil none of these are in foods I keep at home, and I only eat anywhere else 0-2 times per week. I did lose a little weight in the beginning, maybe 10 lbs, but then stabilized. But what I have mostly noticed is that when I eat lots of fried food, depending on the oil used (which to some degree you can taste), I'm either fine, or feel exhausted/congested/thirsty for basically a full day. I think you may have a point about trans fata from reusing oil, since anecdotally this seems even worse for leftovers.
Of course, another thing I did at the same time is switch to grass-fed butter and pasture-raised eggs. Organic meats and milk, not always pasture raised. Conventional cheeses. I've read things claiming the fatty acid composition is significantly different for these foods depending on what the animals eat, in terms of omega 3/6 ratios, saturated/unsaturated fat ratios, and fatty acid chain lengths. I've never looked too deeply into checking those claims, because for me the fact that they taste better is reason enough. As far as I can tell, it wasn't until WWII or later that we really started feeding cows corn and raising chickens in dense indoor cages with feed? Yet another variable/potential confounder for studies.
In what sense are these two viewpoints in tension?
This seems more a question of "observable by whom" vs "observable in principle."
This works too, yeah.
Yeah, I was thinking it's hard to beat dried salted meat, hard cheese, and oil or butter.
You also don't have to assume that all the food travels the whole way. If (hypothetically) you want to send 1 soldier's worth of food and water 7 days away, and each person can only carry 3 days worth at a time, then you can try to have 3 days worth deposited 6 days out, and then have a porter make a 2 day round trip carrying 1 day's worth to leave for that soldier to pickup up on day 7. Then someone needs to have carried that 3 days worth to 6 days out, which you can do by having more porters make 1 day round trips from 5 days out, etc. Basically it you need exponentially more people and supplies the farther out your supply chains stretch. I think I first read about this in the context of the Incas, because potatoes are less calorie dense per pound than dried grains so it's an even bigger problem? Being able to get water along the way, and ideally to pillage the enemy's supplies, are also a very big deal.
I think at that point the limiting factors become the logistics of food, waste, water, and waste heat. In Age of Em Robin Hanson spends time talking about fractal plumbing systems and the like, for this kind of reason.
All good points, many I agree with. If nothing else, I think that humanity should pre-commit to following this strategy whenever we find ourselves in the strong position. It's the right choice ethically, and may also be protective against some potentially hostile outside forces.
However, I don't think the acausal trade case is strong enough that I would expect all sufficiently powerful civilizations to have adopted it. If I imagine two powerful civilizations with roughly identical starting points, one of which expanded while being willing to pay costs to accommodate weaker allies while the other did not and instead seized whatever they could, then it is not clear to me who wins when they meet. If I imagine a process by which a civilization becomes strong enough to travel the stars and destroy humanity, it's not clear to me that this requires it to have the kinds of minds that will deeply accept this reasoning.
It might even be that the Fermi paradox makes the case stronger - if sapient life is rare, then the costs paid by the strong to cooperate are low, and it's easier to hold to such a strategy/ideal.
This seems to completely ignore transaction costs for forming and maintaining an alliance? Differences in the costs to create and sustain different types of alliance-members? Differences in the potential to replace some types of alliance-members with other or new types? There can be entities for whom forming an alliance that contains humanity will cause them to incur greater costs than humanity's membership can ever repay.
Also, I agree that in a wide range of contexts this strategy is great for the weak and for the only-locally-strong. But if any entity knows it is strong in a universal or cosmic sense, this would no longer apply to it. Plus everyone less strong would also know this, and anyone who truly believed they were this strong would act as though this no longer applied to them either. I feel like there's a problem here akin to the unexpected hanging paradox that I'm not sure how to resolve except by denying the validity of the argument.
On screen space:
When, if ever, should I expect actually-useful smart glasses or other tech to give me access to arbitrarily-large high-res virtual displays without needing to take up a lot of physical space, or prevent me from sitting somewhere other than a single, fixed desk?
On both the Three Body Problem and economic history: It really is remarkably difficult to get people to see that 1) Humans are horrible, and used to be more horrible, 2) Everything is broken, and used to be much more broken, and 3) Actual humans doing actual physical things have made everything much better on net, and in the long run "on net" is usually what matters.
On the Paul Ehrlich organization: Even if someone agrees with these ideas, do they not worry what this makes kids feel about themselves? Like, I can just see it: "But I'm the youngest of 3! My parents are horrible and I'm the worst of all!"
And this, like shame-based cultural norm enforcement, disproportionately punishes those who care enough to want to be pro-social and conscientious, with extra suffering.
Got it, makes sense, agreed.
I agree that filling a context window with worked sudoku examples wouldn't help for solving hidouku. But, there is a common element here to the games. Both look like math, but aren't about numbers except that there's an ordered sequence. The sequence of items could just as easily be an alphabetically ordered set of words. Both are much more about geometry, or topology, or graph theory, for how a set of points is connected. I would not be surprised to learn that there is a set of tokens, containing no examples of either game, combined with a checker (like your link has) that points out when a mistake has been made, that enables solving a wide range of similar games.
I think one of the things humans do better than current LLMs is that, as we learn a new task, we vary what counts as a token and how we nest tokens. How do we chunk things? In sudoku, each box is a chunk, each row and column are a chunk, the board is a chunk, "sudoku" is a chunk, "checking an answer" is a chunk, "playing a game" is a chunk, and there are probably lots of others I'm ignoring. I don't think just prompting an LLM with the full text of "How to solve it" in its context window would get us to a solution, but at some level I do think it's possible to make explicit, in words and diagrams, what it is humans do to solve things, in a way legible to it. I think it largely resembles repeatedly telescoping in and out, to lower and higher abstractions applying different concepts and contexts, locally sanity checking ourselves, correcting locally obvious insanity, and continuing until we hit some sort of reflective consistency. Different humans have different limits on what contexts they can successfully do this in.
Oh, by "as qualitatively smart as humans" I meant "as qualitatively smart as the best human experts".
I think that is more comparable to saying "as smart as humanity." No individual human is as smart as humanity in general.
inside the fuzz
This is an excellent short mental handle for this concept. I'll definitely be using it.
I was going to say the same. I can't count the number of times a human customer service agent has tried to do something for me, or told me they already did do something for me, only for me to later find out they were wrong (because of a mistake they made), lying (because their scripts required it or their metrics essentially forced them into it), or foiled (because of badly designed backend systems opaque to both of us).
Here's a simple test: Ask an AI to open and manage a local pizza restaurant, buying kitchen equipment, dealing with contractors, selecting recipes, hiring human employees to serve or clean, registering the business, handling inspections, paying taxes, etc. None of these are expert-level skills. But frontier models are missing several key abilities. So I do not consider them AGI.
I agree that this is a thing current AI systems don't/can't do, and that aren't considered expert-level skills for humans. I disagree that this is a simple test, or the kind of thing a typical human can do without lots of feedback, failures, or assistance. Many very smart humans fail at some or all of these tasks. They give up on starting a business, mess up their taxes, have a hard time navigating bureaucratic red tape, and don't ever learn to cook. I agree that if an AI could do these things it would be much harder to argue against it being AGI, but it's important to remember that many healthy, intelligent, adult humans can't, at least not reliably. Also, remember that most restaurants fail within a couple of years even after making it through all these hoops. The rate is very high even for experienced restauranteurs doing the managing.
I suppose you could argue for a definition of general intelligence that excludes a substantial fraction of humans, but for many reasons I wouldn't recommend it.
Chris' latest reply to my other comment resolved a confusion I had, so I now realize my comment above isn't actually talking about the same thing as you.
I'm definitely one of those non-experts who has never done actual machine learning, but AFAICT that article you linked both is tied to and does not explicitly mentioned that the 'principle of indifference' is about the epistemological taste of the reasoner, while arguing that the cases where the reasoner lacks knowledge to hold a more accurate prior means the principle itself is wrong.
The training of an LLM is not a random process, therefore indifference will not accurately predict the outcome of this process. This does not imply anything about other forms of AI, or about whether people reasoning in the absence of knowledge about the training process were making a mistake. It also does not imply sufficient control over the outcome of the training process to ensure that the LLM will, in general, want to do what we want it to want to do, let alone to do what we want it to do.
The section where she talks about how evolution's goals are human abstractions and an LLM's training has a well-specified goal in terms of gradient descent is really where that argument loses me, though. In both cases, it's still not well specified, a priori, how the well-defined processes cash out in terms of real-world behavior. The factual claims are true enough, sure. But the thing an LLM is trained to do is predict what comes next, based on training data curated by humans, and humans do scheme. Therefore, a sufficiently powerful LLM should, by default, know how to scheme, and we should assume there are prompts out there in prompt-space that will call forth that capability. No counting argument needed. In fact, the article specifically calls this out, saying the training process is "producing systems that behave the right way in all scenarios they are likely to encounter," which means the behavior is unspecified in whatever scenarios the training process deems "unlikely," although I'm unclear what "unlikely" even means here or how it's defined.
One of the things we want from our training process is to not have scheming behavior get called up in a hard-to-define-in-advance set of likely and unlikely cases. In that sense, inner-alignment may not be a thing for the structure of LLMs, in that the LLM will automatically want what it is trained to want. But, it is still the case that we don't know how to do outer-alignment for a sufficiently general set of likely scenarios, aka we don't actually know precisely what behavioral responses our training process is instilling.
Yes, true, and often probably better than we would be able to write it down, too.
I was under the impression that this meant that a sufficiently powerful AI would be outer-aligned by default, and that this is what enables several of the kinds of deceptions and other dangers we're worried about.
Is the difference between the goal being specified by humans vs being learned and assumed by the AI itself?
My mental shorthand for this has been that outer alignment is getting the AI to know what we want it to do, and inner alignment is getting it to care. Like the difference between knowing how to pass a math test, and wanting to become a mathematician. Is that understanding different from what you're describing here?
This is an interesting thought I hadn't come across before. Very much Sun Tzu, leave-your-enemy-a-path-of-retreat. As you said, its efficacy would depend very much on the nature of the AI in question. Do you think we'll be able to determine which AIs are worthwhile to preserve, and which will just be more x-risk that way?
If we are lucky enough to find ourselves in one of in those situations you describe, where we have an AGI that wants to do what humanity wants (or would/should want) it to want to do, then to what degree is additional training actually required? I'm sure there are many possible such scenarios that vary widely, but my mental default assumption is that such a system would already be motivated to seek out answers to questions such as, "What is it the humans want me to want, and why?" which would naturally include studying... well, pretty much the entire moral philosophy literature. I wouldn't put super high odds on it, though.
That said, one of my main concern about a classical virtue ethics training regimen specifically is that it doesn't really give a clear answer about how to prioritize among virtues (or more broadly, things some subset of cultures says are virtues) that are in conflict, and real humans do in fact disagree and debate about this all the time.
That seems entirely plausible to me.
I'd add that it might be much easier to make long-distance trucks self-driving than local distribution in many cases. My dad was a wholesaler to restaurants, I rode in the trucks to make deliveries sometimes, and I can tell you that making a self-driving truck work bringing food to pizzerias in NYC requires a willingness to double park and get tickets, and human-level maneuvering to carry stuff into weirdly shaped storage rooms. So even when we do have automated trucks, there will likely be humans carrying stuff on and off them for a while.
I was thinking about Musk's self driving vs AGI timeline predictions, wondering if there wasn;t a way they might not be as in conflict as it seems. Like, maybe Musk believes AI will happen in such a way that large-scale retooling of the vehicle fleet and sales of new vehicles will still take years. Then it occurred to me that the "easiest" (and quickest and cheapest) way to build myself a self-driving car in Musk's 2029 could be for me to buy smart glasses, and exoskeleton arms and legs with actuators, then connect them to an AGI via my my phone and let it move my limbs around.
What do you mean by "agreement"? Endorsement of a shared verbal description of some assertion about the topic? Complete agreement about the probability distribution among hypotheses? The wording here suggests the latter. For flawless reasoners agreement should always be possible in principle, but may require the sharing of literally every piece of information available, including everything they've ever experienced and the complete structure of their brains, which is not something humans are physically capable of sharing (let alone processing).
In other words, when you say conversations can be long, are you thinking about how they can very easily become millennia-per-discussion-point complicated if you really seek complete agreement? If that's your goal in reasoning and your criteria for success, then yes, you will usually fail. It is not my goal.
Also, to the title question, reasoning does not require a conversation partner at all. Having one can make things easier, or sometimes harder, but not having one does not learning or reasoning.
Yeah, you're right, but for most of history they were net population sinks that generated outsized investment returns. Today they're not population sinks because of sanitation etc. etc.
I know I'm being imprecise and handwavy, so feel free to ignore me, but really my thought was just that lots of things look vaguely like ponzi schemes without getting into more details than most people are going to pay attention to.
From the outside, depending on your level of detail of understanding, any franchise could look that way. Avon and Tupperware look a bit that way. Some MLM companies are more legitimate than others.
From a more abstract point of view, I could argue that "cities" are an example. "Hey, send your kids to live here, let some king and his warriors be in charge, and give up your independence, and you'll all get richer!" It wasn't at all clear in the beginning how "Pay taxes and die of diseases!" was going to be good for anyone but the rulers, but the societies that did it more and better thrived and won.
To the latter: my point is that except to the extent we're resource constrained, I'm not sure why anyone (and I'm not saying you are necessarily) would argue against any safe line of research even if they thought it was unlikely to work.
To the former: I think one of the things we can usefully bestow on future researchers (in any field) is a pile of lines of inquiry, including ones that failed and ones we realized we couldn't properly investigate yet, and ones where we made even a tiny bit of headway.
I'd say their value is instrumental, not terminal. The sun and stars are beautiful, but only when there are minds to appreciate them. They make everything else of value possible, because of their light and heat and production of all the various elements beyond Helium.
But a dead universe full of stars, and a sun surrounded by lifeless planets, have no value as far as I'm concerned, except insofar as there is remaining potential for new life to arise that would itself have value. If you gave me a choice between a permanently dead universe of infinite extent, full of stars, or a single planet full of life (of a form I'm capable of finding value in, so a planet full of only bacteria doesn't cut it) but surrounded by a bland and starless sky that only survives by artificial light and heat production (assume they've mastered controlled fusion and indoor agriculture), I'd say the latter is more valuable.
Unfortunately in the world I live in, the same people who would accept "This is obviously a Ponzi scheme" (but who don't understand AI x-risk well) have to also contend with the fact that most people they hear talking about AI are indistinguishable (to them) from people talking about crypto as an investment, or about how transformative AI will lead to GDP doubling times dropping to years, months, or weeks. So, the same argument could be used to get (some of) them to dismiss the notion that AI could become that powerful at all with even less seeming-weirdness.
Arguments that something has the form of a Ponzi scheme are, fortunately and unfortunately, not always correct. Some changes really do enable permanently (at least on the timescales the person thinks of as permanent) faster growth.
Yes on the overall gist, and I feel like most of the rest of the post is trying to define the word "things" more precisely. The Spokesperson things "past annual returns of a specific investment opportunity" are a "thing." The Scientist thinks this is not unreasonable, but that "extrapolations from established physical theories I'm familiar with" are more of a "thing." The Epistemologist says only the most basic low-level facts we have, taken as a whole set, are a "thing" and we would ideally reason from all of them without drawing these other boundaries with too sharp and rigid a line. Or at least, that in places where we disagree about the nature of the "things," that's the direction in which we should move to settle the disagreement.
I often think about this as "it's hard to compete with future AI researchers on moving beyond this early regime". (That said, we should of course have some research bets for what to do if control doesn't work for the weakest AIs which are very useful.)
I see this kind of argument a lot, but to my thinking, the next iteration of AI researchers will only have the tools today's researchers build for them. You're not trying to compete with them. You're trying to empower them. The industrial revolution wouldn't have involved much faster growth rates if James Watt (and his peers) had been born a century later. They would have just gotten a later start at figuring out how to build steam engines that worked well. (Or at least, growth rates may have been faster for various reasons, but at no single point would the state of steam engines in that counterfactual world be farther along than it was historically in ours).
(I hesitate to even write this next bit for fear of hijacking in a direction I don't want to go, and I'd put it in a spoiler tag if I knew how. But, I think it's the same form of disagreement I see in discussions of whether we can 'have free will' in a deterministic world, which to my viewpoint hinges on whether the future state can be predicted without going through the process itself.)
Who are these future AI researchers, and how did they get here and get better if not by the efforts of today's AI researchers? And in a world where Sam Altman is asking for $7 trillion and not being immediately and universally ridiculed, are we so resource constrained that putting more effort into whatever alignment research we can try today is actually net-negative?
Ok, then that I understand. I do not think it follows that I should be indifferent between those two ways of me dying. Both are bad, but only one of them necessarily destroys everything I value.
In any case I think it's much more likely a group using an aligned-as-defined-here AGI to kill (almost) everyone by accident, rather than intentionally.
Before anything else, I would note that your proposed scenario has winners. In other words, it's a horrible outcome, but in this world where alignment as you've defined it is easy but a group of humans uses it to destroy most of the population, that group of humans likely survives, repopulates the Earth, and continues on into the future.
This, to put it mildly, is a far, far better outcome than we'd get from an AI that wants to kill everyone of its own agency, or that doesn't care about us enough to avoid doing so as a side effect of other actions.
I don't remember where or when, but IIRC EY once wrote that if an evil person or group used ASI to take over the world, his reaction would be to 'weep for joy and declare victory' that any human-level agents whatsoever continue to exist and retain enough control to be able to be said to "use" the AI at all.
That said, yes, if we do figure out how to make an AI that actually does what humans want it to do, or CEV-would-want it to do, when they ask it to do something, then preventing misuse becomes the next major problem we need to solve, or ideally to have solved before building such an AI. And it's not an easy one.
- @justinpombrio is right that Bayesian updates move probability estimates between hypotheses, not towards or away from specific hypotheses.
- Yes, clearly, we made a mistake in our hypotheses about this experiment. Professor, you believe the mistake was in how you all applied Newtonian mechanics to the experimental system. Students, you believe that Newtonian mechanics is in some way incorrect.
Each of you, go and make a list of all the assumptions you made in setting up and carrying out the experiment, along with your priors for the likelihood of each.
Compare lists, make sure you're in agreement on what the set of potential failure points is for where you went wrong.
Then figure out how to update your probabilities based on this result, and what experiment to perform next.
(Basically, the students' world models are too narrow to even realize how many things they're assuming away, and then pointing a figure at the only thing they knew to think of as a variable. And the professor (and all past science teachers they've had) failed to articulate what it was they were actually teaching and why).
I think you are looking at the wrong conditional probability. You're saying "Most virtual minds will not be misled into no knowing they're in a simulation." I have no objection to that. You then conclude that we, specifically, are probably not in a simulation, because we don't think we are. But this ignores the question of relative population sizes.
Suppose from the origin of humans to the time when ancestor simulations become cheap enough to consider, that there are a total of ~1e12 humans. Suppose that this civilization, over the subsequent several millennia, creates 1e18 virtual minds, of which only 1 in 10k are simulations as defined here. Then, 99% of all human minds that believe they are living before the advent of ancestor simulations, are wrong and being misled.
Obviously these numbers aren't coming from anywhere, they're just to show that the quantities you're estimating don't map to the conclusion you're trying to draw, if the future population is sufficiently large in total.
Fundamentally I agree, and I think it sounds like we both agree with Spock. Christianity tries to get around this by distinguishing between timeless/eternal and within-time/everlasting viewpoints, among other approaches, but I think very much fails to make a good case. I do think there are a few plausible counterarguments here, none of which are standard AFAIK.
One is Scott Alexander's Answer to Job, basically that we're mistaken to think this is the best possible world (assuming "world" means "Earth"), because God actually created all possible net-good universes, and (due to something like entropy) most of those are going to be just-barely-net-good. That post combines it with a discussion of what the words "create" and "exist" actually might mean, in terms of identity, value, quantity, simulation, computation and how to sum utilities.
Another is dkirmani's answer below, that for some functions and initial conditions there might not be a well-defined analytical solution to the problem of future-prediction, only a computational solution, such that even God has to simulate the whole process to do the prediction or the goodness-summation (which might be equivalent to creating minds and experiences and worlds). This one is also a plausible solution to the question of why God would create anything at all.
Yeah, there are lots of ways to be useful, and not all require any superhuman capabilities. How much is broadly-effective intelligence vs targeted capabilities development (seems like more the former lately), how much is cheap-but-good-enough compared to humans vs better-than-human along some axis, etc.
Fair enough, "trivial" overstates the case. I do think it is overwhelmingly likely.
That said, I'm not sure how much we actually disagree on this? I was mostly trying to highlight the gap between an AI have a capability and us having the control to use an AI to usefully benefit from that capability.
Trivially, any AI smart enough to be truly dangerous is capable of doing our "alignment homework" for us, in the sense of having enough intelligence to solve the problem. This is something EY has also pointed out many times, but which often gets ignored. Any ASI that destroys humanity will have no problem whatsoever understanding that that's not what humanity wanted, and no difficulty figuring out what things we would have wanted it to do instead.
What is very different and less clear of a claim is whether we can use any AI developed with sufficient capabilities, but built before the "homework" was done, to do so safely (for likely/plausible definitions of "we" and "use").
It seems totally reasonable to say that AI is rapidly getting many very large advantages with respect to humans, so if it gets to ‘roughly human’ in the core intelligence module, whatever you want to call that, then suddenly things get out of hand fast, potentially the ‘fast takeoff’ level of fast even if you see more signs and portents first.
In retrospect, there had been omens and portents.
But if you use them as reasons to ignore the speed of things happening, they won't help you.
Lots of thoughts here. One is that over the course of our lives we encounter so many stories that they need to have variety, and Tolstoy's point makes pure heroes less appealing: "Happy families are all alike; every unhappy family is unhappy in its own way." Heroes and conflicts in children's stories are simple, and get more complex in stories for teens and adults. This is not just about maturity and exposure to the complexities of life and wanting to grapple with real dilemmas, it's also about not reading the hundredth identical plot.
Milton's Lucifer was also my first thought reading this, but I'm not sure I agree with your take. I think the point, for me, is that he makes us question whether is actually is the villain, at all. The persuasion element is, I think, an artifact of the story being told in a cultural context where there's an overwhelming presumption that he is the villain. The ancient Greeks had a different context, and had no problem writing complex and flawed and often doomed heroes who fought against fate and gods without their writers/composers ever thinking they needed to classify them as villains or anti-heroes, just larger-than-life people.
Perhaps it's just my American upbringing, but I think I want to live in a world where agents can get what they want, even with the world set against them, if only they are clever and persistent enough.
I'm American too, and I don't want that. At least not in general. I do share the stereotypical American distrust of rigid traditions and institutions to a substantial degree. I want agents-in-general to get much of what they want, but when the world is set against them? Depends case-by-case on why the world is set against them, and on why the agents have the goals they have. Voldemort was a persistent and clever (by Harry Potter standards) agent as much as Gandhi was. I can understand how each arrived at their goals and methods, and why their respective worlds were set against them, but that doesn't mean I want them both to get what they want. Interpolate between extremes however you like.
I know opinions about these kinds questions differ widely, and I think you shouldn't take too much advice from people who don't know anything about you. Regardless, I think the answers depend a lot on what set of options is or seems available to you.
For the first set of questions, do any of the options you'd consider seem likely to change the answer of "how many years?" If not, I would probably not use that as a deciding factor. You're building a life for yourself, it's unlikely the things you value only have value in the future and not the present, and there's enough probability that the answer is "at least decades" to make accounting for long timelines in your plans worthwhile.
For the second, this is harder and depends a lot on where you are, where you can easily go, where you have personal or family ties, how much money and other resources you have available, and how exposed you currently are to different kinds of economic and geopolitical changes.
As for personal anecdotes: none of the career options I considered had to do with AI, so I've treated the first set of questions as irrelevant to my own career path. I do understand that AI going well is extremely high-importance and high-variance, but I'm still focusing on the much lower-variance problem of climate change (and relatedly, energy, food, water, transportation, etc.). Sure, it won't make a difference if humanity goes extinct in 2035, but neither would any other path I took. I've also had the luxury of mostly being able to ignore the second set of questions, but FWIW I work fully remote and travel full time, which has the side effects of preserving optionality and of teaching me how to be transplantable and not get tied to hard-to-transport stuff.
The actual WSJ article centers on companies not sure they want to pay $30/month per user for Microsoft Copilot.
I understand that this is a thing, but I find it hard to imagine there are that many people making significant use of Windows and Microsoft Office at work who wouldn't be able to save an hour or two a month using Copilot or it's near-term successors. For me the break-even point would be saving somewhere between 5-30 minutes a month depending on how I calculate the cost and value of my work time.
On the protest acceptability: whenever I read about these polls I have no idea how much I'm supposed to think about the actual question. Personally I find it very easy to imagine fliers someone could hand out, and audiences they could hand them to, that I would find unacceptable but that both causes I support and those I oppose might decide are great ideas. "Always" to me means "probability ~1" but maybe that is too high a thresholdfor the intended question.
On binging: shows have gotten more complex since the advent of DVR and then streaming platforms. Yes, some space is better than binging, but also a lot of what's actually good is going to demand a lot of memory from me until I reach a natural breakpoint where there's not so many loose ends and intertwined plot points. Sometimes a week is fine. Other times even a day might leave me having to go rewatch bits of previous episodes.
Yes, agreed, and just to be clear I'm not talking about delays in granting a patent, I'm talking about delays in how long it takes to bring a technology to market and generate a return once a patent has been filed and/or granted.
Also, I'm not actually sure I'm 100% behind extending patent terms. I probably am. I do think they should be longer than copyright terms, though.
I think there could be a lot of value in having a sequence of posts on, basically, "What is this 'science' thing anyway?" Right now all the core ideas (including various prerequisites and corollaries) exist on this site or in the Sequences, but not a single, clear, cohesive whole that isn't extremely long.
However, I think trying to frame it this way, in one post, doesn't work. It's unclear who the target audience it, how they should approach it, and what they should hope to get out of it. Even knowing and already understanding these points, I read it wondering, "Why are these here, together, in this order? What is implied by the point numbering? Who, not already knowing these, would be willing to read this and able to understand it?"
It looks like the author created this account only a day before posting this. IDK if they've been here lurking or using another account a long time before that or not. In any case, my suggestion would be to look at how the Sequences are structured, find the bits that tie into what you're writing here, and then refactor this post into a series. Try and make it present ideas in a cohesive order, in digestible chunks, with links to past posts by others that expand on important points in more detail or other styles.
I agree with pretty much all of this. If anything I think it somewhat understates the case. We're going to need a lot more clean power than current electricity demand suggests if and when we make a real effort to reduce fossil fuel consumption in chemical production and transportation, and the latter will necessitate building a whole lot of some form of energy storage whether or not batteries get much cheaper.
Is the slow forward march of copyright terms the optimal response to the massive changes in information technology we’ve seen over the past several centuries?
Of course not! Even without the detailed analysis and assorted variables needed to figure out anything like an optimal term, economics tells us this can't actually help much in increasing idea, tech, patent, or creative work output. Market size for copies of a given work changes over time (speed and direction vary), but to a first approximation assume you can get away with holding it steady. Apply a 7% discount rate to the value of future returns and by year 20 you've gotten roughly 75% of all the value you'll ever extract. By year 50 you're over 95%.
Even without copyright, Disney must still own the trademark rights to use Mickey in many ways, so that part is really about copyright. Honestly outside of an occasional TV special I can't remember the last time there was an actual new creative work about Mickey at all. Who can claim that copyright was still incentivizing anything at all? What court would believe it?
Patents are trickier, because they start at date of filing, and in some industries it can take most of the patent term just to bring it to market, leaving only a few years to benefit from the protection. Something as simple as a lawsuit from a competitor, or any form of opposition to building a plant, or a hiccup in raising funds, could create enough delay to wipe out a significant chunk of a patent's value, in a way that wasn't really true in a century ago. It makes little sense to me to have the same patent durations across software, automotive, energy, aviation, and pharmaceutical inventions/discoveries.
Yes, agreed, that is one of the points of disagreement about free will. I find it more strange to think the future is more steerable in a world where you can't predict the outcomes of actions even in principle.
In the case of steering the future and AI, the thing in question is more about who is doing the steering, less about the gears-level question of how steering works as a concept. It's similar to how a starving man cares more about getting a loaf of bread than he does about getting a lesson on the biochemistry of fermentation. Whether humans or AIs or aliens decide the direction of the future, they all do so from within the same universal laws and mechanisms. Free will isn't a point of difference among options, and it isn't a lever anyone can pull that affects what needs to be done.
I am also happy to concede, that yes, creating an unfriendly AI that kills all humans is a form of steering the future. Right off a cliff, one time. That's very different than steering in a direction I want to steer (or be steered) in. It's also very different from retaining the ability to continue to steer and course correct.
Indeterminism is, tautologously, freedom from determinism.
Yes, and determinism isn't the thing I want freedom from. External control is, mostly.
Why would it be a deterministic fact in an indeterministic world?
The "may" is important there, and I intended it to be a probabilistic may, not a permission-granting may. It is a deterministic fact that it might be invoked, not that it necessarily will.