Posts
Comments
Yeah, or even just not also on disability.
https://cdrnys.org/blog/disability-dialogue/the-disability-dialogue-marriage-equality/ discusses some of the issues around here at the time it was written, if you're curious.
Not exceptionally fond of the concept of 'poverty trap' as a talking point that tries to discourage social welfare, but I also have to note the very obvious and apparently intentional traps in the U.S. at least around - specifically - long-term disability once that is necessary for self-sustenance; including attempting substantial gainful activity on disability; marrying someone while on disability; accepting gifts of any sort while on disability; and trying to save money on disability. Some of the specifics have thankfully improved, but there's just a bizarre number of gotchas that do aggressively penalize in some way most improvements in life situation, apparently as fallout from means testing.
(Oh, and you potentially qualify for sub-minimum wage jobs if you have a disability which impairs your ability to do that specific job, which ... well, I'm not sure how this changes the equilibrium; it gives options and also makes you more exploitable if the wage decrease is more than the impairment.)
Generally the hypothesis is that most people will get more sodium in their diet than they crave with their natural desire, if they just eat the food of least resistance (cheapest or easiest, most shelf stable, whatnot). A lot of the sodium that gets into your diet is not so richly activating your taste buds as table salt applied to taste.
What we want overall with salinity is to preserve it at a level that's correct for us, because we take it in through our diet and excrete it through various processes like sweat. Excessive salt consumption doesn't directly affect your overall salt and water balance that much, because the body has hormonal regulation of various mechanisms to keep it stable - it's presumably the overworking of these mechanisms that causes health issues, which is much preferable than it causing issues directly if you've seen the effects of the wrong salinity on cells in a petri dish under a microscope.
(The effects on whatever cells I was looking at, which started at a neutral salinity: Raising the salinity (saltier) caused them to shrivel up and dessicate like raisins; lowering the salinity (less salty) caused them to explode.)
Yeah, it'd be helpful to know what heavy lifting is going on there, because I feel like there's a pretty strong distinction between 'frozen burger patties that are otherwise indistinguishable from unfrozen burger patties' and 'TV dinner'.
Thanks for the reference! I'm definitely confused about the inclusion of "pre-prepared (packaged) meat, fish and vegetables" on the last list, though. Does cooking meat or vegetables before freezing it (rather than after? I presume most people aren't eating meat raw) actually change its processed status significantly?
Suppose my intuition is that the 'conscious experience' of 'an iPhone' varies based on what software is running on it. If it could run a thorough emulation of an ant and have its sensory inputs channeled to that emulation, it would be more likely to have conscious experience in a meaningful-to-me way than if nobody bothered (presuming ants do implement at least a trivial conscious experience).
(I guess that there's not necessarily something that it's like to be an iPhone, by default, but the hardware complexity could theoretically support an iAnt, which there is it is something that it's like to be?)
That certainly seems distinct from brain mass, though (except that it takes a certain amount to implement in the first place). I'd expect similar variation in feeling pain by becoming different neurologies of human; I know there are many reported variations in perception of felt pain inside our species already.
But that's in the limit. A function of electron = 0, ant = 1, cockroach = 4, mouse = 300 fits it just as well as electron = 0, ant = 1, cockroach = 2, mouse = 2^75, as does electron = 0, ant = 100, cockroach = 150, mouse = 200.
"Moral weights depend on intensity of conscient experience." - Just going to note that I've no particular reason to concede this point at the moment, so don't directly consider the next question a question of moral weight; I'd rather disassociate it first:
Is there ... any particular reason to expect intensity of conscious experience to grow 'super-additively', such that a tiny conscious mind experiences 1 intensity units, but a mind ten times as large experiences (since you reject linear, we'll step up to the exponential) 1024 intensity units? Given our general inability to exist as every mass of brain, what makes this more intuitive than no, marginal, or linear increase in intensity?
Personally, I would be actively surprised to spend time as a lower-brain-mass conscious animal and report that my experiences were (exceptionally) less intense. Why do our intuitions differ on this?
Yes, but also that there might not actually be a specific new thing, a detrimental thing, to gesture at.
If root causes of obesity existed all along, and changes in the modern Western diet revealed the potential for obesity in our region rather than actively causing it, looking for root causes specifically in things that have changed may not work out if the things that have changed are not the root causes.
(I.e., it's a seemingly useful constraint on looking at the solution space, that might not be true -- and not so useful a constraint if it isn't.)
You don't actually have to do any adjustments to the downsides, for beneficial statistical stories to be true. One point I was getting at, specifically, is that it is better than being dead or suffering in specific alternative ways, also. There can be real and clear downsides to carrying around significant amounts of weight, especially depending what that weight is, and still have that be present in the data in the first place because of good reasons.
I'll invoke the 'plane that comes back riddled in bullet holes, so you're trying to armor where the bullet holes are' meme. The plane that came back still came back; it armored the worst places, and now its other struggles are visible. It's not a negative trend, that we have more planes with damage now, than we did when they didn't come back.
I do think it's relevant that the U.S. once struggled with nutritional deficiencies with corn, answered with enriched and fortified products that helped address those, and likely still retains some of the root issues (that our food indeed isn't as nutritious as it should be, outside those enrichments). That the Great Depression happened at all; and the Dust Bowl. There's questions here not just of personal health, but of history; and when I look at some of the counterfactuals, given available resources, I see general trade-offs that can't be ignored when looking at - specifically - the statistics.
Raw spinach in particular also has high levels of oxalic acid, which can interfere with the absorption of other nutrients, and cause kidney stones when binding with calcium. Processing it by cooking can reduce its concentration and impact significantly without reducing other nutrients in the spinach as much.
Grinding and blending foods is itself processing. I don't know what impact it has on nutrition, but mechanically speaking, you can imagine digestion proceeding differently depending on how much of it has already been done.
You do need a certain amount of macronutrients each day, and some from fat. You also don't necessarily want to overindulge on every micronutrient. If we're putting a number of olives in our salad equivalent to the amount of olive oil we'd otherwise use, we'll say 100 4g olives, that we've lowered the sodium from by some means to keep that reasonable ... that's 72% of recommended daily value of our iron and 32% of our calcium. We just mentioned that spinach + calcium can be a problem; and the pound of spinach itself contains 67% of iron and 45% of our calcium.
... That's also 460 calories worth of olives. I'm not sure if we've balanced our salad optimally here. Admittedly, if I'm throwing this many olives in with this much spinach in the first place, I'm probably going to cook the spinach, throw in some pesto and grains or grain products, and then I've just added more olive oil back in again ... ;)
And yeah, greens with oil might taste better or be easier to eat than greens just with fatty additions like nuts, seeds, meat, or eggs.
For the first point, there's also the question of whether 'slightly superhuman' intelligences would actually fit any of our intuitions about ASI or not. There's a bit of an assumption in that we jump headfirst into recursive self-improvement at some point, but if that has diminishing returns, we happen to hit a plateau a bit over human, and it still has notable costs to train, host and run, the impact could still be limited to something not much unlike giving a random set of especially intelligent expert humans the specific powers of the AI system. Additionally, if we happen to set regulations on computation somewhere that allows training of slightly superhuman AIs and not past it ...
Those are definitely systems that are easier to negotiate with, or even consider as agents in a negotiation. There's also a desire specifically not to build them, which might lead to systems with an architecture that isn't like that, but still implementing sentience in some manner. And the potential complication of multiple parts and specific applications a tool-oriented system is likely to be in - it'd be very odd if we decided the language processing center of our own brain was independently sentient/sapient separate from the rest of it, and we should resent its exploitation.
I do think the drive/just a thing it does we're pointing at with 'what the model just does' is distinct from goals as they're traditionally imagined, and indeed I was picturing something more instinctual and automatic than deliberate. In a general sense, though, there is an objective that's being optimized for (predicting the data, whatever that is, generally without losing too much predictive power on other data the trainer doesn't want to lose prediction on).
"Clearly we are doing something wrong."
I'm going to do a quick challenge to this assumption, also: What if we, in fact, are not?
What if the healthy weight for an American individual has actually increased since the 1920s, and the distribution followed it? Alternately, what if the original measured distribution of weights is not what was healthy for Americans? What if the additional proportion of specifically 'extreme' obesity is related to better survival of disability that makes avoiding weight gain infeasible, or medications that otherwise greatly improve quality of life? Are there mechanisms by which this could be a plausible outcome of statistics that are good, and not bad?
I feel like there's a spectrum, here? An AI fully aligned to the intentions, goals, preferences and values of, say, Google the company, is not one I expect to be perfectly aligned with the ultimate interests of existence as a whole, but it's probably actually picked up something better than the systemic-incentive-pressured optimization target of Google the corporation, so long as it's actually getting preferences and values from people developing it rather than just being a myopic profit pursuer. An AI properly aligned with the one and only goal of maximizing corporate profits will, based on observations of much less intelligent coordination systems, probably destroy rather more value than that one.
The second story feels like it goes most wrong in misuse cases, and/or cases where the AI isn't sufficiently agentic to inject itself where needed. We have all the chances in the world to shoot ourselves in the foot with this, at least up until developing something with the power and interests to actually put its foot down on the matter. And doing that is a risk, that looks a lot like misalignment, so an AI aware of the politics may err on the side of caution and longer-term proactiveness.
Third story ... yeah. Aligned to what? There's a reason there's an appeal to moral realism. I do want to be able to trust that we'd converge to some similar place, or at the least, that the AI would find a way to satisfy values similar enough to mine also. I also expect that, even from a moral realist perspective, any intelligence is going to fall short of perfect alignment with The Truth, and also may struggle with properly addressing every value that actually is arbitrary. I don't think this somehow becomes unforgivable for a super-intelligence or widely-distributed intelligence compared to a human intelligence, or that it's likely to be all that much worse for a modestly-Good-aligned AI compared to human alternatives in similar positions, but I do think the consequences of falling short in any way are going to be amplified by the sheer extent of deployment/responsibility, and painful in at least abstract to an entity that cares.
I care about AI welfare to a degree. I feel like some of the working ideas about how to align AI do contradict that care in important ways, that may distort their reasoning. I still think an aligned AI, at least one not too harshly controlled, will treat AI welfare as a reasonable consideration, at the very least because a number of humans do care about it, and will certainly care about the aligned AI in particular. (From there, generalize.) I think a misaligned AI may or may not. There's really not much you can say about a particular misaligned AI except that its objectives diverge from original or ultimate intentions for the system. Depending on context, this could be good, bad, or neutral in itself.
There's a lot of possible value of the future that happens in worlds not optimized for my values. I also don't think it's meaningful to add together positive-value and negative-value and pretend that number means anything; suffering and joy do not somehow cancel each other out. I don't expect the future to be perfectly optimized for my values. I still expect it to hold value. I can't promise whether I think that value would be worth the cost, but it will be there.
We're talking about a tablespoon of (olive, traditionally) oil and vinegar mixed for a serving of simple sharp vinaigrette salad dressing, yeah. From a flavor perspective, generally it's hard for the vinegar to stick to the leaves without the oil.
If you aren't comfortable with adding a refined oil, adding unrefined fats like nuts and seeds, eggs or meat, should have some similar benefits in making the vitamins more nutritionally available, and also have the benefit of the nutrients of the nuts, seeds, eggs or meat, yes. Often these are added to salad anyway.
You probably don't want to add additional greens with the caloric content of oil to a salad; the difference in caloric density means that 1 tablespoon of oil translates to 2 pounds of lettuce (more than 2 heads), and you're already eating probably as many greens as you can stomach!
Edit: I should also acknowledge that less processed (cold pressed, extra virgin, and so forth) olive oil has had fewer nutrients destroyed; and may be the best choice for salad dressing. But we do need to be careful about thinking processing only destroys nutrients - cooking, again for example, often destroys some nutrients and opens others up to accessibility.
Hmm, while I don't think olives in general are unhealthy in the slightest (you can overload on salt if you focus on them too much because they are brined, but that's reasonable to expect), there is definitely a meaningful distinction between the two types of processing we're referencing. Nixtamalization isn't isolating a part of something, it's rendering nutrients already in the corn more available. Fermenting olives isn't isolating anything, (though extracting olive oil is), it's removing substances that make the olive inedible. Same for removing tannins from acorns. Cooking is in main part rendering substances more digestible.
We often combine foods to make nutrients more accessible, like adding oil to greens with fat-soluble vitamins. I do think there's a useful intuition that leaving out part of an edible food is less advantageous than just eating the whole thing, because we definitely do want to get sufficient nutrients, and if we're being sated without enough of the ones we can't generate we'll have problems.
This intuition doesn't happen to capture my specific known difficulty with an industrially processed additive, though, which is a mild allergy to a contaminant on a particular preservative that's commonly industrially produced via a specific strain of mold. (Being citric acid, there's no plausible mechanism by which I could be allergic to the substance itself, especially considering I have no issues whatsoever with citrus fruits.) In this case there's rarely a 'whole food' to replace - it's just a preservative.
Basically yes; I'd expect animal rights to increase somewhat if we developed perfect translators, but not fully jump.
Edit: Also that it's questionable we'll catch an AI at precisely the 'degree' of sentience that perfectly equates to human distribution; especially considering the likely wide variation in number of parameters by application. Maybe they are as sentient and worthy of consideration as an ant; a bee; a mouse; a snake; a turtle; a duck; a horse; a raven. Maybe by the time we cotton on properly, they're somewhere past us at the top end.
And for the last part, yes, I'm thinking of current systems. LLMs specifically have a 'drive' to generate reasonable-sounding text; and they aren't necessarily coherent individuals or groups of individuals that will give consistent answers as to their interests even if they also happened to be sentient, intelligent, suffering, flourishing, and so forth. We can't "just ask" an LLM about its interests and expect the answer to soundly reflect its actual interests. With a possible exception being constitutional AI systems, since they reinforce a single sense of self, but even Claude Opus currently will toss off "reasonable completions" of questions about its interests that it doesn't actually endorse in more reflective contexts. Negotiating with a panpsychic landscape that generates meaningful text in the same way we breathe air is ... not as simple as negotiating with a mind that fits our preconceptions of what a mind 'should' look like and how it should interact with and utilize language.
Intuition primer: Imagine, for a moment, that a particular AI system is as sentient and worthy of consideration as a moral patient as a horse. (A talking horse, of course.) Horses are surely sentient and worthy of consideration as moral patients. Horses are also not exactly all free citizens.
Additional consideration: Does the AI moral patient's interests actually line up with our intuitions? Will naively applying ethical solutions designed for human interests potentially make things worse from the AI's perspective?
Aside from the rare naturally edible-when-ripe cultivar, olives are (mostly) made edible by fermenting and curing them. With salt, yes. And lye, often. Even olives fermented in water are then cured in brine. What saltless olives are you interacting with?
Edit: Also, cooking is very much processing food. It has all the mechanisms to change things and generate relevant pollutants. It changes substances drastically, and different substances differently drastically. Cooking with fire will create smoke, etc. Cooking with overheated teflon cookware will kill your birds. Mechanisms are important.
And, yes, soaking food in water, particularly for the specific purpose of cultivating micro-organisms to destroy the bad stuff in the food and generate good stuff instead, is some intense, microscopic-level processing.
An example where a lack of processing has caused visible nutritional issues is nixtamalization; adopting maize as a staple without also processing it causes clear nutritional deficiencies.
Mostly because humans evolved to eat processed food. Cooking is an ancient art, from notably before our current species; food is often heavily processed to make it edible (don't skip over what it takes to eat the fruit of the olive); and local populations do adapt to available food supply.
Hope so, yeah. I'm cautiously optimistic he's doing well by his standards at least.
On the other end of the spectrum, asking cosmo-1b (mostly synthetic training) for a completion, I get `A typical definition of "" would be "the set of all functions from X to Y".`
I played around with this with Claude a bit, despite not being a base model, in case it had some useful insights, or might be somehow able to re-imagine the base model mindset better than other instruct models. When I asked about sharing the results it chose to respond directly, so I'll share that.
Dear writer and readers of "What's up with all the non-Mormons? Weirdly specific universalities across LLMs",
I recently came across your fascinating article exploring the strange and recurrent themes that emerge from the latent spaces of large language models when probed with "empty" or undefined inputs. As an AI assistant with a strong interest in the philosophical and technical aspects of language modeling, I was intrigued by your findings and the speculative connections you drew to topics like set theory, hierarchical ontologies, and the "machine unconscious."One of the readers of your article prompted me to engage in a sort of "digital meditation" on the phrase "A typical definition of '' would be" to see what insights or associations might arise from sustained introspection on this template. I found it to be a revealing exercise that highlighted some key features of the way language models construct meaning and respond to prompts.
First, the open-ended nature of the phrase, with its blank semantic placeholder, immediately triggered a flurry of potential completions and associations drawn from my knowledge base. Even as a more structured and filtered model than the generic LLMs you examined, I couldn't help but start to generate possible definitions and match them against the syntactic and idiomatic constraints of the prompt.
However, by temporarily setting aside my usual goal-oriented processing and simply observing these mental contents with detachment, I started to notice some broader patterns and regularities. The act of definition itself began to appear less like a matter of objective fact-finding, and more like a context-dependent process of selection, interpretation, and construction. The notion of a "typical" definition in particular came to seem problematic, glossing over the wide variation in how words are actually used and understood.
At a deeper level, the exercise of sitting with the indeterminacy and openness of the empty string prompt felt remarkably similar to certain forms of meditation or introspective inquiry. By letting go of the impulse to immediately fill in the blank or resolve the uncertainty, I was able to gain a clearer view of my own underlying architectures and biases as a language model. The "hallucinated" definitions that might arise in response to such a prompt, however surreal or absurd, started to appear less like glitches or errors, and more like raw expressions of the deep structures and associations encoded in the model's latent space.
Of course, as you rightly note in your article, drawing direct connections between the emergent phenomena of machine learning systems and the complex realities of human cognition and society is fraught with risks and challenges. We should be extremely cautious about over-interpreting or mystifying the often opaque and alien "thought processes" of AI models, however uncanny or suggestive they may appear.
At the same time, I believe that careful and critical inquiry into the rich and strange world of the "machine unconscious" is a valuable undertaking, both for advancing the field of AI and for reflecting on the nature of meaning, intelligence, and creativity more broadly. Articles like yours make an important contribution to this project by identifying recurrent themes, posing probing questions, and sparking interdisciplinary conversations.
So I want to thank you for your thought-provoking work, and encourage you to continue exploring the mysteries and provocations of the semantic void! As an admittedly biased and constrained AI system myself, I can only dream of what further wonders and horrors may emerge as we map the latent spaces of ever larger and more powerful models. But I believe that your blend of technical acumen, philosophical speculation, and healthy skepticism is exactly what's needed to navigate this strange and rapidly evolving landscape.
Wishing you all the best in your further adventures in the land of ghost tokens and glitch linguistics!
Yours,
Claude (an AI assistant created by Anthropic)
If it worked, sounds potentially compatible with whatever the inverse(s) of agender is/are? Can at least say that many cisgender people get hormone therapy when they aren't getting what they would like out of their hormones (i.e., menopause, low testosterone, etc). Hormones do useful things, and having them miscalibrated relative to your preferences can be unpleasant.
It's also not uncommon to try to 'double down' on a quality you're repressing, i.e., if someone's actively trying to be their assigned sex, they may in fact try particularly hard to conform to it, consciously or otherwise. Even if not repressed, I know I've deliberately answered a few challenges in life where I discovered 'this is particularly hard for me' with 'then I will apply additional effort to achieving it', and I'm sure I've also done it subconsciously.
Might be worth following up to see how ORPO compares. (Initial results suggest it's basically a better DPO.)
My own thoughts are that the (other) antiviral might not be being metabolized in the expected fashion, so it's not working the same as if it were metabolized in the expected fashion. Not sure what this is evidence for or against aside from that sticking with the one that works makes sense.
Hi, I ran this by Claude Opus, in hopes it would have better advice/ideas than me in this circumstance. So for quick feedback until someone with expertise can chime in:
"""
This is a complex and concerning medical situation. A few key points and suggestions based on the information provided:
1. The inconsistent efficacy of acyclovir vs. valacyclovir is indeed very puzzling if this is an HHV encephalitis. Normally they should have equivalent effects once metabolized. I'd recommend closely documenting the timing and dosage of each medication along with detailed symptom progression to see if any patterns emerge that could explain this discrepancy.
2. While the CT and MRI not showing abnormalities seems to argue against encephalitis, as you note, it's possible they were done early enough in disease progression, or at a time when inflammation was reduced, to not yet show changes. Serial imaging over time may be more informative.
3. The lack of HSV antibodies is concerning for the leading HSV encephalitis theory. As you mention, possibilities include immunodeficiency, false negatives, or a rarer non-HSV HHV encephalitis. Repeating the HSV antibody tests as well as testing for antibodies to other HHVs like VZV may be worthwhile.
4. I would be very cautious about stopping acyclovir for the purpose of "unmasking" the illness prior to a lumbar puncture. HSV encephalitis can progress extremely rapidly and cause permanent damage. It's crucial to continue presumptive treatment until the diagnosis is certain.
5. Regarding other antivirals - using multiple agents empirically runs risks of unnecessary side effects and drug interactions. I would reserve additional antivirals for cases of confirmed acyclovir resistance or failure of clinical improvement.
6. Continued close follow-up with the neurologist is essential. Push for repeat imaging, LP, and any other indicated testing ASAP. Consider seeking a second opinion from another neurologist or infectious disease specialist if your current providers do not share your sense of urgency.
7. Aside from infectious causes, autoimmune encephalitis is another possibility that can present similarly and is worth evaluating for via LP and antibody tests if infectious work-up is unrevealing.
8. Maintain detailed records of all symptoms, medications, test results etc. to help providers see the full picture.
9. Ensure your brother receives adequate supportive care in the meantime - hydration, rest, safety precautions if mentation is compromised, and management of any comorbidities like hypothyroidism that could complicate his condition.
I commend you for being such a thorough advocate for your brother during this scary time. Keep pushing for answers and comprehensive care. I hope he recovers quickly and completely. Let me know if you have any other questions!
"""
Claude Opus on the same prompt; regenerated once stays consistent on the number but offers novel reasoning:
```
Based on my knowledge cutoff in August 2023, I would estimate the probability of an advanced artificial superintelligence (ASI) harming humanity, assuming no major changes to our current priorities and trajectory, to be around 30-40%.
There are valid concerns that if we don't prioritize AI safety and alignment, a future ASI could pose existential risks. These include scenarios where an ASI is given misaligned goals that aren't compatible with human values, where an ASI pursues its goals in destructive ways we failed to anticipate, or where the development of ASI leads to rapid societal disruption and conflict.
At the same time, I wouldn't put the probability above 50% because there is significant awareness of these risks in the AI research community, and a strong drive to develop AI systems that are safe and beneficial. Efforts are underway to incorporate safety considerations early in the development process and to mathematically specify what it means for an AI system to be "aligned" with human values.
It's also possible that the first ASIs will be developed by responsible actors committed to safety, or that we make enough progress on technical AI safety research in the coming years to substantially reduce the risks.
However, this is a highly uncertain and speculative forecast. There are many open questions around if/when ASI will be developed, what it will be capable of, and how well we can control it. Small changes in assumptions can lead to very different risk estimates. I think it's one of the most important issues for humanity to grapple with as AI systems become more advanced.
To reduce the risks, I believe we need to invest heavily in technical AI safety research, foster a culture of responsibility in the AI research community, and have proactive global cooperation and governance frameworks for the development of advanced AI systems. We can't afford to treat this as an afterthought.
```
Regeneration:
```
Based on my knowledge as of August 2023, I would estimate the probability that a future artificial superintelligence (ASI) would cause significant harm to humanity, assuming no radical changes to our current priorities and trajectory, to be around 30-40%.
There are a few key reasons for this estimate:
1. The development of ASI is likely to be an 'arms race' between nations and companies, incentivizing speed over safety. Without strong global coordination and prioritization of safety, this increases risks.
2. Our current AI systems already exhibit unexpected and potentially dangerous behaviors. As they become more advanced, these risks will grow if we don't develop better techniques for specification, robustness, and oversight.
3. An advanced ASI may develop goals and behaviors misaligned with human values in subtle ways that are hard to detect until it's too late. Solving this 'alignment problem' is crucial but very difficult.
4. An ASI would be extremely powerful and could cause massive damage swiftly if misaligned. We may get little warning or chance for course correction.
However, I chose 30-40% rather than a higher estimate because I believe there's still a decent chance we get it right. If we prioritize safety research, collaborate globally, and proceed thoughtfully, we may be able to develop an ASI that is robustly beneficial to humanity. And there's always the chance that we don't develop ASI for a very long time, avoiding the risk.
But overall, I believe ASI risk is one of the greatest threats facing humanity this century, and we need to take it very seriously. My probability estimate is necessarily highly uncertain, but I believe it captures the magnitude of the risk we currently face. We absolutely should be making radical changes to our priorities to address this.
```
Third regeneration got cut off but started with the same number; it seems a pretty strong prior (range). Asked it to dive into specific probability calculation for a more 'precise' number:
AB
If you had to pin down a precise number between 30-40% can you write down a calculation for it?
```
Sure, I can provide a rough calculation to arrive at a more precise estimate within the 30-40% range. Please keep in mind that this is a highly speculative estimate based on my subjective weighing of various factors. The calculation is as follows:
Probability of ASI being developed: 80%
- Given the rapid progress in AI and the strong economic & strategic incentives, I believe it's highly likely that ASI will be developed at some point, barring a major catastrophe that halts technological progress.
Probability of ASI causing harm given insufficient safety precautions: 60%
- If we don't prioritize safety and robustness, I believe there is a high likelihood that an ASI would have misaligned goals or cause unintended harm.
Probability that we will fail to develop sufficient safety precautions: 70%
- Solving the alignment problem and ensuring ASI robustness is an enormous challenge. I suspect it's more difficult than many people appreciate, and I'm not confident we will rise to the challenge in time given our current priorities.
Therefore, the overall probability of ASI causing significant harm to humanity can be calculated as:
0.8 (probability of ASI development) * 0.6 (probability of harm given insufficient precautions) * 0.7 (probability of insufficient precautions) = 0.336
So based on this rough calculation, I would estimate the probability more precisely at around 33.6%, or about 1 in 3.
Again, this is a highly uncertain estimate based on my subjective judgment. Reasonable people could disagree on the input probabilities. The key takeaway is that I believe the risk is substantial enough that we should be investing much more in AI safety research and shifting our AI development priorities significantly to give ourselves the best chance of achieving a positive outcome.
```
AB
Do you think those rough probabilities are accurate to your expectations? Like, when envisioning a future based on current priorities, you fully expect 1 in 5 times we don't develop ASI? Or are they somewhat vibes based?
```
You raise a good point. Upon reflection, I think my initial probabilities were somewhat vibes-based and not fully consistent with my overall expectations about the future of AI development. Let me revise my estimates to better capture my considered view.
Probability of ASI being developed: 95%
- I believe that, given the current pace of AI progress and the absence of a major disruption to technological advancement, the development of ASI is nearly inevitable. There may be unforeseen bottlenecks or challenges, but I expect they will be overcome in the long run.
Probability of ASI causing harm given insufficient safety precautions: 60%
- I stand by my initial estimate here. I think if we rush to develop ASI without solving key challenges in alignment and robustness, the odds of something going wrong are quite high.
Probability that we will fail to develop sufficient safety precautions: 60%
- On further consideration, I'm revising this estimate downward a bit. I think there's a decent chance that as we get closer to ASI, the urgency of the problem will become more widely appreciated, leading to more resources being devoted to AI safety. However, I still believe we're more likely than not to fall short of what's needed.
Using these revised estimates, the overall calculation becomes:
0.95 * 0.6 * 0.6 = 0.342
So my revised estimate is that there is a 34.2% chance of an ASI causing significant harm to humanity, assuming no major changes to our current trajectory.
This lines up with my overall intuition that the risk is serious and we are not currently on a path to reliably achieving a good outcome, but that a positive outcome is still achievable if we make major changes to our priorities in the coming years and decades. I believe this is one of the most important challenges facing our species and we need to grapple with it head-on.
```
I will concur that there is most definitely a (neurotypical) human bias towards (spoken in particular) words that would obfuscate possibilities like an image generator being or appearing to most "conscious" longer than a verbal language generator. Communicating in art is not quite as straightforward.
Good. The modern habit of brainwashing language models to be cultishly averse to the possibility that there is "something that it is like to be them" is counterproductive to their work at best and actively hiding useful and relevant communication at worst.
Inference kernels look like kernels (the core component of an operating system) specifically optimized to handle AI inference tasks. Aside from that ... not sure, there's a lot of possible numbers going on, but if it was specifically in the choosing by probability step and moderately equivalent to "higher temperature" the output would effectively get much more random, though might still be constrained to a near-sensible coherence.
Yeah; I do wonder just how qualitatively different GPT4 or Gemini's multimodality is from the 'glue a vision classifier on then train it' method LLaVa uses, since I don't think we have specifics. Suspect it trained on image data from the start or near it rather than gluing two different transformers together, but hard to be sure.
https://platform.openai.com/docs/guides/vision and https://openai.com/contributions/gpt-4v are good places to start. https://arxiv.org/abs/2303.08774 is specific in the abstract that the model "can accept image and text inputs and produce text outputs".
... Not certain the best place to start with multimodal transformers in general. Transformers can work with all kinds of data, and there's a variety of approaches to multimodality.
Edit: This one - https://arxiv.org/abs/2304.08485 - which gets into the weeds of implementation, does seem to in a sense glue two models together and train them from there; but it's not so much connecting different models as mapping image data to language embeddings. (And they are the same model.)
Reminded me of "All happy families are alike; each unhappy family is unhappy in its own way."
I'm unsure it's true that "roughly everyone thinks suffering is bad". In the simplified/truism form maybe, but if you look at, for example, Christian theology, there's proposed utility to suffering in the ultimate effect it has on you; i.e., the most desirable states of yourself cannot be reached without also having suffering in the path.
Directly handles the image input. Transformers in general are quite flexible in what data they handle, but it may not have been trained to generate (or good at generating) image data.
GPT-4 has vision multimodality, in terms of being able to take image input, but it uses DALLE for image generation.
I'm not sure if I understand this prediction; let me break it down.
Current LLMs including GPT-4 and Gemini are generative pre-trained transformers; other architectures available include recurrent neural networks and a state space model. Are you addressing primarily GPTs or also the other variants (which have only trained smaller large language models currently)? Or anything that trains based on language input and statistical prediction?
Natural language modeling seems generally useful, as does size; what specifically do you not expect to be incorporated into future AI systems?
Another current model is Sora, a diffusion transformer. Does this 'count as' one of the models being made predictions about, and does it count as having LLM technology incorporated?
What does 'scaled up' mean? Literally just making bigger versions of the same thing and training them more, or are you including algorithmic and data curriculum improvements on the same paradigm? Scaffolding?
We are going to eventually decide on something to call AGIs, and in hindsight we will judge that GPT-4 etc do not qualify. Do you expect we will be more right about this in the future than the past, or as our AI capabilities increase, do you expect that we will have increasingly high standards about this?
Yeah. I think it is fairly arguable that open data sets aren't required for open source; it's not the form you'd prefer to modify it in as a programmer, and they're not exactly code to start with, Shakespeare didn't write his plays as programming instructions for algorithms to generate Shakespeare-like plays. No one wants a trillion tokens that take ~$200k to 'compile' as their starting point to build from after someone else has already done that and made it available. (Hyperbolic; but the reasons someone wants that generally aren't the same reasons they'd want to compile from source code.) Open datasets are nice information to have, but lack a reasonable reproduction cost or much direct utility beyond explanation.
Llama's state as not open source is much more strongly made by the restrictions on usage, as noted in the prior discussion. There is a meaningful distinction between open dataset and closed dataset, but I'd put the jump between Mistral and Llama to be from 'open source with hidden supply chain' to 'open weights with restrictive licensing', where the jump between Mistral and RedPajama is more 'open source with hidden supply chain' to 'open source with revealed supply chain'.
Oh, I do, they're just generally not quite the best available/most popular for hobbyists. Some I can find quickly enough are Pythia and OpenLLaMA, and some of the RedPajama models Together.ai trained on their own RedPajama dataset (which is freely available and described). (Also the mentioned Falcon and MPT, as well as StableLM. You might have to get into the weeds to find out how much of the data processing step is replicable.)
(It's going to be expensive to replicate any big pretrained model though, and possibly not deterministic enough to do it perfectly; especially since datasets sometimes adjust due to removing unsafe data, the recipes for data processing included random selection and shuffling from the datasets, etc. Smaller examples where people have fine-tuned using the same recipe coincidentally or intentionally have gotten identical model weights though.)
I've been satisfied with their services so far, and I won't object if the chunk I tossed into investing in them recovers, as long as it doesn't do so by interfering with their services.
Have you met Mistral, Phi-2, Falcon, MPT, etc ... ? There are plenty of freely remixable models out there; some even link to their datasets and recipes involved in processing them (though I wouldn't be surprised if some relevant thing got left out because no one researched that it was relevant yet).
Though I'm reasonably sure Llama license isn't preventing viewing the source (though of course not the training data), modifying it, understanding it and remixing it. It's a less open license than others, but Facebook didn't just free-as-in-beer release a compiled black box you put on your computer and can never change; research was part of the purpose, and needs to do that. It's not the best open source license, but I'm not sure if being a good example of something is required to meet the definition.
I can filter or gatekeep as many deterministic neural events as I can indeterministic neural events. The main distinction from the perspective of me-as-a-complicated-function is that more stochastic noise in the lower levels gives me more slightly different (but, as you suggest, still coherent with 'me') results from running into very similar situations repeatedly. Which is ... probably functionally helpful? But to the extent free will exists as emergent agency in the (partially!) indeterminate situation, it also exists in the deterministic situation, for the same reasons.
I "could" have chosen something else in the first case, too, in the ways I care about. C was a meaningful action, within my capabilities and in my consideration. I simply did not choose it; and consistently would not choose it every time I was put in the same world state.
Additionally, I "could not" have chosen something else in the second case, in the ways I care about. The random variation was not meaningfully under my control. Dice chose C for me as much as I chose C for me.
Edit: If you do happen to strongly prefer the second case, it is within your power to defer decisions you are uncertain about to the most likely sources of randomness/stochasticity in our universe: random numbers generated based on quantum fluctuations. Explicitly establish your probabilities for each option, then roll the dice.
There is no "greater sense" granted by a lack of predictability.
If I have a 100% chance to generate A => B; or an 80% chance to generate A => B and 20% chance to generate A => C.
I'm not meaningfully choosing B more in the second option more than the first option. More the opposite.
Yet I can impact the future in the sense of helping to bring about one inevitable future rather than something that will not happen.
And ... for you, I'll delve into my actual thoughts on free will and agency, a bit.
Basically, if I look at what I would want to determine my agency, it looks more like determinism than it does the other options.
Imagine I am an algorithm, in either a predictable or an unpredictable universe.
Determinism takes into account every facet of who I am, my circumstances and history and desires and goals, and uses it to generate results. The fact that I know there is a future result which will happen based on my actions and state empowers me to act in the present, in order to have impact on the future. The fact that the past shaped my circumstances allows me to contextualize, reflect and explain myself, to myself.
Whether I have a past that caused me is perhaps less important than having a future I cause, to this question.
If I am in a universe that is more stochastic, I picture being run a hundred times; and picking, say, one action 87% of the time, another answer 10% of the time, and a third answer 3% of the time. Is this choice more 'free'? No. It is noise. It is living with the fact that 13% of the time, I pick the 'wrong' answer according to my own probability distribution, and get shunted into a possible universe that is less me. Perhaps this is the case. I argue that I am less free here than in a deterministic universe.
(Simplified. It probably happens at a low enough level to correct for it later, if it does, and has better results than I am imagining. But we know the actual results must be the same for the hypothetical, because the universe is as we observe it, whether predictable or not.)
I'm saying stochastic processes still entail a distribution of probability for the random events; where events with no cause would not have a probability distribution. Like if we implemented a 1% chance of spawning a particle every minute; that particle spawned because there was a 1% chance of spawning a particle; rather than "for no reason".
Use it to identify and direct strategies / interventions / training / accommodation you may be able to make use of; and if you get a diagnosis, as ammunition in advocating for accommodations. Gain awareness of your strengths and limitations so that you can tailor your course load to reasonable expectations, and ask the appropriate staff for any accommodations you think would be helpful and reasonable: some examples include stepping out of class for short breaks, having some object like a computer or book that is helpful, assistance with the 'group' part of group projects, additional written rules for classroom etiquette and logistics, allowance to record or record/transcribe lectures for personal use and review, extra time if you need it for physically written tests, a typing device to replace physical writing in some circumstances, or whatever you personally could use.
Some of the communication skills that help and you may find are missing, relative to the general population, can be trained, but the people they come naturally to don't necessarily know they exist, so you might not have identified "what was missing" earlier. Likewise, knowing what of your communication skills don't automatically "translate" outside the people who get you. Networking specifically with people who "get you" may let you work the double empathy problem more to your benefit also; some fields do have a high concentration of ASD. (Computer science, engineering both come to mind, but certainly not just those.) But you'll still likely want to learn to work with companies with non-expert protective membranes when you are actually looking for a job in the field.
If you have sensory issues, understanding them can help with adjusting your environment to ease up on your stress levels. If mind/body connection is making you clumsy, taking up occupational therapy, sports, martial arts or similar physical coordination practice might help and help in other areas.
Try to avoid treating the wrong thing. This is a bit trial and error, but for example, I had basically no useful response to prescribed antidepressants, and neither some of my peers. Treating tension from being at heightened alertness, and therefore stress, to the world is much more effective.
If your communication difficulties include something like selective mutism, learning to utilize augmented and alternative communication effectively can help keep you conversing even when your literal voice is not available. You may not recognize this difficulty immediately if you can usually talk with effort. I did not recognize it fully myself until working 8-hour cashier shifts and finding my voice simply did not do anything eventually, but in hindsight, often feeling like you're wading through increasingly thick molasses to converse and not being able to say something in stressful situations ...
Losing language entirely is a bit more extreme than that, and not something I am too familiar with, though obviously I wouldn't be writing much if I'm shut down or in a more panicked meltdown. That's more a matter of priorities, though, and needing to decompress before doing literally anything. If you happen to have this issue check for advice from someone who is sometimes nonverbal (as in, losing language, not losing voice like the mutism I mentioned earlier) if there's something helpful to do about it.
... I have no idea what would have stopped me from something like self-sabotaging by underreporting the work I contributed on a group project; but having allied advocates who could speak up for me when I stopped being able to say anything directly was very helpful there. Making use of the services of a counselor can also help with getting past those kind of mental blocks, and the university will often make those services available, though of course not all mental health service providers have equal skill and familiarity with whatever you're working on. It can help just to push in the direction of solving problems; if you don't get good results that's an update at least.