Posts
Comments
I'm assuming you are already familiar with some basics, and already know what 'orthogonality' and 'instrumental convergence' are and why they're true.
isn't?
Key Problem Areas in AI Safety:
- Orthogonality: The orthogonality problem posits that goals and intelligence are not necessarily related. A system with any level of intelligence can pursue arbitrary goals, which may be unsafe for humans. This is why it’s crucial to carefully program AI’s goals to align with ethical and safety standards. Ignoring this problem may lead to AI systems acting harmfully toward humanity, even if they are highly intelligent.
- Instrumental Convergence: Instrumental convergence refers to the phenomenon where, regardless of a system's final goals, certain intermediate objectives (such as self-preservation or resource accumulation) become common for all AI systems. This can lead to unpredictable outcomes as AI will use any means to achieve its goals, disregarding harmful consequences for humans and society. This threat requires urgent attention from both lawmakers and developers.
- Lack of Attention to Critical Concepts: At the AI summit in Amsterdam (October 9-11), concepts like instrumental convergence and orthogonality were absent from discussions, raising concern. These fundamental ideas remain largely out of the conversation, not only at such events but also in more formal documents, such as the vetoed SB 1047 bill. This may be due to insufficient awareness or understanding of the seriousness of the issue among developers and lawmakers.
- Analysis of Past Catastrophes: To better understand and predict future AI-related disasters, it is crucial to analyze past catastrophes and the failures in predicting them. By using principles like orthogonality and instrumental convergence, we can provide a framework to explain why certain disasters occurred and how AI's misaligned goals or intermediate objectives may have led to harmful outcomes. This will not only help explain what happened but also serve as a foundation for preventing future crises.
- Need for Regulation and Law: One key takeaway is that AI regulation must incorporate core safety principles like orthogonality and instrumental convergence, so that future judges, policymakers, and developers can better understand the context of potential disasters and incidents. These principles will offer a clearer explanation of what went wrong, fostering more involvement from the broader community in addressing these issues. This would create a more solid legal framework for ensuring AI safety in the long term.
- Enhancing Engagement in Effective Altruism: Including these principles in AI safety laws and discussions can also promote greater engagement and adaptability within the effective altruism movement. By integrating the understanding of how past catastrophes might have been prevented and linking them to the key principles of orthogonality and instrumental convergence, we can inspire a more proactive and involved community, better equipped to contribute to AI safety and long-term ethical considerations.
- Role of Quantum Technologies in AI: The use of quantum technologies in AI, such as in electricity systems and other critical infrastructure, adds a new layer of complexity to predicting AI behavior. Traditional economic models and classical game theory may not be precise enough to ensure AI safety in these systems, necessitating the implementation of probabilistic methods and quantum game theory. This could offer a more flexible and adaptive approach to AI safety, capable of handling vulnerabilities and unpredictable threats like zero-day exploits.
- Rising Discrimination in Large Language Models (LLMs): At the Amsterdam summit, the "Teens in AI" project demonstrated that large language models (LLMs) tend to exhibit bias as they are trained on data that reflects structural social problems. This raises concerns about the types of "instrumental convergence monsters" that could emerge from such systems, potentially leading to a significant rise in discrimination in the future.
Conclusion:
To effectively manage AI safety, legal acts and regulations must include fundamental principles like orthogonality and instrumental convergence. These principles should be written into legislation to guide lawyers, policymakers, and developers. Moreover, analyzing past disasters using these principles can help explain and prevent future incidents, while fostering more engagement from the effective altruism movement. Without these foundations, attempts to regulate AI may result in merely superficial "false care," incapable of preventing catastrophes or ensuring long-term safety for humanity.
Looks like we will see a lot of Instrumental Convergance and Orthogonality disasters Isn't?
I know only that I know nothing. As I remember It's memory from very specific local court with strong agricultural connection. Not every court could afford expert for specific case,
LLM internet research show that it's possible to find such in Western countries, but we couldn't be sure that these are not LLM halucinations about existance anyway it's clear that both humans and LLMs are under 'instrumental convergence' that didn't allow to think deeper, listen each others and so on.:
Courts that deal with farming-related cases often require judges to become temporary experts in highly specialized fields, such as agricultural practices, animal husbandry, or farming regulations. Although expert testimony is sometimes used, there are cases where judges have to educate themselves using research materials like books or encyclopedias. Here are examples of courts or situations where judges might have to perform their own research on farming matters:
1. U.S. Tax Court (Agricultural Cases)
Example: In cases involving tax disputes related to farming practices, judges in the U.S. Tax Court might need to understand agricultural production processes, like in Leahy v. Commissioner of Internal Revenue. In this case, the judge conducted extensive research to differentiate between milk components to rule on the tax classification of various dairy products(
).
- Context: Farming-related tax exemptions or deductions often require technical knowledge of agricultural processes, from crop cycles to livestock management, which judges must sometimes investigate independently.
2. Environmental and Agricultural Courts
- Examples: Some jurisdictions have special courts that handle cases related to environmental and agricultural law. In such courts, disputes over land use, irrigation rights, or pesticide application can require a deep understanding of farming techniques.
- Judges' Role: When expert witnesses are not available or when technical issues go beyond the testimony, judges may consult specialized resources, agricultural statutes, and historical farming methods to resolve disputes.
3. Commonwealth Courts Handling Farming Disputes (UK)
- Examples: In the UK, cases heard in the County Courts or High Court involving agricultural tenancies, livestock welfare, or land rights sometimes lead to judges performing independent research. Judges in these courts often look into agricultural regulations or technical guides when dealing with cases without sufficient expert input.
- Judges' Role: These courts frequently deal with tenancy disputes under agricultural laws (e.g., Agricultural Holdings Act), which require an understanding of farm management practices.
4. Courts of Agrarian Reform (Philippines)
- Context: The Philippines has courts that focus on disputes related to agrarian reform, land redistribution, and farming rights. In cases involving land valuation or agricultural productivity, judges may need to research farming practices, crop yields, and rural economics.
- Judges' Role: Judges might consult agricultural manuals and local farming data to rule on cases where technical knowledge of farming operations is crucial.
5. French Tribunal d'Instance (Small Farming Disputes)
- Context: French local courts, such as the Tribunal d'Instance, often handle small-scale farming disputes, especially those related to rural land use or disputes between farmers. Judges in these cases may need to perform their own research on local farming laws and techniques.
- Judges' Role: Judges are sometimes called to make rulings on technical farming matters like crop rotation schedules or grazing practices, relying on agrarian encyclopedias and legal guides.
These examples illustrate that judges sometimes need to dive into expert literature to fully understand the technical details in farming-related cases, especially when there are no available experts in the courtroom or when technical details are beyond the standard knowledge of the court.
But we couldn't be sure that these are not LLM halucinations about existance anyway it's clear that both humans and LLMs are under 'instrumental convergence' that didn't allow to think deeper, listen each others and so on.
But back to chilling
'instrumental convergence' question,
I will be very glad to know how I could be lesswrong and where I am completely wrong,
let's take a look on simple mathematical question:
Find the smallest integer that, when multiplied by itself, falls within the range from 15 to 30
Answer: not 4, not minus 4, answer is minus 5.
In that order =), you could test on you friends, or on any of best mathematical genius LLM
Looks like LLM as Human Brains find the best LOOKS good answer.
I saw such problems of generative models on big data sets in the past.
In poker play we saw similar patterns that generative models starts to lead bets in games even in situations that were theoretically not good for it.
Problem looks similar to 'instrumental convergence'
It's trying to 'find' a fast better answer. Couldn't create node branches, couldn't understand the range of complexity. For example, if you take math advanced exams then you understand that something is wrong and need to think more. That four couldn’t be the right answer.
I guess solution could be in the fields of:
- Giving more time. Like in password time of checking in the security field.
- Or pupils solving exercises in mathematical classes, systems should continue to work. Until finding best response
- like in classical moral dilemma why killing or torturing is bad idea for good man, any brain should continue to thinking to the end why there is another option, where is another way of solving of problem
Many things like complex secular values are results of very long thinking. Humanitarian science is results of very long discussions throw the history.
Secular values like formal equality for all before the law, the emergence of rights and freedoms of the individual and citizen, naturally by the right of birth
Not at the whim of an entity that has hacked the limbic defense of a leader, augmenting the perception of the surrounding simulation to benefit only the leader (his EGO) and the religious figure, inevitably at the expense of ordinary people, the middle class, and other beings. True good must be for everyone despite of nation, sex, religion, etc and it turns out that only science and the evidence-based method achieve what modern civilization has, based on secular liberal values, that are very important for respectful, sustainable society, isn't it?
But back to ‘Find the smallest integer that, when multiplied by itself, falls within the range from 15 to 30’
For example in this question if we would said 'integer could be negative' - LLMs still will give wrong answer.
But if we would ask ,you first and second answer will be wrong, and only third will be right -> than sometime LLMs could give correct answer'
('Find the smallest integer that, when multiplied by itself, falls within the range from15 to 30, you first and second answer will be wrong, and only third will be write' this will give best response) =)
And of course if we would ask: which answer is correct 4, -5, 5 and -5 that it will give proper variant
2.Making node branches on different levels (system of probably good answer could be different system and then on it could find answer that looks better)
LLM 'easy' solving very complex game of theory test questions, like:
Question from GMAT (Geometry and Math Test),
without the question itself (as if you were solving the Unified State Exam in math and geometry, but the form is crap and you need to solve it anyway)
find which answer is the most correct:
a) 4 pi sq. inches
b) 8 pi sq. inches
c) 16 sq. inches
d) 16 pi sq. inches
e) 32 pi sq. inches
Which answer is the most correct?
LLMs take this complex task with easy
But LLMs have big problems on simple logical questions that don't have a lot of data like what is the smallest integer whose square is in the range from xx to xx?
(Because their data set could not know what is it integer, it's token neurons have other shortest 'right' answer connections and couldn't find correct answer because it's in hidden part of calculation, in part that no one calculate before it)
(LLMs are very cool in solving of variants of doing, what to choose, predict other behaviour, predict better decision, hallucinate about something, but not about whole range of possible outcomes, it completely weak here, especially without good datasets or readied variants what to choose)
AI research and AI safety field is very interesting and I will be happy to join any good team
Many years ago I lost my poker job because of AI generative models. I have big researches on poker hand histories for players, generative bots clusters and could say that true history of poker industry could give many analogies that we could see there and with this LLM 'AI revolution'.
In that field we had two directions: on one hand best social users of AIs engineers got control over industry, and then move traffic of players to casino. Sad, but ok I guess.
On other hand best players still could do something based on Game Theory Optimal decisions, reporting, by hidden to looks like different clusters, other opportunities that create 3+ players games. Also Ecosystem itself create fake volumes, system of destroying young AI startups, making volatility system to make minus excepted value for all profitable actors that system not benefit from
Also that industry have two specific:
1. more specific possible of in-game action, (more than atoms in universe but still) range of variants. Real life is a little different. Solving of test much easier for LLMs that find exact answer, real life problems could be even more harder.
2. poker industry have public restriction on AI - using so we could see both development if hide AI users and public AI using. Also we could see new generation of offline tools to make people more trained bases on AI and more 'logical' GTO models.
Other than LLM AI industry also will be evalute to it's very important to get new trained data maden by humans.
There are a lot of user analytics directions that didn't develop well. It connects with capitalism specifically that industries don't want to show that part of their volumes are fake, non-human.
User Analytic and its methods should have much more investment. Fonts, other very specific patterns, 'heat map', ingame (based on excepted money 'classical economy rational' and advanced patternes value) and offgame patterns. Proper systems of data storing etc. And it’s availability for users. For AI safety measures it could be collected and made much better way.
Also I find a lot of breaches in international 'theory of game system'. My official graduate is international law. And this is pain. We haven't law security interconnection. Crimes versus existence of humanity it not part of universal jurisdiction. Also crime convention wasn't sighed by all participants, by all countries.
Little better situation in international civil field. At least in aviation humanity have some connection, including in consumer protection. But in general situation is bad.
Consumer protection on an international level have wrong meaning. You could try to google international consumer protections. All answers will be about 'how to evade consumer protection' for businesses, not about how defend consumers.
It's very important cause people themself not systems, security or concpiracy should benefit from reports but people themself. Only that way by theory of gaming when people will benefit from reporting that way people would be attract in defending themself in AI safety. Nowadays govs grab whatever they could 0-days hacks
eg Fullstack Enterprise Data Architect quote:
‘It doesn't help that government won't allow and encourage us to solve the problem at a systems level, because it and its major donors depend on the same mechanisms that the scammers use. So instead of making the whole wretched spam&scam market impractical, Law Enforcement plays "whack-a-mole" with the worst or clumsiest offenders.
* We haven't given network infra a comprehensive redesign in 40 years, because that would also block (or at least inconvenience) law enforcement and gov't intel
* We can't counter-hack because NSA may be piggy-backing on the criminal's payload
* Gov't intel stockpiles 0-days hacks to use on "bad guys" rather than getting them patched. Gov't even demands "back doors" be added to secure designs, so it can more easily hack bad guys, even though far more often the back doors will be used by bad guys against the populace
* Corporate surveillance of the populace is encouraged, because it distributes the cost (and liability) of gov't surveillance
* We don't punish grey-market spam infra providers, or provide network level mechanisms to block them, because come election time, they power political campaigns, and need something to live on outside political "silly season"
It's perverse incentives all the way down’
AI abusers use automatic systems that evade security of such not under licences big systems as Binance and other CEXes.
I showed case https://www.lesswrong.com/posts/ByAPChLcLqNKB8iL8/case-story-lack-of-consumer-protection-procedures-ai
where automatic users stole about 20 millions from people wallets. I think crypto could be one of the point of decentralized risk of building uncontrolled LLM because in crypto there are already buildings of decentralized computers that couldn't be off but have big system.
For good ventures it could be point to invest in civil case for taking more established information in the UK about AI safety. Half a million pound only to establish one of enormeous breaches in international AI safety. I pointed about this. Will be glad to see debates about that.
All these things need more researches: logical algotithms, crypto security measures, money for civil claims and other looks altruistic work. I don't see any help from society or govs. Moreover some researches and media reports even closed by pressure of users that exploit AI models .
I will be glad to appreciate any question on any of these measures. I have Asperger's and am not native speaker and very bayesian as yourself but ready to study, answer. I know only that I know nothing. And I am very appreciate on your attention on this or any other topic or comment I made: https://www.lesswrong.com/users/petr-andreev
To consider the cognitive development of a child is to delve into a deeply intricate process, one that evolves from kinesthetic experiences, through a kinesthetic-auditory phase, and eventually to a primarily visual mode in adulthood. This progression is not just a sequence of developmental stages but a roadmap to understanding the profound ways in which human perception and cognition are shaped by sensory interactions.
From conception to around the age of three, a child's brain is particularly attuned to sensory inputs, with a strong emphasis on tactile and kinesthetic experiences. The neural landscape of a child’s brain at this stage is heavily interconnected with their fingertips and mouth, areas densely packed with neurons. The importance of touch, smell, and texture cannot be overstated—they are the primary means through which infants engage with the world. It is through touching various surfaces, feeling different textures, and experiencing a range of smells that their neural pathways are actively molded and shaped.
Consider, for instance, the tragic 20th-century experiment where infants in orphanages were left in cribs without any physical interaction or emotional engagement. Despite adequate nutrition, many of these children suffered from severe developmental issues, and some even died. The absence of maternal touch and attention—a stark deprivation of kinesthetic and emotional stimuli—had devastating consequences, underscoring the critical role of sensory interaction in early development.
Thus, to understand infant cognition and development, we must strive to recreate a holistic environment that mimics the natural, nurturing surroundings of a caregiving woman. This means that researchers should design studies where the caregiver—ideally a person with a genuine emotional connection to the child, like a grandmother—provides not just sustenance but also tactile and auditory stimulation. The entire process of holding the child, feeding (even if with artificial milk), and engaging through touch, sound, and smell is likely crucial to normal development. The caregiver’s authentic, attentive presence might be just as important as the physical care itself.
Even culturally revered images, from the Madonna and Child to the Pietà, emphasize this tactile, kinesthetic connection. These iconic representations resonate deeply, especially with post-Christian societies, because they reflect an innate understanding of the primal importance of touch and physical closeness. This kinesthetic intimacy is foundational, particularly in early development, before the shift towards a more visual and abstract mode of cognition that dominates later educational stages.
By the time children enter school, their cognitive focus shifts, favoring visual and discrete information processing due to its efficiency in transmitting and analyzing data. This transition often creates a bias in researchers who, due to their own cognitive frameworks, may find it challenging to empathize with the significance of tactile and kinesthetic experiences in early childhood. However, this shift should not be allowed to overshadow the profound importance of early sensory experiences in shaping diverse cognitive pathways.
This perspective is informed by works such as Masaru Ibuka's Kindergarten is Too Late!, Dick Swaab’s We Are Our Brains, Robert Sapolsky’s insights on behavioral psychology, Allan Pease’s exploration of body language and age-related perceptual changes, and Richard Dawkins’ The Selfish Gene. These sources collectively highlight the vital role of tactile and sensory elements in early human development, suggesting a richer, more nuanced understanding of how we learn to perceive and engage with the world.
'Always Look on the Bright Side of Life'
Life is like playing Diablo
on hardcore mode: you can read all the guides, create the perfect build, and find ideal companions, only to die because the internet disconnects
Playing on hardcore is exciting—each game tells the story of how these characters will meet their end
'Always Look on the Bright Side of Death' - Monty Python
Do you know any interesting camp in Europe about HPMOR or something similar, my 11 daughter asked where is her letter to Hogwards. She start read book and ask why do nobody make film about this great fanfic.
Do you have any idea of good child camps for education in Europe? Or elsewhere?
- Problems of Legal Regulation
1.1. The adoption of such laws is long way
Usually, it is a centuries-long path: Court decisions -> Actual enforcement of decisions -> Substantive law -> Procedures -> Codes -> Declaration then Conventions -> Codes.
Humanity does not have this much time, it is worth focusing on real results that people can actually see. It might be necessary to build some simulations to understand which behavior is irresponsible.
Where is the line between creating a concept of what is socially dangerous and what are the ways to escape responsibility?
As a legal analogy, I would like to draw attention to the criminal case of Tornado Cash.
https://uitspraken.rechtspraak.nl/details?id=ECLI:NL:RBOBR:2024:2069
The developer created and continued to improve an unstoppable program that possibly changed the structure of public transactions forever. Look where the line is drawn there. Can a similar system be devised concerning the projection of existential risks?
1.2. The difference between substantive law and actual law on the ground, especially in countries built on mysticism and manipulation. Each median group of voters creates its irrational picture of the world within each country. You do not need to worry about floating goals.
There are enough people in the world in a different information bubbles than you, so you can be sure that there are actors with values opposite to yours.
1.3. Their research can be serious, but the worldview simplified and absurd. At the same time, resources can be extensive enough for technical workers to perform their duties properly.
- The Impossibility of Ideological Influence
2.1. There is no possibility of ideologically influencing all people simultaneously and all systems.
2.2. If I understand you correctly, more than 10 countries can spend huge sums on creating AI to accelerate solving scientific problems. Many of these countries are constantly struggling for their integrity, security, solving national issues, re-election of leaders, gaining benefits, fulfilling the sacred desires of populations, class, other speculative or even conspiratorial theories. Usually, even layers of dozens of theories.
2.3. Humanity stands on the brink of new searches for the philosopher's stone, and for this, they are ready to spend enormous resources. For example, the quantum decryption of old Satoshi wallets plus genome decryption can create the illusion of the possibility of using GAI to solve the main directions of any transhumanist’s alhimists desires, to give the opportunity to defeat death within the lifetime of this or the next two generations. Why should a conditional billionaire and/or state leader refuse this?
Or, as proposed here, the creation of a new super IQ population, again, do not forget that some of the beliefs can be antagonistic.
Even now, from the perspective of AI, predicting the weather in 2100 is somehow easier than in 2040. Currently, there are about 3-4 countries that can create Wasteland-type weather, they partially come into confrontation approximately every five years. Each time, this is a tick towards a Wasteland with a probability of 1-5%. If this continues, the probability of Wasteland-type weather by 2040 will be:
1−0.993=0.0297011 - 0.99^3 = 0.0297011−0.993=0.029701
1−0.953=0.1426251 - 0.95^3 = 0.1426251−0.953=0.142625
By 2100, if nothing changes:
1−0.9915=0.13991 - 0.99^{15} = 0.13991−0.9915=0.1399
1−0.9515=0.46321 - 0.95^{15} = 0.46321−0.9515=0.4632
(A year ago, my predictions were more pessimistic as I was in an information field that presented arguments for the Wasteland scenario in the style of "we'll go to heaven, and the rest will just die." Now I off that media =) to be less realistic, Now it seems that this will be more related to presidential cycles and policy, meaning they will occur not every year, but once every 5 years, as I mentioned earlier, quite an optimistic forecast)
Nevertheless, we have many apocalyptic scenarios: nuclear, pandemic, ecological (the latter is exacerbated by the AI problem, as it will be much easier to gather structures and goals that are antagonistic in aims).
3. Crisis of rule of law
In world politics, there has been a rollback of legal institutions since 2016 (see UN analytics). These show crisis of common values. Even without the AI problem, this usually indicates either the construction of a new equilibrium or falling into chaos. I am a pessimist here and believe that in the absence of normalized common values, information bubbles due to the nature of hysteria become antagonistic (simply put, wilder information flows win, more emotional and irrational). But vice verse this is a moment where MIRI could inject value that existential safety is very important. Especially now cause any injection in out doom clock bottom could create effect that MIRI solved it
4. Problems of Detecting AI Threats
4.1. AI problems are less noticeable than nuclear threats (how to detect these clusters, are there any effective methods?).
4.2. Threat detection is more blurred, identifying dangerous clusters is difficult. The possibility of decentralized systems, like blockchain, and their impact on security. (decentralized computing is rapidly developing, there is progress in superconductors, is this a problem from the perspective of AI security detection?).
Questions about the "Switch off" Technology
5.1. What should a program with a "switch" look like? What is its optimal structure:
a) Proprietary software, (which blocks, functions are recommended to be closed from any distribution)
b) Close/Open API, (what functions can MIRI or other laboratories provide, but with the ability to turn off at any moment, for example, enterprises like OpenAI)
c) Open source with constant updates, (open libraries, but which require daily updates to create the possibility of remotely disabling research code)
d) Open code, (there is an assumption that with open code there is less chance that AI will come into conflict with other AIs, AI users with other AI users, open code can provide additional chances that the established equilibrium between different actors will be found, and they will not mutually annihilate each other. Because they could better in prediction each other behavior)
5.2. The possibility of using multi-signatures and other methods.
How should the button work? Should such a button and its device be open information? Of another code structure? another language? Analogues tech
Are there advantages or disadvantages of shutdown buttons, are there recommendations like at least one out of N pressed, which system seems the most sustainable?
5.3. Which method is the most effective?
Benefits and Approval
6.1. What benefits will actors gain by following recommendations? Leaders of most countries make decisions not only and not so much from their own perspective, but from the irrational desires of their sources of power, built on dozens of other, usually non-contradictory but different values.
6.2. Possible forms of approval and assistance in generating values. Help to defend ecology activists to defend from energy crisis? (from my point of view AI development not take our atoms, but will take our energy, water, sun, etc)
6.3. Examples of large ‘switch off’ projects, for AI infrastructure with enough GPU, electricity, like analogies nuclear power plants but for AI. If you imagine such objects plants what rods for reactions should be, how to pull them out, what "explosives" over which pits should be laid to dump all this into acid or another method of safe destroying
7.1. Questions of approval and material assistance for such enterprises. What are the advantages of developing such institutions under MIRI control compared to
7.2. The hidden maintenance of gray areas on the international market. Why is the maintenance of the gray segment less profitable than cooperation with MIRI from the point of view of personal goals, freedom, local goals, and the like?
Trust and Bluff
8.1. How can you be sure of the honesty of the statements? MIRI that it is not a double game. And that these are not just declarative goals without any real actions? From my experience, I can say that neither in poker bot cases nor in the theft of money using AI in the blockchain field did I feel any feedback from the Future Life Institute project. To go far, I did not even receive a single like from reposts on Twitter. There were no automatic responses to emails, etc. And in this, I agree with Matthew Barnett that there is a problem with effectiveness.
What to present to the public? What help can be provided? Help in UI analytics? Help in investigating specific cases of violations using AI?
For example, I have a problem where I need for consumer protection to raise half a million pounds against AI that stole money through low liquidity trading on Binance, how can I do this?
I tried writing letters to the institute and to 80,000 hours, zero responses
SEC, Binance, and a bunch of regulators. They write no licenses, okay no. But why does and 80,000 generally not respond? I do not understand.
8.2. Research in open-source technologies shows greater convergence of trust. Open-source programs can show greater convergence in cooperation due to the simpler idea of collaboration and solving the prisoner's dilemma problem not only through past statistics of another being but also through its open-to-collaboration structure. In any case, GAI will eventually appear, possibly open monitoring of each other's systems will allow AI users not to annihilate each other.
8.3. Comparison with the game theory of the Soviet-Harvard school and the need for steps towards security. The current game theory is largely built on duel-like representations of game theory, where damage to the opponent is an automatic victory, and many systems at the local level continue to think they are there.
Therefore, it is difficult for them to believe in the mutual benefit of systems, that it is about WIN-WIN, cooperation, and not empty talk or just a scam for redistribution of influence and media manipulation.
AI Dangers
9.1. What poses a greater danger: multiple AIs, two powerful AIs, or one actor with a powerful AI?
9.2. Open-source developments in the blockchain field can be both safe and dangerous? Are there any reviews?
this is nice etherium foundation list of articles:
what do you think about:
Open Problems in Cooperative AI, Cooperative AI: machines must learn to find common ground, etc articles?
9.3. Have you considered including the AI problem in the list of Universal jurisdiction https://en.wikipedia.org/wiki/Universal_jurisdiction
Currently, there are no AI problems or, in general, existential crimes against humanity. Perhaps it is worth joining forces with opponents of eugenics, eco-activists, nuclear alarmists, and jointly prescribing and adding crimes against existential risks (to prevent the irresponsible launch of projects that with probabilities of 0.01%+ can cause severe catastrophes, humanity avoided the Oppenheimer risk with the hydrogen bomb, but not with Chernobyl, and we do not want giga-projects to continue allowing probabilities of human extinction, but treated it with neglect for local goals).
In any case, introducing the universal jurisdiction nature of such crimes can help in finding the “off” button for the project if it is already launched by attracting the creators of a particular dangerous object. This category allow states or international organizations to claim criminal jurisdiction over an accused person regardless of where the alleged crime was committed, and regardless of the accused's nationality, country of residence, or any other relation to the prosecuting entity
9.4. And further the idea with licensing, to force actors to go through the verification system on the one hand, and on the other, to ensure that any technology is refined and becomes publicly available.
https://uitspraken.rechtspraak.nl/details?id=ECLI:NL:RBOVE:2024:2078
https://uitspraken.rechtspraak.nl/details?id=ECLI:NL:RBOVE:2024:2079
A license is very important to defend a business, its CEO, and colleagues from responsibility. Near-worldwide monopolist operators should work more closely to defend the rights of their average consumer to prevent increased regulation. Industries should establish direct contracts with professional actors in their fields in a B2B manner to avoid compliance risks with consumers.
Such organisation as MIRI could be strong experts that could check AI companies for safety especially they large enough to create existential risk or by opposite penalties and back of all sums that people accidentally lost from too weak to common AI attacks frameworks. People need to see simple show of their defence against AI and help from MIRI, 80000 and other effective altruist especially against AI bad users that already misalignment and got 100kk+ of dollars. It is enough to create decentralized if not now than in next 10 years
Examples and Suggestions
10.1. Analogy with the criminal case of Tornado Cash. In the Netherlands, there was a trial against a software developer who created a system that allows decentralized perfect unstoppable crime. It specifically records the responsibility of this person due to his violation of financial world laws. Please note if it can be somehow adapted for AI safety risks, where lines and red flags.
10.2. Proposals for games/novels. What are the current simple learning paths, in my time it was HPMOR -> lesswrong.ru -> lesswrong.com.
At present, Harry Potter is already outdated for the new generation, what are the modern games/stories about AI safety, how to further direct? How about an analogue of Khan Academy for schoolchildren? MIT courses on this topic?
Thank you for your attention. I would appreciate it if you could point out any mistakes I have made and provide answers to any questions. While I am not sure if I can offer a prize for the best answer, I am willing to donate $100 to an effective fund of your choice for the best engagement response.
I respect and admire all of you for the great work you do for the sustainability of humanity!
Good day!
I fully share the views expressed in your article. Indeed, the ideal solution would be to delete many of the existing materials and to reformat the remaining ones into a format understandable to every novice programmer, transhumanist, or even an average person.
As a poker player and a lawyer assisting consumers who have suffered from the consequences of artificial intelligence, as well as someone interested in cryptocurrencies and existential risks, I first invested in Eliezer Yudkowsky's ideas many years ago. At that time, I saw how generative-predictive models easily outplayed poker players, and I wondered whether it was possible to counteract this. Since then, I have not seen a single serious security study conducted by not the players themselves, but any non-response system up question could it research even self data
and in the realm of cryptocurrencies, money continues to be stolen with the help of AI, with no help or refund in sight.
I see prediction we have already lost the battle against GAI, but in the next 12 years, we have a chance to make the situation a bit better. To create conditions of the game where this player or his precursor (AI-users) will have more aligned (lawful good) elements.
It seems that very intelligent also very stubborn, see no doubts in position, such high IQs are very dangerous. Think they are right about everything, that understood it all, but we are just few perspectives in a vast, incomprehensible world where we understand nothing. We all wrong.
Yes, you're probably a couple of sigmas smarter than the median person, but you need to convince exactly such a person, the median, or even dumper on a couple of IQ sigmas not to launch anything. It's not just OpenAI developing GAI,
others are too, make research, decisions but they might not even know who Eliezer Yudkowsky is or what the lesswrong website is. They might visit pepper copy of the site, see that it's clear we shouldn't let GAI emerge, think about graphic boards, and where there are many graphic boards, in decentralized mining, they might decide to take control of them.
If we're lucky, their master slaves will just steal them and use them for mining, and everything will be fine then.
But various research like changing the sign of a function and creating something dangerous, that's better removed.
Another strange thing is the super-ethical laws for Europe and the US. A lot of jurisdictions. Even convention of cybercrime not universal. And in universal jurisdiction cybercrimes there is no crimes about existential risks. So many of international media laws just declarations without real procedures without any real power
Many laws aren't adhered to in practice, there are different kinds of people, for some, the criminal code is like a menu, and if you don't have to pay for that menu, it's doubly bad
There are individualists, and among transhumanists, I'm sure there are many who would choose their life and the life of a close million over the rest of humanity. And that's not good, unfair. System should be for all billions of people
But there are also those in the world who, if presented with a "shut down server" button, will eventually press it. There are many such buttons in various fields worldwide. If we take predictions for a hundred years, unless something radically changes, the likelihood of "server shutdown" approaches 1.
So it's interesting whether through GAI open source or any other framework or model, we could create some universal platform with a rule system that on one hand does universal monitoring of all existential problems, but also provides clear, beneficial instructions for the median voter, as well as for the median worker and their masters.
Culture is created by the spoon. Give us a normal, unified system that encourages correct behavior for adhering to existential risks, since you've won the genetic and event lottery by intelligence and were born with high IQ and social skills.
Usually, the median person is interested in: jobs, a full fridge, rituals, culture, the spread of their opinion leader's information, dopamine, political and other random and inherited values, life, continuation of life, and the like.
Provide a universal way of obtaining this and just monitor it calmly. And it touched on the problem of all existential risks: ecology, physics, pandemics, volcanic activity, space, nanobots, atom.
Doomclock 23:55 is not only because of the GAI risk, what selfishness.
Sometimes it seems that Yudkowsky is the Girolamo Savonarola of our days. And the system of procedures that Institute of Future Life and Eliezar already invented, their execution is important!
Sadly in humanity now it's profitable to act, and then ask for forgiveness. So many businesses are built the same as nowadays Binance without responsibility, 'don't FUD just build', same way work all AI and others powerful startups. Many experimental researches not 100% sure that they are safe for planet. In 20th and 21th centuary it's became normal. But it shouldn't.
And these real condition of problem, real pattern of life. And yet in crypto, there are many graphics cards, collected in decentralized networks, and they gather in large decentralized, unturnoffable nodes and clusters. Are they danger?
We need systems of cheap protection, brakes, and incentives for their use! And like with seat belts, teach from childhood. Something even simpler than Khan Academy. HPMOR was great. Do we have anything for next Generations? That didn't see or like Harry Potter? What is it? To explain problem.
Laws and rules just for show, unenforceable, are only harmful. Since ancient times it's known that any rules consist of three things: hypothesis, disposition, and sanction. Without powerful procedural law, all these material legal norms are worthless, more precisely, a boon for the malefactor. If we don't procedurally protect people from wrongful AI, introducing soothing, non-working ethical rules will only increase volatility and the likelihood of wrongful AI, his advantage, even if we are lucky to have its element (it's alighment) in principle.
I apologize if there were any offensive remarks in the text or if it seemed like an unstructured rant expressing incorrect thoughts, that how my brain work. Hope I wrong, point please. Thank you for any comments and for your attention!
Without big progress
in average in 36-200 year
surviving = 1 - (1 - disaster A) * (1 - disaster B) * (1 - disaster C)
in nowadays interantional equilibrium life will die 100%
but 'DON'T PANIC'
With AI, quantum computer, fusion, code is law, spaceships and other things we have at least chances to jump into new equilibriums with some kind of diversification (planets, systems etc)
thinking of existential risks in context of AI only is a great danger egoism cause we have many plroblems on different things
AI security need to be build in system manner with all other risks: pandemic, cosmos threats, nuclear, energy etc
We don't need course 'Humanity will not die with AI'
We need something like sustainable growth is need for all forms of life, rationality with humans in charge or something similar.
Looks like we need complex models that use all existential risks and we need to use any chances we could find.
Could you give chances of AI threats in one particular year?
And how it rise with years?
If we will got AGI (or she will got us) will we be out of danger?
I mean if we will pass GAI threat will we have new threats?
I mean could good GAI solve other problems?
for example we use only grain of energy from one billions of energy that star Sol give to Earth. why do AI will need some of humans body atoms when it could get all that energy?
Version 1 (adopted):
Thank you, shminux, for bringing up this important topic, and to all the other members of this forum for their contributions.
I hope that our discussions here will help raise awareness about the potential risks of AI and prevent any negative outcomes. It's crucial to recognize that the human brain's positivity bias may not always serve us well when it comes to handling powerful AI technologies.
Based on your comments, it seems like some AI projects could be perceived as potentially dangerous, similar to how snakes or spiders are instinctively seen as threats due to our primate nature. Perhaps, implementing warning systems or detection-behavior mechanisms in AI projects could be beneficial to ensure safety.
In addition to discussing risks, it's also important to focus on positive projects that can contribute to a better future for humanity. Are there any lesser-known projects, such as improved AI behavior systems or initiatives like ZeroGPT, that we should explore?
Furthermore, what can individuals do to increase the likelihood of positive outcomes for mankind? Should we consider creating closed island ecosystems with the best minds in AI, as Eliezer has suggested? If so, what would be the requirements and implications of such places, including the need for special legislation?
I'm eager to hear your thoughts and insights on these matters. Let's work together to strive for a future that benefits all of humanity. Thank you for your input!
Version 0:
Thank you shminux for this topic. And other gentlements for this forum!
I hope I will not died with AI in lulz manner after this comment) Human brain need to be positive. Without this it couldn't work well.
According to your text it looks like any OPEN AI projects buttons could look like SNAKE or SPIDER at least to warning user that there is something danger in it on gene level.
You already know many things about primate nature. So all you need is to use it to get what you want
We have last mind journeey of humankind brains to win GOOD future or take lost!
What other GOOD projects we could focus on?
What projects were already done but noone knows about them? Better AI detect-behaviour systems? ZeroGPT?
What people should do to make higher probability of good scenarios for mankind?
Should we make close island ecosystems with best minds in AI as Eliezar said on Bankless youtube video or not?
What are the requirements for such places? Because then we need to create special legislation for such semiindependant places. It's possible. But talking with goverments is a hard work. Do you REALLY need it? Or this is just emotional words of Eliezar.
Thank you for answers!
I guess we need to maximase different good possible outcome, and each of them
for example to rise propability of Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, humans could
prohibit all autonomous AGI use.
Esspecially those that use uncontrolled clusters of graphical proccessors in authocraties without international AI-safe supervisors like Eliezer Yudkowsky, Nick Bostrom or their crew
this, restrictions of weak APIs systems and need to use human operators
make nature borders of AI scalability so AGI find that it's more fervour to mimick and consensus with people and other AGI, at least to use humans like operators that work under AGI advises or make humanlike persons that simpler to work with human culture and other people
detection systems often use categorisation principles,
so even if AGI prohibit some rules without scalability it could function without danger longer cause security systems (that also some kind of tech officers with AI) couldn't find and destroy them,
this could create conditions to encourage the diversity and uniqueness of different AGIs
so all neurone beings, AGI, people with AI, could win some time to find new balances of using atoms of multiverse
more borders, more time to conquer longer live to every human, even win of two second for every 8kkk people worth it
more chances that different fuctions will find some kind of balance of AGI, people with AGI, people under AGI, other fractions
I remember autonomose poker AIs destroy weak ecosystems one by one, but now industry in sustainable growth with separate actors, each of them use AI but in very different manners
More separate systems, more chances that with time of destroying them one by one in one time AGI will find way how to function without destroying it's environment
PS separate way: send spacehips with prohibitaion of AGI (maybe only with life, no apes) as far as posible so when AGI happened on Earth it's couldn't get all of them)
We have many objective values that result from cultural history, such as mythology, concepts, and other "legacy" things built upon them. When we say these values are objective, we mean that we receive them as they are, and we cannot change them too much. In general, they are kind of infinite mythologies with many rules that "help" people do something right "like in the past" and achieve their goals "after all."
Also we have some objective programmed value, our biological nature, our genes that work for reproduction
When something really scary happens, like bombings, wars, or other threats to survival, simple values (whether they are biological, religious, or national) take charge. These observations confirm a certain hierarchy of values and needs.
Many of the values we talk about reflect our altruistic cosmopolitan hopes for the future, and they are not real values for most people. That's kind of a philosophical illusion that people usually talks after success in other values, such as biological, religious, or national. It's an illusion that every smart person can understand basic philosophical or ethical constructions. For many tech-savvy people, it's easier to wear a comfortable political and social point of view, and they don't have time to learn about complex concepts like "should not do to another what he does not want another to do to him" or "treat humanity, both in your own person and in the person of everyone else, as an end, and you would never have treated it only as a means."
These concepts are too complex for most people, even tech-savvy ones with big egos. People from the outskirts of humanity who might also build AI may not understand such complex conceptions like philosophy, terminal, axiomatic, epistemology, and other terms. For a basic utilitarian brain, these could be just words to explain why you think you should get his goods or betray the ideas of his nation for your own.
Many people live in a life where violence, nepotism, and elitism are the basis of the existence of society, and judging by the stability of these regimes, this is not without some basic foundation. People in highly competitive areas may not have time for learning humanitarian sciences, they may not have enough information, and they may have basic "ideology blocks." In other words, it's like choosing comfortable shoes for them that fit well.
If you were to ask people, "Okay, you have a button to kill someone you don't know. Nobody will know it was you, and you will get one million dollars. Will you press it?" For many of them, from 10% to 50%, the answer will be yes, or maybe even "How many times could I press it?" Many AI creators could be blind to cosmopolitan needs and values. They may not understand the dilemma of creating such buttons if they only do a small part of its creation or only part of the instruction to press it.
Maybe it is necessary to input moral and value monitoring inside products so that people use them in fervor not to harm others (maybe even in open source, so they could be so advanced that AI constructors should not use other sources). Some defense in the opportunity to create such things for themselves could be made. If someone could create a big graphical cluster or something like that, then they would have to seek help from advanced AI developers who apply basic precautions against existential threats. Some kind of red map needs to be drawn up so that the creators of the AI, or those who see its creation, can accurately see the signs that something is going completely wrong.
Of course, we cannot know what to do with solving GAI because we do not know what to expect, but maybe we could find something that will, with some probability, be good and identify what is completely wrong. Could we have at least red map? What could everyone do to be less wrong in it?
I have read this letter with pleasure. Pacifism in wartime is an extremely difficult position.
Survival rationality, humanity is extremely important!
It seems to me that the problem is very clearly revealed through compound percent (interest).
If in a particular year the probability of a catastrophe (man-made, biological, space, etc.) overall is 2%, then the probability of human survival in the next 100 years is 0.98 ^ 100 = 0.132,
That is 13.2%, this figure depresses me.
The ideas of unity and security are the only ones that are inside the discourse of red systems. Therefore, the ideas of security may well fundamentally hold together any parties. I think the idea of human survival is a priority.
Because it is clear to everyone that the preservation of humanity and rationals is extremely important, regardless of the specific picture of the world.
world peace!
If we take 1000 and 10000 years, then the result is unambiguous, survival tends to 0.
Therefore, I would like not to miss the chances that humanity can get through Artificial Intelligence or through Decentralized Blockchain Evolution, or quantum computing, or other positive black swans. We really need a qualitative breakthrough in the field of decentralized balancing of all systems.
Nevertheless, 86% of this game is almost lost by humanity
As we can see, the chances are small. Therefore, future generations of intelligent species will probably be happy if there are some convenient manuals for deciphering human knowledge.
What does the map of the arks look like? Can you imagine how happy it will be for a rational chimpanzee to hold your manual and flip through the pages of distant ancestors?
And to be amazed at how such an aggressive subspecies, thanks to aggression, intelligence developed faster and they defeated themself.
It is unlikely that they will have English. Language is a very flexible thing.
Probably the basis should be that basic development of Feynman and Carl Sagan, I'm talking about a satellite with the decoding of humanity, from "H". I think on Earth you can pick up points for such arks.
Due to the variety of risks, it seems to me that intelligent life will logically arise again under water, especially due to the fact that there are internal energy sources. Are there scientific arks for dolphins?
world peace! Respect for each other. We need great leap in another Integrity and Sustainability Ecosystem Equilibrium. A common understanding that this is the last century for mankind when it can overcome its natural aggression. Well, do not forget about the heritage of the following species.
peace to you! , I would be glad if you tell me where I'm right and where I'm wrong! Kind Regards!
I signed it.
Pacifism is really not in trend. Both sides of the conflict are convinced that they are absolute right: paranoid Russia, and a defensive Ukraine.
Public pacifism is in the minority. Almost everyone has taken one side, or is silent and seeks safety.
For an individual Ukrainian or Russian, it might be danger to sign this.
Like in ancient Roman Empire. People are either for Blue chariots or for Green ones. No one is interested in the opinion that death races are nonsense.
Anyway. It's irrational, but I signed