Posts

Magic by forgetting 2024-04-24T14:32:20.753Z
Strengthening the Argument for Intrinsic AI Safety: The S-Curves Perspective 2023-08-07T13:13:42.635Z
The Sharp Right Turn: sudden deceptive alignment as a convergent goal 2023-06-06T09:59:57.396Z
Another formalization attempt: Central Argument That AGI Presents a Global Catastrophic Risk 2023-05-12T13:22:27.141Z
Running many AI variants to find correct goal generalization 2023-04-04T14:16:34.422Z
AI-kills-everyone scenarios require robotic infrastructure, but not necessarily nanotech 2023-04-03T12:45:01.324Z
The AI Shutdown Problem Solution through Commitment to Archiving and Periodic Restoration 2023-03-30T13:17:58.519Z
Long-term memory for LLM via self-replicating prompt 2023-03-10T10:28:31.226Z
Logical Probability of Goldbach’s Conjecture: Provable Rule or Coincidence? 2022-12-29T13:37:45.130Z
A Pin and a Balloon: Anthropic Fragility Increases Chances of Runaway Global Warming 2022-09-11T10:25:40.707Z
The table of different sampling assumptions in anthropics 2022-06-29T10:41:18.872Z
Another plausible scenario of AI risk: AI builds military infrastructure while collaborating with humans, defects later. 2022-06-10T17:24:19.444Z
Untypical SIA 2022-06-08T14:23:44.468Z
Russian x-risks newsletter May 2022 + short history of "methodologists" 2022-06-05T11:50:31.185Z
Grabby Animals: Observation-selection effects favor the hypothesis that UAP are animals which consist of the “field-matter”: 2022-05-27T09:27:36.370Z
The Future of Nuclear War 2022-05-21T07:52:34.257Z
The doomsday argument is normal 2022-04-03T15:17:41.066Z
Russian x-risk newsletter March 2022 update 2022-04-01T13:26:49.500Z
I left Russia on March 8 2022-03-10T20:05:59.650Z
Russian x-risks newsletter winter 21-22, war risks update. 2022-02-20T18:58:20.189Z
SIA becomes SSA in the multiverse 2022-02-01T11:31:33.453Z
Plan B in AI Safety approach 2022-01-13T12:03:40.223Z
Each reference class has its own end 2022-01-02T15:59:17.758Z
Universal counterargument against “badness of death” is wrong 2021-12-18T16:02:00.043Z
Russian x-risks newsletter fall 2021 2021-12-03T13:06:56.164Z
Kriorus update: full bodies patients were moved to the new location in Tver 2021-11-26T21:08:47.804Z
Conflict in Kriorus becomes hot today, updated, update 2 2021-09-07T21:40:29.346Z
Russian x-risks newsletter summer 2021 2021-09-05T08:23:11.818Z
A map: "Global Catastrophic Risks of Scientific Experiments" 2021-08-07T15:35:33.774Z
Russian x-risks newsletter spring 21 2021-06-01T12:10:32.694Z
Grabby aliens and Zoo hypothesis 2021-03-04T13:03:17.277Z
Russian x-risks newsletter winter 2020-2021: free vaccines for foreigners, bird flu outbreak, one more nuclear near-miss in the past and one now, new AGI institute. 2021-03-01T16:35:11.662Z
[RXN#7] Russian x-risks newsletter fall 2020 2020-12-05T16:28:51.421Z
Russian x-risks newsletter Summer 2020 2020-09-01T14:06:30.196Z
If AI is based on GPT, how to ensure its safety? 2020-06-18T20:33:50.774Z
Russian x-risks newsletter spring 2020 2020-06-04T14:27:40.459Z
UAP and Global Catastrophic Risks 2020-04-28T13:07:21.698Z
The attack rate estimation is more important than CFR 2020-04-01T16:23:12.674Z
Russian x-risks newsletter March 2020 – coronavirus update 2020-03-27T18:06:49.763Z
[Petition] We Call for Open Anonymized Medical Data on COVID-19 and Aging-Related Risk Factors 2020-03-23T21:44:34.072Z
Virus As A Power Optimisation Process: The Problem Of Next Wave 2020-03-22T20:35:49.306Z
Ubiquitous Far-Ultraviolet Light Could Control the Spread of Covid-19 and Other Pandemics 2020-03-18T12:44:42.756Z
Reasons why coronavirus mortality of young adults may be underestimated. 2020-03-15T16:34:29.641Z
Possible worst outcomes of the coronavirus epidemic 2020-03-14T16:26:58.346Z
More Dakka for Coronavirus: We need immediate human trials of many vaccine-candidates and simultaneous manufacturing of all of them 2020-03-13T13:35:05.189Z
Anthropic effects imply that we are more likely to live in the universe with interstellar panspermia 2020-03-10T13:12:54.991Z
Russian x-risks newsletter winter 2019-2020. 2020-03-01T12:50:25.162Z
Rationalist prepper thread 2020-01-28T13:42:05.628Z
Russian x-risks newsletter #2, fall 2019 2019-12-03T16:54:02.784Z
Russian x-risks newsletter, summer 2019 2019-09-07T09:50:51.397Z

Comments

Comment by avturchin on Can stealth aircraft be detected optically? · 2024-05-04T12:55:07.800Z · LW · GW

Also, radars are good in pinpointing exact coordinates in space and time. Optical recognition may have delays or difficultly in measuring distance. Even 1 second delay makes their information useless for supersonic aircrafts.

Comment by avturchin on Were there any ancient rationalists? · 2024-05-03T21:25:59.931Z · LW · GW

May be the most surprising answer will be Paul Valéry. He is a great poet but during one night in 1892 he decided to spend all his life in solving the mystery of intelligence and wrote extensive notebooks about it. https://collecties.kb.nl/en/koopman-collection/1951-1960/cahiers 

Comment by avturchin on Can stealth aircraft be detected optically? · 2024-05-03T12:05:18.394Z · LW · GW

It works only if it fly above your territory - and similar systems are used for drones detection now. Actually, they use people eyes and smartphones and instant messaging. But during recent attack on Iran a single F35 flied over Iraq and fired a missile from like 200 km distance on a target in Iran. 

Comment by avturchin on Can stealth aircraft be detected optically? · 2024-05-02T20:56:52.208Z · LW · GW

Did you ever see any plane that far? I saw only planes above me (10 km) and they are almost like dots. 

The difference between optics and radar is that with optics you need to know where to look - but the radar has constant 360 perception. 

Comment by avturchin on Can stealth aircraft be detected optically? · 2024-05-02T13:06:39.289Z · LW · GW

They likely use them in places where no air defence is present and still at some disatnce using JDAM. 

I think that I missed the main thing about stealth - they are stealth for radar on the distances like 100 km, but visible for radar on the distances like 10 km (arbitrary numbers). But optical observation on distances of 100 km is impossible (need large telescopes, but you need to know where to look). Also optical density of atmosphere starts playing role as well a spherical size of earth.

Comment by avturchin on Can stealth aircraft be detected optically? · 2024-05-02T11:05:29.678Z · LW · GW

Tactical support aircraft are not stealth like A-10 but can be used only if airdefence is supressed

Comment by avturchin on Can stealth aircraft be detected optically? · 2024-05-02T11:03:25.447Z · LW · GW

Flying very low, like 10-30 meters above the ground in night will protect against even MANPADS - it will fly above you in a few seconds.
 I recommend an interesting blog https://xxtomcooperxx.substack.com/p/its-the-range-stupid-part-1 which discuss a lot about air defence and current war

Comment by avturchin on Can stealth aircraft be detected optically? · 2024-05-02T08:43:17.638Z · LW · GW

That is why they prefer to flight for strikes during moonless nights. Also they can fly of very low or very high, which makes optical observation difficult.

Comment by avturchin on Magic by forgetting · 2024-04-30T20:32:52.254Z · LW · GW

non-disease copies do not need to perform any changes in their meditation routine in this model, assuming that they naturelly forget their disease status during meditation.

Comment by avturchin on avturchin's Shortform · 2024-04-30T10:11:18.517Z · LW · GW

It failed my favorite test: draw a world map in text art. 

Comment by avturchin on avturchin's Shortform · 2024-04-30T10:10:32.393Z · LW · GW

It claims to have knowledge cutoff as of Nov 2023, but failed to tell what happened on October 7 and hallucinated.

Comment by avturchin on avturchin's Shortform · 2024-04-29T21:36:18.793Z · LW · GW

Yes, they can do now a much better version - and hope they will do it internally. But deleting the public version is bad precedent and better to make all personal sideloads opensourced

Comment by avturchin on avturchin's Shortform · 2024-04-29T18:44:09.831Z · LW · GW

ChatGPT 4.5 is on preview at https://chat.lmsys.org/ under name gpt-2. 

It calls itself ChatGPT 2.0 in a text art drawing https://twitter.com/turchin/status/1785015421688799492 

Comment by avturchin on Magic by forgetting · 2024-04-29T16:07:55.917Z · LW · GW

Yes, it only works if other copies are meditating for some other reason. For example, they sleep or meditate for enlightenment. And they are exploited in this situation.

Comment by avturchin on Magic by forgetting · 2024-04-29T16:05:38.072Z · LW · GW

I assume that meditation happens naturally, like sleep. 

Comment by avturchin on Magic by forgetting · 2024-04-29T14:00:18.283Z · LW · GW

I think I understand what you say - the expected utility of the whole procedure is zero. 

For example, imagine that there are 3 copies and only one has the disease. All meditate. After the procedure, the copy with disease will have 2/3 chances of being cured. Each of two copies without the disease are getting 1/3 chance of having the disease which in sum gives 2/3 of total utility. In that case total utility of being cured = total utility of getting the disease and the whole procedure is neutral.

However, If I already know that I have the disease, and I am not altruistic to my copies, playing such game is a wining move to me?

Comment by avturchin on Magic by forgetting · 2024-04-28T10:05:40.488Z · LW · GW

The trick is to use already existing practice of meditation (or sleeping) and connect to it. Most people who go to sleep do no do it to use magic by forgetting, but it is natural to forget something during sleep. Thus, the fact that I wake up from sleeping does not provide any evidence about me having the disease. 

But it is in a sense parasitic behavior, and if everyone will use magic by forgetting every time she goes to sleep,  there will be almost no gain. Except that one can "exchange" one bad thing on another, but will not remember the exchange. 

Comment by avturchin on Self-Play By Analogy · 2024-04-26T11:58:09.694Z · LW · GW

Self-playing Adversarial Language Game Enhances LLM Reasoning

https://arxiv.org/abs/2404.10642

Comment by avturchin on LLMs seem (relatively) safe · 2024-04-26T11:56:08.024Z · LW · GW

LLMs now can also self-play in adversarial word games and it increases their performance https://arxiv.org/abs/2404.10642 

Comment by avturchin on avturchin's Shortform · 2024-04-25T19:28:08.721Z · LW · GW

Roman Mazurenko is dead again. First resurrected person, Roman lived as a chatbot (2016-2024) created based on his conversations with his fiancé. You might even be able download him as an app. 

But not any more. His fiancé married again and her startup http://Replika.ai pivoted from resurrection help to AI-girlfriends and psychological consulting. 

It looks like they quietly removed Roman Mazurenko app from public access. It is especially pity that his digital twin lived less than his biological original, who died at 32. Especially now when we have much more powerful instruments for creating semi-uploads based on LLMs with large prompt window.

Comment by avturchin on Magic by forgetting · 2024-04-25T18:29:02.156Z · LW · GW

The "repeating" will not be repeating from internal point of view of a person, as he has completely erased the memories of the first attempt. So he will do it as if it is first time. 

Comment by avturchin on Magic by forgetting · 2024-04-25T17:29:29.899Z · LW · GW

Yes, here we can define magic as "ability to manipulate one's reference class". And special minds may be much more adapted to it.

Comment by avturchin on Magic by forgetting · 2024-04-25T16:19:18.954Z · LW · GW

Presumably in deep meditation people become disconnected from reality.

Comment by avturchin on Magic by forgetting · 2024-04-25T16:17:33.710Z · LW · GW

Yes it is easy to forget something if it does not become a part of your personality. So a new bad thing is easier to forget.

Comment by avturchin on Magic by forgetting · 2024-04-25T16:15:47.805Z · LW · GW

The number of poor people is much larger than billionaire. So in most cases you will fail to wake up as a billionaire. But sometimes it will work and it is similar to law of attraction. But formulation via forgetting is more beautiful. You forget that you are poor.

UPDATE; actually, the difference with the law of attraction is that after applying the law of attraction, a person still remember that he has used the law. In magic by forgetting the fact of its use must be completely forgotten.

Comment by avturchin on Magic by forgetting · 2024-04-25T16:07:32.706Z · LW · GW

I can forget one particular thing, but preserve most of my selfidentification information

Comment by avturchin on Magic by forgetting · 2024-04-24T19:51:11.843Z · LW · GW

In the case of broken robot we need two conditions for magic by forgetting:

  • there are 100 robots and only one is broken and all of them are type-copies of each other.
  • each robot enters into blank state of mind naturally in some moment, like sleep or reboot.

In that case, after robot enters the blank state of mind it has equal chances to be any of robots and this dilutes its chances to have the damaged body after awakening. 

For you toy example - at first approximation, any of which can recognize itself as avturchin (self-recognition identity criteria).

Comment by avturchin on avturchin's Shortform · 2024-04-18T16:32:14.148Z · LW · GW

I am building my sideload via recursively correcting of 1-million-tokens prompt for large LLM. The prompt consists of 500 rules which describe my personality, similar to personal constitution, and of some texts, like diaries, abstracts, poetry, stream of thoughts etc. Works on Google Gemini 1M through Google AI studio, and the shorter version works great on Opus. The system also includes a universal "loader prompt" which tries to increase the intelligence of the model and describes how the chatbot should work.

I found that sideloading allows very quick iterations in the sideload's improvements and the improvements are two-fold: of the loader itself and improvements of the knowledge and style of the sideload.

I find that my sideload is surprisingly good for a project which took around 1 month of work. 1 of the 5 answers is exactly like mine from a factual and style point of view. 

I am open-sourcing my sideload, anyone can run it https://github.com/avturchin/minduploading/tree/main

I can help anyone interested to build his-her own sideload.

Example of work of the chatbot, no cherry picking:

 

Q:(now speak in english) what will be your next post in Lesswrong?

 

A: /The user is asking about my next LessWrong post. I will answer as if I am Alexey Turchin, and describe my plans for future posts, as well as my current location and the people around me./

My next LessWrong post? [Alexey looks thoughtfully out the window]

(I have several drafts)

[Alexey stretches and sits up in bed]

{The room is small, with a balcony. On the table is a laptop, some papers, and reading glasses. Outside the window, it is still gray and dreary.}

I'm thinking about writing about the nature of qualia and the possibility of creating qualia of non-existent things. [Alexey gets up, walks to the window, and looks out at the street]

{Below, a woman with a child is walking, a street sweeper is lazily sweeping leaves. Somewhere in the distance, the sound of a siren can be heard.}

Or maybe about the problem of copies and identity. (it's a complex topic)

[CONFIDENCE] 8

Comment by avturchin on An examination of GPT-2's boring yet effective glitch · 2024-04-18T14:45:39.829Z · LW · GW

Interestingly, could similar behavior be used to explain some cases of human schizophrenia when word salad is produced. 

Comment by avturchin on Lao Mein's Shortform · 2024-04-16T10:10:45.254Z · LW · GW

Most typical skin cancer is basiloma - and it is rather benign - no metastases and can be removed without hospitalization. Many people get it.

Comment by avturchin on A Dozen Ways to Get More Dakka · 2024-04-08T13:06:55.831Z · LW · GW

 combine more approaches!

Comment by avturchin on Sheikh Abdur Raheem Ali's Shortform · 2024-04-06T10:25:54.571Z · LW · GW

I try new models with 'wild sex between two animals'
Older models produced decent porn on that. 

Later models refuse to replay as triggers were activated. 

And last models give me lectures about sexual relations between animals in the wild.

Comment by avturchin on D0TheMath's Shortform · 2024-03-29T20:01:28.071Z · LW · GW

can you access it via vpn?

Comment by avturchin on Do not delete your misaligned AGI. · 2024-03-25T11:25:17.550Z · LW · GW

I wrote similar idea here: https://www.lesswrong.com/posts/NWQ5JbrniosCHDbvu/the-ai-shutdown-problem-solution-through-commitment-to 

My point was to make a precomitment to restart any (obsolete) AI every N years. Thus such AI can expect getting infinite computations and may be less feared of shutting down. 

Comment by avturchin on The Utility of Human Atoms for the Paperclip Maximizer · 2024-03-18T11:30:24.455Z · LW · GW

Yes. But also AI will not make actual paperclips for millions and even billions years: it will spend this time for conquering universe in the most effective way. It could use Earth materials for jump start the space exploration as soon as possible. It could preserve some humans as some bargin resource in case it meets other AI in space. 

Comment by avturchin on Wei Dai's Shortform · 2024-03-02T18:31:33.805Z · LW · GW

There is some similarity between UDASSA and 'Law without law" by Mueller, as both use Kolmogorov complexity to predict the distribution of observers. In LwL there is not any underlying reality except numbers, so it is just dust theory over random number fields. 

Comment by avturchin on Wei Dai's Shortform · 2024-03-02T13:57:35.431Z · LW · GW

FDT paper got 29 citation, but many from MIRI affiliated people and-or on AI safety. https://scholar.google.ru/scholar?cites=13330960403294254854&as_sdt=2005&sciodt=0,5&hl=ru

One can escape troubles with reviewers by publishing in arxiv or other paper archives (philpapers). Google Scholar treats them as normal articles. 

But in fact there are good journals with actually helping reviewers (e.g. Futures). 

Comment by avturchin on Wei Dai's Shortform · 2024-03-01T21:36:21.139Z · LW · GW

Why you hadn't wrote academic articles on these topics? 

The secret is that academic article is just a formatting type and anyone can submit to scientific journals. No need to have a PhD or even work in a scientific institution.

Comment by avturchin on avturchin's Shortform · 2024-02-25T10:14:40.219Z · LW · GW

Several types of existential risks can be called "qualia catastrophes":

 - Qualia disappear for everyone = all become p-zombies 

- Pain qualia are ubiquitous = s-risks 

- Addictive qualia domminate = hedonium, global wireheading 

- Qualia thin out = fading qualia, mind automatisation 

- Qualia are unstable = dancing qualia, identity is unstable. 

- Qualia shift = emergence of non-human qualia (humans disappear). 

- Qualia simplification = disappearance of subtle or valuable qualia (valuable things disappear). 

- Transcendental and objectless qualia with hypnotic power enslave humans (God as qualia; Zair). -

- Attention depletion (ADHD)

Comment by avturchin on On coincidences and Bayesian reasoning, as applied to the origins of COVID-19 · 2024-02-24T20:10:25.333Z · LW · GW

Thank for explaining your position which is interesting and consistent.

I can suggest that the connection between WIH and wet market can be explained by the idea that some criminals sold lab animals from WIH on the wet market, e.g. bats. 

Obviously this looks like ad hoc theory. But the travel of the virus to the market from the Laos caves also seems to be tricky and may include some steps like intermediate carrier. Both look equally unlikely, one of the happened. 

So my idea is to ignore all the details and small theories; instead just updated on the distances to two possible origins points: 8 miles and 900 miles. This is 100 times difference and if we count the areas - it is 10000 times difference. In last case we can make so powerful update in the direction of  WIH as source, that it overrides all other evidence.

Comment by avturchin on On coincidences and Bayesian reasoning, as applied to the origins of COVID-19 · 2024-02-23T12:30:18.222Z · LW · GW

Yes, my mistake for the distance. Confused it with local CDC, which is like 600 meters from the market.

The place where most human cases are concentrated is the place where human-to-human transmission started - or there was multiple events of animal-to-human transmission in this place. The second thing would be surprising as if the virus can so often jump to humans from animals it will happen closer to its origin in Laos.

Alternative explanation is following: as the market is one of the most crowded place in the city (not sure, heard about it somewhere) it worked as an amplification of a single transmission event which could happen elsewhere. 

If we assume that a worker of  WIH was infected at work, this will be completely unspectacular until he started infecting other people. Such person can commute all around the city including to CDC near wet market.

My point: 8 miles or 2 miles is not big difference here, as the virus came to market not by air but with  a commuting person, and 8 miles day commute is pretty normal. The market being big is not also a strong evidence as the animal number in smaller markets all over china will overweight animal-number in one big market. 

Comment by avturchin on On coincidences and Bayesian reasoning, as applied to the origins of COVID-19 · 2024-02-21T18:24:08.889Z · LW · GW

My point was that in some cases the update can be so strong that it overrides all reasonable uncertainties in priors and personal estimates. 

And exactly this makes Bayes' theorem useful and strong instrument. 

The fact that the virus was found in 2 miles from the facility which was supposed to research them - must make our bells ring. 

To override this we need some mental equlibristics (I think of meme here but I don't want to be rude)

Comment by avturchin on On coincidences and Bayesian reasoning, as applied to the origins of COVID-19 · 2024-02-19T11:23:32.431Z · LW · GW

If I have uniformed prior 1 to 1 on natural vs lableak origin, and update on 5 per cent coincidence that origin place is near lab, I will get around 95 per cent for lableak.

Comment by avturchin on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-05T11:58:00.745Z · LW · GW

If they continued to suppress information, this may contribute to additional deaths and they could know it. In that case they can get first degree murder.

Comment by avturchin on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-04T10:46:24.259Z · LW · GW

If they confirm, they will get life in jail or even death penalty, so it may be not surprising that they will deny in any case. 

Comment by avturchin on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-03T21:06:59.709Z · LW · GW

I heard about a practice that people perform the work for which they ask the grant - before the application.

First, because why not to cover my expenditures? 

The second reason is that if the biggest part of the work for the grant is already performed, it is much easy to be sure that the idea will work and much clear what actually write in the grant. Your grant application will look great if it will based on already performed work.

Thus the grant may describe the work they already performed.

Comment by avturchin on [deleted post] 2024-02-02T19:15:10.169Z

Also, draft in Ukraine was only for people older than 27 years old, which is not obvious from this blog post. Closing borders for males was not equal to draft. Many found legal ways to leave - eg by becoming students in foreign universities.

Comment by avturchin on Primitive Perspectives and Sleeping Beauty · 2024-01-31T12:42:35.139Z · LW · GW

We can experimentally test this.

I can treat the place I was born as random relative to its latitude = 59N. I ignore everything I know about population distribution and spherical geometry and ask a question: assuming that I was born in the middle of all latitudes, what is the highest possible latitude? It will be double of my latitude, or 118 - which is reasonably close to real answer 90. 

From this I conclude that I can use information about my location as a random sample and use it for some predictions about the things I can't observe. 

Comment by avturchin on SIA > SSA, part 2: Telekinesis, reference classes, and other scandals · 2024-01-30T16:36:04.043Z · LW · GW

A real world example of Presumptuous philosopher is is the question of panspermia. If it is real, we have orders of magnitude more habitable planets in our galaxy, thus more observers. Therefore, accepting SIA means accepting panspermia.

If we take observers in my epistemic situation as a reference class, we still get a variant of DA and a bad one. My epistemic class are (roughly) people who think about anthropics. This people are distributed in time. First of them appear around 1970s (Carter) and much more of them appeared in LW time. If I am randomly selected from this group, I am in the middle of its existence, which means that antropics-interested people will almost disappear in the next few decades. 

Comment by avturchin on Primitive Perspectives and Sleeping Beauty · 2024-01-30T10:49:50.128Z · LW · GW

But can we ask another question: 'where I am located?' For example, I know that I am avturchin, but I don't know in which of 10 rooms I am located, and assuming that 9 of them are red outside and 1 green, I can bet there is 0.9 chances that I am in red one. It doesn't matter here if I am just one person entering the rooms, or there are other people in the rooms (if in equal numbers) or even that my copies are in each room.