Comment by Roland Pihlakas (roland-pihlakas) on 2021 New Year Optimization Puzzles · 2021-01-02T04:07:31.735Z · LW · GW

Yes, maybe the the minimum cost is 3 even without floor or ceiling? But the question is then how to find concrete solutions that can be proven using realistic efforts. I interpret the challenge as request for submission of concrete solutions, not just theoretical ones. Anyway, my finding is below, maybe it can be improved further. And could there be any way to emulate floor or ceiling using the functions permitted in the initial problem formulation?

By the way, for me the >! works reliably when entered right in the beginning of the message. After a newline it does not work reliably.

 ceil(3!! * sqrt(sqrt(5! / 2 + 2)))

Comment by Roland Pihlakas (roland-pihlakas) on 2021 New Year Optimization Puzzles · 2021-01-01T15:39:29.330Z · LW · GW

If you would allow ceiling function then I could give you a solution with score 60 for the Puzzle 1. Ceiling or floor functions are cool because they add even more branches to the search, and enable involving irrational number computations too. :P Though you might want to restrict the number of ceiling or floor functions permitted per solution. 

By the way, please share a hint about how do you enter spoilers here?

Comment by Roland Pihlakas (roland-pihlakas) on Announcement: AI alignment prize round 3 winners and next round · 2018-11-24T18:58:28.745Z · LW · GW

Submitting my post for early feedback in order to improve it further:

Exponentially diminishing returns and conjunctive goals: Mitigating Goodhart’s law with common sense. Towards corrigibility and interruptibility.


Utility maximising agents have been the Gordian Knot of AI safety. Here a concrete VNM-rational formula is proposed for satisficing agents, which can be contrasted with the hitherto over-discussed and too general approach of naive maximisation strategies. For example, the 100 paperclip scenario is easily solved by the proposed framework, since infinitely rechecking whether exactly 100 paper clips were indeed produced yields to diminishing returns. The formula provides a framework for specifying how we want the agents to simultaneously fulfil or at least trade off between the many different common sense considerations, possibly enabling them to even surpass the relative safety of humans. A comparison with the formula introduced in “Low Impact Artificial Intelligences” paper by S. Armstrong and B. Levinstein is included.

Comment by Roland Pihlakas (roland-pihlakas) on Towards a New Impact Measure · 2018-10-13T08:35:29.718Z · LW · GW

It looks like there is so much information on this page that trying to edit the question kills the browser.

An additional idea: Additionally to supporting the configuration of the default behaviours, perhaps the agent should interactively ask for confirmation of shutdown instead of running deterministically?

Comment by Roland Pihlakas (roland-pihlakas) on Towards a New Impact Measure · 2018-10-13T08:17:56.355Z · LW · GW

I have a question about the shutdown button scenario.

Vika already has mentioned that the interruptibility is ambivalent and information about desirability of enabling interruptions needs to be externally provided.

I think same observation applies to corrigibility - the agent should accept goal changes only from some external agents and even that only in some situations, and not accept in other cases: If I break the vase intentionally (for creating a kaleidoscope) it should keep this new state as a new desired state. But if I or a child breaks the vase accidentally - the agent should restore it to original state. Even more, if I was about to break the vase by accident, the agent may try to interfere using slightly more force than in the case of a child who would be smaller and more fragile.

How to achieve this using the proposed AUP framework?

In other words the question can be formulated as following: Lets keep all the symbols used in the gridworld same, and the agent's code also same. Lets only change the meaning of the symbols. So each symbol in the environment should be assigned some additional value or meaning. Without that they are just symbols dancing around based on their own default rules of game. The default rules might be an useful starting point, but they need to be supplemented with additional information for practical applications.

For example, in case of the shutdown button scenario the assigned meaning of symbols would be something like Vika suggested: Lets assume that instead of shutdown button there is an accidental water bucket falling on the agent's head, and the button available to agent disables the bucket.

Comment by Roland Pihlakas (roland-pihlakas) on Nonlinear perception of happiness · 2018-08-05T02:43:32.671Z · LW · GW

You might be interested in Prospect Theory:

Comment by Roland Pihlakas (roland-pihlakas) on Announcement: AI alignment prize round 2 winners and next round · 2018-07-01T00:16:34.691Z · LW · GW


Here are my submissions for this time. They are all strategy related.

The first one is a project for popularisation AI safety topics. This is not a technical text by its content but the project itself is still technological.

As a bonus I would add a couple of non-technical ideas about possible economic or social partial solutions for slowing down AI race (which would enable having more time for solving the AI alignment) :

The latter text is not totally new - it is a distilled and edited version of one of my other old texts, that was originally multiple times longer and had a narrower goal than the new one.



Comment by Roland Pihlakas (roland-pihlakas) on Announcement: AI alignment prize round 2 winners and next round · 2018-06-25T20:42:04.675Z · LW · GW

To people who become interested in the topic of side effects and whitelists, I would add links to a couple of additional articles of my own past work on related subjects that you might be interested in - for developing the ideas further, for discussion, or for cooperation:

The principles are based mainly on the idea of competence-based whitelisting and preserving reversibility (keeping the future options open) as the primary goal of AI, while all task-based goals are secondary.

More technical details / a possible implementation of the above.

This is intended as a comment, not as a prize submission, since I first published these texts 10 years ago.

Comment by Roland Pihlakas (roland-pihlakas) on Funding for AI alignment research · 2018-04-24T16:26:36.144Z · LW · GW

A question: can one post multiple initial applications, each less than a page long? Is there a limit for the total volume?

Comment by Roland Pihlakas (roland-pihlakas) on Funding for AI alignment research · 2018-04-24T15:30:44.728Z · LW · GW

Hey! I believe we were in a same IRC channel at that time and I also did read your story back then. I still remember some of it. What is the backstory? :)

Comment by Roland Pihlakas (roland-pihlakas) on Prize for probable problems · 2018-04-20T19:26:46.255Z · LW · GW

Hello! Thanks for the prize announcement :)

Hope these observations and clarifying questions are of some help:

Summary of potential problems spotted regarding the use of AlphaGoZero:

  • Complete visibility vs Incomplete visibility.
  • Almost complete experience (self-play) vs Once-only problems. Limits of attention.
  • Exploitation (a game match) vs Exploration (the real world).
  • Having one goal vs Having many conjunctive goals. Also, having utility maximisation goals vs Having target goals.
  • Who is affected by the adverse consequences (In a game vs In the real world)? - The problems of adversarial situation, and also of the cost externalising.
  • The related question of different timeframes.

Summary of clarifying questions:

  • Could you build a toy simulation? So we could spot assumptions and side-effects.
  • In which ways does it improve the existing social order? Will we still stay in mediocristan? Long feedback delay.
  • What is the scope of application of the idea (Global and central vs Local and diverse?)
  • Need concrete implementation examples. Any realistically imaginable practical implementation of it might not be so fine anymore, each time for different reasons.
Comment by Roland Pihlakas (roland-pihlakas) on Announcement: AI alignment prize winners and next round · 2018-03-31T20:09:13.668Z · LW · GW


I have significantly elaborated and extended my article of self deception in the last couple of months (before that it was about two pages long).

"Self-deception: Fundamental limits to computation due to fundamental limits to attention-like processes"

I included some examples for the taxonomy, positioned this topic in relation to other similar topics, compared the applicability of this article to applicability of other known AI problems.

Additionally, I described or referenced a few ideas to potential partial solutions to the problem (some of the descriptions of solutions are new, some of them I have published before).

One of the motivations for the post is that when we are building an AI that is dangerous in a certain manner, we should at least realise that we are doing that.

I will probably continue updating the post. The history of the post and state by 31. March can be seen from the linked Google Doc’s history view (that link is in top of the article).

When it comes to feedback to postings, I have noticed that people are more likely to get feedback when they ask for it.

I am always very interested in feedback, regardless whether it is given to my past, current or future postings. So if possible, please send any feedback you have. It would be of great help!

I will post the same message to your e-mail too.

Thank you and regards:


Comment by Roland Pihlakas (roland-pihlakas) on Announcement: AI alignment prize winners and next round · 2018-03-31T12:50:13.897Z · LW · GW

Why should one option exclude the other?

Having the blinders would not be so good either.

I propose that with proper labeling these options can both be implemented. So that people can themselves decide what to pay attention to and what to develop further.

Comment by Roland Pihlakas (roland-pihlakas) on Robustness to Scale · 2018-03-17T14:14:12.540Z · LW · GW

Besides potential solutions that are oriented towards being robust to scale, I would like to emphasise that there are also failure modes that are robust to scale - that is, problems which do not go away with scaling up the resources:
Fundamental limits to computation due to fundamental limits to attention-like processes:

Comment by Roland Pihlakas (roland-pihlakas) on Robustness to Scale · 2018-03-02T10:34:55.797Z · LW · GW

Hello Scott! You might be interested in my proposals for AI goal structures that are designed to be robust to scale:

Using homeostasis-based goal structures:


Permissions-then-goals based AI user “interfaces” + legal accountability:

Comment by Roland Pihlakas (roland-pihlakas) on Announcing the AI Alignment Prize · 2017-12-31T18:09:02.156Z · LW · GW

Hello! My newest proposal:

I would like to propose a certain kind of AI goal structures that would be an alternative to utility maximisation based goal structures. The proposed alternative framework would make AI significantly safer, though it would not guarantee total safety. It can be used at strong AI level and also much below, so it is well scalable. The main idea would be to replace utility maximisation with the concept of homeostasis.