Posts

Sayan's Braindump 2019-09-04T08:51:56.549Z · score: 2 (1 votes)
What is your Personal Knowledge Management system? 2019-07-16T20:20:05.558Z · score: 16 (8 votes)
Unknown Unknowns in AI Alignment 2019-06-14T05:07:21.193Z · score: 6 (3 votes)

Comments

Comment by sayan on Value Impact · 2019-09-24T08:08:21.650Z · score: 10 (4 votes) · LW · GW

As far as I understand, this post decomoses 'impact' into value impact and objective impact. VI is dependent on some agent's ability to reach arbitrary value-driven goals, while OI depends on any agent's ability to reach goals in general.

I'm not sure if there exists a robust distinction between the two - the post doesn't discuss any general demarcation tool.

Maybe I'm wrong, but I think the most important point to note here is that 'objectiveness' of an impact is defined not to be about the 'objective state of the world' - rather about how 'general to all agents' an impact is.

Comment by sayan on Reframing Impact · 2019-09-21T06:49:04.336Z · score: 7 (3 votes) · LW · GW

I think this post is broadly making two claims -

  1. Impactful things fundamentally feel different.

  2. A good Impact Measure should be designed in a way that it strongly safeguards against almost any imperfect objective.

It is also (maybe implicitly) claiming that the three properties mentioned completely specify a good impact measure.

I am looking forward to reading the rest of the sequence with arguments supporting these claims.

Comment by sayan on Sayan's Braindump · 2019-09-18T09:17:23.574Z · score: 1 (1 votes) · LW · GW

What gadgets have improved your productivity?

For example, I started using a stylus few days ago and realized it can be a great tool for a lot of things!

Comment by sayan on What are some of your "Crazy Ideas" that you're currently thinking about? · 2019-09-18T09:12:00.373Z · score: 2 (2 votes) · LW · GW

I am thinking about these questions about a lot without actually reaching anywhere.

What is the nature of non-dual epistemology? What does it mean to 'reason' from the (Intentional Stance)[https://en.wikipedia.org/wiki/Intentional_stance], from inside of an agent?

Comment by sayan on Sayan's Braindump · 2019-09-05T15:37:35.986Z · score: 3 (2 votes) · LW · GW

Seems like this has been done already.

https://www.alignmentforum.org/posts/yXPT4nr4as7JvxLQa/classifying-specification-problems-as-variants-of-goodhart-s

Comment by sayan on Sayan's Braindump · 2019-09-05T14:50:38.043Z · score: 1 (1 votes) · LW · GW

Okay, natural catastrophes might not be a good example. (Edited)

Comment by sayan on Sayan's Braindump · 2019-09-04T09:07:28.573Z · score: 1 (1 votes) · LW · GW

If there is no self, what are we going to upload to the cloud?

Comment by sayan on Sayan's Braindump · 2019-09-04T09:03:58.701Z · score: 2 (3 votes) · LW · GW

It is so difficult to understand the difference and articulate in pronunciation some accent that is not one's native, because of the predictive processing of the brain. Our brains are constantly appropriating signals that are closely related to the known ones.

Comment by sayan on Sayan's Braindump · 2019-09-04T09:02:54.300Z · score: 3 (2 votes) · LW · GW

Is there a good bijection between specification gaming and wireheading vs different types of Goodhart's law?

Comment by sayan on Sayan's Braindump · 2019-09-04T09:01:55.031Z · score: 8 (4 votes) · LW · GW

Extremely low probability events are great as intuition pumps, but terrible as real world decisionmaking.

Comment by sayan on Sayan's Braindump · 2019-09-04T09:00:10.393Z · score: 3 (5 votes) · LW · GW

Speculation: People never use pro-con lists to actually make decisions, they rather use them rationalizingly to convince others.

Comment by sayan on Sayan's Braindump · 2019-09-04T08:59:05.731Z · score: 3 (3 votes) · LW · GW

The internet might be lacking multiple kind of curation and organization tools? How can we improve?

Comment by sayan on Sayan's Braindump · 2019-09-04T08:58:03.213Z · score: 1 (1 votes) · LW · GW

Pathological examples of math are analogous to adversarial examples in ML. Or are they?

Comment by sayan on Sayan's Braindump · 2019-09-04T08:57:31.481Z · score: 1 (1 votes) · LW · GW

What are the possible failure modes of AI-aligned Humans? What are the possible misalignment scenarios? I can think of malevolent uses of AI tech to enforce hegemony and etc etc. What else?

Comment by sayan on Sayan's Braindump · 2019-09-04T08:56:52.318Z · score: 1 (1 votes) · LW · GW

What's a good way to force oneself outside their comfort zone where most expectations and intuitions routinely fail?

This might become useful to build antifragility about expectation management.

Quick example - living without money in a foreign nation.

Is it possible to design a personal or group retreat for this?

Comment by sayan on Sayan's Braindump · 2019-09-04T08:55:24.374Z · score: 6 (2 votes) · LW · GW

Would CIRL with many human agents realistically model our world?

What does AI alignment mean with respect to many humans with different goals? Are we implicitly assuming (with all our current agendas) that the final model of AGI is to being corrigible with one human instructor?

How do we synthesize goals of so many human agents into one utility function? Are we assuming solving alignment with one supervisor is easier? Wouldn't having many supervisors restrict the space meaningfully?

Comment by sayan on Sayan's Braindump · 2019-09-04T08:53:21.081Z · score: 1 (3 votes) · LW · GW

Where is the paradigm for Effective Activism? On a first thought, it doesn't even seem to be difficult to do better than status quo.

Comment by sayan on Towards a New Impact Measure · 2019-09-04T08:28:26.060Z · score: 8 (5 votes) · LW · GW

Quick question. Given that now the Conservative Agency paper is available, what am I missing if I just read the paper and not this post? It seems easier to me to follow the notations of the paper. Is there any significant difference between the formalization of this post and the paper?

Comment by sayan on What is your Personal Knowledge Management system? · 2019-07-18T05:36:06.279Z · score: 1 (1 votes) · LW · GW

I read books on multiple devices - GNU/Linux, Android, and Kindle. Last time I checked, Calibre was too feature-rich and heavy, but lacked a simple getting-out-of-my way workflow for syncing my reading between devices. Is there a better solution now?

Comment by sayan on What is your Personal Knowledge Management system? · 2019-07-17T06:00:52.421Z · score: 1 (1 votes) · LW · GW

I love how you emphasized learning Unix tools. I use other things mentioned here except tmux. Would you be willing to share your tmux workflow in more detail with keybindings?

Comment by sayan on Overcoming Akrasia/Procrastination - Volunteers Wanted · 2019-07-16T17:22:34.787Z · score: 1 (1 votes) · LW · GW

I am interested!

Comment by sayan on Open Thread September 2018 · 2018-09-02T11:27:10.318Z · score: 2 (4 votes) · LW · GW

Just finished reading Yuval Noah Harari's new book 21 Lessons for the 21st Century. Primary reaction: even if you already know all the things being presented in the book, it is worth a read just because of the clarity into the discussion the book offers.

Comment by sayan on New paper: Long-Term Trajectories of Human Civilization · 2018-08-13T17:43:40.699Z · score: 7 (4 votes) · LW · GW

This is an amazingly comprehensive and useful paper. I wish it was longer with little summaries of some papers it references, rather than just citing them.

I also wish somebody creates a video version of it in the spirit of CGP Grey's video on the classic Bostrom paper, so that I can just redirect people to the video instead of sub-optimally trying to explain all these things myself.

Comment by sayan on Open Thread August 2018 · 2018-08-13T04:12:44.651Z · score: 1 (1 votes) · LW · GW

Shared the draft with you. Please let me know your feedback.

Comment by sayan on Open Thread August 2018 · 2018-08-12T13:05:01.046Z · score: 1 (1 votes) · LW · GW

Shared the draft with you. Feel free to comment and question.

Comment by sayan on Open Thread August 2018 · 2018-08-09T08:17:57.335Z · score: 8 (5 votes) · LW · GW

I have started to write a series of rigorous introductory blogposts on Reinforcement Learning for people with no background in it. This is totally experimental and I would love to have some feedback on my draft. Please let me know if anyone is interested.