Pattern's Shortform Feed 2019-05-30T21:21:23.726Z · score: 13 (3 votes)


Comment by pattern on NaiveTortoise's Short Form Feed · 2020-04-03T16:51:52.695Z · score: 2 (1 votes) · LW · GW

Solutions might be better to go with than proofs - if the answer is wrong, that's more straightforward to show that whether or not a proof is wrong.

Comment by pattern on Two Alternatives to Logical Counterfactuals · 2020-04-02T07:10:39.856Z · score: 2 (1 votes) · LW · GW

Source code doesn't entirely determine the result, inputs are also required.* Thus "logical counterfactuals" -reasoning about what a program will return if I input y? This can be done by asking 'if I had input y instead of x' or 'if I input y' even if I later decide to input x.

While it can be said that such considerations render one's "output" conditional on logic, they remain entirely conditional on reasoning about a model, which may be incorrect. It seems more useful to refer to such a relation as conditional on one's models/reasoning, or even processes in the world. A calculator may be misused - a 2 instead of a 3 here, hitting "=" one too many times, there, etc.

(Saying it is impossible for a rational agent that knows X to do Y, and agent A is not doing Y, does not establish that A is irrational - even if the premises are true, what follows is that A is not rational or does not know X.)

*Unless source code is defined as including the inputs.

Comment by pattern on Two Alternatives to Logical Counterfactuals · 2020-04-02T06:58:38.791Z · score: 2 (1 votes) · LW · GW

I think

  • when modeling a complex/not entirely understood system, probabilities may be a more effective framework.
  • Just as, if the output of a program were known before it was run, it probably wouldn't need to be run, we don't know what we'll decide before we decide, though we do after, and we're not sure how we could have predicted the outcome in advance.
Comment by pattern on Two Alternatives to Logical Counterfactuals · 2020-04-02T06:55:09.866Z · score: 2 (1 votes) · LW · GW

If the agent is 'caused' then in order for its source code to be different, something about the process that produced it must be different. (I haven't seen this addressed.)

Comment by pattern on Two Alternatives to Logical Counterfactuals · 2020-04-02T06:54:00.795Z · score: 2 (1 votes) · LW · GW


Hence, they would not have seen that their source code is "A".

Unless something interfered with what they saw - there need not be pure/true observations.

Instead, if the agent were to take action Y upon seeing that their source code is "A", their source code must be something else, perhaps "B".

And something might have incentive to do so if the agent were to do X if it "saw its source code was A" and were to do Y if it "saw its source code was B". While A and B may be mutually exclusive, the actual policy "might" be dependent on observations of either.


[1] If a program takes long enough to run, it may never be found that it does halt. In a sense, the fact that its output is determined does not mean it can (or will) be deduced.

there is no way for two different policies to be compatible with the same source code.

And set of inputs.

Formally, a policy is a function mapping an observation history to an action. It is distinct from source code, in that the source code specifies the implementation of the policy in some programming language, rather than itself being a policy function.
Logically, it is impossible for the same source code to generate two different policies. There is a fact of the matter about what action the source code outputs given an observation history (assuming the program halts). Hence there is no way for two different policies to be compatible with the same source code.

Overall take:

Dynamic versus static:

Consider the numbers 3, 1, 2, 4.

There exists more than one set of actions that 'transforms' the above into: 1, 2, 3, 4.

(It can also be transformed into a sorted list by deleting the 3...)

A sorting method however, does not always take a list and move the first element to the third position, or even necessarily do so in every case where the first element is three.

While deterministic, its behavior depends upon an input. Given the input, the actions it will take are known (or follow from the source code in principle[1]).

This can be generalized further, in the case of a sorting program that takes both a set of objects, and a way of ordering. Perhaps a program can even be written that reasons about some policy, and based on the results, makes an output conditional on what it finds. Thus the "logical counterfactual" does not exist per se, but is a way of thinking used in order to handle the different cases, as it is not clear which one is the case, though only one may be possible.

More specific:

Formally, a policy is a function mapping an observation history to an action. It is distinct from source code, in that the source code specifies the implementation of the policy in some programming language, rather than itself being a policy function.

Though a policy may include/specify (simpler) policies, and thus by extension, a source code may as well, though the different threads will probably be weaved together.

Comment by pattern on "No evidence" as a Valley of Bad Rationality · 2020-04-01T23:14:58.750Z · score: 2 (1 votes) · LW · GW

I was suggesting that there might be ways of assigning the label of "null hypothesis".

X is good, more X is good. (intuition favors "more chemo should kill more cancer cells")

X has a cost, we go as far as the standards say, and stop there. (Chemo kills cells. This works on your cells, and cancer cells. Maybe chemo isn't like shooting someone - they aren't that likely to die as a result - but just as you wouldn't shoot someone to improve their health unless it was absolutely necessary, and no more, chemo should be treated the same way.) "Do no harm." (This may implicitly distinguish between action and inaction.)

Comment by pattern on Categorization of Meta-Ethical Theories (a flowchart) · 2020-03-31T22:21:49.216Z · score: 2 (1 votes) · LW · GW

Epistemic status[1]: Do meta-ethics matter? (see below), and what does it mean for something to be empirical if it can't be verified?


Ethics can have a basis for going from is to ought, whether or not that base is justified*, which can render later** judgements 'empirical'. But people can lie, especially in favor of what they prefer being ethical, when they are not going from 'ethics' to 'action'.

*What does it means for an ethical basis to be justified?


Further Contents

1. Conditional Judgements

2. Understanding intent (of message, of speaker)

3. Ethics 'provide' a way to get from 'is' to 'ought'. (see also: 6.)

4. (Turning one ethical statement into another.)

5. Subjectivism and the Death Penalty

6. Ethical decision making, or justification? (continues 3)

1. Conditional Judgements

But there is still a fact about what the speaker is actually experiencing. It’s possible she might not dislike stealing, for example—maybe after robbing a bank I get caught by the cops, and in a pitiful attempt at deceiving them I exclaim, “I don’t even like stealing!”

Or someone could like stealing conditional on getting away with it.

  • Such a thing can (in theory) be determined about a specific event - particularly after the fact, based on whether or not they were caught.
  • With more difficulty, and some assumptions, this could be translated from a binary statement into one of:
  • -- probability (of getting caught) or
  • -- a statement about expected utility, though these approaches seems odd.

2. Understanding intent (of message, of speaker)

The question of what a speaker intends to communicate when they make a moral claim is an empirical question, and I think it’s safe to say that different people mean a variety of different things when they make moral claims. So I don’t think it makes sense to unreservedly jump on board with a conclusion about how to correctly interpret all moral claims. Someone can always come along and be like “Actually, when I make a moral claim I mean something totally different from what your theory says I mean,” and I guess you could just insist that they are lying or very confused about their own internal experience, but that seems like a silly thing to commit yourself to doing all the time.

Classify such statements as emotive or subjective. (Perhaps someone can say "I see the sky as red" and have this be empirically true - it is still (or was) subjective.) Does emotive include the empirical? (It seems to include a notion about meaning being encoded in the act of saying or doing something, and a person saying something true is also an act.)

3. Ethics 'provide' a way to get from 'is' to 'ought'.

Do you think that you have ever made a moral claim that turned out to be false? Think back over the course of your life, maybe to when you were a child. If you can recall a time when you made a moral claim that you feel you later learned was wrong (you didn’t just change your mind, but discovered the claim to be false) that’s an indicator that you think of your moral claims as truth-apt.

Or there's an ethical "basis" of some form like:

Suffering is bad. (Ethical)

X is bad. (Empirical aspect: Does it lead to suffering? More suffering than the alternative?)

4. (Turning one ethical statement into another.)

This statement has the capacity to be true or false (and I actually think it’s false, but we’ll get to that later), so it is a truth-apt statement.

Compromise: Lies should be held to a higher stand than truth. (Caveat: in some circumstances.)

5. Subjectivism and the Death Penalty

The aspiring subjectivist might have some bullets to bite when they consider the existence of people generally considered to be really bad. If a psychopath believes murder is good, then is it really true that murder is the right thing for him to do?

This relies on an condition, that seems to be mentioned out of proportion to its frequency. For something that might be more common, some people are in favor of the death penalty (at least for certain crimes, if a certain standard of evidence is reached, etc).

6. Ethical decision making, or justification? (continues 3)

You might worry about the fact that relativism doesn’t seem to allow for moral progress over time.

If morality is 'determined' by some fact, then what is moral may not be about 'slavery' but instead about how a given practice measures up to that standard. From Wikipedia's Proving Too Much:

The Georgia-born American educator Henry Coppée in 1850 described in his "Elements of Rhetoric" that if one argues that slavery is evil because masters are put into situations where they can beat slaves to death, then marriage and parenthood are also evil because domestic violence exists.

On this basis, the argument (above) could be furthered on the grounds that if violence exceeds some threshold, then the practice should be abolished/made illegal/etc., and then argue in favor of 'current standards' that slavery exceeds this amount while marriage and parenthood do not. But if data is never examined, and all that is taking place is justification, then a claim that empirically based morality/etc. is occurring is false, and the activity is rendered somewhat inauthentic. And without a way to determine whether or not someone is telling the truth, why should their statements be taken as "empirical"?

[1] This phrase has nothing to do with the meaning of the constituent words.

Comment by pattern on Outperforming the human Atari benchmark · 2020-03-31T21:35:00.581Z · score: 2 (1 votes) · LW · GW

So the "2" after "Coronavirus" indicated that "Coronavirus" is the 2nd tag, not that some algorithm determined it's relevance to that subject is "2"?

Comment by pattern on Outperforming the human Atari benchmark · 2020-03-31T21:20:47.756Z · score: 3 (2 votes) · LW · GW

1. Meta: Why does this have the coronavirus marker?[1] (I could understand if there was a comment or the OP said "and this can help with developing a vaccine by ...," but that doesn't seem to be the case.)

2. Specific: It would be interesting to see a comparison of different approaches all compared to each other, in therms of compute, cost, performance, etc., so it's clear when "a better algorithm has been found" versus "more compute was used".[2] I guess I'm interested in some basic credit assignment w.r.t. these different "AI" approaches.

[1] Any tips on how to add 'crossed out' to this part of the text if that changes, would be appreciated.

[2] If an algorithm doesn't work as well, stopping after a certain point makes sense - but it's nice to know if a later work does better, or is just 'more compute'.

Comment by pattern on Rational Manifesto · 2020-03-31T18:27:35.449Z · score: 2 (1 votes) · LW · GW


Some of those items should be indented more, to indicate that they're sub-bullets, so they're easier to read.

Comment by pattern on Pattern's Shortform Feed · 2020-03-31T18:24:34.727Z · score: 2 (1 votes) · LW · GW

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.

-Antoine de Saint-Exupéry

Comment by pattern on Three Kinds of Competitiveness · 2020-03-31T16:52:01.858Z · score: 2 (1 votes) · LW · GW

Speed is an aspect of Product Performance (Quality). (Example: are you using Internet Explorer as your web browser?)

Comment by pattern on What marginal returns now? · 2020-03-31T16:50:32.967Z · score: 2 (1 votes) · LW · GW
Do the extend we can think in larger timelines and people read LW because we gave



To the [extent]
Comment by pattern on "No evidence" as a Valley of Bad Rationality · 2020-03-30T02:27:17.608Z · score: 3 (2 votes) · LW · GW

Having the reverse as the null hypothesis is also bad. Which is worse?

Comment by pattern on "No evidence" as a Valley of Bad Rationality · 2020-03-29T05:47:26.292Z · score: 4 (2 votes) · LW · GW

Upon seeing the title, I guessed this piece was going to argue that people are often right without evidence. Instead the OP argued against believing something without evidence.

In the intro class, he took one step backwards. At that point he's in the Valley of Bad Rationality: education made him worse than where he started.

But is the doctor worse or better for it (even assuming that this story, second hand or third hand or more is accurate)? And how do we know?

Comment by pattern on Bogus Exam Questions · 2020-03-29T05:36:29.643Z · score: 4 (2 votes) · LW · GW

It could be an attempt to be more reality based*. (If you know the names of processes models are produced by, you can start to learn about them. Predictive models could start out more simply, and the course could have sophisticated pre-requisites.) But perhaps models coming from somewhere is a start, not an end.

*Like calculus problem that "involve" physics.

Comment by pattern on TurnTrout's shortform feed · 2020-03-28T06:18:45.520Z · score: 2 (1 votes) · LW · GW

What are your thoughts on infinitely small quantities?

Comment by pattern on TikTok Recommendation Algorithm Optimizations · 2020-03-27T03:55:47.524Z · score: 3 (2 votes) · LW · GW
natural language processing [NLP] (presumably analyzing any text which is overlaid in the video),

Audio -> text programs do exist, so it seems possible they could be doing NLP on the output of such a program, though even Youtube's audio to text program can be really bad.

Comment by pattern on How important are MDPs for AGI (Safety)? · 2020-03-27T03:47:35.721Z · score: 2 (1 votes) · LW · GW
TD-style update


Comment by pattern on 3 Interview-like Algorithm Questions for Programmers · 2020-03-27T03:45:02.673Z · score: 2 (1 votes) · LW · GW

The rotors move after each typed character, but the way they move is independent of which character it is?

Comment by pattern on Can crimes be discussed literally? · 2020-03-27T03:36:32.220Z · score: -1 (1 votes) · LW · GW

Since whether or not something "is" a 'central' example is defined relative to an ontology, I am asking "what is your ontology", in particular that the OP's remarks serve as a 'non-central' usage?

Comment by pattern on Can crimes be discussed literally? · 2020-03-26T03:23:10.437Z · score: 2 (1 votes) · LW · GW

You made a jump from:

asserting Y, when we know Y to be false


Asserting Y when it is common knowledge that Y is false

The difference between the two may be relevant to the contents of the OP.

Comment by pattern on Can crimes be discussed literally? · 2020-03-26T03:18:52.835Z · score: -1 (1 votes) · LW · GW
are you talking about the URL?


I said a descriptor could be misleading.

What descriptors do you find to be misleading?

Comment by pattern on ODE to Joy: Insights from 'A First Course in Ordinary Differential Equations' · 2020-03-25T23:08:06.691Z · score: 2 (1 votes) · LW · GW
I don't fully understand why this doesn't work for some functions which are infinitely differentiable (like logx), but apparently this becomes clearer after some complex analysis.

Because the derivative isn't zero? (x^2 is infinitely differentiable but ends up at zero fast. (x^2, 2x, 2, 0, 0, 0...))

Comment by pattern on True Habits · 2020-03-25T21:51:10.659Z · score: 4 (2 votes) · LW · GW


gabitual behavior

habitual behavior


Change/create contexts by:

Armed with the insights above, I added two desks to my room; 1 for work, 1 for leisure. Two days later, my desire for gaming during work was completely gone. 


This post seems both useful (testable) and left me with questions. What factors contribute/give rise to 'contexts', and how can they be combined with habits?

Comment by pattern on 3 Interview-like Algorithm Questions for Programmers · 2020-03-25T18:30:25.054Z · score: 3 (2 votes) · LW · GW

3. Crypto.

The Germans another create

create another

For example, if the daily code is SEC and the message code is MES then the Germans will use the daily code SEC to encode MESMES. This might result in the ciphertext XFYZQM. Only these first 6 letters are encrypted with SEC. The rest of the message is encrypted with MES. The next message (on the same day) might have a message code COD. In this case, the daily code SEC is used to encode CODCOD which might result in the ciphertext IWVUBB.

Let's imagine the daily code the machine is set it with is AAA.

Would this turn the plaintext "X" into the 'ciphertext' "X"?

But not "XX" to "XX", because typing changes the configuration?

Comment by pattern on Occam's Guillotine · 2020-03-25T17:28:53.512Z · score: 5 (2 votes) · LW · GW
but why not describe it as a source of wonder / beauty / power / progress instead?

Tell someone they can fly, and they may be excited to learn how. Tell someone they can't and they may be reluctant to believe you.

If we accept only what we want to believe* how will we:

  • Find the truth
  • Obtain the power/knowledge/etc. necessary to make things better?**

*This can go either way. If we want to be able to do things, then things being possible is great. If we want to do nothing, then things being impossible is great. (Or to make a better case: we may not want to believe

  • people are capable of doing terrible things, etc.
  • It is 'possible' to do terrible things (consider nukes, biorisk, etc. - AI risk may include the claim that 'agency' is not required to 'do evil'.))


**And what if some things can't be improved?

Comment by pattern on Occam's Guillotine · 2020-03-25T17:20:31.845Z · score: 2 (1 votes) · LW · GW
For example, I recently saw a discussion of necessity on a ‘rationalist’ forum where someone pointed out that it was impossible to fly unassisted.

If you don't consider a device like a bike assistance, I think it is possible.

Comment by pattern on Vanessa Kosoy's Shortform · 2020-03-25T07:11:20.152Z · score: 2 (1 votes) · LW · GW

And here I thought the reason was going to be that Bayesianism doesn't appear to include the cost of computation. (Thus, the usual dutch book arguments should be adjusted so that "optimal betting" does not leave one worse off for having payed, say, an oracle, too much for computation.)

Comment by pattern on Breaking quarantine is negligence. Why are democracies acting like we can only ask nicely? · 2020-03-25T07:01:26.567Z · score: 2 (1 votes) · LW · GW

I disagree with the OP. That being said, logic is logic. (Obviously when there is insufficient action more action may be called for, and just the opposite when there is 'too much'.)

At the worst extremes we have people throwing up their hands and saying, “well we’re screwed, white people will never do what they’re told, unlike those obedient Asians — and it’s not like we can make them, right?”. That attitude is completely crazy. It’s not true, and it’s part of the general atmosphere of extremely mixed messages that have made the crisis so bad.

"Oh no," say the pessimists," there is nothing to be done, we are all going to die." A pathetic excuse for inaction.

1. Tell people what they need to do. If you can do nothing else, then do what you can.

People who are tested or presumed positive will obey quarantine if they understand that they might go to prison if they don’t. This is in no way an extreme or authoritarian response. It’s completely consistent with civil liberties. Any individual freedom is always constrained by reasonable expectations of harm. None of us have a general-purpose freedom to act however we want regardless of the risks to other people. The specifics of the coronavirus pandemic are unusual, but the general principle isn’t.

Is giving people money to quarantine a violation of civil liberties?

2. Whatever you think of a given plan, there are both 'positive' and 'negative' incentives.

Take the most extreme case: someone has tested positive and been instructed to self-isolate, but the person ignores the instruction and infects someone else, who later dies of the infection. This is an act of extreme and callous negligence. The person who left quarantine was informed of a specific risk, they ignored that information, and they have caused someone else’s death for a frivolous purpose. It is not unreasonable to imprison that person for many years in order to deter that sort of negligence.

Assuming causality is easily established... (Outside of a hypothetical, things can get way more complicated, especially preventative measures.)

3. Why should the sentence occur, after the fact? If someone is infected and therefore they are a danger to others (until X amount of time has passed), then there exists, if anything a 'natural' sentence - until they can't infect others. (Sanitation, etc. at the end of quarantine may help with keeping that sentence short.) But if we don't want them in prison, hospital, or a special quarantine facility, all that is needed is much lighter version of house arrest. The arguments for prison are flimsy by comparison - are the risks to others increased or decreased as a result of such a sentence? In the case of quarantine, it seems as though a more solid proof could not be asked for, by comparison.

(What crime is punishable by such a short sentence?)

Our society is built on both individual freedom and individual responsibility.

A powerful statement. Perhaps there is some disagreement about what that means, and today we are seeing the price of a lack of clarity.

Comment by pattern on How do you study a math textbook? · 2020-03-24T22:53:07.039Z · score: 8 (3 votes) · LW · GW

1. I am not familiar with any empirical work on the subject.

2. Some textbooks give advice in the beginning, esp. about how ideas are organized. (Which can be helpful if you don't want to read the whole thing - it can explain dependencies between chapters, like 5 requires 1-3, but not 4, etc.)

3. Speculative:

  • Every subject has basic 'building blocks'. If a subject builds on an earlier subject, those remain important. Example: For polynomials there's: 1, x, x^2, x^3, etc. (All the other polynomials can be assembled out of those.) In calculus, there are functions that basically map between them (give or take multiplying/dividing by a constant factor), that are important.
  • you know times/addition tables? That's a great format for two dimensional functions that take positive integers as input. (Though f(1-10, 1-10) might not all be necessary if there's symmetry, and depending on the function, other constants might be more useful to know, like 1, 0, -1, e, pi, the primes, etc. For 3d, you might want to involve color or variables for associations or distinctions.)
  • These answers are all about how you study a math textbook. Answers based around other people might capture low hanging fruit, for one reason or another.
Comment by pattern on Can crimes be discussed literally? · 2020-03-24T20:26:20.917Z · score: -1 (1 votes) · LW · GW

Yes. (Content-wise, the linked essay is also the post.**)

What part of the essay do you find to be misleading (explicitly or implicitly*)?


**The linked essay is at the address, but doesn't include the address, so actually the post includes more than the linked essay.

Comment by pattern on Can crimes be discussed literally? · 2020-03-22T23:54:15.622Z · score: -1 (3 votes) · LW · GW

What are these 'misleading connotations'?

Comment by pattern on What is optimization power, formally? · 2020-03-22T19:32:48.325Z · score: 2 (1 votes) · LW · GW
statistical learning theory
Those of you interested in the historical literature on this sort of thing may be interested in cyberneticist's Rosenblueth, Weiner, and Bigelow's 1943 paper"Behavior, Purpose and Teleology",

Thanks for the references, though the link doesn't work (and is missing a space after it). I think it should point here.

EDIT: This post is old, so the link was probably broken in the move to LW 2.0.

Comment by pattern on Pulse Oximetry & the Oxygen–Haemoglobin Dissociation Curve · 2020-03-22T19:19:33.166Z · score: 4 (2 votes) · LW · GW
Since I can't post an image in a comment I've created this post.

Where's the comment that links to this?

Comment by pattern on [AN #91]: Concepts, implementations, problems, and a benchmark for impact measurement · 2020-03-22T06:57:56.676Z · score: 2 (1 votes) · LW · GW

Intuitively, if you forget how to do something one way, but you remember how to do it other ways, then that could make figuring out the other way again easier, thought I don't have a reason to suspect that would be the case for NNs/etc, and might depend on the specifics of the task.

Comment by pattern on [AN #91]: Concepts, implementations, problems, and a benchmark for impact measurement · 2020-03-22T06:53:51.644Z · score: 2 (1 votes) · LW · GW
Not apparent.
I may be missing your point, but isn't the fact that the Memento agent works on Montezuma's Revenge evidence that learning is not generalizing across "sections" in Montezuma's Revenge?

I was indicating that I hadn't found the answer I sought (but I included those quotes because they seemed interesting, if unrelated).

Automatic / program. See Section 4, whose first sentence is "To generalize this observation, we first propose a simple algorithm for selecting states associated with plateaus of the last agent."

Thanks for highlighting that. The reason I was interested is because I was thinking of the neural networks as being deployed to complete tasks rather than the entire game by themselves.

I ended up concluding the game was being divided up into 'parts' or epochs, each with their own respective agent deployed in sequence. The "this method makes things easy as long as there's not interference" thing is interesting when compared to multi-agent learning - they're on the same team, but cooperation doesn't seem to be easy under these circumstances (or at least not an efficient strategy, in terms of computational constraints), and reminded me of my questions about those approaches, like: Does freezing one agent (for a round) so it's predictable, then train the other one (or have it play with a human) improve things? How can 'learning to cooperate better' be balanced with 'continuing to be able to cooperate/coordinate with the other player'?

Comment by pattern on Donald Hobson's Shortform · 2020-03-22T02:10:04.404Z · score: 2 (1 votes) · LW · GW

But your disagreement only kicks in after a million years. If we meet the first alien civilization we meet, before then, then it doesn't seem to apply. A million (and 10,000?) years is also an even bigger interval than 10,000 - making what appears to be an even stronger case than the post you referenced.

Comment by pattern on [AN #91]: Concepts, implementations, problems, and a benchmark for impact measurement · 2020-03-20T17:56:44.199Z · score: 2 (1 votes) · LW · GW
The thing about Montezuma's revenge and similar hard exploration tasks is that there's only one trajectory you need to learn; and if you forget any part of it you fail drastically; I would by default expect this to be better than adversarial dynamics / populations at ensuring that the agent doesn't forget things.

But is it easier to remember things if there's more than one way to do them?

Comment by pattern on [AN #91]: Concepts, implementations, problems, and a benchmark for impact measurement · 2020-03-20T17:54:35.234Z · score: 2 (1 votes) · LW · GW
If this hypothesis were true, there would be an easy way to improve performance: once you have learned to perform the first subtask, just create a brand new neural net for the next subtask, so that training for this next subtask doesn’t interfere with past learning. Since the new agent has no information about what happened in the past, and must just “pick up” from wherever the previous agent left off, it is called the Memento agent (a reference to the movie of the same name). One can then solve the entire task by executing each agent in sequence.

This leaves unclear how it is decided that old "agents" should be used.

The paper says:

Crucially, non-interference between the two learners is identified as the salient difference.

It seems earlier agents are activated only when (what might be called) "their checkpoint" is reached ('after' the game starts over). This makes it seem like once an agent is no longer the cutting edge it is frozen, and might (depending on the environment) be replaceable* by a replay of actions.

*Not that this replacement takes place.

Was this switch to a new agent automatic or done by hand? (Was 'the agent has plateaued' determined by a program or the authors of the paper?)

Furthermore, in MONTEZUMA’S REVENGE we find that instead starting the second agent as randomly initialized (the weights encode no past context) performs equivalently.
We have now seen that learning about one portion of PONG or QBERT typically improves the prediction error in other contexts, that is, learning generalizes. However, we now present that this is not generally the case.

Not apparent.

independent reproducibility requires us to be able to reproduce similar results using only what is written in the paper. Crucially, this excludes using the author's code.

The alternative might be interesting. Given the code, but not the paper, see what insights can be found.

I’d be interested to see (and curious if the authors tried) more continuous variants of this where older information is compressed at a higher rate than newer information, since it seems rather arbitrary to split into two FIFO queues where one has a fixed compression rate.

Set a size for queues, when one is full, a new one is made. (Continuity seems expensive if it means compressing every time something is added.)

Comment by pattern on Mati_Roy's Shortform · 2020-03-19T23:21:13.380Z · score: 3 (2 votes) · LW · GW

(I aimed for non-"political" examples, which ended up sounding ridiculous.)

Suppose you believed that the color blue is evil, and want there to be less blue things in the world.

Suppose I believed the same as you, except for me the color is red.

Perhaps we could agree on a moral trade - we will both be against the colors blue and red! Or perhaps something less extreme - you won't make things red unless they'd look really good red, and I won't make things blue unless they'd look really good blue. Or we trade in some other manner - if we were neighbors and our houses were blue and red we might paint them different colors (neither red nor blue), or trade them.

Comment by pattern on Donald Hobson's Shortform · 2020-03-19T23:09:54.697Z · score: 2 (1 votes) · LW · GW

Some suggestions:

Suggest that if there are things they want to do before they die, they should probably do them. (Perhaps give more specific suggestions based on their interests, or things that lots of people like but don't try.)

Introduce Alice and Bob. (Perhaps one has a more effective approach to life, or there are things they could both learn from each other.)

Investigate/help investigate to see if the premise is incorrect. Perhaps Alice's life isn't so nice. Perhaps there are ways Bob's life could be improved (perhaps risky ways*).

*In the Sequences, lotteries were described as 'taxes on hope'. Perhaps they can be improved upon; by

  • decreasing the payout and increasing the probability
  • using temporary (and thus exploratory) rather than permanent payouts (see below)
  • seeing if there's low hanging fruit in domains other than money. (Winning a lot of money might be cool. So might winning a really nice car, or digital/non-rivalrous goods.)
Comment by pattern on Lessons from Isaac: Poor Little Robbie · 2020-03-19T22:44:52.107Z · score: 2 (1 votes) · LW · GW

Can you imagine a story about a machine that cares about humans ending badly?

Comment by pattern on A Sketch of Answers for Physicalists · 2020-03-19T22:40:39.308Z · score: 2 (1 votes) · LW · GW
The argument here is that in order to explain our justification of physics we need a concept of agents. Otherwise we [won't] be able to talk about running or observing experiments.

I think it might be possible to create an account of when an information processing system can interact with the world and develop an accurate understanding/model for it (even if the system is deterministic, etc.).

Comment by pattern on The absurdity of un-referenceable entities · 2020-03-19T22:22:25.069Z · score: 2 (1 votes) · LW · GW

How can there be an unreferenceable entity?

Perhaps I am missing the point of reference, but this phrase might be about something like:

  • If you've never seen an alien city, it might be difficult to describe it in advance. (reference to unknown parts of reality)
  • Things which cannot be referenced "directly"* - perhaps by existence. The members of the class of "colors we can see" for instance is one we know exists (has members, which we can point to). But colors we can't see may be a recently established phenomenon. While it could be "referenced", the reference didn't always point to a phenomenon with known examples - in fact its defining trait was that (via the known means at the time) it wasn't (easily, known to be) observable.


Comment by pattern on How would you take math notes to make the most of them? · 2020-03-17T22:30:13.352Z · score: 8 (3 votes) · LW · GW

It might depend on the area of math.

Things I have found useful:

1. Make up problems, and solve them. (Calculus: what is the volume of a sphere? Not using 'the' formula - integrate it from top to bottom. (Getting to the general formula eventually requires plugging in variables or functions rather than numbers.) There's also a^b=b^a.)

2. Find different 'languages' and convert between them. (It may seem obvious how to convert between addition and multiplication: 5+5+5 = 3x5 = 3+3+3+3+3. But what is 1+2+3+...+100?)

3. Generalizing (The easiest explanations are 'make functions/algorithms'. 3+5=5+3=8 replaced by f(3, 5) = 8, perhaps because f(3, 5) = f(2, 6) = f(1, 7) = f(0, 8) = 8. Less trivially: Everything is a function. Derivative operator? Turns functions into (usually other) functions. (Looking into what functions it doesn't change/are their own derivatives might be useful for reasons similar to why knowing about 1 is good for understanding/working with multiplication.))

4. What makes sense to you? Some people find it easier to understand derivatives by analogy to the (related) discrete case: If one of your character's stats in a video game automatically go up by some amount when your character levels up, then the amount a stat goes up between level 11 and 12 is like the derivative of that stat with regards to level.* Aside from the particular mathematical connections between these two things, the advantage is that something has been put in other terms: discrete rather than continuous.

5. Just practicing might be more important than note taking. (If I want to get good at trig functions, I don't have amazing notes on trig functions - I have a set of starting equations ("identities") which I can take and combine and fiddle with, and then I will be better with handling trig functions.) I guess I'm saying "you don't have to work with "first" principles, but working from a starting point gets you somewhere."

Note that 4 ties into 2.

*There's a post here on LW about that, though it didn't dive into why that was a great connection.

Comment by pattern on A critical agential account of free will, causation, and physics · 2020-03-16T16:59:49.916Z · score: 2 (1 votes) · LW · GW
An agent believing this statement may falsify it by taking an action B not equal to f(). Note that, since the agent is assumed to be able to compute things, f() may be determined. So, indeed, this statement does not hold as a law, either.
This contradicts a certain strong formulation of naive determinism: the idea that one's action is necessarily determined by some known, computable function.

The contradiction is conditional on said process working. More complicated ways of looking at determinism:

  • Enumerated possible futures: the agent will do A or B (at time t). False if the agent does something else.*
  • The agent will do A (eventually). Without a time given this may be unfalsifiable.
  • Another agent observes A, and successfully predicts what they will do. (A is unaware of the prediction, and thus unable to falsify it intentionally.)

*If "or" enumerates 'exclusive possibilities, then by doing more than one of the enumerated items, such a statement would be 'false'. Also, what does time mean? You might suppose that a prophecy that you will die in the year 2000 is obviously false - but if in 2022 calendars were rolled back to a year before 2000 then such a prediction becomes something that might be true.

Here is the assumption about how the agent interacts with gravity. The agent will choose some natural number as the height of an object. Thereafter, the object will fall, while a camera will record the height of the object at each natural-number time expressed in milliseconds, to the nearest natural-number millimeter from the ground. The agent may observe a printout of the camera data afterwards.

This seems straightforward.

Some more complicated cases:

Combine the camera and the dropped object. (If this renders things too immediately difficult, add a piece of tape or create some distant marker with regular markings, or get another camera, and use the data to figure out how to convert between the two 'frames'.)

Replace cameras with telescopes. Does the Earth go around the sun, or the sun around the earth?

(This comment might:

  • go better under a later post
  • mix up commentary and quoting, or otherwise 'jump the gun' (see above))
Let C be the computer that the agent is considering whether it is running on. C has, at each time, a tape-state, a Turing machine state, an input, and an output. The input is attached to a sensor (such as a camera), and the output is attached to an actuator (such as a motor).

It might have been in another post, but I remember reading about the possibility that the agent could modify C or create a machine in order to alter the tape in order to 'falsify' the prediction that at time t the tape would read a certain way. This seemed to point at "if "the agent is a turing machine", that needs to include the possibility that it can be both a turing machine "t" messing with tape and a turing "t+" that includes systems of sensors, motors, etc. that will read "t" and modify it (under certain conditions).".

Given the usual workings of physics, absent security, the agent could be a turing machine in the sense that the universe (it is in) is. (This seems to put issues/questions around "closed individualism versus alternatives" into this agent framework in a fashion that seems close to rendering the answers an empirical matter.)

Comment by pattern on More Dakka for Coronavirus: We need immediate human trials of many vaccine-candidates and simultaneous manufacturing of all of them · 2020-03-13T23:14:25.631Z · score: 7 (4 votes) · LW · GW
where prisoners volunteer

This sounds like something that could turn into a scandal where it later turns out prisoners were 'volunteered' against their will.

Comment by pattern on [AN #90]: How search landscapes can contain self-reinforcing feedback loops · 2020-03-13T02:13:37.660Z · score: 2 (1 votes) · LW · GW

Exploring AI Futures Through Role Play

What's the name of the game?

Comment by pattern on [AN #90]: How search landscapes can contain self-reinforcing feedback loops · 2020-03-13T01:18:39.260Z · score: 2 (1 votes) · LW · GW


the most efficient way to train Transformer-based language models is to train very large models and stop before convergence, rather than training smaller models to convergence.

I understand this refers to some notion of economic efficiency, but what is meant by convergence? Getting a perfect score on the training set?


I think that a large body of historical literature supports the conclusion that American civilizations fell primarily because of their exposure to diseases which they lacked immunity to, rather than because of European military power.

European biological military power

I think the primary insight here should instead be that pandemics can kill large groups of humans, and therefore it would be worth exploring the possibility that AI systems use pandemics as a mechanism to kill large numbers of biological humans.

Non-pandemic conditions as infrastructure:

Humans which rely on other humans in ways that involve contact with other humans, directly or indirectly*, are subject to, and are themselves, a vector for disease/viruses.

*Such as via shared surfaces, like door nobs.

Though we do not usually think of "there not being a pandemic" as infrastructure, systems which operate under the assumption that there isn't one are necessarily vulnerable to such. Methods, procedures, and work that can make this state of affairs more stable/robust may serve a similar role to infrastructure designed with the possibility of earthquakes in mind - perhaps useless when things are still, but an incredibly important foundation when a quake hits.

Rohin's opinion: It makes sense that role playing can help find extreme, edge case scenarios.

I'd say it might be useful to find obvious scenarios - though if the tech tree is an abstract thing rather than one involving making decisions it might not enable that.

Made up examples:

  • Before we release our new image recognition system to the public, let's quietly test it on a database of celebrities' facebook profile pictures and check the results. (And which celebrity does it think I look most like?)
  • There have been some concerns about how malicious actors could use this tech. So as CEO I suggest we see if this can be used to impersonate me.
  • Instead of pursuing just one approach, let's try multiple in parallel, possibly combining different approaches to see if that improves performance...
  • I may not know what a "general quantum computer" is, but if it can't make our search algorithms better or beat humans at video games, what did we get it for?

Some of the difficult about this being realistic is expertise/accessibility, though it might be interesting to see how far people can get with black box outlines or extrapolation.

style commentary:

"Beyond Near- and Long-Term: Towards a Clearer Account of Research Priorities in AI Ethics and Society (Carina Prunkl and Jess Whittlestone) (summarized by Rohin): This paper argues that the existing near-term / long-term distinction conflates four different axes on which research could differ: "

  • the capability level of AI systems (current pattern-matching systems vs. future intelligent systems)
  • the impacts of AI systems (impacts that are being felt now like fairness vs. ones that will be felt in the future like x-risks)
  • certainty (things that will definitely be problems vs. risks that are more speculative)
  • extremity (whether to prioritize particularly extreme risks)