Formal Open Problem in Decision Theory

post by Scott Garrabrant · 2018-11-29T03:25:46.134Z · score: 29 (15 votes) · LW · GW · 23 comments

(This post was originally published on March 31st 2017, and has been brought forwarded as part of the AI Alignment Forum launch sequence on fixed points.)

In this post, I present a new formal open problem. A positive answer would be valuable for decision theory research. A negative answer would be helpful, mostly for figuring out what is the closest we can get to a positive answer. I also give some motivation for the problem, and some partial progress.

Open Problem: Does there exist a topological space (in some convenient category of topological spaces) such that there exists a continuous surjection from to the space (of continuous functions from to )?


Topological Naturalized Agents: Consider an agent who makes some observations and then takes an action. For simplicity, we assume there are only two possible actions, and . We also assume that the agent can randomize, so we can think of this agent as outputting a real number in , representing its probability of taking action .

Thus, we can think of an agent as having a policy which is a function from the space of possible observations to . We will require that our agent behaves continuously as a function of its observations, so we will think of the space of all possible policies as the space of continuous functions from to , denoted .

We will let denote the space of all possible agents, and we will have a function which takes in an agent, and outputs that agent's policy.

Now, consider what happens when there are other agents in the environment. For simplicity, we will assume that our agent observes one other agent, and makes no other observations. Thus, we want to consider the case where , so .

We want to be continuous, because we want a small change in an agent to correspond to a small change in the agent's policy. This is particularly important since other agents will be implementing continuous functions on agents, and we would like any continuous function on policies to be able to be considered valid continuous function on agents.

We also want to be surjective. This means that our space of agents is sufficiently rich that for any possible continuous policy, there is an agent in our space that implements that policy.

In order to meet all these criteria simultaneously, we need a space of agents, and a continuous surjection .

Unifying Fixed Point Theorems: While we are primarily interested in the above motivation, there is another secondary motivation, which may be more compelling for those less interested in agent foundations.

There are (at least) two main clusters of fixed point theorems that have come up many times in decision theory, and mathematics in general.

First, there is the Lawvere cluster of theorems. This includes the Lawvere fixed point theorem, the diagonal lemma, and the existence of Quines and fixed point combinators. These are used to prove Gödel's incompleteness Theorem, Cantor's Theorem, Löb's Theorem, and achieve robust cooperation in the Prisoner's Dilemma in modal framework and bounded variants. All of these can be seen as corollaries of Lawvere's fixed point theorem, which states that in a cartesian closed category, if there is a point-surjective map , then every morphism has a fixed point.

Second, there is the Brouwer cluster of theorems. This includes Brouwer's fixed point theorem, The Kakutani fixed point theorem, Poincaré–Miranda, and the intermediate value theorem. These are used to prove the existence of Nash Equilibria, Logical Inductors, and Reflective Oracles.

If we had a topological space and a continuous surjection , this would allow us to prove the one-dimensional Brouwer fixed point theorem directly using the Lawvere fixed point theorem, and thus unify these two important clusters.

Thanks to Qiaochu Yuan for pointing out the connection to Lawvere's fixed point theorem (and actually asking this question three years ago).

Partial Progress:

Most Diagonalization Intuitions Do Not Apply: A common initial reaction to this question is to conjecture that such an does not exist, due to cardinality or diagonalization intuitions. However, note that all of the diagonalization theorems pass through (some modification of) the same lemma: Lawvere's fixed point theorem. However, this lemma does not apply here!

For example, in the category of sets, the reason that there is no surjection from any set to the power set, , is because if there were such a surjection, Lawvere's fixed point theorem would imply that every function from to itself has a fixed point (which is clearly not the case, since there is a function that swaps and ).

However, we already know by Brouwer's fixed point theorem that every continuous function from the interval to itself has a fixed point, so the standard diagonalization intuitions do not work here.

Impossible if You Replace with e.g. : This also provides a quick sanity check on attempts to construct an . Any construction that would not be meaningfully different if the interval is replaced with the circle is doomed from the start. This is because a continuous surjection would violate Lawvere's fixed point theorem, since there is a continuous map from to itself without fixed points.

Impossible if you Require a Homeomorphism: When I first asked this question I asked for a homeomorphism between and . Sam Eisenstat has given a very clever argument why this is impossible. You can read it here. In short, using a homeomorphism, you would be able to use Lawvere to construct a continuous map that send a function from to itself to a fixed point of that function. However, no such continuous map exists.


If you prefer not to think about the topology of , you can instead find a space , and a continuous map , such that for every continuous function , there exists an , such that for all , .

Many of the details in the motivation could be different. I would like to see progress on similar questions. For example, you could add some computability condition to the space of functions. However, I am also very curious which way this specific question will go.

This post came out of many conversations, with many people, including: Sam, Qiaochu, Tsvi, Jessica, Patrick, Nate, Ryan, Marcello, Alex Mennen, Jack Gallagher, and James Cook.

This post was originally published on March 31st 2017, and has been brought forwarded as part of the AI Alignment Forum launch sequences.

Tomorrow's AIAF sequences post will be 'Iterated Amplification and Distillation' by Ajeya Cotra, in the sequence on iterated amplification.


Comments sorted by top scores.

comment by Charlie Steiner · 2018-12-01T22:26:05.727Z · score: 4 (2 votes) · LW · GW

When thinking about agents, the first motivation might not quite work out. Small changes in observation might introduce discontinuous changes in policy - e.g. in the Matching Pennies game. Suppose there are agents (functions) in that output a fixed , no matter their input. If you can continuously vary by moving in , then Matching Pennies play will be discontinuous at . So right away you've committed to some unusual behavior for the agents in by asking for continuity - they can't play perfect Matching Pennies at the very least.

comment by Stuart_Armstrong · 2017-04-04T07:48:20.000Z · score: 4 (3 votes) · LW · GW

A small note: it's not hard to construct spaces that are a bit too big, or a bit too small (raising the possibility that a true lies between them).

For instance, if is the unit interval, then we can map onto the countable hypercube ( ). Then if we pick an ordering of the dimensions of the hypercube and an ordering of , we can see any element of - hence any element of - as a function from to .

Let be the space of continuous functions . Then any element of defines a unique function (the converse is not true - most functions do not correspond to continuous functions ). Pulling back to via we define the set .

Thus maps surjectively onto . However, though maps into by restriction (any function from is a function from ), this map is not onto (for example, there are more continuous functions from than there are from , because of the potential discontinuity at ).

Now, there are elements of that map to functions in but not in . So there's a hope that there may exist an with , , and mapping onto . Basically, as `gets bigger', its image in grows, while itself shrinks, and hopefully they'll meet.

comment by Donald Hobson (donald-hobson) · 2018-12-01T00:37:56.497Z · score: 5 (3 votes) · LW · GW

I think you made a mistake here

The Hahn–Mazurkiewicz theorem states that

A non-empty Hausdorff topological space is a continuous image of the unit interval if and only if it is a compact, connected, locally connected second-countable space.

I will agree that is connected and locally connected. I'm not sure if its second countable. It is not compact.

Just to be clear Now let . Clearly each is open. Let And . Now clearly this family covers all of . However, remove any from and is no longer covered. So is a family of open sets, which cover and don't have any finite subcover.

comment by Stuart_Armstrong · 2018-12-03T20:56:36.238Z · score: 7 (3 votes) · LW · GW

Hum, should be compact by Tychonoff's theorem (see also the Hilbert Cube, which is homeomorphic to ).

For your proof, I think that is not open in the product topology. The product topology is the coarsest topology where all the projection maps are continuous.

To make all the projection maps continuous we need all sets in to be open, where we define iff there exists an , such that is open in and .

Let be the set of finite intersection of these sets. For any , there exists a finite set such that if and for , then as well.

If we take to be the arbitrary union of , this condition will be preserved. Thus is not contained in the arbitrary unions and finite intersections of , so it seems it is not an open sent.

Also, is second-countable. From the wikipedia article on second-countable:

Any countable product of a second-countable space is second-countable

comment by Donald Hobson (donald-hobson) · 2018-12-04T23:58:05.214Z · score: 3 (2 votes) · LW · GW

I've figured out the difference, I was using the box topology , while you were using the

You are correct. I knew about finite topological products and made a natural generalization, but it turns out not to be the standard meaning of .

comment by Stuart_Armstrong · 2018-12-05T15:56:17.583Z · score: 2 (1 votes) · LW · GW

Thanks for introducing me to the box topology - seeing it defined so explicitly, and seeing what properties it fails, cleared up a few of my intuitions.

comment by jessicata (jessica.liu.taylor) · 2017-03-31T19:32:51.000Z · score: 4 (3 votes) · LW · GW

If I were studying this, I would be looking at domain theory, in which (among other things) there has been found a topological space homeomorphic with . The page I linked links to some notes at the bottom. (h/t Qiaochu for pointing out domain theory)

comment by Marcello · 2017-04-05T00:39:07.000Z · score: 3 (3 votes) · LW · GW

From discussions I had with Sam, Scott, and Jack:

To solve the problem, it would suffice to find a reflexive domain with a retract onto .

This is because if you have a reflexive domain , that is, an with a continuous surjective map , and is a retract of , then there's also a continuous surjective map .

Proof: If is a retract of then we have a retraction and a section with . Construct . To show that is a surjection consider an arbitrary . Thus, . Since is a surjection there must be some with . It follows that . Since was arbitrary, is also a surjection.

comment by Stuart_Armstrong · 2018-12-13T13:59:50.989Z · score: 2 (1 votes) · LW · GW

EDIT: This idea is wrong:

Ok, here's an idea for constructing such a map, with a few key details left unproven; let me know if people see any immediate flaws in the approach, before I spend time filling in the holes.

Let be a countable collection of open intervals (eg ), given the usual topology. Let be the closed unit interval, and the set of continuous functions from to . Give the compact-open topology.

By the properties of the compact-open topology, since is (Tychonoff), then so is . I'm hoping that the proof can be extended, at least in this case, to show that is (normal Haussdorff).

It seems clear that is second-countable: let consist of all functions that map into , where is the intersection of with a closed interval with rational endpoints, and is an open interval with rational endpoints. The set of all such is countable, and forms a subbasis of . A countable subbasis means a countable basis, as the set of finite subsets of countable set, is itself countable.

If is and second countable, then it is homeomorphic to a subset of the Hilbert Cube. To simplify notation, we will identify with its image in the Hilbert Cube.

Take the closure of within the Hilbert Cube. This closure is compact and second countable (since the Hilbert Cube itself is both). It seems clear that is connected and locally connected; connected will extend to the closure, we'll need to prove that locally connected does as well.

Then we can apply the Hahn-Mazurkiewicz theorem:

  • A non-empty Hausdorff topological space is a continuous image of the unit interval if and only if it is a compact, connected, locally connected second-countable space.

So there is a continuous surjection . Pull back , defining . If is open in (with the subspace topology), then is open in . Even if is not open, we can hope that, at worst, it consists of a countable collection of points, open intervals, half-closed intervals, and closed intervals (this is not a general property of subsets of the interval, cf the Cantor set, but it feels very likely that it will apply here).

In that case, these is a continuous surjection from to , mapping each open interval to one of the points or intervals ("folding" over the ends when mapping to those with closed end-points).

Then is the continuous surjection we are looking for.

Note: I'm thinking now that might not be connected, but this would not be a problem as long as it has a countable number of connected components.

comment by Dacyn · 2017-04-17T21:16:14.000Z · score: 1 (1 votes) · LW · GW

I am going to take some license with your question because I think you are asking the wrong question. Arbitrary topological spaces and abstract continuity are rarely the right notions in real-world situations. Rather, uniform continuity on bounded sets usually better corresponds to the intuitive notion of "a small change in input produces a small change in output".

Thus, suppose that is a complete separable metric space and that is uniformly continuous on bounded sets. Then we can show that there exists a function which is uniformly continuous on bounded sets but not a fiber of (i.e. there is no such that for all ). Indeed, consider two cases:

  1. is uniformly discrete. Then every map from to is uniformly continuous, so we get a contradiction from cardinality considerations.

  2. is not uniformly discrete. Then for each n, since f is uniformly continuous on it has a modulus of continuity on this set, i.e. a continuous increasing function such that for all . Since is not uniformly discrete, there is a function such that for infinitely many , there exist such that . (To construct it, take pairs with , extract a subsequence that behaves geometrically nicely, and then find a function such that for all in the subsequence.) Clearly, cannot be a fiber of .

comment by AlexMennen · 2017-04-23T05:00:11.000Z · score: 1 (1 votes) · LW · GW

can be a fiber of , since for each , and could be distance greater than from the basepoint.

Example: let , with for and . Let be the basepoint (so that is the point you were calling ""). Let , , , and .

I also don't see how to even construct the function , or, relatedly, what you mean by "geometrically nicely", but I guess it doesn't matter.

Also, I'm not convinced that metric spaces with uniform continuity on bounded subsets is a better framework than topological spaces with continuity.

comment by AlexMennen · 2017-04-29T18:20:01.000Z · score: 0 (0 votes) · LW · GW

This is intended as a reply to David Simmons's comment, which for some reason I can't reply to directly.

In the new version of your proof, how do we know isn't too close to for some ? And how do we know that is uniformly continuous on bounded subsets?

About continuity versus uniform continuity on bounded sets:

It seems to me that your point 1 is just a pithier version of your point 4, and that these points support paying attention to uniform continuity, rather than uniform continuity restricted to bounded sets. This version of the problem seems like it would be a less messy version of the "fixed modulus of continuity" version of the problem you mentioned (which I did not understand your solution to, but will look again later).

I'm not sure what you're getting at about singularities in point 3. I wouldn't have asked why you were considering uniform continuity on bounded sets instead of uniform continuity away from singularities (in fact, I don't know what that means). I would ask, though, why uniform continuity on bounded sets instead of uniform continuity on compact sets? As you point out, the latter is the same as continuity.

Your point 2 is completely wrong, and in fact this is the primary reason I was convinced that continuity is a better thing to pay attention to than uniform continuity on bounded sets. The type of object you are describing is an effective Polish space that remembers its metric. Typically, descriptive set theorists forget the metric, and the isomorphisms between Polish spaces are homeomorphisms (and the isomorphisms between effective Polish spaces are computable homeomorphisms). Changing the metric in a computably homeomorphic way does not change what can be done when points are represented as descending chains of basic open sets with singleton intersection. So the thing you described was really topological rather than metric in nature, even though it involves introducing a metric in the setup. I am not aware of any notions of computability in metric spaces in which the metric matters in the way you are suggesting. It is not true that uniform continuity gives you an algorithm for computing the function. As a counterexample, let be an uncomputable number, and let be given by for every . is clearly uniformly continuous, but not computable. It is also not true that uniform continuity is necessary in order for the function to be computable. For instance, is computable on . Of course, is not complete, but for another example, consider an effective infinite-dimensional Hilbert space, and let be an effective orthonormal basis. Let . This sequence of functions is computable, they have disjoint support, and for any point, a sufficiently small neighborhood around it will be disjoint from the supports of all but at most one of these functions. Thus is computable, but of course it is not uniformly continuous on the unit ball, which is bounded. However, it is true that every computable function is continuous, and conversely, every continuous function is computable with respect to some oracle. Of course, we really want computability, not computability with respect to some oracle, but this still seems to show that continuity is at least a good metaphor for computability, whereas uniform continuity on bounded sets doesn't seem so to me.

Of course, all this about continuity as a metaphor for computability makes the most sense in the context of Polish spaces, and we can only talk about actual computability in the context of effective Polish spaces. Scott's problem involves a space and the exponential . If is a locally compact Polish space, then is also Polish. I think that this might be necessary (that is, if and are Polish, then is locally compact), although I'm not sure. If so, and if your proof is correct, it seems plausible that your proof could be adapted to show that there is no locally compact Polish space with the property that Scott was looking for, and that would show that there is no solution to the problem in which and are both Polish spaces, and hence no computable solution, if computability is formalized as in effective descriptive set theory.

comment by Dacyn · 2017-05-01T11:10:45.000Z · score: 1 (1 votes) · LW · GW

I don't know why my comment doesn't have a reply button. Maybe it is related to the fact that my comment shows up as "deleted" when I am not logged in.

Sorry, I seem to be getting a little lazy with these proofs. Hopefully I haven't missed anything this time.

New proof: ... We can extract a subsequence such that if and , then for all , and for all and , either (A) and or (B) and . By extracting a further subsequence we can assume that which of (A) or (B) holds depends only on and not on . By swapping and if necessary we can assume that case (A) always holds.

Lemma: For each there is at most one such that .

Proof: Suppose and , with . Then , a contradiction.

It follows that by extracting a further subsequence we can assume that for all .

Now let be an increasing uniformly continuous function such that and for all . Finally, let . Then for all we have . On the other hand, for all we have , for we have , and for we have . Thus . Clearly, cannot be a fiber of . Moreover, since the functions and are both uniformly continuous, so is .

Regarding your responses to my points:

I guess I don't disagree with what you write regarding my points 1 and 4.

It seems to be harder than expected to explain my intuitions regarding singularities in point 3. Basically, I think the reasons that abstract continuity came to be considered standard are mostly the fact that in concrete applications you have to worry about singularities, and this makes uniform continuity a little more technically annoying. But in the kind of problem we are considering here, it seems that continuity is really more annoying to deal with than uniform continuity, with little added benefit. I guess it also depends on what kinds of functions you expect to actually come up, which is a heuristic judgement. Anyway it might not be productive to continue this line of reasoning further as maybe our disagreements just come down to intuitions.

Regarding my point 2, I wasn't very clear when I said that uniform continuity gives you an algorithm, what I meant was that if you have an algorithm for computing the images of points in the dense sequence and for computing the modulus of continuity function, then uniform continuity gives you an algorithm. The function would be the kind of thing I would handle with uniform continuity away from singularities (to fix a definition for this, let us say that you are uniformly continuous away from singularities if you are uniformly continuous on sets of the form , where is some set of singularities).

In your definition of , I think you mean to write max instead of min. But I see your point, though the example seems a little pathological to me.

Anyway, it seems that you agree that it makes sense to restrict to Polish spaces based on computability considerations, which is part of what I was trying to say in 2.

If you have a locally compact Polish space, then you can find a metric with respect to which the space is proper (i.e. bounded subsets are compact): let , where is compact. With respect to this metric, continuity is the same as uniform continuity on bounded sets, so my proof should work then.

Proposition: Let be a Polish space that is not locally compact. Then (with the compact-open topology) is not first countable.

Proof: Suppose otherwise. Then the function has a countable neighborhood basis of sets of the form where is compact and is open. Since is not locally compact, there exists a point such that is not compact for any . For each , we can choose . Let , and note that is compact. Then is a neighborhood of . But then for some . This contradicts the fact that , since we can find a bump function which is on but at .

It does still seem to me that most of the useful intuition comes from point 4 of my previous comment, though.

comment by AlexMennen · 2017-04-30T19:22:02.000Z · score: 0 (0 votes) · LW · GW

It appears that comments from new users are collapsed by default, and cannot be replied to without a "Like". These seem like bad features.

Your proof that there's no uniformly continuous on bounded sets function admitting all uniformly continuous on bounded sets functions as fibers looks correct now. It also looks like it can be easily adapted to show that there is no uniformly continuous admitting all uniformly continuous functions as fibers. Come to think of it, your proof works for arbitrary metric spaces , not just complete separable metric spaces, though those are nicer.

I see what you mean now about uniform continuity giving you an algorithm, but I still don't think that's specific to uniform continuity in an important way. After all, if you have an algorithm for computing images of points in the countable dense set, and a computable "local modulus of continuity" in the sense of a computable function with and , then is computable, and this does not require to be uniformly continuous. Although I suppose you could object that this is a bit circular, in that I'm assuming the "local modulus of continuity" is computable only in the standard sense, which does not require uniform continuity.

I'm not sure why you would allow singularities at some points (presumably a uniformly discrete set, or something like that) while still insisting on uniform continuity elsewhere. It still seems to me that the arguments for uniform continuity rather than continuity all point to wanting uniform continuity entirely, rather than some sense of local uniform continuity in most places.

Thanks for pointing out the error in my definition of ; I've fixed it.

In your argument that locally compact Polish spaces can be given metrics with respect to which they are proper, it isn't true that is necessarily a proper metric. For instance, consider a countably infinite set with for . This is a locally compact Polish space, but for every , so , and the space is not proper.

Your last proposition looks correct (though with a typo: last in the proof should be ). However, if is not locally compact, then the compact-open topology isn't necessarily the right topology to consider on . We want a topology making into an exponential object, and it isn't clear that such a topology even exists, or that it is the compact-open topology if it does exist (though it must be a refinement of the compact-open topology if it does exist). Maybe asking about non-locally compact Polish spaces with a Polish exponential space is a kind of weird question, though, and if we're even considering non-locally compact Polish spaces, we should turn to the version of the question where we just want a continuous function admitting all continuous functions as fibers.

comment by Dacyn · 2017-05-01T11:15:35.000Z · score: 1 (1 votes) · LW · GW

I will have to think more about the issue of continuity vs uniform continuity. I suppose my last remaining argument would be the fact that Bishop--Bridges' classic book on constructive analysis uses uniform continuity on bounded sets rather than continuity, which suggests that it is probably better for constructive analysis at least. But maybe they did not analyze the issue carefully enough, or maybe the relevant issues here are for some reason different.

To fix the argument that every locally compact Polish space admits a proper metric, let be as before and let if and if . Next, let , where is a countable dense sequence. Then is continuous and everywhere finite. Moreover, if , then and thus is compact. It follows that the metric is proper.

Anyway I have fixed the typo in my previous post.

comment by AlexMennen · 2017-05-02T04:02:08.000Z · score: 0 (0 votes) · LW · GW

Hm, perhaps I should figure out what the significance of uniform continuity on bounded sets is in constructive analysis before dismissing it, even though I don't see the appeal myself, since constructive analysis is not a field I know much about, but could potentially be relevant here.

is the reciprocal of what it was before, but yes, this looks good. I am happy with this proof.

comment by Dacyn · 2017-04-23T12:12:19.000Z · score: 0 (0 votes) · LW · GW

Ah, you're right. The proof can be fixed by changing the division between the two cases. So here is the new proof, with more details added regarding the construction of :

  1. is uniformly discrete for all . Then every map from to is uniformly continuous on bounded sets, so we get a contradiction from cardinality considerations.

  2. is not uniformly discrete for some . Then for each , since f is uniformly continuous on it has a modulus of continuity on this set, i.e. a continuous increasing function such that for all . Since is not uniformly discrete, there exist such that and . We can extract a subsequence such that if and , then for all , and for all and , either (A) or (B) . By extracting a further subsequence we can assume that which of (A) or (B) holds depends only on and not on . By swapping and if necessary we can assume that case (A) always holds. Now let be an increasing continuous function such that for all . Finally, let . Then for all we have but . Clearly, cannot be a fiber of .

Regarding the appropriateness of metric spaces / uniform continuity rather than topological spaces / abstract continuity, here are some of the reasons behind my intuition here (developed working in mathematical analysis, specifically Diophantine approximation, and also constructive mathematics):

  1. The obvious: metric spaces are explicitly meant to represent the intuitive notion of alikeness as a quantitative concept (i.e. distance), whereas topological spaces have no explicit notion of alikeness.

  2. In computability theory, one is interested in the question of how to computationally represent a point or an approximation to a point in a space. The standard way to do this is via restricting to the class of complete separable metric spaces, fixing a countable dense sequence (assumed to be representative of the structure of the metric space), and defining a computational approximation to a point to be an expression of the form . Since and are integers this expression can be coded as finite data. One then defines a computational encoding of a point to be an infinite bitstream consisting of computational approximations that converge to the point.

    In practical applications, in the end you will want everything to be computable. So it makes sense to work in a framework where there are natural notions of computability. I am not aware of any such notions for general topological spaces.

  3. Regarding continuity vs uniform continuity in metric spaces, both are saying that if two points are close in the domain, their images are also close. But the latter gives you a straightforward estimate as to how close, whereas the former says that the degree of closeness may depend on one of the points. Now, there are good reasons to consider such dependence, since even natural functions on the real numbers (such as or ) have "singularities" where they are not uniformly continuous.

    So the question is whether to modify the notion of uniform continuity to directly account for singularities, or to use the standard definition of continuity instead. But if one works with the standard definition, then most of the time one is really looking for ways to sneak back to uniform continuity, e.g. by using the fact that a continuous function on a compact set is uniformly continuous.

    An intuitive way of thinking about the fact that a continuous function on a compact set is uniformly continuous is that the notion of compactness means that there are no singularities present "within the space". For example, if we go back to the functions or , then the singularity of the first occurs at infinity, while the singularity of the latter occurs at . If we take a compact subset of the domain of either function, then what it really means is that we are avoiding the singularity.

    By contrast, non-compactness should mean that there are singularities. In some cases like it is easy to identify what the singularities are. But if we are dealing with spaces that are not locally compact like or an infinite-dimensional Hilbert space, then it is not as clear what the singularities are, there is just a general sense that they are dispersed "throughout the space" (because the space is not not locally compact).

    But you have to ask yourself, are these singularities real or just imagined? In many cases, imagined. For example, in the theory of Banach spaces continuous linear maps are always uniformly continuous.

    What about a map that is not uniformly continuous, like the inversion map in infinite-dimensional Hilbert space? In this case, there is still a singularity -- at -- and the definition of continuity needs to reflect that. But it doesn't help to imagine all sorts of other singularities dispersed throughout the space, because that prevents you from making useful statements like: if are at least away from and , then , where is an absolute constant.

    Now the example in the previous paragraph is an example of quantitative continuity, which is stronger than uniform continuity away from singularities. But the point is that it can be seen as an extension of uniform continuity away from singularities.

  4. Maybe my last reason will be the most relevant from a naturalized agent perspective. The notion of uniform continuity is important because it introduces the modulus of continuity, which can be viewed as a measure of how continuous a function is. The restriction that an agent must be uniformly continuous can be then thought of in a quantitative sense, with "better" agents less having to follow this restriction. So a more powerful agent may have a looser (larger) modulus of continuity, because it can react more precisely to different possible inputs.

    In this terminology, my proof can be thought of as giving an intuitive reason for why the agent cannot implement every possible policy: the agent has limited resources to distinguish different inputs, so it can only implement those policies that can be implemented with these limited resources.

    The obvious followup question would be whether if you restrict your attention to the policies that the agent isn't prevented from implementing due to its limited resources, then can it implement every possible policy? Or in other words, if you fix a modulus of continuity from the outset, can you include all functions with that modulus of continuity as fibers?

    If you allow the every-policy function to have an arbitrary modulus of continuity unrelated to the modulus of continuity you are trying to imitate, then it is not hard to see that this is possible at least for some spaces. (By Arzela-Ascoli the space of functions with a fixed modulus of continuity is compact, so there exists a continuous surjection from to this space.) But this may require greatly increasing the resources that the agent must spend to differentiate inputs. On the other hand, requiring the exact same modulus of continuity seems like too rigid an assumption. So the right question is probably to ask how close can the modulus of continuity of the every-policy function be to the modulus it is trying to imitate.

    For this kind of question it is probably better to work with a concrete example rather than trying to prove something in generality, so I will work with the Cantor space with the metric . Suppose we want to imitate all functions such that implies . (I know this is not quite the same as the original question, but I think it is close enough.) If then there are such functions. So if we have a single function that has all of them as fibers, then by the pigeonhole principle there is some ball of the form that contains two such fibers. But then if and are the two fibers, then there exists such that . It follows that if we want to choose such that implies (i.e. the analogue of the assumption on but with replaced by ) then we need .

    In conclusion, the required accuracy of is doubly exponential with respect to the required accuracy of . Thus, it is not feasible to implement such a function.

comment by Scott Garrabrant · 2017-04-10T22:42:30.000Z · score: 1 (1 votes) · LW · GW

I give a stronger version of this problem here.

comment by Marcello · 2017-04-05T00:26:00.000Z · score: 1 (1 votes) · LW · GW

"Self-Reference and Fixed Points: A Discussion and an Extension of Lawvere's Theorem" by Jorge Soto-Andrade and Francisco J. Varela seems like a potentially relevant result. In particular, they prove a converse Lawvere result in the category of posets (though they mention doing this for in an unsolved problem.) I'm currently reading through this and related papers with an eye to adapting their construction to (I think you can't just use it straight-forwardly because even though you can build a reflexive domain with a retract to an arbitrary poset, the paper uses a different notion of continuity for posets.)

comment by Stuart_Armstrong · 2017-04-03T12:07:06.000Z · score: 1 (1 votes) · LW · GW

Can you argue that must have a semi-metric compatible with the topology by using ?

I'm wondering if you can generalise this to some sort of argument that goes like this. Using X, project down via from to . Let be our initial surjection; it's now a bijection between and maps from to .

If the projection is continuous, then every map from to lifts to a map from to . Restricting to the subset of maps that are lifts like this, and applying , gives a subset . We now have a new equivalence relationship, maps from that are equal to each other on . Project down from by this relationship, to generate . Continue this transfinitely often (?) to generate a space where is a homeomorphism, and find a contradiction?

This feels dubious, but maybe worth mentioning...

comment by AlexMennen · 2017-04-03T17:51:03.000Z · score: 2 (2 votes) · LW · GW

I haven't checked that argument carefully, but that sounds like it should give you with a continuous bijection , which might not necessarily be a homeomorphism.

comment by Stuart_Armstrong · 2017-04-03T18:43:39.000Z · score: 0 (0 votes) · LW · GW

Yes, you're right.

comment by AlexMennen · 2017-05-21T21:10:19.000Z · score: 0 (0 votes) · LW · GW

This comment is to explain partial results obtained by David Simmons in this thread, since the results and their proofs are difficult to follow as a result of being distributed across multiple comments, with many errors and corrections. The proofs given here are due to David Simmons.


Theorem 1: There is no metric space and uniformly continuous function such that every uniformly continuous function is a fiber of .

Theorem 2: There is no metric space and function that is uniformly continuous on bounded sets, such that every function which is uniformly continuous on bounded sets is a fiber of .

Theorem 3: There is no locally compact Polish space and continuous function such that every continuous function is a fiber of .


Theorem 1 says that there is no solution to the version of Scott's problem with continuity in topological spaces replaced with uniform continuity in metric spaces. A plausible reason that this version of the problem might be important is that if an agent has to be able to compute what its policy says to do to any desired precision using an amount of computational resources that does not depend on the input, then its policy must be uniformly continuous. The gist of the proof is that for any uniformly continuous , it is possible to construct a uniformly continuous function that requires greater precision in its input than does to determine the output to any desired precision. I suspect it might be possible to adapt the proof to work in uniform spaces rather than just metric spaces, but I have not studied uniform spaces.

Theorem 2 is similar to theorem 1, but with uniform continuity replaced with uniform continuity on bounded subsets. I was not convinced that this is an important notion in its own right, but theorem 2 is useful as a lemma for theorem 3. See the thread under David Simmons's comment for more discussion about what sort of continuity assumption is appropriate. The gist of the proof is to apply the proof of theorem 1 on a bounded set. The function constructed will be uniformly continuous everywhere, so the proof actually shows a stronger result that unifies both theorems 1 and 2: there is no that is uniformly continuous on bounded sets and admits every uniformly continuous function as a fiber.

Theorem 3 almost says that Scott's problem cannot be solved with Polish spaces. It doesn't quite say that, because there are Polish spaces that are not locally compact. However, non-locally-compact Polish spaces are not exponentiable, so in the version of the problem in which we want a surjection , it isn't even clear whether there exists an exponential object , which may mean that non-exponentiable spaces are not promising, although I'm not sure. A reason to restrict attention to Polish spaces is that effective Polish spaces provide a topological setting in which there is a good notion of computability, so the nonexistence of a solution in Polish spaces would make it impossible to provide a computable solution in that sense. That said, there may be other notions of computability in topological spaces (domain theory?), which I am unfamiliar with. The gist of the proof is to find a metric in which bounded sets are compact, and apply theorem 2.


Proof of theorem 1: Let be a metric space, and be uniformly continuous. If is uniformly discrete, then all functions from are uniformly continuous, so there is a uniformly continuous function that is not a fiber of by Cantor's theorem.

So assume that is not uniformly discrete. Let be such that . Note that for all and , either (A) and or (B) and . By extracting a subsequence we can assume that which of (A) or (B) holds depends only on and not on . By swapping and if necessary we can assume that case (A) always holds.

For each there is at most one such that , because if and with , then , a contradiction.

It follows that by extracting a further subsequence we can assume that