Proofs Section 2.2 (Isomorphism to Expectations)

post by Diffractor · 2020-08-27T07:52:08.121Z · LW · GW · 0 comments

Contents

No comments

Previous proof post is here. [AF · GW]

Theorem 2: Isomorphism Theorem: For (causal, pseudocausal, acausal, surcausal) ${\Theta }^{st}$ or ${\Theta }^{\omega }$ which fulfill finitary or infinitary analogues of all the defining conditions, $\uparrow \left({\Theta }^{st}\right)$ and $\downarrow \left({\Theta }^{\omega }\right)$ are (causal, pseudocausal, acausal, surcausal) hypotheses. Also, $\uparrow$ and ${\to }^{st}$ define an isomorphism between $\Theta$ and ${\Theta }^{st}$, and $\downarrow$ and ${\to }^{\omega }$ define an isomorphism between $\Theta$ and ${\Theta }^{\omega }$.

Proof sketch: The reason this proof is so horrendously long is that we've got almost a dozen conditions to verify, and some of them are quite nontrivial to show and will require sub-proof-sketches of their own! Our first order of business is verifying all the conditions for a full belief function for $\uparrow \left({\Theta }^{st}\right)$. Then, we have to do it all over again for $\downarrow \left({\Theta }^{\omega }\right)$. That comprises the bulk of the proof. Then, we have to show that taking a full belief function $\Theta$ and restricting it to the infinite/finite levels fulfills the infinite/finite analogues of all the defining conditions for a belief function on policies or policy-stubs, which isn't quite as bad. Once we're done with all the legwork showing we can derive all the conditions from each other, showing the actual isomorphism is pretty immediate from the Consistency condition of a belief function.

Part 1:Let's consider $\uparrow \left({\Theta }^{{\pi }_{st}}\right)$. This is defined as: $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right):={\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{st}\right))$

We'll show that all 9+2 defining conditions for a belief function are fulfilled for $\uparrow \left({\Theta }^{st}\right)$. The analogue of the 9+2 conditions for a ${\Theta }_{st}$ is:

1: Stub Nirvana-free Nonemptiness: $\forall {\pi }_{st}:{\Theta }^{st}\left({\pi }_{st}\right)\cap NF\ne \varnothing$

2: Stub Closure: $\forall {\pi }_{st}:{\Theta }^{st}\left({\pi }_{st}\right)=\overline{{\Theta }^{st}\left({\pi }_{st}\right)}$

3: Stub Convexity. $\forall {\pi }_{st}:{\Theta }^{st}\left({\pi }_{st}\right)=c.h\left({\Theta }^{st}\right({\pi }_{st}\left)\right)$

4: Stub Nirvana-free Upper-Completion.

$\forall {\pi }_{st}:{\Theta }^{st}\left({\pi }_{st}\right)\cap NF=\left(\right({\Theta }^{st}\left({\pi }_{st}\right)\cap NF)+M^{sa}(F^{NF}\left({\pi }_{st}\right)\left)\right)\cap M^a\left(F\right({\pi }_{st}\left)\right)$

5: Stub Restricted Minimals:

$\exists {\lambda }^{\odot },b^{\odot }\forall {\pi }_{st}:(\lambda \mu ,b)\in {\left({\Theta }^{st}\right({\pi }_{st}\left)\right)}^{min}\to \lambda +b\le {\lambda }^{\odot }+b^{\odot }$

6: Stub Normalization: ${inf}_{{\pi }_{st}}{E}_{{\Theta }^{st}\left({\pi }_{st}\right)}\left(0\right)=0$ and ${sup}_{{\pi }_{st}}{E}_{{\Theta }^{st}\left({\pi }_{st}\right)}\left(1\right)=1$

7: Weak Consistency: $\forall {\pi }_{st}^{lo},{\pi }_{st}^{hi}\ge {\pi }_{st}^{lo}:p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left({\Theta }^{st}\right({\pi }_{st}^{hi}\left)\right)\subseteq {\Theta }^{st}\left({\pi }_{st}^{lo}\right)$

8: Stub Extreme Point Condition: for all $M,{\pi }_{st}^{lo}$:

$M\in \left({\Theta }^{st}\right({\pi }_{st}^{lo}){)}^{xmin}\cap NF\to$

$\exists \pi \ge {\pi }_{st}^{lo}\forall {\pi }_{st}^{hi}:(\pi >{\pi }_{st}^{hi}\ge {\pi }_{st}^{lo}\to (\exists M^{\prime }\in {\Theta }^{st}\left({\pi }_{st}^{hi}\right)\cap NF:p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(M^{\prime }\right)=M\left)\right)$

9: Stub Uniform Continuity: The function ${\pi }_{st}\mapsto \left(p{r}_{\ast }^{\infty ,{\pi }_{st}}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{st}\right)\cap NF\cap \{\le \odot \})$ is uniformly continuous.

C: Stub Causality: $\forall {\pi }_{st},M\in {\Theta }^{st}\left({\pi }_{st}\right)\exists of\forall {\pi }_{st}^{\prime }:of\left({\pi }_{st}\right)=M\wedge of\left({\pi }_{st}^{\prime }\right)\in {\Theta }^{st}\left({\pi }_{st}^{\prime }\right)$

where the outcome function $of$ is defined over all stubs.

P: Stub Pseudocausality: 

$\forall {\pi }_{st},{\pi }_{st}^{\prime }:\left(\right(M\in {\Theta }^{st}\left({\pi }_{st}\right)\wedge \text{supp}\left(M\right)\subseteq F^{NF}\left({\pi }_{st}^{\prime }\right))\to M\in {\Theta }^{st}({\pi }_{st}^{\prime }\left)\right)$

Let's begin showing the conditions. But first, note that since we have weak consistency, we can invoke Lemma 6 to reexpress $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$ as ${\bigcap }_n\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{pa}^n\right))$ Where ${\pi }_{pa}^n$ is the n'th member of the fundamental sequence of ${\pi }_{pa}$.

Also note that, for all stubs, $\uparrow \left({\Theta }^{st}\right({\pi }_{st}\left)\right)={\Theta }^{st}\left({\pi }_{st}\right)$. We'll be casually invoking this all over the place and won't mention it further.

Proof: By Lemma 6 with weak consistency, $\uparrow \left({\Theta }^{st}\right({\pi }_{st}\left)\right)={\bigcap }_{n\ge m}\left(p{r}_{\ast }^{{\pi }_{st},{\pi }_{st}^n}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{st}^n\right))$ Now, m can be anything we like, as long as it's finite. Set m to be larger than the maximum timestep that the stub is defined for. Then ${\pi }_{st}^n={\pi }_{st}$ no matter what n is (since it's above m) and projection from a stub to itself is identity, so the preimage is exactly our original set ${\Theta }^{st}\left({\pi }_{st}\right)$.

We'll also be using another quick result. For all stubs ${\pi }_{st}^{hi}\ge {\pi }_{st}^{lo}$, given stub causality,

$p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left({\Theta }^{st}\right({\pi }_{st}^{hi}\left)\right)={\Theta }^{st}\left({\pi }_{st}^{lo}\right)$

Proof: Fix an arbitrary point $M\in {\Theta }^{st}\left({\pi }_{st}^{lo}\right)$. By causality, we get an outcome function which includes $M$, giving us that there's something in ${\Theta }^{st}\left({\pi }_{st}^{hi}\right)$ that projects down onto $M$. Use weak consistency to get the other subset direction.

Condition 1: Nirvana-free Nonemptiness.

Invoking Stub Nirvana-free Nonemptiness, $\forall {\pi }_{st}:{\Theta }^{st}\left({\pi }_{st}\right)\cap NF\ne \varnothing$ so we get nirvana-free nonemptiness for $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{st}\right)$.

Now, assume ${\pi }_{pa}$ is not a stub. By stub-bounded-minimals, there is some ${\lambda }^{\odot }+b^{\odot }$ bound on the set of minimal points, regardless of stub. Let $\left({\Theta }^{st}\right({\pi }_{pa}^n){)}^{clip}$ be ${\Theta }^{st}\left({\pi }_{pa}^n\right)\cap \{\le \odot \}\cap NF$

This contains all the minimal nirvana-free points for ${\pi }_{pa}^n$. This set is nonempty because we have stub nirvana-free nonemptiness, so a nirvana-free point $M$ exists. We have stub-closure and stub minimal-boundedness, so we can step down to a minimal nirvana-free point below $M$, and it obeys the ${\lambda }^{\odot }+b^{\odot }$ bound.

Further, by weak consistency and projection preserving $\lambda$ and $b$ and nirvana-freeness,

$p{r}_{\ast }^{{\pi }_{pa}^{n+1},{\pi }_{pa}^n}\left(\right({\Theta }^{st}\left({\pi }_{pa}^{n+1}\right){)}^{clip})\subseteq \left({\Theta }^{st}\right({\pi }_{pa}^n){)}^{clip}$

Invoking Lemma 9, the intersection of preimages of these is nonempty. It's also nirvana-free, because if there's nirvana somewhere, it occurs after finite time, so projecting down to some sufficiently large finite stage preserves the presence of Nirvana, but then we'd have a nirvana-containing point in a nirvana-free set $\left({\Theta }^{st}\right({\pi }_{pa}^n){)}^{clip}$, which is impossible. This is also a subset of the typical intersection of preimages used to define $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$. Pick an arbitrary point in said intersection of preimages of clipped subsets.

Bam, we found a nirvana-free point in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$ and we're done.

Time for conditions 2 and 3, Closure and Convexity. These are easy.

$\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)={\bigcap }_n\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{pa}^n\right))$

The preimage of a closed set (stub-closure) is a closed set, and the intersection of closed sets is closed, so we have closure.

Also, $p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}^n}$ is linear, so the preimage of a convex set (stub-convexity) is convex, and we intersect a bunch of convex sets so it's convex as well.

Condition 4: Nirvana-free upper completion. 

Let $M\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)\cap NF$. Let's check whether $M+M^{\ast }$ (assuming that's an a-measure and $M^{\ast }$ is nirvana-free) also lies in the set. A sufficient condition on this given how we defined things is that for all ${\pi }_{pa}^n$, $p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}(M+M^{\ast })\in {\Theta }^{st}\left({\pi }_{pa}^n\right)$, as that would certify that $M+M^{\ast }$ is in all the preimages.

$p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}$ is linear, so $p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}(M+M^{\ast })=p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}\left(M\right)+p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}\left(M^{\ast }\right)$

The first component is in ${\Theta }^{st}\left({\pi }_{st}\right)$, obviously. And then, by stub nirvana-free-upper-completion, we have a nirvana-free a-measure plus a nirvana-free sa-measure (projection preserves nirvana-freeness), making a nirvana-free a-measure (projection preserves a-measures), so $p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}\left(M\right)+p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}\left(M^{\ast }\right)$ is in ${\Theta }^{st}\left({\pi }_{pa}^n\right)\cap NF$, and we're done.

Condition 5: Bounded-Minimals

So, there is a critical ${\lambda }^{\odot }+b^{\odot }$ value by restricted-minimals for ${\Theta }^{st}$

Fix a ${\pi }_{pa}$, and assume that there is a minimal point $M$ in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$ with a $\lambda +b$ value that exceeds the bound. Project $M$ down into each ${\Theta }^{st}\left({\pi }_{pa}^n\right)$. Projection preserves $\lambda$ and $b$ so each of these projected points $M_n$ lie above some $M_n^{min}$.

Now, invoke Lemma 7 to construct a $M_n^{\prime }\in M^a\left(F\right({\pi }_{pa}\left)\right)$ (or the nirvana-free variant) that lies below $M$, and projects down to $M_n^{min}$. Repeat this for all n. All these $M_n^{\prime }$ points are a-measures and have the standard ${\lambda }^{\odot }+b^{\odot }$ bound so they all lie in a compact set and we can extract a convergent subsequence, that converges to $M^{\prime }$, which still obeys the ${\lambda }^{\odot }+b^{\odot }$ bound.

$M^{\prime }$ is below $M$ because $\left(\right\{M\}-M^{sa}(F\left({\pi }_{pa}\right)\left)\right)\cap M^a\left(F\right({\pi }_{pa}\left)\right)$ (or the nirvana-free variant) is a closed set. Further, by Lemma 10, $M^{\prime }$ is in the defining sequence of intersections for $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$. This witnesses that $M$ isn't minimal, because we found a point below it that actually obeys the bounds. Thus, we can conclude minimal-point-boundedness for $\uparrow \left({\Theta }^{st}\right)$.

Condition 6: Normalization. We'll have to go out of order here, this can't be shown at our current stage. We're going to have to address Hausdorff continuity first, then consistency, and solve normalization at the very end. Let's put that off until later, and just get extreme points.

Condition 8: Extreme point condition: 

The argument for this one isn't super-complicated, but the definitions are, so let's recap what condition we have and what condition we're trying to get.

Condition we have: for all $M,{\pi }_{st}^{lo}$:

$M\in \left({\Theta }^{st}\right({\pi }_{st}^{lo}){)}^{xmin}\cap NF\to$

$\exists \pi \ge {\pi }_{st}^{lo}\forall {\pi }_{st}^{hi}:(\pi >{\pi }_{st}^{hi}\ge {\pi }_{st}^{lo}\to (\exists M^{\prime }\in {\Theta }^{st}\left({\pi }_{st}^{hi}\right)\cap NF:p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(M^{\prime }\right)=M\left)\right)$

Condition we want: for all $M,{\pi }_{st}$,

$M\in \left({\Theta }^{st}\right({\pi }_{st}){)}^{xmin}\cap NF\to \exists \pi >{\pi }_{st},M^{\prime }:M^{\prime }\in \uparrow ({\Theta }^{st}\left)\right(\pi )\cap NF\wedge p{r}_{\ast }^{\pi ,{\pi }_{st}}(M^{\prime })=M$

Ok, so $M\in \left({\Theta }^{st}\right({\pi }_{st}){)}^{xmin}\cap NF$ By the stub extreme point condition, there's a $\pi \ge {\pi }_{st}$, where, for all ${\pi }_{st}^{hi}$ that fulfill $\pi >{\pi }_{st}^{hi}\ge {\pi }_{st}$, there's a $M^{\prime }\in {\Theta }^{st}\left({\pi }_{st}^{hi}\right)\cap NF$, where $p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}}\left(M^{\prime }\right)=M$.

Lock in the $\pi$ we have. We must somehow go from this to a $M^{\prime }\in \Theta \left(\pi \right)\cap NF$ that projects down to our point of interest. To begin with, let ${\pi }^n$ be the n'th member of the fundamental sequence for $\pi$. Past a certain point m, these start being greater than ${\pi }_{st}$. The $M^{\prime }\in {\Theta }^{st}\left({\pi }^n\right)\cap NF$ which projects down to $M$ that we get by the stub-extreme-point condition will be called $M_n^{{}^{\prime }lo}$. Pick some random-ass point in $\left(p{r}_{\ast }^{\pi ,{\pi }_{st}^n}{)}^{-1}\right(M_n^{{}^{\prime }lo})$ and call it $M_n^{{}^{\prime }hi}$.

$M_n^{{}^{\prime }hi}$ all obey the $\lambda$ and $b$ values of $M$, because it projects down to $M$. We get a limit point of them, $M^{\prime }$, and invoking Lemma 10, it's also in $\uparrow \left({\Theta }^{st}\right)\left(\pi \right)$. It also must be nirvana-free, because it's a limit of points that are nirvana-free for increasingly late times. It also projects down to $M$ because the sequence $M_n^{{}^{\prime }hi}$ was wandering around in the preimage of $M$, which is closed.

Condition 9: Hausdorff Continuity: 

Ok, this one is going to be fairly complicated. Remember, our original form is:

"The function ${\pi }_{st}\mapsto \left(p{r}_{\ast }^{\infty ,{\pi }_{st}}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{st}\right)\cap NF\cap \{\le \odot \})$ is uniformly continuous"

And the form we want is:

"The function ${\pi }_{pa}\mapsto \left(p{r}_{\ast }^{\infty ,{\pi }_{pa}}{)}^{-1}\right(\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)\cap NF\cap \{\le \odot \})$ is uniformly continuous"

Uniform continuity means that if we want an $ϵ$ Hausdorff-distance between two preimages, there's a $\delta$ distance between partial policies that suffices to produce that. To that end, fix our $ϵ$. We'll show that the $\delta$ we get from uniform continuity on stubs suffices to tell us how close two partial policies must be.

So, we have an $ϵ$. For uniform continuity, we need to find a $\delta$ where, regardless of which two partial policies ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$ we select, as long as they're $\delta$ or less apart, the sets $\left(p{r}_{\ast }^{\infty ,{\pi }_{pa}}{)}^{-1}\right(\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)\cap NF\cap \{\le \odot \})$ (and likewise for ${\pi }_{pa}^{\prime }$) are only $ϵ$ apart. So, every point in the first preimage must have a point in the second preimage only $ϵ$ distance away, and vice-versa. However, we can swap ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$ (our argument will be order-agnostic) to establish the other direction, so all we really need to do is to show that every point in the preimage associated with ${\pi }_{pa}$ is within $ϵ$ of a point in the preimage associated with ${\pi }_{pa}^{\prime }$.

First, $m:={\log }_{\gamma }\left(\delta \right)$ is the time at which two partial policies $\delta$ apart may start differing. Conversely, any two partial policies which only disagree at-or-after m are $\delta$ apart or less. Let ${\pi }_{st}^{\ast }$ be the policy stub defined as follows: take the inf of ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$ (the partial policy which is everything they agree on, which is going to perfectly mimic both of them up till time m), and clip things off at time m to make a stub. This is only $\delta$ apart from ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$, because it perfectly mimics both of them up till time m, and then becomes undefined (so there's a difference at time m) Both ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$ are $\ge {\pi }_{st}^{\ast }$.

Let $M$ be some totally arbitrary point in $\left(p{r}_{\ast }^{\infty ,{\pi }_{pa}}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{pa}\right)\cap NF\cap \{\le \odot \})$. $M$ is also in $\left(p{r}_{\ast }^{\infty ,{\pi }_{st}^{\ast }}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{st}^{\ast }\right)\cap NF\cap \{\le \odot \})$, because $M$ projects down to some point in ${\Theta }^{st}\left({\pi }_{st}^{\ast }\right)$ that's nirvana-free.

Let ${\pi }_{pa}^{{}^{\prime }n}$, where $n\ge m$, be the n'th stub in the fundamental sequence for ${\pi }_{pa}^{\prime }$. These form a chain starting at ${\pi }_{st}^{\ast }$ and ascending up to ${\pi }_{pa}^{\prime }$, and are all $\delta$ distance from ${\pi }_{st}^{\ast }$.

Anyways, in $M^a\left(\infty \right)$, we can make a closed ball $B_{\le ϵ}$ of size $ϵ$ around $M$. This restricts $\lambda$ and $b$ to a small range of values, so we can use the usual arguments to conclude that $B_{\le ϵ}$ is compact.

Further, because ${\pi }_{st}^{\ast }$ is $\delta$ or less away from ${\pi }_{pa}^{{}^{\prime }n}$, the two sets $\left(p{r}_{\ast }^{\infty ,{\pi }_{st}^{\ast }}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{st}^{\ast }\right)\cap NF\cap \{\le \odot \})$ and $\left(p{r}_{\ast }^{\infty ,{\pi }_{pa}^{{}^{\prime }n}}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{pa}^{{}^{\prime }n}\right)\cap NF\cap \{\le \odot \})$ are within $ϵ$ of each other, so there's some point of the latter set that lies within our closed $ϵ$-ball.

Consider the set ${\bigcap }_{n\ge m}\left(\right(p{r}_{\ast }^{\infty ,{\pi }_{pa}^{{}^{\prime }n}}{)}^{-1}\left({\Theta }^{st}\right({\pi }_{pa}^{{}^{\prime }n})\cap NF\cap \{\le \odot \left\}\right)\cap B_{\le ϵ})$

the inner intersection is an intersection of closed and compact sets, so it's compact. Thus, this is an intersection of an infinite family of nonempty compact sets. To check the finite intersection property, just observe that since preimages of the sets $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{{}^{\prime }n}\right)\cap NF\cap \{\le \odot \}$ get smaller and smaller as n increases due to weak-consistency but always exist.

Pick some arbitrary point $M^{\prime }$ from the intersection. it's $\le ϵ$ away from $M$ since it's in the $ϵ$-ball. However, we still have to show that $M^{\prime }$ is in $\left(p{r}_{\ast }^{\infty ,{\pi }_{pa}^{\prime }}{)}^{-1}\right(\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{\prime }\right)\cap NF\cap \{\le \odot \})$ to get Hausdorff-continuity to go through.

To begin with, since $M^{\prime }$ lies in our big intersection, we can project it down to any ${\Theta }^{st}\left({\pi }_{pa}^{{}^{\prime }n}\right)\cap NF\cap \{\le \odot \}$. Projecting it down to stage n makes $M_n^{\prime }$. Let $M_{\infty }^{\prime }$ be the point in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{\prime }\right)\cap NF\cap \{\le \odot \}$ defined by: ${\bigcap }_{n\ge m}\left(p{r}_{\ast }^{{\pi }_{pa}^{\prime },{\pi }_{st}^n}{)}^{-1}\right(M_n^{\prime })$

Well, we still have to show that this set is nonempty, contains only one point, and that it's in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{\prime }\right)$, and is nirvana-free, to sensibly identify it with a single point.

Nonemptiness is easy, just invoke Lemma 9. It lies in the usual intersections that define $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{\prime }\right)$, so we're good there. If it had nirvana, it'd manifest at some finite point, but all finite projections are nirvana-free, so it's nirvana-free. If it had more than one point in it, they differ at some finite stage, so we can project to a finite ${\pi }_{pa}^{{}^{\prime }n}$ to get two different points, but they both project to $M_n^{\prime }$, so this is impossible. Thus, $M_{\infty }^{\prime }$ is a legit point in the appropriate set. If the projection of $M^{\prime }$ didn't equal $M_{\infty }^{\prime }$, then we'd get two different points, which differ at some finite stage, so we could project down to separate them, but they both project to $M_n^{\prime }$ for all n so this is impossible.

So, as a recap, we started with an arbitrary point $M$ in $\left(p{r}_{\ast }^{\infty ,{\pi }_{pa}}{)}^{-1}\right(\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right))\cap \{\le \odot \}$, and got another point $M^{\prime }$ that's only $ϵ$ or less away and lies in $\left(p{r}_{\ast }^{\infty ,{\pi }_{pa}^{\prime }}{)}^{-1}\right(\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{\prime }\right))\cap \{\le \odot \}$ This argument also works if we flip ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$, so the two preimages are only $ϵ$ or less apart in Hausdorff-distance.

So, given some $ϵ$, there's some $\delta$ where any two partial policies which are only $\delta$ apart have preimages only $ϵ$ apart from each other in Hausdorff-distance. And thus, we have uniform continuity for the function mapping ${\pi }_{pa}$ to the set of a-measures over infinite histories which project down to $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)\cap NF\cap \{\le \odot \}$ Hausdorff-continuity is done.

Condition 7: Consistency.

Ok, we have two relevant things to check here. The first, very easy one, is that

$\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)={\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\right(\uparrow \left({\Theta }^{st}\right)\left({\pi }_{st}\right))$

From earlier, we know that $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{st}\right)={\Theta }^{st}\left({\pi }_{st}\right)$, and from how $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$ is defined, this is a tautology.

The other, much more difficult direction, is that

$\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)=\overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(\uparrow \left({\Theta }^{st}\right)\left(\pi \right)\left)\right)$

We'll split this into four stages. First, we'll show one subset direction holds in full generality. Second, we'll get the reverse subset direction for causal/surcausal. Third, we'll show it for policy stubs for pseudocausal/acausal, and finally we'll use that to show it for all partial policies for pseudocausal/acausal.

First, the easy direction. $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)\supseteq \overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(\uparrow \left({\Theta }^{st}\right)\left(\pi \right)\left)\right)$

If we pick an arbitrary $M\in \uparrow \left({\Theta }^{st}\right)\left(\pi \right)$, it projects down to ${\Theta }^{st}\left({\pi }_{st}\right)$ for all stubs ${\pi }_{st}$ below $\pi$. Since ${\pi }_{pa}\le \pi$, it projects down to all stubs beneath ${\pi }_{pa}$. Since projections commute, $M$ projected down into $M^a\left(F\right({\pi }_{pa}\left)\right)$ makes a point that lies in the preimage of all the ${\Theta }^{st}\left({\pi }_{st}\right)$ where ${\pi }_{st}\le {\pi }_{pa}$, so it projects down into $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$.

This holds for all points in $\uparrow \left({\Theta }^{st}\right)\left(\pi \right)$, so $p{r}_{\ast }^{\pi ,{\pi }_{pa}}(\uparrow ({\Theta }^{st}\left)\right(\pi \left)\right)\subseteq \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$. This works for all $\pi \ge {\pi }_{pa}$, so it holds for the union, and then due to closure and convexity which we've already shown, we get that the closed convex hull of the projections lies in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$ too, establishing one subset direction in full generality.

Now, for phase 2, deriving $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)\subseteq \overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(\uparrow \left({\Theta }^{st}\right)\left(\pi \right)\left)\right)$ in the causal/surcausal case.

First, observe that if $\pi \ge {\pi }_{pa}$, then ${\pi }^n\ge {\pi }_{pa}^n$. Fix some $M\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$ and arbitrary $\pi \ge {\pi }_{pa}$. We'll establish the existence of a $M^{\prime }\in \uparrow \left({\Theta }^{st}\right)\left(\pi \right)$ that projects down to $M$.

To begin with, $M$ projects down to $M_n$ in ${\Theta }^{st}\left({\pi }_{pa}^n\right)$. Lock in a value for n, and consider the sequence that starts off $M_0,M_1...M_n$, and then, by causality for stubs and ${\pi }^n\ge {\pi }_{pa}^n$, you can find something in ${\Theta }^{st}\left({\pi }^n\right)$ that projects down onto $M_n$, and something in ${\Theta }^{st}\left({\pi }^{n+1}\right)$ that projects down onto that, and complete your sequence that way, making a sequence of points that all project down onto each other that climb up to $\pi$. By Lemma 10, we get a $M_n^{\prime }\in \uparrow \left({\Theta }^{st}\right)\left(\pi \right)$. You can unlock n now. All these $M_n^{\prime }$ have the same $\lambda$ and $b$ value because projection preserves them, so we can isolate a convergent subsequence converging to some $M^{\prime }\in \uparrow \left({\Theta }^{st}\right)\left(\pi \right)$.

Assume $p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left(M^{\prime }\right)\ne M$. Then we've got two different points. They differ at some finite stage, so there's some n where can project down onto $M^a\left(F\right({\pi }_{pa}^n\left)\right)$ to witness the difference, but from our construction process for $M^{\prime }$, both $M^{\prime }$ and $M$ project down to $M_n$, and we get a contradiction.

So, since $p{r}_{\ast }^{\pi ,{\pi }_{pa}}(\uparrow ({\Theta }^{st}\left)\right(\pi \left)\right)=\uparrow \left({\Theta }^{st}\right({\pi }_{pa}\left)\right)$, this establishes the other direction, showing equality, and thus consistency, for causal/surcausal hypotheses.

For part 3, we'll solve the reverse direction for pseudocausal/acausal hypotheses in the case of stubs, getting $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{st}\right)\subseteq \overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right(\uparrow \left({\Theta }^{st}\right)\left(\pi \right)\left)\right)$

Since we're working in the Nirvana-free case and are working with stubs, we can wield Lemma 3

$c.h\left(\right({\Theta }^{st}\left({\pi }_{st}\right){)}^{min})=c.h\left(\right({\Theta }^{st}\left({\pi }_{st}\right){)}^{\text{xmin}})$

So, if we could just show that the union of the projections includes all the extreme minimal points, then when we take convex hull, we'd get the convex hull of the extreme minimal points, which by Lemma 3, would also nab all the minimal points as well. By Lemmas 11 and 12, our resulting convex hull of a union of projections from above would be upper-complete. It would also get all the minimal points, so it'd nabs the entire ${\Theta }^{st}\left({\pi }_{st}\right)$ within it and this would show the other set inclusion direction for pseudocausal/acausal stubs. Also, we've shown enough to invoke Lemma 20 to conclude that said convex hull is closed. Having fleshed out that argument, all we need that all extreme minimal points are captured by the union of the projections.

By our previously proved extreme minimal point condition, for every extreme minimal point $M$ in ${\Theta }^{st}\left({\pi }_{st}\right)$, there's some $\pi >{\pi }_{st}$ and $M^{\prime }$ in $\uparrow \left({\Theta }^{st}\right)\left(\pi \right)$ that projects down to $M$, which shows that all extreme points are included, and we're good.

For part 4, we'll show that in the nirvana-free pseudocausal/acausal setting, we have

$\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)\subseteq \overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(\uparrow \left({\Theta }^{st}\right)\left(\pi \right)\left)\right)$

Fix some arbitrary $M\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$. Our task is to express it as a limit of some sequence of points that are mixtures of stuff projected from above.

For this, we can run through the same exact proof path that was used in the part of the Lemma 21 proof about how the nirvana-free part of $\Theta \left({\pi }_{pa}\right)$ is a subset of closed convex hull of the projections of the nirvana-free parts of $\Theta \left(\pi \right)$, $\pi \ge {\pi }_{pa}$. Check back to it. Since we're working in the Nirvana-free case, we can apply it very straightforwardly. The stuff used in that proof path is the ability to project down and land in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^n\right)$ (we have that by how we defined $\uparrow$), Hausdorff-continuity (which we have), and stubs being the convex hull, not the closed convex hull, of projections of stuff above them (which we mentioned recently in part 3 of our consistency proof).

Thus, consistency is shown.

Condition 6: Normalization. 

Ok, now that we have all this, we can tackle our one remaining condition, normalization. Then move on to the two optional conditions, causality and pseudocausality.

A key thing to remember is, in this setting, when you're doing $E_H\left(f\right)$, it's actually $E_{H\cap NF}\left(f\right)$, because if Murphy picked a thing with nirvana in it, you'd get infinite value, which is not ok, so Murphy always picks a nirvana-free point.

Let's show the first part, that ${inf}_{\pi }{E}_{\uparrow \left({\Theta }^{st}\right)\left(\pi \right)}\left(0\right)=0$. This unpacks as:${inf}_{\pi }{min}_{(\lambda \mu ,b)\in \uparrow \left({\Theta }^{st}\right)\left(\pi \right)\cap NF}b=0$ and we have that ${inf}_{{\pi }_{st}}{min}_{(\lambda \mu ,b)\in {\Theta }^{st}\left({\pi }_{st}\right)\cap NF}b=0$

Projections preserves $b$ values, so we can take some nirvana-free point with a $b$ value of nearly 0, and project it down to ${\Theta }^{st}\left({\pi }_{\varnothing }\right)$ (belief function of the empty policy-stub) where there's no nirvana possible because no events happen.

So, we've got $b$ values of nearly zero in there. Do we have a point with a $b$ value of exactly zero? Yes. it's closed, and has bounded minimals, so we can go "all positive functionals are minimized by a minimal point", to get a point with a $b$ value of exactly zero. Then, we can invoke Lemma 21 (we showed consistency, extreme points, and all else required to invoke it) to decompose our point into the projections of nirvana-free stuff from above, all of which must have a $b$ value of 0. So, there's a nirvana-free point in some policy with 0 $b$ value.

Now for the other direction. Let's show that ${sup}_{\pi }{E}_{\uparrow \left({\Theta }^{st}\right)\left(\pi \right)}\left(1\right)=1$.

This unpacks as: ${sup}_{\pi }{min}_{(\lambda \mu ,b)\in \uparrow \left({\Theta }^{st}\right)\left(\pi \right)\cap NF}(\lambda +b)=1$

We'll show this by disproving that the sup is <1, and disproving that the sup is >1.

First, assume that, regardless of $\pi$, ${min}_{(\lambda \mu ,b)\in \uparrow \left({\Theta }^{st}\right)\left(\pi \right)\cap NF}(\lambda +b)\le 1-ϵ$ Then, regardless of ${\pi }_{st}$ we can pick some totally arbitrary $\pi >{\pi }_{st}$, and there's a nirvana-free point with a $\lambda +b$ value of $1-ϵ$ or less. By consistency, we can project it down into $\Theta \left({\pi }_{st}\right)$, to get a nirvana-free point with a $\lambda +b$ value of $1-ϵ$ or less. Thus, regardless of the stub we pick, there's nirvana-free points where Murphy can force a value of $1-ϵ$ or less, which contradicts ${sup}_{{\pi }_{st}}{min}_{(\lambda \mu ,b)\in {\Theta }^{st}\left({\pi }_{st}\right)\cap NF}(\lambda +b)=1$

What if it's above 1? Assume there's some $\pi$ where ${min}_{(\lambda \mu ,b)\in \uparrow \left({\Theta }^{st}\right)\left(\pi \right)\cap NF}(\lambda +b)\ge 1+ϵ$

From uniform-continuity-Hausdorff, pick some $\delta$ to get a $\frac{ϵ}{2}$ Hausdorff-distance or lower (for stuff obeying the ${\lambda }^{\odot }+b^{\odot }$ bound, which all minimal points of $\uparrow \left({\Theta }^{st}\right)\left(\pi \right)\cap NF$ do for all $\pi$). This $\delta$ specifies some extremely large n, consider ${\pi }^n$. Now, consider the set of every policy ${\pi }^{\prime }$ above ${\pi }^n$. All of these are $\delta$ or less away from $\pi$. Also, remember that the particular sort of preimage-to-infinity that we used for Hausdorff-continuity slices away all the nirvana.

So, Murphy, acting on $\pi$, can only force a value of $1+ϵ$ or higher. Now, there can be no nirvana-free point in $\uparrow \left({\Theta }^{st}\right)\left({\pi }^{\prime }\right)$ with $\lambda +b<1+\frac{ϵ}{2}$. The reason for this is that, since ${\pi }^{\prime }$ is $\delta$ or less away from $\pi$, there's a nirvana-free point in $\uparrow \left({\Theta }^{st}\right)\left(\pi \right)$ that's $\frac{ϵ}{2}$ away, and thus has $\lambda +b<1+ϵ$, which is impossible.

Ok, so all the nirvana-free points in $\uparrow \left({\Theta }^{st}\right)\left({\pi }^{\prime }\right)$ where ${\pi }^{\prime }>{\pi }_{st}^n$ have $\lambda +b\ge 1+\frac{ϵ}{2}$.

Now, since we have Lemma 21, we can go "hm, ${\Theta }^{st}\left({\pi }^n\right)$ equals the convex hull of the projections of $\uparrow \left({\Theta }^{st}\right)\left({\pi }^{\prime }\right)\cap NF$. Thus, any minimal point with $\lambda +b\le 1$ is a finite mix of nirvana-free stuff from above, one of which must have $\lambda +b\le 1$. But we get a contradiction with the fact that there's no nirvana-free point from ${\pi }^{\prime }$ above ${\pi }^n$ with a $\lambda +b$ value that low, they're all $\ge 1+\frac{ϵ}{2}$"

So, since we've disproved both cases, ${sup}_{\pi }{E}_{\uparrow \left({\Theta }^{st}\right)\left(\pi \right)}\left(1\right)=1$. And we're done with normalization! On to causality and pseudocausality.

Condition C: Causality.

An "outcome function" $of$ for $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$ is a function that maps a ${\pi }_{pa}$ to a point in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$, s.t. for all ${\pi }_{pa}^{hi},{\pi }_{pa}^{lo}:p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left(of\right({\pi }_{pa}^{hi}\left)\right)=of\left({\pi }_{pa}^{lo}\right)$.

Causality is, if you have a $M\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$, you can always find an outcome function $of$ where $of\left({\pi }_{pa}\right)=M$. Sadly, all we have is causality over stubs. We'll be using the usual identification between $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{st}\right)$ and ${\Theta }^{st}\left({\pi }_{st}\right)$.

Anyways, fix a ${\pi }_{pa}$ and a point $M$ in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$. Project $M$ down to get a sequence $M_n\in {\Theta }^{st}\left({\pi }_{pa}^n\right)$. By causality for stubs, we can find an $o{f}_n$ where, for all ${\pi }_{st}$, $o{f}_n\left({\pi }_{pa}^n\right)=M_n$. Observe that there are countably many stubs, and no matter the n, all the $\lambda$ and $b$ values are the same because projection preserves those. We can view $o{f}_n$ as a sequence in

${\prod }_{{\pi }_{st}}\left({\Theta }^{st}\right({\pi }_{st})\cap \{({\lambda }^{\prime }\mu ,b^{\prime })|{\lambda }^{\prime }=\lambda \wedge b^{\prime }=b\left\}\right)$

By stub closure, and a $\lambda$ and $b$ bound, this is a product of compact sets, and thus compact by Tychonoff (no axiom of choice needed, its just a countable product of compact metric spaces) so we can get a limiting $o{f}^{st}$ (because it's only defined over stubs).

An outcome function for stubs fixes an outcome function for all partial policies, by

$of\left({\pi }_{pa}\right)={\bigcap }_n\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}{)}^{-1}\right(o{f}^{st}\left({\pi }_{pa}^n\right))$

We've got several things to show now. We need to show that $o{f}^{st}$ is an outcome function, that $of$ is well-defined, that $of\left({\pi }_{pa}\right)=M,$ and that it's actually an outcome function.

For showing that $o{f}^{st}$ is an outcome function, observe that projection is continuous, and, letting n index our convergent subsequence of interest, regardless of stub ${\pi }_{st}$, ${lim}_{n\to \infty }o{f}_n\left({\pi }_{st}\right)=o{f}^{st}\left({\pi }_{st}\right)$. With this,

$p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(o{f}^{st}\right({\pi }_{st}^{hi}\left)\right)=p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left({lim}_{n\to \infty }o{f}_n\right({\pi }_{st}^{hi}\left)\right)={lim}_{n\to \infty }p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(o{f}_n\right({\pi }_{st}^{hi}\left)\right)$

$={lim}_{n\to \infty }o{f}_n\left({\pi }_{st}^{lo}\right)=o{f}^{st}\left({\pi }_{st}^{lo}\right)$

Now, let's show that $of$ is well-defined. Since $o{f}^{st}$ is an outcome function, all the points project down onto each other, so we can invoke Lemma 9 to show that the preimage is nonempty. If the preimage had multiple points, we could project down to some finite stage to observe their difference, but nope, they always project to the same point. So it does pick out a single well-defined point, and it lies in $\uparrow \left({\Theta }^{st}\right)({\pi }_{pa}$) by being a subset of the defining sequence of intersection of preimages.

Does $of\left({\pi }_{pa}\right)=M$? Well, $M$ projected down to all the $M_n$. If $n\ge m$, then $o{f}_n\left({\pi }_{pa}^m\right)=p{r}_{\ast }^{{\pi }_{pa}^n,{\pi }_{pa}^m}\left(o{f}_n\right({\pi }_{pa}^n\left)\right)=p{r}_{\ast }^{{\pi }_{pa}^n,{\pi }_{pa}^m}\left(M_n\right)=M_m$ So, the limit specification $o{f}^{st}$ has $o{f}^{st}\left({\pi }_{pa}^n\right)=M_n$ for all n. The only thing that projects down to make all the $M_n$ is $M$ itself, so $of\left({\pi }_{pa}\right)=M$.

Last thing to check: Is $of$ an outcome function over partial policies? Well, if ${\pi }_{pa}^{hi}\ge {\pi }_{pa}^{lo}$, then for all n, ${\pi }_{pa}^{hi,n}\ge {\pi }_{pa}^{lo,n}$. Assume $p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left(of\right({\pi }_{pa}^{hi}\left)\right)\ne of\left({\pi }_{pa}^{lo}\right)$. Then, in that case, we can project down to some ${\pi }_{pa}^{lo,n}$ and they'll still be unequal. However, since projections commute, it doesn't matter whether you project down to ${\pi }_{pa}^{lo}$ and then to ${\pi }_{pa}^{lo,n}$, or whether you project down to ${\pi }_{pa}^{hi,n}$ (making $o{f}^{st}\left({\pi }_{pa}^{hi,n}\right)$), and then project down to ${\pi }_{pa}^{lo,n}$ (making $o{f}^{st}\left({\pi }_{pa}^{lo,n}\right)$). Wait, hang on, this is the exact point that $of\left({\pi }_{pa}^{lo}\right)$ projects down to, contradiction. Therefore it's an outcome function.

And we're done, we took an arbitrary ${\pi }_{pa}$ and $M\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$, and got an outcome function $of$ with $of\left({\pi }_{pa}\right)=M$, showing causality.

Condition P: Pseudocausality: If $(m,b)\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$, and $m$'s support is on $M^a\left(F^{NF}\right({\pi }_{pa}^{\prime }\left)\right)$, then $(m,b)\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{\prime }\right)$.

But all we have is, if $(m,b)\in {\Theta }^{st}\left({\pi }_{st}\right)$ and $m$'s support is on $M^a\left(F\right({\pi }_{st}^{\prime }\left)\right)$, then $(m,b)\in {\Theta }^{st}\left({\pi }_{st}^{\prime }\right)$.

There's a subtlety here. Our exact formulation of pseudocausality we want is the condition $\text{supp}\left(m\right)\subseteq F\left({\pi }_{pa}^{\prime }\right)$, so if the measure is 0, then support is the empty set, which is trivially a subset of everything, then pseudocausality transfers it to all partial policies.

Ok, so let's assume that $M\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$, and the measure part $m$ has its support being a subset of $M^a\left(F^{NF}\right({\pi }_{pa}^{\prime }\left)\right)$ but yet is not in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{\prime }\right)$. Then, since this is an intersection of preimages from below, there should be some finite level ${\pi }_{pa}^{{}^{\prime }n}$ that you can project $M$ down to (it's present in $M^a\left(F^{NF}\right({\pi }_{pa}^{\prime }\left)\right)$, just maybe not in $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}^{\prime }\right)$) where the projection of $M$ (call it $M_n^{\prime }$) lies outside ${\Theta }^{st}\left({\pi }_{pa}^{{}^{\prime }n}\right)$ (lying outside the intersection of preimages)

This is basically "take $m$, chop it off at height n". However, since $M\in \uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right)$, you can project it down to ${\Theta }^{st}\left({\pi }_{pa}^n\right)$. Which does the exact same thing of chopping $m$ off at height n, getting you $M_n^{\prime }$ exactly. We can invoke stub-pseudocausality (because with full measure, the history will land in $F\left({\pi }_{pa}^{\prime }\right)$, then with full measure, the truncated history will land in $F\left({\pi }_{pa}^{{}^{\prime }n}\right)$ as the latter is prefixes of the former, or maybe the full measure is 0 in which case pseudocausality transfer still works) to conclude that $M^{\prime }$ actually lies inside ${\Theta }^{st}\left({\pi }_{pa}^{{}^{\prime }n}\right)$, getting a contradiction. This establishes pseudocausality in full generality.

Ok, so we have one direction. $\uparrow \left({\Theta }^{st}\right)$ is a hypothesis, if ${\Theta }^{st}$ fulfills analogues of the hypothesis conditions for the finitary stub case. Our proof of everything doesn't distinguish between causal and surcausal, and the arguments work for all types of hypotheses, whether causal, surcausal, pseudocausal, or acausal. Ok, we're 1/4 of the way through. Now we do the same thing, but for building everything from infinitary hypotheses.

 

PART 2: Infinitary hypotheses. We now consider $\downarrow \left({\Theta }^{\omega }\right)$, defined as$\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)=\overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right({\Theta }^{\omega }\left(\pi \right))$

We'll show that with 8+2 defining conditions, the 9+2 defining conditions for a hypothesis hold for $\downarrow \left({\Theta }^{\omega }\right)$. The the 8+2 conditions for a ${\Theta }^{\omega }$ are:

1: Infinitary Nirvana-Free Nonemptiness: $\forall \pi :{\Theta }^{\omega }\left(\pi \right)\cap NF\ne \varnothing$

2: Infinitary Closure: $\forall \pi :{\Theta }^{\omega }\left(\pi \right)=\overline{{\Theta }^{\omega }\left(\pi \right)}$

3: Infinitary Convexity. $\forall \pi :{\Theta }^{\omega }\left(\pi \right)=c.h\left({\Theta }^{\omega }\right(\pi \left)\right)$

4: Infinitary Nirvana-free Upper-Completeness 

$\forall \pi :{\Theta }^{\omega }\left(\pi \right)\cap NF=\left(\right({\Theta }^{\omega }\left(\pi \right)\cap NF)+M^{sa}(F^{NF}\left(\pi \right)\left)\right)\cap M^a\left(F\right(\pi \left)\right)$

5: Infinitary Bounded Minimals: $\exists {\lambda }^{\odot },b^{\odot }\forall \pi :(\lambda \mu ,b)\in {\left({\Theta }^{\omega }\right(\pi \left)\right)}^{min}\to \lambda +b\le {\lambda }^{\odot }+b^{\odot }$

6: Normalization: ${inf}_{\pi }{E}_{{\Theta }^{st}\left(\pi \right)}\left(0\right)=0$ and ${sup}_{\pi }{E}_{{\Theta }^{st}\left(\pi \right)}\left(1\right)=1$

7: Nirvana-free consistency.

$\forall {\pi }_{st}:c.h\left({\bigcup }_{\pi >{\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\cap NF\left)\right)=\overline{c.h}\left({\bigcup }_{\pi >{\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)\cap NF$

8: Infinitary Uniform Hausdorff Continuity:

The function $\pi \mapsto {\Theta }^{\omega }\left(\pi \right)\cap NF\cap \{\le \odot \}$ is uniformly continuous.

C: Infinitary Causality: Regardless of $\pi$ and $M\in {\Theta }^{\omega }\left(\pi \right)$, there's an outcome function $of$ over full policies s.t. $of\left(\pi \right)=M$, and for all ${\pi }^{\prime }$ and ${\pi }^{\prime \prime }$, $p{r}_{\ast }^{{\pi }^{\prime },inf({\pi }^{\prime },{\pi }^{\prime \prime })}\left(of\right({\pi }^{\prime }\left)\right)=p{r}_{\ast }^{{\pi }^{\prime \prime },inf({\pi }^{\prime },{\pi }^{\prime \prime })}\left(of\right({\pi }^{\prime \prime }\left)\right)$

P: Infinitary Pseudocausality: $\forall \pi ,{\pi }^{\prime }:(M\in {\Theta }^{\omega }(\pi ),\text{supp}(M)\subseteq F^{NF}({\pi }^{\prime })\to {\pi }_{st}\in {\Theta }^{\omega }({\pi }^{\prime })$

Let's begin showing the conditions.

Condition 1: Nirvana-free Nonemptiness.

This one is trivial. Pick some $\pi \ge {\pi }_{pa}$. There's a nirvana-free point. Project it down. You get a nirvana-free point and you're done.

Conditions 2 and 3, Closure and convexity. We explicitly took the closed convex hull when defining everything, these are tautological.

Condition 4: Nirvana-free upper completion. 

For the pseudo/acausal case, it's doable by Lemmas 10, 11, and 12. The projection of an upper-complete set (by infinitary nirvana-free upper-completion) is upper-complete, so the union of projections is upper-complete, and then the convex hull is upper-complete, and then the closure is upper-complete and we're done.

We'll have to loop back to the causal case of Nirvana-free Upper Completion later, because we need Lemma 21 to make it go through and that requires consistency and the extreme point condition to make it work.

Condition 5: Bounded Minimals.

We can break down into three phases. First is showing that all points in the projection set have something under them that respects the ${\lambda }^{\odot }+b^{\odot }$ bound. Second is showing that all points in the convex hull of the union of projection sets have something under them that respects the ${\lambda }^{\odot }+b^{\odot }$ bound. Third is showing that all points in the closure have something under them that respects the usual bound. The reason we have to phrase it this way is that we don't necessarily know that our sets of interest are closed until the end, so we can't find a minimal point, just a bounded one that is lower, but that suffices to show that a "minimal point" that violates the restricted minimal condition isn't actually minimal.

For part 1, let $M\in p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left({\Theta }^{\omega }\right(\pi \left)\right)$. Then, $M$ is the projection of some point $M^{\prime }\in {\Theta }^{\omega }\left(\pi \right)$. By infinitary bounded-minimals, we can find a minimal point $M^{min}\in {\Theta }^{\omega }\left(\pi \right)$ below $M^{\prime }$ that obeys the ${\lambda }^{\odot }+b^{\odot }$ bound, so $M^{min}+M^{\ast }=M^{\prime }$. Projecting down is linear, so we get $p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left(M^{min}\right)+p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left(M^{\ast }\right)=M$, and $p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left(M^{min}\right)$ is below $M$ and fulfills the ${\lambda }^{\odot }+b^{\odot }$ bound.

For part 2, let $M\in c.h\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)$ We can rewrite $M$ as $E_{\zeta }{M}_i$, and then, by part 1, decompose the $M_i$ into $M_i^{lo}$ (not actually minimal, just a point obeying the ${\lambda }^{\odot }+b^{\odot }$ bound) and $M_i^{\ast }$. Then we can decompose $M$ further into $E_{\zeta }{M}_i^{lo}+E_{\zeta }{M}_i^{\ast }$. The former is an a-measure (mix of a-measures) and obeys the ${\lambda }^{\odot }+b^{\odot }$ bound since all its components do, and it's in the relevant convex hull, witnessing that $M$ has a point below it in the convex hull that obeys the bounds.

For part 3, let $M$ be in the closure of the convex hull. There's some sequence $M_n$ in the convex hull that limits to $M$. Below each $M_n$ we can find a $M_n^{lo}$ (again, not actually minimal) that obeys the ${\lambda }^{\odot }+b^{\odot }$ bound. Invoke Lemma 16 to get a point below $M$ that respects the bounds, and we're done.

Condition 6: Normalization. 

We literally have the exact phrasing of normalization we need already, this is a tautology.

Condition 7: Consistency. 

Ok, one direction is trivial because $\downarrow \left({\Theta }^{\omega }\right)\left(\pi \right)={\Theta }^{\omega }\left(\pi \right)$, so we can just use the definition of $\downarrow$. $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)=\overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(\downarrow \left({\Theta }^{\omega }\right)\left(\pi \right)\left)\right)$

The other direction, that everything equals the intersection of preimages of stuff below it, is trickier. One subset direction isn't too bad, the one that

$\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)\subseteq {\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\right(\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right))$

If we take a $M\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$ that wasn't added in the final closure step, it's expressible as $E_{\zeta }{M}_i$, and all the $M_i$ come from points $M_i^{\infty }$ in ${\Theta }^{\omega }\left({\pi }_i\right)$ where ${\pi }_i\ge {\pi }_{pa}$. Projecting the $M_i^{\infty }$ down to a ${\pi }_{st}\le {\pi }_{pa}$ instead makes $M_i^{lo}$, which mix together in the same way to make $M^{lo}\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)$. Because projections are linear and commute, $M_{lo}$ is the projection of $M$. So, any point in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$ (without the closure step) projects down to lie in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)$ for any ${\pi }_{st}\le {\pi }_{pa}$.

Then, for the closure step, we just fix a sequence $M_n$ limiting to $M$. The $M_n$ can project down to whichever $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)$ you wish, and by continuity of projection, the $M$ comes along for the ride as a limit point. However, $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)$ is closed, so $M$ projects down to land in that set as well. Bam, any old $M\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$ projects down to land in any $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)$ set you wish with ${\pi }_{st}\le {\pi }_{pa}$, certifying that $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$ lies in the intersection of preimages of stubs below.

Now, we just have to establish $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)\supseteq {\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\right(\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right))$ which splits into two cases. The causal/surcausal case, and the pseudocausal/acausal case where you don't have to worry about nirvana.

For the nirvana-free case... We can use the same proof strategy as the last part of Lemma 21, where we were showing the result for partial policies. It may be a bit nonobvious why it works. We do need to swap things around a bit, and will mention important changes without fleshing out the fiddly details, which are already given in the last part of Lemma 21.

Start with a $M$ in the intersection of preimages of stubs below. To show it's in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$, we need a sequence limiting to it, where each member of the sequence is a mix of finitely many points projected down from policies above ${\pi }_{pa}$. The end part of Lemma 21 gives how to construct such a sequence. The fact that we're working in a nirvana-free setting means you can ignore all fiddly details about points being nirvana-free and preimages of only the nirvana-free parts, because everything fulfills that. The key steps in that proof path are:

1:   being able to project down $M$ to make a sequence $M_n\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}^n\right)$. We trivially have this by $M$ being defined as "in the intersection of preimages of stubs below it".

2: Having uniform Hausdorff-continuity for the policies. This is our condition 8 we're assuming, so we're good there.

3: The ability to shatter our $M_n$ into finitely many $M_{i,n}$ which are the projections of various $M_{i,n}^{\infty }$ points from above. This is the key difference. The proof of Lemma 21 had to set up that fact beforehand. However, in our case, we have the Nirvana-free consistency condition, which says

$\forall {\pi }_{st}:c.h\left({\bigcup }_{\pi >{\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\cap NF\left)\right)=\overline{c.h}\left({\bigcup }_{\pi >{\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)\cap NF$

But, since we're working in the nirvana-free setting, this turns into:

$\forall {\pi }_{st}:c.h\left({\bigcup }_{\pi >{\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)=\overline{c.h}\left({\bigcup }_{\pi >{\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)$

And that right-hand term... is just the definition of $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)$! So, swapping that out, and specializing our arbitrary stub to ${\pi }_{pa}^n$, we have:

$c.h\left({\bigcup }_{\pi >{\pi }_{pa}^n}p{r}_{\ast }^{\pi ,{\pi }_{pa}^n}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)=\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}^n\right)$

So, since our $M_n$ lie in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}^n\right)$, they can be written as a finite mix of nirvana-free things from above projected down, and the Lemma 21 argument goes through.

Now, for the nirvana cases where we can assume infinitary causality. We'll do this by showing a little sublemma, that, if $\pi \ge {\pi }_{pa}$, then $p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left({\Theta }^{\omega }\right(\pi \left)\right)=\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$

First, we'll show that if $\pi ,{\pi }^{\prime }\ge {\pi }_{pa}$, then $p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left({\Theta }^{\omega }\right(\pi \left)\right)=p{r}_{\ast }^{{\pi }^{\prime },{\pi }_{pa}}\left({\Theta }^{\omega }\right({\pi }^{\prime }\left)\right)$

Fix an arbitrary $M$ in the projection of ${\Theta }^{\omega }\left(\pi \right)$, we can get a preimage point $M^{hi}\in {\Theta }^{\omega }\left(\pi \right)$. Then, by infinitary causality, we can make a point $M^{{}^{\prime }hi}\in {\Theta }^{\omega }\left({\pi }^{\prime }\right)$ that projects down to $M$. Just make an outcome function $of$ where $of\left(\pi \right)=M^{hi}$, feed in ${\pi }^{\prime }$, that gets you your point $M^{{}^{\prime }hi}$, the two agree when you project them down to $inf(\pi ,{\pi }^{\prime })$, and ${\pi }_{pa}$ is further down than that and projections commute so they both hit the same point $M$ if you project down. Flipping $\pi$ and ${\pi }^{\prime }$ shows our equality.

Alright, so now $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$ can be written as $\overline{c.h}\left(p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)$ where $\pi$ is arbitrary above ${\pi }_{pa}$.

Projection is linear, so the projection of a convex set is convex. To get the closure points, just take a sequence $M_n$ in the projection limiting to some $M$. Take preimage points $M_n^{\infty }\in {\Theta }^{\omega }\left(\pi \right)$. There's a bound on the $\lambda$ and $b$ values of this sequence because projections preserve $\lambda$ and $b$ and our sequence $M_n$ converges, so we can apply the Compactness Lemma and get a convergent subsequence limiting to a point $M^{\infty }$, which must be in ${\Theta }^{\omega }\left(\pi \right)$ because closure. Projection is continuous, so $M^{\infty }$ projects down to $M$. And we have $p{r}_{\ast }^{\pi ,{\pi }_{pa}}{\Theta }^{\omega }\left(\pi \right)=\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$ proved! Wow, that was a sublemma of case 2 of part 3 of the proof of condition 7 in part 2 of the proof of the Isomorphism theorem, we're really in the weeds at this point.

Moving on, how can we use this to show $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)\supseteq {\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\right(\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right))$ for the causal case, which is the last bit we need to show consistency?

Well, fix a $M$ in the intersection of preimages, and an arbitrary $\pi \ge {\pi }_{pa}$. $M$ projects down to make some $M_n\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}^n\right)$. Since $\pi \ge {\pi }_{pa}\ge {\pi }_{pa}^n$, and we have our sublemma, there's a point $M_n^{\infty }$ in ${\Theta }^{\omega }\left(\pi \right)$ that projects down to some $M_n^{\prime }\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$, and further down to $M_n$.

This sequence $M_n^{\infty }$ all has the same $\lambda$ and $b$ value since projection preserves those, so by the Compactness Lemma and closure, there's a convergent subsequence and limit point $M^{\infty }\in {\Theta }^{\omega }\left(\pi \right)$. Does $M^{\infty }$ project down onto $M$? (witnessing that $M\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right))$?

Well, let's say it didn't and projecting down gets you a distinct point. Then there's some n where projecting down further to ${\pi }_{pa}^n$ would keep the points distinct, since they have to differ at some finite time. But... after time n, our sequence $M_n^{\infty }$ is roaming around entirely in the preimage of $M_n$, so the limit point is in there too, and it projects down to $M_n$ and we have a contradiction. Therefore, $M^{\infty }\in {\Theta }^{\omega }\left(\pi \right)$ projects down onto $M$, witnessing that $M\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$, and $M$ was arbitrary in the intersection of preimages. 

So, we have $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)\supseteq {\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\right(\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right))$ for the nirvana-containing causal/surcausal case, which is the last piece we needed to show consistency.

Condition 8: Extreme point condition.

The thing we want is that an extreme nirvana-free minimal point $M^{ex}$ in $\downarrow \left(B{F}^{\omega }\right)\left({\pi }_{st}\right)\cap NF$ is the projection of a nirvana-free point from a policy above it. By the nirvana-free consistency property, $M^{ex}$ lies in $c.h\left({\bigcup }_{\pi \ge {\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\cap NF\left)\right)$

$M^{ex}$ is extreme, so it lies in ${\bigcup }_{\pi \ge {\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\left({\Theta }^{\omega }\right(\pi )\cap NF)$

So, there's some point in some ${\Theta }^{\omega }\left(\pi \right)\cap NF$ that projects down to $M^{ex}$ and we're done.

Condition 9: Hausdorff Continuity:

Ok, this one is going to be complicated. We'll work with the Lemma 15 version of Hausdorff-continuity, where a $\delta$ difference between two policies means that if you start off in one preimage, you've gotta travel $ϵ(1+\lambda )$ distance or less to get to the preimage associated with the other policy, and vice-versa.

We split into two parts. Part 1 is showing that if ${\pi }_{pa}\ge {\pi }_{st}$, and the distance between ${\pi }_{pa}$ and ${\pi }_{st}$ is $\delta$ or less, the distance between their respective preimages is low. Part 2 is showing that if ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$ are $\delta$ apart, then we can exploit part 1 to get that the distance between the preimages is low, and will be pretty easy after we get part 1.

Our Hausdorff-continuity condition links $\delta$ and $ϵ$. So, when we fix an $ϵ$ and are like "how close do the policies have to be to guarantee the preimages are $ϵ$ apart", pick the $\delta$ that gets you $\frac{ϵ}{3}$ distance w.r.t our original Hausdorff-continuity condition, and also have $\delta <\frac{ϵ}{3}$.

Our first part uses $M^{lo}$ and $M^{{}^{\prime }lo}$ for points in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)\cap NF$, $M^{{}^{\prime }mid}$ for a point in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)\cap NF$, $M^{hi}$ and $M^{{}^{\prime }hi}$ for points in $M^a\left(\infty \right)$ (that are expressible as a finite mix of points from ${\Theta }^{\omega }\left(\pi \right)$ for varying $\pi$), and $M,M^{\prime }$ for two more general points in $M^a\left(\infty \right)$.

So, one half of showing the two preimages are close to each other is trivial. Everything in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)\cap NF$ projects down into $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)\cap NF$ by consistency, and projection preserving nirvana-freeness, so the preimage associated with ${\pi }_{st}$ is a superset of the preimage associated with ${\pi }_{pa}$, so there's distance 0 from a point in the ${\pi }_{pa}$ preimage to a point in the ${\pi }_{st}$ preimage.

The other half is trickier. Pick an arbitrary point $M$ in $\left(p{r}_{\ast }^{\infty ,{\pi }_{st}}{)}^{-1}\right(\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)\cap NF)$ and $\lambda$ is the $\lambda$ value of this point. $M$ projects down to some point $M^{lo}$ in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)\cap NF$. From nirvana-free consistency, $M\in c.h\left({\bigcup }_{\pi \ge {\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\cap NF\left)\right)$ ,$M^{lo}$ can then be produced by (keeping in mind that it doesn't matter whether we mix before or after projecting) finitely many points in varying ${\Theta }^{\omega }\left(\pi \right)\cap NF$ sets that are mixed to make a point $M^{hi}$, and then projected down.

Important note: $M^{hi}$ is not necessarily equal to, or even close to, $M$.

Because ${\pi }_{st}$ is within $\delta$ of ${\pi }_{pa}$, every policy above ${\pi }_{st}$ has a corresponding policy within $\delta$ that lies above ${\pi }_{pa}$. Thus, we can perturb the component points (indexed by i) that mix to make $M^{hi}$ by $\frac{ϵ}{3}(1+{\lambda }_i)$ (infinitary Hausdorff-uniform-continuity Lemma 15 variant, $\delta$ was assumed to be small enough for that to be the case), and mix them, to get a $M^{{}^{\prime }hi}$ in $c.h\left({\bigcup }_{\pi \ge {\pi }_{pa}}\left({\Theta }^{\omega }\right(\pi )\cap NF)\right)$

$M^{{}^{\prime }hi}$ projects down to $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)\cap NF$ to make a $M^{{}^{\prime }mid}$, and further projects down to $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)\cap NF$ to make a $M^{{}^{\prime }lo}$. Because $M^{{}^{\prime }hi}$ is within $\frac{ϵ}{3}(1+\lambda )$ of $M^{hi}$, projecting down (and projecting being nonexpansive) means that $M^{{}^{\prime }lo}$ is within $\frac{ϵ}{3}(1+\lambda )$ of $M^{lo}$.

Now, we can take $M^{{}^{\prime }lo}$ and fill in all the missing measure data to get a $M^{\prime }$ that projects down onto $M^{{}^{\prime }mid}$ (certifying that it's in the preimage of $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)\cap NF)$ as follows. Our most important constraint is that, when extending $m^{{}^{\prime }lo}$, it should perfectly mimic $m^{{}^{\prime }mid}$ so it can project down onto it. Our second constraint is that, if $m^{lo}$ doesn't specify what comes after a finite history and it doesn't conflict with the first constraint, it should exactly mimic the conditional probabilities of $m$. Also, our $\delta$ fixes a first time n (ie, ${\log }_{\gamma }\left(\delta \right)$) at which ${\pi }_{pa}$ is defined where ${\pi }_{st}$ isn't, so all conflicts of the second constraint with the first constraint must happen after then. This does the following: We can slice the histories assigned measures by $M^{\prime }$ into three parts.

Part 1 is prefixes of histories in $F\left({\pi }_{st}\right)$. There's only $ϵ(1+\lambda )$ difference in these between $M$ and $M^{\prime }$ (after all, projecting down to ${\pi }_{st}$ leaves these unchanged, and $M$/$M^{\prime }$ project down to $M^{lo}$ and $M^{{}^{\prime }lo}$ which are only $\frac{ϵ}{3}(1+\lambda )$ apart).

Part 2 is histories which have as a prefix something in $F\left({\pi }_{st}\right)$ less than length n. In that case, we're mimicking the conditional probabilities of $m$.

Part 3 is histories which have as a prefix something in $F\left({\pi }_{st}\right)$ of length n or higher. Because this is the threshold where ${\pi }_{pa}$ and ${\pi }_{st}$ start differing, we've got to obey the $m^{{}^{\prime }mid}$ probabilities. But this only occurs after time n.

Let's analyze the difference between $M^{\prime }$ and $M$, shall we? Our two relevant results are Vanessa's folk result that two distributions that differ by an amount will differ by the same amount if we extend them with the same conditional probabilities, and the result from the proof of Lemma 15 that arbitrarily reshuffling the measure/amount of dirt after time n takes ${\gamma }^n{\lambda }^{\prime }$effort, where ${\lambda }^{\prime }$ is the $\lambda$ value of the measure you're reshuffling.

So, we start off with a $\frac{ϵ}{3}(1+\lambda )$ distance (includes the $b$ term) between $M^{lo}$ and $M^{{}^{\prime }lo}$. Then, extending up further to fill in everything up till time n, $m$ and $m^{\prime }$ mimic the conditional probabilities of each other. Still a $\frac{ϵ}{3}(1+\lambda )$ distance between them at this stage. Finally, after time n, $M^{\prime }$ may go its own arbitrary way because it's gotta be compliant with $M^{{}^{\prime }mid}$, and to reshuffle this around, it takes ${\gamma }^n{\lambda }^{\prime }$ effort. So, the net distance between $M$ (arbitrary point in the preimage of $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)$, and $M^{\prime }$ (specially crafted point in the preimage of $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$ is below $\frac{ϵ}{3}(1+\lambda )+{\gamma }^n{\lambda }^{\prime }$.

Wait, n (time of first difference) was ${\log }_{\gamma }\left(\delta \right)$ since ${\pi }_{st}$ and ${\pi }_{pa}$ are only $\delta$ apart, and ${\lambda }^{\prime }$ can be at most $\lambda +\frac{ϵ}{3}(1+\lambda )$ because $\lambda$ values are preserved by projections, and $M^{lo}$ and $M^{{}^{\prime }lo}$ are only $\frac{ϵ}{3}(1+\lambda )$ distance apart, so no more than that amount of dirt is the difference between the two. Finally, we assumed $\delta <\frac{ϵ}{3}$. So, we get:

$d(M,M^{\prime })\le \frac{ϵ}{3}(1+\lambda )+{\gamma }^n{\lambda }^{\prime }=\frac{ϵ}{3}(1+\lambda )+{\gamma }^{{\log }_{\gamma }\left(\delta \right)}(\lambda +\frac{ϵ}{3}(1+\lambda \left)\right)<\frac{ϵ}{3}(1+\lambda )+\delta (2\lambda +2)$

$<\frac{ϵ}{3}(1+\lambda )+\frac{ϵ}{3}(2\lambda +2)=ϵ(1+\lambda )$

And we have our appropriate distance bound between preimages! Now to use this in part 2, which should go a lot faster.

Time for part 2, to get full generality. Pick two partial policies ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$ and assume the distance between them is $\delta$. Then, the stub ${\pi }_{st}$ given by "$inf({\pi }_{pa},{\pi }_{pa}^{\prime })$ but cut it off so it's undefined after time n (where n is ${\log }_{\gamma }\left(\delta \right)$)" is within distance $\delta$ of both ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$. Further, ${\pi }_{st}\le {\pi }_{pa},{\pi }_{pa}^{\prime }$. Then, take some point in the preimage of ${\pi }_{pa}$. It's also in the preimage of ${\pi }_{st}$. Because ${\pi }_{st}$ is at a distance of $\delta$ from ${\pi }_{pa}^{\prime }$, we only have to go $ϵ(1+\lambda )$ distance to get a point in the preimage of ${\pi }_{pa}^{\prime }$, and then reverse ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$ and we're done!

By Lemma 15, this establishes uniform continuity for the function mapping partial policies to the preimage of their nirvana-free part in the space of all nirvana-free measures over infinite histories.

Condition 4: Nirvana-free upper completeness (causal case) 

Now that we've nabbed every nice condition other than this one, we can invoke Lemma 21 (we only require upper completion on the infinite levels, which we have) to get that the nirvana-free part is the (closed) convex hull of the projections of nirvana-free stuff from above. Then, just appeal to lemmas 11, 12, and 13, that the closed convex hull of projections of nirvana-free upper-complete sets is nirvana-free upper-complete.

Condition C: Causality.

We showed part of this all the way back in our consistency argument. For causal/surcausal, $p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left({\Theta }^{\omega }\right(\pi \left)\right)=\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$ regardless of which $\pi \ge {\pi }_{pa}$ we picked. We'll be using this.

Pick some arbitrary ${\pi }_{pa}$ and $M\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$. $M$ has a preimage point $M^{\pi }\in {\Theta }^{\omega }\left(\pi \right)$ where $\pi \ge {\pi }_{pa}$. We get an outcome function $o{f}^{\omega }$ mapping policies to points in their associated sets s.t. $o{f}^{\omega }\left(\pi \right)=M^{\pi }$. Extend this $o{f}^{\omega }$ to all points by defining $of\left({\pi }_{pa}^{\prime }\right):=p{r}_{\ast }^{{\pi }^{\prime },{\pi }_{pa}^{\prime }}\left(o{f}^{\omega }\right({\pi }^{\prime }\left)\right)$

Ok, we need to show that: This actually singles out a unique point and isn't an invalid definition, said point is in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}^{\prime }\right)$, that $of\left({\pi }_{pa}\right)=M$, and that it's an outcome function.

Assuming this is actually well-defined, $of\left({\pi }_{pa}^{\prime }\right)$ is in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}^{\prime }\right)$ trivially because it's a projection of a point from above. Also, $of\left({\pi }_{pa}\right)=p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left(o{f}^{\omega }\right(\pi \left)\right)=p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left(M^{\pi }\right)=M$ which clean ups that part. Now for showing that it's an outcome function.

$p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left(of\right({\pi }_{pa}^{hi}\left)\right)=p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left(p{r}_{\ast }^{\pi ,{\pi }_{pa}^{hi}}\right(o{f}^{\omega }\left(\pi \right)\left)\right)=p{r}_{\ast }^{\pi ,{\pi }_{pa}^{lo}}\left(o{f}^{\omega }\right(\pi \left)\right)=of\left({\pi }_{pa}^{lo}\right)$

So, we got everything assuming the extension is well-defined, let's show that. Pick any two $\pi ,{\pi }^{\prime }$ above any ${\pi }_{pa}$. We'll show that they project to the same point.

$p{r}_{\ast }^{\pi ,{\pi }_{pa}}\left(o{f}^{\omega }\right(\pi \left)\right)=p{r}_{\ast }^{inf(\pi ,{\pi }^{\prime }),{\pi }_{pa}}\left(p{r}_{\ast }^{\pi ,inf(\pi ,{\pi }^{\prime })}\right(o{f}^{\omega }\left(\pi \right)\left)\right)=p{r}_{\ast }^{inf(\pi ,{\pi }^{\prime }),{\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }^{\prime },inf(\pi ,{\pi }^{\prime })}\right(o{f}^{\omega }\left({\pi }^{\prime }\right)\left)\right)$

$=p{r}_{\ast }^{{\pi }^{\prime },{\pi }_{pa}}\left(o{f}^{\omega }\right({\pi }^{\prime }\left)\right)$

And we're done with causality! Now for pseudocausality.

Condition P: Pseudocausality.

We'll do this in two steps. One is showing that for stubs, points which meet the appropriate conditions are also present in all the requisite other stubs. Step 2 is generalizing this to all points in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$. 

Let's say you have some $M\in \downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}\right)$. By Nirvana-free consistency, $M\in c.h\left({\bigcup }_{\pi \ge {\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)$ so we can shatter it into finitely many $M_i$ that are projections of stuff from above, $M_i^{\infty }$. The support of the measure component of $M$ is a subset of $F^{NF}\left({\pi }_{st}\right)\cap F^{NF}\left({\pi }_{st}^{\prime }\right)$, so the same must apply to all the $M_i$.

Now, what we can do is make a ${\pi }_i^{\prime }$ that mimics the behavior of ${\pi }_i$ for all prefixes and extensions of strings in $F^{NF}\left({\pi }_{st}\right)\cap F^{NF}\left({\pi }_{st}^{\prime }\right)$, but otherwise mimics ${\pi }_{st}^{\prime }$, and extends if needed in some random-ass way, and is above ${\pi }_{st}^{\prime }$.

The reason we can do this is because, if there's a contradiction in this construction, it would be from ${\pi }_{st}^{\prime }$ and ${\pi }_i$ behaving differently on some prefix or extension of a string in $F^{NF}\left({\pi }_{st}\right)\cap F^{NF}\left({\pi }_{st}^{\prime }\right)$. But, ${\pi }_{st}^{\prime }$ can't specify what to do for any nodes in $F^{NF}\left({\pi }_{st}^{\prime }\right)$ or later (because $F^{NF}\left({\pi }_{st}^{\prime }\right)$ is basically a coat of leaf observation nodes around the extent of ${\pi }_{st}^{\prime }$), and if ${\pi }_{st}^{\prime }$ and ${\pi }_i$ differ on a strict prefix of something in $F^{NF}\left({\pi }_{st}\right)\cap F^{NF}\left({\pi }_{st}^{\prime }\right)$, then that means that ${\pi }_{st}^{\prime }$ and ${\pi }_{st}$ branch different ways so there's no node in both $F^{NF}\left({\pi }_{st}\right)$ and $F^{NF}\left({\pi }_{st}^{\prime }\right)$ after the branch point, so again we get a contradiction.

Anyways, we've crafted our finitely many ${\pi }_i^{\prime }$ which lie above ${\pi }_{st}^{\prime }$, and mimic ${\pi }_i$ going forward. Our $M_i^{\infty }$ is an extension of $M_i$ whose measure component is only supported on $F^{NF}\left({\pi }_{st}\right)\cap F^{NF}\left({\pi }_{st}^{\prime }\right)$. Also, before and past that, ${\pi }_i^{\prime }$ mimics ${\pi }_i$ perfectly, so we can transfer $M_i^{\infty }$ to ${\Theta }^{\omega }\left({\pi }_i^{\prime }\right)$ by infinitary pseudocausality. Do this for all the i. Then, projecting all those down to ${\pi }_{st}^{\prime }$, we get that all the $M_i$ lie in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}^{\prime }\right)$, and mixing them together, we get that $M$ itself lies in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{st}^{\prime }\right)$.

Now for part 2, where we show it for partial policies in general. Let $M$ be arbitrary in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right)$. Project $M$ down to all the ${\pi }_{pa}^n$ to make a sequence of $M_n$. Since the support of the measure component of $M$ is a subset of $F\left({\pi }_{pa}\right)\cap F\left({\pi }_{pa}^{\prime }\right)$ (pseudocausality assumption) the support of the measure component of $M_n$ is a subset of $F\left({\pi }_{pa}^n\right)\cap F\left({\pi }_{pa}^{{}^{\prime }n}\right)$, so by pseudocausality for stubs which we've shown, $M_n$ is also present in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}^{{}^{\prime }n}\right)$. Then, take the preimage in ${\pi }_{pa}^{\prime }$ of all those $M_n$ points. By consistency and Lemma 9 and the usual argument about "there can only be one preimage point for a series of points", we get that $M$ itself (the only thing that could project down on $M_n$ for all n) lies in $\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}^{\prime }\right)$ and we're done with pseudocausality.

 

Alright, that's most of the proof out of the way, all that's left is showing that the full belief function conditions imply the finitary and infinitary versions, respectively, and getting isomorphism. Let's begin.

Let's check whether ${\to }^{st}\left(\Theta \right)$ makes a stub-hypothesis, and whether ${\to }^{\omega }\left(\Theta \right)$ makes an infinitary-hypothesis, if $\Theta$ is a hypothesis/fulfills all the conditions. ${\to }^{st}$ is just "restrict $\Theta$ to only reporting sets for stubs", and ${\to }^{\omega }$ is just "restrict $\Theta$ to only reporting sets for full policies"

The variants of nonemptiness, closure, convexity, nirvana-free upper-completion, bounded minimals, hausdorff-continuity, and pseudocausality for the finite and infinite case are trivially implied by the corresponding condition for hypotheses, leaving the four moderately nontrivial cases of the analogues of normalization, consistency, the extreme point condition, and causality.

Extreme point condition: The infinitary case doesn't have an analogue of the extreme point condition. So that leaves the finitary case. What we can do is take a nirvana-free extreme minimal point $M^{ex}$ in some $\Theta \left({\pi }_{st}\right)$, apply the general extreme point condition to get a nirvana-free $M\in \Theta \left(\pi \right)$ for some suitable $\pi$ that projects down to $M^{ex}$, and, clipping away the infinite parts by ${\to }^{st}$, the projections of $M$ fill the role of the points in $\Theta \left({\pi }_{st}^{\prime }\right)$ all below some policy that project down to $M^{ex}$.

Causality. The finite case is that we can take a point associated with some stub, and craft an outcome function for stubs that matches up with our point. This is trivially implied by the general case of causality, where you can take any partial policy and point and get an outcome function that matches up with it. The infinite case is that we can take a point $M$ in $\Theta \left(\pi \right)$, and get points for all the other $\Theta \left({\pi }^{\prime }\right)$ that project down appropriately. For this, again, we just take an outcome function for $M$ and clip it off to the infinite levels.

Consistency: The finite case of weak consistency is pretty easy. We get

$p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left({\to }^{st}\right(\Theta \left)\right({\pi }_{st}^{hi}\left)\right)=p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(\Theta \right({\pi }_{st}^{hi}\left)\right)\subseteq \Theta \left({\pi }_{st}^{lo}\right)=\left({\to }^{st}\right)\left(\Theta \right)\left({\pi }_{st}^{lo}\right)$

Where the subset came from full consistency because everything is the closed convex hull of projections from above, so projecting down gets you a subset. For the Nirvana-free consistency condition for the infinite case, it's a simple consequence of Lemma 21.

Normalization: ${inf}_{{\pi }_{st}}{E}_{\Theta \left({\pi }_{st}\right)}\left(0\right)=0$ and ${sup}_{{\pi }_{st}}{E}_{\Theta \left({\pi }_{st}\right)}\left(1\right)=1$

To begin, the normalization condition for infinitary hypotheses and general hypotheses is the exact same, so we can ignore that and work on the stub hypothesis case. The inf one is pretty easy. From general normalization, at the infinite level, there are $\pi$ and nirvana-free points in $\Theta \left(\pi \right)$ with a $b$ value at-or-near zero, and you can just project them down to any stub you want.

The sup one is a bit trickier. It's obviously not above 1, because no matter what policy $\pi$ you pick, you've got a nirvana-free point with $\lambda +b\le 1$ in $\Theta \left(\pi \right)$, which you can project down to whichever stub you're looking at, to certify that the expectation of 1 is 1 or less. Showing that it isn't below 1 is a bit harder.

Let's say there's some $\pi$ where ${min}_{(\lambda \mu ,b)\in \Theta \left(\pi \right)\cap NF}(\lambda +b)=1$ (or arbitrarily close to 1, doesn't really matter, although we'll show later that there is indeed a maximizing policy where Murphy can only force a value of 1)

From Hausdorff-continuity, pick some $\delta$ to get an $ϵ$ Hausdorff-distance or lower. This $\delta$ specifies some extremely large n, consider ${\pi }^n$. Now, consider the set of every policy ${\pi }^{\prime }$ above ${\pi }^n$. All of these are $\delta$ or less away from $\pi$.

By Hausdorff-continuity, there can't be a nirvana-free point in any $\Theta \left({\pi }^{\prime }\right)$ with $\lambda +b<1-ϵ$, because we could do an $ϵ$ perturbation to get a point in $\Theta \left(\pi \right)\cap NF$ with $\lambda +b<1$, because small changes in $M$ induce small changes in $\lambda$ and $b$. Or, we can add a little bit of wiggle room if the minimizing value of $\lambda +b$ in $\pi$ is slightly less than 1

However, any nirvana-free point in $\Theta \left({\pi }^n\right)$ must originate as a mix of finitely many points from $\Theta \left({\pi }_i^{\prime }\right)\cap NF$ (varying ${\pi }_i^{\prime }$ as long as it's above ${\pi }^n$) that have been projected down. This is because, by our earlier proof of nirvana-free consistency from consistency in general, $\Theta \left({\pi }^n\right)\cap NF=c.h\left({\bigcup }_{{\pi }^{\prime }\ge {\pi }^n}p{r}_{\ast }^{{\pi }^{\prime },{\pi }^n}\right(\Theta \left({\pi }^{\prime }\right)\cap NF\left)\right)$

All of these projected points have $\lambda +b\ge 1-ϵ$, so the mix point has $\lambda +b\ge 1-ϵ$, so Murphy can only force a value of $1-ϵ$ or higher. And we can make $\delta$ as small as we wish to get a stub ${\pi }^n$ below $\pi$ (n extremely large) where $ϵ$ is as small as we wish, so the sup of the $\lambda +b$ values Murphy can force over all stubs can't be below 1. So it must be 1.

Isomorphism! Let's go! As a quick recap, 

$\downarrow \left({\Theta }^{\omega }\right)\left({\pi }_{pa}\right):=\overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)$

$\uparrow \left({\Theta }^{st}\right)\left({\pi }_{pa}\right):={\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\right({\Theta }^{st}\left({\pi }_{st}\right))$

And ${\to }^{\omega }$/${\to }^{st}$ is just "clip down your hypothesis to full policies/stubs".

So, two parts of this are trivially easy. From earlier in the proof (the start of the first section for the stub one, and an obvious corollary of definitions for the full policy one), we established that $\uparrow \left({\Theta }^{st}\right)\left({\pi }_{st}\right)={\Theta }^{st}\left({\pi }_{st}\right)$ and $\downarrow \left({\Theta }^{\omega }\right)\left(\pi \right)={\Theta }^{\omega }\left(\pi \right)$. Using this, ${\to }^{\omega }(\downarrow ({\Theta }^{\omega }\left)\right)\left(\pi \right)=\downarrow \left({\Theta }^{\omega }\right)\left(\pi \right)={\Theta }^{\omega }\left(\pi \right)$ and${\to }^{st}(\uparrow ({\Theta }^{st}\left)\right)\left({\pi }_{st}\right)=\uparrow \left({\Theta }^{st}\right)\left({\pi }_{st}\right)={\Theta }^{st}\left({\pi }_{st}\right)$

So, ${\to }^{\omega }(\downarrow ({\Theta }^{\omega }\left)\right)={\Theta }^{\omega }$ and ${\to }^{st}(\uparrow ({\Theta }^{st}\left)\right)={\Theta }^{st}$

Let's get fancier and show the other two.

$\downarrow \left({\to }^{\omega }\right(\Theta \left)\right)\left({\pi }_{pa}\right)=\overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right({\to }^{\omega }\left(\Theta \right)\left(\pi \right)\left)\right)=\overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(\Theta \left(\pi \right)\left)\right)=\Theta \left({\pi }_{pa}\right)$

The first two equalities are unpacking definitions, the third is consistency for $\Theta$.

$\uparrow \left({\to }^{st}\right(\Theta \left)\right)\left({\pi }_{pa}\right)={\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\right({\to }^{st}\left(\Theta \right)\left({\pi }_{st}\right))={\bigcap }_{{\pi }_{st}\le {\pi }_{pa}}(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{st}}{)}^{-1}\left(\Theta \right({\pi }_{st}\left)\right)=\Theta \left({\pi }_{pa}\right)$

Again, first two equalities are unpacking definitions, the third is consistency for $\Theta$. So, $\downarrow \left({\to }^{\omega }\right(\Theta \left)\right)=\Theta$ and $\uparrow \left({\to }^{st}\right(\Theta \left)\right)=\Theta$

Putting it together, ${\to }^{\omega }$ and $\downarrow$ make an isomorphism between ${\Theta }^{\omega }$ and $\Theta$, and ${\to }^{st}$ and $\uparrow$ make an isomorphism between ${\Theta }^{st}$ and $\Theta$. We're finally done!

Proposition 1: If $\Theta$ fulfills the causality condition, nonemptiness, closure, and convexity, then $S^{\Theta }$ is a nonempty, closed, convex set of a-environments or a-survironments. ${\Theta }^{S^{\Theta }}=\Theta$. Also, $S\subseteq S^{{\Theta }^S}$.

Ok, what $S^{\Theta }$ is, is the set of a-environments $(\lambda e,b)$ where, regardless of ${\pi }_{pa}$, $\left(\lambda \right({\pi }_{pa}\cdot e),b)$ lies in $\Theta \left({\pi }_{pa}\right)$. For nonemptiness, pick some arbitrary point in one of your $\Theta \left({\pi }_{pa}\right)$, use causality to get an outcome function, and then you fill in the conditional probabilities for an action-observation sequence with your outcome function points. This never produces a contradiction anywhere because if there was a contradiction, you'd be able to project two specified points down and have them disagree somewhere, which is impossible because we have an outcome function.

For closure, if you take a limit of a-environments, this makes a limiting sequence in all the $S^{\Theta }$, which are all closed, so the limit point environment has all its induced distributions lying in the usual $\Theta \left({\pi }_{pa}\right)$, and is in $S^{\Theta }$

For convexity, if you take a mix of a-environments, this makes the same mix in all the $S^{\Theta }$ which are all convex, so the mixed environment has all its induced distributions lying in the usual $\Theta \left({\pi }_{pa}\right)$, and is in $S^{\Theta }$.

For equality, if $M\in {\Theta }^{S^{\Theta }}\left({\pi }_{pa}\right)$, then it originated from some a-environment made from an outcome function for $\Theta$, which... just gets your original point so $M\in \Theta \left({\pi }_{pa}\right)$. In the other direction, if $M\in \Theta \left({\pi }_{pa}\right)$, by causality, we can project down and extend the specification and make an a-environment that acts like $M$ on ${\pi }_{pa}$, and then going back gets you $M\in {\Theta }^{S^{\Theta }}\left({\pi }_{pa}\right)$. 

In the other direction, if $(\lambda e,b)\in S$, then it induces an outcome function and you can go back from that to $(\lambda e,b)\in S^{{\Theta }^S}$, so $S\subseteq S^{{\Theta }^S}$

Theorem 3.1: Pseudocausal Translation: For all pseudocausal ${\Theta }^{st}$ hypotheses defined only  on policy stubs, ${\to }^c\left({\Theta }^{st}\right)$ is a causal hypothesis only defined on policy stubs. ${\to }^{NF}\left({\to }^c\right({\Theta }^{st}\left)\right)={\Theta }^{st}$. For all causal ${\Theta }^{st}$ hypotheses defined only on policy stubs, ${\to }^{NF}\left({\Theta }^{st}\right)$ is a pseudocausal hypothesis only defined on policy stubs.

Theorem 3.2: Acausal Translation: For all acausal ${\Theta }^{st}$ hypotheses defined only on policy stubs, ${\to }^{sc}\left({\Theta }^{st}\right)$ is a surcausal hypothesis only defined on policy stubs. ${\to }^{NF}\left({\to }^{sc}\right({\Theta }^{st}\left)\right)={\Theta }^{st}$. For all surcausal ${\Theta }^{st}$ hypotheses defined only on policy stubs, ${\to }^{NF}\left({\Theta }^{st}\right)$ is an acausal hypothesis only defined on policy stubs.

Both these theorems have highly similar proofs, so let's group them together. First, we'll need to set up how ${\to }^c$ and ${\to }^{sc}$ work, and then knock out two lemmas we'll need before we can proceed to the main result. ${\to }^c$ is defined by ${\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}\right)=\overline{c.h}\left({\bigcap }_{{\pi }_{st}^{\prime }\le {\pi }_{st}}\right(I_{\ast }^{{\pi }_{st}^{\prime },{\pi }_{st}}\left({\Theta }^{st}\right({\pi }_{st}^{\prime }\left)\right)\left)\right)$${\to }^{sc}$ is defined identically, just with $I_{\ast s}$ instead of $I_{\ast }$, and closed convex hull permitting us to mix with $0^{+}$ probability.

$I^{{\pi }_{st}^{\prime },{\pi }_{st}}$ where ${\pi }_{st}^{\prime }\le {\pi }_{st}$ (this is like the inverse of projection, it's going up instead of down) is a function $F\left({\pi }_{st}^{\prime }\right)\to F\left({\pi }_{st}\right)$ defined by: If $h\in F\left({\pi }_{st}^{\prime }\right)\cap F\left({\pi }_{st}\right)$, then $I^{{\pi }_{st}^{\prime },{\pi }_{st}}\left(h\right)=h$. If $h\in F\left({\pi }_{st}^{\prime }\right)$ and isn't in $F\left({\pi }_{st}\right)$, then $I^{{\pi }_{st}^{\prime },{\pi }_{st}}\left(h\right)=h{\pi }_{st}\left(h\right)N$

$I_{\ast }^{{\pi }_{st}^{\prime },{\pi }_{st}}$ from $M^a\left(F\right({\pi }_{st}^{\prime }\left)\right)\to M^a\left(F\right({\pi }_{st}\left)\right)$ is just pushing $(m,b)$ through the mapping $I^{{\pi }_{st}^{\prime },{\pi }_{st}}$. You keep the $b$ term the same, and push the measure terms up. $I_{\ast s}^{{\pi }_{st}^{\prime },{\pi }_{st}}$ is defined identically on the measure part, except that it has the rule that all nirvana events in $F\left({\pi }_{st}\right)$ and not in $F\left({\pi }_{st}^{\prime }\right)$ with 0 measure get $0^{+}$ measure instead.

Intuitively, what $I_{\ast }$ and $I_{\ast s}$ are doing, is capping off whatever they need to (in order to extend appropriately) with Nirvana. $I_{\ast }$ is capping off positive-probability histories with guaranteed Nirvana immediately afterwards, where $I_{\ast s}$ is more paranoid and caps off every 0-probability Nirvana history that got added with "it is possible that Nirvana occurs here".

Let's go over some properties that $I_{\ast }$ and $I_{\ast s}$ fulfill. $I_{\ast }$ is an injective continuous map $M^a\left(F\right({\pi }_{st}^{\prime }\left)\right)\to M^a\left(F\right({\pi }_{st}\left)\right)$, and $I_{\ast s}$ is an injective continuous map ${SM}^a\left(F\right({\pi }_{st}^{\prime }\left)\right)\to {SM}^a\left(F\right({\pi }_{st}\left)\right)$. $I_{\ast }$ and $I_{\ast s}$ are undone by projecting back down, $p{r}_{\ast }^{{\pi }_{st},{\pi }_{st}^{\prime }}\left(I_{\ast }^{{\pi }_{st}^{\prime },{\pi }_{st}}\right(M\left)\right)=M$. Both $I_{\ast }$ and $I_{s\ast }$ are linear, the latter in the stronger sense that it's linear when you mix stuff with $0^{+}$ probability, it doesn't matter whether you mix before or after injecting up. Further, injections up commute, $I_{\ast }^{{\pi }_{st}^{\prime },{\pi }_{st}}\left(I_{\ast }^{{\pi }_{st}^{\prime \prime },{\pi }_{st}^{\prime }}\right(M\left)\right)=I_{\ast }^{{\pi }_{st}^{\prime \prime },{\pi }_{st}}\left(M\right)$, and the same for $I_{\ast s}$.

In order to make progress, we want to get two important lemmas. The first one, Lemma 22, is that slicing away the nirvana from this thing recovers the original pseudocausal hypothesis. The second one I call the "Diamond Lemma", and it says that injecting up and projecting down is the same as projecting down and then injecting up, and if you sketch it out, it looks like a diamond.

Lemma 22: $\left({\to }^c\right({\Theta }^{st}\left)\right({\pi }_{st}\left)\right)\cap NF={\Theta }^{st}\left({\pi }_{st}\right)$, and the same holds for ${\to }^{sc}$.

Proof sketch: One direction is trivial, the other direction that ${\to }^c$ doesn't add any new nirvana-free points is trickier. Working in the pseudocausal-to-causal setting, we can take some $M$ that's nirvana-free in the closed convex hull, and get a sequence $M_n$ limiting to it where each $M_n$ is in the convex hull. Now, indexing stubs below ${\pi }_{st}$ by i, the $M_n$ can all be viewed as a mix of $M_{i,n}$ points projected up from below. The problem is, the mix varies as n does. What we can do is separate into "good" i where we can get a suitable limit point and limit probability, and "bad" i that we have to treat as a special chunk, and reexpress $M$ as a sum of a probabilistic mix of "good" $M_i$ injected up, and an additional "bad" chunk. We can show that the "good" $M_i$ can all be transferred up to ${\Theta }^{st}\left({\pi }_{st}\right)$ itself by pseudocausality and mixed in there, and the "bad" chunk is a nirvana-free a-measure. So, $M$ is the sum of a point in ${\Theta }^{st}\left({\pi }_{st}\right)$, plus a nirvana-free a-measure, so $M$ lies in ${\Theta }^{st}\left({\pi }_{st}\right)$ by nirvana-free upper completion.

Working in the surcausal-to-acausal setting, we take our $M$ in the closed convex hull and a sequence $M_n$, but injection up in this setting is much more effective at adding Nirvana, and the surmetric is much more sensitive than the usual metric for noticing the presence or absence of Nirvana. So, only an initial segment of $M_n$ is "contaminated" with Nirvana since the limit point is Nirvana-free, and we can clip that part off, and the "uncontaminated tail" can only have come from ${\Theta }^{st}\left({\pi }_{st}\right)$ itself because injection up is very aggressive with adding Nirvana, so we get it from just closure on ${\Theta }^{st}\left({\pi }_{st}\right)$.

Proof: For $\left({\to }^c\right({\Theta }^{st}\left)\right({\pi }_{st}\left)\right)\cap NF\supseteq {\Theta }^{st}\left({\pi }_{st}\right)$, just observe that the identity injection $I_{\ast }^{{\pi }_{st},{\pi }_{st}}$ leaves ${\Theta }^{st}\left({\pi }_{st}\right)$ completely unchanged and adds no nirvana, so any point in ${\Theta }^{st}\left({\pi }_{st}\right)$ also lies in the closed convex hull of the injections up, and is nirvana-free because the original point that we mapped through identity was nirvana-free. This works with the surcausal case too.

Now for the considerably more difficult reverse direction, for the pseudocausal-to-causal case first.

If $M\in {\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}\right)\cap NF$, then unpacking that, $M$ is nirvana-free, and lies in the closed convex hull of the injections up. So, we can fix a sequence $M_n$ in the convex hull of injections up that limits to $M$. Index the stubs below ${\pi }_{st}$ by i, there's only finitely many of them.

The $M_n$ can be written as ${\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\left(M_{i,n}\right)$ where $M_{i,n}\in {\Theta }^{st}\left({\pi }_{st}^i\right)$. ${\zeta }_{i,n}$ may be 0. This is because, if there's multiple points in the injection of a particular stub that are mixed, you can mix them before injecting up to get a single $M_{i,n}$ that's injected up, because injections are linear and we're injecting a convex set.

Blessed by the gift of finitely many i to worry about, use repeated picking of subsequences to get a subsequence of n where:

For all i, ${\zeta }_{i,n}$ converges. Call the limiting values ${\zeta }_i$. Now split the i into good i where ${\zeta }_i>0$, and bad i where ${\zeta }_i=0$. The ${\zeta }_i$ will sum up to 1.

For all good i, ${\zeta }_{i,n}>0$ always. The fact that they all limit to above 0 helps you out because you only have to trim off an initial segment.

For all good i, $M_{i,n}$ converges, call the limit point $M_i$. This is because injection up preserves $\lambda$ and $b$, and ${\zeta }_{i,n}$ is bounded above 0, so the $\lambda +b$ value of the $M_{i,n}$ is upper-bounded by $\frac{{\lambda }^{\prime }+b^{\prime }}{{min}_n{\zeta }_{i,n}}$, which is a finite nonnegative number divided by a finite positive number, and we can apply the Compactness Lemma to establish that a convergent subsequence exists. In this case, ${\lambda }^{\prime }+b^{\prime }$ is the bound as a whole for the sequence $M_n$, which converges so it must have a bound of that form, and not the bound on minimal points.

Finally, ${\sum }_{\text{bad}i}\left({\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\right(M_{i,n}\left)\right)$ converges. This is doable because the sequence $M_n$ has bounded $\lambda$ and $b$ because it converges to something, so the partial sum of bad i has the same bound, so we can invoke the Compactness Lemma to get our convergent subsequences.

Putting all this together (we kept selecting from compact sets so that is what let us build a subsequence with all these great properties at once) we have a decomposition of $M$ itself into:

${\sum }_{\text{good}i}\left({\zeta }_i{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\right(M_i\left)\right)+{lim}_{n\to \infty }\left({\sum }_{\text{bad}i}\left({\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\right(M_{i,n}\left)\right)\right)$

Now, since ${\sum }_{\text{good}i}{\zeta }_i=1$ (all the bad i had their probability components limit to 0), that first sum part looks like an actual mixture of points injected up! Since $M$ is nirvana-free, both parts must be nirvana-free, and the sums are also a-measures.

First, by closure of all our original ${\Theta }^{st}\left({\pi }_{st}^i\right)$, all the $M_i$ components (where i is good) do lie in ${\Theta }^{st}\left({\pi }_{st}^i\right)$. And when we inject the $M_i$ up,  since the mix of them is nirvana-free, this means that each individual $M_i$ must be nirvana-free after injection.

Now, what injection does, is it caps Nirvana on everything that is in $F\left({\pi }_{st}^i\right)$ and not in $F\left({\pi }_{st}\right)$ that has positive probability. So, if $M_i$ is nirvana-free after injection, this must mean that its measure component is only supported on $F\left({\pi }_{st}\right)$. Via pseudocausality, this means that $M_i$ lies in ${\Theta }^{st}\left({\pi }_{st}\right)$ itself! Also, $I_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\left(M_i\right)=M_i$.

So, our sum over good i components (by convexity), is actually a probabilistic mixture of stuff in ${\Theta }^{st}\left({\pi }_{st}\right)$ itself! Abbreviating ${\sum }_{\text{good}i}{\zeta }_i{M}_i$ as $M^{\prime }$, which lies in ${\Theta }^{st}\left({\pi }_{st}\right)$ by convexity, and rewriting the sum, we can reepress $M$ as:

$M^{\prime }+{lim}_{n\to \infty }\left({\sum }_{\text{bad}i}\left({\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\right(M_{i,n}\left)\right)\right)$

This is a nirvana-free a-measure in ${\Theta }^{st}\left({\pi }_{st}\right)$, plus a nirvana-free a-measure, so, by nirvana-free upper-completion, $M$ lies in ${\Theta }^{st}\left({\pi }_{st}\right)$ and we're done. Now, let's hit up the surcausal case.

Assume $SM\in {\to }^{NF}\left({\Theta }^{st}\right)\left({\pi }_{st}\right)\cap NF$. $SM$ is nirvana-free, and lies in the closed convex hull of the injections up. So, we can fix a sequence $S{M}_n$ in the convex hull of injections up that limits to $SM$. Index the stubs below ${\pi }_{st}$ by i, there's only finitely many of them, reserve i=0 for ${\pi }_{st}$ itself.

The $S{M}_n$ can be written as ${\sum }_i{\zeta }_{i,n}{I}_{\ast s}^{{\pi }_{st}^i,{\pi }_{st}}\left(M_{i,n}\right)$ where $M_{i,n}\in {\Theta }^{st}\left({\pi }_{st}^i\right)$. ${\zeta }_{i,n}$ may be 0 or $0^{+}$. This is because, if there's multiple points in the injection of a particular stub, you can mix them before injecting up to get your single point, one for each ${\pi }_{st}^i$, because injections are linear and we're injecting a convex set.

Note that $SM$ is nirvana-free, and there's only finitely many spots where nirvana could be since we're working in a stub, so past a certain point all the $S{M}_n$ will be nirvana-free due to the surmetric we're using. Let's clip off that initial segment that's contaminated with Nirvana. Now, we can get something very interesting. If ${\pi }_{st}^i<{\pi }_{st}$, then injecting up anything at all is going to stick nirvana (maybe with $0^{+}$ measure) somewhere. Having ${\zeta }_{i,n}$ be $0^{+}$ doesn't help you, because mixing with a nirvana-containing thing with $0^{+}$ probability means the mixture contains the nirvana-spots of that thing you mixed in. So, past a certain point, all the $S{M}_n$ can only be written as $M_{0,n}$ (the identity injection, anything else either has exactly 0 probability so it gets clipped out of the sum, or it has Nirvana somewhere and can't be present).

Therefore, in the tail, the sequence of $S{M}_n$ limiting to $SM$ is the same as $M_{0,n}\in {\Theta }^{st}\left({\pi }_{st}\right)$ limiting to some $M_0\in {\Theta }^{st}\left({\pi }_{st}\right)$, so $SM\in {\Theta }^{st}\left({\pi }_{st}\right)$ and it's actually an a-measure, not an a-surmeasure. This establishes Lemma 22 for the sur-case.

Lemma 23/Diamond Lemma: For any ${\pi }_{st},{\pi }_{st}^{\prime }$, and any ${\pi }_{st}^{hi}\ge {\pi }_{st},{\pi }_{st}^{\prime }$, and any $M\in M^a\left(F\right({\pi }_{st}\left)\right)$, then: $p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{\prime }}\left(I_{\ast }^{{\pi }_{st},{\pi }_{st}^{hi}}\right(M\left)\right)=I_{\ast }^{inf({\pi }_{st},{\pi }_{st}^{\prime }),{\pi }_{st}}\left(p{r}_{\ast }^{{\pi }_{st},inf({\pi }_{st},{\pi }_{st}^{\prime })}\right(M\left)\right)$ (and same for $I_{\ast s}$ and the sur variants)

it's called the Diamond Lemma because if you sketch out the injections as going diagonally up and the projections as going diagonally down, the commutative diagram looks like a diamond.

To begin with, we can go "hm, there's an upper bound on ${\pi }_{st}$ and ${\pi }_{st}^{\prime }$. For every finite history in $F\left({\pi }_{st}\right)$, there's an extension of that history in $F\left({\pi }_{st}^{hi}\right)$, which has a prefix in $F\left({\pi }_{st}^{\prime }\right)$, and vice-versa. This establishes that for all the finitely many histories in $F\left({\pi }_{st}\right)$, either a prefix of that history lies in $F\left({\pi }_{st}^{\prime }\right)$, or an extension of that history lies in $F\left({\pi }_{st}^{\prime }\right)$, and vice-versa for $F\left({\pi }_{st}^{\prime }\right)$

Now, we can split into three possible cases and show that up-then-down equals down-then-up in terms of what measure is assigned to a history in $F\left({\pi }_{st}^{\prime }\right)$ by mapping $M$ through the injections and projections, which shows the diamond lemma in full generality.

In the first case, our history $h$ in $F\left({\pi }_{st}^{\prime }\right)$ is also in $F\left({\pi }_{st}\right)$ (the equality case)In this case, $h$ also lies in $F(inf({\pi }_{st},{\pi }_{st}^{\prime }\left)\right)$. Projecting down to inf does nothing to the measure on $h$, and embedding up also does nothing to the measure on $h$. Embedding up to ${\pi }_{st}^{hi}$ also does nothing to the measure on $h$, and projecting down doesn't affect it either.

In the second case, our history $h$ in $F\left({\pi }_{st}^{\prime }\right)$ isn't in $F\left({\pi }_{st}\right)$, but there are strict extensions that lie in $F\left({\pi }_{st}\right)$ (this requires $h$ to be nirvana-free). $h$ is still assigned a measure by $M$, though, being a prefix of stuff with measure. In this case, $h$ also lies in $F(inf({\pi }_{st},{\pi }_{st}^{\prime }\left)\right)$. The same analysis from our first case works, $h$ doesn't have its measure disrupted.

In the third case, our history $h$ in $F\left({\pi }_{st}^{\prime }\right)$ isn't in $F\left({\pi }_{st}\right)$, but a strict prefix $h^{\prime }$ lies in $F\left({\pi }_{st}\right)$. We can distinguish three subcases. In the first subcase, $h$ is of the form $h^{\prime }aN$. In the second subcase, $h$ still ends with Nirvana, but it isn't immediately after $h^{\prime }$ happens, some stuff happens in the meantime first. In the third subcase, $h$ doesn't end with Nirvana. Also, $h^{\prime }$ lies in $F(inf({\pi }_{st},{\pi }_{st}^{\prime }\left)\right)$.

For the first subcase where $h$ is of the form $h^{\prime }aN$, injecting up means $h^{\prime }aN$ now has the measure originally associated with $h^{\prime }$ and nirvana is marked as "possible" there (if we're using the sur-injection). Projecting down leaves this alone. Projecting down leaves the measure on $h^{\prime }$ alone, and injecting up means $h^{\prime }aN$ now has the measure originally associated with $h^{\prime }$ and nirvana is marked as "possible" there (if we're using the sur-injection). In both paths, $h^{\prime }aN$ ends up with the measure that $h^{\prime }$ started with, and nirvana marked as "possible" in the sur-case.

For the second subcase where $h$ is of the form "$h^{\prime }$, then some stuff happens, then Nirvana occurs", then in the causal case, the injection up would assign $h$ 0 measure (all the measure of $h^{\prime }$ got channeled into $h^{\prime }aN$ instead of $h$), and then projecting down, it stays the same. Similarly, projecting down means $h^{\prime }$ has some measure, then it's all channeled into $h^{\prime }aN$ on the injection up, so $h$ itself gets 0 measure. For the surcausal case, the injection up assigns $h$ $0^{+}$ measure (by the same argument and sur-injections tagging every freshly-added nirvana outcome with $0^{+}$ measure). and projecting down, it remains with $0^{+}$ measure. Projecting down leaves $h^{\prime }$ alone and then injecting up tags $h$ with $0^{+}$ measure.

For the third subcase where $h$ is an extension of $h^{\prime }$ that doesn't add any Nirvana, we can run through the same argument as the second subcase to conclude that we get 0 measure for both the causal case and the surcausal one.

Thus, no matter whether we inject up and project down, or project down and inject up, the measure assigned to $h\in F\left({\pi }_{st}^{\prime }\right)$ by the measure component of the p-(sur)measure will agree.

An important thing to note with this is that we can use any stub above ${\pi }_{st}$ and ${\pi }_{st}^{\prime }$ for the injection up, but we must use $inf({\pi }_{st},{\pi }_{st}^{\prime })$ for the projection down.

 

Now, we can finally embark on the proof of the two translation theorems! There's enough similarities between the proofs that we can just do one big proof and remark on any differences we come across. The things we must show are that slicing off the Nirvana from a causal/surcausal hypothesis makes a pseudocausal/acausal hypothesis, and that adding in those injections up can turn a pseudocausal/acausal hypothesis to a causal/surcausal one, and that going nirvana-free to nirvana-containing back to nirvana-free is identity.

Proof sketch: While at first this may look like the proof will be almost as long as the Isomorphism theorem because we're verifying a list of 9 conditions twice over, it'll actually be considerably shorter. The only nontrivial part of the first part where we check that slicing off the Nirvana makes a pseudocausal/acausal hypothesis is deriving pseudocausality from causality, and even that is fairly easy.

Going from pseudocausal/acausal to causal/surcausal is trickier, though thankfully most conditions are trivially true, there's only three notable ones. There's the bound on minimal points, which is done by taking a sequence $M_n$ limiting to a $M$ that violates the bound, using the definition of the causal translation to get a point below each $M_n$ which obeys the ${\lambda }^{\odot }+b^{\odot }$ bound (fairly nontrivial), and appealing to Lemma 16 to construct a limit point below $M$ that obeys the ${\lambda }^{\odot }+b^{\odot }$ bound. Showing weak consistency (projecting down makes a subset) requires the Diamond Lemma to write the projection of your point of interest as a mix of injections up from below, and the last tricky one is causality. Which requires first showing that injecting up ${\to }^c\left({\Theta }^{st}\right({\pi }_{st}\left)\right)$ to a higher stub won't add any new points, and then coming up with a clever way of building our outcome function, and using the Diamond Lemma to show that it indeed an outcome function.

Finally, nirvana-free to causal to nirvana-free is instant by Lemma 22.

Proof: Referring back to the conditions for a hypothesis on policy stubs, we'll show that they're fulfilled when you slice away the Nirvana, and that pseudocausality can be derived from causality if we're just dealing with a-measures and not a-surmeasures.

Stub Nirvana-free Nonemptiness was a property already possessed by the causal hypothesis, so it's preserved when we clip away the Nirvana. Stub closure and convexity also hold because we're intersecting with the closed convex set (of nirvana-free a-measures). Nirvana-free upper completion also holds. Bounded minimals holds because a minimal point in the Nirvana-free part must also be a minimal point in the original set, because adding anything to a nirvana-containing a-measure makes a nirvana-containing a-measure, so there can be no nirvana below our minimal point in the nirvana-free part, so it's minimal in general. Normalization holds because the expectation values only depend on the nirvana-free part. Nirvana-free stuff projects down to nirvana-free stuff, getting stub-consistency. The stub extreme point condition carries over due to the preexisting intersection with nirvana-free used to define it, and the same applies to uniform continuity. This wraps up surcausal-to-acausal, and just leaves deriving pseudocausality from causality for causal-to-pseudocausal.

Let's say we have a nirvana-free $M\in {\Theta }^{st}\left({\pi }_{st}\right)$, where the measure part of $M$ is supported over $F\left({\pi }_{st}^{\prime }\right)$, and we want to show that it's also present in ${\Theta }^{st}\left({\pi }_{st}^{\prime }\right)$. Then $M$ is present in ${\Theta }^{st}(inf({\pi }_{st},{\pi }_{st}^{\prime }\left)\right)$, because it's supported entirely over histories that both the different policies produce, so it's supported over histories that the intersection of the policies produces, just project it down to the inf. Now, by causality, we can find something in ${\Theta }^{st}\left({\pi }_{st}^{\prime }\right)$ that projects down onto $M$, which must be $M$ itself because the measure part of $M$ is supported entirely on histories in $F\left({\pi }_{st}^{\prime }\right)$. This gets pseudocausality.

Now, let's show that ${\to }^c\left({\Theta }^{st}\right)$ and ${\to }^{sc}\left({\Theta }^{st}\right)$ fulfill the defining conditions for finitary causal.

1: Stub Nirvana-Free Nonemptiness: This one is trivial, because ${\Theta }^{st}\left({\pi }_{st}\right)$ is present as a subset via the identity injection $I_{\ast }^{{\pi }_{st},{\pi }_{st}}$, and is nirvana-free.

2,3: Stub Closure/Convexity: We took a closed convex hull, these are tautological.

4: Stub Nirvana-Free Upper-Completeness: Just apply Lemma 22 to get that the nirvana-free part of ${\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}\right)$ (and same with surcausal) is just the original ${\Theta }^{st}\left({\pi }_{st}\right)$, which is nirvana-free and upper-complete by stub nirvana-free upper-completeness, so we're good there.

Condition 5: Stub Bounded Minimals: 

By stub bounded minimals on the ${\Theta }^{st}$ we have a ${\lambda }^{\odot }+b^{\odot }$ bound on $\lambda +b$ for minimal points in ${\Theta }^{st}\left({\pi }_{st}\right)$ for all stubs. Pick a $M\in {\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}\right)$ (or the surcausal analogue) with $\lambda +b>{\lambda }^{\odot }+b^{\odot }$. There's a sequence of points $M_n$ limiting to $M$ that lie in the convex hull. All these $M_n$ (or $S{M}_n$) can be written as ${\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\left(M_{i,n}\right)$ where $M_{i,n}\in {\Theta }^{st}\left({\pi }_{st}^i\right)$. Decompose $M_{i,n}$ into a minimal point and something else, getting $M_{i,n}^{min}+M_{i,n}^{\ast }$.

Then, do a further rewrite as $M_{i,n}^{min}+(m_{i,n}^{\ast -},-m_{i,n}^{\ast -}(1\left)\right)+(m_{i,n}^{\ast +},b_{i,n}^{\ast }+m_{i,n}^{\ast -}(1\left)\right)$

Note that the $\lambda +b$ value of the sum of the first two terms is bounded above by ${\lambda }^{\odot }+b^{\odot }$, because $M_{i,n}^{min}$ obeys that bound, and for the second term, it deletes exactly as much from the $\lambda$ term as it adds to the $b$ term. Also, since $M_{i,n}$ is an a-measure, adding in just the negative component to $M_{i,n}^{min}$ doesn't make it go negative anywhere, so the sum of the first two terms is an a-measure, and by nirvana-free upper completeness, it lies in ${\Theta }^{st}\left({\pi }_{st}^i\right)$. The third component of the sum is also an a-measure.

By linearity of $I_{\ast }$ or $I_{\ast s}$, injecting up the first two terms and the last term, and adding them afterwards, is the same as injecting up the bulk of them (we can only inject up a-measures). Let's abbreviate $M_{i,n}^{min}+(m_{i,n}^{\ast -},-m_{i,n}^{\ast -}(1\left)\right)$ as $M_{i,n}^{◊}$ and abbreviate $(m_{i,n}^{\ast +},b_{i,n}^{\ast }+m_{i,n}^{\ast -}(1\left)\right)$ as $M_{i,n}^{♡}$ Now, we can rewrite $M_n$ as:

$M_n={\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\left(M_{i,n}\right)={\sum }_i\left({\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\right(M_{i,n}^{◊}\left)\right)+{\sum }_i\left({\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\right(M_{i,n}^{♡}\left)\right)$

The first component is in ${\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}\right)$ and has the ${\lambda }^{\odot }+b^{\odot }$ bound (injection up preserves $\lambda$ and $b$) and $M_{i,n}^{◊}$ lies below the ${\lambda }^{\odot }+b^{\odot }$ bound, and mixing stuff below the bound produces a point below the bound. Abbreviate the first component as $M_n^{◊}$.

So, $M_n$ isn't minimal, it lies above $M_n^{◊}$. Because the $M_n^{◊}$ have a ${\lambda }^{\odot }+b^{\odot }$ bound, and there's only finitely many places where nirvana could be, we can extract a convergent subsequence, limiting to some $M^{◊}$ which obeys the ${\lambda }^{\odot }+b^{\odot }$ bound, and by Lemma 16, $M^{◊}$ lies below $M$.

Therefore, $M$ isn't minimal, and it was an arbitrary point that violated the ${\lambda }^{\odot }+b^{\odot }$ bound, so all minimal points in any ${\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}\right)$ obey the same ${\lambda }^{\odot }+b^{\odot }$ bound, and we get bounded minimals.

6: Stub Normalization. By Lemma 22, we didn't introduce any new nirvana-free points, so stub normalization of ${\Theta }^{st}$ carries over.

Condition 7: Weak Consistency. 

This is "projecting down makes a subset". All the following arguments work with sur-stuff. Fix some $M\in {\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}^{hi}\right)$. It's a limit of points $M_n$ in the convex hull. We can decompose the $M_n$ as ${\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}^{hi}}\left(M_{i,n}\right)$. Then,

$p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(M_n\right)=p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left({\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}^{hi}}\right(M_{i,n}\left)\right)={\sum }_i{\zeta }_{i,n}p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(I_{\ast }^{{\pi }_{st}^i,{\pi }_{st}^{hi}}\right(M_{i,n}\left)\right)$

Which, by the Diamond Lemma, can be rewritten as: ${\sum }_i{\zeta }_{i,n}{I}_{\ast }^{inf({\pi }_{st}^{lo},{\pi }_{st}^i),{\pi }_{st}^{lo}}\left(p{r}_{\ast }^{{\pi }_{st}^i,inf({\pi }_{st}^{lo},{\pi }_{st}^i)}\right(M_{i,n}\left)\right)$

The projections of the $M_{i,n}\in {\Theta }^{st}\left({\pi }_{st}^i\right)$ lie in ${\Theta }^{st}(inf({\pi }_{st}^{lo},{\pi }_{st}^i\left)\right)$ by weak consistency for ${\Theta }^{st}$. So, actually, the projection of $M_n$ down to ${\pi }_{st}^{lo}$ can be written as a mix of injections up from stubs below ${\pi }_{st}^{lo}$, so the projection of $M_n$ lies in ${\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}\right)$. Then, just use continuity of projection, and closure, to get $M$ itself projecting down into ${\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}^{lo}\right)$, so we're good on weak consistency.

8: Stub Extreme Point Condition: By Lemma 22, we didn't introduce any new nirvana-free points, so any nirvana-free extreme minimal point was present (and nirvana-free extreme minimal) in ${\Theta }^{st}$ already, so the extreme point condition carries over from there.

9: Stub Hausdorff Continuity: By Lemma 22, we didn't introduce any new nirvana-free points, and the preimages for Hausdorff continuity are of the nirvana-free parts, so this is completely unaffected and carries over.

Condition C: Causality: As a warmup to this result, we'll show that if ${\pi }_{st}\le {\pi }_{st}^{hi}$, then

$I_{\ast }^{{\pi }_{st},{\pi }_{st}^{hi}}\left({\to }^c\right({\Theta }^{st}\left)\right({\pi }_{st}\left)\right)\subseteq {\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}^{hi}\right)$

Pick a $M\in {\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}\right)$. It's a limit of $M_n$ which are finite mixtures of injections of stuff from below, and $M_n$ can be written as ${\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\left(M_{i,n}\right)$ Then,

$I_{\ast }^{{\pi }_{st},{\pi }_{st}^{hi}}\left(M_n\right)=I_{\ast }^{{\pi }_{st},{\pi }_{st}^{hi}}\left({\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\right(M_{i,n}\left)\right)={\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st},{\pi }_{st}^{hi}}\left(I_{\ast }^{{\pi }_{st}^i,{\pi }_{st}}\right(M_{i,n}\left)\right)$

And then, by commutativity of injections, the injection of $M_n$ rewrites as ${\sum }_i{\zeta }_{i,n}{I}_{\ast }^{{\pi }_{st}^i,{\pi }_{st}^{hi}}\left(M_{i,n}\right)$ All the ${\pi }_{st}^i$ are below ${\pi }_{st}$ so they're also under ${\pi }_{st}^{hi}$, witnessing that the injection of $M_n$ lies in ${\to }^c\left({\Theta }^{{\pi }_{st}}\right({\pi }_{st}^{hi}\left)\right)$. Then, just appeal to closure and continuity of $I_{\ast }$ or $I_{\ast s}$ to get that $M$ injects up into ${\to }^c\left({\Theta }^{{\pi }_{st}}\right({\pi }_{st}^{hi}\left)\right)$ Again, all this stuff works for the sur-situation as well.

With this out of the way, fix some $M\in {\to }^c{\Theta }^{{\pi }_{st}}\left({\pi }_{st}\right)$. Let's try to make an outcome function from this, shall we? Let's do $of\left({\pi }_{st}^{\prime }\right):=I_{\ast }^{inf({\pi }_{st},{\pi }_{st}^{\prime }),{\pi }_{st}^{\prime }}\left(p{r}_{\ast }^{{\pi }_{st},inf({\pi }_{st},{\pi }_{st}^{\prime })}\right(M\left)\right)$

Yup, that does indeed specify one point for everything. It obviously spits out $M$ when you feed ${\pi }_{st}$ in because both the injection and projection turn into identity. Further, by weak-consistency, the projection of $M$ down lies in ${\to }^c\left({\Theta }^{{\pi }_{st}}\right)(inf({\pi }_{st},{\pi }_{st}^{\prime }\left)\right)$, and by our freshly-proved result, injecting up lands you in ${\to }^c\left({\Theta }^{st}\right)\left({\pi }_{st}^{\prime }\right)$.

So, all that's left is showing that it's an outcome function! That, for any two ${\pi }_{st}^{hi}$ and ${\pi }_{st}^{lo}$ where ${\pi }_{st}^{hi}\ge {\pi }_{st}^{lo}$, that $p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(of\right({\pi }_{st}^{hi}\left)\right)=of\left({\pi }_{st}^{lo}\right)$ Let's begin.

$p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(of\right({\pi }_{st}^{hi}\left)\right)=p{r}_{\ast }^{{\pi }_{st}^{hi},{\pi }_{st}^{lo}}\left(I_{\ast }^{inf({\pi }_{st},{\pi }_{st}^{hi}),{\pi }_{st}^{hi}}\left(p{r}_{\ast }^{{\pi }_{st},inf({\pi }_{st},{\pi }_{st}^{hi})}\right(M\left)\right)\right)$

And then, by the Diamond Lemma, this equals

$I_{\ast }^{inf({\pi }_{st}^{lo},inf({\pi }_{st},{\pi }_{st}^{hi}\left)\right),{\pi }_{st}^{lo}}\left(p{r}_{\ast }^{inf({\pi }_{st},{\pi }_{st}^{hi}),inf({\pi }_{st}^{lo},inf({\pi }_{st},{\pi }_{st}^{hi}\left)\right)}\left(p{r}_{\ast }^{{\pi }_{st},inf({\pi }_{st},{\pi }_{st}^{hi})}\right(M\left)\right)\right)$

And then, $inf({\pi }_{st}^{lo},inf({\pi }_{st},{\pi }_{st}^{hi}\left)\right)=inf({\pi }_{st}^{lo},{\pi }_{st}^{hi},{\pi }_{st})=inf({\pi }_{st},{\pi }_{st}^{lo})$ because ${\pi }_{st}^{lo}\le {\pi }_{st}^{hi}$. Rewriting a bit, and grouping the two projections together because they commute, we have:

$I_{\ast }^{inf({\pi }_{st},{\pi }_{st}^{lo}),{\pi }_{st}^{lo}}\left(p{r}_{\ast }^{{\pi }_{st},inf({\pi }_{st},{\pi }_{st}^{lo})}\right(M\left)\right)=of\left({\pi }_{st}^{lo}\right)$

And we're finally done with everything, we showed causality.

Lemma 24: Given a $\Theta$ (maybe just defined on stubs or full policies) that fulfills all hypothesis conditions except normalization, if it's normalizable, then all belief-function conditions are preserved (works in the sur-case too)

Nirvana-free nonemptiness, closure, convexity, nirvana-free upper-completion, and bounded-minimals are all obviously preserved by scale-and-shift/Proposition 7 in section 1 of proofs. For consistency, due to projections preserving $\lambda$ and $b$, the scale-and-shift in the stubs (or full policies) is perfectly reflected in whichever partial policy you're evaluating, so consistency holds too. For the extreme point condition, any nirvana-free minimal extreme point post-renormalization is also nirvana-free minimal extreme pre-renormalization, so we can undo the renormalization, get a point in the nirvana-free component of a full policy that projects down accordingly, and scale-shift that point to get something that projects down to the scale-shifted extreme point. For Hausdorff-continuity, the scaling just scales the distance between two sets by the scale term, so Hausdorff-continuity carries over. Pseudocausality is preserved by normalization (un-normalize, transfer over to the partial policy of interest, then normalize back again), and so is causality (unnormalize the point and outcome function, complete it, normalize your batch of points back again).

Proposition 2: Given a nirvana-free ${\Theta }^{?\omega }$, the minimal constraints we must check of ${\Theta }^{?\omega }$ to turn it into an acausal hypothesis are: Nonemptiness, Restricted Minimals, Hausdorff-Continuity, and non-failure of renormalization. Every other constraint to make a hypothesis can be freely introduced.

Ok, we have our ${\Theta }^{?\omega }$, and we want to produce an acausal hypothesis. The obvious way to do it is: ${\Theta }^{\omega }\left(\pi \right)={\left(\overline{c.h}{\left({\Theta }^{?\omega }\right(\pi \left)\right)}^{uc}\right)}^R$ We'll use ${\Theta }^{\omega ,\neg R}\left(\pi \right)$ to refer to the set before renormalization.

Proof sketch: We basically just run through the infinitary hypothesis conditions, and show that they're fulfilled by ${\Theta }^{\omega ,\neg R}$, and then appeal to Lemma 24 that we didn't destroy our hard work when we normalize. As for the hypothesis conditions themselves, they're all pretty simple to show except for bounded-minimals and Hausdorff-continuity, which is where the bulk of the work is.

1: Nirvana-free nonemptiness. Trivial, because all the ${\Theta }^{?\omega }\left(\pi \right)$ are nonempty.

2: Closure: Appeal to Lemma 2, not in this document, but of section 1 in basic inframeasure theory, that the upper completion of a closed set is closed. Then we just intersect with the cone of a-measures (closed) to get our set of interest, so it's closed.

3: Convexity: If you have $M$ and $M^{\prime }$ which decompose into $M^{lo}+M^{\ast }$ and $M^{{}^{\prime }lo}+M^{{}^{\prime }\ast }$ ($M$ and $M^{\prime }$ lie in the upper completion and $M^{lo}$/$M^{{}^{\prime }lo}$ lie in the closed convex hull), then

$pM+(1-p)M^{\prime }=p(M^{lo}+M^{\ast })+(1-p)(M^{{}^{\prime }lo}+M^{{}^{\prime }\ast })$

$=(p{M}^{lo}+(1-p\left)M^{{}^{\prime }lo}\right)+(p{M}^{\ast }+(1-p\left)M^{{}^{\prime }\ast }\right)$

The first component lies in the closed convex hull because it's a mix of two points from the closed convex hull, the second component is an sa-measure, and by our upper completion, then $pM+(1-p)M^{\prime }$ lies in our ${\Theta }^{\omega ,\neg R}\left(\pi \right)$

4: Nirvana-free Upper-completeness: Trivial, we took the upper completion.

Condition 5: Bounded Minimals: 

This can be shown by demonstrating that, if ${\lambda }^{\odot }+b^{\odot }$ is our bound for ${\Theta }^{?\omega }$ (ie, every point in ${\Theta }^{?\omega }\left(\pi \right)$, regardless of $\pi$, either respects the bound or lies strictly above a point in ${\Theta }^{?\omega }\left(\pi \right)$ that respects the bound), then every point in ${\Theta }^{\omega ,\neg R}\left(\pi \right)$ lies above a point that obeys the ${\lambda }^{\odot }+b^{\odot }$ bound.

Take a point $M\in \overline{c.h}\left({\Theta }^{?\omega }\right(\pi \left)\right)$. We don't have to worry about points in the upper completion that weren't part of the original closed convex hull, because they're above something in the closed convex hull, so we just have to show that everything in the closed convex hull lies above something that respects the bounds.

$M$ can be written as a limit of points $M_n\in c.h\left({\Theta }^{?\omega }\right(\pi \left)\right)$, which split into a mixture of finitely many $M_{i,n}\in {\Theta }^{?\omega }\left(\pi \right)$. We can then split the $M_{i,n}$ into $M_{i,n}^{lo}+M_{i,n}^{\ast }$, where $M_{i,n}^{lo}$ respects the appropriate bounds (everything in ${\Theta }^{?\omega }\left(\pi \right)$ either obeys the bounds or is above a point which obeys the bounds). Now, we can rewrite $M_n$ as: ${\sum }_i{\zeta }_{i,n}\left(M_{i,n}^{lo}\right)+{\sum }_i{\zeta }_{i,n}\left(M_{i,n}^{\ast }\right)$

That first sum term is a mixture of stuff that respects the ${\lambda }^{\odot }+b^{\odot }$ bound, so it respects the same property and lies below $M_n$ by the addition of the second term making $M_n$. All these lower points lie in the closed convex hull, and obey the ${\lambda }^{\odot }+b^{\odot }$ bound, so there's a convergent subsequence that limits to some limit point that's also in the closed convex hull, respects the ${\lambda }^{\odot }+b^{\odot }$ bound, and by Lemma 16, is below $M$.

So, any $M\in {\Theta }^{\omega ,\neg R}\left(\pi \right)$, regardless of $\pi$, which violates the ${\lambda }^{\odot }+b^{\odot }$ bound on minimal points, has a point lower than it which does respect the bound, showing Minimal Boundedness.

We normalize at the end, and need uniform Hausdorff Continuity to show nirvana-free consistency, so let's skip to that one, which is hard.

Condition 8: Uniform Hausdorff Continuity: 

We'll be working with the Lemma 15 variant of Hausdorff-continuity, that given any $ϵ$, there's a $\delta$ where two policies $\pi ,{\pi }^{\prime }$ being $\delta$ apart or less guarantees that if $M$ is in ${\Theta }^{?\omega }\left(\pi \right)$, then there's a point $M^{\prime }$ in ${\Theta }^{?\omega }\left({\pi }^{\prime }\right)$ that's only $ϵ(1+\lambda )$ away, where $\lambda$ is the $\lambda$ value associated with $M$, and establish that variant for ${\Theta }^{\omega ,\neg R}$.

Fix an $ϵ$. How close do two policies have to be to guarantee that for any $M\in {\Theta }^{\omega ,\neg R}\left(\pi \right)$, there's a point $M^{\prime }$ in ${\Theta }^{\omega ,\neg R}\left({\pi }^{\prime }\right)$ within $ϵ(1+\lambda )$? Well, for our original Hausdorff-continuity condition, pick a $\delta$ that forces a $\frac{ϵ}{2(1+{\lambda }^{\odot }+b^{\odot })}$ "distance", and $\delta <\frac{ϵ}{2(1+{\lambda }^{\odot }+b^{\odot })}$.

Since we've got closure and bounded-minimals, write $M$ as $M^{lo}+M^{\ast }$ where $M^{lo}$ respects the ${\lambda }^{\odot }+b^{\odot }$ bound, and it lies in the closed convex hull and is a limit of $M_n^{lo}$ points, which decompose into a mixture of finitely many $M_{i,n}^{lo}$ points.

Now, each of these $M_{i,n}^{lo}$ points in ${\Theta }^{?\omega }\left(\pi \right)$, by Hausdorff-continuity of ${\Theta }^{?\omega }$, have a $M_{i,n}^{{}^{\prime }lo}$ point in ${\Theta }^{?\omega }\left({\pi }^{\prime }\right)$, that's only $\frac{ϵ}{2(1+{\lambda }^{\odot }+b^{\odot })}(1+{\lambda }_{i,n})$ away, by ${\pi }^{\prime }$ and $\pi$ being $\delta$ or less apart.

We can mix the $M_{i,n}^{{}^{\prime }lo}$ in the same way as usual to make a $M_n^{{}^{\prime }lo}$ that's only $\frac{ϵ}{2(1+{\lambda }^{\odot }+b^{\odot })}(1+{\lambda }_n)$ or less away from $M_n^{lo}$

Because the $M_n^{lo}$ sequence converges, there's some bound on the ${\lambda }_n$ and $b_n$ values, and the (at most) $\frac{ϵ}{2(1+{\lambda }^{\odot }+b^{\odot })}(1+{max}_n({\lambda }_n\left)\right)$ change to make $M_n^{{}^{\prime }lo}$ still keeps the $\lambda$ and $b$ values of our new sequence bounded, so by the Compactness Lemma, the $M_n^{{}^{\prime }lo}$ sequence has a convergent subsequence, with a limit point $M^{{}^{\prime }lo}$, that lies in ${\Theta }^{\omega ,\neg R}\left({\pi }^{\prime }\right)$ by closure. Also, for all n,

$d(M^{lo},M^{{}^{\prime }lo})\le d(M^{lo},M_n^{lo})+d(M_n^{lo},M_n^{{}^{\prime }lo})+d(M_n^{{}^{\prime }lo},M^{{}^{\prime }lo})$

The two distances on either side limit to 0, and the middle distance limits to $\frac{ϵ}{2(1+{\lambda }^{\odot }+b^{\odot })}(1+{\lambda }^{\odot }+b^{\odot })$ or less, because eventually the $\lambda$ value of $M_n^{lo}$ gets really close to the $\lambda$ value of $M^{lo}$, which is subject to the constraint that it can't be bigger than ${\lambda }^{\odot }+b^{\odot }$ due to $M^{lo}$ being picked to  have its $\lambda +b$ value below ${\lambda }^{\odot }+b^{\odot }$, so $d(M^{lo},M^{{}^{\prime }lo})\le \frac{ϵ}{2}$

Ok, so those two things are pretty close to each other. But what we really want is to find a point in ${\Theta }^{\omega ,\neg R}\left({\pi }^{\prime }\right)$ that's close to $M$, ie, $M^{lo}+M^{\ast }$. We can invoke the proof path from direction 2 of Lemma 15 (we have enough tools to do it, most notably upper completion) to craft a $M^{\prime }\in {\Theta }^{\omega ,\neg R}\left({\pi }^{\prime }\right)$ where $d(M,M^{\prime })\le \frac{ϵ}{2}+\delta (\frac{ϵ}{2}+\lambda )$

Further, $\delta <\frac{ϵ}{2(1+{\lambda }^{\odot }+b^{\odot })}\le \frac{ϵ}{2}$. So, we get $d(M,M^{\prime })\le ϵ(1+\lambda )$ and we're done.

7: Nirvana-free Consistency: We're working in a nirvana-free setting, so we can simplify things. Our formulation that we're going to show is, regardless of stub ${\pi }_{st}$, $c.h\left({\bigcup }_{\pi >{\pi }_{st}}p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }^{\omega }\left(\pi \right)\left)\right)$is closed. Just invoke Lemma 20 and we have it and that's the last one we needed besides renormalization. Now all we have to do is to show that every property is preserved when we do the necessary rescaling. Invoke Lemma 24.

Proposition 3: Given some arbitarary ${\Theta }^{?\omega }$ which can be turned into an acausal hypothesis, turning it into ${\Theta }^{\omega }$ has $E_{{\Theta }^{\omega }\left(\pi \right)}\left(f\right)=\alpha \left(E_{{\Theta }^{?\omega }\left(\pi \right)}\right(f)-\beta )$ for all $\pi$ and $f$.

The steps to make your full ${\Theta }^{\omega }$ are convex hull, closure, upper completion, and renormalization. For convex hull, because $f$ induces a positive functional, which is linear, convex hull doesn't affect the worst-case value (mixing a-measures mixes the score you get w.r.t the function), closure just swaps inf out for min, and upper completion doesn't add any new minimal points so it preserves the same minimal values for everything. Let's use ${\Theta }^{\omega ,\neg R}$ for the upper completion of the closed convex hull (no renormalization) so, unpacking definitions, for all $\pi ,f$:

$E_{{\Theta }^{?\omega }\left(\pi \right)}\left(f\right)={inf}_{(m,b)\in {\Theta }^{?\omega }\left(\pi \right)}\left(m\right(f)+b)={inf}_{(m,b)\in c.h\left({\Theta }^{?\omega }\right(\pi \left)\right)}\left(m\right(f)+b)$

$={inf}_{(m,b)\in \overline{c.h}\left({\Theta }^{?\omega }\right(\pi \left)\right)}\left(m\right(f)+b)={inf}_{(m,b)\in \left(\overline{c.h}\right({\Theta }^{?\omega }\left(\pi \right)){)}^{uc}}\left(m\right(f)+b)=E_{{\Theta }^{\omega ,\neg R}\left(\pi \right)}\left(f\right)$

Continuing onwards, let's use $\beta$ for our shift constant and $\alpha$ for our rescaling constant.

$E_{{\Theta }^{\omega }\left(\pi \right)}\left(f\right)={inf}_{(m,b)\in {\Theta }^{\omega }\left(\pi \right)}\left(m\right(f)+b)={inf}_{(m,b)\in {\Theta }^{\omega ,\neg R}\left(\pi \right)}\left(\alpha m\right(f)+\alpha (b-\beta \left)\right)$

$=\alpha (\left({inf}_{(m,b)\in {\Theta }^{\omega ,\neg R}\left(\pi \right)}\right(m\left(f\right)+b\left)\right)-\beta )=\alpha \left(E_{{\Theta }^{\omega ,\neg R}\left(\pi \right)}\right(f)-\beta )$

So, regardless of your $\pi$ and $f$, $E_{{\Theta }^{\omega }\left(\pi \right)}\left(f\right)=\alpha \left(E_{{\Theta }^{?\omega }\left(\pi \right)}\right(f)-\beta )$

and we're done. In particular, since this scale and shift is completely uniform across everything, it keeps the set of optimal policies unchanged.

Proposition 4: For all hypotheses $\Theta$ and ${\Theta }^{\prime }$,$(\forall \pi ,f:E_{\Theta \left(\pi \right)}(f)=E_{{\Theta }^{\prime }\left(\pi \right)}(f\left)\right)\leftrightarrow \left({\to }^{NF}\right(\Theta )={\to }^{NF}({\Theta }^{\prime }\left)\right)$

We can use that Murphy never picks something with Nirvana in it, and ${\to }^{NF}\left(\Theta \right)\left(\pi \right)=\Theta \left(\pi \right)\cap NF$ to rewrite our desired property as:

$(\forall \pi ,f:E_{{\to }^{NF}\left(\Theta \right)\left(\pi \right)}(f)=E_{{\to }^{NF}\left({\Theta }^{\prime }\right)\left(\pi \right)}(f\left)\right)\leftrightarrow \left({\to }^{NF}\right(\Theta )={\to }^{NF}({\Theta }^{\prime }\left)\right)$

one direction of this is pretty easy, if the belief functions are identical when you slice off the nirvana, then regardless of $\pi$ and $f$, Murphy forces the same value. The other direction of this can be done by Theorem 3 from Section 1. Fixing a $\pi$, the property $\forall f:E_{{\to }^{NF}\left(\Theta \right)\left(\pi \right)}\left(f\right)=E_{{\to }^{NF}\left({\Theta }^{\prime }\right)\left(\pi \right)}\left(f\right)$ implies ${\to }^{NF}\left(\Theta \right)\left(\pi \right)={\to }^{NF}\left({\Theta }^{\prime }\right)\left(\pi \right)$, but this holds for all $\pi$, so we get ${\to }^{NF}\left(\Theta \right)={\to }^{NF}\left({\Theta }^{\prime }\right)$

Proposition 5: For all hypotheses $\Theta$, and all continuous functions $g$ from policies to functions $f\in C\left(\right(A\times O{)}^{\omega },[0,1])$, then ${\text{argmax}}_{\pi }{E}_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$ exists and is closed.

Proof sketch: We'll prove this in four phases, where ${\pi }_n$ is some arbitrary sequence of policies limiting to the policy $\pi$. 

Our first phase will be establishing that ${lim}_{n\to \infty }|E_{\Theta \left({\pi }_n\right)}\left(g\right({\pi }_n\left)\right)-E_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)|=0$

Our second phase will be establishing that ${limsup}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)\le E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$

Our third phase will be establishing that ${liminf}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)\ge E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$

Putting phase 2 and 3 together, ${lim}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)=E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$ and then, in conjunction with phase 1, ${lim}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right({\pi }_n\left)\right)=E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$ This establishes that the function $\pi \mapsto E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$ is continuous. Phase 4 is then deriving our desired result from the continuity of that function.

Begin phase 1. To begin with, because $g$ is a function from a compact metric space to a metric space, by the Heine-Cantor theorem, it's uniformly continuous. So, there's some $\delta$ difference between policies that guarantees an $ϵ$ difference between the functions produced, and our distance metric on functions in this case is ${sup}_x|f\left(x\right)-f^{\prime }\left(x\right)|$, the distance metric associated with uniform convergence.

By Proposition 3 (Section 1), every positive functional (and, by Proposition 1 (Section 1), continuous functions induce positive functionals) is minimized within the set of minimal points, so we can fix an $(m,b)$ and $(m^{\prime },b^{\prime })$ within $\left(\Theta \right({\pi }_n){)}^{min}$ (specifically, the nirvana-free component) which minimize the positive functionals associated with $g\left({\pi }_n\right)$ and $g\left(\pi \right)$, respectively. Being able to get an actual minimizing point follows from minimal-boundedness, so the closure of the set of minimal points (due to having bounds) is compact, and a continuous function from a compact set to $[0,1]$ has an actual minimizing point, so we can pick such a point and then step down to a minimal point if needed

Note that, by the way we picked these, $m\left(g\right({\pi }_n\left)\right)+b=E_{\Theta \left({\pi }_n\right)}\left(g\right({\pi }_n\left)\right)$ and also $m^{\prime }\left(g\right(\pi \left)\right)+b^{\prime }=E_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)$

First, we'll bound the following two terms. $|\left(m\right(g\left({\pi }_n\right))+b)-\left(m\right(g\left(\pi \right))+b)|$ and $|\left(m^{\prime }\right(g\left(\pi \right))+b^{\prime })-\left(m^{\prime }\right(g\left({\pi }_n\right))+b^{\prime })|$ The same arguments work for both, so we'll just show one of them.

$|\left(m\right(g\left({\pi }_n\right))+b)-\left(m\right(g\left(\pi \right))+b)|=|m\left(g\right({\pi }_n)-g(\pi \left)\right)|\le m\left(|g\right({\pi }_n)-g(\pi \left)|\right)\le {\lambda }^{\odot }{ϵ}_n$

The argument for this is that the first $\le$ is because $m$ is a measure (never negative), so an upper-bound on the absolute value of the expectation is given by the expectation of the absolute value of the distance between the two functions. For the second $\le$, if n is large enough to make ${\pi }_n$ and $\pi$ be only ${\delta }_n$ apart, then $g\left({\pi }_n\right)$ and $g\left(\pi \right)$ are only ${ϵ}_n$ apart, so that absolute value is upper bounded by ${ϵ}_n$, getting us an upper bound of $m\left({ϵ}_n\right)$ Then, because the total amount of measure for minimal points is upper bounded by some ${\lambda }^{\odot }$ regardless of which policy we picked (minimal-point boundedness for $\Theta$), we can finally impose an upper bound of ${ϵ}_n{\lambda }^{\odot }$ on the distance. The sort of argument works for the second thing, and gets us the exact same upper bound.

Further, $m\left(g\right({\pi }_n\left)\right)+b\le m^{\prime }\left(g\right({\pi }_n\left)\right)+b^{\prime }$ and $m^{\prime }\left(g\right(\pi \left)\right)+b^{\prime }\le m\left(g\right(\pi \left)\right)+b$. This is because $(m,b)$ is specialized to minimize $g\left({\pi }_n\right)$ and $(m^{\prime },b^{\prime })$ is specialized to minimize $g\left(\pi \right)$. Therefore, in one direction:

$\left(m^{\prime }\right(g\left(\pi \right))+b^{\prime })-\left(m\right(g\left(\pi \right))+b)\le 0$ so $\left(m^{\prime }\right(g\left(\pi \right))+b^{\prime })-\left(m\right(g\left({\pi }_n\right))+b)\le {ϵ}_n{\lambda }^{\odot }$

In the other direction,

$\left(m\right(g\left({\pi }_n\right))+b)-\left(m^{\prime }\right(g\left({\pi }_n\right))+b^{\prime })\le 0$ so $\left(m\right(g\left({\pi }_n\right))+b)-\left(m^{\prime }\right(g\left(\pi \right))+b^{\prime })\le {ϵ}_n{\lambda }^{\odot }$

Thus, putting the two parts together,

$|E_{\Theta \left({\pi }_n\right)}\left(g\right({\pi }_n\left)\right)-E_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)|=|\left(m\right(g\left({\pi }_n\right))+b)-\left(m^{\prime }\right(g\left(\pi \right))+b^{\prime })|\le {ϵ}_n{\lambda }^{\odot }$

We can make n go to infinity, which makes ${\delta }_n$ (distance between policies) go to 0, which makes ${ϵ}_n$ (distance between functions) go to 0, and ${\lambda }^{\odot }$ is a constant, so we get that the distance between the two expectations limits to 0 and we're done with the first phase.

Time for phase 2, showing that ${limsup}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)\le E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$

Fix some $(m,b)$ in $\Theta \left(\pi \right)\cap NF$ that minimizes the positive functional associated with $g\left(\pi \right)$. By Hausdorff-Continuity, we can find a sequence of points $(m_n,b_n)\in \Theta \left({\pi }_n\right)\cap NF$ that limit to $(m,b)$. By continuity of $g\left(\pi \right)$, this means that $m_n\left(g\right(\pi \left)\right)+b_n$ limits to $m\left(g\right(\pi \left)\right)+b$, which is $E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$. However, $m_n\left(g\right(\pi \left)\right)+b_n\ge E_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)$

Thus, ${limsup}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)\le E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$

Now for phase 3, showing that ${liminf}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)\ge E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$

Assume it's false, we'll get a proof-by contradiction. That is,

${liminf}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)<E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$

Then, we can get some subsequence where the expectations converge to the liminf. For each n in that subsequence, fix a $(m_n,b_n)\in \left(\Theta \right({\pi }_n){)}^{min}\cap NF$ that minimizes the positive functional associated with $g\left(\pi \right)$ within $\Theta \left({\pi }_n\right)$. Ie, $m_n\left(g\right(\pi \left)\right)+b_n=E_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)$

By bounded minimals, there's some ${\lambda }^{\odot }+b^{\odot }$ bound on all of these, so we can isolate another convergent subsequence (the expectation values still limit to the liminf), where the $(m_n,b_n)$ limit to some $(m,b)$. For the following arguments, we'll use n to denote numbers from our original sequence (ranges over all natural numbers) and j to denote numbers from our convergent subsequence of interest (where the expectations converge to liminf and our sequence of minimizing points converges to a limit point)

First, this $(m,b)$ limit point lies in $\Theta \left(\pi \right)$, because it's arbitrarily close to points that are arbitrarily close to $\Theta \left(\pi \right)$(Hausdorff-continuity), so the distance to that set shrinks to 0, and $\Theta \left(\pi \right)$ is closed so said point limits to be in it. Now, we can go

${liminf}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)={lim}_{j\to \infty }\left(m_j\right(g\left(\pi \right))+b_j)=m\left(g\right(\pi \left)\right)+b$

By $m_j\left(g\right(\pi \left)\right)+b_j=E_{\Theta \left({\pi }_j\right)}\left(g\right(\pi \left)\right)$, and the j's making a subsequence where we attain the liminf value in the limit, and then the second equality is a convergent sequence of a-measures having their expectation value limit to the expectation value of the limit point.

But then we get something impossible. $(m,b)\in \Theta \left(\pi \right)$, and yet somehow (by our original assumption that the liminf undershot the expectation value of $g\left(\pi \right)$ in $\Theta \left(\pi \right)$),

$m\left(g\right(\pi \left)\right)+b={liminf}_{n\to \infty }{E}_{\Theta \left({\pi }_n\right)}\left(g\right(\pi \left)\right)<E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)={min}_{(m^{\prime },b^{\prime })\in \Theta \left(\pi \right)\cap NF}\left(m^{\prime }\right(g\left(\pi \right))+b^{\prime })$

Which cannot be. This shows that the liminf is $\ge E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$.

Now for phase 4. Again, from the proof sketch, phases 1, 2, and 3 establish that $\pi \mapsto E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$ is a continuous function $\Pi \to R^{\ge 0}$ Let's abbreviate this function as $\chi$. Since we're mapping $\Pi$ (which is compact) through a continuous function, the image $\chi \left(\Pi \right)$ is compact. Thus, it has a maximum value, which is attained by some policy. Take that maximum value (it's a single point so it's closed), take the preimage (which is a nonempty closed set of policies), and that's your ${\text{argmax}}_{\pi }{E}_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$ set. And $E_{\Theta \left(\pi \right)}\left(g\right(\pi \left)\right)$ unpacks as ${min}_{(m,b)\in \Theta \left(\pi \right)\cap NF}\left(m\right(g\left(\pi \right))+b)$Thus showing our result.

A quick corollary of it is, if $g$ just returns the constant 1 function, you can find a policy $\pi$ where ${min}_{(\lambda \mu ,b)\in \Theta \left(\pi \right)}(\lambda +b)$ is 1, by normalization, so we can use $max$ in normalization instead of $sup$.

Lemma 25: In the nirvana-free setting, with all the $B_i\subseteq F^{NF}\left({\pi }_{pa}\right)$ being nonempty and upper complete, then $E_{\zeta }{B}_i$ is upper-complete.

The proof of this is nearly identical to the proof of Lemma 12. Except in this case, our $M_i$ aren't finitely many points selected from a nonconvex $B$, they're countably many points selected from the various $B_i$. Apart from that difference, the proof path works as it usually does.

Lemma 26: A belief function $\Theta$ fulfilling all conditions except normalization, which is renormalized by subtracting ${inf}_{\pi }{E}_{\Theta \left(\pi \right)}\left(0\right)$ and scaling by $\left({sup}_{\pi }{E}_{\Theta \left(\pi \right)}\right(1)-{inf}_{\pi }{E}_{\Theta \left(\pi \right)}(0){)}^{-1}$, only has the renormalization fail if, for all $\pi$, $\Theta \left(\pi \right)\cap NF$ has a single minimal point of the form $(0,b)$, with the same $b$ for all $\pi$. (works in the sur-case too)

First, fixing an arbitrary ${\pi }^{\prime }$, then, if there's a divide-by-zero when scaling,

$E_{\Theta \left({\pi }^{\prime }\right)}\left(1\right)\le {max}_{\pi }{E}_{\Theta \left(\pi \right)}\left(1\right)={min}_{\pi }{E}_{\Theta \left(\pi \right)}\left(0\right)\le E_{\Theta \left({\pi }^{\prime }\right)}\left(0\right)$

However, $E_{\Theta \left({\pi }^{\prime }\right)}\left(1\right)\ge E_{\Theta \left({\pi }^{\prime }\right)}\left(0\right)$ always, so the two terms are equal.

Now, just invoke Proposition 6 from Section 1, which says that if $E_{\Theta \left({\pi }^{\prime }\right)\cap NF}\left(1\right)=E_{\Theta \left({\pi }^{\prime }\right)\cap NF}\left(0\right)$, then there's a single minimal point of the form $(0,b)$, and the $b$ is $E_{\Theta \left({\pi }^{\prime }\right)\cap NF}\left(0\right)$, which is the same for all policies ${\pi }^{\prime }$. The converse is, if all $\Theta \left(\pi \right)$ are of this form, then renormalization fails.

 

Let's define "nontriviality" for a belief function $\Theta$. A $\Theta$ is nontrivial if there exists some $\pi$ where $E_{\Theta \left(\pi \right)}\left(1\right)\ne E_{\Theta \left(\pi \right)}\left(0\right)$

In other words, there's some policy you can feed in where the minimal points of $\Theta \left(\pi \right)$ aren't just a single $(0,b)$ point. This is a very weak condition. Also, for the upcoming proposition 6, mixing just doesn't interact well with nirvana-free consistency, so we have to do it just for pseudocausal and acausal hypotheses.

Proposition 6: For pseudocausal and acausal hypotheses ${\Theta }_i$ where ${\sum }_i{\zeta }_i{\lambda }_i^{\odot }<\infty$ and there exists a nontrivial ${\Theta }_i$, then mixing them and renormalizing produces a pseudocausal or acausal hypothesis.

Mixing is defined on the infinite levels by $\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)=E_{\zeta }\left({\Theta }_i\right(\pi \left)\right)$, and is then extended down to the finite levels by the usual process of projecting down and taking the closed convex hull. Then, we can renormalize if we wish. We'll distinguish these by "raw" or "renormalized" mix. Thus, the only conditions we need to check are the infinite conditions. For everything else, we can just go "the infinite conditions work, so we can extend to a full belief function" by the Isomorphism theorem.

If you want what mixing does at finite levels, it's $\overline{c.h}\left({\bigcup }_{\pi >{\pi }_{st}}\left(E_{\zeta }\left(p{r}_{\ast }^{\pi ,{\pi }_{st}}\right({\Theta }_i\left(\pi \right)\left)\right)\right)\right)$ So it isn't "mix all the finite levels", it's "mix all the projections individually and then take convex hull"

Proof sketch: Neglecting normalization (because Lemma 24 shows we can just renormalize and all nice conditions are preserved), we just need to verify all the relevant infinitary conditions, and then we can extend to lower levels by isomorphism, and get our result. We also need to show that nontriviality implies that the renormalization doesn't fail, but that's easy. As for the conditions, our lemmas let us get most of them with little trouble. Bounded-minimals from just the ${\lambda }^{\odot }$ bound is slightly more difficult and relies on showing that $(0,1)$ is in all the ${\Theta }_i\left(\pi \right)$ sets regardless of i and $\pi$ by normalization to eliminate the $b$ term, and Hausdorff-continuity is also fairly nontrivial (we have to split our mix into three pieces and bound each one individually via a different argument) and relies on the same $(0,1)$ is in all the ${\Theta }_i\left(\pi \right)$ result. For causality, we'll knock it out with Tychonoff in a slightly more complicated way than usual so we just use a countable product and don't have to invoke the full Axiom of Choice.

We'll take a detour and show that $(0,1)\in E_{\zeta }{\Theta }_i\left(\pi \right)$ for all $\pi$, we need this in a few places. First, pick an arbitrary i and $\pi$ and look at ${\Theta }_i\left(\pi \right)$. Find a minimal point $(m,b)$ that minimizes $m\left(1\right)+b$. Now, consider $(m,b)+(-m,m(1\left)\right)$. This is $(0,m(1)+b)$. However...

$m\left(1\right)+b={min}_{(m^{\prime },b^{\prime })\in {\Theta }_i\left(\pi \right)\cap NF}\left(m\right(1)+b)=E_{{\Theta }_i\left(\pi \right)}\left(1\right)\le {max}_{\pi }{E}_{{\Theta }_i\left(\pi \right)}\left(1\right)=1$

(by $(m,b)$ minimizing $m\left(1\right)+b$, and normalization, respectively)

So, by nirvana-free upper-completion for ${\Theta }_i$, we get the point $(0,1)\in {\Theta }_i\left(\pi \right)$ for all i and $\pi$. Then, mixing these gets that $(0,1)$ lies in $\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)$.

1: Infinitary Nirvana-Free Nonemptiness, it's easy, all our $\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)$ contain the a-measure $(0,1)$, which is nirvana free.

2,3: Closure, convexity: Closure follows from the proof of closure in Proposition 11 of section 1, and we mixed convex sets so the mixture is convex.

4: Nirvana-free Upper-Completion: Just invoke Lemma 25.

5: Bounded minimals: Since $(0,1)\in \left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)$ for all i, any a-measure with $b\ge 1$ isn't minimal (add whatever you want to $(0,1)$), so we have a bound on the $b$ values of minimals. What about a bound on the $\lambda$ values?

Well, just take a $M$ in $\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)$ with $b<1$, and split it into a mixture of $M_i$ points from the ${\Theta }_i\left(\pi \right)$. Then, by bounded minimality for the ${\Theta }_i$, we can take each $M_i$ and find a minimal point below it that fulfills the ${\lambda }_i^{\odot }$ bound on the $\lambda$ values. Mixing those minimals produces a point $M^{lo}$ that's below $M$, and has a $\lambda$ value below $E_{\zeta }{\lambda }_i^{\odot }$, which, by assumption, is finite. So, every minimal point in any $\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)$ has a $\lambda +b$ value below $E_{\zeta }{\lambda }_i^{\odot }+1$ and we have bounded-minimals.

Nirvana-free consistency is something we'll have to loop back to after Hausdorff-continuity.

Condition 8: Hausdorff-continuity: 

Pick an arbitrary $M\in \left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)$. It shatters into $M_i\in {\Theta }_i\left(\pi \right)$. We'll be showing the Lemma 15 variant, which is that for all $ϵ$, there's a $\delta$ where if $d({\pi }^{\prime },\pi )<\delta$, then there's a point in $\left(E_{\zeta }{\Theta }_i\right)\left({\pi }^{\prime }\right)$ that's $ϵ(1+\lambda )$ away.

First, we'll shuffle around what our $M_i$ are supposed to be, we need a certain decomposition to make it work. Reindex your probability distribution so the highest-probability thing is assigned to $i=0$. All the $M_i$ can be decomposed as a $M_i^{min}+M_i^{\ast }$. Now, let our new $M_i$ for $i>0$ be defined as: $M_i^{min}+(m_i^{\ast -},-m_i^{\ast -}(1\left)\right)$, 

and our $M_i$ for $i=0$ is defined as: $M_0^{min}+(m_0^{\ast -},-m_0^{\ast -}(1\left)\right)+{\sum }_i\frac{{\zeta }_i}{{\zeta }_0}(m_i^{\ast +},b_i^{\ast }+M_i^{\ast -}(1\left)\right)$

These new $M_i$ still lie in ${\Theta }_i\left(\pi \right)\cap NF$ (nirvana-free upper-completion), and they're all a-measures (the negative part isn't enough to cancel out the positive measure on $m_i^{min}$ otherwise our old $M_i$ wouldn't be an a-measure). Further, mixing them together still makes $M$, and if $i>0$, then ${\lambda }_i\le {\lambda }_i^{\odot }$ (because we start off at a minimal point with ${\lambda }_i\le {\lambda }_i^{\odot }$, and then add something to it that saps some measure from it).

Fixing some $ϵ$... well, $E_{\zeta }{\lambda }_i^{\odot }<\infty$, so find a j where ${\sum }_{i>j}{\zeta }_i({\lambda }_i^{\odot }+1)<\frac{ϵ}{3}$. For $i\le j$... well, using the Lemma 15 variant of Hausdorff-continuity, note that fixing a $\delta$ gets you a different ${ϵ}_i$ value for Hausdorff-continuity of each ${\Theta }_i$. We only have to worry about the $i\le j$, though, and there's finitely many. So, pick a $\delta$ where the induced ${ϵ}_0$ is below $\frac{ϵ}{3}$, and for all $0<i\le j$, ${ϵ}_i<\frac{ϵ}{E_{\zeta }{\lambda }_i^{\odot }+1}$.

So... we take our $M_i$ in ${\Theta }_i\left(\pi \right)$, and go to nearby $M_i^{\prime }$ in ${\Theta }_i\left({\pi }^{\prime }\right)$. We should break down exactly how this is done. For $M_0$, the $\lambda$ value relative to $M$ is at most $\frac{\lambda }{{\zeta }_0}$ (in the degenerate case where it contributes all the measure to the mixture $M$), so, by Hausdorff-continuity for ${\Theta }_0$, the gap between $M_0$ and $M_0^{\prime }$ is at most $\frac{ϵ}{3}(1+\frac{\lambda }{{\zeta }_0})$ because we picked $\delta$ low enough to get that scale factor on the front. 

For $M_i$ where $0<i\le j$, the gap between $M_i$ and $M_i^{\prime }$ is at most $\frac{ϵ}{3(E_{\zeta }{\lambda }_i^{\odot }+1)}(1+{\lambda }_i^{\odot })$ Because we picked $\delta$ low enough to guarantee that scale factor on the front, and $M_i$ is made by adding a minimal point and an sa-measure where the measure component is entirely negative, so ${\lambda }_i^{\odot }$ is a bound on the $\lambda$ value for $M_i$.

And finally, for $i>j$, we can specially craft a $M_i^{\prime }$ where the gap between $M_i$ and $M_i^{\prime }$ is at most ${\lambda }_i^{\odot }+1$. This is because $M_i$ has measure below ${\lambda }_i^{\odot }$ due to being a minimal point that lost some measure. So, we can expend ${\lambda }_i^{\odot }$ effort to completely reshuffle it however we wish, and then add $(0,1)$ to our reshuffled a-measure to make an a-measure that lies above $(0,1)$, which must be in ${\Theta }_i\left({\pi }^{\prime }\right)$, so our reshuffled a-measure plus $(0,1)$ lies in ${\Theta }_i\left({\pi }^{\prime }\right)$ by nirvana-free upper-completion, and we only had to spend ${\lambda }_i^{\odot }+1$ effort to travel to it (first term is the reshuffling, second term is adding 1 to the $b$ term)

Now, let's analyze the distance between $M$ and the point $E_{\zeta }{M}_i^{\prime }$ which lies in $\left(E_{\zeta }{\Theta }_i\right)\left({\pi }^{\prime }\right)$. $d(M,E_{\zeta }{M}_i^{\prime })$ equals...

$d({\zeta }_0{M}_0+{\sum }_{0<i\le j}({\zeta }_i{M}_i)+{\sum }_{i>j}({\zeta }_i{M}_i),{\zeta }_0{M}_0^{\prime }+{\sum }_{0<i\le j}({\zeta }_i{M}_i^{\prime })+{\sum }_{i>j}({\zeta }_i{M}_i^{\prime }\left)\right)$

$\le d({\zeta }_0{M}_0,{\zeta }_0{M}_0^{\prime })+{\sum }_{0<i\le j}d({\zeta }_i{M}_i,{\zeta }_i{M}_i^{\prime })+{\sum }_{i>j}d({\zeta }_i{M}_i,{\zeta }_i{M}_i^{\prime })$

$={\zeta }_{0}d(M_0,M_0^{\prime })+{\sum }_{0<i\le j}\left({\zeta }_{i}d\right(M_i,M_i^{\prime }\left)\right)+{\sum }_{i>j}\left({\zeta }_{i}d\right(M_i,M_i^{\prime }\left)\right)$

$<{\zeta }_0\frac{ϵ}{3}(1+\frac{\lambda }{{\zeta }_0})+{\sum }_{0<i\le j}\left({\zeta }_i\frac{ϵ}{3(E_{\zeta }{\lambda }_i^{\odot }+1)}\right(1+{\lambda }_i^{\odot }\left)\right)+{\sum }_{i>j}{\zeta }_i({\lambda }_i^{\odot }+1)$

$<\frac{ϵ}{3}({\zeta }_0+\lambda )+\frac{ϵ}{3}\frac{1}{E_{\zeta }{\lambda }_i^{\odot }+1}{\sum }_{0<i\le j}\left({\zeta }_i\right(1+{\lambda }_i^{\odot }\left)\right)+\frac{ϵ}{3}$

$<\frac{ϵ}{3}({\zeta }_0+1+\lambda +\frac{1}{E_{\zeta }{\lambda }_i^{\odot }+1}{\sum }_i{\zeta }_i({\lambda }^{\odot }+1\left)\right)<\frac{ϵ}{3}(1+1+\lambda +\frac{E_{\zeta }({\lambda }_i^{\odot }+1)}{E_{\zeta }{\lambda }_i^{\odot }+1})$

$=\frac{ϵ}{3}(3+\lambda )\le \frac{ϵ}{3}(3+3\lambda )=ϵ(1+\lambda )$

And bam, we've got Hausdorff-continuity!

9: Nirvana-free consistency: Invoke Lemma 20, notice that we're in the nirvana-free setting, we're done.

Pseudocausality. Fix some $M\in \left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)$, whose support is a subset of $F^{NF}\left({\pi }^{\prime }\right)$. $M$ shatters into $M_i\in {\Theta }_i\left(\pi \right)$. All of them have their support being a subset of $F^{NF}\left({\pi }^{\prime }\right)$ (otherwise there'd be measure-mass outside of there), so all the $M_i$ transfer over to ${\Theta }_i\left({\pi }^{\prime }\right)$ by pseudocausality for the ${\Theta }_i$, and then we can mix them back together to get $M\in \left(E_{\zeta }{\Theta }_i\right)\left({\pi }^{\prime }\right)$.

And we're done, mixing works just fine, as long as we can show that the renormalization preserves everything and the renormalization doesn't fail. Renormalization fails iff (By Lemma 26), for all $\pi$, then $E_{\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)}\left(1\right)=E_{\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)}\left(0\right)$

We can then go $E_{\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)}\left(1\right)=E_{E_{\zeta }\left({\Theta }_i\right(\pi \left)\right)}\left(1\right)=E_{\zeta }\left(E_{{\Theta }_i\left(\pi \right)}\right(1\left)\right)$ and similar for 0, (by the definition of the mix of belief functions and Proposition 10 in Section 1) and then we can use that $E_{\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)}\left(1\right)-E_{\left(E_{\zeta }{\Theta }_i\right)\left(\pi \right)}\left(0\right)=0$ to go

$E_{\zeta }\left(E_{{\Theta }_i\left(\pi \right)}\right(1\left)\right)-E_{\zeta }\left(E_{{\Theta }_i\left(\pi \right)}\right(0\left)\right)=0$

${\sum }_i{\zeta }_i\left(E_{{\Theta }_i\left(\pi \right)}\right(1)-E_{{\Theta }_i\left(\pi \right)}(0\left)\right)=0$

So, then, for all $\pi$ and i, $E_{{\Theta }_i\left(\pi \right)}\left(1\right)=E_{{\Theta }_i\left(\pi \right)}\left(0\right)$ However, this is incompatible with the existence of a nontrivial ${\Theta }_i$, because nontriviality just says that there's a $\pi$ where $E_{{\Theta }_i\left(\pi \right)}\left(1\right)\ne E_{{\Theta }_i\left(\pi \right)}\left(0\right)$.

So, nontriviality for some ${\Theta }_i\left(\pi \right)$ means that the renormalization of your mix can be done. It's a very weak condition, just saying "there's some possibility of starting with a hypothesis (${\Theta }_i$) which has a policy $\pi$, where murphy actually has to pay attention to what function you're maximizing.

Now, we just need to show that renormalization preserves all nice conditions. Just invoke isomorphism to complete to a full psuedo/acausal belief function lacking normalization, apply Lemma 24 and renormalize everything, and we're done.

Proposition 7: For pseudocausal and acausal hypotheses, $E_{\left(E_{\zeta }{\Theta }_i\right)\left({\pi }_{pa}\right)}\left(f\right)=E_{\zeta }\left(E_{{\Theta }_i\left({\pi }_{pa}\right)}\right(f\left)\right)$

Proof: $E_{\left(E_{\zeta }{\Theta }_i\right)\left({\pi }_{pa}\right)}\left(f\right)=E_{E_{\zeta }\left({\Theta }_i\right({\pi }_{pa}\left)\right)}\left(f\right)=E_{\zeta }\left(E_{{\Theta }_i\left({\pi }_{pa}\right)}\right(f\left)\right)$ by Proposition 10 of Section 1.

Proposition 8: For pseudocausal and acausal hypotheses,

$p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left(\right(E_{\zeta }{\Theta }_i\left)\right({\pi }_{pa}\left)\right)=E_{\zeta }\left(p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\right({\Theta }_i\left({\pi }_{pa}^{\prime }\right)\left)\right)$

This is an easy one. If $M\in p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left(\right(E_{\zeta }{\Theta }_i\left)\right({\pi }_{pa}^{hi}\left)\right)$, then there's a preimage point $M^{\prime }\in \left(E_{\zeta }{\Theta }_i\right)\left({\pi }_{pa}^{hi}\right)$, which then decomposes into $M_i^{\prime }\in {\Theta }_i\left({\pi }_{pa}^{hi}\right)$. These $M_i^{\prime }$ project down to $M_i$, which then mix to make $M$ (project then mix equals mix then project because of linearity) witnessing that $M\in E_{\zeta }\left(p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\right({\Theta }_i\left({\pi }_{pa}^{hi}\right)\left)\right)$

Now for the reverse direction. If $M\in E_{\zeta }\left(p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\right({\Theta }_i\left({\pi }_{pa}^{hi}\right)\left)\right)$, then it shatters into $M_i\in p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left({\Theta }_i\right({\pi }_{pa}^{hi}\left)\right)$, which have preimage points $M_i^{\prime }\in {\Theta }_i\left({\pi }_{pa}^{hi}\right)$. The $M_i^{\prime }$ mix to make a $M^{\prime }\in \left(E_{\zeta }{\Theta }_i\right)\left({\pi }_{pa}^{hi}\right)$, which projects down to $M$, by project-mix equaling mix-project, witnessing that $M\in p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left(\right(E_{\zeta }{\Theta }_i\left)\right({\pi }_{pa}^{hi}\left)\right)$.

Next proof post! [AF · GW]

0 comments

Comments sorted by top scores.