Asymptotic Logical Uncertainty: The Benford Test

post by Scott Garrabrant · 2015-05-31T00:30:37.000Z · LW · GW · 4 comments

This is the second post in a series on Asymptotic Logical Uncertainty.

All Turing Machines we discuss will have no input, and will output an infinite (or finite) sequence of bits. We let $M\left(n\right)$ denote the $n$th bit output by the Turing machine $M$.

Let $A\left(n\right)=n{\uparrow }^{n}n$ be the $n$th Ackermann number as defined in Conway and Guy's "The Book of Numbers." Let ${\varphi }_1,{\varphi }_2,\dots$ be a simple enumeration of sentences in first order logic over ZFC. Let $L$ be a deterministic Turing machine such that $L\left(n\right)=1$ if and only if ${\varphi }_n$ is provable with proof length at most $A\left(n\right)$. We will be discussing Turing machines which are designed to try to guess the output of $L$ in a very limited time. It will be clear later why we need $L$ to be computable, and therefore must define $L$ using bounded proofs.

Given a probabilistic Turing machine $M$, and a Time complexity function $T$, we define an infinite collection of dependent random variables $M^T\left(n\right)$ for $n\ge 1$. $M^T\left(n\right)$ is defined to be $M\left(n\right)$ if $M$ outputs the first $n$ bits in time at most $T\left(n\right)$. If $M$ does not output $n$ bits in time $T\left(n\right)$, then $M^T\left(n\right)$ is either 0 or 1 uniformly at random. These random variables are dependent, because we only run $M$ once to determine $M^T\left(n\right)$ for all $n$.

Let $M$ be a probabilistic Turing machine designed to try to guess the output of $L$ in time at most $T\left(n\right)=2^n$. Let $S=\{s_1,s_2,\dots \}$ be the subset of natural numbers defining the sentences ${\varphi }_{s_n}=$"The first digit of $A\left(n\right)$ in base 10 is 1." Let $B\left(n\right)$ be defined to be 1 if $n$ is a power of 10, and $\frac{1}{\log 10}$ otherwise. We say that $M$ satisfies the Benford test if

${lim}_{n\to \infty }|P\left(M^T\right(s_n)=1)-B\left(n\right)|=0.$

The Benford test is a desired property, because we believe that ${\varphi }_{s_n}$ is always true when $n$ is a power of 10, and otherwise is pseudorandomly true with probability $\frac{1}{\log 10}$. Formally, we are making the assumption that for any probabilistic Turing machine $M$, the probability that $M^T\left(s_i\right)=L\left(s_i\right)$ for all $1\le i\le n$ is in $O\left({\prod }_{i=1}^n|L\right(s_i)+B(i)-1|)$.

The number $\frac{1}{\log 10}$ comes from Benford's Law. Intuitively, we expect that when $n$ is not a power of 10, ${\varphi }_{s_n}$ is true $\frac{1}{\log 10}$ of the time, and $A\left(n\right)$ is too large for there to be any procedure which predicts ${\varphi }_{s_n}$ better than randomly.

The Benford test is extremely easy to pass if you are allowed to hard code $B\left(i\right)$ into your algorithm. What is more difficult is finding an algorithm which passes the test in a natural way.

The weak Benford test will refer to the version of the test where the algorithm is allowed to know that it will only be judged on its answers to ${\varphi }_{s_n}$. This means that instead of trying to predict $L$, the algorithm tries to predict the output of $L_S$, which is defined to agree with $L$ on the $n$th bit when $n\in S$ and output a 0 for all other bits. The distinction between the Benford test and the weak Benford test is informal. The only difference is changing the bit string that the algorithm is "trying to predict."

Next, we will present an algorithm which uses a strategy similar to Solomonoff induction to satisfy the weak Benford test.

4 comments

Comments sorted by top scores.