post by Eliezer Yudkowsky (Eliezer_Yudkowsky)
score: 29 (31 votes) ·
Followup to: Archimedes's Chronophone.
Suppose you could send messages back in time to Archimedes of Syracuse, using a chronophone which - to avoid transmitting anachronistic information - transmits the results of executing cognitive strategies, rather than words. If you say "Women should have the vote", it comes out as "Install a tyrant of great personal virtue", because you repeated what your culture considers a wise form of political arrangement, and what comes out of the chronophone is the result of executing the same cognitive policy in Archimedes's era.
The chronophone won't transmit arguments you rationalize using your home culture's foreknowledge of the desired conclusion - it will substitute the result of executing that cognitive policy using Archimedes's culture's belief as the intended conclusion. A basic principle of the chronophone is that if you say something considered obvious in your home culture, it comes out as something considered obvious in Archimedes's culture.
The challenge was to say something useful under this restriction. This challenge is supposed to be difficult. It's really hard to get somewhere when you don't already know your destination. If there were some simple cognitive policy you could follow to spark moral and technological revolutions, without your home culture having advance knowledge of the destination, you could execute that cognitive policy today - which is what the whole parable is about!
A surprising number of respondents seemed to completely miss the point of the chronophone, just thinking up things they would like to say directly to Archimedes. The classic question of "If you went back in time, how would you start up an industrial civilization?" has been done many times in science fiction (Lord Kalvan of Otherwhen, The Cross-Time Engineer). There are thousands of things we'd like to say to the Past. The difficult part of the question is: How do you get it to come out of the chronophone?
Ger suggested teaching Archimedes decimal notation. Well, if you speak decimal notation - our home culture's standard representation of numbers - into the chronophone, then the chronophone outputs the standard representation of numbers used in Syracuse. To get a culturally nonobvious output, you need a culturally nonobvious input. Place notation is revolutionary because it makes it easier for ordinary people, not just trained accountants, to manipulate large numbers. Maybe an equivalent new idea in our own era would be Python, which makes it easier for novices to program computers - or a mathematician trying to standardize on category theory instead of set theory as a foundation for mathematics. Coming up with that chronophone input suggests that maybe we should pay more attention, in this era, to Python or category theory! A new representation that makes math easier can add up to a lot of benefit over time.
Hertzlinger remarked: "Some of Archimedes's most potentially-important research involved things he regarded as trivial toys. So if we advise him to get interested in Rubik's cube..." Of course you cannot directly describe a Rubik's Cube into the chronophone. So I asked what corresponding input Hertzlinger would say into the chronophone - has Hertzlinger followed the cognitive policy of playing with toy ideas? Maybe if this would have been such a good policy for Archimedes to follow, we should follow it ourselves.
Robin Hanson proposed an (admittedly clever) meta-trick for fine-tuning the chronophone's output. If that worked, Robin wanted to suggest trying to make useful devices that make money, and creating a tradition of this activity. I asked Robin if he'd ever tried to make such useful devices himself - if this is so important to human progress, why isn't Robin doing it? Perhaps Robin could reply that we've already gotten a huge amount of progress out of inventing gadgets, so now this no longer offers the greatest marginal returns. But that, in turn, points up one of the essential difficulties of the challenge. In this era it is culturally obvious - a non-surprising idea - that money-making new technologies benefit humanity. What could you say into the chronophone that would correspond to the nonobviousness of that idea in Archimedes's era? I don't know if it's important enough to qualify, but, for example, Robin's thoughts about prediction markets are not considered obvious in modern culture. That makes them a better bet for chronophone input than if Robin were to describe his efforts to invent a fancy new gadget. Everyone's doing that these days; it would probably come out of the chronophone as a suggestion to become a great warrior.
Richard Hamming used to ask his fellow researchers two questions: "What are the most important problems of your field?" and "Why aren't you working on them?"
What kind of ideas have provided the greatest benefit to humanity? Why aren't you thinking them?
Most of what we desperately want to say to Archimedes is not obvious relative to Archimedes's culture. This strongly suggests that the most important things the Future would want to say to us are, amazingly enough, not things that everyone already knows. If you want to really benefit humanity, you've got to do some original thinking - come up with the sort of nonobvious idea that you would speak into a chronophone. And you have to do some hard thinking about areas of application, directions of effort. You can't just run off in the direction of what your contemporary culture has instilled as the reflex answer to the question "How can I benefit humanity?" In those orchards the low-hanging fruit is gone.
The point of the chronophone dilemma is to make us think about what kind of cognitive policies are good to follow when you don't know your destination in advance. If you can just tell Archimedes to build a capitalist society because your culture already knows this is a good idea, it defeats the purpose of the dilemma. The chronophone transmits cognitive policies, not sentences. What sort of thinking are we doing now that is analogous to the kind of thinking we wish Archimedes had done then?
Comments sorted by oldest first, as this post is from before comment nesting was available (around 2009-02-27).
comment by Dagon
· score: 0 (2 votes) · LW
Do we get feedback on the modifications the chronophone makes to our statements? I think I would find it useful in identifying my own biases to see how it translates statements I think are probably true into things I think probably aren't.
comment by Nick_Hay2
· score: 0 (0 votes) · LW
It seems the point of the exercise is to think of non-obvious cognitive strategies, ways of thinking, for improving things. The chronophone translation is both a tool both for finding these strategies by induction, and a rationality test to see if the strategies are sufficiently unbiased and meta.
But what would I say? The strategy of searching for and correcting biases in thought, failures of rationality, would improve things. But I think I generated that suggestion by thinking of "good ideas to transmit" which isn't meta enough. Perhaps if I discussed various biases I was concerned about, gave a stream of thought analysis of how to improve a particular bias (say, anthropomorphism), this would be invoking the strategy rather than referencing it, thus passing the filter. Hmmm.
comment by Tom3
· score: 5 (7 votes) · LW
I would say:
"To understand recursion, you must first understand recursion."
Find the meta level in that, Chronophone!
comment by DSimon
· score: 14 (14 votes) · LW
The Chronophone would output one of Zeno's Paradoxes (the one about the arrows), which like your quote is also a clever mathematics-related statement that, while potentially enlightening to think about, is clearly not actually true.
comment by Sebastian_Hagen2
· score: 4 (4 votes) · LW
I think the "work on FAI theory" suggestion made in a comment to the previous post was a good one; not because it would yield an FAI design when the answer was passed back through the cronophone, but because the output would get Archimedes working on the most important problem visible to him.
Alternatively, if we think in hindsight that Archimedes simply doesn't have the necessary resources to trigger a major catastrophe, and we want him to focus on doing good instead of not doing bad, that could be modified to "build a seed AI, any seed AI".
Since I'm not currently working on either, I probably shouldn't be the one speaking that advice, though.
comment by michael_vassar2
· score: 1 (1 votes) · LW
Build a seed AI seems like something that would translate into a religious proposal to me. Discover God perhaps? Might it work better if proposed by someone who didn't understand the implications of AGI? Would discussion of MNT translate ad the suggestion of miniaturizing Hiero's engine and the Antikythera mechanism, or scaling them up?
The rules of this game really don't seem clear enough. We need to get feedback from Archimedes on what he's hearing.
Personally, I strongly suspect that when it comes to ethical and political ideas the Ancient Greeks will understand us fairly well without much need for metaphorical translation, so long as we keep the discussion abstract. Women's suffrage = wise dictator? For people who's ancestors were inventing democracies of a sort and writing Lysistrata centuries earlier? Make the machine reversible and if you suggest women's suffrage and he responds about Hiero II, you will hear a sensible to modern ears argument for moving to Singapore.
comment by _Gi
· score: 1 (1 votes) · LW
What would the chronophone transmit, if I suggested that Archimedes joined the Diogenes and the cynics?
comment by Gordon_Worley
· score: 1 (5 votes) · LW
I still haven't come up with something that I feel fits the spirit of the question, but my start is that I could tell Archimedes about atheism. Up until I was maybe 11 or 12 years old I never really considered the question of religion. My parents taught me the basic Christian tradition, but I never attended church or was deeply indoctrinated. At that age, though, other kids started asking me about religion as they began to become adult members of their religions. "What religion are you?" they would ask and my answer was "I don't know". Someone asked if I celebrated Christmas and I said yes so they told me I was a Christian, but I didn't really know what that meant. But over the next several years the more I learned about religion the less it seemed to make sense to me, until eventually someone shunned me for being an atheist, so then that put me on the path to actively learning what atheism was and seriously thinking about the question of whether anything supernatural exists.
In the end I concluded on atheism. If I recounted my steps to Archimedes, I think they would come through clearly, except rather than the Cult of Jesus he might hear Cult of Zeus. To what extent this might actually improve things I don't know, since I'm not even sure if promoting atheism for its own sake is a good idea.
comment by Gordon_Worley
· score: 0 (0 votes) · LW
As I've thought about the chronophone, a big part of the trouble with it is that we can't successfully transmit any idea where we already know what result we want. Thus to pick something desirable now that will be translated into something desirable then is essentially impossible, since if I already know it to be desirable, I must know enough of the result to know it's desirable, hence tainting all my thoughts. At best, I can tell Archimedes about things I'm working on now that are non-obvious and hope that they translate into something similarly non-obvious that would have generated from the same motivation in his own time. That is, if my motion to have fun causes me to research mathematical topic X, his motivation to have fun will cause him to research topic Y in his favorite field. X and Y may not even be analogues, just generated by the same kind of line of thinking.
comment by Stuart_Armstrong
· score: 0 (0 votes) · LW
Hum... We can't present Archimedes with the scientific method, but we could present him with simple experiments we did to confirm and infirm some of our simpler beliefs. That might be enough to give him a hint of the method...
Thanks Eliezer, for this example! Even though it's been making my head spin - there seem to be ways of cheating the chronophone (such as presenting the scientific method through specific experiments), but they don't seem "fair" to your project - deciding what is best for us today, if we don't know the destination.
Since what we'd most want to do is to get Archimedes to doubt some of his assumptions, maybe what we'd need most today is doubt?
But the one thing I'd most like to tell him is "watch out for that roman soldier!"
comment by RedMan
· score: 0 (0 votes) · LW
"Take a bath, please repeat my suggestion back to me."
Encouraging people to think thoughts that inspire them to run screaming through the streets naked is a value I wish to promote.
comment by Basil Marte
· score: 9 (2 votes) · LW
Hallo, this is a chronophone call from 12 years in the future.
I would transmit a technical analysis of various voting systems, and the fundamental process they are meant to solve (simplistically: the process itself doesn't add any information whatsoever, just condenses the noisy information in the ballots into a set of winning candidates—single-winner systems weakly tend to reduce variety). Some do a terrible job (FPTP selects the mode of the distribution) while some are quite involved (given the input containing orderings/scorings of labels, reconstruct a metric space those labels come from, and return the median).
Hopefully, this comes out as "autocrats sometimes rule according to the subjects' morality, but when they don't, there's no system to remove/correct them", and helps implement some form of democracy. Or I could just read The Dictator's Handbook, though I have no idea what that would come out as.
- I think the globally "standard" four-year terms (I was primarily thinking about my own country's cabinets when I thought about this) are too short, as they leave no incentive to invest in infrastructure-like public goods (e.g. education) beyond its role as a crowd-pleaser. Even if there is no lead time for gathering information, planning, and implementation, most investments don't make back their initial cost so soon.
- On the other hand, even in a comparatively healthy system with multiple parties, the distribution of their popularities has approximately 2 bits of entropy (I used my own country's election data). So, ~0.5bits/year public input... (Yes, there are other elections for local government that I didn't count.)
- Cheaper (thus more frequent), higher-bandwidth information input: legally binding polls, focus groups (analogy with juries?), "citizen boards" and other market research tools. I've not put too much thought into how to make them cheater-resistant yet.
- Rethinking the separation of powers into those three bins. For instance, it's not obvious to me why collecting evidence for the criminal side of the judicial system and putting down riots cluster together (with other functions) into what we label police. (Obvious counterexample—not meant as endorsement—being the places where armed policing is done by the military.) Or why are there near-duplicates for rule creation ("obey or else") e.g. laws and decrees (or however executive-generated rules are called). (I've made these notes years before reading Legal Systems Very Different from Ours.)