Comment by IAFF-User-95 (Imported-IAFF-User-95) on An Idea For Corrigible, Recursively Improving Math Oracles · 2015-07-27T21:26:05.000Z · LW · GW

In practice, the development of such oracles will almost certainly go through weak AI systems already useful for proof. I believe these weak systems will be enough to switch over to provably secure kernels, compilers, sandboxing, etc., after which it is much harder for an oracle to break out (it would require hardware bugs).