Comment by NixPhenom on Friendlier AI through politics · 2009-08-24T05:29:26.212Z · LW · GW

What counts as an AGI? A system that can solve resource-allocation problems that satisfy its inbuilt values in a dynamic environment?

If one were to take Hayek's view of human marketplaces, we could consider our currently existing markets as greater-than-human intelligences by that standard.

What legal principles enable markets to work well? Theory and practice suggest that the necessary and sufficient conditions are: 1) property ownership and enforcement and 2) a sound currency of value-measurement.

So: how well have we imbued our emergently-superintelligent markets with values consistent with human well-being? I would suggest that we have succeed to the extent that the conditions above are met. Further, I'd suggest that FAI development should also hinge on those principles.