post by [deleted] · · ? · GW · 0 comments

This is a link post for


Comments sorted by top scores.

comment by Anon User (anon-user) · 2023-06-02T00:54:31.153Z · LW(p) · GW(p)

When an AGI takes on values for the first time, it must draw from the set of values which already exist or construct something similar from what already exists

The values come into the picture well before it's an AGI. First, a random neural network is initialized, and its "values" is a completely arbitrary function chosen as random. Over time, NN is trained towards an AGI and it's "values" take shape. By the time AGI emerges, it does not "take on values for the first time", the values emerge from an extremely long sequence of tiny mutations, each creating something very similar to what already existed, becoming more complex and coherent over time.