lukemarks's Shortform

What's the epistemic backing behind this claim, how much data, what kind? Did you do it, how's it gone? How many others do you know of dropping out and did it go well or poorly?

It might be useful to have some test for "have self-motivation", to reduce the number of people who believe they have it, they quit school, and then it turns out they actually don't.

Or maybe, it's not just whether you feel motivated right now, but how long that feeling stays, on average.

I do think you're correct that it would be a good decision for some. I would also say establishing this as a norm might induce some to take the easy way out and it be a mistake for them.

Might be the case that councelors should be prepared to have a real converstation with HS students that come to that decision but not really make it one schools promote as a path forward. But I do know I was strongly encouraged to complete HS even when I was not really happy with it (and not doing well by many metrics) but recognized as an intelligent kid. I often think I should have just dropped out, got me GED, worked (which I was already doing and then skipping school often) and then later pursued college (which I also did a few years after I graduated HS). I do feel I probably lost some years playing the expected path game.

April is still n=9 months away.

Singular Learning Theory explains/predicts this. If you go to a random point in the loss landscape, you very likely land in a large region implementing the same behaviour, meaning the network has a small effective parameter count. Just because most of the loss landscape is taken up by the biggest, and thus simplest, behavioural regions.

You can see this happening if you watch proxies for the effective parameter count while models train. E.g. a modular addition transformer or MNIST MLP start out with very few effective parameters at initialisation, then gain more as the network trains. If the network goes through a grokking transition, you can watch the effective parameter count go down again.

For example: If this proposition were true and a significant decider of generalization ability, would this make mesaoptimization less likely? More likely?

≈ no change I'd say. We already knew neural network training had a bias towards algorithmic simplicity of some kind, because otherwise it wouldn't work. So we knew general algorithms, like mesa-optimisers, would be preferred over memorised solutions that don't generalise out of distribution. SLT just tells us how that works.

One takeaway might be that observations about how biological brains train are more applicable to AI training than one might have previously thought. Previously, you could've figured that since AIs use variants of gradient descent as their updating algorithm, while the brain uses we-don't-even-know-what, their inductive biases could be completely different.

Now, it's looking like the updating rule you use doesn't actually matter that much for determining the inductive bias. Anything in a wide class of local optimisation methods might give you pretty similar stuff. Some methods are a lot more efficient than others, but the real pixie fairy dust that makes any of this possible is in the architecture, not the updating rule. 

(Obviously, it still matters what loss signal you use. You can't just expect that an AI will converge to learn the same desires a human brain would, unless the AI's training signals are similar to those used by the human brain. And we don't know what most of the brain's training signals are.)

I don't think I understand your hypothetical.  Is your hypothetical about a future AI which has:

• Very accurate measurements of the state of the universe in the future
• A large amount of compute, but not exponentially large
• Very good algorithms for retrodicting* the past

I think it's exponentially hard to retrodict the past.  It's hard in a similar way as encryption is hard.  If an AI isn't power enough to break encryption, it also isn't powerful enough to retrodict the past accurately enough to break secrets.

If you really want to keep something secret from a future AI, I'd look at ways of ensuring the information needed to theoretically reconstruct your secret is carried away from the earth at the speed of light in infrared radiation.  Write the secret in sealed room, atomize the room to plasma, then cool the plasma by exposing it to the night sky.

*predicting is using your knowledge of the present to predict the state of the future.  Retrodicting is using your knowledge of the present to predict retrodict the state of the past