LessWrong 2.0 Reader

their Frontiermath score is way more than i expected. i also would like to know the python tool, that they used and the amount of reasoning tokens they used for the 32% run. it used thousands of tokens to solve a altered crossing the river puzzle. 

libraries abstract away the low level implementation details; you tell them what you want to get done and they make sure it happens. frameworks are the other way around. they abstract away the high level details; as long as you implement the low level details you're responsible for, you can assume the entire system works as intended.

a similar divide exists in human organizations and with managing up vs down. with managing up, you abstract away the details of your work and promise to solve some specific problem. with managing down, you abstract away the mission and promise that if a specific problem is solved, it will make progress towards the mission. 

(of course, it's always best when everyone has state on everything. this is one reason why small teams are great. but if you have dozens of people, there is no way for everyone to have all the state, and so you have to do a lot of abstracting.)

when either abstraction leaks, it causes organizational problems -- micromanagement, or loss of trust in leadership.

If you've been collecting data on yourself, it's now moderately easy to let an LLM write code that'll tease out causal relationships between those variables, e.g. with a time series causal inference library like tigramite.

I've done that with some of my data, and surprisingly it both fails to back out results from RCTs I've run, and also doesn't find many (to me) interesting causal relationships.

The most surprising one is that creatine influences creativity. Meditating more doesn't influence anything (e.g. has no influence on mood or productivity), as far as could be determined. More mindful meditations cause happiness.

Because of the resampling/interpolation necessary, everything has a large autocorrelation.

Mainly posting this here so that people know it's now easy to do this, and shill tigramite. I hope I can figure out a way of dealing with rare interventions (e.g. MDMA, CBD, ketamine), and incorporate more collected data. Code here, written by Claude 3.5.1 Sonnet.

Agreed, downvoted my comment. (You can't strong downvote your own comment, or I would have done that.)

I was mostly just trying to point to prior arguments against similar arguments while expressing my view.

I think it's a bit sad that this comment is being so well-received -- it's just some opinions without arguments from someone who hasn't read the paper in detail.  

What are the illegal things that would be needed?

Video games are good, but Magic the Gathering is the best general IQ test I've ever seen.

(Yes, it's available in video game form, MTG Arena is one of the better F2P games out there. The game UX handles more of the rules for you, which limits some IQ-testing, but still a good translation.)

How quickly people pick up the game, how quickly they achieve competitive competency, and whether they can reach the highest level of mastery are all good illustrations of IQ.

Unlike Chess, Go, or Poker, which are purely abstract. MTG has a large amount of language skill involved. Rules text and card text are best interpreted as legal texts. It tests Bayesian reasoning, unlike purely deterministic games. There is a wide skillset to test, from deck construction limited play to reading opponents. The game is constantly refreshed with new cards, sets, and rules to learn.

I agree that a superintelligence might make mistakes. In fact I still believe the first AGI may be a good engineer but bad strategist [LW · GW]. I completely agree a smart but unwise superintelligence is dangerous, and may build a greater superintelligence misaligned to even it.

However I think mistakes will almost completely disappear above a certain level of extreme superintelligence.

A truly intelligent being doesn't just fit models to empirical data, but fits simulations to empirical data.

After it fits a simulation to the empirical data, it then fits a model to the simulation. This "model fitted to a simulation fitted to empirical data" will generalize far better than a model directly fitted to empirical data.

It can then use this model to run "cheap simulations" of higher level phenomena, e.g. it models atoms to run cheap simulations of molecules, it models molecules to run cheap simulations of cells, it models cells to run cheap simulations of humans, it models humans to run cheap simulations of the world.

Simulations fit empirical data much better than directly fitting models to empirical data, because a simulation may be made out of millions of "identical" objects. Each "identical" object has the same parameters. This means the independent parameters of the simulation may be a millionfold fewer than a model with the same number of moving parts. This means you need far fewer empirical data to fit the simulation to empirical data, assuming the real world has the same shape as the simulation, and is also made up of many moving parts "with the same parameters."

I copied this from my own post about Scanless Whole Brain Emulation [LW · GW], but that's very off topic :)

EDIT: one thing simulations cannot predict are even smarter superintelligences, since simulating them equals building them. As long as it is wise enough to understand this, it can find solutions which prevent smarter superintelligences from being built, and then do a risk analysis. The universe might last for a trillion years, so building a smarter superintelligence 1000 years sooner has negligible benefit. The more time it spends planning how to build it safely, the lower the risk it gets it wrong.

I’ve just read the article, and found it indeed very thought provoking, and I will be thinking more about it in the days to come.

One thing though I kept thinking: Why doesn’t the article mention AI Safety research much?

In the passage

The only policy that AI Doomers mostly agree on is that AI development should be slowed down somehow, in order to “buy time.”

I was thinking: surely most people would agree on policies like “Do more research into AI alignment” / “Spend more money on AI Notkilleveryoneism research”?

In general the article frames the policy to “buy time” as to wait for more competent governments or humans, while I find it plausible that progress in AI alignment research could outweigh that effect.

—

I suppose the article is primarily concerned with AGI and ASI, and in that matter I see much less research progress than in more prosaic fields.

That being said, I believe that research into questions like “When do Chatbots scheme?”, “Do models have internal goals?”, “How can we understand the computation inside a neural network?” will make us less likely to die in the next decades.

Then, current rationalist / EA policy goals (including but lot limited to pauses and slow downs of capabilities research) could have a positive impact via the “do more (selective) research” path as well.

Not as far as I know, feel free to create one.