Epistemic belief updating: Not noticeably different.

Task stickiness: Massively increased, but I believe this is improvement (at baseline my task stickiness is too low so the change is in the right direction).

My personal ranking of impact would be regularization, then AI control (at least for automated alignment schemes), with interpretability a distant 3rd or 4th at best.

I'm pretty certain that we will do a lot better than evolution, but whether that's good enough is an empirical question for us.

Pet peeve: AI community defaulted to von Neumann as being the ultimate smart human and therefore the basis of all ASI/human intelligence comparison when the mathematician Alexander Grothendieck exists somehow.

Von Neumann arguably had the highest processor-type "horsepower" we know of plus his breadth of intellectual achievements is unparalleled.
But imo Grothendieck is a better comparison point for ASI as his intelligence, while being strangely similar to LLMs in some dimensions, arguably more closely resembles what alien-like intelligence would be: 
- solving "impossible" problem through meta-language and abstractions. 
- able to think deeply on his own (re-discovered measure theory alone when he was a teenager, re-discovered Poincaré results when undergrad, apparently solved multiple PhD theses in parallel in less than a year)
- almost solely built algebraic geometry (which in turn provided the blueprint for category theory) a domain which scares a part of the mathematics community to this day. 
- not your typical child prodigy 
- famously bad at computations: "take a prime number. 57 for instance."

Even from the AI alignment perspective, Grothendieck is fascinating.
Unaligned with "society" incentives and rewards yet having strong moral preferences, in the sense of choosing to work for a public university when he probably could have earned a higher wage elsewhere, holding hardcore communist beliefs, refusing the Fields medal in protest of Soviet Union and on top of that chosing to be stateless.
Disappeared the moment he understood that despite all of that his discoveries were still fueling the industrial-military complex.
 

Sorry if it's not appropriate for this site. But is anybody interested in chess research? I've seen that people here might be interested in chess. For example, here's a chess post [LW · GW] barely related to AI.

Intro

In chess, what positions have the longest forced wins? "Mate in N" positions can be split into 3 types:

1. Positions which use "tricks" to get a big number of moves before checkmate. Such as cycles of repeating moves. For example, this manmade mate in 415 (see the last position) uses obvious cycles. Not to mention mates in omega.
2. Tablebase checkmates, discovered by brute force, showing absolutely incomprehensible play with no discernible logic. See this mate in 549 moves. One should assume it's based on some hidden cycles or something?
3. Positions which are similar to immortal games. Where the winning variation requires a combination without any cycles. For example: Kasparov's Immortal (14 moves long combination), Stoofvlees vs. Igel (down a rook for 21 moves) - the examples lack optimal play tho.

Surprisingly, nobody seems to look for the longest mates of Type 3. Well, I did look for them and discovered some. Down below I'll explain multiple ways to define what exactly I did. Won't go into too much detail. If you want more detail - Research idea: the longest non-trivial middlegames. There you also can see the puzzles I've created.

My longest puzzle is 42 moves: https://lichess.org/study/sTon08Mb/JG4YGbcP Overall, I've created 7 unique puzzles. Worked a lot on 1 more (mate in 52 moves), but couldn't make it work.

Among other things, I made this absurd mate in 34 puzzle. Almost the entire board is filled with pieces (62 pieces on the board!), only two squares are empty. And despite that the position has deep content. It's kinda a miracle. I think it deserves recognition.

Definition 1

Unlike Type 1 and Type 2 mates, my mates involve many sacrifices of material. So my mates can be defined as "the longest sacrificial combinations".

Definition 2

We can come up with important metrics which make a long mate more special, harder to find, more rare. Material disbalance, amount of non-check moves, amount of freedom of pieces, etc. Then we can search for the longest mates compatible with high enough values of those metrics.

Well, that's what I did.

Definition 3

This is an idea of a definition rather than a definition. But it might be important.

• Take a sequential game with perfect information.
• Take positions with the longest forced wins.
• Out of those positions, choose positions where the defending side has the greatest control over the attacking side's optimal strategy.

My mates are an example of positions where the defending side has especially great control over the flow of the game.

Deeper meaning?

Can there be any deep meaning behind researching my type of mates? I think yes. There are two relevant things.

1. First thing is hard to explain, because I'm not a mathematician. But I'll try. Math can often be seen as skipping stuff which is the most interesting to humans. For example, math can prove theorems about games in general, without explaining why a specific game is interesting or why a specific position is interesting. However, here it seems like we can define something very closely related to subjective "interestingness".
2. Hardness of defining valuable things is relevant to Alignment. The definitions above reveal that maybe sometimes valuable things are easier to define than it seems.

Reception

How did chess community receive my work?

• On Reddit, some posts got a moderate amount of upvotes (enough to get into daily top). A silly middlegame position. With checkmate in 50-80 moves? (110+); Does this position set any record? (60+). Sadly the pattern didn't continue: New long non-trivial middlegame mate found. Nobody asked for this. (1).
• On a computer chess forum, people mostly ignored it. I hoped they could help me find the longest attacks in computer games.
• On the Discord of chess composers, a bunch of people complimented my project. But nobody showed any proactive interest (e.g. "hey, I'd like to preserve your work"). One person reacted like ~"I'm not a specialist on that type of thing, I don't know with whom you could talk about that"
• On Reddit communities where you can ask mathematicians things, people told that game theory is too abstract for tackling such things.

Sorry, fixed

I'm unable to open the google docs file in the third link.

• Develop an architecture which has very little opaque reasoning (e.g. not much more than we see in current LLMs) but is sufficiently competitive using legible (e.g. CoT reasoning)

Don't you think CoT seems quite flawed right now? From https://arxiv.org/abs/2305.04388: "Building more transparent and explainable systems will require either improving CoT faithfulness through targeted efforts or abandoning CoT in favor of alternative methods."

Thanks your great paper on alignment faking, by the way.

willingness seems likely to be understating it. a context where the capability is even part of the author context seems like a prereq. finetuning would produce that, with fewshot one has to figure out how to make it correlate. I'll try some more ideas.

if it's a fully general argument, that's a problem I don't know how to solve at the moment. I suspect it's not, but that the space of unblocked ways to test models is small. I'm bouncing ideas about this around out loud with some folks the past day, possibly someone will show up with an idea for how to constrain on what benchmarks are worth making soonish. but the direction I see as maybe promising is, what makes a benchmark reliably suck as a bragging rights challenge?

Stanley Kubrick is perhaps one of the most influential Sci-Fi filmmakers of the 20th century, therefore I believe he has some authority on this matter. What may answer the need for dystopia can be extend to war and crime films:
 

...one of the attractions of a war or crime story is that it provides an almost unique opportunity to contrast an individual of our contemporary society with a solid framework of accepted value, which the audience becomes fully aware of, and which can be used as a counterpoint to a human, individual, emotional situation. Further, war acts as a kind of hothouse for forced, quick breeding of attitudes and feelings. Attitudes crystallize and come out into the open. Conflict is natural, when it would in a less critical situation have to be introduced almost as a contrivance, and would thus appear forced, or - even worse - false. Eisenstein, in his theoretical writings about dramatic structure, was often guilty of oversimplification. The black and white contrasts of Alexander Nevsky do not fit all drama. But war does permit this basic kind of contrast - and spectacle. And within these contrasts you can begin to apply some of the possibilities of film - of the sort explored by Eisenstein."
https://www.archiviokubrick.it/english/words/interviews/1959independence.html 

More specifically he explains the way he believes Speculative Fictional genres, such as fantasy and Sci-Fi can be effective towards expressing certain ideas which realist drama - the kinds you're advocating albeit within a Sci-Fi environment - may not be. Interviews taken from these transcripts: http://visual-memory.co.uk/amk/doc/interview.html 

Michel Ciment: You are a person who uses his rationality, who enjoys understanding things, but in2001: A Space Odyssey and The Shining you demonstrate the limits of intellectual knowledge. Is this an acknowledgement of what William James called the unexplained residues of human experience?

Stanley Kubrick: Obviously, science-fiction and the supernatural bring you very quickly to the limits of knowledge and rational explanation. But from a dramatic point of view, you must ask yourself: 'If all of this were unquestionably true, how would it really happen?' You can't go much further than that. I like the regions of fantasy where reason is used primarily to undermine incredulity. Reason can take you to the border of these areas, but from there on you can be guided only by your imagination. I think we strain at the limits of reason and enjoy the temporary sense of freedom which we gain by such exercises of our imagination.

Michel Ciment: Don't you think that today it is in this sort of popular literature that you find strong archetypes, symbolic images which have vanished somehow from the more highbrow literary works?

Stanley Kubrick: Yes, I do, and I think that it's part of their often phenomenal success. There is no doubt that a good story has always mattered, and the great novelists have generally built their work around strong plots. But I've never been able to decide whether the plot is just a way of keeping people's attention while you do everything else, or whether the plot is really more important than anything else, perhaps communicating with us on an unconscious level which affects us in the way that myths once did. I think, in some ways, the conventions of realistic fiction and drama may impose serious limitations on a story. For one thing, if you play by the rules and respect the preparation and pace required to establish realism, it takes a lot longer to make a point than it does, say, in fantasy. At the same time, it is possible that this very work that contributes to a story's realism may weaken its grip on the unconscious. Realism is probably the best way to dramatize argument and ideas. Fantasy may deal best with themes which lie primarily in the unconscious. I think the unconscious appeal of a ghost story, for instance, lies in its promise of immortality. If you can be frightened by a ghost story, then you must accept the possibility that supernatural beings exist. If they do, then there is more than just oblivion waiting beyond the grave. 

And to finish, I can't the source at the moment (I think it was in "A Life in Pictures"), but it is like Jack Nicholson said of Kubrick "then someone like Stanley comes along and asks: it's realistic, but is it interesting?".

A dystopia provides a background, a framework that allows a highly catalytic environment for dramatizing ideas that cannot be done by means of regular small stakes interpersonal conflict. Even Plato knew this with regards to pedagogy: hence why his Socrates suggested like the way you use big handwriting to make a manuscript more visible, he expanded the vision of justice in one single person to the entire polis.