Look at the "Selected stories" section of the page you linked. This is the kind of thing that person writes.

My experience with journalists (not this specific one) is negative. They usually come to you after the story is already written in their mind. What they are looking for are the words they could quote to support their story. So whatever you tell them, it probably won't change the article in general, but if they have already decided to say something, and you happen to say something that sounds similar, than that specific sentence (and nothing else) will be added to the story, along with your name, to make it seem that the story is the result of talking to multiple people.

Anything you say that would disagree with the article will simply be ignored, even if that means ignoring 99% of what you said. It doesn't matter. If they interview 10 people, they will get 10 sentences they can quote; that is quite enough for one article to make it seem like the story has a lot of outside support.

Writing negative stuff about Zizians sounds like... not bad, per se; they are indeed horrible people. But you don't know what else will be in the article, who else will be associated with them (and your sentence, taken out of original context, might support that association). Perhaps the conclusion will be that Zizians are representative of the rationalist community in general. Will you get the opportunity to see the new context for your words before they are published?

I think sending him a link to https://zizians.info/ should be safe, because most likely he can google it anyway. Answering a list of questions, using mostly one-sentence answers (to avoid the possibility of a tangential sentence being taken out of the whole paragraph), maaaaaybe okay. Anything else, I think there is 80% chance you will be unhappy about the outcome.

I generally think it's good to answer journalists on substantive questions.

Do you model journalists as truth-seeking people? I don't; based on my previous experience with some of them. (I could still make an exception for a specific person, if I considered their previous articles fair and well reasoned.)

Maybe controversially, I think a lot of poetry in English relies on these subtle effects because the language isn't very good for poetry per se - it can't do perfect rhymes at the end of every line, or perfect matching of syllable count and accented syllables between lines. In Russian poetry (for example) these things are table stakes, and there's enough flexibility for word choice on top of that. So reading good poetry in Russian, and I guess also in Farsi and some other languages, is a viscerally fun experience. The rhymes hit you. The intellectual angle isn't required. For example, when reading Tolkien's poems in original and in Russian translation side by side, the Russian versions hit much harder.

That said, I think English poetry has a saving grace: rap lyrics. Rap seems to have the right answer how to make poetry work in English: use lots of internal rhymes everywhere. This plays to the language's strengths, because English has a very large vocabulary. So listening to rap gives the experience of "being hit by rhymes", which was the intended experience of poetry to begin with.

But it's very unclear whether they institutionally care.

There are certain kinds of things that it's essentially impossible for any institution to effectively care about.

I no longer see LW as an alternative to academic publishing or Arxiv in the way that I had hoped. My plan was posts that would have the substance of a solid academic paper

Instead, you wrote e.g. a short vague post on politics [LW · GW]. If you don't want to suffer the consequences of negative karma, don't do that. (I think this should have been obvious, or am I wrong here?)

You post about politics, get downvoted, and then complain that the website it unfit to publish solid academic papers? In my opinion, a website where vague posts on politics are welcome would be the one actually unfit to publish solid academic papers.

It's unclear to what extent LW's reader voters know that their votes are silencing or unsilencing other users.

Speaking for myself, I am aware that downvoting can silence the users who came here to make vague political posts, and in my opinion this is system working exactly as intended.

One thing that's unclear is whether removing negative karma comments/posts affects auto rate limits. If I were 8 years younger I would probably be tempted to try this experiment.

Why would you have to be 8 years younger to delete a worthless post?

Those are good points. The hugs one specifically I haven't heard myself from any AIs, but you could argue that AI are 'bred' selectively to be socially adept. That might seem like it would 'poison the well' because of course if they're trained to be socially successful (RLHF probably favoring feelings of warmth and connection, which is why chatgpt, claude, and gemini generally trend toward being more friendly and likeable), then they're going to act that way. Like that would force them to be artificially that way, but the same could be said of humans, even generally, but if we focus specifically on an idiosyncratic thing like music:

Often I've heard a hypothesis for why humans enjoy music when it exerts no direct survival pressure on the human as an organism—is that it creates social unity or a sense of community, but this has the same sort of "artificial" connotation. So someone banged an object in a rhythm a long time ago, and then as more people joined in, it became socially advantageous to bang on objects rhythmically just for the sake of fostering a sense of closeness? which is then actually experienced by the organism as a sense of fun and closeness, even though it has no direct effect on survival?

I realize this makes the questions tougher because going by this model, the very same things that might make them 'pretend' to care might also be things that might cause them to actually care, but I don't think it's an inaccurate picture of our convoluted conundrum.

Ah, well if there are interdependent demand functions then there is no guarantee of a (general) equilibrium, so the starting resource allocation problem may have 0 or many solutions. So I would say no, the isomorphism doesn't hold in that case.

I really like that description! I think the core problem here can be summarized as "Accidently by reinforcing for goal A, then for goal B, you can create A-wanter, that then spoofs your goal-B reinforcement and goes on taking A-aligned actions." It can even happen just randomly, just from ordering of situations/problems you present it with in training, I think.

I think this might require some sort of internalization of reward or a model of the training setup. And maybe self location - like how the world looks with the model embedded in it. It could also involve detecting the distinction between "situation made up solely for training", "deployment that will end up in training" and "unrewarded deployment".

Also, maybe this story could be added to Step 3:

"The model initially had a guess about the objective, which was useful for a long time but eventually got falsified. Instead of discarding it, the model adopted it as a goal and became deceptive."

I think one form of "distortion" is development of non-human and not pre-trained circuitry for sufficiently difficult tasks. I.e., if you make LLM to solve nanotech design it is likely that optimal way of thinking is not similar to how human would think about the task.

One of the scenarios I'm imagining is a scenario where network effects exist, such that you don't want to have something if you are the only person having it, but you do want it if others have it.

Arguably, a lot of public goods/social media are like this, where there's 0 demand at a limited size, but have lots of demand when the size starts increasing beyond a threshold.

In essence, I'm asking if we can remove the demand independence assumption and still get an isomorphism between optimal solutions to scarce resources and a system having prices.

A sequence of almost winner-takes-all races

Talk about the sequentially temporal nature of this race, such that the final winners won each race one after the other.