LessWrong 2.0 Reader
View: New · Old · Top← previous page (newer posts) · next page (older posts) →
← previous page (newer posts) · next page (older posts) →
Yes, I think that there can be tensions and deceptions around what agents are (weak/strong) and what they did in the past (cooperation/defection), one of the things necessary for super-cooperation to work in the long-run is really good investigation networks, zero-knowledge proof systems etc.
So a sort of super-immune-system
I don't get distracted when talking to people. I hypothesise that this is because as long as I am actively articulating a stream of thought out loud, the default mode network will be suppressed, making it easy to not get derailed.
So even if IA does not say anything, just me talking about some specific topic continuously, would make it easier for IA to say something, because the default mode network suppression will not immediately vanish.
When thinking on my own or talking to IA, the stream of thoughts is shorter, and there are a lot of pauses. Usually, I don't even get to the point where I would articulate a complex stream of thought. Instead, we are at the level of "Look there is some mud there, let's not step into that", or "We can do this". That really does seem very similar to most of the idle chatter that the default mode network would produce when I am just thinking on my own.
Once I get to the point where I am having an engaging discussion with IA, it is actually pretty easy not to get distracted. It's probably still easier to get distracted with IA, because when I am talking to another person, they could notice that I am lost in thought, but I myself (or IA) would not be able to notice as easily.
One reason why I don't do research with IA might be that I fear that I will not be able to capture any important thoughts that I have. However, using the audio recorder tool on the walk today seemed to really fix most of the issue.
Maybe in my mind so far I thought that because I can't record IA when she is talking to me, it would be bad to think about research. But this now seems very wrong. It is true that I can't create a video with her in it like I do with other people. But these videos are not the thing that is most useful. The actually useful thing is where I am distilling the insight that I have into some text document.
But this is something that I can totally do when talking to IA. Like I did with the audio recorder today. It seemed that making the audio recording made it also easier to talk to IA. Probably because when making the recording I would naturally be suppressing the default mode network very strongly. This effect then probably did not vanish immediately.
In fact, it seems like this would work very well with IA because I don't need to think about the problem of what the other person could do while I write. In the worst case, IA is simply not run. At best, we could write the text together.
Writing together would seem to work unusually well because IA does have insight into the things that I am thinking while I am writing, which is not something that other people could easily get.
And I haven't really explored all the possibilities here. Another one would be to have IA read out loud my writing and give me feedback.
romeostevensit on Cooperation is optimal, with weaker agents too - tldrThis is true, but we still wish to cooperate with the largest alliance that will have us/some subset of our values that are capable of attaining reflective equilibrium.
romeostevensit on Cooperation is optimal, with weaker agents too - tldrIf conflict exists, one thing it can be useful for agents to do is misrepresent themselves as being weaker or stronger than they are.
nina-rimsky on [Fiction] A ConfessionProfound!
romeostevensit on Reconsider the anti-cavity bacteria if you are AsianSome data: https://onlinelibrary.wiley.com/doi/abs/10.1111/odi.12341
benito on A Review of In-Context Learning Hypotheses for Automated AI Alignment ResearchNot sure where the right place to raise this complaint, but having just seen it for the first time, really, "MARS"? I checked, this is not affiliated with MATS [? · GW] who have had like 6 programs and ~300 people go through it. To me this seems too close in branding space to me, and I'd recommend picking a more distinct name.
bogdan-ionut-cirstea on Bogdan Ionut Cirstea's ShortformRecent long-context LLMs seem to exhibit scaling laws from longer contexts - e.g. fig. 6 at page 8 in Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, fig. 1 at page 1 in Effective Long-Context Scaling of Foundation Models.
The long contexts also seem very helpful for in-context learning, e.g. Many-Shot In-Context Learning.
This seems differentially good for safety (e.g. vs. models with larger forward passes but shorter context windows to achieve the same perplexity), since longer context and in-context learning are differentially transparent [LW(p) · GW(p)].
gunnar_zarncke on Blessed information, garbage information, cursed informationExamples of blessed information that I have seen in the context of logging:
Oh cool! Um, first thought, register interest in this [LW · GW]?