Posts
Comments
These are quizzes you make yourself. Did OKC ever have those? It's not for a matching percentage.
A quiz in paiq is 6 questions, 3 multiple choice and 3 open. If someone gets the right answer on the multiple choice, then you get to see their open question answers as a match request, and you can accept or reject the match based in that. I think it's really great.
You can also browse other people's tests and see if you want to take any. The tests seem more descriptive of someone than most written profiles I've read cause it's much harder to misrepresent personal traits in a quiz then in a self-declared profile
I discovered the Netherlands actually has a good dating app that doesn't exist outside of it... I'm rather baffled. I have no idea how they started. I've messaged them asking if they will localize and expand and they thanked me for the compliment so... Dunno?
It's called Paiq and has a ton of features I've never seen before, like speed dating, picture hiding by default, quizzes you make for people that they can try to pass to get a match with you, photography contacts that involve taking pictures of stuff around and getting matched on that, and a few other things... It's just this grab bag of every way to match people that is not your picture or a blurb. It's really good!
That sounds great! I have to admit that I still get a far richer experience from reading out loud than subvocalizing, and my subvocalizing can't go faster than my speech. So it sounds like you have an upgraded form with more speed and richness, which is great!
Thanks! :D
Attention is a big part of it for me as well, yes. I feel it's very easy to notice when I skip words when reading out loud, and getting the cadence of a sentence right only works if you have a sense of how it relates to the previous and next one.
Yeah, that's my understanding as well.
Oh interesting! Maybe I'm wrong. I'm more curious about something like a survey on the topic now.
This is really good! Thank you for sharing _ competition drive and wanting to achieve certain things are great motivations, and I think in any learning process the motivation one can tap into is at least as important as the actual learning technique. I'm glad you had access to that.
I tend to feel a little confused about the concept of "intelligence", as I guess my post already illustrated, haha. I think the word as we use it is very imprecise for cases like this. I'd roughly expect people with higher general intelligence to be much faster and successful at finding workarounds for their language processing issues, but I'd also expect the variance in this to be so high as to make plotting your general intelligence against "how quickly did you tame your dyslexia" to not make super much sense.
Then again, I do agree with a comment somewhere else here that Typical Minding is a thing, and my intuitions here may be wrong cause I'm failing to understand what it's like for other minds and I might have overcorrected due to 25 years of incorrectly concluding I was kind of dumb. Lol.
Interesting! Thank you for sharing! I'd love to know the answer as well.
Anecdotally, I can say that I did try to learn Japanese a little, and I found Kanji far easier to learn than words in hiragana or katakana, cause relating a "picture" to a word seemed far easier for me to parse and remember than to remember "random phonetic encodings". I'm using quotation marks to indicate my internal experience, cause I'm a little mistrustful by now if I'm even understanding how other people parse words and language.
Either way, that anecdote would point to my pictoral->meaning wiring being stronger than my phoneme-encoding->meaning wiring. Which might explain why processing language as drawings helped me. I really have no idea how much this would generalize. But I agree people must run in to this when learning new alphabets.
[mind blown]
Minds are so interesting! Thank you for sharing!
Yeah, that sounds about right. Dutch culture has additionally strong reinforcement of typical mind fallacy cause being "different" in any direction is considered uncomfortable or unsocial, and everyone is encouraged to conform to the norm. There is a lot of reference to how all humans are essentially the same, and you shouldn't think you are somehow different or special. I think I absorbed these values quite a bit, and then applied some motivated cognition to not notice the differences in how I was processing information compared to my peers.
Thank you! I appreciate you sharing that _
My mother is/was very aware of historical practices and I think she often normalized my reading out loud with these types of references as well :)
I'm now going to admit your question made me realize I'm not sure "subvocalize" refers to the same thing for everyone ... I could always read in my head, but the error rate was huge. Only in my early 20s did I switch to a way of reading in my head that also does cadence and voices etc. The latter is what I mean by subvocalizing: The entire richness of an audiobook, generated by my own voice, but just so softly no one else can hear. It's a gradient from normal speech volume, to whisper, to whispering so softly no one can hear, to moving my lips and no sound coming out, to entire subvocalization.
Anyway, my prediction is that non-dyslectics do not subvocalize - it's much too slow. You can't read faster than you speak in that case.
Thank you for sharing!
Would it be correct to say that the therapy gave you the tools to read and write correctly with effort, and that the bullet point list shows motivations you experienced to actually apply that effort?
Cause my problem was mostly that I didn't know how to even notice the errors I was making, let alone correct for them. Once I knew how to notice them, I was, apparently, highly motivated to do so.
aaaaw thank you for saying that! _ I appreciate it!
Oh, that does help to know, thank you!
Hi! comment so everyone gets a msg about this:
Location is The Refter in Zwolle at Bethlehemkerkplein 35a, on the first floor!
If you have trouble finding it feel free to ping me here, on the discord, or the what's app group. Link to discord can be found below!
We are moving to Science Park Library
The ACX meeting on the same day is unfortunately cancelled. For that reason we are extending the deadline for sign up:
If you have a confirmation email, then you can definitely get in.
Otherwise, fill out the form and we'll select 3 people for the remaining spots. If people show up without signing up, they can get in if we are below 20. If we are on 20 or more, then no dice :D
(Currently 17)
Update: So far 11 people have been confirmed for the event. If you filled out the sign up form, but did not receive an email with confirmation, and you think you should, please DM me here on LW.
The last review cycle will be Friday morning, so if you want to attend, be sure to fill out the form before then.
Looking forward to seeing you there!
Here is the sign-up form. Please fill it out before Friday. People who are accepted in to the workshop will receive an email to that effect.
We have hit 15 signups!
Keep an eye on your inboxes for the signup form.
Well damn... Well spotted.
I found the full-text version and will dig in to this next week to see what's up exactly.
Thank you! I wholeheartedly agree to be honest. I've added a footnote to the claim, linking and quoting your comment. Are you comfortable with this?
Oooh gotcha. In that case, we are not remotely any good at avoiding the creation of unaligned humans either! ;)
Could you paraphrase? I'm not sure I follow your reasoning... Humans cooperate sufficiently to generate collective intelligence, and they cooperate sufficiently due to a range of alignment mechanics between humans, no?
Should we have a rewrite the Rationalist Basics Discourse contest?
Not that I think anything is gonna beat this. But still :D
Ps: can be both content and/or style
Thank you! I appreciate the in-depth comment.
Do you think any of these groups hold that all of the alignment problem can be solved without advancing capabilities?
Thanks!
And I appreciate the correction -- I admit I was confused about this, and may not have done enough of a deep-dive to untangle this properly. Originally I wanted to say "empiricists versus theorists" but I'm not sure where I got the term "theorist" from either.
Thanks!
And to both examples, how are you conceptualizing a "new idea"? Cause I suspect we don't have the same model on what an idea is.
Two things that worked for me:
-
Produce stuff, a lot of stuff, and make it findable online. This makes it possible for people to see your potential and reach out to you.
-
Send an email to anyone you admire asking if they are interested in going for a coffee (if you have the funds to fly out to them) or do a video call. Explain why you admire them and why this would be high value to you. I did this for 4 people without limit of 'how likely are they to answer' and one of them said 'yeah sure' and I think the email made them happy cause a reasonable subset of people like learning how they have touched other's lives in a positive way.
Even in experiments, I think most of the value is usually from observing lots of stuff, more than from carefully controlling things.
I think I mostly agree with you but have the "observing lots of stuff" categorized as "exploratory studies" which are badly controlled affairs where you just try to collect more observations to inform your actual eventual experiment. If you want to pin down a fact about reality, you'd still need to devise a well-controlled experiment that actually shows the effect you hypothesize to exist from your observations so far.
If you actually go look at how science is practiced, i.e. the things successful researchers actually pick up during PhD's, there's multiple load-bearing pieces besides just that.
Fair!
Note that a much simpler first-pass on all these is just "spend a lot more time reading others' work, and writing up and distilling our own".
I agree, but if people were both good at finding necessary info as an individual and we had better tools for coordinating (e.g.,finding each other and relevant material faster) then that would speed up research even further. And I'd argue that any gains in speed of research is as valuable as the same proportional delay in developing AGI.
There is an EU telegram group where they are, among other things, collecting data on where people are in Europe. I'll DM an invite.
That makes a lot of sense! And was indeed also thinking of Elicit
Note: The meetup this month is Wednesday, Jan 4th, at 15:00. I'm in Berkeley currently, and I couldn't see how times were displayed for you guys cause I have no option to change time zones on LW. I apologize if this has been confusing! I'll get a local person to verify dates and times next time (or even set them).
Did you accidentally forget to add this post to your research journal sequence?
I thought I added it but apparently hadn't pressed submit. Thank you for pointing that out!
- optimization algorithms (finitely terminating)
- iterative methods (convergent)
That sounds as if as if they are always finitely terminating or convergent, which they're not. (I don't think you wanted to say they are)
I was going by the Wikipedia definition:
To solve problems, researchers may use algorithms that terminate in a finite number of steps, or iterative methods that converge to a solution (on some specified class of problems), or heuristics that may provide approximate solutions to some problems (although their iterates need not converge).
I don't quite understand this. What does the sentence "computational optimization can compute all computable functions" mean? Additionally, in my conception of "computational optimization" (which is admittedly rather vague), learning need not take place.
I might have overloaded the phrase "computational" here. My intention was to point out what can be encoded by such a system. Maybe "coding" is a better word? E.g., neural coding. These systems can implement Turing machines so can potentially have the same properties of turing machines.
these two options are conceptually quite different and might influence the meaning of the analogy. If intelligence computes only a "target direction", then this corresponds to a heuristic approach in which locally, the correct direction in action space is chosen. However, if you view intelligence as an actual optimization algorithm, then what's chosen is not only a direction but a whole path.
I'm wondering if our disagreement is conceptual or semantic. Optimizing a direction instead of an entire path is just a difference in time horizon in my model. But maybe this is a different use of the word "optimize"?
You write "Learning consists of setting the right weights between all the neurons in all the layers. This is analogous to my understanding of human intelligence as path-finding through reality"
- Learning is a thing you do once, and then you use the resulting neural network repeatedly. In contrast, if you search for a path, you usually use that path only once.
If I learn the optimal path to work, then I can use that multiple times. I'm not sure I agree with the distinction you are drawing here ... Some problems in life only need to be solved exactly once, but that's the same as any thing you learn only being applicable once. I didn't mean to claim the processes are identical, but that they share an underlying structure. Though indeed, this might an empty intuitive leap with no useful implementation. Or maybe not a good matching at all.
I do not know what you mean by "mapping a utility function to world states". Is the following a correct paraphrasing of what you mean?
"An aligned AGI is one that tries to steer toward world states such that the neurally encoded utility function, if queried, would say 'these states are rather optimal' "
Yes, thank you.
I don't quite understand the analogy to hyperparameters here. To me, it seems like childbirth's meaning is in itself a reward that, by credit assignment, leads to a positive evaluation of the actions that led to it, even though in the experience the reward was mostly negative. It is indeed interesting figuring out what exactly is going on here (and the shard theory of human values might be an interesting frame for that, see also this interesting post looking at how the same external events can trigger different value updates), but I don't yet see how it connects to hyperparameters.
A hyperparameter is a parameter across parameters. So say with childbirth, you have a parameter pain on physical pain which is a direct physical signal, and you have a hyperparameter 'Satisfaction from hard work' that takes 'pain' as input as well as some evaluative cognitive process and outputs reward accordingly. Does that make sense?
What if instead of trying to build an AI that tries to decode our brain's utility function, we build the process that created our values in the first place and expose the AI to this process?
Digging in to shard theory is still on my todo list. [bookmarked]
Many models that do not overfit also memorize much of the data set.
Is this on the sweet spot just before overfitting or should I be thinking of something else?
Thank you for you extensive comment! <3
Oh my, this looks really great. I suspect between this and the other list of AIS researchers, we're all just taking different cracks at generating a central registry of AIS folk so we can coordinate at all different levels on knowing what people are doing and knowing who to contact for which kind of connection. However, maintaining such an overarching registry is probably a full time job for someone with high organizational and documentation skills.
I'll keep it in mind, thank you!
Great idea!
So my intuition is that letting people edit a file that is publicly linked is inviting a high probability of undesirable results (like accidental wipes, unnoticed changes to the file, etc). I'm open to looking in to this if the format gains a lot of traction and people find it very useful. For the moment, I'll leave the file as-is so no one's entry can be accidentally affected by someone else's edits. Thank you for the offer though!
Thank you for sharing! I actually have a similar response myself but assumed it was not general. I'm going to edit the image out.
EDIT: Both are points are moot using Stuart Armstrong's narrower definition of the Orthogonality thesis that he argues in General purpose intelligence: arguing the Orthogonality thesis:
High-intelligence agents can exist having more or less any final goals (as long as these goals are of feasible complexity, and do not refer intrinsically to the agent’s intelligence).
Old post:
I was just working through my own thoughts on the Orthogonality thesis and did a search on LW on existing material and found this essay. I had pretty much the same thoughts on intelligence limiting goal complexity, so yay!
Additional thought I had: Learning/intelligence-boosting motivations/goals are positively correlated with intelligence. Thus, given any amount of time, an AI with intelligence-boosting motivations will become smarter than those do not have that motivation.
It is true that instrumental convergence should lead any sufficiently smart AI to also pursue intelligence-boosting (cognitive enhancement) but:
- At low levels of intelligence, AI might fail at instrumental convergence strategies.
- At high levels of intelligence, AI that is not intelligence-boosting will spend some non-zero amount of resources on its actual other goals and thus be less intelligent than an intelligence-boosting AI (assuming parallel universes, and thus no direct competition).
I'm not sure how to integrate this insight in to the orthogonality thesis. It implies that:
"At higher intelligence levels, intelligence-boosting motivations are more likely than other motivations" thus creating a probability distribution across the intelligence-goal space that I'm not sure how to represent. Thoughts?
Hmm, that wouldn't explain the different qualia of the rewards, but maybe it doesn't have to. I see your point that they can mathematically still be encoded in to one reward signal that we optimize through weighted factors.
I guess my deeper question would be: do the different qualias of different reward signals achieve anything in our behavior that can't be encoded through summing the weighted factors of different reward systems in to one reward signal that is optimized?
Another framing here would be homeostasis - if you accept humans aren't happiness optimizers, then what are we instead? Are the different reward signals more like different 'thermostats' where we trade off the optimal value of thermostat against each other toward some set point?
Intuitively I think the homeostasis model is true, and would explain our lack of optimizing. But I'm not well versed in this yet and worry that I might be missing how the two are just the same somehow.
Clawbacks refer to grants that have already been distributed but would need to be returned. You seem to be thinking of grants that haven't been distributed yet. I hope both get resolved but they would require different solutions. The post above is only about clawbacks though.
As a grantee, I'd be very interested in hearing what informs your estimate, if you feel comfortable sharing.
Netherlands
Small celebration in Amsterdam: https://www.lesswrong.com/events/mTxNWEes265zkxhiH/winter-solstice-amsterdam
Sure. For instance, hugging/touch, good food, or finishing a task all deliver a different type of reward signal. You can be saturated on one but not the others and then you'll seek out the other reward signals. Furthermore, I think these rewards are biochemically implemented through different systems (oxytocin, something-sugar-related-unsure-what, and dopamine). What would be the analogue of this in AI?
ah, like that. Thank you for explaining. I wouldn't consider that a reversal cause you're then still converting intuitions into testable hypotheses. But the emphasis on discussion versus experimentation is then reversed indeed.
What would the sensible reverse of number 5? I can generate those them for 1-4 and 6, but I am unsure what the benefit could be of confusing intuitions with testable hypotheses?
I really appreciate that thought! I think there were a few things going on:
- Definitons and Degrees: I think in common speech and intuitions it is the case that failing to pick the optimal option doesn't mean something is not an optimizer. I think this goes back to the definition confusion, where 'optimizer' in CS or math literally picks the best option to maximize X no matter the other concerns. While in daily life, if one says they optimize on X then trading off against lower concerns at some value greater than zero is still considered optimizing. E.g. someone might optimize their life for getting the highest grades in school by spending every waking moment studying or doing self-care but they also spend one evening a week with a romantic partner. I think in regular parlance and intuitions, this person is said to be an optimizer cause the concept is weighed in degrees (you are optimizing more on X) instead of absolutes (you are disregarding everything else except X).
- unrepresented internal experience: I do actually experience something related to conscious IGF optimization drive. All the responses and texts I've read so far are from people that say that they don't, which made me assume the missing piece was people's awareness of people like myself. I'm not a perfect optimizer (see above definitional considerations) but there are a lot of experiences and motivations that seemed to not be covered in the original essay or comments. E.g. I experience a strong sense of identity shift where, since I have children, I experience myself as a sort of intergenerational organism. My survival and flourishing related needs internally feel secondary to that of the aggregate of the blood line I'm part of. This shift happened to me during my first pregnancy and is quite a disorienting experience. It seems to point so strongly at IGF optimization that claiming we don't do that seemed patently wrong. From examples I can now see that it's still a matter of degrees and I still wouldn't take every possible action to maximize the number of copies of my genes in the next generation.
- where we are now versus where we might end up: people did agree we might end up being IGF maximizers eventually. I didn't see this point made in the original article and I thought the concern was that training can never work to create inner alignment. Apparently that wasn't the point haha.
Does that make sense? Curious to hear your thoughts.
good to know, thank you!
On further reflection, I changed my mind (see title and edit at top of article). Your comment was one of the items that helped me understand the concepts better, so just wanted to add a small thank you note. Thank you!