Posts
Comments
Andy Matuschak @andymatuschak:
Finally found a single actual screenshot of the DARPA Digital Tutor (sort of—a later commercial adaptation). Crazy-making that there were zero figures in any of the papers about its design, and not enough details to imagine one.
Some observations:
* An instructional interface is presented alongside a live machine.
* Student presented with a concrete task to achieve in the live system.
* The training system begins by “discussing the situation”, probing the student’s understanding with q's, and responding with appropriate feedback and follow-up tasks.
* It can observe the student’s actions in the live system and respond appropriately.
* The instructional interface uses a text-conversational modality.
* I see strong influence from Graesser's AutoTutor, and some from Anderson's Cognitive Tutors.
(from https://www.edsurge.com/news/2020-06-09-how-learning-engineering-hopes-to-speed-up-education )
https://twitter.com/andy_matuschak/status/1782095737096167917
This all seems very teleological. Do you have thoughts on what the teleology of the universe could be under this model?
Thanks for asking. This is the intention of Mathematical Boundaries Workshop which is running now. Let me know if you'd like to come on Sunday
New better link: https://www.aria.org.uk/programme-safeguarded-ai/
yea
right, yeah, i think precisely formalizing boundaries is less useful for the cyborgism angle
Personal anecdote:
Ever since reading George's post, I've been noticing ways in which I have been (subconsciously) tensing muscles in my neck-- and possibly around my vagus nerve and inside my head. I wonder if by tensing these muscles, I'm reducing blood flow.
(I can think of reasons why someone might learn to do this on purpose actually, eg in response to some social stress.)
So now I'm experimenting with relaxing those muscles whenever I notice myself tensing them. Maybe this increases blood flow, idk. It maybe feels a little like that.
re Q2-
So I don't doubt that improvements in subjective wellbeing are reported essentially unanimously.
But, to give a sense of the kind of thing I'm expecting here, consider that a child who doesn't learn to be emotionally insecure around their parents is probably much worse off. In some societies, parents who dislike a child starve/kill them, and emotional insecurity can be one way to predict and therefore avoid others disliking you.
In which case, I wonder, if you don't have these common delusions about the mind (or you're ~enlightened), does this put you in a worse place physically or socially?
(Probably not in all possible environments, but maybe this is true in some [social] environments that are common today.)
Some various questions:
Q1: To what extent do you think ~unenlightenment in an individual is caused by the need to fit in socially?
Ie: In order to get other people to take care of you or not kill you (especially when you're a vulnerable child), you contort your mind in all sorts of ways and construct an ego (very much in the Elephant in the Brain way) and adopt all sorts of delusions.
For example, you might want to be able to control other people, and one way to do that is to exile your emotional emotions so you can tell them "You made me so angry! Stop doing that!" (Then later, if that doesn't work, you can say, "I'm so sorry, my emotions got the best of me" -- as if your emotions are separate from you, lol. Have your cake and eat it too.)
I write a little bit about how my experience of depression seems like this here.
Q1.b: To what extent do you think become more spiritually skilled is just about learning how to integrate with other people safely, but without having those common-but-helpful-but-wrong delusions about how your own mind works?
Q2: Do you think people benefit from being ~unenlightened or spiritually unskilled? Precisely how so?
whoa @Joe Carlsmith wrote a whole thing about boundaries I had no idea https://www.lesswrong.com/posts/rdTgtHn3neGzkyCrL/being-nicer-than-clippy#Boundaries
taking something negative in one's brain and asking, "but how is this useful? What is it doing for me? What is this piece of me trying to protect me from?"
yes yes this!
The Coherence Therapy Institute case studies are great for this btw
Just curious, how related do you think your symptoms are to social interaction?
Though they have very different methods, both anxiety and depression tend to have the same result, at least for me: I don’t do anything.
This reminds me of the post I wrote about my own depression: https://chipmonk.substack.com/p/depression-was-useful
I'd like to reply to your comment but I didn't understand your first sentence
Where do you think the boundary is here?
The curation failed? the email is empty for me
New version of this?
Me reading this post:
- wow wtf these results, cool if true!
- … * a bunch of explanation * ...
- *the post ends*
- wait what did you actually do for "increasing cerebral vascularization and broadening my proprioception"?
What were your interventions?
Update: found them on your substack:
The method that I used consisted of targeted NIR interference therapy, short UV during the morning, a lot of inversion-based exercises where I focused on contracting/relaxing neck and face muscles, a few customized breathing exercises (think wim hof), figuring out the correct levels for a bunch of cholinergic vaso[dilators/modulators] (think noopept), massage therapies to reduce tension on the spine, some proprioception-heavy movement practices, a niche tibetan metta meditation series… and about 5 other things that are even harder to compress. The main point is that “the method” doesn’t matter so much, you can just google “intervention to increase IQ”, find 50 things, dig through the evidence, select 20, combine them, and assume 5 work
---
I think the core point of "how" is really unimportant, since I didn't do something optimal... not even close, I did something "silly" that I could execute part time with pocket change.
So I don't want to bias people towards this particular method.
I discuss this in Protecting agent boundaries.
Synthesis says worked with DARPA to make a tool to teach kids math: https://www.synthesis.com/tutor
(Also, I asked a friend who worked there about this last summer and, if I recall correctly, they said it does not use LLMs, it's something else.)
- show up to office hours for classes you aren’t a part of, just to chat with the professor
this is how I became friends with Po-Shen Loh. I would be the only person to show up to office hours, and we wouldn't even talk about math.
oh this would make a lot more sense if true.
How were you getting SLIT for $25/week
I don't know, but the online shops that sell it are still $100/mo today.
how much did SLIT cost in total for you
$25/wk * 2 years
were the doses tailored to your particular allergies based on tests?
yes, there were several allergens in there, just the ones i was allergic to
The advice I wish I had earlier is:
- https://dilbertblog.typepad.com/the_dilbert_blog/2007/06/the_day_you_bec.html
- read one of Derek Sivers's books (as an example)
- Derek Sivers also has some recommendations for some writing books that are probably good. I tried Several Short Sentences About Writing and idk it was okay for me, some people really like it though
LW is full of good writers
I disagree with this definitively. I can't read most if not almost all LW posts. You can do much better if you know what you want to communicate and only say the essential words.
If you don't like fluoride polish you can instead bring your own nano-hydroxyapatite tooth polish. (It's essentially tooth polish made from [synthetic] teeth.) I ship this one from Japan (it's also sometimes available on US amazon).
Yes, see Agent membranes/boundaries and formalizing “safety” and davidad's comment.
(Also, I'm not necessarily agreeing that your examples are not violations of boundaries. First one isn't a violation of end-person (although probably the farmer). Second one could be.)
in full generality, what's a "threat"?
in full generality, what's a "dangerous" collision?
Hm I'm not immediately sure how to define these
is that the defense is actually used to strike inside another's boundary, as has been the case for ~all weapons
Yeah, I am worried about this.
This is notably not the case for infosec and encryption, where defensive capability doesn't imply offensive capability. However, I'm unsure if this is also true for any physical interventions. (e.g.: Vaccines? No, bioweapons… Nanotech? No…)
That said, physical interventions do seem to be defense-dominant when there is coordination among a sufficiently large portion of society/power.
Today, I revised it to be much more clear and complete.
Thread of revisions to this post. (This post was originally published on 2024 Jan 3.)
any thoughts on davidad's shutdown timer?
this exists for some of those -- texts.com
I think the more general form of the emotions thing is: reductionism and "i can't understand it consciously therefore it's not rational".
The counter is deep respect for Chesterton's Fence.
This is also how many people get into woo
"selective, corrective, and structural" are kinda like:
- Selective: Interventions that occur before the critical event. t < T_c
- Corrective: Interventions that occur during the critical event. t ~= T_c
- Structural: Interventions that occur after the critical event. t > T_c
The non-linear drama might be another good case study
Just checking: are your timelines still this long?
Here's a tricky example I've been thinking about:
Is a cell getting infected by a virus a boundary violation?
What I think makes this tricky is that viruses generally don't physically penetrate cell membranes. Instead, cells just "let in" some viruses (albeit against their better judgement).
Then once you answer the above, please also consider:
Is a cell taking in nutrients from its environment a boundary violation?
I don't know what makes this different from the virus example (at least as long as we're not allowed to refer to preferences).
Where is the 2024 fellowship going to be?
edit: oh i see website
- Fellows can join the program from anywhere in the world. We provide tailored solutions to support fellows in their research, such as access to one of several AI safety-specific office spaces in Europe or the US, or support to visit their mentors in-person.
If you're an agent observing other agents, other agents stop seeming sovereign from your perspective if you can mind-read those agents in high resolution.
And "Vingean Agency" is the opposite of that, got it.
he says a bit in this direction- see my other comment
any proposed actions which can be meaningfully interpreted by sandboxed human-level supervisory AIs as messages with nontrivial semantics could be rejected.
I want to give a big +1 on preventing membrane piercing not just by having AIs respect membranes, but also by using technology to empower membranes to be stronger and better at self-defense.
Thanks for writing this! I largely agree (and the rest I need to think more about)
thx to friend:
The key scholars are Amartya Sen and Martha Nussbaum. SEP is a good place to get oriented: https://plato.stanford.edu/entries/capability-approach/
thanks just listened to it. This reminds me a lot of what Scott Garrabrant has been thinking about. Perhaps intentionally setting up membranes within a society so that failure/infection/etc. in one region doesn't infect every other region. They talk about the same thing but for insight in solving problems
Any link to Critch suggesting this?
Psychology may [… be …] ultimately very important, perhaps critically important
oh good, my other project is psychology, so I really hope it circles back.
I originally got into the AI safety stuff because I was trying to understand psychological boundaries properly and realized that no one did. Then I realized that many people were maybe doing the same thing when thinking about interactions between AIs and humans.
This is also maybe one of my qualms about anchoring this idea on the words "membranes" and "boundaries"... i think the actual structure is a bit more continuous. "Membranes" are a nice anchor abstraction for other reasons though so I'm sticking with it for now
Hmm I'm confused about why people ask "where is the boundary?" in this situation.
I visualize this situation as more of a topological map, kinda like
where the horizontal axes are kinda like physical space (more precisely: such that pieces (e.g.: your arm, your brain, web search, personal note taking app) thar are more closely connected are closer horizontally)
and the vertical axis is like "communication bandwidth". For example, the communication bandwidth (ability to sense and control) between your brain/nervous system and your arm is pretty darn high, probably higher than almost anything else in the universe.
Yeah, you could try to cut out a particular hill in that topology and say "that hill is Bob" but that's obviously cutting a lot of detail.
Oh, yes! I forgot about your original comment about this
Oh.
But why shouldn't it be hardcoded?
essentially also "membranes for safety" idea but Critch takes it a broader and more civilizational
could you restate your argument again plainly i missed it