Posts
Comments
Oh no, that was a bad typo. It has now been corrected.
Wow. I'm running 3.1 now, and my laptop's fan isn't running at all. Wild!
What a fantastic product. Reminds me of the 3-d Reddit museum app.
run LessWrong 2.0 by moving everything to a javascript based web-app architecture, so we consider this a natural next step for us to take.
Okay, but actually, though, I'm still hoping for the day where lesswrong.com loads more comparably to greaterwrong.com.
Hi, thanks for sharing and experimentally trying out the theory in the previous post! Super cool.
Do you have the code for this up anywhere?
I'm also a little confused by the training procedure. Are you just instantiating a random vector and then doing GD with regards to the loss function you defined? Do the charts show the loss averaged over many random vectors (and function variants)?
Overall enjoying this series and your take on CFAR-style rationality. Thanks for putting in the time to write this up.
Michael Nielsen also has some great stuff. Especially his quantum.country and neural networks one.
Slight ntipick: Simply because logistic isn't actually used in practice anymore, it might be better to start people new to the sequence with a better activation function like reLU or tanh?
When I was first learning this material, due to people mentioning sigmoid a lot, I thought it would be a good default, and then I learned later on that it's actually not the activation function of choice anymore, and hasn't been for a while. (See, for example, Yann LeCun here in 1998 on why normal sigmoid has drawbacks.)
As a university student with ties to EA (and also looking at future opportunities), the EA forum post you linked gave some useful anecdotes to think about. Thank you for sharing the list.
Just wanted to thank you for writing up this series. I've been slowly going through the book on my own. Just finished Chapter 2 and it's awesome to have these notes to review.
A friend I know actually goes everyday with a GoPro recording his interactions.
Also, I'm wondering if you have thoughts on where to store this preserved information? Making sure that future people have access to it seems like the important part. But obviously just making it all available publicly online for everyone seems too vulnerable. Maybe some sort of dead-man's switch type setup, where it gets made public after you die?
Fading Novelty is the first post, so it's supposed to be read from top to bottom.
Finally finished up polishing old posts in my series on instrumental rationality. Didn't cross-post it to LW because much of the stuff is cannibalized, but the link is here https://mlu.red/. Posts are meant to be read sequentially, but I haven't added "next post" functionality yet.
Whoa! I wrote about something similar here a while ago under the same name, at least about the aesthetics part.
Note: Anna Salamon has a public response on the FB post here (unsure to what extent it's official)
Seconded. In my view, the anecdotes are there such that the idea is more salient and hangs around longer in your head.
Sure, you can read 10 self-help summaries in an hour, but I don't think that gives you 10x the same amount of benefit as reading about one concept for an hour. (If anything, I don't even think you get 1x the same amount of benefit, as you have to factor in potential confusion sorting everything out, etc.)
The padding can also be useful if you're trying to learn via example, or learn what the stereotype of The Concept looks like.
Ah, gotcha.
LFD was my first intro to statistical learning theory, and I think it's pretty clear. It doesn't cover the No Free Lunch Theorem or Uniform Convergence, though, so your review actually got me wanting to read UML. I think that if you're already getting the rigor from UML, you probably won't get too much out of LFD.
I'm curious if you've looked at Learning From Data by Abu-Mostafa, Magdon-Ismail, and Lin? (There's also a lecture series from CalTech based off the book.)
I haven't read Understanding Machine Learning, but it does seem to be an even more technical, given my skimming of your notes. However, the Mostafa et al book does give a proof of why you can expect the VC dimension to be polynomially bounded for a set of points greater than the break point (if the VC dimension is finite), as well as a full proof of the VC Bound in the appendix.
Hmmm. I agree with you that fingernail biting didn't seem to fit the paradigm. However, I did Google "stop biting fingernails", though, to see if there was any domain specific suggestions. (You may have already done this.)
Two things that maybe seemed promising:
- Wear gloves to prevent easy access to hands
- Getting a fidget toy to keep your hands otherwise busy
Something else which seems maybe useful is to be mindful/reflective after you've noticed that you've done it.
Otherwise, I (at least right now) don't know much about breaking habits without knowing the trigger.
Thanks for the info, Ozzie!
I checked out Observable some more. I think it might actually be a little heavier then what I want. Unsure if I'll do the coding exercises beforehand (and just post the results + code), or if I'll go through the work of setting up an interactive notebook so readers can follow along.
I looked into self-hosting it because it seems the default option is creating a notebook hosted on their site. My understanding is that there's a way to embed notebooks onto my own sites (or the runtime environment is open-sourced?)
I'm going to spend some of the winter holidays working on Abu-Mostafa et al's Learning From Data's problem set. I think this should be fun, and I'll also look into learning Observable for some interactive notebooks for the coding problems.
This piece was helpful in outlining how different people in the AI safety space disagree, and what the issues with Paul's approaches seem to be. Paul's analogies with solving hard problems was especially interesting to me (the point where most problems don't seem to occupy a position midway between totally impossible and solvable). The inline comments by Paul were also good to read as counterpoints to Eliezer's responses.
Sidenote: Loved the small Avatar reference in the picture of the cabbage vendor.
I've been thinking about interpretable models. If we have some system making decisions for us, it seems good if we can ask it "Why did you suggest action X?" and get back something intelligible.
So I read up about what sorts of things other people have come up with. Something that seemed cool was this idea of tree regularization. The idea being that decision trees are sort of the standard for interpretable models because they typically make splits along features. You essentially train a regularizer (which is a neural net) which proxies average tree length (i.e. the complexity of a decision tree which is comparable to the actual model you're training). Then, when you're done, you can train a new decision tree which mimics the final neural net (the one you trained with the regularizer).
The author pointed out that, in the process of doing so, you can see what features the model thinks are relevant. Sometimes they don't make sense, but the whole point is that you can at least tell that they don't make sense (from a human perspective) because the model is less opaque. You know more than just "well, it's a linear combination of the inputs, followed by some nonlinear transformations, repeated a bunch of times".
But if the features don't seem to make sense, I'd still like to know why they were selected. If the system tells us "I suggested decision X because of factors A, B, and C" and C seems really surprising to us, I'd like to know what value it's providing to the prediction.
I'm not sure what sort of justification we could expect from the model, though. Something like "Well, there was this regularity that I observed in all of the data you gave me, concerning factor C," seems like what's happening behind the scenes. Maybe that's a sign for us to investigate more in the world, and the responsibility shouldn't be on the system. But, still, food for thought.
Yeah! The screenshot you shared helps. I think most of the frontpage stuff ends up greyed out for me because I click on most things.
I use the GW styling that mimics the old LW. I think that certain things which seem to matter a lot to me are darker borders and higher contrast.
Even now, I still don't think I like the LW redesign, mostly because of speed and aesthetics reasons. I know stuff is in place to speed up the site, so I guess that's a work in progress. The grey on grey for text, though, feels like it's way too long contrast; there's something else aesthetically going on where because everything is the same shade of light grey, nothing feels like it has "weight", and the focus isn't fully on the content either because of how it all doesn't seem to pop out.
For me, all of the nifty new features like sequences, meetup pages, and shortform feeds feel like they're missing the point. If the site feels slow and doesn't seem to have the visual affordances, I don't feel compelled to participate as fully, regardless of what else I can do on the site.
I'm glad greaterwrong exists because it addresses both of these issues, but I'm curious if these are turn-offs for other people.
Ack! My error! I see now.
Hmmm, am I doing something wrong?
My karma:
What I see when I click the three dots on a page:
I think I have enough karma, but I can't figure out where the nomination button is. Could someone share a screenshot?
I had this happen to me as well. Firefox 70 on Ubuntu 18.04
Also, anecdotally, there have been lots of Indian applicants (and attendees) at ESPR throughout the years. Seems like people there also think rationality is cool (lots of the people I interviewed had read HPMOR, there are LW meetups there, etc. etc.)
Thought as I worked through the exercise:
- Is there something I'm missing? It seems like TurnTrout's already given us all the pieces. Seems like we can say that "Something has high impact to someone if it either affects something they value (the personal side) or affects their ability to do things more broadly (the objective side)."
- Something is a big deal if it affects our ability to take future actions? (That seems to be the deal about objectively being bad.)
- Is the point here to unify it into one sort of coherent notion?
- Okay, so let's back up for a second and try to do all of this from scratch...When I think about what "impact" feels like to me, I imagine something big, like the world exploding.
- But it doesn't necessarily have to be a big change. A world where everyone has one less finger doesn't seem to be a big change, but it seems to be high impact. Or a world where the button that launches nukes is pressed rather than not pressed. Maybe we need to look some more into the future? (Do we need discounting? Maybe if nukes get launched in the far future, it's not that bad?)
- I think it's important to think relative to the agent in question, in order to think about impact. You also want to look at what changed. Small changes aren't necessarily low impact, but I think large changes will correspond to high impact.
- It seems like "A change has has high impact if the agent's valuation of the after state is very different than their valuation of the current state" is the best I have after 15 minutes...
Michael Sipser's Introduction to the Theory of Computation goes over the recursion theorem and Rice's theorem, IIRC. The proofs are given as well as associated exercises. The textbook walks you through, from DFAs to Turing Machines, so it's pretty self-contained, if you're looking at a source other than Computability and Logic to understand them.
One thing here that seems important to note is what each medium does to your attention and what sort of cognitive work it facilitates:
To borrow a few items from your list:
- Videogames: literally a Skinner box that gives you reinforcement to keep doing the thing.
- Web surfing / news feeds / blogs / movies: makes you a passive consumer of the content.
- Direct messaging: requires you to spend time thinking about your response.
- Writing software / making videos / drawing comics: puts you in a position to think about the message you want to convey, teaching to others requires you to bridge inferential gaps, look at your models.
- Spaced repetition: literally designed to make you remember stuff.
Thanks for trying these out, Ben!
If you ever are interested in learning close-up magic some more, I have lots more thoughts on what good resources are for learning / have strong opinions on what makes a good magic effect. I haven't written about them for the LW audience, but maybe more of this hybrid stuff will manifest later on.
The toy example you gave seems like something that would make for a fun simulation ala Nicky Case's stuff, you can try with multiple groups, different types of evidence (which support either side in varying amounts), and different coordination mechanisms.
I'll look into something this weekend. If anyone else likes doing JS development, ping me, and we can figure something out!
There's jquery UI which maybe counts?
Ben Pace has a new post up on LessWrong that's asking about good exercises for rationality / general LW-adjacent stuff. I think this is a good thing to put up a bounty for, and I started thinking about what makes a good exercise. Exercises are good because they help you further the develop the material; they give you opportunities to put whatever relevant skill to use.
There are differing levels of what you can be trying to assess:
- Identifying the correct idea from a group of different ones
- Summarizing the correct idea
- Transferring the idea to someone else
- Actually demonstrating whatever skill it is (if it's something you can do)
- Actually using the skill to deduce something else (if it's a model thing)
I think there's a good set of stuff to dive into here about the distinction between optimizing for pedagogy versus effectiveness. In the most stark case, you want to teach people using less potent versions of something, at least at first. Think not just training wheels on a bike, but successively more advanced models for physics or arithmetic. There's a gradual shift happening.
More than that, I wonder if the two angles are greatly orthogonal.
Anyway, back to the original idea at hand. When you give people exercises, there's a sense of broad vs narrow that seems important, but I'm still teasing it out. In one sense, you can think of tests that do multiple choice vs open-ended answers. But it's not like multiple-choice questions have to suck. You could give people very plausible-sounding answers which require them to do a lot of work to determine which one is correct. Similarly, open-ended questions could allow for bullshitting.
It's not exactly the format, but what sort of work it induces.
At the very least, it's about pushing for more Generative content. But beyond that, it gets into pedagogy questions:
- How can you give questions which increase in difficulty?
- What does difficulty correspond to? If something is "hard to figure out", what is that quality referring to?
- If you give open-ended questions, how can you assess the answers you get?
- How much of this is covered already by the teaching literature?
I recently wrote about three things you can try with cards to see what your internal calibration feels like. They have some question prompts, but the gist of it is something to do, rather than something with a direct answer.
https://www.lesswrong.com/posts/Ktw9L67Nwq8o2kecP/calibrating-with-cards
I see! Thanks for the breakdown for where the pain points are when it comes to performance. Really appreciate the openness into where things could have gone better / what's happening right now!
Oh, wow! I didn't realize that could have been tripping things up. Thank you for the formatting help!
META: The code block editor wasn't very friendly and ate up all of my tabs. I'm working on better formatting, and this'll probably end up being a post on my own blog later on, which will hopefully also have things like syntax highlighting.
For sure! To be honest, I got a little lost reading your 3-part series here, so I think I'll revisit it later on.
I'm newer to deep learning, so I think my goals are similar to yours (e.g. writing it up so I have a better understanding of what's going on), but I'm still hashing out the more introductory stuff.
I'll definitely link it here after I finish!
Thanks for writing this series!
I'm working on my own post on NNs that focuses more on deriving backprop from computational graphs. I think that method of doing so also builds up a lot of the Chain Rule intuition, as you can easily see how the derivatives for earlier weights can be derived from those in later weights.
I really like that you're doing this! I've tried to get into the series, but I haven't done so in a while. Thanks for the summaries!
(Also, maybe it'd be good for future comments about what you're doing to be children of this post, so it doesn't break the flow of summaries.)
Your advice about demonstrating that you are capable alone is really interesting. Thanks for the extended examples!
Experience As Compounding:
Sometimes I ask myself: "A bunch of cool stuff seems to be happening in the present. So why can't I move faster and let these things in? Why do I feel stuck by past things?"
Well, experience compounds. One reason childhood events can be so influential isn't just that they happened when you were at a formative time and developing your models. In addition, the fact that you pick them up early means they've had the privilege of being part of your thought processes for longer. They're more well-worn tools.
Then, there's also the default answer that each additional year of your life is, relative to the amount of years you've lived, a lesser amount. EX: From year 6 to 7, you've gained an extra ~15% of your total lifespan in new experiences. Whereas from 26 to 27, you've gained closer to 4% of your total lifespan in new experiences.
But, I'd like every year to be measured more equally with one another. I feel like cool stuff is passing by me right now, and I'm just slow on the uptake. I'm not taking it in!
Yes, you can get set in your older ways of thinking, and you will have seen more with each successive year. But experientially speaking I'd like to get my brain to also pay more attention to the recent stuff.
I guess one hacky way to do this would be to spend more time ruminating on the present (which is also harder because if you've lived for 30 years, then by the same proportionality argument, there's just less stuff to think about if you restrict yourself to years 29-30).
I'm confused because there is also:
Experience as a Sliding Window:
There's some sort of cutoff point where I might be able to recall things, but it no longer feels "recent" or directly connected to my identity.
The feeling of recency is quite interesting to me because it seems to imply that important things are going to fade over time. And if you want to preserve certain parts of your identity, there's some sort of "upkeep" you'll need to pay, i.e. having more of those sort of experiences consistently so they stay in recent memory.
Anyway, that's if you equate identity with memory, and that's definitely an oversimplification. But, whatever.
As new things filter in, older things drop out. I'm unsure how to square this with the theory of compounding experience. Presumably if something has effects, even if it falls out of the window, then things it influenced can continue to resound, ala domino effect, but that feels quite contrived. The obvious answer, of course, is that there are several factors at play.
One common theme that I return to, time and time again, is that of addictiveness. More specifically, what makes something habit-forming in a bad way? I've previously talked about this in the context of Attractors. Lately, my thing to hate on is mobile games, or the thing that they represent. Which, yes, is a little late to the game. And I don't even play games on my mobile phone, so it seems a little out of place.
But I digress. The point here is to talk about the Skinner Box. Or, the application of the same concept to human things. Gamification and notification spam both fall into this category. But maybe not games. But maybe some games. Definitely mobile games. The point here is that there's this category I want to get some clarity on, and it's about these things which seem habit-forming and suck you in.
So, what's clearly a Skinner Box? I think that clicker games are totally Skinner Boxes. Also Clash of Clans, Farmville (i.e. everything Zynga / Zynga-clones). But this line is often hazy; Candy Box was innovative and exciting in certain ways. There was a game a while back about alpacas eating one another that seemed surprisingly deep for an idle game. It's one thing to put on a sophisticated veneer on a game, but it still seems fine to critique the underlying mechanics.
What does make a Skinner Box?
- Lack of a challenge
- Despite having progression, idle and clicker games don't really have anything that forces the player to do anything strategic. They just...click things, and they get reinforcement.
- Instant gratification
- Mobile games often leverage this desire by time-locking content, prompting you to pay in order to get something now. The other thing to pay attention here is if the feedback loop is tight.
- Incentives to keep going?
- Intermittent rewards / reward schedules
What doesn't make a Skinner Box?
- Skill and growth
- The more something is like an instrument or a sport, the less it seems like a Skinner Box. Although the many casual LoL players seem to indicate that even something which has a high skill cap can still be addictive.
- Meaning
- The more you invoke artistic purpose, narrative, or some other agenda, we seem to be a lot more forgiving about the actual mechanics involved.
- Instrumentality
- When we're hungry, we eat and eat and eat. And no one bats an eye. The same thing with sleep. Stuff that's useful isn't often seen as dangerous.
It feels like there's been a push towards getting people to start creating their own content. Platforms like YouTube + the Internet make it a lot easier for people to start.
Growing an audience, though, seems hard because there's not often a lot of free attention. Most of the competition is zero-sum between different content. People only have so much free time, so minutes they spend engaging with your stuff is minutes they don't spend engaging in other people's stuff.
There's a cynical viewpoint here which is something like "If you don't think you're creating Good Content, don't broadcast it! We have enough low-quality stuff as it is, out there."
I think people often want to create, though. It's one of the default responses people have if you ask them "Say you could live comfortably without needing to work. What would you do then?" ("Well, I'd write. Or I'd learn to play an instrument...")
Often, though, implementation takes far more time than coming up with the initial idea. There is an asymmetry across many fields where the actual ideation is done by only a small group of people. This then requires maybe 10X as many people to actually put into practice. (EX: the people who design the look/feel of a piece of software at a company vs those who build it.)
Thus, if you want people to join your project (which is of course great because you came up with it), you'll need to convince other people to go with you. On the flip side, I think there's a skill worth practicing where you let go of idea ownership. Stuff is going to get done, and you're going to be doing it; whoever came up with the idea might be less important than whether or not you want the stuff to happen.
But maybe the desire for individual ideation points to something important. A really large amount of people seem to want to partake in creative endeavors.
Here's something that feels like another instance of the deontologist vs consequentialist abstraction, except that the particulars of the situation are what stick out to me: When I choose between doing something sane or something that's endorsed by an official rule, I'll more-often-than-I-like opt to do the endorsed thing, even when it's obviously worse-off for me.
Some examples, of varying quality:
- Not jaywalking, even when it's in a neighborhood or otherwise not-crowded place.
- Asking for permission to do obvious things instead of just doing them
- Focusing on the literal words that someone initially said, rather than their intent, or if they later recant.
- Letting harmful policies happen instead of appealing them.
I'm reminded of that study which showed that people following an evacuation robot were led to stay in a room even when there was a fire, even when the robot was observed to be previously faulty. There's something about rules that overrides appealing to sanity. I'm a little worried that I bias towards this side compared to just doing the thing that works out better.
There are of course benefits to choosing the official option. The biggest one is that if someone questions your judgment later on, you can appeal to the established rules. That gives you a lot of social backing to lean on.
I think there's also a weird masochistic aspect of craving pity, of wanting to be in a situation that seems bad by virtue of nature, so I can absolve myself of responsibility. Something about how this used to be a play to secure ourselves more resources, through a pity play?