Posts
Comments
Ack! My error! I see now.
Hmmm, am I doing something wrong?
My karma:
What I see when I click the three dots on a page:
I think I have enough karma, but I can't figure out where the nomination button is. Could someone share a screenshot?
I had this happen to me as well. Firefox 70 on Ubuntu 18.04
Also, anecdotally, there have been lots of Indian applicants (and attendees) at ESPR throughout the years. Seems like people there also think rationality is cool (lots of the people I interviewed had read HPMOR, there are LW meetups there, etc. etc.)
Thought as I worked through the exercise:
- Is there something I'm missing? It seems like TurnTrout's already given us all the pieces. Seems like we can say that "Something has high impact to someone if it either affects something they value (the personal side) or affects their ability to do things more broadly (the objective side)."
- Something is a big deal if it affects our ability to take future actions? (That seems to be the deal about objectively being bad.)
- Is the point here to unify it into one sort of coherent notion?
- Okay, so let's back up for a second and try to do all of this from scratch...When I think about what "impact" feels like to me, I imagine something big, like the world exploding.
- But it doesn't necessarily have to be a big change. A world where everyone has one less finger doesn't seem to be a big change, but it seems to be high impact. Or a world where the button that launches nukes is pressed rather than not pressed. Maybe we need to look some more into the future? (Do we need discounting? Maybe if nukes get launched in the far future, it's not that bad?)
- I think it's important to think relative to the agent in question, in order to think about impact. You also want to look at what changed. Small changes aren't necessarily low impact, but I think large changes will correspond to high impact.
- It seems like "A change has has high impact if the agent's valuation of the after state is very different than their valuation of the current state" is the best I have after 15 minutes...
Michael Sipser's Introduction to the Theory of Computation goes over the recursion theorem and Rice's theorem, IIRC. The proofs are given as well as associated exercises. The textbook walks you through, from DFAs to Turing Machines, so it's pretty self-contained, if you're looking at a source other than Computability and Logic to understand them.
One thing here that seems important to note is what each medium does to your attention and what sort of cognitive work it facilitates:
To borrow a few items from your list:
- Videogames: literally a Skinner box that gives you reinforcement to keep doing the thing.
- Web surfing / news feeds / blogs / movies: makes you a passive consumer of the content.
- Direct messaging: requires you to spend time thinking about your response.
- Writing software / making videos / drawing comics: puts you in a position to think about the message you want to convey, teaching to others requires you to bridge inferential gaps, look at your models.
- Spaced repetition: literally designed to make you remember stuff.
Thanks for trying these out, Ben!
If you ever are interested in learning close-up magic some more, I have lots more thoughts on what good resources are for learning / have strong opinions on what makes a good magic effect. I haven't written about them for the LW audience, but maybe more of this hybrid stuff will manifest later on.
The toy example you gave seems like something that would make for a fun simulation ala Nicky Case's stuff, you can try with multiple groups, different types of evidence (which support either side in varying amounts), and different coordination mechanisms.
I'll look into something this weekend. If anyone else likes doing JS development, ping me, and we can figure something out!
There's jquery UI which maybe counts?
Ben Pace has a new post up on LessWrong that's asking about good exercises for rationality / general LW-adjacent stuff. I think this is a good thing to put up a bounty for, and I started thinking about what makes a good exercise. Exercises are good because they help you further the develop the material; they give you opportunities to put whatever relevant skill to use.
There are differing levels of what you can be trying to assess:
- Identifying the correct idea from a group of different ones
- Summarizing the correct idea
- Transferring the idea to someone else
- Actually demonstrating whatever skill it is (if it's something you can do)
- Actually using the skill to deduce something else (if it's a model thing)
I think there's a good set of stuff to dive into here about the distinction between optimizing for pedagogy versus effectiveness. In the most stark case, you want to teach people using less potent versions of something, at least at first. Think not just training wheels on a bike, but successively more advanced models for physics or arithmetic. There's a gradual shift happening.
More than that, I wonder if the two angles are greatly orthogonal.
Anyway, back to the original idea at hand. When you give people exercises, there's a sense of broad vs narrow that seems important, but I'm still teasing it out. In one sense, you can think of tests that do multiple choice vs open-ended answers. But it's not like multiple-choice questions have to suck. You could give people very plausible-sounding answers which require them to do a lot of work to determine which one is correct. Similarly, open-ended questions could allow for bullshitting.
It's not exactly the format, but what sort of work it induces.
At the very least, it's about pushing for more Generative content. But beyond that, it gets into pedagogy questions:
- How can you give questions which increase in difficulty?
- What does difficulty correspond to? If something is "hard to figure out", what is that quality referring to?
- If you give open-ended questions, how can you assess the answers you get?
- How much of this is covered already by the teaching literature?
I recently wrote about three things you can try with cards to see what your internal calibration feels like. They have some question prompts, but the gist of it is something to do, rather than something with a direct answer.
https://www.lesswrong.com/posts/Ktw9L67Nwq8o2kecP/calibrating-with-cards
I see! Thanks for the breakdown for where the pain points are when it comes to performance. Really appreciate the openness into where things could have gone better / what's happening right now!
Oh, wow! I didn't realize that could have been tripping things up. Thank you for the formatting help!
META: The code block editor wasn't very friendly and ate up all of my tabs. I'm working on better formatting, and this'll probably end up being a post on my own blog later on, which will hopefully also have things like syntax highlighting.
For sure! To be honest, I got a little lost reading your 3-part series here, so I think I'll revisit it later on.
I'm newer to deep learning, so I think my goals are similar to yours (e.g. writing it up so I have a better understanding of what's going on), but I'm still hashing out the more introductory stuff.
I'll definitely link it here after I finish!
Thanks for writing this series!
I'm working on my own post on NNs that focuses more on deriving backprop from computational graphs. I think that method of doing so also builds up a lot of the Chain Rule intuition, as you can easily see how the derivatives for earlier weights can be derived from those in later weights.
I really like that you're doing this! I've tried to get into the series, but I haven't done so in a while. Thanks for the summaries!
(Also, maybe it'd be good for future comments about what you're doing to be children of this post, so it doesn't break the flow of summaries.)
Your advice about demonstrating that you are capable alone is really interesting. Thanks for the extended examples!
Experience As Compounding:
Sometimes I ask myself: "A bunch of cool stuff seems to be happening in the present. So why can't I move faster and let these things in? Why do I feel stuck by past things?"
Well, experience compounds. One reason childhood events can be so influential isn't just that they happened when you were at a formative time and developing your models. In addition, the fact that you pick them up early means they've had the privilege of being part of your thought processes for longer. They're more well-worn tools.
Then, there's also the default answer that each additional year of your life is, relative to the amount of years you've lived, a lesser amount. EX: From year 6 to 7, you've gained an extra ~15% of your total lifespan in new experiences. Whereas from 26 to 27, you've gained closer to 4% of your total lifespan in new experiences.
But, I'd like every year to be measured more equally with one another. I feel like cool stuff is passing by me right now, and I'm just slow on the uptake. I'm not taking it in!
Yes, you can get set in your older ways of thinking, and you will have seen more with each successive year. But experientially speaking I'd like to get my brain to also pay more attention to the recent stuff.
I guess one hacky way to do this would be to spend more time ruminating on the present (which is also harder because if you've lived for 30 years, then by the same proportionality argument, there's just less stuff to think about if you restrict yourself to years 29-30).
I'm confused because there is also:
Experience as a Sliding Window:
There's some sort of cutoff point where I might be able to recall things, but it no longer feels "recent" or directly connected to my identity.
The feeling of recency is quite interesting to me because it seems to imply that important things are going to fade over time. And if you want to preserve certain parts of your identity, there's some sort of "upkeep" you'll need to pay, i.e. having more of those sort of experiences consistently so they stay in recent memory.
Anyway, that's if you equate identity with memory, and that's definitely an oversimplification. But, whatever.
As new things filter in, older things drop out. I'm unsure how to square this with the theory of compounding experience. Presumably if something has effects, even if it falls out of the window, then things it influenced can continue to resound, ala domino effect, but that feels quite contrived. The obvious answer, of course, is that there are several factors at play.
One common theme that I return to, time and time again, is that of addictiveness. More specifically, what makes something habit-forming in a bad way? I've previously talked about this in the context of Attractors. Lately, my thing to hate on is mobile games, or the thing that they represent. Which, yes, is a little late to the game. And I don't even play games on my mobile phone, so it seems a little out of place.
But I digress. The point here is to talk about the Skinner Box. Or, the application of the same concept to human things. Gamification and notification spam both fall into this category. But maybe not games. But maybe some games. Definitely mobile games. The point here is that there's this category I want to get some clarity on, and it's about these things which seem habit-forming and suck you in.
So, what's clearly a Skinner Box? I think that clicker games are totally Skinner Boxes. Also Clash of Clans, Farmville (i.e. everything Zynga / Zynga-clones). But this line is often hazy; Candy Box was innovative and exciting in certain ways. There was a game a while back about alpacas eating one another that seemed surprisingly deep for an idle game. It's one thing to put on a sophisticated veneer on a game, but it still seems fine to critique the underlying mechanics.
What does make a Skinner Box?
- Lack of a challenge
- Despite having progression, idle and clicker games don't really have anything that forces the player to do anything strategic. They just...click things, and they get reinforcement.
- Instant gratification
- Mobile games often leverage this desire by time-locking content, prompting you to pay in order to get something now. The other thing to pay attention here is if the feedback loop is tight.
- Incentives to keep going?
- Intermittent rewards / reward schedules
What doesn't make a Skinner Box?
- Skill and growth
- The more something is like an instrument or a sport, the less it seems like a Skinner Box. Although the many casual LoL players seem to indicate that even something which has a high skill cap can still be addictive.
- Meaning
- The more you invoke artistic purpose, narrative, or some other agenda, we seem to be a lot more forgiving about the actual mechanics involved.
- Instrumentality
- When we're hungry, we eat and eat and eat. And no one bats an eye. The same thing with sleep. Stuff that's useful isn't often seen as dangerous.
It feels like there's been a push towards getting people to start creating their own content. Platforms like YouTube + the Internet make it a lot easier for people to start.
Growing an audience, though, seems hard because there's not often a lot of free attention. Most of the competition is zero-sum between different content. People only have so much free time, so minutes they spend engaging with your stuff is minutes they don't spend engaging in other people's stuff.
There's a cynical viewpoint here which is something like "If you don't think you're creating Good Content, don't broadcast it! We have enough low-quality stuff as it is, out there."
I think people often want to create, though. It's one of the default responses people have if you ask them "Say you could live comfortably without needing to work. What would you do then?" ("Well, I'd write. Or I'd learn to play an instrument...")
Often, though, implementation takes far more time than coming up with the initial idea. There is an asymmetry across many fields where the actual ideation is done by only a small group of people. This then requires maybe 10X as many people to actually put into practice. (EX: the people who design the look/feel of a piece of software at a company vs those who build it.)
Thus, if you want people to join your project (which is of course great because you came up with it), you'll need to convince other people to go with you. On the flip side, I think there's a skill worth practicing where you let go of idea ownership. Stuff is going to get done, and you're going to be doing it; whoever came up with the idea might be less important than whether or not you want the stuff to happen.
But maybe the desire for individual ideation points to something important. A really large amount of people seem to want to partake in creative endeavors.
Here's something that feels like another instance of the deontologist vs consequentialist abstraction, except that the particulars of the situation are what stick out to me: When I choose between doing something sane or something that's endorsed by an official rule, I'll more-often-than-I-like opt to do the endorsed thing, even when it's obviously worse-off for me.
Some examples, of varying quality:
- Not jaywalking, even when it's in a neighborhood or otherwise not-crowded place.
- Asking for permission to do obvious things instead of just doing them
- Focusing on the literal words that someone initially said, rather than their intent, or if they later recant.
- Letting harmful policies happen instead of appealing them.
I'm reminded of that study which showed that people following an evacuation robot were led to stay in a room even when there was a fire, even when the robot was observed to be previously faulty. There's something about rules that overrides appealing to sanity. I'm a little worried that I bias towards this side compared to just doing the thing that works out better.
There are of course benefits to choosing the official option. The biggest one is that if someone questions your judgment later on, you can appeal to the established rules. That gives you a lot of social backing to lean on.
I think there's also a weird masochistic aspect of craving pity, of wanting to be in a situation that seems bad by virtue of nature, so I can absolve myself of responsibility. Something about how this used to be a play to secure ourselves more resources, through a pity play?
Malcolm Ocean gets it. There's a terrible thing that happens when you try to encapsulate your essay with a title. Somehow, the label takes on a life of its own, and you sometimes forget the content inside the essay.
This happens to my own essays where I think "Oh, huh, this essay is called 'Learning from Past Experiences'". Sounds kinda boring.
And in fact it was not boring and it was good.
I'm thinking of maybe instead transitioning to just numbers + summaries instead.
For example, a format like: Essay 10 [Fading novelty, ways to address it, and a brief digression into typography.]
Update: currently using Remarkbox for comments.
FWIW, both linear algebra and multivariable calculus are required for students at UC San Diego, which is a large public institution.
(Although it's a little tricky as our university has sub-colleges, not all of which require both).
Thanks! I'm a little worried about my own signal-to-noise ratio for now, so I'll do that later on if I feel my short-form quality improves.
Meant for this to be a reply.
Yeah, Netlify was really easy to set up, so I recommend them!
I don't think I'll be able to migrate blog comments because I'm still not sure what I'll do for comments on the new blog, actually.
I don't want to use Disqus because it's bulky, other options cost money, so maybe a self-hosted option...
Long-term, I'll probably do some more organizing to put posts into sequences / other things to improve readability.
Right now, the focus is on updating/editing old posts, so that the main page is a set of polished essays that work as standalones.
This seems reasonable.
See also Yudkowsky in Inadequate Equilibria for a similar sentiment:
Try to spend most of your time thinking about the object level. If you’re spending more of your time thinking about your own reasoning ability and competence than you spend thinking about Japan’s interest rates and NGDP, or competing omega-6 vs. omega-3 metabolic pathways, you’re taking your eye off the ball.
I don't think the author disagrees all that much with you. I'm reading his claim as something more like "the default attitude (some) people have towards reading does not set them up for good learning".
In the essay, he acknowledges the role that effort and metacognition play in making the actual learning happen. The actionable parts I found useful were at the end where he was hypothesizing about improved mediums, e.g. an online textbook with spaced repetition built in to facilitate recall.
I think I agree that the generalizations you cited were hasty, especially as there is no formal review on those matters. I, too, find that I can get value out of books and can probably do a better job than just giving a brief summary.
Let me try to pick out the parts that I liked and see if you disagree with those:
The main part of the thesis that I found interesting was the analogy between books and lectures. In both cases, there is the potential of transmissionism as one naive way of thinking about how information gets absorbed.
It's clearer, however, that lectures in an educational setting aren't enough. Without problem sets, examples, and other applications, most of the "real learning" doesn't happen. (i.e. "mathematics is not a spectator sport") Good professors seem to recognize this and supplement their teachings accordingly.
Then, the author points out that a similar acknowledgment for books is not the norm. There is still room for improving the medium, and one example he gave was the spaced repetition enhanced online book. There is an undercurrent of "you as the reader need to put in effort to get value out of your reading", which I agree with. It's a different state of mind when I'm reading for insight vs reading for fun. In the first case, I might re-read passages, skip around, stop for a bit, take some notes, etc. etc. whereas in the second case, I'm probably just reading rather quickly from cover to cover.
I see. Thanks for providing the additional info!
I've read over briefly both this article and the previous one in the series. Thank you for putting these together!
What I'm curious about in quant trading is the actual implementation. Once you, say, have a model which you think works, how important is latency? How do you make decisions about when to buy / sell? (Partially echoing Romeo's sentiment about curiosity around stop losses and the actual nitty-gritty of extracting value after you think you've figured something out.)
Heads-up: nowadays, when people talk about neural networks for games, they really mean deep learning combined with reinforcement learning.
Back to your question: When you don't have a log of games, you typically have some other way of assessing performance, e.g. assigning a "score" to the state of the game, which you can quantify and optimize.
For a specific well-known example, I think this paper on training to play Atari games with deep reinforcement learning goes over a lot of the actual math / implementation details.
I've looked a little bit at the RAISE website, and I've looked at the overview of curriculum topics, and I'm finding it a little...sparse, maybe? (I haven't actually looked at the class materials on grasple though, so maybe there's more stuff there.) I'm wondering how realistic it would be for someone to start engaging with MIRI-esque topics after learning just the courses RAISE has outlined.
At least for the prerequisites course, these are all topics covered throughout the first two years of a typical undergraduate computer science degree. And that doesn't seem like quite enough.
EX: TurnTrout's sequence of essays on their journey to become able to contribute towards MIRI-esque topics seems to span a much greater gamut of topics (linear algebra, analysis, etc.) at greater depth, closer to what one might cover in graduate school.
I guess, to operationalize, I'm curious about:
1. What target audience RAISE has in mind (technical people looking for a refresher, people who have had zero real exposure to technical subjects before, etc. etc.) for their materials.
2. What degree of competence RAISE expects people to come out of the curriculum with, either best-case or average-case.
3. In the best case, how many units of material do you think RAISE can product? In other words, is it enough for students to study RAISE's material for a 6-month long curriculum? 1 year long?
****
(Of course, it's also much easier from my position to be engaging/critiquing existing works, than to actually put in the effort to make all of this happen. I don't mean any of the above as an indictment. It's admirable and impressive that y'all have coordinated to make this happen at all!)
If you want a proof-based approach, Linear Algebra Done Right is the typical go-to that's also on the MIRI page. I went through maybe the first 3/4ths of it, and I thought it was pretty good, in terms of number of exercises and helping you think about manipulating vector spaces, etc. in a more abstract sense.
Otherwise, I've heard good things about Gilbert Strang's MIT OCW course here: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/.
In general, I think that 3B1B's videos are really good for building intuition about a concept, but trying to do exercises off of the pedagogy in his videos alone can be quite challenging, especially as he often assumes some mastery with the subject already. (EX: In the eigen-stuffs video, he doesn't actually explain how to find the eigenvalues of a matrix.)
Thus, I think it makes more sense to stick to a traditional textbook / course for learning linear algebra and using 3B1B as supplementary stuff for when you want a visual / different way of looking at a concept.
Also, it might be worth checking in to see what you want to learn linear algebra for. I suspect there are more domain specific resources if, for example, you cared about just the useful parts of linear algebra used in machine learning (dimensionality reduction, etc.).
I think the overarching thing to do is to simply write more. To that end, there are a lot of ways to make this happen. For example: participating in NaNoWriMo, committing to writing a sequence of articles about topic X, engaging in discourse online, summarizing research papers for a larger audience, writing guest articles for a blog, or journaling.
As for improving your actual workflow, I think that the Typical Writing Class you take in school largely gets it right. For example, writing outlines / summaries before starting the actual writing, asking people to proofread, have several drafts / edit, and try to imagine reading it from the reader's perspective w/o the extra context you have as a writer.
(I think most of the dissatisfaction I had with the skills taught to me during class had a lot more to do with the context of "Oh man, I have to write this thing using this technique for school and not of my own volition?" rather than the skills themselves not being very good.)
Do you have any thoughts written up anywhere on what a game you'd design would look like? I've been following all of your MTG / digital CCG articles with a lot of interest; I'd be really curious which aspects of game design you'd like to incorporate.
I am wondering if there are commonalities between what different self-help things are doing. For example, it seems that a lot of self-help is focused on changing our default actions, ala debiasing, so there is a train of thought that starts with cognitive biases and goes from there.
A related question I'm wondering about, which seems related to this is "Why does self-help work? What is it doing?"
Not 100% related to the question, but maybe the discussion here in the comments could spark more (or another question).
Oh, right, yes, I didn't think about that.
That is a good example where the above advice I listed, as well as the following models, break.
Oh, right. I think an implicit thing here is "for people you want to be close to, this makes sense to do".
In other cases, as I sorta skimmed over, having simplified models, relying on norms/roles, etc. etc. is usually enough to get by.
The Learning from Untrusted Data paper was really clear, and I found the results surprising. Thanks for sharing and for consistently putting this out week after week. I really like the combination of the summary + your personal thoughts.
Specifically for the example of social influencers, I wonder if roles like this might be limited in capacity because, by virtue of their role, they require a large following. Given the way popularity and status tends to be non-uniformly distributed across social groups, it might not be a very large role for most people to fit into.
But overall, I think the underlying theme of humans-as-valuable-because-they-are-human and the examples to Etsy charging a premium on human-made things hits the nail on the head as for where residual value might lie in an automated future.
I suppose one corollary, too, is that it will be human desires which comprise the demand for these goods. If it turns out that automation also shifts the demand for certain services, via machines sending orders to machines, then this might also limit future options for humans?
Bug report: I can no longer view old messages.
I clicked on my profile, went to private messages, and then clicked on something from the conversations sidebar.
Every message had the following red error:
"Error: TypeError: Cannot read property '_id' of null"
You might already be aware, but there's the Unofficial CFAR Canon List which compiles a lot of their earlier material (though some things have now changed) that someone put together a while back.
If you're looking for more derivative content written by people who have gone to CFAR workshops:
There's also the Hammertime sequence alkjash wrote and the Instrumental Rationality sequence I wrote.
Yup! I just set up one up here. I just grabbed an off-the-shelf Python package so let me know if there are any issues.