Posts
Comments
I don't mean to assert that one effect is bigger than the other, more that together they create a vicious cycle. No one disputes that bad decisions can lead to poverty, that's common sense, or that other factors influence it, but if poverty itself is a multiplier it stands to reason that that needs to be addressed as part of any potential solution. The next post (dropping Saturday) is about how, in such coordination problems, multiple factors must align in order for any one solution to be effective.
Oh, sorry, that was largely boiler-plate, while this post did have some hover-over info which didn't translate to LW (which was actually kind of important, as it provided some disclaimers and caveats to points made) it's probably not what you'd call a "full experience". Some other posts on the site have simulations.
Though I do think the overall aesthetic of the posts on the site is subtly important for the tone of my writing (generally a not too serious tone).
I'd be interested in reading up on the replication problems with priming if you have any links. I wasn't on guard for this sort of research, so it seemed plausible to me. All of this goes against our general intuitions that people need to feel their poverty to get motivated for working, so I'm more likely to accept scientific research than assume it's wrong and that my intuitions are correct.
Wait, are you trying to tell me that drug addiction and mental illness contribute to poverty? That seems like a stretch... (jokes)
I feel like that's is a given, the factors that perpetuate poverty are manifold. The article focuses on this finding because it reveals a counter-intuitive and therefore easily overlooked contributory factor (and a significant one at that).
I didn't mean at all to suggest that this finding was the only contributory factor, or that it could be solved with any one solution (which is why the title of the last section reads "a" solution, not "the" solution).
Thanks for reading and responding :)
I think the idea is really interesting. As someone who spent 5 years creating student video resources, I appreciate the impact they can have, and I have at times tried to convince my father—a life-long maths teacher to collaborate with me on replicating his course... but the fool didn't take me up on the offer.
the cost of showing it to every student in the country is approximately zero
I feel like the cost-effectiveness argument is valid but might run into issues. To begin with, as you have in one of your comments pointed out, video resources with a teacher who can respond dynamically, adds much more than a video alone. So, this means there is no cost saving in terms of teachers time—which I think is a good thing (I'll put a pin in that for later) and then video production on top of that is not at all cheap. One thing that was consistent, in my experience creating educational resources, was the need to constantly update the resources (there was a team of us working full time to just maintain one course).
So, while the cycle of feedback and constant improvement of the resources is a vital part of the process, it makes what seems like a one-off expense into a perpetual expense.
Furthermore teachers are already underpaid, relative to other professions requiring similar skills, so the additional funding for these new resources would need to result from an unprecedented increase in education funding (which could have gone to teachers) or would have to be taken from the budget at the expense of teachers.
Unless of course you leave it to the private sector in which case you have to worry about advertising, special interests and competition leading optimisation for what is appealing to students rather than what is necessarily effective—Hollywood, after all only has the mandate to entertain, they don't have to also educate.
To get back to that pin: If we did manage to create a resource perhaps incorporating generative AI that can present ideas in an engaging way and provide dynamic feedback, making teachers unnecessary we run into another issue. There's something to be said for having well-rounded educators in society, learning in a non-specialised way is enriching for people in general. One negative side of chat GPT is this way drastic drop-off in activity on forums like Stack Overflow, because people don't need other people any more.
There's something about the person to person trading of ideas that I think contributes to a robust community, in the same way that international trade helps to curb international conflicts—we might find that making human interaction unnecessary to education whether in schools or on forums might lead to a fragmentation of the social fabric. Personally I really like the idea of lots of amateurs sharing ideas—like on LessWrong and other forums, there's something uniquely human about learning from sharing, with benefits for the teacher also (à la the Feynman Technique).
But, I think you make a good case. Thanks for sharing the idea.
Hey, again good points.
But I have recognized sparks of true understanding in one-shot AI works.
I absolutely agree here, this is what I was referring to when I wrote...
I think we can appreciate the beauty of connecting with humanity as a whole, knowing that it is the big data of humanity that has informed AI art - I suspect this is what we find so magical about it.
I suspect that AI has an appeal not just because of its fantastic rendering capacity but also the fact that it is synthesising works not just from a prompt but from a vast library of shared human experience.
you're looking at hours of thought and effort
Regarding the arduous* process of iteratively prompting and selecting AI art, I think the analogy with photography works in terms of evoking emotions. Photographers approach their works in a similar way, shooting multiple angles and subjects and selecting those that resonate with them (and presumably others) for exhibition or publication. I think there is something special about connecting with what a human artist recognised in a piece whether it came from a camera or an algorithm. I acknowledge this is a form of connection that is still present in AI art, just as it is in photography.
* I caveat "arduous" because, while it might take hours of wrangling the AI to express something approximating what we intend, the skill that takes artists years to master—that of actually creating the work, is largely performed, in the case of AI art, by the non-sentient algorithm. It is not the hours of work that goes into one painting that impresses the viewer generally, it's the unseen years of toil and graft that allowed the artist to make something magic within those hours. The vast majority of the magic in AI art is provided by the algorithm.
This is why I see it as analogous to photography. Still a valid art form, but not one that need make actual painting obsolete.
Okay, so I think I get you now, in the imbalanced game, if the payoff is 100 or 1 as in "Zero Sum" is a misnomer, a rational player will still make the same decision, regardless of the imbalance with the other player, given the resulting preference ordering.
However while this imbalance makes no difference to the players' decisions, it does make a difference to the total payoff, making it non-zero-sum. I'm having difficulty understanding why values such as happiness or resources cannot be substituted for utility—surely at some point this must happen if game theory is to have any application in the real world. Personally I'm interested in real world applications, but I fully acknowledge my ignorance on this particular aspect.
I find a practical way to look at a zero-sum game is to imagine that each of two players must contribute half of the total payoff in order to play. This takes a game that is constant-sum, and makes it zero-sum, and does so in a way that doesn't break the constant-sumness. In the case of the imbalanced game, because it is not constant-sum it doesn't reduce to a zero-sum game in this way, remaining a non-zero-sum game with terrible odds for one player, meaning that a rational player won't opt in if given the option.
If I'm not mistaken, this is generally what is meant when someone refers to a zero-sum game. In chess for instance you enter a competition with your rating (essentially your bet) and the outcome of the game has either a negative or positive (or no) impact on your rating and an opposite impact on your opponent's rating.(I'm not exactly sure if chess ratings are calculated as exactly zero-sum, but you get the idea). So, the game is zero-sum. Of course there are outside factors that make it beneficial to both players; enjoyment, brain-exercise, socialising etc which may have positive utility on another level, but the game itself and the resulting rating changes are essentially zero-sum.
This is the sense in which I am using the term "zero-sum", in the most basic sense for someone to win (relative to their starting point, bet or rating) another must lose by an equal amount.
There is probably a more mathematically succinct way of expressing this, but I don't have those tools at my disposal at present. Again, thanks for your comments. Please don't feel the need to continue your labours educating me on this topic, I understand that you clearly have a better understanding of game theory than I do, so I appreciate your time. I should probably continue reading further to level up my understanding. Of course if you feel like continuing the floor is yours.
I’m not sure how the game is the same when you add a constant. The game as proposed in the example is clearly different. I can see that multiplication makes no difference, and as such doesn’t make the sum non-constant. I don’t see how asymmetrically changing the parameters is a “mere change in notation”.
By the way, I’m sure you’re entirely correct about this, I just simply don’t see how there is a problem with using the concept of zero-sum understood as constant-sum.
Hi Vladimir, thanks for your input, it has been fascinating going down the rabbit hole of nuance regarding the term "zero-sum".
I agree that the term is more accurately denoting "constant-sum", I think this is generally implied by most people using it. There was the interesting "zero-sum" example in the linked article that veered away from "constant-sum" with asymmetrical payoffs, 100,0 or 0,1, meaning that depending on the outcome of the game the total sum would be different. This, to me disqualifies it from being called a zero-sum game, given the common understanding that zero-sum denotes constant-sum. The example seemed to solve the problem by conflating zero-sum and constant-sum and then proceeded to stick to a strict definition of zero-sum, which was confusing. But perhaps I just need to sit with it longer.
To your point about Kaldor-Hicks, yes I guess many positive-sum situations could be described in these terms but I'm really referring to something more general—any situation where the total sum payoff increases regardless of Pareto improvements or promised reimbursement by other means to any party left worse off. For instance if a left-wing government were to increase taxes on the wealth, not offering them any reimbursement, but rather doing this based on the mandate that comes with being democratically elected, then this policy might be positive-sum due to the fact that dollar-for-dollar money makes a bigger difference to a poor person than a rich person, due to diminishing returns on happiness.
I really appreciate your comments, and intend to continue exploring the nuances you've raised. I think for a primer on non-zero-sum games, particularly with a site that is focused on practical solutions in the real world rather than pure theory, the more accessible (perhaps less nuanced) definitions I've used are probably appropriate.
Hi Vladimir,
Thanks for your comment, please excuse the delay in getting back, I'm actually busily digesting your response and the various branches of dependencies that comprise it (in terms of links to other concepts). I intend to get back to you with a considered answer, but am enjoying taking my time exploring the ideas you've linked to.
Hi bideup, thanks for your comment. The graph is simplified from one in the Pew Report with the left bar representing the lower quintile and the right representing the upper. I see what you mean, but the intention of pointing to the 20% mark is to show where it should be given 100% social mobility. Perhaps the omission of the central quintiles didn’t help.
There's a tiny possibility he may have influenced my thinking. I did spend 6 months editing him, among others for a documentary.
That looks interesting, will read :) Thanks.
What an insightful post!
I have difficulty judging how likely that is, but the odds will improve if semi-wise humans keep getting input from their increasingly wise AGIs.
I think we're on the same page here, positing that AGI could actually help to improve alignment—if we give it that task. I really like one of your fundamental instructions being to ask about potential issues with alignment.
And on the topic of dishing out tasks, I agree that pushing the industry toward Instruction Following is an ideal path, and I think there will be a great deal of consumer demand for this sort of product. A friend of mine has mentioned this as the no-brainer approach to AI safety and even a reason what AI safety isn't actually that big a deal... I realise you're not making this claim in the same way.
My concern regarding this is that the industry is ultimately going to follow demand and as AI becomes more multi-faceted and capable, the market for digital companions, assistants and creative partners will incentivise the production of more human, more self-motivated agents (sovereign AGI) that generate ideas, art and conversation autonomously, even spontaneously.
Some will want a two-way partnership, rather than master-slave. This market will incentivise more self-training, self-play, even an analogue to dreaming / day-dreaming (all without a HITL). Whatever company enables this process for AI will gain market share in these areas. So, while Instruction Following AI will be safe, it won't necessarily satisfy consumer demand in the way that a more self-motivated and therefore less-corrigible AI would.
But I agree with you that moving forward in a piecemeal fashion with the control of an IF and DWIMAC approach gives us the best opportunity to learn and adapt. The concern about sovereign AGI probably needs to be addressed through governance (enforcing HITL, enforcing a controlled pace of development, and being vigilant about the run-away potential of self-motivated agents) but it does also bring Value Alignment back into the picture. I think you do a great job of outlining how ideal an IF development path is, which should make everyone suspicious if development starts moving in a different direction.
Do you think it will be possible to create an AGI that is fundamentally Instruction Following that could satisfy the market for the human-like interaction some of the market will demand?
I apologise if you've, in some way I've not recognised, already addressed this question, there were a lot of very interesting links in your post, not all of which I could be entirely sure I grokked adequately.
Thanks for your comments, I look forward to reading more of your work.
Thanks, very astute point.
Yes, the individual and the collected are tightly coupled with short-term and long-term goals, which exist within individuals too. I think it's interesting to think of yourself as a city, where you need to make systemic changes sometimes to enable individual flourishing.
I really think there is something to making alignment the actual goal of AI—but in a way where the paradoxical nature of alignment is acknowledged, so the AI is not looking for a "final solution" but is rather measuring the success of various strategies in lowering society's (to return to the metaphor of the individual) cognitive dissonance.
Thanks Seth, great points.
Would I like your drawings less if you had conceived them in detail, but not directed the brushstrokes with your own hands and brain?
I guess, for centuries revered artists have been directing apprentices to render works for them, AI is not a significant departure from this practice, and I think you're right—there is still a connection with an artist even if their role was to prompt for an emotion, and then, in their selection, recognise that emotion when the AI successfully captured it.
This is making me hope that we see more detailed accounts of the creative process
This is really interesting. As your rightly state, making their intentions explicit is not a common practice for artists, my cynicism again tells me this could be c) part of the inflationary imperative of artists to play on the allure of ineffable qualities in art to make something relatively shallow seem deep and meaningful (and therefore valuable).
Imagine a space where truly deep thoughts, and profound messages well put could generate art that really connects people—not because it means something different to every person but because it evokes the very same thing, common humanity, among unique individuals. Pretty lofty.
Thanks again for your kind words and insight.
Well, yes, good point—people consume art for all sorts of reasons.
Though I wasn't meaning to say that anyone consciously looks at an artwork with the intention of connecting with the artist, only that it's an implied prerequisite, as in, if we're impressed by the skill, we're impressed because we have a sense of how difficult that would be for a human (being a human ourselves) or if we think the work has captured an emotion we might implicitly assume that the artist recognised that same emotion in creating the work. These features of the art-consumption experience are largely absent in AI art, when we are pretty certain that the "artist" has no conscious experience.
But, yes, I take your point, and people can appreciate AI art for many reasons besides.
That was fascinating, and well written, thanks for sharing. There I thought eventually having my wisdom that out would be a hassle, now I’m looking forward to it.
Interesting the use of the word “fascinating” as that carries with it some sense of an emotional experience, some excitement. But perhaps, like a language model can be used as simple hyperbole. Also the enthusiasm noted, suggesting that enthusiasm and conscious experience can be decoupled.
Thanks for sharing, definitely one of those experiments that will inform my thinking next time I’m thinking about AI sentience.
Hi Seth,
Thanks for your kind words. It's funny, I think I naturally write in a longer more convoluted style, but have worked hard to make my writing accessible and short—nice to know the effort pays off.
The cartoons are drawn with an Apple Pencil on an iPad Pro using Procreate (the studio pen is great for cartooning if you're really interested). I set up a big canvas 1000px wide and about 5000px high, then go about drawing all of them top to bottom. Then I export to photoshop, crop and export to png with a transparent background so that whatever colour the page is shows through. Those I've used here on LW are screenshots from the blog itself as the image backgrounds don't work well on white or black (the only options here—my site is generally a pastel blue). I've explained in another post why I'm keeping my crappy drawings in the face of the generative AI revolution.
Thanks for the extra info around terms like "coordination", good to know. I actually mention Moloch in part 2 and have written a series on Moloch, funny you use the word "thorny", as the cartoon characters I use for that series are called "Thorny Devils" (Moloch Horridus).