Better thinking through experiential games
post by Morendil · 2009-10-23T12:59:12.901Z · LW · GW · Legacy · 36 commentsContents
36 comments
A few years ago I came across The Logic of Failure by Dietrich Doerner (previously mentioned on LW) which discusses cognitive failures in people dealing with "complex situations".
One section (p.1 28) discusses a little simulation game, where participants are told to "steer" the temperature of a refrigerated storeroom with a defective thermostat, the exact equation governing how the thermostat setting affects actual temperature being unknown. Players control a dial with settings numbered 0 through 100, and can read actual temperature off a thermometer display. The only complications in this task are a) that there is a delay between changing the dial and the effects of the new setting; b) the possibility of "overshoot".
I found the section's title chilling as well as fascinating: "Twenty-eight is a good number." Doerner says this statement is typical of what participants faced with this type of situation tend to say. People don't just make ineffective use of the data they are presented with: they make up magical hypotheses, cling to superstitions or even call into question the very basis of the experiment, that there is a systematic link between thermostat setting and temperature.
Reading about it is one thing, and actually playing the game quite another, so I got a group of colleagues together and we gave it a try. We were all involved in one way or another with managing software projects, which are systems way more complex than the simple thermostat system; our interest was to confirm Doerner's hypothesis that humans are generally inept at even simple management tasks. By the reports of all involved it was one of the most effective learning experiences they'd had. Since then, I have had a particular interest in this type of situation, which I have learned is sometimes called "experiential learning".
As I conceive of it, experiential learning consists of setting up a problematic situation, in such a way that the students ("players") should rely on their own wits to explore the situation, invent ways of dealing with it (sometimes by incorporating conceptual tools provided by an instructor), and test their newfound insights against the original problem - or against real-world situations. My preferred setting for experiential learning is a small-group format, with individual or group interaction with the situation, and group discussion for the debrief.
In experiential learning there is no "right" or "wrong" lesson to be taken from a game or simulation. Everything that happens, not just the ostensible game but also the myriad meta-games that accompany it, is fodder for observation and analysis. Neither is realism a requirement for experiential learning; it is an understood convention of the genre that such games present an abstraction of some "real world" situation that necessarily deviates from it in many respects.
The important part of an experiential learning situation is the debrief. In the debrief, you initially refrain from drawing conclusions about the experiment. The first thing you want from the session is data. A good question to ask is "What happened in this session that stood out for you?"
Because you want to map the game back to the real world, perhaps in unforeseen ways, another thing you want from the session is analogies. A good question to ask is "What did the experiences of this session remind you of?"
For learning to take place there should also be some puzzles arising from either the observations made during the game, or their transposition to real life. For instance, your preexisting mental model - derived from real life interactions - would have led you to different predictions about the game.
The intended outcome of experiential learning is for students (and, sometimes, teacher) to construct an updated mental model that resolves these tensions and can be transposed back to real world situations and applied there. A constructivist approach doesn't expect students to draw exactly the same conclusions as the teacher, even when the teacher makes available the ingredients out of which students build their updated model. Knowledge obtained in that way is more truly a part of you - it sticks better than anything the teacher could have merely told you.
An experiential learning game focusing on the basics of Bayesian reasoning might be a valuable design goal for this community - and a game I'd definitely have an interest in playing. Such is my "hidden agenda" in publishing this post...
Any takers ?
36 comments
Comments sorted by top scores.
comment by AndrewKemendo · 2009-10-23T13:32:19.231Z · LW(p) · GW(p)
A few months ago I stumbled upon a game wherein the goal is to guide an elephant from one side of the screen to a pipe; perhaps you have seen it:
Here's the rub: The rules change on every level. In order to do well you have to be quick to change your view of how the new virtual world works. That takes a flexible mind and accurate interpretation of the cues that the game gives you.
I sent this to some of my colleagues and have concluded anecdotally that their mental flexibility is in rough correlation with their results from the game. I think that experimental games are great and would, if done in a controlled setting, be an interesting way to evaluate mental acuity.
Replies from: PeerInfinity, SilasBarta, Morendil, taw, taryneast↑ comment by PeerInfinity · 2009-10-24T18:27:23.620Z · LW(p) · GW(p)
While I'm at it, here are links to a bunch of other games that require some degree of thinking outside the box and adapting to changing rules:
Factory Balls and Factory Balls 2
Easy. Each level introduces new puzzle pieces, but no dramatic changes in the rules. The solutions are all inside the box, once you figure out what the rules are.
Easy. Requires solving some puzzles without any hints what the puzzle is or what a solution would look like. Solvabe just by trying random things until something happens.
Medium. A series of puzzles, each of which has different rules.
Medium. A series of puzzles, each of which has different rules.
Medium. Each level introduces new puzzle pieces, but no dramatic changes in the rules. The solutions are all inside the box, but you have to figure out how to put them together.
Medium. Each level introduces new puzzle pieces, but no dramatic changes in the rules. The solutions are all inside the box, but you have to figure out how to put them together.
Hard! A series of puzzles, connected by other puzzles, each of which have different rules, and most of which have a counterintuitive solution. Hints about the solutions are cleverly hidden in the game. Hint: Gur cevagfperra ohggba vf lbhe sevraq. (rot13'd)
Hard. A series of puzzles, each of which has a deliberately counterintuitive, and often malevolent, solution. Don't worry if you can't solve all of them, some of the solutions require specific computer hardware or softwae to win.
The Impossible Quiz and The Impossible Quiz 2
Almost Impossible. A series of quiz questions and other challenges that have deliberately counterintuitive solutions. Some of the quiz questions are solvable only by trial and error. Some of the challenges require extremely fast reflexes. Many of the puzzles are blatantly evil. Do not expect to win this. You have been warned.
The rest of these games don't really fit with the theme of thinking outside the box and adapting to changing rules, but are unique enough to include in the list anyway:
Medium. A unique game, but the rules don't change much, except when new puzzle elements are introduced. The solutions are mostly inside the box.
Medium. This game doesn't really belong in this list. The rules don't change much, except when new puzzle elements are introduced. The solutions are mostly inside the box.
Medium. This game doesn't really belong in this list. The rules don't change much, except when new puzzle elements are introduced. The solutions are mostly inside the box.
Shift, Shift 2, Shift 3, and Shift 4
Medium. This game doesn't really belong in this list. The rules don't change much, except when new puzzle elements are introduced. The solutions are mostly inside the box.
Medium. Actually, this doesn't belong in this list at all. The rules don't change, and the solutions are all quite literally inside the box.
There are plenty of other good puzzle games I could link to, but they didn't really fit with the theme of thinking outside the box and adapting to changing rules.
Question: Would it be inappropriate to put this list somewhere on the Less Wrong Wiki?
Replies from: phane, AndrewKemendo, Alicorn↑ comment by phane · 2009-10-25T07:45:10.331Z · LW(p) · GW(p)
These kinds of games just remind me of this Monty Python skit. There's no rules by which to play, so you're just trying to guess what the author is thinking.
↑ comment by AndrewKemendo · 2009-10-25T03:29:49.776Z · LW(p) · GW(p)
Would it be inappropriate to put this list somewhere on the Less Wrong Wiki?
I think that would be great if we had a good repository of mind games
Replies from: PeerInfinity↑ comment by PeerInfinity · 2009-10-25T18:59:52.036Z · LW(p) · GW(p)
Ok, I went ahead and added the list to the wiki: http://wiki.lesswrong.com/wiki/Puzzle_Game_Index
Unless anyone objects, I plan to continue adding more games to this list.
Replies from: ABranco↑ comment by SilasBarta · 2009-10-23T22:52:18.318Z · LW(p) · GW(p)
Okay, I completed it without any help (didn't read the comments). My stats are: 15:03:50, Deaths: 85.
Should I be proud of myself?
ETA: Some of them I didn't even understand how the rules were different, I just manipulated the elephant well enough to get it to the end.
Replies from: AndrewKemendo, Peter_de_Blanc↑ comment by AndrewKemendo · 2009-10-24T05:09:51.755Z · LW(p) · GW(p)
I think a lot of it has to do with your experience with computer based games and web applications.
This is why I say it would have to be a controlled study because those with significant computer experience and gaming experience have a distinct edge on those who do not. For example many gamers would automatically go to the WASD control pattern (which is what some first person shooting games use) on the "alternate control" level.
5:57:18 with 15 deaths here
↑ comment by Peter_de_Blanc · 2009-10-24T00:39:39.334Z · LW(p) · GW(p)
10:59:17.5, 64 deaths.
↑ comment by Morendil · 2009-10-23T19:34:43.950Z · LW(p) · GW(p)
Well, I'm stuck at "Time for a refresh" for the moment. I'll have to sleep on it, I guess.
ETA: my kids took over, and naturally they just breezed right through "Time to refresh", I didn't even have time to notice how. I got my revenge when they got stuck at "Credit page", my favorite of all. We got through the whole thing in 35 minutes.
Quite fun, though not quite the kind of "serious game" I have in mind above.
Replies from: listic↑ comment by listic · 2009-10-23T21:02:09.376Z · LW(p) · GW(p)
I was stuck on "Time for a refresh" too. V gbbx n uvag gung vg'f gvzr gb uvg S5 naq erserfu n cntr. Gung'f jung V qvq, naq gur tnzr fgnegrq sebz gur ortvaavat :-)
Nice game, though I'm not sure how serious it is.
UPDATE: Sorry, didn't know there's a spoiler. Is it ok now?
Replies from: AngryParsley, Morendil↑ comment by AngryParsley · 2009-10-24T20:21:53.007Z · LW(p) · GW(p)
Please rot13 your spoiler.
Edit: Thanks!
↑ comment by taw · 2009-10-23T17:54:18.391Z · LW(p) · GW(p)
It's a brilliant game. Are there any others like it?
Replies from: PeerInfinity, cousin_it↑ comment by PeerInfinity · 2009-10-23T18:58:58.904Z · LW(p) · GW(p)
Yes, here's a similar game starring that same elephant:
Another vaguely similar game:
Replies from: taw, CronoDAS↑ comment by taw · 2009-10-24T01:49:55.318Z · LW(p) · GW(p)
I really dislike "Take Something Literally". First, some puzzles don't work on Mac (caps lock, wheel and a few others), what's mentioned nowhere. And second, it feels more like the author is trying to be a smart ass, than like it's a fun experience.
Replies from: PeerInfinity↑ comment by PeerInfinity · 2009-10-24T17:39:02.689Z · LW(p) · GW(p)
Yes, some of those were just annoying. There were a few that I couldn't solve.
Some of the puzzles have alternate solutions if you don't have a microphone or mousewheel or whatever.
For example:
Ba gur yriry jurer lbh'er fhccbfrq gb oybj vagb gur zvpebcubar, na nygreangr fbyhgvba vf gb glcr "oybj" ba gur xrlobneq.
(rot13'd)
↑ comment by CronoDAS · 2009-10-24T03:43:46.293Z · LW(p) · GW(p)
For the version from the Dark Side:
There is no level of thinking sufficient to solve The Impossible Quiz, because the "correct" answers are ridiculous and arbitrary.
Replies from: PeerInfinity↑ comment by PeerInfinity · 2009-10-24T17:34:52.155Z · LW(p) · GW(p)
And so you have to make a guess, and start over if you guess wrong, remembering what not to guess next time. This may have been what the game designer intended.
I got stuck on the part with the toenails, about halfway through the game. I just couldn't click fast enough.
See also The Impossible Quiz 2
I also got about halfway through this one.
↑ comment by cousin_it · 2009-10-23T18:11:37.166Z · LW(p) · GW(p)
This one is somewhat similar: Karoshi Suicide Salaryman.
comment by James_Miller · 2009-10-23T17:03:52.479Z · LW(p) · GW(p)
Here is a simple N player game that might work.
A Jar has 1000 balls, X of which are Red. X is a randomly chosen number between 0 and 1000. Each of the N players is given a randomly selected number of balls. The total number of balls a given player is given is randomly determined.
Each player sees the colors of the balls he has been given, but not the color of the balls the other players have been given. After being given his balls each player lists a 90% confidence interval for how many red balls he thinks are in the jar. After listing this interval each player is shown the confidence interval of one or more of the other players. Each player then picks a new confidence interval, and then this process repeats.
Replies from: nerzhin, Morendil↑ comment by nerzhin · 2009-10-23T19:11:57.193Z · LW(p) · GW(p)
That game could be played (albeit awkwardly) on a Less Wrong post.
The person running the game puts up a top-level post, writes a little program, and private messages everyone who wants to play with their result. Then each player posts a confidence interval as a comment, and away you go.
comment by patrissimo · 2009-11-07T19:52:15.867Z · LW(p) · GW(p)
The format you describe is at the basis of much standard corporate leadership training. I have participated in and led a number of such sessions, it is a lot of fun and can be very illuminating as well.
For example, trying to accomplish a physical task requiring puzzle-solving and geometry while blindfolded, when the task requires coordination among physically distant parties who must pass verbal messages...it's fun and challenging and has some significant analogies to working w/ people remotely.
I totally agree that experiential learning sticks much better than "updating by hearing". Our ears are not good inputs for updating!
comment by dclayh · 2009-10-23T21:43:27.621Z · LW(p) · GW(p)
An experiential learning game focusing on the basics of Bayesian reasoning might be a valuable design goal for this community - and a game I'd definitely have an interest in playing.
Not particularly Bayesian, but I think a game of LessWrong Nomic would be fun.
Replies from: RobinZ↑ comment by RobinZ · 2009-10-23T23:17:05.965Z · LW(p) · GW(p)
Being one of the maintainers of a Livejournal Nomic myself, I have to agree but step aside. It's a fun, geeky game.
Replies from: Morendilcomment by Tom_Talbot · 2009-10-23T18:39:06.412Z · LW(p) · GW(p)
Squeak/Etoys takes a constructivist approach to teaching children. Is this the kind of thing you're thinking of?
Replies from: Morendil↑ comment by Morendil · 2009-10-23T19:34:10.177Z · LW(p) · GW(p)
I'm thinking more specifically of "parlor game" formats, because that's a situation conducive to a rich exchange of ideas in the debrief. Single-player games are interesting as well, but all I usually get from them is some recombined version of my own thoughts and insights.
There is something uniquely interesting about small-group discussions around transforming ideas: I find that every person in the group has seen things under a slightly different angle, and most of the time one or more participants have thought of something I would never have considered on my own.
An Internet discussion forum can work to some extent, but my favorite games tend to involve some kind of physical tokens or game pieces - possibly because I favor the kinesthetic modality for learning.
comment by dclayh · 2009-10-23T21:42:08.805Z · LW(p) · GW(p)
Reading about it is one thing, and actually playing the game quite another, so I got a group of colleagues together and we gave it a try. ... By the reports of all involved it was one of the most effective learning experiences they'd had.
Any specific results you can share? Were the outcomes similar to Doerner?
Replies from: Morendil↑ comment by Morendil · 2009-10-23T22:26:35.266Z · LW(p) · GW(p)
We had a write-up on our wiki (in French). You can probably get an idea from the graph: there were two teams, Team Red did fairly well - they started out cautiously, observed for a while before making any changes, and ended up in control. Team Blue had more oscillation, they panicked a lot at the start. They had most of the "irrational" remarks.