Rethinking Education

Excuse me if I'm misunderstanding your ideas here, but isn't this, almost bullet for bullet, exactly what Khan academy is doing?

This is the beginning of a very good idea. Happily, many, many highly-competent educational researchers have had it already, and some have pursued it to a fair degree of success, particularly in constrained domain fields (think science, technology, engineering, maths, medicine). It certainly seems to be blooming as a field again these last 5-10 years.

Potentially-useful search terms include: intelligent tutoring systems, AI in Education, educational data mining.

One particularly-nifty system is the Pittsburgh Science of Learning Centre's Datashop, which is a shared, open repository of learner interactions with systems designed to teach along these lines. The mass of data there helps get evidence of what sequence of concepts actual learners find helpful, rather than what sequence teachers think they will.

Given that I'm already involved in the creation of similar system (Docademy.com), I figure I've finally found a thread on here where I have something to contribute :).

On the Dependency Tree

First some thoughts on your concept of a dependency tree. I started out thinking this would be the way to go, but I quickly ran into the problem others have mentioned on this thread: There's no one dependency tree. When looking at concepts like basic math, it's easy to think so, but the concept falls apart when you try to come up with a dependency tree for World History, the dependencies depend on the individual teacher.

The elegant solution to this is to tag individual learning resources (such as a video or chapter) with dependencies. The system will never give you a learning resource that requires B1 to learn B2 if you don't know B1, but will serve up other learning resources that teach B2 WITHOUT needing an understanding of B1. This solves all the problem of your dependency tree solution, but has none of the down sides inherent in the rigidity of it.

On The Standardization of Education

I am also in the camp that the standardization of education is a net negative to society and the individual. My research shows that the most effective individuals are those who specialize in the things that they are good at, rather than trying to be well rounded.

However, that doesn't change the fact that there might be certain skills which would be generally good to have. In that case, once you've solved the problem of being able to measure each skill individually, it's trivial for companies and organizations to simply not accept people who don't have these general skills. This would work similar to how modern Applicant Tracking Systems work, but it would allow you to get much more fine grained with the skills, and would have the benefit of being in a standardized format, rather than trying to parse variously formatted resumes.

After that, the free market would take care of the rest. If a group of skills truly allowed people to be generally more effective, the companies that screened for these skills would outcompete the companies that did not, and eventually they would be generally required for someone who wanted to function in society.

On Standardized Tests

You correctly assume that it's important to be able to quickly identify if someone is good at a subject. However, the type of standardized tests you mention, those that can be easily parsed by a computer, are only appropriate for a small subset of skills. Consider the following examples:

• You correctly use expected value calculations when prompted to do so for a test, but completely fail to do so when making real life decisions.
• You know most of the established theory about how to write good fiction, but your actual stories are boring, uninspired, and not engaging.
• You know how to shoot a basketball, how to dribble, and how to pass perfectly. But when trying to combine these skills in a game, you can't execute.

Ultimately, what these examples show is that for many skills, knowing how to do them in a way that can be easily measured by a computer is different then being able to actually use them in the real world. Rather, a more effective way to quickly measure someone's merit in a particular skill would take a portfolio approach. Under each skill it would list:

1. The learning resources they had used to learn that skill.
2. The standardized tests they had completed in the skill.
3. Real world projects they had completed using the skills.
4. Testimonials from the people they had worked on those projects with, and the people they learned with, talking about theiir abilities in those skils

On Motivation

The final problem you mentioned is motivation. You mentioned two problems in motivation

1 . Kids don't know their prerequisites when trying to understand a subject.

This is fixed with the method above.

2 . Kids don't connect what they're learning with what they can do with it.

If you accept my premise above, that education shouldn't be standardized, I think you've actually got this backwards. If you look into the school models that are more self-directed, such as Waldorf, Montesorri, or Sudbury, you'll see that the students don't go looking for something to learn, then figure out how they can use it. Rather, they choose a goal, a project, or an experiment, then learn the skills they need in order to accomplish or create it.

Therefore this ideal school would have a list of goals, projects, and experiments to work on, which would be tied to skills. Students could choose the goal they wanted to accomplish (or create their own), and the system would suggest learning resources that would teach them the necessary skills.

I'd also like to add some of the other issues I see with motivation that you didn't mention.

3 . The lecture method of learning is low immersion and a poor way to keep students engaged.

To fix this, the main role of teachers would be helping students through completing their actual projects, and the teaching would happen more informally, as part of helping them complete the project. If it was a more in-depth subject for which the student needed a book or lecture, the student could request that, but it would never be FORCED on them.

4 . There's absolutely no incentive for good teachers that motivate students to use their skills; they get paid the same as the bad teachers, and can use the same skills for other jobs which are much higher paying.

If students are free to choose their own projects, one would assume that some of those projects will be profit generating. To solve this problem, you would allow teachers a small stake in every project for which a student used their skill, as a bonus on top of their regular salary. This would incentive them to motivate the students to use the skills they teach.

Anyways, I pretty much live and breath this stuff, and would love to discuss it more with anyone who's interested. If you're seriously interested in pursuing a venture that brings into reality the concepts I've discussed above, and see the implications for things like meritocratic voting, disrupting education, and disrupting hiring, feel free to PM on here and we'll talk.

I think that some competition and social pressure could be useful too; maybe it’d be a good idea to divide students into classes, where the most insightful points are voted upon, and the number of mistakes committed would be tallied and posted.

A lot of being rational is about being able to admit that you made a mistake. Rationality is not about being good at shooting down the mistakes other people make.

My groundbreaking idea: make sure that students know A1…An before teaching them A.

I disagree.

Just getting a personal pet project is frequent advice for people who want to start to learn to program. Working on the project leads you to learn the things that are necessary to solve the problem.

You also should explain how this interacts with what you say about essay writing. To me it seems like you advocate two distinct ideas that sound reasonable on the surface without having thought deeply about how they interact with each other.

moved to Discussion (writing quality insufficient for Main)

I read the first 1/3 of the post and then skimmed the rest, because I think that you yourself are "glossing over important bottlenecks".

As I see it, for you the main problem with education is that students don't learn effectively in schools and you try to treat this problem. However, you should be asking: how did this problem appear? What could be done to solve the real problems? And those are in fact more difficult questions and require much deeper changes than creating a dependency tree and applying it to the students.

Creating and following a dependency tree with pupils implies that you consider 5-, 6- or 7-year-old children who enter the school system tabula rasas, which is not true. What is more, during my pedagogy studies one of the lecturers mentioned that 7-year-old children may be 2 years ahead or behind in development in any area (gross motor, fine motor, memorization, social skills, knowledge, you name it) and these differences are normal. This in turn means that the primary school teacher has to manage and teach not a uniform group of tabula rasa children, but a highly differentiated collective of individuals ranging in mental age from 5 to 9 years.

If you are not starting with a uniform group, you cannot follow a dependency tree, because each child is in a different place of this tree. First you would have to sort the children, so that each group is at some level of your tree. But how would you do that? As I said, sorting by age is one of the worst ideas, it has only one advantage: it's cheap. You don't even have to see the kid, you just find one number about her and decide about her life. So, maybe do some tests? (now let's just not talk about how much it would cost and how much time it would take). Sure, but now you can clearly see that some of those kids can read but not count and some can count but not read, so you should either put them in separate classes or order them to go to different classes for different subjects (the former unrealistic and the latter being very frowned upon in elementary schooling; my country even tries to forbid separation of lessons with bells for 7-9-year-olds).

Suppose that you in fact solved that problem and have a group of children on a very similar level in the area that you are going to teach them. Soon you notice one more phenomenon. The same differences that made some of your pupils start education a year earlier and some a year later cause the kids to learn at a different pace. You could be working with some of the pupils on harder problems, advance your tree faster than the average set by bureaucrats, but other kids need more explanation, more time practicing before they advance. If you comply with the needs of the faster kids, the slower ones will completely stop learning, because they will stop understanding what is going on. If you accomodate the slower kids, the faster ones will get bored and unmotivated to learn anything. How do you solve this problem? My lecturers said that teachers should teach the lesson at such a pace that 3/4 of the class can understand it, effectively stiffling half of the class. Maybe you would divide the class further?

But then, how many times can you divide the class without forcing half of the adult population to work as teachers?

This comment is already longish, but let me point to another problem. When learning anything, a dependency tree is an oversimplification.

From Glossary of language teaching terms and ideas

Active vocabulary: Vocabulary that students actually use in speaking and writing.

Passive vocabulary: Vocabulary that students have heard and can understand, but do not necessarily use when they speak or write.

When learning a language, students commonly have much larger passive vocabularies than active. Some textbooks are designed in a way that is supposed to minimize this difference, but at the same time they offer an extremely slow pace of learning. I observe that this concept of active and passive knowledge applies to any field of study. It's quite intuitive: you first have to learn about something before you can use your knowledge. Following a dependency tree means that you cannot expand your passive vocabulary in a given field of study before you master active vocabulary of the lower level, and this in turn slows down you learning. You can learn about multiplication and division when you have not yet mastered calculating addition and subtraction; you only need to understand the concepts of addition and subtraction, have them in your passive vocabulary.

These are just two sample problem with education, but as a newbie, I only wanted to point that you're missing your own point, not solve the whole mess of institutionalized education.

I work for an educational maths company and we are working on implementing a fine-grained Knowledge Graph as are a number of our competitors.

I can't see a government successfully implementing all of this by itself. Fortunately, intelligent online tutoring software has really taken off because of Khan Academy. Charter schools will be the first to structure their lessons to take advantage of the flexibility this software provides in terms of having students working at their own rates. Eventually, I imagine that educational departments will notice that this works and restructure themselves around this model too, but this will be a slow process.

I propose that we pool all of our resources and make a perfect educational web app.

Bad idea. Putting all resources in a single project is stupid. We want diversity.

It's especially bad because it means a single person can't focus on solving part of the problem in a way that actually pays his bills when all his efforts are focused on contributing to the one perfect educational web app.

A few weeks ago a somewhat similar idea came into my mind while thinking through resources I'd have liked to have and ways to improve education (I think what started me off was the way that many concepts taught in early parts of school turn out to be incorrect simplifications?). I dumped some extremely rough notes into my ideas file (at the end of this post), and mostly concluded that it was an immense project which would require many years of focus to become really useful, and unless it got a lot of momentum would easily stall. On the other hand, this kind of resource could if built properly be amazing. On the other other hand, Khan Academy has many of the elements of the resource I imagined, and building from scratch is very likely more effort than encouraging them to add features or just helping build on the existing project.

Some comments on your suggested implementation: Human knowledge is really really big. Even just the bits taught in schools. Trying to rewrite each part in curriculum form before it becomes useful seems like it would cause a project like this to lose steam quickly. One way around this would be to collect existing high quality educational material online and link to it/include it directly from sources which you have arrangements with, allowing contributors to focus on building the dependency tree. If and when it becomes beneficial, switching to producing content may be better.

If I understand correctly (from the talk of deadlines, tutors, qualifications, social atmosphere, essays, classes), you would like to change the way formal education works? Building an educational resource external to the schooling system seems vastly more realistic than enacting radical change on large institutions without extremely strong evidence of the new methods of teaching working. It could possibly work to start a new school (in the UK there's a lot of attention on "free schools" these days, which, so long as enough parents support an idea, can teach in much less orthodox ways and still get funding), but that's also a massive project, and one which requires plenty of interest from parents otherwise you don't have students.

Concretely, how could you get people to invest the effort required to build this? At minimum, if you're trimming it down to just a web app and dependency tree which primarily links out to existing resources, you need a significant amount of programmer and web dev effort plus a good number of active, reliable people with good understanding of each domain covered building the web of knowledge. Paying the non-technical users to write/collect is not going to work unless you have immense funding, and even then there are many pitfalls (see Knol and Encarta). Not paying users means you've got to have something which is very user friendly, stands out in a big way to important volunteers, has a good base of existing example content, and ideally offers something back to them like StackOverflow does with their jobs program. And even if you have those features, mainly volunteer can flop easily.

And my extremely rough notes from the file with all my ideas which seem interesting but I'll probably never do much with. If someone's interested in any of the lines I can write up my actual thoughts on it, these were mostly to help me remember not actually explain my thoughts:

natural curriculum without stupid simplifications

• nested knowledge
• -starting from extremely basic statements a 5 year old can understand, building toward higher understanding with dependencies/inferential gaps filled.
• crowdsourced, wikilike?
• handling original research
• degrees of acceptance in chunks of information (e.g. almost certain, very likely, useful approximation for x, )
• handling conflicting possible knowledge, conflict warnings,
• integrated QA system
• -ask questions about each chunk of knowledge, these are organized/merged into a FAQ, or answered/redirected
• domain-based reputation system
• -based on useful edits and ratings from related knowledge
• -general showed publicly, higher definition information to those who refine the system and people with higher rating
• -fully recalculated on each change to system, arranged so this is socially okay
• -gamification!
• -general reputation includes typo fixes, organizational stuff, as well as directly building knowledge, but this is only minimally counted towards specific domain knowledge.

These ideas are not exactly new but they aren't exactly trivial either. Why are there so many downvotes then? This community confuses me sometimes in their voting behaviour.

I agree with others that the Big Idea here is less exciting than adamzerner appears to think it is.

It seems plausible that learning prerequisites solidly first is a good technique. But it isn't obviously best and I don't see any evidence here that it is actually best. Another possibility is that actually it's best to be exposed informally to ideas before you have all the background to understand them completely, so that (a) they will be more familiar when you see them "properly" and (b) they will give a bit more context when learning the prerequisites.

I have some (weak) anecdotal evidence that sometimes this learning order is effective.

I wonder about the motivational impact of never being allowed to work on something new until you've demonstrably mastered whatever the creators of your study scheme think are the prerequisites. I would imagine that in some cases it would be very bad, but I've seen no evidence either way.

Metacademy already implements a dependency tree that starts at basic statistics and reaches to academic papers in machine learning.

However, if they don’t exist, students probably won’t have the incentive to learn.

Sudbury Valley student manage to learn without any deadlines. If you don't learn because you are curious and motivated to know something but because there's a deadline, you probably won't learn well.

I think that there are likely some problematic aspects to this idea. One is that the general "I learned X, therefore I can learn Y" seems really strongly applicable to all STEM-types of classes, but I think it would likely falter in English, History, and related subjects to some extent - for example, in history, there's a lot of problems trying to organize how and what to learn that mean making a simple graphlike structure seems problematic. All events have preceding events and resulting events necessary to place them in their proper context. You can't really understand World War 2's causes without understand World War 1 and it's resulting effects, and you can't understand that without understanding.... etc. And while you could start all young children at "billions of years ago, the Earth ..." and progress until you are teaching them recent history at the end of highschool, that will necessarily mean they will understand all the earliest stuff at a very low level of sophistication; and you can't really teach the entire history of the world at varying levels of sophistication each year, or even every 3-4 years. And finally, there's really no way to test "Does this student understand the history of the Opium Wars?" in an automatic way that is resistant to cheating and allows retries without being heavily proctored by actual physical humans standing over watching the testing and writing very carefully and doing the marking of essays, which rather defeats the benefits of a lot of this flexibility stuff for those fields. Also, these fields have a lot of required skills that all should be advancing together - history and politics and economics and so forth inform one another a great deal, for example. I have no idea how I would apply something like this to literature and reading skills, etc. either - the dependencies for these fields are far more fluid and less obvious than "Before you can learn subtraction you must learn addition", and necessarily involve a variety of skills in. e.g. vocabulary, grammar, spelling, composition, and the like all rising in somewhat non-discrete ways. The extent to which learning how to spell "definitely" allows you to write memos is extremely non-obvious, to me at least.

Secondly, people don't necessarily know and understand things perfectly because they passed a test on it some time ago. I personally have a very strong memory, and didn't need to be retaught how to find the roots of a quadratic equation after the first time. However, I was personally taught that particular concept at least 3 times that I can recall in public schools, and even the third time I would say that more of my classmates NEEDED it than not, which is depressing but important to realize. I am not immune to it myself - I only retained permanently "how to invert a matrix and why it matters at all" on the second go-round of that concept, and I saw it at least half a dozen times. You have to try to deal with that sort of thing - perhaps retesting before each new concept? Or some time-based decaying proficiency individually for each student based on their history? Regardless, however you attack this problem it is going to be a problem for this sort of approach that ought to be considered.

However, those are sort of nitpicks, really - there are several more fundamental problems. Overall, I don't really see that this project will significantly advance the field of pedagogy relative to the resources it requires to even begin to attack it. A lot of your categories have no suggestion that I consider reasonable for even beginning to attack the roots of the problem as I see them. For example, your section on writing merely states that we should teach people to be good writers somehow, and your rationality section that we should teach them rationality somehow. I found personally that the majority of my teachers in school were not fundamentally extremely competent writers and powerful orators who were able to pass on their gifts of persuasive composition effectively, and similar limits applied for logic and rationality - there is a fundamental chicken and egg problem where you want a lot of people to learn X but there aren't any teachers who currently even know X and little reason exists to suggest that they would be particularly good at teaching X*. And there are a huge number of students who learn best from a teacher, and whom I suspect would make little real progress if left alone. Independent learning works best, so far as I know, for prodigies who are very self-motivated by long-term goals, which characterization I suspect applies to rather less than 1% of the general student body of a typical modern school - Lesswrong's audience is, I suspect self-selected for the sort of people for whom this idea might not be terrible, but there are a huge number of students who would quite simply cheat if left to their own devices with a system such as you suggest, or do minimal real learning but instead cram for tests immediately before taking the tests and then forget them. You could enforce proctoring of official tests to advance, but that seems like it would quickly become a bureaucratic nightmare and loses a lot of the flexibility and so forth you wanted in the first place.

Anecdotally: I personally went to all of my scheduled my classes through all my undergrad because I already knew by that time that I learned best by going to a class where I could listen to a presentation and ask questions, even when the class material was adapted directly from a book I already had in my possession from the first day of class and I had complete sample problems sets similar to those on the final exams as well as solutions. I rather suspect more students are like me in this regard than not, because the regular attendance rates of those classes was >90% even during extremely nasty weather, and they all had the book and problem sets too.

Link to "Good writing" is 410, deleted now.

There is a saying that education is what remains after you forget almost everything you have been taught. Point is, you sort of described a process without stating what end goal you want to achieve.

I think the whole idea of having a centralized curriculum is flawed and holds education back from developing. Schools should be free to teach whatever they consider to be useful for students.

Diversity is good and if children in different schools learn different useful skills that benefits society as a whole as every student can go on to apply their skills.

If you get rid of the whole idea of a curriculum teachers are suddenly free to innovate.

Given the way the post is written I don't understand why you don't list the social aspect under the section "Where does our system fail us?"

I think the dependency tree should at least in part be derivable from the data in Cyc.