The Machine Learning Personality Test
post by PhilGoetz · 2009-08-04T23:36:37.364Z · LW · GW · Legacy · 34 commentsContents
34 comments
You've probably heard of the Briggs-Myers personality test, which is a classification system of 16 different personality types based on the writings of Carl Jung, a man who believed that his library books sometimes spontaneously exploded. Its main advantage is that it manages to classify people without insulting them. (This is accomplished by confounding dimensions: Instead of measuring one property of personality along one dimension, which leads to some scores being considered better than others, you subtract a measurement along one desirable property of personality from a measurement along another desirable property of personality, and call the result one dimension.)
You've probably also heard of the MMPI, a test designed by giving lots of questions to mental patients and seeing which ones were answered differently by people with particular diagnoses. This is more like personality clustering for fault diagnosis than a personality test. You may find it useful if you're crazy. (One of the criticisms of this test is that religious people often test as psychotic: "Do you sometimes think someone else is directing your actions? Is someone else trying to plan events in your life?" Is that a bug, or a feature?)
You may have heard of the Personality Assessment Inventory, a test devised by listing things that psychotherapists thought were important, and trying to come up with questions to test them.
The Big 5 personality test is constructed in a well-motivated way, using factor analysis to try to discover from the data what the true dimensions of personality are.
But these all work from the top down, looking at human behavior (answers), and trying to uncover latent factors farther down. I'm instead going to propose a personality system that, instead, starts from the very bottom of your hardware and leaves it to you to work your way up to the variables of interest: the Machine Learning Personality Test ("MLPT").
Other personality tests try to measure things that people want to measure, but that might not be psychologically real. The MLPT is just the opposite: It tries to measure things that are probably psychologically real, but are at such a low level that people aren't interested in them. Your mission, should you choose to accept it, is to figure out the connection between the dimensions of the MLPT, and personality traits that you understand and care about.
LW readers are familiar with thinking of people as optimizers. Take that idea, and make 3 assumptions:
- People optimize using something like existing algorithms for machine learning.
- A person learns parameters for their learning algorithms according to the data they are exposed to.
- These parameters generalize across tasks.
Assumption 1 is critical for the MLPT to make any sense. What it does is to classify people according to the parameter settings they use when learning and optimizing. I mostly use parameters from classification / categorization algorithms.
Assumption 2 is important if you wish to change yourself. This is the great advantage of the MLPT: It not only tells you your personality type, but also how to change your personality type. Simply expose yourself to data such that the MLPT type you desire is more effective than yours at learning that data.
Assumption 3 is something I have no evidence for at all, and may be wholly false.
Here are the dimensions I've thought of. Can you name others worth adding?
Learning rate: This is a parameter used to say how much you change your weights in response to new information. I was going to say, "how much you change your beliefs", but that would be misleading; because we're talking about a much finer level of detail. In a neural network model, the learning parameter determines how much you change the weight on a connection between 2 neurons each time you want to change the degree to which one of those neuron's output affects the other neuron's input.
People with a high learning rate learn fast and easily, and may be great at memorizing facts. But when it comes to problems where you have a lot of data and are trying to get extremely high performance, they are not able to get as good an optimum. This suggests that expert violinists or baseball players tend to have poor memory and be categorized as slow learners. (Although I'm skeptical that learning rate on motor tasks would generalize to learning rate on history exams.)
Regularization rate: This parameter says how strongly to bias your outcome back towards your priors. If your regularization rate isn't high enough, the parameters you learn may drift to absurdly large values. In some cases, this will cause the entire network to become unstable, at which point learning ceases and you need to be rebooted.
In most ways, regularization is opposed to learning. Increasing the regularization rate without changing the learning rate effectively decreases the learning rate.
People with a high regularization rate might be less prone to mental illness, but not very creative. People with a low regularization rate will get some of the advantages of a high learning rate, without the disadvantages.
Exploration/Exploitation setting: High exploration means you try out new solutions and new things often. High exploitation means you don't. High exploitation is conceptually a lot like high regularization.
Number of dimensions to classify on: When you're learning how to categorize something, how many dimensions do you use? An astonishing percentage of what we do is based on single-dimension discriminations. Some people use only a single dimension even for important and highly complex discrimination tasks, such as choosing a new president, or deciding on the morality of an action.
Using a small number of dimensions results in a high error rate (where "error", since I'm not assuming category labels exist out in the world, is going to mean your error in predicting outcomes based on your categorizations). Using a large number of dimensions results in slow learning and slow thinking, construction of categories no one else understands, stress when faced with complex situations, and errors from overgeneralizing and from perceiving patterns where there are none, because you don't have enough data to learn whether a distinction in outcome is really due to a difference along one of your dimensions, or just chance.
People using too few dimensions will be, well, stupid. They will be incapable of learning many things no matter how much data they're exposed to. But they can make decisions under pressure and with confidence. They may make good managers. People using too many dimensions will take too long to make decisions, wanting too much data. This dimension may correspond closely to "intelligence", of the kind that scores well on IQ tests.
People using different dimensions and different numbers of dimensions will have a very hard time understanding each other.
It may be worth breaking this separately into number of input dimensions and number of output dimensions. But I kinda doubt it. (I guess I'm just a low-dimensional kinda guy.)
Binary / Discrete / Continuous thinking: Do you categorize your inputs before thinking about them, or try to juggle all their values and do regression in your head? Are you trying to put things in bins, or place them along a continuum?
This probably has the same implications as number of input and output dimensions.
Degree of independence/correlation assumed to exist between dimensions: If the things you are categorizing have measurements along different dimensions that are independent on different dimensions, categorization becomes much easier, and you can handle many more dimensions.
People assuming high independence might make good scientists, as science has so far been the art of finding dimensions in the real world that are independent and using them for analysis. People assuming high correlations might be better at art, and at perceiving holistic patterns. They might tend to give credence towards New-Age notions that everything is interconnected.
Degree of linearity/nonlinearity assumed: Assuming linearity has similar advantages and disadvantages as assuming independence, and assuming nonlinearity has similar effects to assuming correlations. (They are not the same; sometimes the real world presents linearity with correlations, or independence with nonlinearity. I just can't think of anything different to say about them personality-wise.)
I'm going to merge independence/correlation and linearity/nonlinearity, because I don't have anything useful to say to distinguish them. I'm going to merge regularization rate and exploration/exploitation for similar reasons; those two are a lot like each other anyway. I'm going to ignore binary/discrete/continuous, because I didn't think of it until after writing the personality types below and I'm too lazy to redo them. It's a lot like number of dimensions anyway.
Now we need to find cute acronyms for our resulting personality types. For this, we will organize our dimensions so that the first and last dimensions are specified with vowels, and the second and third by consonants. (Changing the fourth letter to a vowel and thus providing catchier names is, I think, the main advantage of this test over the Myers-Briggs.)
- Regularization rate: high = (I)nertial, low = (U)nconventional
- Learning rate: high = (F)ast / (S)low
- Number of dimensions: (M)any / (F)ew
- Independence / linearity: (I)ndependent and linear / h(O)listic and nonlinear
Now you may be eager to take the MLPT and find your results!
Sadly, it does not exist. As I said, I'm just proposing it.
But we can at least write fun, horoscope-like personality summaries! (NOTE: These may not be as accurate as an actual horoscope.)
- IFMI: You like things that appear complex, but can be mastered with a few fundamental rules. You may become an engineer.
- ISMI: Like IFMI, but you were on the chess team instead of "It's Academic".
- IFMO: You should go to med school.
- IFFI: You know what you like, and what others should like. You thought four dimensions was too many. You may vote Republican.
- ISMO: You may be a go master.
- UFMI, USMI: You over-analyze everything, often arriving at unconventional answers, and this makes you a pain in the ass to those around you. You probably read Less Wrong.
- USMO: You are very artistic. You don't believe in personality classification schemes. You may have been to Taos, New Mexico.
34 comments
Comments sorted by top scores.
comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2009-08-05T02:09:52.724Z · LW(p) · GW(p)
You've probably heard of the Briggs-Myers personality test, which is a classification system of 16 different personality types based on the writings of Carl Jung, a man who believed that his library books sometimes spontaneously exploded.
This is the most beautiful ad hominem I've read all week.
Replies from: TimFreeman↑ comment by TimFreeman · 2011-05-25T20:29:51.849Z · LW(p) · GW(p)
Carl Jung, a man who believed that his library books sometimes spontaneously exploded.
This is the most beautiful ad hominem I've read all week.
I agree, but from a different week. :-).
I tried looking up the source for this. According to http://soultherapynow.com/articles/carl-jung.html, it's pages 155-156 of:
Jung, C. G. (1989). Memories, Dreams, Reflections (Rev. ed., C. Winston & R. Winston, Trans.) (A. Jaffe, Ed.). New York: Random House, Inc. (Original work published 1963)
This is Jung's autobiography, edited by Jaffe, so we only have Jung's word on it. The only other witness was Freud. I did not find indication that Freud reported the event.
You can persuade Google Books to show you the relevant page of the book. Type in the title, then search in the book for "bookcase". There's another mysterious crack on page 104-105. Search for "walnut".
Replies from: hairyfigment↑ comment by hairyfigment · 2011-05-25T20:57:05.983Z · LW(p) · GW(p)
Unless we can view this part of MDR in Jung's handwriting, we do not have Jung's word for it.
Replies from: TimFreeman↑ comment by TimFreeman · 2011-05-26T12:35:18.062Z · LW(p) · GW(p)
Agreed. I had assumed that Jaffe's role as editor was similar to Random House's role as publisher in the sense that neither would be expected to change the facts of the story, but that does not appear to be the case.
So maybe Jung didn't believe that his library books sometimes exploded and perhaps the original ad hominem didn't apply. However, it's also possible that Jung's other statements went through the same corruption process that the exploding books story did, so the ad hominem does cast doubt on the truth of other things he is claimed to have said, even if Jung didn't really believe in exploding books.
I'm not much interested in tracking down whether other statements attributed to Jung were manipulated by a third party.
comment by Z_M_Davis · 2009-08-05T02:14:35.844Z · LW(p) · GW(p)
As far as I know, there are no personality tests constructed in the proper way, which would be to give a lot of questions and then perform either factor analysis or PCA on the answers in order to discover from the data what the true dimensions of personality are.
But I thought that was exactly how we got the Big Five.
Replies from: PhilGoetzcomment by gjm · 2009-08-05T12:05:52.108Z · LW(p) · GW(p)
I have no idea whether this idea has any actual merit, but it made me smile.
See also: the Lipson-Shiu Corporate Type Test, whose axes are Intelligent-Stupid, Lawful-Chaotic, Important-Unimportant, Good-Evil.
comment by Drahflow · 2009-08-09T23:14:17.784Z · LW(p) · GW(p)
Two additional Dimensions:
Discounting Rate/Function - how much the effects of your actions after ten years affect your choice.
Sliding Window Size/Function - how fast you discard old data, after all the world might have changed
Replies from: PhilGoetzcomment by RobinZ · 2009-08-05T14:52:25.455Z · LW(p) · GW(p)
I'm sorry, but how is this based in machine learning? I don't know a lot about AI, but I don't see any references to the research literature of the field here. The bottom-up approach you endorse sounds interesting, but I don't see any bottom here.
Replies from: PhilGoetzcomment by knb · 2009-08-05T05:29:53.328Z · LW(p) · GW(p)
(One of the criticisms of this test is that religious people often test as psychotic: "Do you sometimes think someone else is directing your actions? Is someone else trying to plan events in your life?" Is that a bug, or a feature?)
One of my neuroscience teachers made us read studies showing that very religious people have near-schizophrenic dopamine levels. He also pointed out that schizophrenics have delusions of grandeur/feelings of persecution that are reminiscent of certain religious beliefs.
Then again, he also made us read "The God Delusion", so he may just cherry-picked the studies because he had an ideological axe to grind.
comment by Jonathan_Graehl · 2009-08-05T00:29:01.403Z · LW(p) · GW(p)
Hey, what a coincidence - LW readers generally fall into the coolest classifications!
This is fantastically clever and amusing. I notice you didn't even try to give a phony silver-lining pat on the back for low-dimensioned learners, but otherwise the non-judging spirit of Briggs-Myers is well emulated.
Replies from: PhilGoetzcomment by [deleted] · 2009-08-05T03:46:43.195Z · LW(p) · GW(p)
This is fantastic. And I certainly agree that personality tests always put being complementary over being accurate.
Replies from: CronoDAS↑ comment by CronoDAS · 2009-08-05T04:39:30.063Z · LW(p) · GW(p)
I just found the Asshole Rating Self-Exam (ARSE), which, although not scientific, is amusing.
comment by [deleted] · 2015-07-09T02:46:52.607Z · LW(p) · GW(p)
This is brilliant! Oh my gosh!
Can someone programming literature hack this up into an actual personality test!
I'd love to give it a go!
It would be even better if there was a feature that describes how your cognitive algorithm fares against different kinds of machine learning problems!
edit: perhaps market it as a 'problem solving style' test?
comment by RHollerith (rhollerith_dot_com) · 2009-08-05T12:22:32.029Z · LW(p) · GW(p)
An intriguing, fun post by Phil Goetz!
The personality trait I am most eager to understand is perfectionism.
Perfectionism is sometimes referred to as obsessive-compulsiveness (but those words are not as apt because they get conflated with "addictive personality," which has no valid connection to perfectionism).
If anyone reading this knows how to define perfectionism using concepts from machine learning or brain science and can teach me the definition, I will kiss your feet and treat you like a king or queen.
The main reason I think it would be a huge win to understand perfectionism better are that people often comment on it and that it almost completely defeats my attempts to attenuate it.
I have found that I can amplify or attenuate most of my psychological dimensions when I set out on a persistent campaign to do so. Here "persistent" means lasting for years. Examples of traits I have been able to attenuate greatly include fearfulness and dysthymia (tendency to be sad for most of the time for days on end). But my perfectionism has proven almost completely unchangable. And the most unchangeable aspects of my mind strike me as the aspects most worth taking the trouble to understand.
P.S. there seems to be some stigmatization of perfectionism. It seems to me that some nonperfectionists do not want to understand perfectionists, they just want to eliminate them from their interpersonal environment.
Replies from: HughRistik, conchis↑ comment by HughRistik · 2009-08-05T19:51:50.743Z · LW(p) · GW(p)
"Perfectionism" is probably related to Conscientiousness from the Big Five. Perhaps it's also related to Neuroticism. My hypothesis is that Conscientiousness causes you to have a high bar for your performance, and Neuroticism causes you anxiety when you aren't meeting that high bar.
Going along with Neuroticism, perfectionism may also be related to self-esteem or a need for agency in a particular area. For instance, during my insanely perfectionistic days, I would try to solve any word or logic problem that I ever came across, even if it wasn't a good use of my time and energy. I think the motivation was that I needed to prove something to myself.
Update: Wikipedia's article on Perfectionism) concurs that it may be related to Conscientiousness and Neuroticism.
This suggests that the main ways to deal with perfectionism are:
- a) lower your bar for success (e.g. don't set it higher than the task actually demands, or higher than is practical given your allotted time for the task): "OK, this is good enough... time to stop..."
and
- (b) better cope with your anxiety when you fail to reach whatever bar you are measuring yourself by: "I'd like to put a bit more work into this, but no sense in beating myself up..."
I've used both strategies successfully in my own life.
↑ comment by conchis · 2009-08-05T12:38:09.167Z · LW(p) · GW(p)
How about "a tendency to optimize even when the costs of optimization (may) outweigh the benefits"?
Replies from: PhilGoetz↑ comment by PhilGoetz · 2009-08-05T14:05:16.846Z · LW(p) · GW(p)
Maybe "stopping criterion".
Replies from: HughRistik↑ comment by HughRistik · 2009-08-05T20:45:27.782Z · LW(p) · GW(p)
Or perhaps "satisfaction criterion." Perfectionists would then have a high satisfaction criterion for any task, some higher than is demanded by the task or actually practical.
Lowering the satisfaction criterion even a little can put completion of an impossible task into the realm of attainability.
Another heuristic is to start a task with a low satisfaction criterion so you can at least get something out, and then come back the next day with a high satisfaction criterion and refine what you started yesterday. This strategy of starting with a low satisfaction criterion that you raise as you get more done works well with writing and programming (e.g. prototyping).
Of course, the drawback of lowering one's satisfaction criterion in one area is that once you get used to that way of thinking, it is easier to start letting things slide in general.
For example, when I was learning to socialize, I had to lower my satisfaction criterion for what constituted an acceptable utterance in a conversation, and what amount of thinking it required. Due to the speed of socializing, I could not use the high-deliberation, high-precision, and mistake-minimizing strategies that I used to excel at other tasks such as chess and school, because they got me stuck in analysis paralysis while the conversation whizzed by.
Yet once I learned to lower my satisfaction criterion, I found myself cutting corners in a lot of other areas, like schoolwork. So I had to learn to switch between my perfectionistic cognitive style, and my more improvisational cognitive style. I think the ideal is to appraise the satisfaction criterion necessary for any stage of any specific task.
Based on this experience, I would suggest that anyone attempting to lower their perfectionism attempt a task that cannot be effectively solved by perfectionistic cognitive strategies, particularly time-constrained or real-time tasks such as socializing, musical improvisation, dance, blitz chess, or doing a painting all in one sitting.
A perfectionist will want to excel at any task they attempt, but to succeed, they will have to cognitively inhibit or disable their own perfectionism: being non-perfectionistic is the only way to approach perfection. Thus, perfectionism implodes.
Replies from: PhilGoetz↑ comment by PhilGoetz · 2009-08-06T00:24:55.638Z · LW(p) · GW(p)
"Stopping criterion" is already the standard terminology for how your machine learning algorithm decides when to stop.
Replies from: HughRistik↑ comment by HughRistik · 2009-08-06T05:46:15.959Z · LW(p) · GW(p)
Thanks for clarifying.
comment by PhilGoetz · 2009-08-05T03:48:41.634Z · LW(p) · GW(p)
By coincidence, just after writing this, I read an article ("Why are modern scientists so dull?") claiming that creativity and psychoticism are strongly positively correlated. The explanation offered was different: that psychoticism (by which they seem to mean being a sociopath, not being mentally unstable) enables one to be more independent.
Replies from: HughRistik↑ comment by HughRistik · 2009-08-05T19:32:04.856Z · LW(p) · GW(p)
Link?
In this context, Psychoticism is probably the trait from Eysenck's 3 factor model of personality.
Psychoticism corresponds to a combination of Agreeableness and Conscientiousness from the Big Five.
Here's a web test for Eysenck's personality inventory. My results:
- Extroversion (sociability) |||||||||||||| 60%
- Neuroticism (emotionality) |||||||||| 36%
- Psychoticism (rebelliousness) |||||||||||| 45%
↑ comment by PhilGoetz · 2009-08-06T15:41:09.298Z · LW(p) · GW(p)
Replies from: ciphergothLink?
↑ comment by Paul Crowley (ciphergoth) · 2010-03-18T12:55:55.061Z · LW(p) · GW(p)
If you have the link right in front of you or if it's easy to find, it's better to save your audience's time by providing it right away.
comment by taw · 2009-08-05T01:13:37.495Z · LW(p) · GW(p)
As far as I know, there are no personality tests constructed in the proper way, which would be to give a lot of questions and then perform either factor analysis or PCA on the answers in order to discover from the data what the true dimensions of personality are.
PCA as used in psychology basically tells you that you're asking the same question in a subtly different way, it has nothing to do with this being important. If you include many variants of the same question in your test, you will get exactly what you wanted in results of PCA.
Replies from: PhilGoetz