Recent updates to gwern.net (2012-2013)
post by gwern · 2013-03-18T19:54:30.809Z · LW · GW · Legacy · 34 commentsContents
34 comments
Previous: Recent updates to gwern.net (2011)
“But where shall wisdom be found? / And where is the place of understanding? / Man knoweth not the price thereof; neither is it found in the land of the living…for the price of wisdom is above rubies.”
As before, here is material I’ve worked on in the 477 days since my last update which LWers may find interesting. In roughly chronological & topical order, here are the major additions to gwern.net
:
- I interviewed translator Michael House about his work in Japan as a translator
- finished data collection for my hafu anime statistics page and begun analysis. (I’ve achieved good coverage of characters, found an astonishingly consistent absence of Korean characters, and confirmed the blond-haired/blue-eyed stereotype; but my original thesis doesn’t seem to work and the data is too unevenly distributed to identify time trends.)
- judged the 2011 & 2012 results for the Haskell Summer of Codes and the accuracy of my predictions
- did a meta-analysis on whether dual n-back increases IQ, and examining possible biases and various claims about what makes the training work or not work
- did another meta-analysis on whether iodine increases IQ, etc
-
modafinil:
- checked for subjective effects of blinded modafinil
- updated my modafinil price-chart twice, and expanded with brand data and a new armodafinil table
- researched modafinil-related prosecutions & convictions in the USA
- and any connection with schizophrenia
- tried kratom
- did a nicotine gum/n-back experiment
- did 2 potassium experiments; neither improved my mood/productivity, and one damaged my sleep
- my Silk Road page has been expanded with a BBC interview, putting SR in a historical cypherpunk context, an updated account of all arrests & law enforcement actions, and application of basic statistics to ordering
- ran 2 sleep experiments on the timing of taking a vitamin D supplement: I found that taking vitamin D before bed substantially damaged my sleep, while taking vitamin D after waking up did not hurt & somewhat helped
- checked whether a walking desk (treadmill) damaged typing speed or accuracy
- I have run 3 Wikipedia experiments establishing that: Talk page edits are ignored by editors; random link deletions (and their restoration) are also ignored by editors; and external link suggestions on Talk pages are also ignored by readers. (I take the former 2 as indicative of the decline in edit activity and rise of deletionist beliefs on Wikipedia.)
- tried some economic/historical analysis: “Reasons of State: Why Didn’t Denmark Sell Greenland to the USA?”
- Defending sunk costs essay (LW discussion)
- “Slowing Moore’s Law: Why You Might Want To and How You Would Do It”
- “The Hyperbolic Time Chamber as Brain Emulation Analogy”
- tried estimating the bandwidth of a Death Note
- compiled predictions for Harry Potter and the Methods of Rationality
- looked into Conscientiousness and online education; studies so far are useless from a meta-analytic standpoint
- tripled length of appendix dealing with the reliability of mainstream science (methodological flaws, replication rates, etc)
- finished meta-ethics essay, “The Narrowing Circle”
- explained the philosophy saying “one man’s modus ponens is another man’s modus tollens”
- speculation about a restoration of the British monarchy
- clean up & exploratory data analysis of SDr’s lucid dreaming data
- Who wrote the Death Note script? (LW discussion)
- 2012 US election predictions: statistical comparison
- Turing-completeness in surprising places (inventory of particularly “weird machines”; relevant to computer and AI security)
Transcribed or translated:
- Nash’s letters on cryptography
- Douglas Hofstadter’s superrationality columns (from Metamagical Themas, 1985)
- “The Iron Law Of Evaluation And Other Metallic Rules”, Rossi 1987 (lessons from the large RCTs evaluating social & welfare interventions)
- “The Ups and Downs of the Hope Function In a Fruitless Search”, Falk et al 1994
- Gene Wolfe on writing
- “Shiny balls of Mud: William Gibson Looks at Japanese Pursuits of Perfection” (2002)
- “Otaku Talk”, Okada et al 2004
- “Earth in My Window”, Murakami 2005
- “On The Battlefield of ‘Superflat’”
- “Ero-Anime: Manga Comes Alive”, Sarrazin 2010
- 1996 NewType interview with Hideaki Anno (translated by me, with the help of an EGFer)
- 1997 Animeland interview with Hideaki Anno (bought, transcribed, and translated by me with the help of other LWers)
- 1997 Utena interviews
More technical:
- added edit history statistics/visualization for
gwern.net
using GitStats - site traffic updates: July-December 2011, January 2012-July 2012, July 2012-Jan 2013
- There’s also been a lot of backend changes: switching to Amazon S3+Cloudflare, adding error pages, metadata like tags, A/B testing, but no need to go into detail.
Personal:
- dumped my notes on my 2011 visit to San Francisco
- posted summaries of my personality & attitudes & my RSS feed collection
- enjoyed some mead; I still like tea better, though
- dumped notes on the 2012 SF convention ICON
34 comments
Comments sorted by top scores.
comment by Barry_Cotter · 2013-03-18T04:47:54.137Z · LW(p) · GW(p)
Man, you are a machine. I go days without writing in a diary that lives in my pocket, you write well researched essays and do research. How many days do you not write something?
Replies from: gwern, Kawoomba↑ comment by gwern · 2013-03-18T15:59:02.570Z · LW(p) · GW(p)
Well, at least 2 of those were for SI, so those would count as a job or work; one has to eat, after all. (Speaking of which, if anyone else wants to hire me for stuff, feel free to contact me.)
But besides that, I think after a while writing/research can be a virtuous circle or autocatalytic. If you look at my repo statistics linked above, you see that I haven't always been writing as much. What seems to happen is that as I write more:
I learn more tools
eg. I learned meta-analysis to answer the burning question of what all the positive & negative n-back studies sum to, but then I was able to use it for iodine; I learned linear models for analyzing MoR reviews but now I can use them in my Touhou material
I internalize a habit of noticing interesting questions that flit across my brain
eg. 2 weeks ago while meditating: 'does more doujin stuff get released when unemployment goes up? Hey! My giant Touhou download could probably answer that!' (One could argue that these questions should probably be ignored and not investigated in depth - to paraphrase Teller, often magic is simply putting in more effort than any sane person would - but nevertheless, this is how things work for me.)
if you aren't writing, you'll ignore useful links or quotes; but if you stick them in small asides or footnotes as you notice them, eventually you'll have something bigger.
I grab things I see on Google Alerts & Scholar, Pubmed, Reddit, Hacker News, my RSS feeds, books I read, and note them somewhere until they finally amount to something. (An on-LW example would be my slowly accreting citations on IQ and economics.)
people leave comments, ping me on IRC, send me emails, or leave anonymous messages, all of which can help
The most recent examples of this come from my most popular page, on Silk Road:
- an anonymous message led me to investigate a vendor in depth and ponder the accusation leveled against them; in a month or two I'll write it up and give my opinions and I'll have another short essay to add to my SR page which I would not have had otherwise, and I think there's a <20% chance that in a few years this will pay off and become a very interesting essay.
- CMU's Christin, who wrote a paper by scraping SR for many months and giving all sorts of overall statistics, emailed me to point out I was citing inaccurate figures from the first version of his paper. I thanked him for the correction and while I was replying, mentioned I had a hard time believing his paper's claims about the extreme rarity of scams on SR as estimated through buyer feedback. After some back and forth and suggesting specific mechanisms how the estimates could be positively biased, he was able to check his database and confirmed that there was at least one large hole in the scraped data and there was probably a general undersampling of scams; so now I have a more accurate feedback estimate for my SR page (important for estimating risk of ordering) and he said he'll acknowledge me in the/a paper, which is nice.
↑ comment by henryaj · 2013-03-19T13:55:12.144Z · LW(p) · GW(p)
Nthing the "you are an inspiration" sentiment expressed here. This has reminded me that you should 'always be shipping'; always be doing stuff and making stuff.
What's your working environment like? You mention doing work for SI; is that at their offices or at home? I've been flirting with the idea of working part-time to pursue other projects in my spare time but I'm not sure I could hack being in a home office all day.
(And, if you don't mind me asking, how do you bankroll all this? Do you have a 'day job' per se?)
Replies from: gwern, henryaj↑ comment by bentarm · 2013-03-19T18:46:52.229Z · LW(p) · GW(p)
(One could argue that these questions should probably be ignored and not investigated in depth - to paraphrase Teller, often magic is simply putting in more effort than any sane person would - but nevertheless, this is how things work for me.)
I can't find a source for this quote (and if it's from a longer interview, I think I'd probably like to read it), possibly because I'm not picking the right words to Google. Do you have a citation?
Replies from: TheOtherDave, ESRogs, arundelo↑ comment by TheOtherDave · 2013-03-19T20:36:22.602Z · LW(p) · GW(p)
Googling "teller magic sane" turned up this interview including the quote:
Replies from: bentarmYou will be fooled by a trick if it involves more time, money and practice than you (or any other sane onlooker) would be willing to invest.
↑ comment by ESRogs · 2013-03-23T16:36:44.588Z · LW(p) · GW(p)
"Sometimes magic is just someone spending more time on something than anyone else might reasonably expect," Teller says.
http://www.esquire.com/print-this/teller-magician-interview-1012?page=all
↑ comment by Kawoomba · 2013-03-18T12:12:25.737Z · LW(p) · GW(p)
This. I'm still trying new ways to motivate me into writing a daily diary, but those ways are as yet resistant.
Replies from: Barry_Cotter↑ comment by Barry_Cotter · 2013-03-18T12:43:29.436Z · LW(p) · GW(p)
I highly recommend just keeping a diary of the correct size with an appropriately sized pen with you at all times. A Moleskine with a pen in it has served me well not that they're tools handed down from Heaven or anything. Whether they're emotioins, thoughts or passing fancies, writing has helped me. More emotional shit than anything else. I don't (re)read it much but just writing is an excellent way of getting self-awareness when it comes to emotions. Saying them would likley work near as well but writing is a great deal easier.
comment by betterthanwell · 2013-03-18T18:04:58.984Z · LW(p) · GW(p)
With some awe and much respect, I would say that you are an inspiration, but that has already been said. I'll upvote that and say something else instead. For whatever reason, some part of my brain tells me; "Yeah, this is pretty much what I would expect the research interests of of a "supervillain"-in-training to look like". I don't pretend to know exactly what awesomeness is, but you have grown a lot of it.
comment by Fadeway · 2013-03-18T17:13:23.111Z · LW(p) · GW(p)
I've read a significant amount of your essays/articles and love the stuff. It's kinda hard to track for new stuff since the RSS feed tends to dump dozens of small changes all at once, so this post is much appreciated.
Replies from: Tenoke, gwern↑ comment by Tenoke · 2013-03-19T14:25:28.496Z · LW(p) · GW(p)
It's kinda hard to track for new stuff since the RSS feed tends to dump dozens of small changes all at once, so this post is much appreciated.
A better (maybe separate?) RSS feed that doesn't do that will be a huge improvement for my experience of the site as well.
Replies from: gwern↑ comment by gwern · 2013-03-19T16:19:51.389Z · LW(p) · GW(p)
I don't know how to do that. There's no automated way to tell when I've added something important and when I just made a lot of minor changes like adding additional citations or formatting tweaks. When something is 'finished' is knowable only in retrospect.
Replies from: TenokeArrakis teaches the attitude of the knife — chopping off what's incomplete and saying: "Now it's complete because it's ended here."
↑ comment by Tenoke · 2013-03-19T17:17:43.045Z · LW(p) · GW(p)
That's fair but broken link fixes and css fiddling are not important enough to be on the RSS feed in almost all cases on both 'finished' and 'unfinished' pages. Out of curiosity, do you have many feeds in your reader that push everything there like you do?
Replies from: gwerncomment by Mestroyer · 2013-03-19T12:30:35.323Z · LW(p) · GW(p)
Your essay "The 3 Grenades" is wrong (as a few commenters have pointed out). Precluding self-referential questions like Boxo comes up with, 4 questions are not enough when the person doesn't have to lie. However, if out of 4 questions, they are forced to lie exactly once, it can be done in 4 questions. If they have to lie exactly once, you can actually pick from among 4 colors in 4 questions. I can prove all of this, and will write it up when I get a chance if you are interested.
Replies from: gwern↑ comment by gwern · 2013-03-19T16:17:18.332Z · LW(p) · GW(p)
Feel free. I don't care about it any more, and as you point out, the commenters correct it.
Replies from: Mestroyer↑ comment by Mestroyer · 2013-03-21T13:49:45.976Z · LW(p) · GW(p)
There's another assumption I didn't realize I was making: you don't change future questions based on what answers you get. But here goes: If every question is dependent only on the colors of the grenades, and you have 4 questions, then if you draw out a grid like this:
Then each square represents a set of four answers, one for each of questions Q1, Q2, Q3, and Q4. The column decides the answers to Q1 and Q2. The row decides the aswers to Q3 and Q4. I'm using "1" for "yes" and "2" for "no." The columns and rows are numberd in grey code, which means that moving one square in any direction is equivalent to flipping one Y/N answer. Because there are 4 answers to flip, and 4 directions to move in (you can move left on the leftmost square to end up on the rightmost, same with top and bottom. Like the "Asteroids" video game), every answer-flip is represented by a move of one square in a particular direction.
So for each possible grenade color, answering the questions truthfully would specify a square in the grid. Answering one incorrectly will move the set of answers one square in one direction. So if when the grenade is really red, the true answers are 0101 (No to Q1, yes to Q2, no to Q3, yes to Q4), the sets of answers the person can give are the ones filled in below:
The light red one is if they don't lie, and the dark red ones are if they ile once. To make it so we can always determine the ball color, we have to arrange 3 of these shapes on the grid so that they don't overlap. If they do overlap, then they could have answered that way if the ball was one of two or more different colors.
Because this grid wraps like the asteroids game, it doesn't matter where you put the first "+". Mentally placing a second plus makes it obvious that there is nowhere to put the last , no matter where you put it.
But, if they have to lie, then the answers they can give look like this:
And you can fit 4 of this shape on the grid:
So to determine the grenade color if 4 colors are possible in 4 questions, if in 4 questions they must lie, just pick the questions so that if the grenade is red, the correct answers are 0101, if blue, 1101, if green, 1010, if pink, 0010. This means Q1 should be "Is the ball blue or green," Q2 should be "Is it red or blue," Q3: "Is it green or pink", Q4: "is it red or blue" (Yes, the last three are basically the same.)
When you get your answers, look up a square in the grid, and whatever color it is, is the color of the ball.
Replies from: Decius↑ comment by Decius · 2013-03-30T22:19:06.679Z · LW(p) · GW(p)
Why wouldn't you change your questions based on the responses you have already gotten? Also, you are assuming that the target has at most one grenade, an assumption which I think is valid.
I agree that asking logically equivalent questions or self-referential questions breaks the spirit of the rules, but asking questions which are subsets of previous questions is not.
Given that case 1 is not having a grenade, and case 2-4 are having grenades of each color, the questions could go like this:
Case 1 or 2? Case 3 or 4? Case 2 or 3?
000: Case 1? (yes 1, no 4)
001: Case 2? (yes 2, no 3)
010: Case 1,2,3? (yes 3, no 4)
011: Case 1,2,3? (yes 3, no 4)
100: Case 2,3,4? (yes 2, no 1)
101: Case 2,3,4? (yes 2, no 1)
110:Case 1? (yes 1, no 4)
111:Case 2? (yes 2, no 3)
This line uses what I consider a cheap trick of asking the same question twice in a row, because once "3 or 4" is known, "2 or 3?" and "1,2, or 3?" both simplify to "3?"
Replies from: Mestroyer↑ comment by Mestroyer · 2013-03-31T01:13:37.663Z · LW(p) · GW(p)
I agree that the assumption that you wouldn't change your questions based on the responses is not a reasonable one, but I realized that I had made it after I came up with my argument, and decided to share it anyway in case people found it interesting.
If later questions are allowed to reference earlier answers by the interrogee, you could get all the benefit of changing your questions by multiplexing among possible later questions based on their answers to earlier questions, all within one question. For example: if your first question was "is the ball red", and if they say yes your next is "is the ball blue", and if they say no you say "is the ball green", your second question would always be be "Is it true that you just said 'yes' and the ball is blue or that you just said 'no' and the ball is green?"
You can also deal with the ruleset where they could have a grenade of one of three colors, or they could have nothing with my method of dealing with 4 possible colors, by replacing all references to "pink" with "no grenade".
comment by MichaelHoward · 2013-03-18T15:53:19.005Z · LW(p) · GW(p)
For the overwhelmed, here's a summary snippet to encourage further investigation... (in rot13 for those who'd consider it spoilers, or just think Down With This Sort Of Thing).
From the Dual N-Back FAQ:
Gb gubfr jubfr gvzr vf yvzvgrq: lbh znl jvfu gb fgbc ernqvat urer. Vs lbh frrx gb vzcebir lbhe yvsr, naq jnag gur terngrfg "onat sbe gur ohpx", lbh ner jryy-nqivfrq gb ybbx ryfrjurer.
Zrqvgngvba, sbe rknzcyr, vf rnfvre, snfgre, naq hygen-cbegnoyr. Glcvat genvavat jvyy qverpgyl vzcebir lbhe snpvyvgl jvgu n pbzchgre, n inyhnoyr fxvyy sbe guvf zbqrea jbeyq. Fcnprq ercrgvgvba zrzbevmngvba grpuavdhrf bssre hacnenyyryrq nqinagntrf gb fghqragf. Abbgebcvpf ner gur rcvgbzr bs rnfr (whfg fjnyybj!), naq gurve rssrpgf ner zhpu zber rnfvyl nffrffrq - bar pna rira eha qbhoyr-oyvaq rkcrevzragf ba barfrys, vzcbffvoyr jvgu qhny A-onpx. Bgure fhccyrzragf yvxr zryngbava pna qryvire orarsvgf vapbzzrafhenoyr jvgu QAO - jung vf gur pbtavgvir inyhr bs nabgure ahzore va jbexvat zrzbel gunaxf gb QAO pbzcnerq gb n tbbq avtug’f fyrrc gunaxf gb zryngbava? Zbqrfg punatrf gb bar’f qvrg naq raivebaf pna shaqnzragnyyl vzcebir bar’f jryy-orvat. Rira onfvp genvavat va ernqvat, jvgu gur pehqrfg gnpuvfgbfpbcr grpuavdhrf, pna cnl ynetr qvivqraqf vs bar vf orybj n onfvp yriry bs ernqvat yvxr 200JCZ & fgvyy fhoibpnyvmvat. Naq nyy bs gurfr pna fgneg cnlvat bss vzzrqvngryl.
...and his favorite nootropics are:
Replies from: Fadeway
- Zbqnsvavy/nezbqnsvavy
- Zryngbava
- Pnssrvar+gurnavar
- Cvenprgnz+pubyvar
- Ivgnzva Q
- Fhyohgvnzvar
- Svfu bvy
↑ comment by Fadeway · 2013-03-18T17:10:45.588Z · LW(p) · GW(p)
Is it useful to increase reading speed, even if it takes a minimal amount of time (to go from basic level to some rudimentary form of training)? I've always been under the impression that speed increases in reading are paid for with a comprehension decrease - which is what we actually care about. Or is this only true for the upper speed levels?
Replies from: moridinamael, gwern↑ comment by moridinamael · 2013-03-21T18:16:54.470Z · LW(p) · GW(p)
In the form of speed-reading in which I was trained, you write a one-sentence summary of each paragraph as you're reading, and after you read a chapter or section, you review each of your one-sentence summaries. In theory this allows you to "process" things like textbooks into knowledge stored in your brain very quickly. In practice, speed reading only works for me if the material doesn't contain any concepts that I don't already understand.
I find it very useful when I need to get the gist of a paper to decide whether I want to actually read it in detail.
Replies from: 9eB1↑ comment by gwern · 2013-03-18T17:49:37.859Z · LW(p) · GW(p)
I think it is. As I mention in my footnote, it's been a long time since I was reading up on the topic and I don't have any notes, but I recall the gist being that it's at the upper levels that you forfeit comprehension and that for lower speeds like <400wpm on nontechnical material you may even get better comprehension.
Replies from: TrE↑ comment by TrE · 2013-03-21T07:48:32.248Z · LW(p) · GW(p)
In summary, it's useful to be capable of different reading speeds to adapt to the current task? Skimming (or Ctrl-F) for searching for useful text passages, fast reading for technical material one is familiar with, slow reading and re-reading for technical material one is not familiar with, slow dabbling in texts one reads for recreational purposes.
comment by MichaelHoward · 2013-03-18T15:51:13.730Z · LW(p) · GW(p)
For the overwhelmed, here's a summary snippet from the Dual N-Back FAQ to encourage further investigation... (in rot13 for those who'd consider it spoilers, or just think Down With This Sort Of Thing).
Gb gubfr jubfr gvzr vf yvzvgrq: lbh znl jvfu gb fgbc ernqvat urer. Vs lbh frrx gb vzcebir lbhe yvsr, naq jnag gur terngrfg "onat sbe gur ohpx", lbh ner jryy-nqivfrq gb ybbx ryfrjurer.
Zrqvgngvba, sbe rknzcyr, vf rnfvre, snfgre, naq hygen-cbegnoyr. Glcvat genvavat jvyy qverpgyl vzcebir lbhe snpvyvgl jvgu n pbzchgre, n inyhnoyr fxvyy sbe guvf zbqrea jbeyq. Fcnprq ercrgvgvba zrzbevmngvba grpuavdhrf bssre hacnenyyryrq nqinagntrf gb fghqragf. Abbgebcvpf ner gur rcvgbzr bs rnfr (whfg fjnyybj!), naq gurve rssrpgf ner zhpu zber rnfvyl nffrffrq - bar pna rira eha qbhoyr-oyvaq rkcrevzragf ba barfrys, vzcbffvoyr jvgu qhny A-onpx. Bgure fhccyrzragf yvxr zryngbava pna qryvire orarsvgf vapbzzrafhenoyr jvgu QAO - jung vf gur pbtavgvir inyhr bs nabgure ahzore va jbexvat zrzbel gunaxf gb QAO pbzcnerq gb n tbbq avtug’f fyrrc gunaxf gb zryngbava? Zbqrfg punatrf gb bar’f qvrg naq raivebaf pna shaqnzragnyyl vzcebir bar’f jryy-orvat. Rira onfvp genvavat va ernqvat, jvgu gur pehqrfg gnpuvfgbfpbcr grpuavdhrf, pna cnl ynetr qvivqraqf vs bar vf orybj n onfvp yriry bs ernqvat yvxr 200JCZ & fgvyy fhoibpnyvmvat. Naq nyy bs gurfr pna fgneg cnlvat bss vzzrqvngryl.
...and his favorite nootropics are...
- Zbqnsvavy/nezbqnsvavy
- Zryngbava
- Pnssrvar+gurnavar
- Cvenprgnz+pubyvar
- Ivgnzva Q
- Fhyohgvnzvar
- Svfu bvy
comment by CronoDAS · 2013-03-18T05:50:28.453Z · LW(p) · GW(p)
There seems to be a "Math Processing Error" occurring somewhere in the system composed of your website and my Firefox browser:
If the message needs to affect something within the day, then the user has a window of only [Math Processing Error] minutes, which is [Math Processing Error] bits; if the user has a window of a year, that’s slightly better, as a death’s timing down to the minute could embody as much as [Math Processing Error] bits.