Posts

How do you do hyperparameter searches in ML? 2020-01-13T03:45:46.837Z · score: 9 (4 votes)
[Personal Experiment] Training YouTube's Algorithm 2020-01-09T09:04:17.459Z · score: 14 (6 votes)
Machine Learning Can't Handle Long-Term Time-Series Data 2020-01-05T03:43:15.981Z · score: 3 (20 votes)
[Book Review] The Trouble with Physics 2020-01-05T01:47:26.368Z · score: 25 (15 votes)
Defining "Antimeme" 2019-12-26T09:35:11.906Z · score: 12 (7 votes)
How to Talk About Antimemes 2019-12-22T11:57:27.828Z · score: 9 (4 votes)
The Arrows of Time 2019-12-21T11:42:16.894Z · score: 3 (2 votes)
[Personal Experiment] One Year without Junk Media 2019-12-14T08:26:05.318Z · score: 39 (15 votes)
Confabulation 2019-12-08T10:18:48.986Z · score: 40 (16 votes)
Connectome-Specific Harmonic Waves 2019-12-05T00:23:53.864Z · score: 17 (9 votes)
Symbiotic Wars 2019-12-04T00:06:08.777Z · score: 23 (11 votes)
Antimemes 2019-11-26T05:58:28.954Z · score: 21 (17 votes)
[Personal Experiment] Counterbalancing Risk-Aversion 2019-11-15T08:34:03.460Z · score: 28 (14 votes)
Indescribable 2019-11-10T13:31:45.298Z · score: 12 (10 votes)
Self-Keeping Secrets 2019-11-10T07:59:15.119Z · score: 35 (16 votes)
The Technique Taboo 2019-10-30T11:22:47.184Z · score: 35 (28 votes)
Prospecting for Conceptual Holes 2019-10-30T08:34:52.769Z · score: 41 (19 votes)
Mediums Overpower Messages 2019-10-20T05:46:19.339Z · score: 37 (13 votes)
Invisible Choices, Made by Default 2019-10-20T02:09:02.992Z · score: 27 (17 votes)
Integrating the Lindy Effect 2019-09-07T17:38:27.348Z · score: 15 (9 votes)
Zeno walks into a bar 2019-08-04T07:00:27.114Z · score: 25 (13 votes)

Comments

Comment by lsusr on Chapter 23: Belief in Belief · 2020-02-13T23:22:33.672Z · score: 1 (1 votes) · LW · GW

Much appreciated. Thank you for fixing this in a timely manner.

Comment by lsusr on Chapter 23: Belief in Belief · 2020-02-12T22:19:33.341Z · score: 1 (1 votes) · LW · GW

SPOILER ALERT FOR PARENT COMMENT

Edit: Fixed.

Comment by lsusr on Looking for books about software engineering as a field · 2020-02-05T03:37:19.608Z · score: 1 (1 votes) · LW · GW

Yes. The shared classes and foundation classes are called "standard libraries". Collections of non-standard libraries are called "repositories". Repositories usually accessed via a "package manager". Repositories tend to be system-specific or language-specific. Here are some of the more popular repositories.

Conda combines several of these specific repositories into a mega package manager like you describe.

Comment by lsusr on Looking for books about software engineering as a field · 2020-02-04T03:21:30.208Z · score: 29 (10 votes) · LW · GW

It's hard to find a book that explains software engineering as "a cohesive whole" because software engineering isn't cohesive. It's a grab-bag of various fields that have never been well-organized.

I'd sort the words you listed into the following classes:

  • Data structures and algorithms: linked lists
  • Libraries: APIs, microservices
  • Deployment: CI/CD, AWS, dockerize
  • Security: SQL injection
  • Optimization: multithreaded program, vectorized code

Of these five categories, the only one that could be called "cohesive" is data structures and algorithms. This is a branch of mathematics. There's dozens of introductory textbooks on the subject. Pick any one you like and skim it. You can skip the example code entirely.

Of these five categories, the only one that's truly "universal" to software development is the concept of libraries. A software library is a bit of code someone else wrote. So instead of having to write software to do X you can issue a command to software someone else wrote that does X. Getting a feel for libraries does require writing real code, but you don't have to write much because libraries are all about leveraging code other people wrote. The best way to learn how libraries work is to copy someone else's Python or JavaScript script. All those import statements at the top are libraries. To learn about APIs or microservices you should write (read: copy) someone else's script that interacts with one.

It's hard to find a good book on deployment because the whole field was transformed recently with the launch of AWS and its clones Azure and Google Cloud. The field continues to change rapidly. Anything you learn about it will go rapidly out-of-date. If you want your knowledge to endure you should start with the broader history of severs and networking. In particular, you should find a book on how the Internet is architected (and maybe a little something on Unix Systems, like Chapter 2 of The Art of Unix Programming). Both would give you an solid idea of how the Internet works without writing any code. This will put the jargon you know into context.

Security the least cohesive of any sub-field of software development. It is the ultimate grab-bag of exploits and counter-defenses. There is no foundation. It's turtles all the way down. The best way to get a feel for security is to read some blogs like Schneier on Security and Krebs on Security. You can look up individual exploits like SQL injection when you need to know what they are.

Optimization is almost as incohesive as security. Basically, multithreading and distributed systems are hard. Optimization is a set of tricks to get around this difficulty—except this time they require a deeper understanding of computer science. In general, optimization is the hardest sub-field to understand without a foundation in computer science. The most important trick is "functional code", something difficult to understand without writing code yourself. However, many bits such as caching and the application of GPUs can be understood without knowledge of how to code.

Don't be too intimidated. Half of professional software engineers don't understand half of these subjects half as well as they'd like.


These bits of humor, philosophy and documentation might help build an intuitive general understanding of software development better than any explicit book on the subject.

Comment by lsusr on The Arrows of Time · 2020-01-20T07:29:12.402Z · score: 1 (1 votes) · LW · GW

It maps to a finite length binary number if you force the particle into one of two states. So you could think of this universe as equidistant (in time) instants of a continuous universe where positions are measured, then they're let to evolve and then positions are measured again. The binary strings refer only to the snapshots where the continuous universe is measured.

This ignores the fact that there must be something to measure the particles with. The goal of this thought experiment is to play around with the Born rule while ignoring the time evolution of a wave function governed by the Schrödinger equation.

Comment by lsusr on How do you do hyperparameter searches in ML? · 2020-01-13T23:31:54.127Z · score: 2 (2 votes) · LW · GW

I think it depends on your problem. If you have lots of compute power, high dimensionality and powerful higher-order emergent behavior from hyperparameters then Bayesian optimization makes sense. And vice-versa.

Comment by lsusr on [Personal Experiment] Training YouTube's Algorithm · 2020-01-10T04:09:29.607Z · score: 1 (1 votes) · LW · GW

Most of the weirder suggestions happen later in the recommendations (lower down on the page). I think the algorithm thinks to itself "The user appears tired of music and probably wants to watch something else".

Comment by lsusr on [Book Review] The Trouble with Physics · 2020-01-08T21:09:59.944Z · score: 1 (1 votes) · LW · GW

The Trouble with Physics does address zero-point energy as a possible explanation/alternative for dark energy. Your points 1 and 2 are correct. The problem is that the cosmological constant calculated from vacuum energy is many orders of magnitude greater than the observed cosmological constant.

Comment by lsusr on Connectome-Specific Harmonic Waves · 2020-01-05T23:41:44.608Z · score: 1 (1 votes) · LW · GW

I didn't know that. Thank you. I have corrected the original article.

Comment by lsusr on [Book Review] The Trouble with Physics · 2020-01-05T23:23:33.797Z · score: 1 (1 votes) · LW · GW

If physicists of the sort I'd like to talk to are around at all that's good enough for me.

Comment by lsusr on Machine Learning Can't Handle Long-Term Time-Series Data · 2020-01-05T07:01:32.852Z · score: 5 (4 votes) · LW · GW

Thank you for the correction. AlphaStar is not completely stateless (even ignoring fog-of-war-related issues).

I think the issue here is more about a lack of 'reasoning' skills than time-scales: the network can't think conceptually...

This is exactly what I mean. The problem I'm trying to elucidate is that today's ML techniques can't create good conceptual bridges from short time-scale data to long time-scale data (and vice-versa). In other words, that they cannot generalize concepts from one time scale to another. If we want to take ML to the next level then we'll have to build a system that can. We may disagree about how to best phrase this but I think we're on the same page concerning the capabilities of today's ML systems.

As for connectome-specific harmonic waves, yes, my suggestion is to store slow-changing data in the largest eigenvectors of the Laplacian. The problem with LSTM (and similar RNN systems) is that there's a combinatorial explosion[1] when you try to backpropagate their state cells. This is the computational cliff I mentioned in the article.

The human brain has no known mechanism for conventional backpropagation in the style of artificial neural networks. I believe no such mechanism exists. I hypothesize instead that the human brain doesn't run into the aforementioned computational cliff because there's no physical mechanism to hit that cliff.

So if the human brain doesn't use backpropagation then what does it use? I think a combination of Laplacian eigenvectors and predictive modeling. If everything so far is true then this sidesteps the RNN computational cliff. I think it uses something involving resonance[2] between state networks instead, but we can reach this conclusion without knowing how the human brain works.

This is promising for a two related reasons: one involving power and the other involving trainability.

  • Concerning power: I think resonance could provide a conceptual bridge between shorter time-scales to longer time-scales. This solves the problem of fractal organization in the time domain and provides a computational mechanism for forming logic/concepts and then integrating them with larger/smaller parts of the internal conceptual architecture.
  • Concerning trainability: You don't have to backpropagate when training the human brain (because you can't). If CSHW and predictive modeling is how the human brain gradient ascends then this could completely sidestep the aforementioned computational cliff involved in training RNNs. Such a machine would require a hyperlinearly smaller quantity of training data to solve complex problems.

I think these two ideas work together; the human brain sidesteps the computational cliff because it uses concepts (eigenvectors) in place of raw low-level associations.


  1. I mean that the necessary quantity of training data explodes, not that it's hard to calculate the backpropagated connection weights for a single training datum. ↩︎

  2. Two state networks in resonance automatically exchange information and vice-versa. ↩︎

Comment by lsusr on The Arrows of Time · 2020-01-05T02:22:05.381Z · score: 1 (1 votes) · LW · GW

Fixed. Thanks.

Comment by lsusr on How to Talk About Antimemes · 2020-01-04T07:07:23.191Z · score: 1 (1 votes) · LW · GW

I didn't know what "straussian" means so I looked it up.

The “Straussian” approach to the history of political philosophy is articulated primarily in the writings of Leo Strauss. Strauss wrote extremely careful, detailed studies of canonical philosophical works along with essays explaining his approach. The most controversial claim Strauss made was that philosophers in the past did not always present their thoughts openly and explicitly. They used an “art of writing” to entice potential philosophers to begin a life of inquiry by following the hints the authors gave about their true thoughts and questions. The overriding purpose of Strauss's own studies was to prove that philosophy in its original Socratic form is still possible by showing the persistence of certain fundamental problems throughout the history of philosophy. The most pertinent of those problems, not merely to political philosophy but to human life as a whole, was the problem of justice. Strauss also insisted that “historicism” is based on a philosophical account of the character and limitations of human knowledge and that it can be refuted, therefore, only on the basis of a philosophical argument. ― The Straussian Approach by Catherine Zuckert in The Oxford Handbook of the History of Political Philosophy

But I can't tell exactly what 'people who engage in philosophy' means and why it's in quotes. It sounds like the title of an essay but a web search doesn't find anything.

Do you feel comfortable giving an example of such a memeplex?

Suffering is indeed an antimeme—and a broad-ranging one too. This is a new addition to my collection. Thanks.

I didn't know what the substitution effect is either so here's a definition.

The substitution effect is the decrease in sales for a product that can be attributed to consumers switching to cheaper alternatives when its price rises. ― Source

Comment by lsusr on How to Talk About Antimemes · 2020-01-04T02:07:15.330Z · score: 2 (2 votes) · LW · GW

I appreciate this article very much. I read the whole thing and was disappointed when I realized Curtis Yarvin hadn't finished the series yet. It already has many great insights and illuminating points. I'll be digesting the implications for a while.

Diversity of approaches is important in this game. My favorite things about it is how Yarvin attacks a closely-related problem from a different perspective. In particular:

  • He focuses on the political economy. (I deliberately de-emphasize politics when choosing where to focus.)
  • He debugs things from first principles. "It is always better to debug forward." (I prefer to debug backwards.)

I agree with almost everything he says. I disagree with his claim that it is "always" better to debug forward. Debugging forward is better when you have a small dataset, as is the case with the historic sweep of broad political ideologies (the subject of Yavin's writing). I think when you're dealing with smaller problems, like niche technical decisions, there's a greater diversity data and therefore a greater opportunity to figure things out inductively.

Comment by lsusr on Defining "Antimeme" · 2019-12-29T01:17:49.112Z · score: 4 (3 votes) · LW · GW

These are all good points. By "mainstream" I'm referring to to the information you acquire by being part of a culture; the stuff you learn just because "that's how it's done". This definition of "mainstream" is necessarily subjective because it can be defined only from the perspective of a specific culture or subculture. To someone growing up Amish, "mainstream" (in this context) is Amish. "Mainstream" is important because holes in this kind of knowledge are uniquely difficult to identify.

I don't think there's a spectrum between "no-platforming" and "unimportant". I think they're opposites in a way that doesn't come full-circle, but haven't thought about this hard enough to be sure. It's certainly worth exploring.

Using the word "anti-meme" absolutely could be used to defend ideas from valid criticism, as illustrated by Isnasene's comment to a similar post. Lisp is indeed an excellent example here. I think the library problem was its biggest issue and this got overlooked by early proselytizers.

Comment by lsusr on What spiritual experiences have you had? · 2019-12-27T06:22:40.702Z · score: 1 (1 votes) · LW · GW

I believe I've had kesho experiences too. This easily meets the criteria of "spiritual experience" and "mystical perception", though it has no hallucinatory component.

Comment by lsusr on Defining "Antimeme" · 2019-12-27T06:05:04.903Z · score: 1 (1 votes) · LW · GW

I hadn't noticed utilitarianism and ethical vegetarianism check these boxes. I wrote this series hoping for exactly this kind of insight. Thanks!

Your comment on the cross-cultural application of utilitarianism makes this extra insightful. I have edited the original post to acknowledge that antimemes are not always culture-specific.

Comment by lsusr on How to Talk About Antimemes · 2019-12-24T00:51:05.734Z · score: 2 (2 votes) · LW · GW

That's a good idea. I hadn't thought about it like that.

Comment by lsusr on Antimemes · 2019-12-22T12:13:29.681Z · score: 1 (1 votes) · LW · GW

I think you're right. I write about this topic precisely because it could "provide some high-value insight to a few people". I trust we can figure out a special set of rules for discussing antimemes to make it work. The tiny amount of progress in this field so far suggests there could be lots of low-hanging fruit.

Comment by lsusr on The Arrows of Time · 2019-12-21T21:23:05.158Z · score: 5 (5 votes) · LW · GW

I could point out the errors in the above, like confusing classical and quantum bits...

Could you point out the errors please?

It's not obvious to me why the difference between classical and quantum bits matters in this thought experiment because superposition and entanglement are not supposed to be part of it. I did admittedly gloss over how a quantum bit needs to be measured to force it into one of two states. Does that really invalidate the thought experiment?

[C]onsider reading an academic book or two on the topic. Then learn and do the math...it's best if you do it yourself some day.

I read several academic books on physics and math when I got my bachelor's degree in physics and math. I drilled freshman-level special relativity and quantum mechanics many times when I tutored these subjects. The footnote on Baryon asymmetry is indeed wild speculation, but the rest of the microstate/macrostate stuff generalizes from a statistical mechanics class I took in my sophomore year of college. Which additional math do you recommend I learn how to do?

Comment by lsusr on [Personal Experiment] One Year without Junk Media · 2019-12-18T17:34:39.296Z · score: 2 (2 votes) · LW · GW

I think there's a sweet spot for difficulty. Too easy and you're not exercising your mind. Too hard and you're not learning because everything is incomprehensible.

Like, I did a couple of things to increase my ability to gather/generate information (learned English, studied fast typing, use automatic grammar check), do you think those have a negative side effect?

You mention learning English so I think I'm in the same boat as you here. I'm a native English speaker so for me the rules here apply only to English. When I'm improving my Chinese, anything that makes things easier to comprehend helps (except translations). I don't place any restrictions on Chinese media except that it don't use English anywhere.

By "fast typing" do you mean typing practice videogames? These also helped me learn to touch type. I don't find any problem with using an automatic spellchecker either. I don't see any negative side effects for these tools listed here.

Can you please elaborate what specific criteria you use to decide did some specific content makes you dumber or smarter?

I don't have specific criteria, nor a single overarching theory (yet). I figured out the effects of different mediums through trial and error. I'd cut certain things from my life for a while (usually at least a week) and then afterward I'd review how much I've been learning. I did this over-and-over again until I got a feel for the medium-term impacts of different mediums. The specific criteria I use is simply "observed effect".

If you mean "how do I observe whether a behavior pattern is making me smarter or dumber?" then I can't answer any better than "subjectively". I'd review I'd learned and how my overall behavior changed. If the observable effect was large enough of an improvement I'd consider removing certain kinds of media.

Comment by lsusr on [Personal Experiment] One Year without Junk Media · 2019-12-18T17:24:29.608Z · score: 1 (1 votes) · LW · GW

I did a cursory search for tools like this but didn't find anything. Instead I've been using crude, less specific methods like editing my /etc/hosts file. I didn't know about leechblock and ublock origin. These could be useful.

Comment by lsusr on [Personal Experiment] One Year without Junk Media · 2019-12-16T23:30:38.174Z · score: 4 (3 votes) · LW · GW

The difference has to do with archives vs. feeds. Websites built around archives are okay but websites designed around feeds are not. Basically, I want to allow websites that let me watch old shows like Firefly but not websites that push the newest television series. In other words, I want websites that let me look up shows I already want to watch but not websites that advertise new shows to me.

Comment by lsusr on [Personal Experiment] One Year without Junk Media · 2019-12-15T19:57:56.213Z · score: 3 (2 votes) · LW · GW

To nitpick, I wouldn't say I get "no" learning benefit from these things. The learning rate from university lectures and recorded conference talks is simply too slow to justify the time investment. As for didactic YouTube channels, I believe they help some (most?) people learn but for me the holistic negative effects of these channels outweigh the educational benefit I get from them. This is too bad because I find 3blue1brown and CGP grey to be both informative and entertaining.

Comment by lsusr on Connectome-Specific Harmonic Waves · 2019-12-15T04:25:42.989Z · score: 2 (2 votes) · LW · GW

Thanks. I was surprised when this initially got downvoted twice in a row.

Comment by lsusr on [Personal Experiment] One Year without Junk Media · 2019-12-15T04:22:58.032Z · score: 15 (7 votes) · LW · GW

There's a long list of rules, definitions and exceptions.

Rules

  • Reddit is forbidden.
  • News is forbidden.
  • The Facebook news feed is forbidden but Facebook messenger is allowed.
  • Books, audiobooks, ebooks, podcasts, comic books, webcomics, etc. are allowed.
  • Reading my favorite bloggers is allowed. Web surfing is not.
  • Recorded university lectures and recorded conference talks are forbidden.
  • Music is allowed.
  • Being sick, travelling or being bored somewhere without books is not an excuse to go off this media diet.
  • I am allowed to amend the rules if I write down the change and then wait a week without changing my mind.
  • Movies and television is allowed but only from certain websites.

Definitions

  • The SCP wiki is allowed because it has more in common withe a science fiction novel than a web forum.
  • Ambient media is fine. If a television is playing in a restaurant I can watch it.

Exceptions

  • Anything is allowed as long as I'm doing it socially. That is, I can play videogames and watch anime if it's with a friend.
  • Research is allowed. That is, I am allowed to use any media format in order to answer a well-defined question. For example, if I hear a news story referenced by something I'm allowed to look up what happened. I can watch a video on how to disassemble my computer if I have to disassemble my computer.
  • Anything created by a friend is allowed.
  • Any link or personal recommendation a friend gives me is allowed.
  • On Less Wrong, I can only read my own posts, those referenced in the comments, and those written by commenters. [Update January 20: I'm letting myself read Less Wrong in general.]
  • Magic trick instructional videos are allowed.
  • Anything is allowed if it's in a foreign language without English subtitles.
  • Anything I create myself is allowed.
  • Being sick (non-emergency) is not an exception.

Any advice about this for other people interested in doing the same thing? Anything you learned as you worked up to two months?

I've got enough advice to write an entire post. Here are a few small tips.

  • First do no harm. If you rules make your life worse then change the rules. Rules that make your life worse are expensive, unsustainable and counterproductive.
  • It doesn't matter how complicated your rules and exceptions are as long as they're a strict gain and it's unambiguous to you what is and isn't allowed.
  • Start with just a few days and then work your way to longer and longer periods of time.
Comment by lsusr on Antimemes · 2019-12-11T21:03:04.731Z · score: 1 (1 votes) · LW · GW

I considered putting in a footnote acknowledging the existence of this kind of textual macro in languages like C and Bash. Demonstrating the uniqueness of defmacro requires a more sophisticated example, like embedding Haskell. Even if you can do this in C, you shouldn't.

Comment by lsusr on Antimemes · 2019-12-11T20:53:57.382Z · score: 2 (2 votes) · LW · GW

Any sane person could write up a list of antimemes, but no sane person would post it.

I really like this sentence. Writing about this antimemetic subjects is difficult because readers will often fixate on a particular example instead of refuting the central thesis. I'm growing to appreciate why Paul Graham wrote How to Disagree. He must get this kind of criticism all the time.

Comment by lsusr on Antimemes · 2019-12-11T20:49:42.199Z · score: 1 (1 votes) · LW · GW

I considered putting in a footnote acknowledging the existence of this kind of textual macro in languages like C and Bash. Demonstrating the uniqueness of defmacro requires a more sophisticated example, like embedding Haskell.

Edit: I replied to the wrong comment. This was supposed to be a reply to TAG's parent comment.

Comment by lsusr on Antimemes · 2019-12-10T11:18:13.891Z · score: 3 (3 votes) · LW · GW

You are correct that I don't give "any good reason to think that they [Lisp, stream entry and entrepreneurship] are antimemes". Thank you for your thoughtful response. This kind of quality feedback is why I post on Less Wrong.

So what allegedly makes these things antimemes, in lsusr's view? I'll hazard a guess: they are things whose merits seem clear to lsusr but that are widely neglected by others.

Good guess, but that's not quite what I mean by the word "antimeme". An antimeme is a meme whose information content provokes a self-suppressing response. In Lisp, this comes not from its nicheness (though nicheness is often a prerequisite) nor its syntax (though this helps keep Lisp unpopular) but rather from the defmacro operator.

Programming languages are distinguished from each other by their syntax. A programming language's syntax is defined by its macros. Macros are the parts of a programming language you can't write your own of. For example, Python's if and not are macros. You can't write your own ifnot macro to abbreviate if and not because you're not allowed to write your own macros. You're only allowed to write functions and variables.

In non-Lisp programming languages, macros are set in stone like a processor's instruction set. Lisp is unique because it lets you define new macros with defmacro. Comprehending Lisp equals unlearning all the antipatterns you acquired from working from a tiny set of immutable macros. Defmacro is an antimeme due to an extreme case of the Blub paradox. The more rigid your understanding of computer science is, the harder it is to understand why Lisp matters because a conventional computer science education assumes macros are immutable. It's this mountain of unlearning that makes Lisp an antimeme.

Stream entry is about unlearning self, attachment and object permanence.

Entrepreneurship means unlearning subservience.

To comprehend an antimeme you must unlearn information. The brain's resistance to unlearning things suppresses the antimeme. But this a different kind of unlearning than replacing false facts with true facts. Error correction replaces facts with facts. Antimemetic unlearning is about replacing a fact with an equation. But not just any equation. Antimemes forces you to generalize in a direction orthogonal to everything you know. It's about replacing a specific equation with a general equation.

If all your experience belongs to a specific case then your specific model of the world is the best model according to Occam's razor. The antimeme's model of the world adds complexity without improving your model for the data collected so far. The antimeme would describe your data better if you had data outside your special case but you won't collect that data until after you absorb the antimeme. This circular reasoning creates a chicken-and-egg problem.

I don't think entrepreneurship is widely ignored at all; I think it's very widely admired.

Once again, that's an excellent point. I should have been more specific. The idea of startups as something "someone else" could do is not an antimeme. The idea of startups as something YOU (assuming you're a programmer) could do is an antimeme. Many people could found profitable startups but have dismissed the idea without having performing a careful analysis.

Comment by lsusr on [deleted post] 2019-12-08T10:36:12.566Z

I found the answer. It's the "Create New Sequence" button on the library page.

Comment by lsusr on The Lesson To Unlearn · 2019-12-08T09:37:15.072Z · score: 8 (6 votes) · LW · GW

This essay hits close to home. It feels personal.

When I was in school my aim was to learn as much as I could while still getting decent grades. I sacrificed perfect grades in my pursuit of learning.

Now I recognize I have an unusually high curiosity, but back then I found it bizarrely Orwellian how much emphasis my classmates put on grades. I didn't mind that they weren't at school to learn. I hated how school got in the way of my actual learning because it was designed with the assumption that students aren't there to learn. I've always had a hard time communicating this frustration.

If you merely read good books on medieval history, most of the stuff you learned wouldn't be on the test. It's not good books you want to read, but the lecture notes and assigned reading in this class.

This explains why I did worst at the broadest subjects like foreign languages. I prefer to learn this way but going to school forced me to read the assigned reading than the most educational material. I enjoyed my degree in physics and mathematics because the good books finally converged with the assigned reading.

In some classes, your professor will have had some sort of political axe to grind, and if so you'll have to grind it too. The need for this varies. In classes in math or the hard sciences or engineering it's rarely necessary, but at the other end of the spectrum there are classes where you couldn't get a good grade without it.

This is why I took as few liberal arts classes as I could in college. I got marked down in a philosophy paper for endorsing the idea that ancient philosophers' ought to test their claims against experiment and real-world evidence.

I liked learning, and I really enjoyed some of the papers and programs I wrote in college. But did I ever, after turning in a paper in some class, sit down and write another just for fun?

It was years after graduating college before I could write for fun. I didn't even write essays in college. This was damage from high school.

If the final exam consisted of a long conversation with the professor, you could prepare for it by reading good books on medieval history. A lot of the hackability of tests in schools is due to the fact that the same test has to be given to large numbers of students.

I think this is insightful and generalizable.

Comment by lsusr on Antimemes · 2019-12-08T07:34:39.839Z · score: 4 (3 votes) · LW · GW

Thanks! I've written some more on the topic just for you. You can find a few more specific antimemes in my article The Technique Taboo. You might also enjoy the science fiction stories that inspired this whole sequence of posts.

Comment by lsusr on Connectome-Specific Harmonic Waves · 2019-12-05T02:40:11.976Z · score: 6 (5 votes) · LW · GW

There were two bits of evidence I used to infer this.

"If I'm not sure what it is, how can I remember what it was doing?" The car wasn't sure whether Herzberg and her bike were a "Vehicle", "Bicycle", "Unknown", or "Other", and kept switching between classifications. This shouldn't have been a major issue, except that with each switch it discarded past observations. Had the car maintained this history it would have seen that some sort of large object was progressing across the street on a collision course, and had plenty of time to stop.

The first bit (above) is that the car throws away its past observations. The second bit of evidence is a consequence of the first.

"If we see a problem, wait and hope it goes away". The car was programmed to, when it determined things were very wrong, wait one second. Literally. Not even gently apply the brakes. This is absolutely nuts. If your system has so many false alarms that you need to include this kind of hack to keep it from acting erratically, you are not ready to test on public roads.

Humans have to write ugly hacks like this when when your system isn't architected bottom-up to handle things like the flow of time. A machine learning system designed to handle time series data should never have human beings in the loop this low down the ladder of abstraction.

Comment by lsusr on Connectome-Specific Harmonic Waves · 2019-12-05T02:19:56.905Z · score: 6 (4 votes) · LW · GW

It was an even more recent post, "Neural Annealing" that inspired me to write this one. The most popular reaction "Neural Annealing" was "I don't get it. How can this be important given the relatively low bandwidth of brainwaves compared to synapses?" I wrote a comment to answer that question, and then this "Connectome-Specific Harmonic Waves" article to address the implications in full.

Comment by lsusr on Connectome-Specific Harmonic Waves · 2019-12-05T02:13:44.105Z · score: 3 (2 votes) · LW · GW

I think "Connectome-specific harmonic waves and meditation" has the right idea. There's not much to say since it's just a list of links to other sources. The most interesting thing about that post is Vaniver's comment. I think my article makes a good point that Veniver missed. Namely, that increasing resonance within a brain increases information flow between different parts. This gives a computational advantage above pure aesthetic symmetry. The goal isn't simplicity. It's coordination.

Vaniver makes a good point in his final paragraph. This isn't addressed in "Connectome-specific harmonic waves and meditation" but is addressed in my post.

[A] 'maximize harmony' story needs to have really strong boundary conditions to create the same sorts of conflicts.

This is accounted for by CSHW because (1) oscillations within state networks are observably contained in their state networks and (2) high frequency oscillations propagate a shorter distance than low frequency oscillations.

Comment by lsusr on Connectome-Specific Harmonic Waves · 2019-12-05T02:04:28.160Z · score: 2 (2 votes) · LW · GW

Mike Johnson's "A future for neuroscience" helped me write this post. I think the biggest problem with it is that Mike Johnson fails to recognize the fractal nature of these harmonics.

The fractal nature of CSHW makes possible a self-organizing system. CSHW's power comes from how its fractal nature makes possible a bottom-up understanding of the human brain. Since Mike Johnson misses this, he instead generalizes downward from the macroscopic structure of the brain. Mike Johnson concludes incorrectly that different harmonics would be the same between different people and then generalizes from his mistake.

By operating top-down, Mike Johnson runs into the same problems neuroscience has struggled with since its inception. Cutting a fractal black box into pieces will never tell you how the box works. All it will do is replace the black box with smaller, identical black boxes. It's better to investigate a fractal via induction than dissection.

Under my interpretation, CSHW is a self-organizing fractal, just like a FFNN. Therefore there's no reason to assume that one person's harmonic signature is likely to resemble someone else's, especially at high frequency oscillations. (I can explain this in more detail if the reasons are not clear from my original post.)

Comment by lsusr on Neural Annealing: Toward a Neural Theory of Everything (crosspost) · 2019-12-04T21:26:33.919Z · score: 1 (1 votes) · LW · GW

I think this post is referring to "high energy" not in terms of electrochemical neural activity but instead as a metaphor for optimization in machine learning.

Machine learning is the process of minimizing an error function. We can conceptualize this error function as a potential gradient such as a gravity well or electrostatic potential. Minimizing the energy of a particle in this potential gradient is mathematically equivalent to minimizing the error function. The advantage of referring to this as "energy" instead of "error" is it lets you borrow other terms like kinetic energy (in both the classical and quantum sense) which makes search algorithms intuitively easy to understand. The post is referring to this kind of entropic energy.

Comment by lsusr on Symbiotic Wars · 2019-12-04T10:33:20.156Z · score: 3 (2 votes) · LW · GW

I was thinking about toxoplasma while writing this post, but didn't know what it was called. I even used the War on Terror as an example in my rough draft before editing it out.

Comment by lsusr on Antimemes · 2019-12-04T08:57:50.521Z · score: 1 (1 votes) · LW · GW

An antimeme is a meme that's hard to find for reasons beyond its complexity. For example, Lisp isn't complicated. Informatically-speaking, it's simpler than popular programming languages.

Comment by lsusr on Neural Annealing: Toward a Neural Theory of Everything (crosspost) · 2019-11-30T09:31:30.373Z · score: 12 (11 votes) · LW · GW

This makes sense to me because I work full-time on the bleeding edge of applied AI, meditate, and have degree in physics where I taught the acoustical and energy-based models this theory is based upon. Without a solid foundation in all three of these fields this theory might seem less self-evident.

Hopefully this explanation can help you understand the theory behind the theory. First I'll address points (1), (2) and (3). Then I'll explain the bandwidth issue in more detail.

(1) While it's true that these harmonic frequencies have less information bandwidth then synapses that doesn't mean they don't perform biologically-useful computations. High-bandwidth pattern matching is trivially easy to do with neural networks. The hardest part about neural networks is time series data. (I know this because I am a specialist in the application of machine learning algorithms to handle time-series inputs.) To simplify the situation right now, we [1] don't know how to use neural networks to handle time series data [2] don't know how to get different machine learning systems of any kind to work together--especially with regard to time series data. If CSHW can make any progress in this direction then that would be useful.

(1.1) You are correct that we need a traditional mass of neurons tuned via gradient descent in order to handle high-bandwidth information like our many nerves and to handle complex actions like muscle control. CSHW does not get in the way of these things. Rather CSHW is a simple, elegant way to coordinate many different sub-networks into a human brain. It's not about "how" do you throw a baseball. It's "when" do you throw a baseball. When different networks are out of phase with each other the inputs of one turn into static for the other, which is literally equivalent to tuning out a radio. In short, the purpose of CSHW is not to replace the massive information processing solved by neural networks. Instead, it's purpose is to combine and separate neural networks, as applicable, in response to time-series inputs. It does this fractally, which is the only way to simplify a design to handle massive complexity in a biological system.

(1.2) All CSHW needs to do is to tell which networks should receive information from which other networks. High-frequency waves both propagate shorter distances and oscillate faster (have higher bandwidth) than low-frequency waves, so the information density and response speed gets higher where it needs to be higher (on smaller scales). Remember: the oscillations don't have to transfer information. That's performed by the traditionally-understood neuronal connections. The oscillations can bring different systems in and out of sync in a coordinated manner. This happens at a lower frequency than individual neuron firings and involves larger masses than individual neurons so the necessary bandwidth is much lower. Frequency space might just be a dozen bits long, but there's three spacial dimensions based on actual physical space too.

(1.3) The low bandwidth of the harmonic frequencies explains an important puzzle about consciousness. You know how you can only keep 3-9 concepts in working memory at once? This could be a reflection of the low bandwidth of the low frequency waves.

(2) We have known neurons and evolution are capable of producing waves like this (especially the low frequency ones) for ages. The question neuroscience has been struggling with isn't "can" neurons produce waves like this. It's "why".

(2.1) This theory describes observed behavior especially well once you compare the theory's predictions to the observed brainwaves in advanced meditators. The brain scans of Tibetan yogis and the traditional subjective descriptions written by Zen masters match descriptions of the low frequency brain resonance predicted by this theory. So does a modern Vipassana manual, though it focuses on the high frequency end of the spectrum. This is 3/3 major Buddhist lineages.

(3) As Michael Edward Johnson (OP) mentioned in another comment, recent advancements in fMRIs have let us observe some of the phenomena described in CSHW.

Comment by lsusr on Neural Annealing: Toward a Neural Theory of Everything (crosspost) · 2019-11-29T22:50:08.388Z · score: 6 (5 votes) · LW · GW

CSHW seems blindingly obvious in hindsight. Why is it a new theory? Why wasn't it invented decades ago? Is CSHW new because the technology necessary to test it didn't exist until recently?

Comment by lsusr on Neural Annealing: Toward a Neural Theory of Everything (crosspost) · 2019-11-29T21:20:14.472Z · score: 2 (2 votes) · LW · GW

I hope this gets curated.

Comment by lsusr on My Anki patterns · 2019-11-26T06:42:10.041Z · score: 2 (2 votes) · LW · GW

What datasets do you use for the people cards? That is, whose names do you memorize and where do you get their names and photos?

Comment by lsusr on What's the largest sunk cost you let go? · 2019-11-25T11:12:41.864Z · score: 8 (7 votes) · LW · GW

Magic.

I'm a magician. I let go of magic because [1] I realized that I'd rather be a scientist or engineer than an entertainer, [2] I don't like talking about magic with laymen and [3] it's hard to give others your full attention when in the back of your mind you're figuring out how to deceive them.

I almost never perform magic tricks anymore. Fortunately the autodidactic skills I learned from magic transferred well to everything else in my life, as does the sense of presence I learned to project.

Comment by lsusr on Comment, Don't Message · 2019-11-18T20:44:38.133Z · score: 9 (3 votes) · LW · GW

That's too bad. Your experience differs from mine. However, I have only a 13th as many posts as you have, so it's possible this may change in the future.

Of the two private messages I've received both made total sense as private messages. One dealt with sensitive issues neither of us felt like stating publicly and the other corrected a minor miscalculation in one of my posts (which I immediately fixed).

With these exceptions aside, yes, I also prefer comments to my posts. One of my favorite reasons has to do with replying. Often commenter B will reply better to commenter A than I could have done myself. This is especially useful when several commenters produce various phenotypes of the same fundamental error.

Comment by lsusr on [Personal Experiment] Counterbalancing Risk-Aversion · 2019-11-16T01:41:36.557Z · score: 4 (3 votes) · LW · GW

In every case the "scary" option was the riskier one, so I always had high confidence in what would happen if I chickened out. I compared that to what actually happened when I took what seemed like a riskier option.

Comment by lsusr on [Personal Experiment] Counterbalancing Risk-Aversion · 2019-11-16T01:40:10.566Z · score: 3 (3 votes) · LW · GW

I began this experiment 9 years ago. I only wrote down whether the decisions were right or wrong so I have no record of what the decisions are.

From what I remember, most of the decisions involved socializing. Perhaps half of the decisions involved starting conversations with people (I was afraid of bothering them). Many involved attending social events. I was in college at the time so some of them involved putting together study groups. At least one was whether to ask someone out on a date.

I consistently underestimated the rewards and overestimated the risks of reaching out to others. I wasn't too worried about personal embarrassment. Rather, I was worried that I would bother other people and worsen their days. This almost never happened. In the rare cases where I did bother someone a little the effect was small.

Comment by lsusr on Prospecting for Conceptual Holes · 2019-11-03T11:42:17.961Z · score: 1 (1 votes) · LW · GW

I understand better now what you mean.

When I originally wrote kenshō I meant to ambiguously refer to both to the Japan-specific conceptual cluster itself and the underlying sector of the meditative map because both of them are orthogonal to the Western philosophical tradition.

I would be surprised if it wasn't possible to experience a kenshō state (under a different name, marked off with different conceptual clusters) in the West using a non-Japanese meditative tradition recently adopted from someone else like India.

Comment by lsusr on Prospecting for Conceptual Holes · 2019-11-03T11:07:08.786Z · score: 2 (2 votes) · LW · GW

Japanese people didn't invent kenshō. People around the word (including Westerners) will experience kenshō randomly. The Japanese merely refined a system for identifying kenshō and fostering kenshō states. I can see how if the Western world discovered kenshō independently or if you don't believe in kenshō then we'd be getting into ontological territory. But it doesn't seem like you're coming from this direction.

So I guess "an idea is invented" is the wrong definition for "a concept to exist". I don't know how to define "existence" in the case of kenshō. I don't even know how to define kenshō itself without tautologies.