How To Be More Confident... That You're Wrong

So it's important to keep "taking seriously" and "believe strongly" or "believe useful" from contaminating each other, or maybe just to take seriously everything (while retaining ability to tell what's plausible/useful to what extent).

I tried to convince my roommate that my comparative advantage was in being wrong about a lot of different things, since everybody else spends time being wrong about stupid boring things but would still never change their mind, and anyways in order to be wrong about something you have to have had anticipations and counter-able intuitions in the first place with is an important virtue. I think my arguments were rather persuasive. Anyway my point is that once you get to the point where you're not afraid of having been wrong or changing your mind then "taking seriously" becomes more useful, more fun, and less dangerous. I'm reasonably sure it's not a problem, since if you can't change your mind anyway you had bigger things to worry about.

I think the largest costs of sticking your neck out and taking things seriously are social ones, which is unfortunate since talking to smart people is the best way to check yourself.

ETA: Hmuh... huge typical mind fallacy alert on what I just said, I do not actually know my audience well enough to have written the above so confidently. ;)

Eliezer developed a hostility towards the outside view, probably because people were using outside view based arguments against the Singularity

Eliezer developed a hostility towards the outside view because people were misusing the outside view, entirely missing the point and making absolutely ridiculous claims based of superficial similarities.

This is best illustrated in this post which I consider borderline anti-epistomology since it can serve as a universal counterargument against anyone invoking the outside view.

The charge of anti-epistomology is not valid. People could apply the reasoning from that post incorrectly in the same way they could apply his outside view post incorrectly yet you cannot thereby (correctly) label the warning anti-epistemic. Using "Outside View!" as a conversation halter is a bad thing, for the reasons specified. Most relevant is the unpacking of the reasoning underlying outside view considerations - see the bottom half of the post.

Cryptography has mailing lists, and hobbyists, and open source influences, and IRC channels. You, as a rank beginner, are not going to get to a level where you can design a crypto scheme that a dilletante hobbyist with the equivalent of a few university courses, and a lot of time can't beat. Not in a few days, probably not even if you're Terry Tao.

Off the top of my head: given standard assumptions about what is considered a valid attack, there's an attack that takes time on the order of the square root of the size of the keyspace.

NB if I don't try to break your proposal, don't think it's secure - cryptanalysis is generally time-consuming work.

I was an intern in the cryptography research group of a large technology company. My boss got an email from someone in product development wanting us to review a cipher they had designed and intended to use in a product. Of course he gave them the standard reply that any cipher not designed by an expert is almost certainly weak (and they should just use a standard cipher that has gone through years of review by the entire cryptography community), but they refused to believe that their cipher is weak, so we had to actually break it. My boss didn't want to bother one of the researchers, so I got the job, and broke it after a couple of days. (An expert would have literally taken minutes, but none of the researchers specialized in symmetric cipher cryptanalysis.)

If you study cryptography for a while you'll hear and see plenty of cautionary tales like this, so you can pretty much absorb the lesson without actually being on the "receiving end" of it. If you're less patient, but happen to work in a large technology company with a cryptography group that you can email, try that. :) Otherwise, yeah, 2 generalizes to "find something that you can generate a feeling of high confidence but where you will subsequently be wrong with high probability".

I'm guessing (1). First, One-Time Pads have already been invented. Second, you're unlikely to find someone willing to spend the time to extract plaintext from an unspecified ciphertext; the idea was to give your full cipher design up for scrutiny. After all, if it's truly secure, it will remain secure even if the attacker knows how it works.