Posts
Comments
There's a lot of content here, and replying to old posts is encouraged, so I think there should be plenty of material.
Depends mainly on how we both learn best. For me, when it comes to learning a new language that tends to be finding a well-defined, small (but larger than toy) project and implementing it, and having someone to rubber-duck with (over IM/IRC/email is fine) when I hit conceptual walls. I'm certainly up for tackling something that would help out MIRI.
if some post or comment is confusing to you, don't skip it and leave it to people who might have the right background. Ask for clarifications.
I am hereby publicly committing to doing this at least once per day for the next week.
Modafinil eliminates the feeling of being tired, but not the body's need for sleep. Being in sleep deficit weakens the immune system, and I've seen long-term modafinil use without sleep recovery end in pneumonia. So, if I take modafinil to work on a project or stay alert for a long drive, and I finish before the modafinil wears off, I'll go to sleep anyway, because even if my brain doesn't care whether it gets sleep or not, my body needs it. With stimulants like caffeine, getting to sleep before the stimulant wears off is difficult and leads to less sleep.
Most social-science studies are designed to elicit answers in such a way that the participant doesn't realize what question is actually being asked. For example, when William Labov studied the distribution of rhoticity in spoken English, he asked people innocuous questions whose answers contained the sound /r/ in (phonetic) environments where rhoticity can occur. He'd go into a multi-story department store, look at the map, and ask an employee something along the lines of "Where can I find towels?" so that the person would answer "Those are on the fourth floor." Similarly, the wi-fi study wasn't looking at usability any more than Labov was interested in towels; they were really eliciting "willingness to do something dangerous" as a proxy for (lack of) risk awareness. As long as the measure is wearing unique clothing, participants shouldn't be able to recognize it.
One class of questions you didn't bring up has to do with perceptions of risk. There was a poster at this year's USENIX Security about a Mechanical Turk experiment that purported to be a Starbucks study evaluating the usability of a new method of accessing the wi-fi at Starbucks locations: click here to install this new root certificate! (Nearly 3/4 of participants did so.) I can't find the poster online, but this short paper accompanied it at SOUPS.
I've TAed a class like the Programming Languages class you described. It was half Haskell, half Prolog. By the end of the semester, most of my students were functionally literate in both languages, but I did not get the impression that the students I later encountered in other classes had internalized the functional or logical/declarative paradigms particularly well -- e.g., I would expect most of them to struggle with Clojure. I'd strongly recommend following up on that class with SICP, as sketerpot suggested, and maybe broadening your experience with Prolog. In a decade of professional software engineering I've only run into a handful of situations where logic programming was the best tool for the job, but knowing how to work in that paradigm made a huge difference, and it's getting more common.
My local hackerspace, and broadly the US and European hacker communities. This is mainly because information security is my primary focus, but I find myself happier interacting with hackers because in general they tend not only to be highly outcome-oriented (i.e., inherently consequentialist), but also pragmatic about it: as the saying goes, there's no arguing with a root shell. (Modulo bikeshedding, but this seems to be more of a failure mode of subgroups that don't strive to avoid that problem.) The hacker community is also where I learned to think of communities in terms of design patterns; it's one of the few groups I've encountered so far that puts effort into that sort of community self-evaluation. Mostly it helps me because it's a place where I feel welcome, where other people see value in the goals I want to achieve and are working toward compatible goals. I'd encourage any instrumental rationalist with an interest in software engineering, and especially security, to visit a hackerspace or attend a hacker conference.
Until recently I was also involved in the "liberation technology" activism community, but ultimately found it toxic and left. I'm still too close to that situation to evaluate it fairly, but a lot of the toxicity had to do with identity politics and status games getting in the way of accomplishing anything of lasting value. (I'm also dissatisfied with the degree to which activism in general fixates on removing existing structures rather than replacing them with better ones, but again, too close to evaluate fairly.)
Also, presuming that the talk Andreas Bogk has proposed for 30c3 is accepted, you'll want to see it -- it's a huge pragmatic leap forward. (I apologize for not being at liberty to go into any more detail than that. The talk will be livestreamed and recorded, FWIW.)
Eh, cryptocurrency has been a thing since the 80s (see Chaum's Blind signatures for untraceable payments, 1983), but it was patent-encumbered for a long time. If you look at the people involved, they're all cypherpunks of one stripe or another, so I'd place cryptocurrency as a subset of hackers at least initially. (The community has certainly expanded since the emergence of bitcoin.)
Toby Ord has some elderly C code (see Appendix II) that he used in his societal iterated prisoner's dilemma tournaments. You'd have to modify it for your purposes, but it's a small codebase.
I model people constantly, but agency and the "PC vs. NPC" distinction don't even come into it. There are classes of models, but they're more like classes of computational automata: less or more complex, roughly scaling with the scope of my interactions with a person. For instance, it's usually fine to model a grocery store cashier as a nondeterministic finite state machine; handing over groceries and paying are simple enough interactions that an NFSM suffices. Of course the cashier has just as much agency and free will as I do -- but there's a point of diminishing returns on how much effort I invest into forming a more comprehensive model, and since my time and willpower are limited, I prefer to spend that effort on people I spend more time with. Agency is always present in every model, but whether it affects the predictions a given model outputs depends on the complexity of the model.
The way I see it, the difference between an online algorithm and a self-training AI is that the latter would not only need such a heuristic -- let's call it "knowledge base evaluation" -- it would also need to be able to evaluate the fitness of novel knowledge base evaluation heuristics.
On reflection, I'm now contradicting my original statement; the above is a stab toward an algorithmic notion of "self-training" that is orthogonal to how restricted an algorithm's training input set is, or who is restricting it, or how. Using this half-formed notion, I observe that Google's ranking algorithm is AFAIK not self-training, and is also subject to a severely restricted input set. I apologize for any confusion.
Hold on, is he trying to imply that prokaryotes aren't competitive? Not only does all single-celled life compete, it competes at a much faster pace than multicellular life does.
The Feynman Lectures on Computation did this for me by grounding computability theory in physics.
I wouldn't call Google's search personalization "self-training" because the user is responsible for adding new data points to his or her own model; it's the same online algorithm it's always been, just tailored to billions of individual users rather than a set of billions of users. The set of links that a user has clicked on through Google searches is updated every time the user clicks a new link, and the algorithm uses this to tweak the ordering of presented search results, but AFAIK the algorithm has no way to evaluate whether the model update actually brought the ordering closer to the user's preferred ordering unless the user tells it so by clicking on one of the results. It could compare the ordering it did present to the ordering it would have presented if some set of data points wasn't in the model, but then it would have to have some heuristic for which points to drop for cross-validation.
The way I see it, the difference between an online algorithm and a self-training AI is that the latter would not only need such a heuristic -- let's call it "knowledge base evaluation" -- it would also need to be able to evaluate the fitness of novel knowledge base evaluation heuristics. (I'm torn as to whether that goalpost should also include "can generate novel KBE heuristics"; I'll have to think about that a while longer.) Even so, as long as the user dictates which points the algorithm can even consider adding to its KB, the user is acting as a gatekeeper on what knowledge the algorithm can acquire.
Are self-training narrow AIs even a going concern yet? DeepQA can update its knowledge base in situ, but must be instructed to do so. Extracting syntactic and semantic information from a corpus is the easy part; figuring out what that corpus should include is still an open problem, requiring significant human curation. I don't think anyone's solved the problem of how an AI should evaluate whether to update its knowledge base with a new piece of information or not. In the Watson case, an iterative process would be something like "add new information -> re-evaluate on gold standard question set -> decide whether to keep new information", but Watson's fitness function is tied to that question set. It's not clear to me how an AI with a domain-specific fitness function would acquire any knowledge unrelated to improving the accuracy of its fitness function -- though that says more about the fitness functions that humans have come up with so far than it does about AGI.
It's certainly the case that an above-human general intelligence could copy the algorithms and models behind a narrow AI, but then, it could just as easily copy the algorithms and models that we use to target missiles. I don't think the question "is targeting software narrow AI" is a useful one; targeting software is a tool, just as (e.g.) pharmaceutical candidate structure generation software is a tool, and an AGI that can recognize the utility of a tool should be expected to use it if its fitness function selects a course of action that includes that tool. Recognition of utility is still the hard part.
In the past I've manually edited my hosts file to point time-wasting sites to 127.0.0.1, which is a small obstacle at most -- I can easily edit it back -- but is enough of a speedbump to remind me "oh, right, I'm trying to avoid that right now." This time I haven't needed to; I managed to become so frustrated with the fact that my queue was saturated with short-term rewards that weren't netting me anything in the long run that choosing not to visit the sites doesn't seem to require much willpower.
This leads me to wonder whether I could eliminate other influences I want to avoid by making myself sufficiently frustrated with them, but that seems like it would be an unpleasant skill to pick up.
Have you read Pierce's Types and Programming Languages? If so, would you say it provides sufficient foundation for this book?
All of these are dummy subjects. English does not allow a null anaphor in subject position; there are other languages that do. ("There", in that last clause, was also a dummy pronoun.)
My experiment in massively curtailing my use of Twitter and Facebook is going well. Objectively, I'm completing more items on my to-do list on a daily basis; subjectively, it appears that my ability to focus has improved, which I'm willing to chalk up to not feeling compelled to check new-message notifications and therefore just not having that distraction, though I would like to be able to measure this in some way.
I was expecting to feel rather cut off from the world, but fortunately this has not been the case. I've been more responsive to more personal methods of contact (email, IM, in-person interaction) that I had previously felt spread too thin to deal with, and this has been hedonically rewarding (e.g., answering email from person asking for some software design advice -> nice thank-you email -> warm fuzzies).
I also started the Tiny Habits course that I saw mentioned in the Useful Habits Repository.
Another meme that arguably reached the Bay Area via the 1960s/1970s counterculture, but predates it, is "intentional community". This influences startup culture and hacker culture (specifically hackerspaces), and to some (lesser?) extent seasteaders, the back-to-the-land movement, and rationalists as well.
My experience with modafinil has been that sleep is possible, but not necessary.
"Security Applications of Formal Language Theory" is a good overview. (If you don't have IEEE access, there's a tech report version.) Much of the work going on in this area has to do with characterizing classes of vulnerabilities in terms of unintended computational automata that arise from the composition of independent systems, often through novel vulnerability discovery motivated by considering the formal characteristics of a composed system and figuring out what can be wedged into the cracks. There's also been some interesting defensive work (Haskell implementation, an approach I'm interested in generalizing). That's probably a good start.
I have not actually learned Idris yet, and I think I could motivate myself better if I had a study partner; would you be interested in something like that?
I am interested in dependent type systems, total languages, and similar methods of proving certain program errors cannot occur, although I would have to do some background research to learn more of the state of the art in that field.
If you're not already familiar with Idris, I highly recommend checking it out -- it's a dependently typed Haskell variant, a bit like Agda but with a much friendlier type syntax. The downside of Idris is that, as a newer language, it doesn't have nearly as robust a standard library of proofs as, say, Coq. That said, the author, Edwin Brady, is keenly focused on making it a good language for expressing security properties in.
The field I work in has to do with proving that program errors related to maliciously crafted input cannot occur; if it'd be useful I'm happy to braindump/linkdump.
less wrong has a lot of opinions of the form "X is obvious and if you don't believe X you are crazy"
This strikes me as a problem of presentation more than anything else. I've had computer science professors whose lecture style contained a lot of "X is obvious and if you don't believe X you are crazy" -- which was extremely jarring at first, as I came into a CS graduate program from a non-CS background, and didn't have a lot of the formal/academic background that my classmates did. Once I brought myself up to speed, I had the methods I needed to evaluate the obviousness and validity of various Xs, but until then, I sure didn't open my mouth in class a lot.
In the classes I TAed, I strove for a lecture style of "X is the case and if you don't understand why then it's my job to help you connect the dots." That was for 101-level classes, which LW is, at least to my impression, not; if the Sequences are the curriculum for undergraduate rationality, LW is kind of like the grad student lounge. But not, like, a snooty exclusive one, anyone's welcome to hang out and contribute. Still, the focus is on contribute, so that's at least perceived social pressure to perform up to a certain standard -- for me it's experientially very similar to the grad school experience I described.
We're talking about opinions here rather than theorems, and there's a distinction I want to draw between opinions that are speculations about something and opinions that are ... personal? ... but I'm having trouble articulating it; I will try again later, but wanted to point out that this experience you describe generalizes beyond LW.
Foxit Reader supports javascript, and libpoppler (which powers evince and okular, among others) does as well.
Without something to measure, though, that's really just a technical curiosity.
If the PDF is signed by a certificate the user has manually installed, it can embed what Adobe calls "high privilege" javascript, which includes the ability to launch any URL. That's an extra step, which would discourage some users, but on the plus side it addresses the "who's given informed consent?" problem.
Momentarily donning a slightly darker hat: it is also possible for a PDF to launch an arbitrary executable (see pp. 30-34 of Julia Wolf's OMG WTF PDF, video). AIUI this requires no additional privileges.
D'oh. My "Elsevier == paywall" assumption kicked in too quickly. Thank you.
You can embed arbitrary javascript in PDFs, so what about including "phone-home" text-boxes for marginalia in rather the same way that online editions of Real World Haskell and other programming books have comment threads for each paragraph? I'd think other metrics like time-on-page could be measured as well. This would need to be disclosed, of course, and not every reader can be expected to comment, but the "how" seems tractable at first impression.
I don't have any useful insights on what response to measure.
Santos, Santos, and Shimony, "Implicitly preserving semantics during incremental knowledge base acquisition under uncertainty".
Santos, Wilkinson, and Santos, "Fusing multiple Bayesian knowledge sources".
The first describes a formalism called the Bayesian knowledge base that is more compact than the usual conditional probability table approach to a Bayesian network, along with other advantages; the second presents an algorithm for aggregating representations in this formalism.
I ran across this in a book on adversarial reasoning, and haven't found anything about it on LW (at least, not apparent from search results). Are paper summaries (e.g., explaining concepts like BKBs) suitable discussion posts?
The plural of anecdote is not data and neither is the singular, but: I am ten years older than my partner, and this is my partner's first relationship. When we got together, we talked a lot about relationship failure modes and the hazards present in our situation (more than just the age difference), committed to voicing and understanding any concerns/fears/etc. that we experienced, and actually followed through on that commitment. We're only about a year and a half in, but so far this has been amazingly successful. It's hard to suss out how much is due to that commitment and how much is due to us apparently winning the compatibility lottery, but I do think there's a lot to be said for being realistic about failure modes and their likelihood. Long-term relationships, and especially cohabitation, require effort, and the sunk cost fallacy applies to them as much as anything else.
I doubt Harry has read many RFCs, but IP over avian carrier had been around for just over two years at this point in the story.
There are some underlying differences between most charities and bone marrow donation, though. The registry gets no utility from prospective donors until they match donors with patients, so if they can't reach a prospective donor, that database entry is worthless (except perhaps to artificially inflate the size of the database and attract financial donations). A charity that's already gotten some of your money has already gotten some utility, and would like to have more, but faces less risk from losing contact with people on their list.
Also, can you even sign up for bone marrow donation anonymously? I suppose you could give them fake information but they'd still need to be able to reach you.
I can't think of very many other charities that exist to match non-fungible donations with people who need them, though -- pints of O+ are interchangeable, as are hours of unskilled labor -- so I think you've found a really interesting outlier.
Hi. I've been a distant LW lurker for a while now; I first encountered the Sequences sometime around 2009, and have been an avid HP:MOR fan since mid-2011.
I work in computer security with a fair bit of software verification as flavoring, so the AI confinement problem is of interest to me, particularly in light of recent stunts like arbitrary computation in zero CPU instructions via creative abuse of the MMU trap handler. I'm also interested in applying instrumental rationality to improve the quality and utility of my research in general. I flirt with some other topics as well, including capability security, societal iterated game theory, trust (e.g., PKI), and machine learning; a meta-goal is to figure out how to organize my time so that I can do more applied work in these areas.
Apart from that, lately I've become disillusioned with my usual social media circles, in part due to a perceived* uptick in terrible epistemology and in part due to facing the fact that I use them as procrastination tools. I struggle with akrasia, and am experiencing less of it since quitting my previous haunts cold turkey, but things could still be better and I hope to improve them by seeking out positive influences here.
*I haven't measured this. It's entirely possible I've become more sensitive to bad epistemology, or some other influence is lowering my tolerance to bad epistemology.