Posts
Comments
Yes!
If you're interested in rerunning this, consider doing it with two tokens that don't have an obvious ordinal relationship. Ie, in the training set, GPT sees "A" then "B" a billion trillion times, and perhaps has a sense that in many contexts "A" is preferred to "B".
Under this view, perhaps a certain set of interpretability techniques might emerge under a paradigm that makes certain assumptions (eg, that ML kernals are "mostly" linear, that systems are "mostly" stateless, that exotic hacks of the underlying hardware aren't in play, etc). If a series of anomalies were to accumulate that couldn't be explained within this matrix, you might expect to see a new paradigm needed.
Kuhn’s view is that during normal science scientists neither test nor seek to confirm the guiding theories of their disciplinary matrix. Nor do they regard anomalous results as falsifying those theories. (It is only speculative puzzle-solutions that can be falsified in a Popperian fashion during normal science (1970b, 19).) Rather, anomalies are ignored or explained away if at all possible. It is only the accumulation of particularly troublesome anomalies that poses a serious problem for the existing disciplinary matrix. A particularly troublesome anomaly is one that undermines the practice of normal science. For example, an anomaly might reveal inadequacies in some commonly used piece of equipment, perhaps by casting doubt on the underlying theory. If much of normal science relies upon this piece of equipment, normal science will find it difficult to continue with confidence until this anomaly is addressed. A widespread failure in such confidence Kuhn calls a ‘crisis’
The Standford Phil Encylopedia gives:
According to Kuhn the development of a science is not uniform but has alternating ‘normal’ and ‘revolutionary’ (or ‘extraordinary’) phases. The revolutionary phases are not merely periods of accelerated progress, but differ qualitatively from normal science. Normal science does resemble the standard cumulative picture of scientific progress, on the surface at least. Kuhn describes normal science as ‘puzzle-solving’ (1962/1970a, 35–42). While this term suggests that normal science is not dramatic, its main purpose is to convey the idea that like someone doing a crossword puzzle or a chess problem or a jigsaw, the puzzle-solver expects to have a reasonable chance of solving the puzzle, that his doing so will depend mainly on his own ability, and that the puzzle itself and its methods of solution will have a high degree of familiarity. A puzzle-solver is not entering completely uncharted territory... Revolutionary science, however, is not cumulative in that, according to Kuhn, scientific revolutions involve a revision to existing scientific belief or practice (1962/1970a, 92). Not all the achievements of the preceding period of normal science are preserved in a revolution, and indeed a later period of science may find itself without an explanation for a phenomenon that in an earlier period was held to be successfully explained...
I wrote "estimate of up to 15% chance", which is compatible with what you're saying here. But I don't mind updating it to be more precise.
This roughly corresponds to the risk stance I've been taking since finishing my primary course of vaxx.
I just tested + for the first time today. At the very least, this is comforting to read. Thanks!
<3
This comment was made before I updated the question to clarify what's in scope and added the moderation guidelines.
I'm sorry to hear about your health issues with LC. They sounds truly terrible. This question isn't addressed at the topic you're asking about, however.
But what is the base rate? How do demographic factors affect the base rate? Vaccination status?
4 month follow up: a lot of heat was very helpful, but underpowered to fully deal with this problem, and also inconvenient to constantly have a hotwater bottle in the back of my sweater like a hunchback of Notre Dame. Gently ramped up, progressive upper back strengthening has been very helpful for getting to a much more sustainably comfortable point. There's still some lingering issues in the spot between the shoulder blades and occasionally I do still treat with heat, but it's much much better now.
A number of answers have already alluded to deprioritizing long-term savings, but you could go farther and borrow from the post-singularity future. Get a mortgage or other loan. This may work out well even in some worlds with friendly superintelligence, because maybe the AGI gives us a luxury automated communism and your debt obligation is somehow disolved.
Umm... this is not financial advice.
Byyyye!
Yeah, as a 1.5 week follow up, the heat treatement has continued to has continued to be effective, with the sense of tightness continuing to resolve. But it's also required a fairly high dose: probably several hours a day sitting with the waterbottle shoved into my t-shirt. I have a more ergonomic chair arriving tomorrow, G-d and B-zzos willing.
Two days later: this is working quite well so far.
I've worked as a professional programmer for nine years now. I think in at some point a few years ago, it actually began to erode my sense of agency working with computers. At a certain point I became less interested in hobby programming. This was 100% a healthy thing, I started doing things like dancing a lot of contact improv and rock climbing. But when I largely stopped hobby programming, almost all my programming experience was coming from working on production systems. Writing production code is slow. I've routinely had the experience of one or two line changes taking hours or days to get merged. I've worked on modest features that take days or weeks to finish. More and more I began to associate any change to a computer system with inertia and working through complex, unpleasant trade offs.
This feeling has also been exacerbated by trialing internet blocking software. Selfcontrol is probably the best one I've tried, but it's def not everything I'd like. I've occasionally thought about trying to extend it, it's open source, but I've never done any Objective-C and have never been motivated enough to figure out how to get a dev environment for it running and then try and situate myself in a new code base. I've looked at freedom and rescuetime and a couple of others, but I've also been gun-shy about giving these apps deep access to my system. I do my banking on here.
Today I decided to do a quick investigation into the minimal amount required to make my machine shutdown at 10pm everynight, with 15 and 3 minute warnings. It was super easy! I think if you asked me, I probably would have predicted this was easy to do, but I was still somehow emotionally surprised to do a thing with a computer in about 30mins all in, including research.
How to set your system up to warn you, then shut down every evening:
- set OS X to shutdown automatically every night @ 10pm
- https://www.wikihow.com/Automatically-Shut-Down-Your-Computer-at-a-Specified-Time
- takes ~45s !
- Write applescripts to trigger notifications: https://apple.stackexchange.com/questions/57412/how-can-i-trigger-a-notification-center-notification-from-an-applescript-or-shel
- Invoke the applescript via a bash script using the
osascriptcommand - Setup crontab to execute the warnings:
- https://crontab.guru/
- crontab -e (this will drop you into vim)
- https://www.wikihow.com/Automatically-Shut-Down-Your-Computer-at-a-Specified-Time
Done! Save your work when the machine politely reminds you, and get a nice night of sleep.
Caveat programmer: this may not be very bulletproof. I tested each part of this individually, but I'll find out over the next few nights how well it works in practice. I'm excluding some steps that might be a bit confusing if you're not comfortable creating bash scripts or in vim (you don't need that much vim, I barely remember how to quit vim each time I open that damn program). Figuring this stuff out might take you a bit longer, but overall it's still probably pretty fast.
Interesting. Maybe you're right.
I think my model here is something more like, "ML agents that can do good text generation get rolled out to the masses by google and apple, and then some amount of glue infrastructure is developed or even just you can say, "hey google, help me with my dating profile" and it'll do a thing that's 70th or 80th percentile for writing quality, diluting out those of us who were doing 85th to 99th percentile writing.
I imagine it will get commercialized.
But shortly after than it'll be available to everyone and we'll have lost another useful signal
See, your first mistake here was not to consult with John Wentworth and Paul Christiano about thermodynamics.
I bet you didn't even ring up JeffTK!
So much for the rationalist virtue of scholarship..
Overall, this post is a bit confusing--it's like someone from a completely different society was suddenly transported to modern USA. What are you asking / telling us?
I had a similar reaction, I had the impression that Zvi is more worldly and jaded than the median LWer.
High quality, interesting, funny writing has been a difficult to manufacture signal up till now. It's possible GPT-n will change that. But folks on LW are probably filtering for people who will filter for making real pretty with the letter forms.
A small bit of anecdata in favor of the vasoconstriction mechanism: I'm about two weeks into a flare up of upper back soreness/tightness/yeck. It's been a bit chronic/mild for... maybe six months, but I did some stuff that flared it up to... 1.3/10ish two weeks ago. Uncomfortable, and concerning, but not crazy. What was worrying was that it didn't seem to be resolving. Sitting at the computer definitely made it worse. I tried some alternations to my seating (currently not very ergonomic for logistic reasons that are taking time to fix). Adding lower back support pillows didn't help. I tried a bunch of self massage; possibly made it slightly worse. I tried a bunch of gentle mobilization and some moderate weight resistance training. Possibly made it slightly worse. I tried walking a lot. That... tended to make it slightly better, but the effect went away quickly if I spent time sitting. I read this post and tried shoving a hotwater bottle down the back of my t-shirts while working at the computer. That produced a large improvement gradient. I guess you could tell the story that the heat is counter the vasoconstriction. The effect doesn't seem to be present if the water bottle is room temperature, but I also haven't tried that much.
It's not the most practical solution -- I just got a kneeing chair, and have a proper office chair en-route too. But it's def been helpful.
I have been the only weirdo I know of who wears a P100. I say this to emphasize that I've been taking covid seriously.
I don't see any reason to believe covid will be over in two months, or N months, for any value of N less than "however long it takes for humans to come into a new equilibrium with a novel virus." I don't know how long that will be, but 2 seems wrong.
As someone who's worn a p100 a lot, I can also say it's hardly cost free. It has all sorts of social, convenience, physical and psychological costs. Maybe those costs are <<< than your covid risk cost. But they do exist. Personally, it's not obviously correct to me any more that the p100 is the right thing to reach for right now. Mine is currently broken, and I expect I will order a new one, but I'm also kind of happy to "just" be wearing n95s right now.
For an answer that follows a very different intuition, take a look at Does Cosmological Evolution Select for Technology? by Jeffery Shainline. This is up there with aestivation and infinite ethics on the fun idea scale. He gives a nice summary at 2:13:23 on Lex Fridman's podcast to around 2:38. Highly recommend listening to the relevant clip on Fridman, it's pretty great. The entire episode is really interesting, and also contains some other supporting context for Shainline's arguement. Caveat: I haven't read the paper yet. The abstract is:
If the parameters defining the physics of our universe departed from their present values, the observed rich structure and complexity would not be supported. This article considers whether similar fine-tuning of parameters applies to technology. The anthropic principle is one means of explaining the observed values of the parameters. This principle constrains physical theories to allow for our existence, yet the principle does not apply to the existence of technology. Cosmological natural selection has been proposed as an alternative to anthropic reasoning. Within this framework, fine-tuning results from selection of universes capable of prolific reproduction. It was originally proposed that reproduction occurs through singularities resulting from supernovae, and subsequently argued that life may facilitate the production of the singularities that become offspring universes. Here I argue technology is necessary for production of singularities by living beings, and ask whether the physics of our universe has been selected to simultaneously enable stars, intelligent life, and technology capable of creating progeny. Specific technologies appear implausibly equipped to perform tasks necessary for production of singularities, potentially indicating fine-tuning through cosmological natural selection. These technologies include silicon electronics, superconductors, and the cryogenic infrastructure enabled by the thermodynamic properties of liquid helium. Numerical studies are proposed to determine regions of physical parameter space in which the constraints of stars, life, and technology are simultaneously satisfied. If this overlapping parameter range is small, we should be surprised that physics allows technology to exist alongside us. The tests do not call for new astrophysical or cosmological observations. Only computer simulations of well-understood condensed matter systems are required.
Thanks for this, Elizabeth.
Do you have any thoughts on trying to use a PReP protocol with herpes antivirals? I spent about twenty minutes the other day doing some initial searches (just duckduckgo didn't have time to get into pubMed), and didn't turn up anything. Valaciclovir inhibits viral DNA synthesis and is, I believe, fairly safe/mostly well tolerated.
Alarming that it freely "lies" (?) or hallucinates or whatever is going on, rather than replying "I don't know".
sorry to be a rube, but.. why is dredging important? I would guess to make shipping lanes and harbors accessible to larger ships, but is shipping lane and harbor capacity a serious bottleneck? Is dredging capacity one? Is it something else?
Oooooh, these are much better than the ones I was got from nightcafe (I just checked, I was actually using "CLIP guided diffusion".)
DALL-E 2's marshes and sunset marshes are slightly better than what I was getting.
I used nightcafe.studio, a VQGAN+CLIP webservice a bunch in March for the worldbuilding.ai entry I was working on. I found it.. okay for generating images that I could then edit in photoshop, but it took many many tries to get something decent. I'd be particularly interested in seeing what DALLE-E 2 does with these prompts:
"Beautiful giant sunset over the saltwater marsh with tiny abandoned buildings in the distance" "Glass greenhouse with a beautiful forest inside, with people and drones flying" "People dropping into a beautiful marsh from flying drones on a sunny day" "Happy children hanging from flying drones on a sunny day beautiful storybook illustration"
I'm not sure that "first-amendment style free speech" is a good frame for discussing speech issues on the internet. Much of what the internet, and social media does, is provide amplification rather than just speech. Maybe there should be more theorization of what rights and expectations citizens should have w/r/t being able to scale or amplify their speech.
As a kid I developed something that basically is a body scan. I worried about monsters while I was lying in bed. At some point I started imagining a field of green energy starting at the very bottom of my feet and working its way very slowly up each leg, then my torso, down my arms, and finally to the very top of my head. Once I was cloaked in this protective green shield I was safe, as long as I didn't move. It broke if I moved. But if I stayed still I could then inflate it outwards, enveloping my house, neighborhood, city, and eventually the entire world in a protective shield.
I'm skeptical of the idea of a collective unconscious, but still find it an interesting/odd coincidence (and/or weak evidence in favor of a collective unconscious) that I basically developed a body scan practice 35 years before I ever learned anything about meditation, and that it also included a universalist component, enveloping the entire world in protection.
I'm sorry you've had to go through this. I wish it was otherwise.
How come these are spoilers?
Wait, rather than some cuckoo idea that this is related to altitude or marginally less cuckoo lithium theory, what if it's about regional cuisine and eating habits?
Pronunciation is left as an exercise to the reader
Also, before and during WW2, Japan had the most shockingly horrifying death-cult-y style leadership and culture. Dan Carlin does a good job sketching this in his Supernova in the East series.
I am not sure if this dichotomy is a helpful one but we can see Templarrr as stating that there is a theoretic 'failing' which need not be mutually exclusive with the pragmatic 'usefulness' of a theory.
That was what I was also trying to say, in a very pithy way : )
All these criticism can be true, and AGI can still be an existential threat.
I saw Katja's post too, and had a reasonably big update from it, although I don't think she addresses the impact of vaccination status on long-cov probability or badness, so my update is smaller than it might have been.
Thank you for this. I'd say prior to reading this I was around 70% that for someone recently vaccinated or boosted Omicron isn't really worth worrying about, and getting Omicron at some point might even be +EV, due to cross immunity effect. I'd say now I'm around.. 45% on this?
My sense is that EAs in general have not been in the "strict lockdown" mode, trying to do some more careful tradeoffs to allow for things like EAGs. This struck me as.. reasonable-ish at the time, (even up through delta). But if Omicron has a similar long-covid story, this suggests that the upcoming EAGs may not be a good idea in person.
Can your survey give P(long-covid | vaccination status) ?
I feel unreasonably validated by this
I posted mine a day after this comment, do you still feel that way?
Some automated phone queuing systems systems (the things where you call in and get put on hold for three days listening to music) offer a service where you can press a button, hang up and they'll call you back when you would have gotten to the end of the queue.
This should be a mandatory for all these systems.
I would have guessed this was related to the Scott Alexander/NYT thing, but didn't that resolve months ago?
A rough typology of music micro-skills from my first year and a half learning saxophone.
-
Reading music
-
Getting notes + annotations correct
- Binding notes + annotations + physical movements together crisply. Annotations were learned significantly after notes.
- Reading the rests as intervals
-
Reading ahead on the sheet
-
Executing the next physical move correctly
-
Keeping your place in the music synchronized with what you're fingering...
-
...even while you're beginning to chunk the music into phrases
-
Knowing where I am in the piece of music, being able to find my place again if I lose it.
-
-
Generating an internal metronome
- Those moments when I notice I'm swaying to the beat vs trying to kickstart with a foot tap
- Maintaining a foot tap in time with the music
- This is close to, but distinct from Synchronizing with the beat/metronome and Continuing to count beats as you play
-
Perception
- I feel like at least half of the learning has been being able to perceive things, especially in the time domain.
- Being able to discern/repeat beat patterns, eg, couldn't even perceive 1/4 3/16 1/16 3/16 1/16 1/4 at first. Slowing down + isolating to the minimal (ie, clapping), localizing to different body parts + drawing out the 16th interval all required to integrate what feel like even more granular nano-skills, just with my hand on my leg.
- Reminds me of cog reflection tests, where my rhythmic intuition intuition STRONGLY force me towards playing 1/4 1/16 3/16 1/16 3/16 1/4, the inverse of the intended swing. This was made worse by the visually even spacing of the notes on the page.
-
Breath
- Getting breaths between notes: not running out of wind
- Exhaling from the diaphragmatic, with an open throat. Mostly the breath stream should be continuous with stops created by the tongue on the reed -- maintaining a steady stream of lung pressure is counter-intuitive.
- Volume control
-
Mouth
- Not destroying your tongue when tonguing notes
- Not destroying your lower lip
- Coordinating fingers, tongue and breath
- Stable embourche
-
Hands
- Keeping the fingers on the keys
- Hand/finger positioning to minimize strain
- Eg, getting crisp transitions between notes
- Slurring
- Eg, getting crisp transitions between notes, eg B -> C
- Crossing the break, from C to high D
-
Sitting and standing postures that don't leave me sore or with soft-tissue issues
-
The ability to love the sound and love the process
-
Instrument maintenance & care
- Daily and less frequent cleaning
- Safe handling
-
Executing multiple micro-skills at the same time or quicky in a chain
-
Breath/mouth/hand coordination
-
Diagnosing problems: is it the reed, mouth, sax, etc? Guidance here is important.
-
Vocalizing the beat to yourself internally?
- Probably not everyone does this?
Sorry, I was being a bit flip/insider-y. Probably inappropriately so.
I'm curious how much you've engaged with the AI Safety literature/arguments?
"Yudkowsky's DM" --> Eliezer Yudkowsky's [Twitter] Direct Messages.
In the software industry we have the concept of responsible disclosure when one finds a security exploit in a published package.
Do we have responsible disclosure procedures for something that may represent a fundamental AI capability advancement? Who would you even disclose to? Slide into Yudkowsky's DMs?
There were periods of time when I would look at the site two or three times a week, but often just to check the 'adjusted prevalence' numbers, which are hidden by default under the "Details" dropdown. Oh, yeah, prevalence hasn't changed much, no need to update cached models, cool. Probably worth ~$20 aggregated.
The biggest update for me was realizing that going to the dentist was much less risky than it intuitively felt. This was great, probably worth at least $100 (or, I dunno, maybe worth negative money if the dentist is actually net harmful to me, which is possible 😂)
I'd agree that quantification of risks for Long Covid: probably at least $200.
Updates for future variants: probably at least $50.