Posts
Comments
Is that TinyStories model a super-wide attention-only transformer (the topic of the mechanistic interp work and Buck’s post you cite). I tried to figure it out briefly and couldn’t tell, but I bet it isn’t, and instead has extra stuff like an MLP block.
Regardless, in my view it would be a big advance to really understand how the TinyStories models work. Maybe they are “a bunch of heuristics” but maybe that’s all GPT-4, and our own minds, are as well…
Just want to flag that oseltamivir is not a vaccine, it is an antiviral drug.
I think in your first paragraph, you may be referring to: https://mason.gmu.edu/~gjonesb/IQandNationalProductivity.pdf
I believe the key issue here is with (i). Standard theories where the universe is infinitely large also suppose it was infinitely large at the moment of the big bang.
The discussion here may be helpful.
I think the basic answer is that your question “why does statistical mechanics actually work?”, actually remains unresolved. There are a number of distinct approaches to the foundations of the subject, and none is completely satisfactory.
This review (Uffink 2006), might be of interest, especially the introduction.
Personally, I have never found maximum entropy approaches very satisfying.
An alternative approach, pursued in a lot of the literature on this topic, is to seek a mathematical reason (e.g. in the Hamiltonian dynamics of typical systems statistical mechanics is applied to) why measured quantities at equilibrium take values as though they were averages over the whole phase space with respect to the microcanonical measure (even though they clearly aren't, because typical measurements are too fast---this can be seen from the fact that in systems that are approaching equilibrium, measurements are able reveal their nonequilibrium nature). This program can pursued without any issues of observer-dependence arising.
If the spike looks a lot like one that was experimentally generated to evade antibody responses, what are the odds that Omicron was created through such experiments?
The research in question seems to be described in this Nature paper, where the authors say (emphasis mine):
To more precisely map the targets of polyclonal neutralizing antibodies in individuals who are convalescent, we passaged a recombinant vesicular stomatitis virus (rVSV)/SARS-CoV-2 chimeric virus1,5 in the presence of each of the RU27 plasmas for up to six passages. rVSV/SARS-CoV-2 mimics the neutralization properties of SARS-CoV-2 (refs. 1,5) but obviates the safety concerns that would accompany such studies with authentic SARS-CoV-2.
So, regarding safety, it seems the place to start would be understanding the properties of this rVSV/SARS-CoV-2 construct. Further details are here.
Following your link and looking for the original source, I found that actually Derek Lowe appears not to say that in his blog post, as least not anymore (he made an edit there---though it is not clear that it ever mentioned the S2 subunit).
https://www.science.org/content/blog-post/antibody-dependent-enhancement-and-coronavirus-vaccines
Specifically, it was the vaccines that targeted the N (nucleoprotein) antigen of the coronavirus that had ADE problems,
while the ones that targeted the S (Spike) protein did not. Update: this isn't accurate. There was trouble after immunization with a nucleoprotein-directed vaccine, but ADE could also be seen with some of the Spike-directed vaccine candidates as well - see reviews here, here, and here.
Anyway, it is possible for us to independently see from many different sources that the vaccines code for full-length spike (with minor modification to stabilize the naturally somewhat "floppy" protein in the desired "prefusion" configuration). For example, from here (https://www.nejm.org/doi/full/10.1056/nejmoa2035389):
The mRNA-1273 vaccine is a lipid nanoparticle–encapsulated mRNA-based vaccine that encodes the prefusion stabilized full-length spike protein of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)
Also, the S1 subunit is that part that contains the receptor binding domain, and it is possible to read in many papers (e.g. https://www.science.org/doi/10.1126/scitranslmed.abi9915) that the vaccines elicit antibodies that target this domain.
I believe the mRNA vaccines are based on the full-length spike protein, not just the S2 subunit. The S1 subunit includes the critical receptor binding domain, which is a common target of neutralizing antibodies induced by vaccination, and is the location of many further mutations seen in Omicron.
Edit: To be clear, this fact doesn't invalidate your point about the new mutations looking possibly quite bad for vaccine efficacy.
I appreciate "ifeff" due to its continuity with "iff". However, just to add, I've always had a soft spot for the use of just when to mean "if and only if". It is short and elegant and conveys approximately the right meaning even when it isn't recognized as a term of art.
Above you write: “ RNA can obviously enter the cell nucleus (that's RNA function). ”
but I believe this is not true.
Normally, mRNA is produced in the nucleus and then is transported out of the nucleus. It is then turned into protein by ribosomes, which reside outside the nucleus.
My understanding is that mRNA from mRNA vaccines is not thought to enter the cell nucleus—and certainly this is not necessary for their function.
I’m not sure the fibrosis due to COVID-19 is really the same thing as the article on Wikipedia you link to. Pulmonary fibrosis that arises with autoimmune cause, or no apparent cause, may be more likely to be progressive than that due to COVID-19.
Damage from COVID-19 could still be disabling, of course, even if not progressive.
See here where this matter is discussed: https://www.newswise.com/coronavirus/fibrosis-or-pulmonary-fibrosis-covid-19-coverage-leads-to-confusion/?article_id=730976
Regarding your premodern city size question---I don't see a real constraint emerging from transportation speed. Here's my reasoning using your figures: a city with N people needs N * 5000 sq. meters of land to supply it with food (assume the land can sustain production indefinitely). If this land is a disk around the city the furthest the food has to come is sqrt(N) * 40 meters. If the food is carried at 10 miles a day, the longest supply lines require transporting food for only ~2-3 days for a city of a million, or ~25 days for a city of a 100 million.
Of course this does not a lead to a clean limit because it seems to me there is no very simple limit on how long the food could travel before it is eaten... But especially the first of these (2-3 days) seems reasonable.
Now, as you note it is transportation cost that is key, not just speed. But this requires estimating many more numbers. What did your calculation / model look like?
This argument has made me start seriously reconsidering my generally positive view of cryonics. Does anyone have a convincing refutation?
The best I can come up with is that if resuscitation is likely to happen soon, we can predict the values of the society we'll wake up in, especially if recovery becomes possible before more potentially "value disrupting" technologies like uploading and AI are developed. But I don't find this too convincing.