LessWrong 2.0 Reader

Can I subscribe to your newsletter?

Given that the model eventually outputs the next token, shouldn't the final embedding matrix be exactly your linear fit matrix multiplied by the probability of each state to output a given token? Could you use that?

This is extremely cool! Can you go into more detail about the step used to project the 64 dimensional residual stream to 3 dimensional space? Did you do a linear fit over a few test points and then used it on all the others?

Can anyone here recommend particular tools to practice grammar? Or with strong opinions on the best workflow/tool to correct grammar on the fly? I already know Grammarly and LanguageTool, but Grammarly seems steep at $30 per month when I don’t know if it is any good. I have tried GPT-4 before, but the main problems I have there, is that it is too slow and changes my sentences more than I would like (I tried to make it do that less through prompting, which did not help that much).

I notice that feeling unconfident about my grammar/punctuation leads me to write less online, especially applying for jobs or fellowships, feels more icky because of it. That seems like an avoidable failure mode.

Ideally, I would like something like the German Orthografietrainer (It was created to teach middle and high school children spelling and grammar). It teaches you on a sentence by sentence basis where to put the commas and why by explaining the sentence structure (Illustrated through additional examples). Because it trains you with particularly tricky sentences, the training is effective, and I rapidly got better at punctuation than my parents within ~3 hours. Is there a similar tool for English that I have never heard of?

While writing this, I noticed that I did not have the free version of Grammarly enabled anymore and tried the free version while writing this. One trick I noticed is that it lists what kinds of error you are making across the whole text. So it is easy to infer what particular mistake I made in which spot, and then I correct it myself. Also, Grammarly did not catch a few simple spelling and punctuation mistakes that Grammarly caught (like “anymore” or the comma at the start of this sentence.). At the end, I also tried ProWritingAid, which found additional issues. Its free version is also just $10, so I will try it first.

I recall a comment on the EA forum about Bostrom donating a lot to global dev work in the early days. I've looked for it for 10 minutes. Does anyone recall it or know where donations like this might be recorded?

Hope so, yeah. I'm cautiously optimistic he's doing well by his standards at least.

Due to the weather, we're moving this from Saturday to Sunday. Same time same place.

I have maybe a naive question. How much do we need to know to find the MSP image within the neural network? Is it only doable if we know the HMM to begin with? Or could it be feasible someday to inspect a neural network, find something that looks like an MSP image, and infer the HMM from it?

Edit: I found the post usefwl, thankmuch!!

Mh, was gonna ask when you were taking it.  I'm preparing to try it as a sleep-aid for when I adjust my polyphasic sleep-schedule (wanting to go fm 16h-cycles potentially down to 9h) bc it seems potentially drowsymaking and has much faster plasma decay-rate^[1] compared to alts.  This is good for polyphasic if not want drowsy aft wake.

The data in ^[1] concerns 100mg tablets, however, and a larger dose (eg 400mg) may be longer. The kinetic model^[2] they use will prob be good estimate of plasma concentrations even if adjust dose.

Questions is whether it's good estimation for duration of action in the brain, esp given that it's "single-compartment model" (the blood is one compartment, and the brain is another). My heuristic for whether plasma T predicts brain T is whether the molecule v easily passes the BBB (as melatonin does), since then I can guess that the curve for the brain will look similar to the curve for the blood, offset slightly down and to the right.
 

1. ^{^}

   

   Typical concentration-time curve of plasma ʟ-theanine of one participant after intake of 100 mg ʟ-theanine via one capsule (A) or 250 mL green tea (B). Circles represent measured concentrations of ʟ-theanine. The line represents the modeled plasma concentration-time curve by the use of the 1-compartment model.
   — Kinetics of ʟ-Theanine Uptake and Metabolism in Healthy Participants Are Comparable after Ingestion of ʟ-Theanine via Capsules and Green Tea, , 4 - ScienceDirect

2. ^{^}

   Hot tip, you can screenshot pdf equations and ask gpt-4-turbo (updated) to write it in latex and/or jupyter-notebook-compatible python^[3].

   

3. ^{^}

   Getting a tweakable python model for it was nontrivial after 35m of trying, so i'll prob j wing it w 200mg 1h pre bedtime and do 3h sleep, and adjust bon feelings.  ig the primary variable that determines how alert i feel upon waking up is getting the sleep-cycle timing right, and planning my wake-up routines (lights & text-to-speech-model as alarm) for j after I've done REM-sleep^[4].

4. ^{^}

   While I probably would feel alert if waking up in middle of REM-sleep (after emerging from deep), I want to avoid that bc studies show bad effects from targeted deprivation of REM (leaving other phases untouched).

I'm gonna guess that you actually wouldn't make people pay for drinks if they said they missed because they had COVID, there was a death in the family, etc.?

This is a tough call. How do you determine what is a "legitimately bad enough" case to miss the event? The examples you mention are clearly bad enough but there are other situation where it's much more personal. If I'm feeling low on energy is that a choice I am making or an unavoidable fact about my metabolism? You would have to set up some kind of tribunal or voting for deciding on these cases. That's a lot of effort and would only create bad vibes. So no, if you don't come you pay, no matter the reason. However, enforcement is lax. Mostly it's up to the people themselves to say "Yeah, today is my turn since two weeks ago I couldn't make it". If someone considers their case to be special they can easily get away with not paying and in all likelihood nobody would even notice let alone question it.