Posts
Comments
Just as a look into a different world, ever since I can remember I have loved eating a lot, have wished that I could do more without consequence, and I have dedicated a lot of mental energy to try to figure out ways of maximizing the amount of eating I get to do at a minimum of effort and penalty, and conversely, eating less than I want to involves a considerable about of dissatisfaction and suffering.
If I could magically eat as much as I wanted to without suffering negative consequences to my health (or getting over full), I would basically be eating constantly and it would be one of the main sources of baseline hedonism I have, the same way others listen to music, take baths, etc. Even though it would be expensive.
Projecting my mind onto others, I think this is why a lot of fad diets exist, and there's a disproportionately large amount of low quality discourse about diet and nutrition. A lot of people want the same thing to be possible and are coping hard to believe that it could be so.
Viewed in this light, the "growing a bigger liver" thing seems like a pretty straightforward idea, although I admit that it sounds pretty grotesque from a certain perspective. (Still, though, by comparison, some people do want to grow more muscle primarily so that they can eat more (something I explicitly did the calculations for), and on the grotesque side, people have taken some pretty nasty drugs or just gotten themselves infected with tapeworms just to stay thin.)
I'm not trying to be derisive; in fact, I relate to you greatly. But it's by being on the outside that I'm able to levy a few more direct criticisms:
-
Were you not paid for the other work that you did, leading dev teams and getting frontier research done? Those things should be a baseline on the worth of your time.
-
If that, have you ever tried to maximize the amount of money you can get the) other people to acknowledge your time as worth (ie, get a high salary offer)?
-
Separately, do you know the going rate for consultants with approximately your expertise? Or any other reference class you cna make up. Consulting can cost an incredible amount of money, and that price can be "fair" in a pretty simple sense if it averts the need to do 10s of hours of labor at high wages. It may be one of the highest leverage activities per unit time that exists as a conventional economic activity that a person can simply do.
-
Aside from market rates or whatever, I suggest you just try asking for unreasonable things, or more money than you feel you're worth (think of it as an experiment, and maybe observe what happens in your mind when you flinch from this).
-
Do you have any emotional hangup about the prospect of trading money for labor generally, or money for anything?
-
Separately, do you have a hard time asserting your worth to others (or maybe just strangers) on some baseline level?
(only if you pull the papers every day!)
Not intended to be an insult: I find it astonishing that it's possible to not notice that people mostly eat for enjoyment, and that eating to sustain oneself is not the norm. (Of course, this is coming from a person who eats mostly for enjoyment, and couldn't imagine that some people mostly eat to (edit: I wrote it backwards) sustain themselves.)
In particular, why do you think that so many people were overweight/obese? You flagged this with your remark about the fact that people mostly "cannot just-", but still, mechanistically, what did you think was causing most people to overeat?
Well, a 5 percent rate of being sent to a concentration camp or combat unit isn't exactly negligible, and a further 17 percent were threatened. So maybe it's correct that these effects are "surprisingly mild", but these stats are more justification for the "just following orders" explanation than I'd have imagined from the main text.
Great idea, I'm going to try this out.
I think there is a confusion in terms here.
The illustration is of a function from R to R, so in that sense it's 1-D. But the function as a vector is infinite dimensional over R.
I didn't learn about disown or nohup until recently, because there was no impetus to, because I'd been using tmux. (My workflow also otherwise depended on tmux; when developing locally I liked its method of managing terminal tabs/splits.)
The tricky thing about doom scenarios like this is that I'm not even sure that the AI is wrong.
Oops. I didn't look at the notation closely and assumed a substantially different thing based on the word "distinguishable". Oh well, I hope you guys will think my application was adequate anyway.
Is anyone else noticing that Claude (Sonnet 3.5 new, the default on claude.ai) is a lot worse at reasoning recently? In the past five days or so its rate of completely elementary reasoning mistakes, which persist despite repeated clarification in different ways, seems to have skyrocketed for me.
For the longest time, I would have used the convolutional architecture as an example of one of the few human-engineered priors that was still necessary in large scale machine learning tasks.
But in 2021, the Vision Transformer paper included the following excerpt:
When trained on mid-sized datasets such as ImageNet without strong regularization, these models yield modest accuracies of a few percentage points below ResNets of comparable size. This seemingly discouraging outcome may be expected: Transformers lack some of the inductive biases inherent to CNNs, such as translation equivariance and locality, and therefore do not generalize well when trained on insufficient amounts of data. However, the picture changes if the models are trained on larger datasets (14M-300M images). We find that large scale training trumps inductive bias.
Taking the above as a given is to say, maybe ImageNet really just wasn't big enough, despite it being the biggest publicly available dataset around at the time.
This is a good post; it articulated several of my own critiques of the ROME paper well, and furthermore, helped save me time in understanding the nuts and bolts level stuff in the paper. It was also somewhat helpful to see the results of some of the experiments you did.
I don't believe you technically mentioned this, though you mentioned many things which are conceptually similar: observing the limitations of the ROME paper made me realize that even given ideal model-editing powers, I think that the task of editing a model's understanding is underspecified:
- Any time you tell a model to believe something which is not true, typically several other things will have to change to accommodate it, but it is not clear by default how deep the rabbit hole goes. (This is something which is technically also true with human lying, or just what happens when you audit factual beliefs that are not true.) For example, if you're to say the Eiffel tower is in Rome, sure, now where in Rome is it? Supposing it's 1km north of the Colosseum, what happened to (the building which in reality actually occupies that location)? Likewise, if a person A speaks French, is it because they were born in France? If they were born in France, how did their parents get there? Maybe world history should now be different.