How do I use caffeine optimally?

It's going to affect different people differently, so take any specific advice as anecdotal, rather than necessarily applicable to you.  It's also going to affect you differently as you age, even separately from any habituation or dependency effects.

Fortunately, it's legal and mostly harmless over any short run, so experimenting is pretty safe.  It is easier to ramp up than to ramp down, so starting small in order to find your minimum effective dose (the smallest amount that you can see any difference) is probably wise, then varying times of day and slight dosage increase (and/or multiple doses per day) to see if it's better or worse.

Note that this is NOT what I did - I just drank a lot of coffee in my youth, to make up for poor sleep hygiene and social/party habits that left me pretty tired for most work mornings.  Only later in life did I cut back, but by then I enjoyed the taste and ritual of coffee enough that I didn't really optimize anything.  

I just quit caffeine a month ago after years of daily dependence on it, and I feel better than I did on it. I now limit myself to 100mg a week. The dependence had a consistent moderate negative affect on my life, so I’d recommend people be very careful to avoid dependence.

I wouldn't recommend regular caffeine at all unless you know from experience that you won't develop a physical dependency. In my experience you get more like short term gain until your body adapts then requires coffee to function normally.

If you do want to try caffeine I recommend trying to pair it with L-theanine (either in pills or green tea) which is supposed to smooth the experience and makes for a cleaner high (YMMV).

If you're looking for a stimulant that you don't take regularly and with shorter half life, consider nicotine gums. Again ymmv, I think gwern has tried it with little effect. Beware the addictive potential (although lower than with cigarettes or vapes)

This is my personal experience. I maintained this schedule all throughout grad school. Eventually, though, I just got a prescription for Adderall and that worked much better.

Caffeine has a steep tolerance curve, and you will rapidly experience diminishing returns if you exclusively use caffeine. This means you are using caffeine to address your caffeine dependency and get you to baseline, rather than to push yourself above baseline. For this reason, you must cycle caffeine with other stimulants (or tolerance breaks) for it to remain effective. You also must have accurate dosage tracking in order to understand the long-term effects. You should also, in my opinion, match caffeine with a 1:2 ratio of caffeine to l-theanine (this is the inverse of the green tea ratio, which has 2:1 caffeine:l-theanine). All of this means coffee is an inadequate source of caffeine. Do not get your caffeine from coffee. Take caffeine pills, starting with a low dose like 25mg caffeine/50mg l-theanine. Track your dosage and dose times in a spreadsheet and also try to rate how effective it was; this self-report is not objective but is better than nothing in the long run. The smallest Starbucks coffee is about 125mg caffeine; on hard days when I was struggling to meet a deadline I might take 200mg (with 400mg l-theanine) twice in a day. While high doses of caffeine paired with 2x l-theanine is the closest you can legally get to adderall, I don't really recommend it outside of making desperate efforts.

The best secondary stimulant I ever found was nicotine. As a non-smoker and non-secondhand smoker with a negligible nicotine tolerance, I needed about 1-2mg sublingually to match the effect of 100mg caffeine. Nicotine has a shorter effect than caffeine and you'll need to redose multiple times in a day for the same effect. It also has a steep curve, and I usually found myself taking up to 5mg by the end of my nicotine weeks. I tried many forms of nicotine, but the most reliably effective with the least side effects was sublingual liquid nicotine at a 1mg/ml concentration. Patches were the worst, gum never really worked, lozenges worked but are kind of high risk high reward.

My schedule was to spend one week on caffeine, one week on nicotine, and one week on adrafinil. Adrafinil never really worked right and eventually I would use this week as a break week. I think it would also have worked fine if I did just caffeine/nicotine, but I never tried that extensively. 

I abused caffeine pretty heavily getting thesis done. Gave up coffee a few years later but it was hard - first thing in the morning, my mouth was ready for coffee and screamed "what is this?!!" when it got tea. My wife got headaches if she didnt get her daily hit so she also went cold turkey which helped. 

When I really need it (up very early for "red-eye special flight" to the capital or a long drive) then I have coffee. We are talking 4-5 times a year. Because I normally only drink tea, I think I get a big hit from it in terms of short term improved concentration.

I personally think the negatives outweigh benefits but I don't have peer-reviewed data to back it. 

There is some observational evidence that coffee drinking increases lifespan. I think the proposed mechanism has to do with promoting autophagy. https://www.acpjournals.org/doi/10.7326/M21-2977 But it looks like decaf works too. (Decaf has a bit of caffeine.)

I think somewhere else I read that unfiltered coffee doesn't improve lifespan, so try to drink the filtered stuff?

In my experience caffeine dependence is not a big deal and might help my sleep cycle.

I cut out morning coffee because of the headaches I got on days I didn't/couldn't have it. After a couple of weeks, I didn't miss it. I still get caffeine from other sources, but it's sporadic, and I think that helps avoid the dependency headaches since my body isn't "expecting" it.

Try delaying caffeine until at least 90 minutes after waking up, preferably a full 2 hours. This was recommended on the Huberman podcast. In my personal experience it removes the caffeine crash later in the day. It also seems to make days without caffeine more tolerable. 

I don't recall the hypothesized mechanism for why this helps (something like it preserves your ability to fully wake up without caffeine) but it's worth a shot.

The focus produced by caffeine is enhanced by theanine (or L-theanine), which also counteracts jitters/headaches caffeine can otherwise induce. You can buy theanine in capsule form. Take 1-2 times as much theanine as caffeine. So for a cup of coffee (either brewed, or containing 2 shots espresso), which contains roughly 150mg caffeine, take say 200mg theanine.

You probably shouldn't routinely have more than 1 cup of (caffeinated) coffee a day if you want to avoid becoming tolerant of it, which removes its effects. And don't drink it in the afternoon or evening, to avoid disturbing your sleep (which may not be obvious, as your sleep can be disturbed even if you have no trouble falling asleep).

Alternatively drink tea, which has far less caffeine than coffee - so you can have as many cups as you like. Tea also contains some theanine (though rather less than the optimal dose).

Anecdata: I aim to never take caffeine  on two consecutive days, and when I do it's normally<50mg. This has worked well for me. 

Consider your current mental health conditions, if you have any. One of my friends (who is diagnosed with anxiety disorder) began drinking only decaf coffee because the caffeine intensified her anxious thoughts. On the opposite side of the caffeine effects spectrum, I have ADHD. I feel caffeine's physical impact (if I placed my hand on my chest I could feel a quicker heartbeat), but I otherwise feel quite relaxed after consuming it rather than jittery. Additionally, caffeine might react negatively with certain medications you take.

I did go through a very intense RedBull binging phase during my freshman year of college, which I absolutely recommend avoiding. Set limits for yourself and stay aware of any changes in your mental or physical responses. Also, congrats on your new job!

Based solely on personal experience (N=1), don't exceed 2 cups/day.

More than that and I used to get headaches on weekends when I didn't drink it. At 2 or less cups/day, no problem.

1. Caffiene dehydrates you, which reduces intelligence, concentration, and makes you expend more of your energy per second of labor (you only have a finite amount of that energy per day). You can mitigate the hydration problems by eating food AND drinking water; it doesn't matter if it makes you use the bathroom more, that's because coffee makes you need to exchange fluids more, not because you're "drinking too much water". Most coffee drinkers do this, and they respond to the tiredness (from dehydration) by drinking more coffee, which is a death spiral.
2. If you're looking for productivity or intelligence, anything related to nootropics or nutrition, even sleep, all takes second priority to avoiding permanent brain damage from Covid [LW · GW]. Brain damage and fatigue-that-will-never-go-away affects far more people than conventional wisdom indicates (even if it wasn't ultra-vague). You can always use lesswrong to improve intelligence/productivity in 6 months, but if Covid ends for good in 2 months and you get long covid between now and then, then you might have to spend the rest of your life working 6 hours per day instead of 8, all because you couldn't wait 2 months for everyone to get infected with like half a dozen different omicron strains (we are closer to herd immunity than ever before). You can avoid permanent brain damage by wearing a cheap p100 mask indoors and eating outdoors, and your vaccination status probably won't help you at all.
3. Caffeine has been widely used for as long as evening alcohol, and similarly long as evening cannabis and cocaine. The fact that everyone uses it, and even that everyone has always been expected to use it, has no bearing on reality whatsoever; it started 500 years ago because it felt good and seemed to help. So it's possible that no matter when you drink it, it might disrupt your sleep, or even reduce your sleep quality in ways you can't perceive (which will reduce intelligence or productivity).
4. Caffeine is a stimulant, which means it is pleasurable, which means that it clearly and provably hacks the reward system of the human brain to reinforce the behavior of drinking coffee. It is very good for business, which drives up the demand and supply of coffee, until billions of people are unwilling to go a day without it.