LessWrong 2.0 Reader

So, it sounds like you'd be in favor of a 1-year pause or slowdown then, but not a 10-year?

That depends on the benefits that we get from a 1-year pause. I'd be open to the policy, but I'm not currently convinced that the benefits would be large enough to justify the costs.

Also, I object to your side-swipe at longtermism

I didn't side-swipe at longtermism, or try to dunk on it. I think longtermism is a decent philosophy, and I consider myself a longtermist in the dictionary sense as you quoted. I was simply talking about people who aren't "fully committed" to the (strong) version of the philosophy.

Picture a dynamic logarithmic scale of discomfort stacking with a ‘hard cap’ where every new instance contributes less and less to the total to the point of flatlining on a graph.

The problem with that is that the agent will tolerate dust specks in everyone's eyes all the time to avoid an extremely tiny probability of one death.

Only vaguely related, but I think you might also enjoy Leonard Stecyk from David Foster Wallace.

(text copied from here)

It is this boy who dons the bright-orange bandolier and shepherds the really small ones through the crosswalk outside school.

This is after finishing the meals-on-wheels breakfast tour of the hospice downtown, whose administrator lunges to bolt her office door when she hears his cart’s wheels in the hall. He has paid out-of-pocket for the steel whistle and the white gloves held palm-out at cars while children who did not dress themselves cross behind him, some trying to run despite WALK DON’T RUN, the happy faced sandwich board he also made himself.

The autos whose drivers he knows he waves at and gives an extra-big smile and tosses some words of good cheer as the crosswalk clears and the cars peel out and move through, some joshing around a little by swerving to miss him only by inches as he laughs and dances aside and makes faces of pretended terror at the flank and rear bumper. The one time that station wagon didn’t miss him really was an accident and he sent the lady several notes to make absolutely sure she knew he understood that and asked a whole lot of people he hadn’t yet gotten the opportunity to make friends with to sign his cast and decorated the crutches very carefully with bits of colored ribbon and tinsel and adhesive sparkles and even before the six weeks the doctor sternly prescribed, he’d given them away to the children’s wing to brighten up some other less lucky and happy kid’s convalescence and by the end of the whole thing he’d been inspired to write a very long theme to enter into the annual Social Studies theme competition about how a positive attitude can make even an accidental injury into an occasion for new friends and bright new opportunities for reaching out to others.
And while the theme didn’t even get honorable mention he honestly didn’t care because he felt like writing the theme had been its own reward and he’d gotten a lot out of the whole nine-draft process and was honestly happy for the kids whose themes did win awards and told them he was 100-plus percent sure they deserved it and that if they wanted to preserve their prize themes and maybe even make displayed items out of them for their parents, he’d be happy to type them up and laminate them and even fix any spelling errors he found if they’d like him to and at home his father puts his hand on Leonard’s shoulder and says he’s really proud that his son’s such a good sport and offers to take him to Dairy Queen as a kind of reward and Leonard tells his father he’s grateful and that the gesture means a lot to him but that in all honesty he’d like it even more if they took the money his father would have spent on the ice-cream and instead donated it either to Easter Seals or, better yet, to UNICEF to go toward the needs of famine-ravaged Biafran kids who he knew for a fact had probably never even heard of ice cream and says that he bets it’ll end up giving both of them a better feeling even then the DQ would and as the father slips the coins in the coin-slot at the special bright-orange UNICEF volunteer cardboard pumpkin bank, Leonard takes a moment to express concern about the father’s facial tick again and to gently rib him about his reluctance to go in and have the family’s MD look at it, noting again that according to the chart on the back of his bedroom door the father is four months overdue for his annual physical and that it’s almost eight months past the date of his recommended tetanus and T.B. boosters.

He serves as hall monitor for period’s one and two but gives far more official warnings than actual citations. He’s there to serve he feels, not run people down. Usually with the official warnings he dispenses a smile and tells them you’re young exactly once so enjoy it and to go get-out here and make this day count why don’t they. Heroes UNICEF and Easter Seals and starts a recycling program in three straight grades. He is healthy and scrubbed and always groomed just well enough to project basic courtesy and respect for the community of which he is a part and he politely raises his hand in class for every question, but only if he’s sure he knows not only the correct answer but the formulation of that answer that the teacher’s looking for that will help advance the discussion of the overall topic they’re covering that day, often staying after class to double-check with the teacher that his take on her general objectives is sound and to ask whether there was any way that his answers could have been better or more helpful.

The boy’s mom has a terrible accident while cleaning the oven and is rushed to the hospital and even though he’s beside himself with concern and says constant prayers former safety, he volunteers to stay home and field calls and relay information to an alphabetized list of concerned family friends and relatives and to make sure the mail and newspaper are brought in and to keep the home’s lights turned on and off in a random sequence at night as officer Chuck of the Michigan State police’s Crime Stoppers public school outreach program sensibly advises when grown-ups are suddenly called away from home and also to call the gas company’s emergency number, which he has memorized, to come check on what may well be a defective valve or circuit in the oven before anyone else in the family is exposed to risk of accidental harm and also, in secret, to work on massive display of bunting and penance and Welcome Homeland World’s Greatest Mom signs which he plans to use the garage’s extendible aluminum ladder—with a responsible neighborhood adult holding it and supervising—to very carefully affix to the front of the home with water-soluble glue so they’ll be there to greet the mom when she’s released from the I.C.U. with a totally clean bill of health which Leonard calls his father repeatedly at the I.C.U. payphone to assure the father that he has absolutely no doubt of (the totally clean bill of health), calling hourly, right on the dot, until there is some kind of mechanical problem with the payphone and when he dials it he just gets a high tone which he duly reports to the telephone company’s new automated 1618 Trouble Line.

He can do several kinds of calligraphy and has been to origami camp twice and can do extraordinary free-hand sketches of local flora with either hand and can whistle all six of Telemann’s Nouveaux Equators and can imitate any birdcall Autobahn could even ever have thought of, don’t even mention spelling bees.
He can make over twenty different kinds of admiral, cowboy, clerical and multi-ethnic hats out of ordinary newspaper and he volunteers to visit the school’s K-through-2nd classrooms teaching the little kids how, a proposal the Carl P. Robinson Elementary principal says he appreciates and has considered very carefully before turning down.

The principal loathes the mere sight of the boy but does not quite know why. He sees the boy in his sleep, at nightmares’ ragged edges; the pressed checked shirt and hair’s hard little part, the freckles and ready, generous smile; anything he can do. The principle fantasizes about sinking a meat hook into Leonard Steel’s bright-eyed little face and dragging the boy face down behind his Volkswagen Beetle over the rough new streets of suburban Grand Rapids.

The fantasies come out of nowhere and horrify the principal, who is a devout Mennonite.
Everyone hates the boy. It is a complex hatred that makes the hater feel guilty and awful and to hate themselves for feeling this way and so makes they involuntarily hate the boy even more for arousing such self-hatred. The whole thing is totally confusing and upsetting. People take a lot of Aspirin when he’s around. The boy’s only real friends among kids are the damaged, the handicapped, the slow, the clinically fat, the last-picked, the non-grata. He seeks them out. All 316 invitations to his eleventh birthday Blow-Out Bash—322 invitations if you count the ones made on audiotape for the blind—are off, sent printed on quality velum with matching high-rag envelopes addressed in ornate Philippian calligraphy he spent three weekends on and each invitation details in Roman Numerated outline-form the itinerary’s half-day at Six Flags, private Ph.D.-guided tour of the Blanford Nature Center and reserved banquette-area-with-free-play at Shakey’s Pizza & Indoor Arcade on Remembrance Drive, the whole day gratis and paid-for out of the paper and aluminum drives the boy got up at 4 a.m. all summer to organize and spearhead, the balance of the drive’s receipts going to the Red Cross and the parents of a Kentwood, MI third-grader with terminal spina bifida who dreams above all-else of seeing Landry and Greer and ‘Night Train’ Lane live from his motorized wheelchair and the invitations explicitly call the party this: A Blow-Out Bash in balloon-shaped font as the caption to an illustrated explosion of good cheer and good will and no-holds-barred, let-out-all-the-stops fun with the bold-faced proviso: Please, no presents required in each of each card’s four corners and the 316 invitations—sent via first-class mail to every student, instructor, substitute, aid, administrator, custodian and physical plant employee at C. P. Robinson Elementary—yield a total attendance of nine celebrants, not counting parents and L.P.N.s of the incapacitated, and yet an undauntedly fine time was had by all was the consensus on the Honest Appraisal and Suggestion cards circulated at party’s end. The massive remainders of chocolate cake, Neapolitan ice cream, pizza, chips, caramel corn, Hershey’s kisses, United Way and Officer Chuck pamphlets on organ tissue donation and the correct procedures to follow if approached by a stranger respectively, kosher pizza for the Orthodox, biodegradable napkins and dietetic soda in souvenir Survived Leonard Steel’s Eleventh Birthday Blow-Out Bash, 1964 plastic glasses with built-in crazy-straws the guests were to keep as mementos all donated to the Kent County Children’s Home via procedures and transport that the birthday-boy had initiated even while the big Twister free-for-all was underway, out of concerns about melted ice cream and staleness and flatness and the waste of a chance to help the less blessed and his father, driving the wood-paneled station wagon and steadying his cheek with one hand, avowed again that the boy beside him had a large, good heart and that he was proud and that if the boy’s mother ever regained consciousness as they so very much hoped, he knew she’d be just awful proud as well.

I'm somewhat confused. I may not be reading the charts you included right, but it sort of looks to me like just rinsing with saline is useful, and that seems like it should be extremely safe and low risk and just about as effective as anything else. Thoughts?

Yeah. It's possible to give quite accurate definitions of some vague concepts, because the words used in such definitions also express vague concepts. E.g. "cygnet" - "a young swan".

Then where are the smart trans men hiding?

There are plenty of stupid and/or distracting behaviors testosterone can push you for without any kind of "chemical brain damage", not only sex. Testosterone is likely to make you seek social status and status-seeking is notoriously incompatible with intellectual pursuits.

This is the strongest alternative explanation by far. I wonder what to look for to check this...

Distributed training seems close enough to being a solved problem that a project costing north of a billion dollars might get it working on schedule. It's easier to stay within a single datacenter, and so far it wasn't necessary to do more than that, so distributed training not being routinely used yet is hardly evidence that it's very hard to implement.

There's also this snippet in the Gemini report:

Training Gemini Ultra used a large fleet of TPUv4 accelerators owned by Google across multiple datacenters. [...] we combine SuperPods in multiple datacenters using Google’s intra-cluster and inter-cluster network. Google’s network latencies and bandwidths are sufficient to support the commonly used synchronous training paradigm, exploiting model parallelism within superpods and data-parallelism across superpods.

I think the crux for feasibility of further scaling (beyond $10-$50 billion) is whether systems with currently-reasonable cost keep getting sufficiently more useful, for example enable economically valuable agentic behavior, things like preparing pull requests based on feature/bug discussion on an issue tracker, or fixing failing builds. Meaningful help with research is a crux for reaching TAI and ASI, but it doesn't seem necessary for enabling existence of a $2 trillion AI company.

First, your non-standard use of the term "counterfactual" is jarring, though, as I understand, it is somewhat normalized in your circles. "Counterfactual" unlike "factual" means something that could have happened, given your limited knowledge of the world, but did not. What you probably mean is "completely unexpected", "surprising" or something similar. I suspect you got this feedback before.

Sticking with physics. Galilean relativity was completely against the Aristotelian grain. More recently, the singularity theorems of Penrose and Hawking unexpectedly showed that black holes are not just a mathematical artifact, but a generic feature of the world. A whole slew of discoveries, experimental and theoretical, in Quantum mechanics were almost all against the grain. Probably the simplest and yet the hardest to conceptualize was the Bell's theorem. 

Not my field, but in economics, Adam Smith's discovery of what Scott Alexander later named Moloch was a complete surprise, as I understand it. 

The number of poor people is much larger than the number of billionaires, but the number of poor people who THINK they're billionaires probably isn't that much larger. Good point about needing to forget the technique, though.