What do you think of the counterargument that OpenAI announced o3 in December and publicly solicited external safety testing then, and isn't deploying until ~4 months later?

Here's my summary of the recommendations:

• National security testing
  • Develop robust government capabilities to evaluate AI models (foreign and domestic) for security risks
  • Once ASL-3 is reached, government should mandate pre-deployment testing
  • Preserve the AI Safety Institute in the Department of Commerce to advance third-party testing
  • Direct NIST to develop comprehensive national security evaluations in partnership with frontier AI developers
  • Build classified and unclassified computing infrastructure for testing powerful AI systems
  • Assemble interdisciplinary teams with both technical AI and national security expertise
     
• Export Control Enhancement
  • Tighten semiconductor export restrictions to prevent adversaries from accessing critical AI infrastructure
  • Control H20 chips
  • Require government-to-government agreements for countries hosting large chip deployments
    • As a prerequisite for hosting data centers with more than 50,000 chips from U.S. companies, the U.S. should mandate that countries at high-risk for chip smuggling comply with a government-to-government agreement that 1) requires them to align their export control systems with the U.S., 2) takes security measures to address chip smuggling to China, and 3) stops their companies from working with the Chinese military. The “Diffusion Rule” already contains the possibility for such agreements, laying a foundation for further policy development.
  • Review and reduce the 1,700 H100 no-license required threshold for Tier 2 countries
    • Currently, the Diffusion Rule allows advanced chip orders from Tier 2 countries for less than 1,700 H100s —an approximately $40 million order—to proceed without review. These orders do not count against the Rule’s caps, regardless of the purchaser. While these thresholds address legitimate commercial purposes, we believe that they also pose smuggling risks. We recommend that the Administration consider reducing the number of H100s that Tier 2 countries can purchase without review to further mitigate smuggling risks.
  • Increase funding for Bureau of Industry and Security (BIS) for export enforcement
     
• Lab Security Improvements
  • Establish classified and unclassified communication channels between AI labs and intelligence agencies for threat intelligence sharing, similar to Information Sharing and Analysis Centers used in critical infrastructure sectors
  • Create systematic collaboration between frontier AI companies and intelligence agencies, including Five Eyes partners
  • Elevate collection and analysis of adversarial AI development to a top intelligence priority, as to provide strategic warning and support export controls
  • Expedite security clearances for AI industry professionals
  • Direct NIST to develop next-generation security standards for AI training/inference clusters
  • Develop confidential computing technologies that protect model weights even during processing
  • Develop meaningful incentives for implementing enhanced security measures via procurement requirements for systems supporting federal government deployments.
  • Direct DOE/DNI to conduct a study on advanced security requirements that may become appropriate to ensure sufficient control over and security of highly agentic models

• Energy Infrastructure Scaling
  • Set an ambitious national target: build 50 additional gigawatts of power dedicated to AI by 2027
  • Streamline permitting processes for energy projects by accelerating reviews and enforcing timelines
  • Expedite transmission line approvals to connect new energy sources to data centers
  • Work with state/local governments to reduce permitting burdens
  • Leverage federal real estate for co-locating power generation and next-gen data centers

• Government AI Adoption
  • across the whole of government, the Administration should systematically identify every instance where federal employees process text, images, audio, or video data, and augment these workflows with appropriate AI systems.
  • Task OMB to address resource constraints and procurement limitations for AI adoption
  • Eliminate regulatory and procedural barriers to rapid AI deployment across agencies
  • Direct DoD and Intelligence Community to accelerate AI research, development and procurement
  • Target largest civilian programs for AI implementation (IRS tax processing, VA healthcare delivery, etc.)

• Economic Impact Monitoring
  • Enhance data collection mechanisms to track AI adoption patterns and economic implications
  • The Census Bureau’s American Time Use Survey should incorporate specific questions about AI usage, distinguishing between personal and professional applications while gathering detailed information about task types and systems employed.
  • Update Census Bureau surveys to gather detailed information on AI usage and impacts
  • Collect more granular data on tasks performed by workers to create a baseline for monitoring changes
  • Track the relationship between AI computation investments and economic performance
  • Examine how AI adoption might reshape the tax base and cause structural economic shifts

If you've liked my writing in the past, I wanted to share that I've started a Substack: https://peterwildeford.substack.com/

Ever wanted a top forecaster to help you navigate the news? Want to know the latest in AI? I'm doing all that in my Substack -- forecast-driven analysis about AI, national security, innovation, and emerging technology!

Here you go: https://chatgpt.com/share/67a34222-e7d8-800d-9a86-357defc15a1d

My current working take is that it is at the level of a median-but-dedicated undergraduate of a top university who is interested and enthusiastic about AI safety. But Deep Research can do in 10 minutes what would take that undergraduate about 20 hours.

Happy to try a prompt for you and see what you think.

This was very helpful for you to put together - thank you!

I'd also add that historically I believe about two-thirds of LTFF's money has also come from OpenPhil, so LTFF doesn't represent a fully independent funder (though the decisionmaking around grants is pretty independent).

This is a great poll and YouGov is a highly reputable pollster, but there is a significant caveat to note about the pause finding.

The way the question is framed provides information about "1000 technology leaders" who have signed a letter in favor of the pause but does not mention any opposition to the pause. I think this would push respondents to favor the pause. Ideal question construction would present more neutrally with both support and oppose statements.

Good to know you won't be out of a job just yet!

I asked GPT4 the same prompt and here is what it said:

Of course! Here's a new contra dance for you:

Contra Dance
Whirlwind Delight
By ChatGPT
(Duple improper)

A1

(8) Circle left 3/4

(8) Neighbor swing

 

A2

(8) Long lines forward and back

(8) Ladies chain

 

B1

(8) Star right 1x

(8) Gents aleman left 1.5

 

B2

(8) Half hey, ladies pass right shoulders to start

(8) Partner balance and swing, end facing down in a line of four

 

Enjoy dancing Whirlwind Delight!

How did it do?

Yep! I was wrong and this is false!

If we want to know what arguments resonate with New York Times articles we can actually use surveys, message testing, and focus groups to check and we don't need to guess! (Disclaimer: My company sells these services.)

Cool - I'll follow up when I'm back at work.

That makes a lot of sense. We can definitely test a lot of different framings. I think the problem with a lot of these kinds of problems is that they are low saliency, and thus people tend not to have opinions already, and thus they tend to generate an opinion on the spot. We have a lot of experience polling on low saliency issues though because we've done a lot of polling on animal farming policy which has similar framing effects.

I'll shill here and say that Rethink Priorities is pretty good at running polls of the electorate if anyone wants to know what a representative sample of Americans think about a particular issue such as this one. No need to poll Uber drivers or Twitter when you can do the real thing!

Yeah, it came from a lawyer. The point being that if you confess to something bad, we may be legally required to repot that, so be careful.

Feel free to skip questions if you feel they aren't applicable to you.

Does the chance evolution got really lucky cancel out with the chance that evolution got really unlucky? So maybe this doesn't change the mean but does increase the variance?as for how much to increase the variance, maybe like an additional +/-1 OOM tacked on to the existing evolution anchor?

I'm kinda thinking there's like a 10% chance you'd have to increase it by 10x and a 10% chance you'd have to decrease it by 10x. But maybe I'm not thinking about this right?

There are a lot of different ways you can talk about "efficiency" here. The main thing I am thinking about with regard to the key question "how much FLOP would we expect transformative AI to require?" is whether, when using a neural net anchor (not evolution) to add a 1-3 OOM penalty to FLOP needs due to 2022-AI systems being less sample efficient than humans (requiring more data to produce the same capabilities) and with this penalty decreasing over time given expected algorithmic progress. The next question would be how much more efficient potential AI (e.g., 2100-AI not 2022-AI) could be given fundamentals of silicon vs. neurons, so we might know how much algorithmic progress could affect this.

I think it is pretty clear right now that 2022-AI is less sample efficient than humans. I think other forms of efficiency (e.g., power efficiency, efficiency of SGD vs. evolution) are less relevant to this.

Yeah ok 80%. I also do concede this is a very trivial thing, not like some "gotcha look at what stupid LMs can't do no AGI until 2400".

This is admittedly pretty trivial but I am 90% sure that if you prompt GPT4 with "Q: What is today's date?" it will not answer correctly. I think something like this would literally be the least impressive thing that GPT4 won't be able to do.

Thanks!

Is it ironic that the link to "All the posts I will never write" goes to a 404 page?

Does it get better at Metaculus forecasting?

This sounds like something that could be done as an organization creating a job for it, which could help with mentorship/connections/motivation/job security relative to expecting people to apply to EAIF/LTFF

My organization (Rethink Priorities) is currently hiring for research assistants and research fellows (among other roles) and some of their responsibilities will include distillation.

These conversations are great and I really admire the transparency. It's really nice to see discussions that normally happen in private happen instead in public where everyone can reflect, give feedback, and improve their own thoughts. On the other hand, the combined conversations combined to a decent-sized novel - LW says 198,846 words! Is anyone considering investing heavily in summarizing the content for people to get involved without having to read all that content?

I don't recall the specific claim, just that EY's probability mass for the claim was in the 95-99% range. The person argued that because EY disagrees with some other thoughtful people on that question, he shouldn't have such confidence.

I think people conflate the very reasonable "I am not going to adopt your 95-99% range because other thoughtful people disagree and I have no particular reason to trust you massively more than I trust other people" with the different "the fact that other thoughtful people mean there's no way you could arrive at 95-99% confidence" which is false. I think thoughtful people disagreeing with you is decent evidence you are wrong but can still be outweighed.

So it looks like we survived? (Yay)

I will be on the lookout for false alarms.

I can see whether the site is down or not. Seems pretty clear.

Attention LessWrong - I am a chosen user of EA Forum and I have the codes needed to destroy LessWrong. I hereby make a no first use pledge and I will not enter my codes for any reason, even if asked to do so. I also hereby pledge to second strike - if the EA Forum is taken down, I will retaliate.

Seems like "the right prompt" is doing a lot of work here. How do we know if we have given it "the right prompt"?

Do you think GPT-4 could do my taxes?

1.) I think the core problem is that honestly no one (except 80K) actually is investing significant effort on growing the EA community since 2015 (especially comparable to the pre-2015 effort and especially as a percentage of total EA resources)

2.) Some of these examples are suspect. The GiveWell numbers definitely look to be increasing beyond 2015, especially when OpenPhil's understandably constant fundraising is removed - and this increase in GiveWell seems to line up with GiveWell's increased investment in their outreach. The OpenPhil numbers also look just to be sensitive to a few dominant eight figure grants, which understandably are not annual events. (Also my understanding is that Open Phil is starting off slowly intentionally but will aim to ramp up significantly in the near future.)

3.) As I capture in "Is EA Growing? EA Growth Metrics for 2018", many relevant EA growth statistics have peaked after 2015 or haven't peaked yet.

4.) There are still a lot of ways EA is growing other than what is captured in these graphs. For example, I bet something like total budget of EA orgs has been growing a lot even since 2015.

5.) Contrary to the "EA is inert" hypothesis, EA Survey data has shown that many people have been "convinced" of EA. Furthermore, our general population surveys show that the vast majority of people (>95% of US university students) have still not heard of EA.

FWIW I I put together "Is EA Growing? EA Growth Metrics for 2018" and I'm looking forward for doing 2019+2020 soon

Mr. Money Mustache has a lot of really good advice that I find a lot of value from. However, I think Mr. Money Mustache underestimates the ease and impact of opportunities to grow income relative to cutting spending - especially if you're in (or can be in) a high-earning field like tech. Doubling your income will put you on a much faster path than cutting your spending a further 5%.

PredictionBook is really great for lightweight, private predictions and does everything you're looking for. Metaculus is great for more fully-featured predicting and I believe also supports private questions, but may be a bit of overkill for your use case. A spreadsheet also seems more than sufficient, as others have mentioned.

Thanks. I'll definitely aim to produce them more quickly... this one got away from me.

My understanding is that we also have and might in the future also spend a decent amount of time in a "level 2.5", where some but not all non-essential businesses are open (i.e., no groups larger than ten, restaurants are closed to dine-in, hair salons are open).

A binary search strategy still could be more efficient, depending on the ratio of positives to negatives.

What about binary search?

This is a good answer.

Not really an answer, but a statement and a question - I imagine this is literally the least neglected issue in the world right now. How much does that affect the calculus? How much should we defer to people with more domain expertise?

Introduction to Statistical Learning

This paper also seems helpful: https://arxiv.org/pdf/1812.11118.pdf

Answered here: https://forum.effectivealtruism.org/posts/YAwLfgwhg7opp3rTp/please-take-the-2019-ea-survey#G8Hn64AEyh3uMY2SG

The EA Survey is closing today! Please take! https://www.surveymonkey.co.uk/r/EAS2019LW

Thanks!

It could also be on the list of pros, depending on how one uses LW.

Are you offering to take donations in exchange for pressing the button or not pressing the button?

What happens if you don't check off everything for the day?