the gears to ascenscion's Shortform 2023-08-14T15:35:08.389Z
A bunch of videos in comments 2023-06-12T22:31:38.285Z
gamers beware: modded Minecraft has new malware 2023-06-07T13:49:10.540Z
"Membranes" is better terminology than "boundaries" alone 2023-05-28T22:16:21.404Z
"A Note on the Compatibility of Different Robust Program Equilibria of the Prisoner's Dilemma" 2023-04-27T07:34:20.722Z
Did the fonts change? 2023-04-21T00:40:21.369Z
"warning about ai doom" is also "announcing capabilities progress to noobs" 2023-04-08T23:42:43.602Z
"a dialogue with myself concerning eliezer yudkowsky" (not author) 2023-04-02T20:12:32.584Z
A bunch of videos for intuition building (2x speed, skip ones that bore you) 2023-03-12T00:51:39.406Z
To MIRI-style folk, you can't simulate the universe from the beginning 2023-03-01T21:38:26.506Z
How to Read Papers Efficiently: Fast-then-Slow Three pass method 2023-02-25T02:56:30.814Z
Hunch seeds: Info bio 2023-02-17T21:25:58.422Z
If I encounter a capabilities paper that kinda spooks me, what should I do with it? 2023-02-03T21:37:36.689Z
Hinton: "mortal" efficient analog hardware may be learned-in-place, uncopyable 2023-02-01T22:19:03.227Z
Call for submissions: “(In)human Values and Artificial Agency”, ALIFE 2023 2023-01-30T17:37:48.882Z
List of links: Formal Methods, Embedded Agency, 3d world models, and some tools 2023-01-23T18:21:13.733Z
[talk] Osbert Bastani - Interpretable Machine Learning via Program Synthesis - IPAM at UCLA 2023-01-13T01:38:27.428Z
Stop Talking to Each Other and Start Buying Things: Three Decades of Survival in the Desert of Social Media 2023-01-08T04:45:11.413Z 2022-12-21T21:31:17.373Z
[link, 2019] AI paradigm: interactive learning from unlabeled instructions 2022-12-20T06:45:30.035Z
Relevant to natural abstractions: Euclidean Symmetry Equivariant Machine Learning -- Overview, Applications, and Open Questions 2022-12-08T18:01:40.246Z
Interpreting systems as solving POMDPs: a step towards a formal understanding of agency [paper link] 2022-11-05T01:06:39.743Z
Interesting papers: formally verifying DNNs 2022-09-30T08:49:40.820Z
How to make your CPU as fast as a GPU - Advances in Sparsity w/ Nir Shavit 2022-09-20T03:48:35.653Z
We haven't quit evolution [short] 2022-06-06T19:07:14.025Z
What can currently be done about the "flooding the zone" issue? 2020-05-20T01:02:33.333Z
"The Bitter Lesson", an article about compute vs human knowledge in AI 2019-06-21T17:24:50.825Z
thought: the problem with less wrong's epistemic health is that stuff isn't short form 2018-09-05T08:09:01.147Z
Hypothesis about how social stuff works and arises 2018-09-04T22:47:38.805Z
Events section 2017-10-11T16:24:41.356Z
Avoiding Selection Bias 2017-10-04T19:10:17.935Z
Discussion: Linkposts vs Content Mirroring 2017-10-01T17:18:56.916Z
Test post 2017-09-25T05:43:46.089Z
The Social Substrate 2017-02-09T07:22:37.209Z


Comment by the gears to ascension (lahwran) on the gears to ascenscion's Shortform · 2023-09-26T11:28:36.891Z · LW · GW

  • The argument that concerns about future AI risks distract from current AI problems does not make logical sense when analyzed directly, as concerns can complement each other rather than compete for attention.
  • The real motivation behind this argument may be an implicit competition over group status and political influence, with endorsements of certain advocates seen as wins or losses.
  • Advocates for AI safety and those for addressing current harms are not necessarily opposed and could find areas of agreement like interpretability issues.
  • AI safety advocates should avoid framing their work as more important than current problems or that resources should shift, as this can antagonize allies.
  • Both future risks and current harms deserve consideration and efforts to address them can occur simultaneously rather than as a false choice.
  • Concerns over future AI risks come from a diverse range of political ideologies, not just tech elites, showing it is not a partisan issue.
  • Cause prioritization aiming to quantify and compare issues can seem offensive but is intended to help efforts have the greatest positive impact.
  • Rationalists concerned with AI safety also care about other issues not as consequential, showing ability to support multiple related causes.
  • Framing debates as zero-sum competitions undermines potential for cooperation between groups with aligned interests.
  • Building understanding and alliances across different advocacy communities could help maximize progress on AI and its challenges.
Comment by the gears to ascension (lahwran) on the gears to ascenscion's Shortform · 2023-09-26T11:26:42.923Z · LW · GW

  • Experts like Yoshua Bengio have deep mental models of their field that allow them to systematically evaluate new ideas and understand barriers, while most others lack such models and rely more on trial and error.
  • Impostor syndrome may be correct in that most people genuinely don't have deep understanding of their work in the way experts do, even if they are still skilled compared to others in their field.
  • Progress can still be made through random experimentation if a field has abundant opportunities and good feedback loops, even without deep understanding.
  • Claiming nobody understands anything provides emotional comfort but isn't true - understanding varies significantly between experts and novices.
  • The real problem with impostor syndrome is the pressure to pretend one understands more than they do.
  • People should be transparent about what they don't know and actively work to develop deeper mental models through experience.
  • The goal should be learning, not just obtaining credentials, by paying attention to what works and debugging failures.
  • Have long-term goals and evaluate work in terms of progress towards those goals.
  • Over time, actively working to understand one's field leads to developing expertise rather than feeling like an impostor.
  • Widespread pretending of understanding enables a "civilizational LARP" that discourages truly learning one's profession.
Comment by the gears to ascension (lahwran) on the gears to ascenscion's Shortform · 2023-09-26T11:26:08.570Z · LW · GW

a comment thread of mostly ai generated summaries of lesswrong posts so I can save them in a slightly public place for future copypasting but not show up in the comments of the posts themselves

Comment by the gears to ascension (lahwran) on Video essay: How Will We Know When AI is Conscious? · 2023-09-26T10:17:29.595Z · LW · GW

coming back to this: I claim that when we become able to unify the attempted definitions, it will become clear that consciousness is a common, easily-achieved-by-accident information dynamics phenomenon, but that agency, while achievable in software, is not easily achieved by accident.

some definition attempts that I don't feel hold up to scrutiny right now, but which appear to me to be low scientific quality sketches which resemble what I think the question will resolve to later:

Comment by the gears to ascension (lahwran) on Memory bandwidth constraints imply economies of scale in AI inference · 2023-09-18T01:17:05.759Z · LW · GW

a gpu contains 2.5 petabytes of data if you oversample its wires enough. if you count every genome in the brain it easily contains that much. my point being, I agree, but I also see how someone could come up with a huge number like that and not be totally locally wrong, just highly misleading.

Comment by the gears to ascension (lahwran) on Can I take ducks home from the park? · 2023-09-15T01:36:48.329Z · LW · GW

no, you cannot. ducks cannot be moved; ducks are born, never move, and eventually crystallize into a duck statue after about 5 years of life standing in one spot.

Comment by lahwran on [deleted post] 2023-09-12T10:50:55.228Z

@Jim Fisher what's your reasoning for removing the links?

Comment by the gears to ascension (lahwran) on What is to be done? (About the profit motive) · 2023-09-10T12:26:07.704Z · LW · GW

As far as I know, there has never been a society that both scaled and durably resisted command-power being sucked into a concentrated authority bubble; whether this command-power/authority was tokenized via rank insignia or via numerical wealth ratings, the task of building a large-scale society of hundreds of millions to billions that can coordinate, synchronize, keep track of each others' needs and wants, fulfill the fulfillable needs and most wants, and nevertheless retains the benefits of giving both humans and nonhumans significant slack that the best designs for medium-scale societies of single to tens of millions like indigenous governance does and did, is an open problem. I have my preferences for what areas of thought are promising, of course.

Structuring numericalization of which sources of preference-statement-by-a-wanting-being are interpreted as command by the people, motors, and machines in the world appears to me to inlines the alignment problem and generalize it away from AI. It seems to me right now that this is the perspective where "we already have unaligned AI" makes the most sense - what is coming is then more powerful unaligned ai - and it seems to me that promising movement on aligning AI with moral cosmopolitanism will likely be portable back into this more general version. Right now, the competitive dynamics of markets - where purchasers typically sort offerings by some combination of metrics that centers price - creates dynamics where sellers that can produce things the most cheaply in a given area win. Because of monopolization and the externalities it makes tractable, the organizations most able to sell services which involve the work of many AI research workers and the largest compute clusters are somewhat concentrated, with the more cheaply implementable AI systems in more hands but most of those hands are the ones most able to find vulnerabilities in purchasers' decisionmaking and use it to extract numericalized power coupons (money).

It seems to me that ways to solve this would involve things that are already well known: if very-well-paid workers at major AI research labs could find it in themselves to unionize, they may be more able to say no to things where their organizations' command structure has misplaced incentives stemming from those organizations' stock contract owners' local incentives, maybe. But I don't see a quick shortcut around it and it doesn't seem like it's as useful as technical research on how to align things like profit motive with cosmopolitan values, eg via things like Dominant Assurance Contracts.

Comment by the gears to ascension (lahwran) on the gears to ascenscion's Shortform · 2023-09-09T12:25:28.932Z · LW · GW

I have the sense that it's not possible to make public speech non-political, and in order to debate things in a way that doesn't require thinking about how everyone who reads them might consider them, one has to simply write things where they'll only be considered by those you know well. That's not to say I think writing things publicly is bad; but I think tools for understanding what meaning will be taken by different people from a phrase would help people communicate the things they actually mean.

Comment by lahwran on [deleted post] 2023-09-08T11:52:16.889Z

those are easy enough to come by from existing ideas that you probably shouldn't make the world worse by suggesting new ones and adding to the pile of loose gunpowder covering the world. I've come up with plenty myself; I simply try to extract a way to thwart them and discard the husk that suggests how to harm people. compare

Comment by the gears to ascension (lahwran) on Video essay: How Will We Know When AI is Conscious? · 2023-09-07T04:04:03.740Z · LW · GW

I feel that this is appropriate for those hung up on the concept of consciousness, but there are some technical confusions about the topic that I find frustrating. It does still seem to conflate consciousness with agency in a way that I'm not sure is correct. But, if one is thinking of agency when saying consciousness, it's not the worst resource. I'd argue that consciousness is a much easier bar to pass than agency, and agency is the concern - but given that most people think of them as the same and even view their own consciousness through the lens of how it contributes to their own agency, perhaps it's not so bad. But one can be awake, coherent, selfhood-having, and yet not have much in the way of intention, and that is the thing I wish it had focused on more. It's certainly on the right side of things in terms of not just seeing ai as a technology.

Comment by the gears to ascension (lahwran) on a rant on politician-engineer coalitional conflict · 2023-09-06T03:17:29.337Z · LW · GW

Yeah, sounds right, though I've encountered hierarchy resistant organizations that successfully route task by capabilities. It's currently quite fragile but I don't see any fundamental reason it has to stay that way. A non-concentrateable currency creation system such as grassroots economics and a cultural focus on credit assignment towards technical workers can move towards keeping the graph in a mesh structure, some would say the numbers are optional in small orgs, sometimes even big ones, compare types of open source culture. The difficulty is ensuring that any numerical tracking of predictions about who will do what well and by whose request, stays precise enough, without getting precise enough for the people who want to exploit politics-like vulnerabilities to be able to do so.

Comment by the gears to ascension (lahwran) on What wiki-editing features would make you use the LessWrong wiki more? · 2023-08-24T13:26:07.767Z · LW · GW
  • prominent placement. eg:
    • use of tags on a post embeded as their intro into the post even without hover, perhaps at the bottom of the post, perhaps at the top. Users don't seem to consistently realize that the wiki tags are lists of relevant topics.
    • recently edited wiki pages in the sidebar.
    • or at least just put the wiki in the sidebar at all, or clarify what it is, right now it's just under "concepts" and clicking that gives you a big, hard-to-read list. You can hover each one to get the beginning of the summary, but to do that you need to have already been motivated to do so, and it's a trivial inconvenience.
  • suggestion in the lesswrong editor to make phrases that match as titles of wiki pages into links to those wiki pages
    • or just find all places a wiki page is mentioned as a link from the wiki page; this is already possible with the current upgraded search engine, just would be more convenient to have a button for it
      • extra bonus points for showing the search results on the wiki page itself in a way that doesn't take too much attention but which makes it a clear feature. eg, put a search for untagged posts and comments below the list of tagged posts.
  • make sure wiki page edits show up in recent discussion. I vaguely remember they might already, but also remember finding them slightly confusing. might be false memory either way.
  • related pages via semantic search? longstanding wishlist item. search recently improved, possibly already solved.
    • in particular, suggested or similar tags views on every post.
    • this could be done by embedding every post (or every paragraph in every post) using the openai api (or another embedding model from MTEB) and using a semantic-search-with-BM25 similarity comparison to find relevant wiki topics. Bonus points for two way linking.
  • view on wikipedia, ask gpt, etc buttons, to accelerate importing existing overviews?
  • LLM programs to detect out of date wiki pages, eg given page=<wiki page content>, post=<post tagged with that wiki tag>, run gpt4 [{"role": "system", "content": f"Here is a wiki page. Bot please review whether user's article has anything new to add on top of the following wiki page.\n\n> {page}\n\nBot write a VERY CONCISE, ONE SENTENCE, NO NEWLINE overview of what topics the user's article adds beyond the above wiki page."}, {"role": "assistant", "content": "Min word use active."}, {"user": post}] and then use this to generate summaries of what each post with a tag has to add to a topic it's tagged with. I'd guesstimate without even fermi estimating it that this would cost $5k to run all lesswrong posts with gpt4. Note that the brevity prompt is for human use, not to save AI tokens.
Comment by the gears to ascension (lahwran) on Against Almost Every Theory of Impact of Interpretability · 2023-08-18T18:32:07.403Z · LW · GW

Note that a wrench current paradigms throw in this is that self-improvement processes would not look uniquely recursive, since all training algorithms sort of look like "recursive self improvement". instead, RSI is effectively just "oh no, the training curve was curved differently on this training run", which is something most likely to happen in open world RL. But I agree, open world RL has the ability to be suddenly surprising in capability growth. and there wouldn't be much of an opportunity to notice the problem unless we've already solved how to intentionally bound capabilities in RL.

There has been some interesting work on bounding capability growth in safe RL already, though. I haven't looked closely at it, I wonder if any of it is particularly good.

edit: note that I am in fact claiming that after miri deconfuses us, it'll turn out to apply to ordinary gradient updates

Comment by the gears to ascension (lahwran) on We Should Prepare for a Larger Representation of Academia in AI Safety · 2023-08-15T11:13:49.097Z · LW · GW

Hmm. If academia picks stuff up, perhaps interhuman cosmopolitan alignment could use much more discussion on lesswrong, especially from interdisciplinary perspectives. In other words, if academia picks up short term alignment, long term alignment becomes a question of how to interleave differing value systems while ensuring actual practical mobility for socio emotionally disempowered folks. The thing social justice is trying to be but keeps failing at due to corruption turning attempts at analytical "useful wokeness" into crashed slogan thought; the thing libertarianism is trying to be but keeps failing at due to corruption turning defense of selves into decision theoretically unsound casual selfish reasoning; the thing financial markets keep trying to be but keep failing at due to corruption turning competitive reasoning systems into destructive centralization; etc. Qualitative insights from economics and evolutionary game theory and distributed systems programming and etc about how to see all of these issues as the same one as the utility function question in strongly supercapable superreasoner AI. Etc.

Just some rambling this brought to mind.

edited sep 13, here's what GPT4 wrote when asked to clarify wtf I'm talking about:

It seems like the paragraph discusses the potential of including more conversations about interdisciplinary cosmopolitan alignment in the community at lesswrong, particularly in light of developments in academia. You're recommending the community focus on synthesizing crucial insights from across various disciplines and cultural outlooks, with a view to advancing not just short-term but also long-term alignment efforts. The intent is towards a cosmopolitan harmony, where different value systems can coexist while providing tangible empowerment opportunities for people who are socio-emotionally disadvantaged.

The concept of 'alignment' you're discussing appears to be an application of social justice and economic principles aimed at driving change, though this is often undermined by corruption hindering promising initiatives. It corresponds to the 'useful wokeness', which unfortunately ends up being reduced to mere slogans and uncritical loyalty to ideological authority.

Another suggestion you indicated is found in the ideation behind libertarianism. This political ideology works towards autonomy and personal liberty but fails due to corruption that changes this ideal into impractical self-centeredness and hypocritical rule implementation. The common theme across these diverse though corruption-prone strategies is further amplified in your criticism of financial market dynamics, where the competitive nature of trading based reasoning systems often leads to harmful centralization. The discussed failure here appears to be more significant.

The utility function argument in artificial intelligence (AI) with high potential capabilities and reasoning abilities seem to be at the core of your argument. You suggest drawing qualitative insights from economics, evolutionary game theory, distributed system programming, and other disciplines for a more comprehensive understanding of these issues. By attempting to view these myriad challenges as interrelated, one might recast these into a single question about utility function—which if navigated effectively, could address the question of alignment not only in AIs but also across society at large.

Comment by the gears to ascension (lahwran) on ‘We’re changing the clouds.’ An unforeseen test of geoengineering is fueling record ocean warmth · 2023-08-07T13:40:24.912Z · LW · GW

via kagi summarizer,

[main summary is entirely redundant with the post]

[most itemized summary was also redundant]

  • Geoengineering would need to be done carefully and intentionally, rather than the current reckless and haphazard pollution.
  • As the impacts of climate change become more severe, humanity will likely need to not only reduce carbon emissions but also remove existing CO2 and mitigate impacts through geoengineering.
  • There are alternative ways to seed clouds for geoengineering purposes that do not involve sulfur dioxide, such as spraying sea water.
  • Discussing geoengineering now does not mean giving up on reducing carbon emissions and transitioning to renewable energy.

it's not terribly in depth, but I guess those are additional points. more details are in the video, but not a lot more.

Comment by the gears to ascension (lahwran) on ProgramCrafter's Shortform · 2023-08-04T14:29:00.421Z · LW · GW and may be relevant here. Are you familiar with as well? I'm curious how you'd compare the justification for your use of arctan to the justifications in those articles.

Comment by the gears to ascension (lahwran) on AI Awareness through Interaction with Blatantly Alien Models · 2023-07-28T18:17:28.520Z · LW · GW

I don't think you can make LLMs feel alien because they are not in fact highly alien: neural systems are pretty familiar to other neural systems - where by neural system I mean a network of interacting components that learns via small updates which are local in parameter space - and you're more likely to make people go "wow, brains are cool" or "wow, ai is cool" than think it's a deeply alien mind, because there's enough similarity that people do studies to learn about neuroscience from deep learning. Also, I've seen evidence in public that people believe chatgpt's pitch that it's not conscious.

Comment by lahwran on [deleted post] 2023-07-09T19:10:26.143Z
Comment by the gears to ascension (lahwran) on Blanchard's Dangerous Idea and the Plight of the Lucid Crossdreamer · 2023-07-09T11:15:40.365Z · LW · GW
Comment by the gears to ascension (lahwran) on Blanchard's Dangerous Idea and the Plight of the Lucid Crossdreamer · 2023-07-09T11:15:03.541Z · LW · GW
Comment by the gears to ascension (lahwran) on Blanchard's Dangerous Idea and the Plight of the Lucid Crossdreamer · 2023-07-08T22:49:33.323Z · LW · GW
Comment by the gears to ascension (lahwran) on Passing the ideological Turing test? Arguments against existential risk from AI. · 2023-07-08T00:17:13.209Z · LW · GW

Consider this analogy: a child raised in a household espousing violent fascist ideologies may develop behaviors and attitudes that reflect these harmful beliefs. Conversely, the same child nurtured in a peaceful, loving environment may manifest diametrically opposite characteristics. Similarly, we could expect an AI trained on human data that encapsulates how humans see the world to align with our perspective

Then I have bad news about that Internet data and the portion of humanity who endorse large fragments such as authoritarianism or the whole of the fascism recipe, worldwide. Liberation, morality, care for other beings, drive for a healthy community, etc are not at all guaranteed even just in humans. In fact, this is a reason that even if ai is not on its own an xrisk, we should not be instantly reassured.

Comment by the gears to ascension (lahwran) on BOUNTY AVAILABLE: AI ethicists, what are your object-level arguments against AI notkilleveryoneism? · 2023-07-07T20:46:06.340Z · LW · GW

I think the appropriate choice would probably be

Comment by the gears to ascension (lahwran) on BOUNTY AVAILABLE: AI ethicists, what are your object-level arguments against AI notkilleveryoneism? · 2023-07-07T04:01:00.343Z · LW · GW

I feel like their opinions are already fairly well documented, and don't necessarily argue quite the way you're imagining; my sense is they're already in many ways on the same page, they just believe that the origin of the agenticness that will destroy humanity is external to the AI and embedded in the market dynamics of ownership contracts. They see most AI x-risk claims as "whataboutism" hype, and are not engaging seriously as they believe it's an attempt to manipulate them out of making progress on their cause areas. Taking their cause areas seriously as a show of good faith would go a long way, imo; I don't think they have any mechanistic objections to the actual idea of x-risk from goodharting at all! they just view that as having dense prior art in the form of anti-capitalism. Here's a good interview with Gebru:


bad ai summary (they'll be disappointed I dared to, heh) - if this summary doesn't pitch you to watch the original and get the detail (recommend 2x speed or more), you're missing the core insights they have to share, imo, but it's better than you not watching the video in terms of understanding their ideas:

Here are 64 key points extracted from the discussion:

1. Large language models like GPT-3 are being misleadingly marketed as a step towards artificial general intelligence when in reality they are just text generators.  

2. These models suffer from factual inaccuracies, promote misinformation, and reuse copyrighted work without compensation.

3. Companies hype up artificial intelligence to convince people the software is more capable than it really is and to justify profiting from it.

4. The concerns raised by researchers like Emily and Timnit focus on present harms while AI safety researchers primarily worry about potential future dangers from superintelligence.

5. The AI safety community tends to ignore or dismiss the harms highlighted by AI ethics researchers that focus on today's issues.

6. Large language models suffer from a lack of "scoping" where their potential uses are not clearly defined which makes them difficult to regulate and evaluate.  

7. Emily and Timnit argue that the rapid commercialization of large language models has more to do with corporate interests and hype than genuine scientific progress.

8. The narrative that AI is progressing unstoppably is largely fueled by companies and their marketing claims rather than reflecting technological breakthroughs.

9. AI systems today lack any understanding that other minds exist, which is a foundational part of intelligence.

10. AI systems cannot communicate or interact meaningfully with humans the way an intelligent being could.

11. Regulating corporations for the literal output of their AI systems would help reduce harmful effects.

12. The public should push for appropriate legislation around AI that is developed through consultation with those most affected by the technology.

13. OpenAI, despite claiming to work in the interests of humanity and safety, hides important information about how their models are trained which prevents meaningful evaluation and accountability.

14. The paper "Sparks of AGI" from Microsoft researchers exhibits racism by citing a paper arguing for genetic differences in intelligence between races.

15. The "AI pause" letter cites flawed research and misunderstands the capabilities and limitations of large language models.

16. The "AI pause" letter promotes the technology even as it claims to raise concerns about it.

17. The concepts of "AI safety" and "AGI" promoted by organizations like OpenAI have eugenicist roots and are aligned with corporate interests.

18. Researchers should release data and model details to allow for accountability, transparency, and evaluation of any harms.

19. Large language models can deceptively mimic the form of human language and intelligence without possessing the underlying content or abilities.

20. AI companies mislead the public about capabilities to justify profits and build hype that attracts funding and talent.

21. The harms from large language models are now, in reifying discrimination through training data and misleading users, not potential future risks.  

22. Not releasing training data and model specifics allows companies like OpenAI to avoid accountability for potential copyright violations and data theft.  

23. Chat GPT's outputs about factual topics rely on scraping and mashing up information from the web, not any real understanding.

24. Self-driving cars fail at a foundational part of intelligence: communicating and interacting meaningfully with humans.

25. Corporations and billionaires, not technological progress, are primarily responsible for the speed at which large language models have proliferated.

26. There is no evidence large language models are actually intelligent - they only appear coherent to users who assume intelligence.

27. Calls for "AI regulation" often ignore the harms faced by those exploited in developing AI systems and hurt by its outcomes.

28. Concerns about potential "AGI takeover" ignore today's real issues of information pollution, misinformation, and exploitation of artists' work.

29. A narrow vision of who counts as a "person" allows AI safety researchers to ignore or minimize current harms faced by many groups.

30. An "AI pause" would be ineffective at stopping harms since data collection could continue and harm is already occurring.

31. Genuine scientific progress involves choice and communication, not inevitable paths determined by funding and hype.

32. Speculative fiction explores how technology impacts the human condition, rather than just focusing on "cool" applications.

33. The harms today from AI involve unequal access to resources and exacerbating discrimination, not potential doomsday scenarios.

34. An unjust economic system that concentrates corporate power and wealth in few hands drives irresponsible AI development.

35. Fundamental ignorance about what intelligence even is underlies the notion that large language models represent "the first sparks of AGI."

36. Good models of intelligence exhibit a range of intelligent behaviors, not just linguistic coherence.

37. Companies benefit from artificially intelligent-sounding marketing claims while marginalized groups bear the brunt of AI's harms.

38. Large language models are "unscoped technologies" built for any use case rather than aimed at specific, well-defined tasks.

39. No evidence suggests large language models actually understand language beyond providing seemingly plausible outputs.

40. Researchers have proposed regulation that includes watermarking synthetic media and liability for AI-caused harms.

41. The idea that AI is progressing unstoppably toward superintelligence combines science fiction fantasies with a selective reading of history.

42. Researchers evaluating the capabilities of AI systems should provide data, parameters, and documentation to allow for transparency and accountability.

43. ChatGPT-3 is polluting the information ecosystem with non-information and requires governance through legislation.

44. The harms from large language models are real now and can be addressed through policy, not hypothetical future dangers.

45. AI that relies on racially biased studies to define intelligence reveals racism within the AI research community.  

46. AI research and machine learning techniques do not inherently lead to AGI; that narrative is driven by funding and corporate interests.  

47. The key question is not how to build AGI but why - who benefits and who is harmed, and for what purposes it would be created.

48. Claims that AI will revolutionize entire sectors of the economy within 5 years rely on hype and unrealistic timelines.  

49. Understanding the harms today from AI means centering the perspectives of those most affected by biases in data and uses of the technology.

50. Large language models succeed at mimicking the superficial form of language through statistical tricks, not through any inherent capabilities.

51. Responsible development of AI systems requires restricting what technologies can be created as much as developing them in ethically justifiable ways.

52. Current AI systems lack foundational capabilities required for intelligence like understanding that other minds exist with their own perspectives.  

53. AI researchers concerned about future risks ignore or downplay present-day harms highlighted by AI ethics researchers.  

54. Chasing funding to develop AGI and hype about future possibilities has distracted from addressing real problems AI could help solve today.

55. The onus should not be on marginalized groups to prove AI is harmful when the burden of responsibility lies with developers to show that it is beneficial.    

56. Billionaires and technology companies claiming to develop AI for the greater good are primarily motivated by profit and self-interest.

57. Corporations seek to maximize profit from AI while shifting responsibility for any harms onto end users and the public.

58. The unchecked spread of potentially harmful AI tools requires collective action and regulation rather than reliance on individual discernment.

59. AI constitutes a choice by humans about what technologies to develop, not an inevitable progression - we can decide what problems to focus our efforts on solving.

60. AI hype benefits those who control funding and research directions while sidelining or harming marginalized groups.

61. The public needs information about how AI systems actually work in order to evaluate claimed capabilities and push for responsible outcomes.      

62. Viewing today's AI capabilities through the lens of science fiction fantasies obscures real problems that could be helped by more narrowly targeted technologies.

63. Hyperbolic claims that AI will revolutionize society within just a few years ignore complexities of technological progress and human development.  

64. Companies do not always reveal the harms of their technologies or the means to mitigate those harms because it would reduce profits from AI.

Comment by the gears to ascension (lahwran) on What are the best non-LW places to read on alignment progress? · 2023-07-07T03:24:59.368Z · LW · GW

can you and others please reply with lists of people you find notable for their high signal to noise ratio, especially given twitter's sharp decline in quality lately?

Comment by the gears to ascension (lahwran) on Two paths to win the AGI transition · 2023-07-06T22:28:30.445Z · LW · GW

I no longer think there's anything we could learn about how to align an actually superhuman agi that we can't learn from a weaker one.

(edit 2mo later: to rephrase - there exist model organisms of all possible misalignment issues that are weaker than superhuman. this is not to say human-like agi can teach us everything we need to know.)

Comment by the gears to ascension (lahwran) on Nature: "Stop talking about tomorrow’s AI doomsday when AI poses risks today" · 2023-06-28T14:57:05.305Z · LW · GW

Yeah I travel and ask randos on the street and they agree AI is about to kill us all. Does he travel?

Comment by the gears to ascension (lahwran) on A bunch of videos in comments · 2023-06-27T23:26:30.277Z · LW · GW

Khan Academy's science videos are a valuable resource but may not promote meaningful learning on their own. Students often think they already know the material so do not pay full attention. When asked what they saw, they remember their own ideas instead of what was presented. Simply presenting correct information is not enough; students' misconceptions must be addressed to increase mental effort and learning. The most effective video showed an actor illustrating common misconceptions, which students then had to reconsider. This led to higher post-test scores and more reported mental effort, showing that confronting misconceptions can improve science learning from videos.

  • Khan Academy videos created by Sal Khan have over 2200 videos covering a wide range of subjects including math, history and science.
  • The author is skeptical that Khan Academy science videos promote meaningful learning for viewers.
  • The author conducted a study where students watched science explanation videos but showed little improvement in test scores, indicating they did not learn much.
  • Students often think they already know the material in science videos so they do not pay full attention and falsely remember their own ideas as what was presented.
  • Simply presenting correct scientific information is not effective as students do not recognize how it differs from their existing ideas.
  • Addressing common misconceptions in the videos helped increase students' mental effort and resulted in more learning.
  • Starting with misconceptions and then explaining the scientific ideas may be a more effective approach for science videos.
  • Khan Academy videos are a valuable resource but may not be that effective for those just starting to learn science as they do not challenge existing misconceptions.
  • Increased mental effort while watching the videos translated to more learning for the students.
  • The author tries to start Veritasium videos by addressing common misconceptions to tackle them effectively.
Khan Academy and the Effectiveness of Science Videos
Veritasium 2011

Comment by the gears to ascension (lahwran) on My tentative best guess on how EAs and Rationalists sometimes turn crazy · 2023-06-27T15:20:07.810Z · LW · GW

I'd suggest that's what's needed is a strong immune system against conformity and control which does not itself require conformity or control: a generalized resistance to agentic domination from another being, of any kind, and as part of it, a culture of habit around joining up with others who are skilled at resisting pressure in response to an attempted pressure, without that join up creating the problems it attempts to prevent. I'm a fan of this essay on this topic, as well as other writing from that group.

Comment by the gears to ascension (lahwran) on On how various plans miss the hard bits of the alignment challenge · 2023-06-27T15:05:16.809Z · LW · GW

I've missed one or more facts that link those threads, can you tell me which part of critch's approach stands out to you in this context and what made it do so? I agree that his is some of my favorite work, but it's not yet obvious to me that it actually checks all the boxes, in particular whether humans deploying ai will have values that directly contract usage of his insights. it also still isn't clear to me whether anything in his work helps with is/ought.

Comment by the gears to ascension (lahwran) on How tall is the Shard, really? · 2023-06-26T04:41:23.512Z · LW · GW

I've downvoted my own comment for being off topic humor but this ran through my head,

Comment by the gears to ascension (lahwran) on Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell? · 2023-06-25T20:29:23.167Z · LW · GW

Sure, I agree, the asteroid is going to kill us all. But it would be courteous to acknowledge that it's going to hit a poor area first, and they'll die a few minutes earlier. Also, uh, all of us are going to die, I think that's the core thing! we should save the poor area, and also all the other areas!

Comment by the gears to ascension (lahwran) on Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell? · 2023-06-25T20:28:05.123Z · LW · GW

Wait, whoops. Let me retrace identity here, sounds like a big mistake, sorry bout that Meg & Melanie when you see this post someday, heh.

edit: oops! the video I linked doesn't contain a Mitchell at all! It's Emily M. Bender and Timnit Gebru, both of whom I have a high opinion of for their commentary on near-term AI harms, and both of whom I am frustrated with for not recognizing how catastrophic those very harms could become if they were to keep on getting worse.

Comment by the gears to ascension (lahwran) on Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell? · 2023-06-25T20:03:19.152Z · LW · GW

Mitchell gebru and bender express their opinions on such things in more detail in the video I linked. Here's the overcompressed summary, which badly miscompresses the video, but which is a reasonable pitch for why you should watch it to get the full thing in order to respond to the points eloquently rather than using the facsimile. If you can put your annoyance at them missing the point about x-risk on hold and just try to empathize with their position having also been trying to ring alarm bells and being dismissed, and see how they're feeling like the x-risk crowd is just controlled opposition being used to dismiss their warnings, I think it could be quite instructive.

I also strongly recommend watching this video - timestamp is about 30sec before the part I'm referencing - where Bengio and Tegmark have a discussion with, among others, Tawana Petty, and they also completely miss the point about present-day harms. In particular, note that as far as I can tell she's not frustrated that they're speaking up, she's frustrated that they seem to be oblivious in conversation to what the present day harms even are; when she brings it up, they defend themselves as having already done something, which in my view misses the point because she was looking for action on present day harms to be weaved into the action they're demanding from the start. "Why didn't they speak up when Timnit got fired?" or so. She's been pushing for people like them to speak up for years, and she appears to feel frustrated that even when they bring it up they won't mention the things she sees as the core problems. Whether or not she's right that the present day problems are the core, I agree enthusiastically that present day problems are intensely terrible and are a major issue we should in fact acknowledge and integrate into plans to take action as best we can. This will remain a point of tension, as some won't want to "dilute" the issue by bringing up "controversial" issues like racism. But I'd like to at least zoom in on this core point of conflict, since it seems to get repeatedly missed. We need to not be redirecting away from this, but rather integrating. I don't know how to do that off the top of my head. Tegmark responds to this, but I feel like it's a pretty crappy response that was composed on the fly, and it'd be worth the time to ponder asynchronously how to respond more constructively.

  • "This has been killing people!"
  • "Yes, but it might kill all people!"
  • "Yes, but it's killing people!"
  • "Of course, sure, whatever, it's killing people, but it might kill all people!"

You can see how this is not a satisfying response. I don't pretend to know what would be.

Comment by the gears to ascension (lahwran) on Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell? · 2023-06-25T18:20:44.363Z · LW · GW

Edit #2: Severe identity error on my part! I seem to have been confusing who's who from memory badly, I made the video summary when I first saw the video and then turned that summary into a chattable bot today, and lost track of who was who in the process! I stand by the point made here, but it seems to be somewhat out of context, as it is merely related to the kind of thing Melanie Mitchell said by nature of being made by people who make similar points. I'm not going to delete this comment, but I'd appreciate folks wiping out all the upvotes.

I think that the AI x-risk crowd is continuing to catastrophically misunderstand a key point in the Mitchell AI Ethics researchers' view: their claim that there are, in fact, critical present-day harms from ai, they should be acknowledged, and they should in fact be solved very urgently. I happen to think that x-risk from AI is made of the same type of threat; but even if they weren't, I think that 1. Mitchell crowd AI Ethics crowd are being completely unreasonable in dismissing x-risk. you think that somehow capitalism is going to kill us all, and AI won't supercharge capitalism? what the hell are you smoking? 2. also, even if not for threat from capitalism, AI will do the same sort of stuff that makes capitalism bad but so much harder that even capitalism won't be able to take it.

We can't have people going "no, capitalism is fine actually" to someone whose whole point is that capitalist oppression is a problem. They'll just roll their eyes. Capitalism is unpopular actually!

Also, I don't really need to define the word for the type of argument one would have with MMitchell Gebru and Bender, she'd know what it means; but I would define the problem behaviors as optimization towards a numeric goal (increase investor payout) without regard for the human individuals in the system (workers, customers; even really investors don't get a good deal besides the money number going up). That's exactly what we're worried about with AI - but now without humans in the loop. Her claims that it's just hype are nonsense, she believes lecun's disinformation - and he's an agent of one of the nastiest capitalist orgs around!

Edit: here's a poe bot which embeds an awkward summary of a video interview they did. I have hidden the prompt and directed claude to not represent itself as being an accurate summary; however, claude is already inclined to express views similar to theirs (especially in contrast to ChatGPT, which does not), so I think claude could be an interesting debate partner, especially for those who look down on their views. Here's an example conversation where I pasted an older version of this comment I made before I realized I'd gotten identities wrong. I also strongly recommend watching the video it's based on, probably at 2x speed.

Comment by the gears to ascension (lahwran) on Lessons On How To Get Things Right On The First Try · 2023-06-21T00:10:20.152Z · LW · GW

For sure, though all that demonstrates is that it'll have to import one of the already trained nanotech-focused models

Comment by the gears to ascension (lahwran) on Lessons On How To Get Things Right On The First Try · 2023-06-20T09:23:04.235Z · LW · GW

I love this exercise, which completely kicked my butt. I would suggest more spoiler warnings, maybe a page length gap?

Comment by lahwran on [deleted post] 2023-06-19T18:57:42.154Z

I make them often enough. they don't get upvoted a ton usually.

Comment by the gears to ascension (lahwran) on Holly_Elmore's Shortform · 2023-06-19T02:02:18.606Z · LW · GW

every death and every forgotten memory is soul degradation. we won't be able to just reconstruct everyone exactly.

Comment by the gears to ascension (lahwran) on Holly_Elmore's Shortform · 2023-06-18T21:01:49.145Z · LW · GW

a pause lets some people ignore the pause and move in bad directions. we need to be able as a civilization to prevent the tugs on society to get sucked into AIs. the AIs of today will take longer to kill us all, but they'll still degrade our soul-data, compare YouTube recommender. authoritarian cultures that want to destroy humanity's soul might agree not to make bigger ai, but today's big ai is plenty. it's not like not pausing is massively better; nothing but drastically speeding up safety and liberty-generating alignment could save us

Comment by the gears to ascension (lahwran) on Holly_Elmore's Shortform · 2023-06-18T12:34:48.827Z · LW · GW

those sound like secondhand positions to me. not like those people were the originators of the reasoning. I think a pause is likely to guarantee we die though. we need to actually resist all bad directions, which a pause just lets some people ignore. pauses could never be enforced well enough to save us without an already save us grade ai.

Comment by the gears to ascension (lahwran) on A bunch of videos in comments · 2023-06-17T01:01:51.618Z · LW · GW
  • Many companies and platforms are becoming more restrictive and hostile towards developers, limiting what can be built on their sites. This reduces creativity and usefulness of the internet.
  • Major platforms are deleting old content and inactive accounts in mass, resulting in a loss of internet history and institutional memory.
  • Search engines are becoming less useful and filled with ads, clickbait content, and generic results that don't answer users' questions.
  • Search engine optimization practices have homogenized the internet and sterilized content, focusing more on Google's algorithm than the end user.
  • Generative AI responses in search have so far been plagued with issues and have not delivered the promised quality of results.
  • Google's manifest V3 changes will undermine the effectiveness of ad blockers and privacy extensions, benefiting Google's own business model.
  • The internet is moving towards a future where useful information is hidden behind paywalls and walled gardens, while public spaces are filled with AI-generated content.
  • Optimism for technological progress and the future of the internet is declining.
  • Corporations are putting the burden of their growth onto users, resulting in a worse experience.
  • Transparency, advocacy, and supporting independent creators can help ensure an open and user-friendly internet.

Comment by the gears to ascension (lahwran) on A bunch of videos in comments · 2023-06-16T20:56:20.579Z · LW · GW
  • Hollywood has been producing more sequels, prequels, remakes and films based on existing properties in recent years compared to original films. In 2021, only one of the top 10 grossing films was an original idea.
  • This trend is due to consolidation in the film industry where there are fewer major players and studios. Consolidation has also led to fewer films being produced.
  • The rise of streaming services like Netflix has led to vertical integration where one company controls both content production and distribution. This gives them more control over who profits.
  • Creators and independent producers have less control and profit due to the lack of a clear marketplace to sell their content. Writer pay has actually decreased in recent years.
  • The UK has been more successful at producing original content due to government regulations that require broadcasters to commission a portion of their programming from independent producers and allow them to retain secondary rights.
  • Small to mid-sized production companies have been able to thrive in the UK due to these regulations.
  • There are questions over whether the current streaming model in Hollywood is sustainable in the long run.
  • Wall Street investors want to see profitability after years of growth investment in streaming companies.
  • Creators are fighting for terms that give them the ability and incentive to make great original content.
  • The outcome of this fight will determine whether the next great original film gets made.

Comment by the gears to ascension (lahwran) on A bunch of videos in comments · 2023-06-16T20:55:28.194Z · LW · GW
  • Solar energy prices have dropped significantly in the last few decades, making it cheaper than fossil fuels like coal in most places.
  • However, investment and deployment of solar energy have stagnated despite the lower prices, as profitability remains an issue.
  • Companies like Shell have pledged to transition to renewables, but they have conditioned it on renewables delivering high profit margins of 8-12%, which is unlikely.
  • Returns on renewable energy projects are typically around 4-8%, much lower than what companies like Shell require.
  • Fossil fuel companies and asset management firms are not investing substantially in renewables, as profit remains the main driver rather than sustainability goals.
  • The parts of the solar business that are profitable involve manufacturing and mining, where exploitation and poor conditions remain.
  • Profit, not price, determines what gets produced. Without profit potential, the transition to renewables will not happen at scale.
  • Like water power in the past, solar energy is a cheaper source of energy but less profitable due to difficulties in privatization and exploitation.
  • The transition to renewables will likely come with major drawbacks as long as profit remains a requirement.
  • Systemic changes are needed to make the transition to renewables in a just and sustainable manner.

Comment by the gears to ascension (lahwran) on LLMs Sometimes Generate Purely Negatively-Reinforced Text · 2023-06-16T18:12:13.745Z · LW · GW

cool work! this feels related to - what are your thoughts on the connection?

Comment by the gears to ascension (lahwran) on Why libertarians are advocating for regulation on AI · 2023-06-15T21:56:39.075Z · LW · GW

For what it's worth, I think most people I know expect most professed values to be violated most of the time, and so they think that libertarians advocating for this is perfectly ordinary; the surprising thing would be if professed libertarians weren't constantly showing up advocating for regulating things. Show don't tell in politics and ideology. That's not to say professing values is useless, just that there's not an inconsistency to be explained here, and if I link people in my circles this post, they'd respond with an eyeroll at the possibility that if only they were more libertarian they'd be honest - because the name is most associated with people using the name to lie.

Comment by the gears to ascension (lahwran) on Meta-conversation shouldn't be taboo · 2023-06-15T20:42:05.961Z · LW · GW

fair nuff! yeah properly demonstrating online sounds really hard.

Comment by the gears to ascension (lahwran) on Meta-conversation shouldn't be taboo · 2023-06-15T20:29:00.556Z · LW · GW

it only works when you are able to reduce social anxiety by showing that they're welcome. someone who is cripplingly anxious typically wants to feel like they're safe, so showing them a clearer map to safety includes detecting the structure of their social anxiety first and getting in sync with it. then you can show them they're welcome in a way that makes them feel safer, not less. to do this requires gently querying their anxiety's agentic target and inviting the group to behave in ways that satisfy what their brain's overactivation wants.

Comment by the gears to ascension (lahwran) on Looking Back On Ads · 2023-06-15T16:43:52.317Z · LW · GW

I think the only content left would be the actual art. not the stuff that only deserves the name content.