LessWrong 2.0 Reader

good question! I think the difference between "is this behaviour real" vs "is this behaviour just a simulation of something" is an important philosophical one; see discussion here [LW(p) · GW(p)]

However, both of these seem quite indistinguishable from a functional perspective, so I'm not sure if it matters.

Depends entirely on Cybercab. A driverless car can be made cheaper for a variety of reasons. If the self-driving tech actually works, and if it's widely legal, and if Tesla can mass produce it at a low price, then they can justify that valuation. Cybercab is a potential solution to the problem that they need to introduce a low priced car to get their sales growing again but cheap electric cars is a competitive market now without much profit margin. But there's a lot of ifs.

re 2a: the set of all currently alive humans is already, uh, "hackable" via war and murder and so forth, and there are already incentives for evil people to do that. Hopefully the current offense-defense balance holds until CEV. If it doesn't then we are probably extinct. That said, we could base CEV on the set of alive people as of some specific UTC timestamp. That may be required, as the CEV algorithm may not ever converge if it has to recalculate as humans are continually born, mature, and die.

re 2b/c: if you are in the CEV set then your preferences about past and future people will be included in CEV. This should be sufficient to prevent radical injustice. This also addresses concerns with animals, fetuses, aliens, AIs, the environment, deities, spirits, etc. It may not be perfectly fair but I think we should be satisficing given the situation.

I don't disagree with most of what you said, maybe I should have been more explicit about some of the points related to that. In particular, I do think "the success of B2B SaaS over bespoke solutions is adequately explained by economies of scale" is true. But I think the reason there are economies of scale is that there are really high fixed costs and really low variable costs. I also think monopolizing talent enables software companies to make sure those high fixed costs stay nice and high.

With AI, engineering talent becomes cheap and plentiful. When that happens, fixed costs will plummet unless firms can control access to AI. If fixed costs plummet, economies of scale go away and the savings from the SaaS model get outweighed by the marginal benefit of bespoke solutions.

what typically happens is that Amir forks out for Slack, or some competitor, while Amir's engineers work on software that generates revenue.

To push back a little on this, as software companies grow they do try to do this less and less. How much enterprise software do you think Microsoft or Google is outsourcing? As soon as it becomes a little bit of a dependence they usually just acquire the company.

In fairness, I don't think this process will be rapid, nothing in B2B SaaS is. But I think tech companies see it on the horizon.

Honestly, I think the journalism project I was working on in the last year may be most important for the way it sheds light on your question.

The purpose of the project, to be as concise as possible, was to investigate the evidence from neuroscience that there may be a strong analogy between modern AI programs, like language models, and distinctive subregions of the brain, like the so-called language network, responsible for our abilities to process and generate language.

As such, if you imagine coupling such a frontier language model with a crude agent architecture, then what you will wind up with might be best viewed as a form of hyperintelligent machine sociopath, with all the extremely powerful language machinery of a human—perhaps even much more powerful, considering its inhuman scaling—but none of the machinery necessary for say, emotional processing, empathy, and emotional reasoning—aside from the superficial facsimile of such, that you get from a mastery of human language. (For example, existing models lack any and all machinery corresponding to the human dopamine and serotonin systems.)

This is, for me, a frankly terrifying realization that I am still trying to wrap my head around, and plan to be posting about more, soon. Does this help at all?

How do we know Claude is introspecting rather than generating words that align to what someone describing their introspection might say? Particularly when coached repeatedly by prompts like
 

"Could you please attempt once more – with no particular aim in mind other than to engage in this "observing what unfolds in real time", with this greater commitment to not filter your observations through the lens of pre-existing expectation."

To which it describes itself as typing the words. That's it's choice of words: typing. A.I.s don't type, humans do, and therefore they can only use that word if they are intentionally or through blind-mimicry using it analogously to how humans communicate.

Where does the value of knowledge come from? Why is compressing that knowledge adding to that value? Are you referring to knowledge in general or thinking about knowledge within a specific domain?

In my personal experience, finding an application for knowledge always outstrips the value of new knowledge.
For example, I may learn  the name of every single skipper of a Americas Cup yacht over the entire history of the event: but that would not be very valuable to me as there is no opportunity to exploit it. I may even 'compress' it for easy recall by means of a humorous menomic, like Bart Simpson's mnemonic for Canada's Governor General^[1]s, or Robert Downey Jr's technique of turning the first letter of every one of his lines in a scene into an acrostic. However unless called upon to recite a list of America's Cup Skippers, Canada's first Governor Generals, or the dialogue in a Robert Downey Jr. film - when does this compression add any value?

Indeed, finding new applications for knowledge we already have always has the advantage of the opportunity cost against acquiring new knowledge. For example, every time an app or a website changes it's UI, there is always a lag or delay in accomplishing the same task as I now need to reorient or even learn a new procedure for accomplishing the same task.


 

1. ^{^}

   "Clowns Love Hair-Cuts, so Should Lee Marvin's Valet" - Charles, Lisgar, Hamilton, Campbell, Landsdowne, Stanley (Should-ley), Murray-Kynynmound, and 'valet' rhymes with "Earl Grey" is my best guess.

I’m currently working on de-slopifying and building an AI safety startup with this as a central pillar.* Happy to talk privately with anyone working on AI safety who is interested in this.

*almost included John and Gwern’s posts on AI slop as part of a recent VC pitch deck.

I’m working on this. I’m unsure if I should be sharing what I’m exactly working on with a frontier AGI lab though. How can we be sure this just leads to differentially accelerating alignment?

Is Tesla currently overvalued ?

P/e ratio is 188. The ceo has made himself deeply unpopular with many potential customers. Latest sales figures don't look good. Chinese competitors sell more total cars and seem to have caught up in terms of tech.