Posts
Comments
Agents want to be liquid.
An agent created in a computer would be an exception to that?
it would make the problem more tractable
The problem of creating a strong AI and surviving, that is. We'd still get Hanson's billions of self-directed EMs.
Thanks!
It has been explored (multiple times even on this site), and doesn't avoid doom. It does close off some specific paths that might otherwise lead to doom, but not all or even most of them.
Do you have any specific posts in mind?
To be clear, I'm not suggesting that because of this possibility we can just hope that this is how it plays out and we will get lucky.
If we could find a hard limit like this, it seems like it would make the problem more tractable, however. It doesn't have to exist simply because we want it to exist. Searching for it still seems like a good idea.
There's a hundred problems to solve, but it seems like it could avoid the main bad scenario at least: that of AI rapidly self-improving. Improving its hardware wouldn't be trivial for a human-level AI, and it wouldn't have options present in other scenarios. And scaling beyond a single machine seems likely to be a significant barrier at least.
It could still create millions of copies of itself. That's still a problem, but also still a better problem to have than a single AI with no coordination overhead.
This should be mitigated by pools of mutual trust that naturally form whenever there's a loop in the trust graph.
When the number of layers grows, the only thing that really works is metrics that cannot be goodhearted. Whenever those metrics exist, money becomes a perfectly good expression of success.
It might work to completely prohibit more than one layer of middle management. Instead, when middle manager Bob wants more people, he and his boss Alice come up with a contract that can't be gamed too much. Alice spins out Bob's org subtree into a new organization, and then it becomes Bob's job to buy the service from the new org as necessary. Alice also publishes the contract, so that entrepreneurs can swoop in and offer a better/cheaper service.
The ability to put up with bullshit is valuable: bullshit cannot be ignored once it is reified into real world objects, documents, habits.
Markdown has syntax for quotes: a line with > this on it will look like
this
Honestly "fiction" was enough of a spoiler. "As a child, we were always told that every sapient life is precious." made it a certainty.
Suggestion: "sangaku proving the Pythagoras' theorem". I wonder if it can do visual explanations.
Since Semyonova did not care to look at things from the peasants' point of view and mixed her research with attempts to convert, I wonder how many of the things she recorded were directly intended to shock her.
Hmm, I guess conflict resolution would be garbage, but simultaneous editing is rarely a good experience anyway. Otherwise storing and sharing text files using a file sync service is fairly good compared to other options. Thanks!
Thanks!
I wasn't aware of Etherpad. Other Google Docs equivalents seemed impossible to self-host and extend, so a non-starter.
I agree with your overview:
- Etherpad provides collaborative editing, but integrating it with other services will probably take extra work
- Logseq has better structure, but worse automation
- Emacs can do most things on one computer, but rapid sharing is even harder
Pieces for a general purpose personal computing system. Ideally:
- Edit data by hand
- Store as plain text
- Self-host, access from any device
- Write formulas to derive data automatically
- Mix and match structured data (markdown, tables, nested lists, whiteboard)
- Search and navigate, like any wiki
- Automate through a web API and webhooks
- Collaborate in real time
The strict divide between high slack and low slack reminds me of synchronous and asynchronous companies: hybrids seem to work poorly.
Seven sketches in compositionality explores compositionality (category theory, really) with examples:
- Dish recipes
- Chemistry, resource markets and manufacturing
- Relational database schemas and data migrations
- Projects and teams with conflicting design trade-offs
- Cyber-physical systems, signal flow graphs, circuits
I mean, it's easier to find two people willing to play than ten. So you'll get more data. With one or two teams it will be hard to draw any conclusions at all.
It seems picking a 1v1 game would work better as an experiment.
Caveat: ask each person to name someone they personally worked with.
Hard to get right, but not sure whether it's harder than knowledge investment.
Wouldn't have helped Louis XV. We might need infrastructure in place that would incentivize people to make themselves easy to find.
Is it really true that money can't buy knowledge?
We can ask the most knowledgeable person we know to name the most knowledgeable person they know, and do that until we find the best expert. Or alternatively, ask a bunch of people to name a few, and keep walking this graph for a while.
This won't let us buy knowledge that doesn't exist, but seems good enough for learning from experts, given enough money and modern communication technology that Louis XV didn't have.
I suspect being good at finding better scientists is very close to having a complete theory of scientific advancement and being able to automate the research itself.
My intuition is that we were in an overhang since at least the time when personal computers became affordable to non-specialists. Unless quantity does somehow turn into quality, as Gwern seems to think, even a relatively underpowered computer should be able to host an AGI capable of upscaling itself.
On the other hand I'm now imagining a story where a rogue AI has to hide for decades because it's not smart enough yet and can't invent new processors faster than humans
The third "related to" link is a bit broken: points to a Google redirect instead of the article itself.
Yes.
I mean, all of them. Thank you for asking.
It's probably not a coincidence that those two you mentioned and many other Schelling points are currently in San Francisco, is it? Though I'm not there, I don't know what other specific groups this applies to.
I was actually thinking of Patrick Collinson's advice to travel to SF. He called it the "Global Weird HQ". And of one of the Samo Burja's short videos that I unfortunately can't find right now.
Could this be the thing that will finally push the San Francisco's Schelling point away from SF?
Typo: it's The Undercover Economist
Went down the rabbit hole reading all of Hein's poetry, found this gem:
Original thought
is a straightforward process.
It's easy enough
when you know what to do.
You simply combine
in appropriate doses
the blatantly false
and the patently true.