Posts

Offering service as a sensayer for simulationist-adjacent spiritualities. 2024-05-22T18:52:05.576Z
[Cosmology Talks] New Probability Axioms Could Fix Cosmology's Multiverse (Partially) - Sylvia Wenmackers 2024-04-14T01:26:38.515Z
All About Concave and Convex Agents 2024-03-24T21:37:17.922Z
Do not delete your misaligned AGI. 2024-03-24T21:37:07.724Z
Elon files grave charges against OpenAI 2024-03-01T17:42:13.963Z
Verifiable private execution of machine learning models with Risc0? 2023-10-25T00:44:48.643Z
Eleuther releases Llemma: An Open Language Model For Mathematics 2023-10-17T20:03:45.419Z
A thought about the constraints of debtlessness in online communities 2023-10-07T21:26:44.480Z
The point of a game is not to win, and you shouldn't even pretend that it is 2023-09-28T15:54:27.990Z
Cohabitive Games so Far 2023-09-28T15:41:27.986Z
Do agents with (mutually known) identical utility functions but irreconcilable knowledge sometimes fight? 2023-08-23T08:13:05.631Z
Apparently, of the 195 Million the DoD allocated in University Research Funding Awards in 2022, more than half of them concerned AI or compute hardware research 2023-07-07T01:20:20.079Z
Using Claude to convert dialog transcripts into great posts? 2023-06-21T20:19:44.403Z
The Gom Jabbar scene from Dune is essentially a short film about what Rationality is for 2023-03-22T08:33:38.321Z
Will chat logs and other records of our lives be maintained indefinitely by the advertising industry? 2022-11-29T00:30:46.415Z
[Video] How having Fast Fourier Transforms sooner could have helped with Nuclear Disarmament - Veritaserum 2022-11-03T21:04:35.839Z
The Mirror Chamber: A short story exploring the anthropic measure function and why it can matter 2022-11-03T06:47:56.376Z
I just watched the Open C3 Subcommittee Hearing on Unidentified Aerial Phenomena (UFOs). Here's a succinct summary and commentary + some background 2022-05-18T04:15:11.681Z
Alex Tabarrok advocates for crowdfunding systems with *Refund Bonuses*. I think this might be a natural occurrence of a money pump against Causal Decision Theory pledgers 2022-03-14T07:27:06.955Z
Grabby Aliens could be Good, could be Bad 2022-03-07T01:24:43.769Z
Would (myopic) general public good producers significantly accelerate the development of AGI? 2022-03-02T23:47:09.322Z
Are our community grouphouses typically rented, or owned? 2022-03-02T03:36:58.251Z
We need a theory of anthropic measure binding 2021-12-30T07:22:34.288Z
Venture Granters, The VCs of public goods, incentivizing good dreams 2021-12-17T08:57:30.858Z
Is progress in ML-assisted theorem-proving beneficial? 2021-09-28T01:54:37.820Z
Auckland, New Zealand – ACX Meetups Everywhere 2021 2021-08-23T08:49:53.187Z
Violent Unraveling: Suicidal Majoritarianism 2021-07-29T09:29:05.182Z
We should probably buy ADA? 2021-05-24T23:58:05.395Z
Deepmind has made a general inductor ("Making sense of sensory input") 2021-02-02T02:54:26.404Z
In software engineering, what are the upper limits of Language-Based Security? 2020-12-27T05:50:46.772Z
The Fermi Paradox has not been dissolved - James Fodor 2020-12-12T23:18:32.081Z
Propinquity Cities So Far 2020-11-16T23:12:52.065Z
Shouldn't there be a Chinese translation of Human Compatible? 2020-10-09T08:47:55.760Z
Should some variant of longtermism identify as a religion? 2020-09-11T05:02:43.740Z
Design thoughts for building a better kind of social space with many webs of trust 2020-09-06T02:08:54.766Z
Investment is a useful societal mechanism for getting new things made. Stock trading shares some functionality with investment, but seems very very inefficient, at that? 2020-08-24T01:18:19.808Z
misc raw responses to a tract of Critical Rationalism 2020-08-14T11:53:10.634Z
A speculative incentive design: self-determined price commitments as a way of averting monopoly 2020-04-28T07:44:52.440Z
MakoYass's Shortform 2020-04-19T00:12:46.448Z
Being right isn't enough. Confidence is very important. 2020-04-07T01:10:52.517Z
Thoughts about Dr Stone and Mythology 2020-02-25T01:51:29.519Z
When would an agent do something different as a result of believing the many worlds theory? 2019-12-15T01:02:40.952Z
What do the Charter Cities Institute likely mean when they refer to long term problems with the use of eminent domain? 2019-12-08T00:53:44.933Z
Mako's Notes from Skeptoid's 13 Hour 13th Birthday Stream 2019-10-06T09:43:32.464Z
The Transparent Society: A radical transformation that we should probably undergo 2019-09-03T02:27:21.498Z
Lana Wachowski is doing a new Matrix movie 2019-08-21T00:47:40.521Z
Prokaryote Multiverse. An argument that potential simulators do not have significantly more complex physics than ours 2019-08-18T04:22:53.879Z
Can we really prevent all warming for less than 10B$ with the mostly side-effect free geoengineering technique of Marine Cloud Brightening? 2019-08-05T00:12:14.630Z
Will autonomous cars be more economical/efficient as shared urban transit than busses or trains, and by how much? What's some good research on this? 2019-07-31T00:16:59.415Z
If I knew how to make an omohundru optimizer, would I be able to do anything good with that knowledge? 2019-07-12T01:40:48.999Z

Comments

Comment by mako yass (MakoYass) on K-complexity is silly; use cross-entropy instead · 2024-06-12T23:18:19.623Z · LW · GW

where  denotes logarithm base , aka the negative of the (base 2) logarithm

I've never heard of  before and I'm having difficulty looking it up. Is it distinct from , or ?

Edit: No, it's equivalent to , but this is only confirmed later in the post. Okay, where does this obscuritan "" notation come from?

Comment by mako yass (MakoYass) on Two easy things that maybe Just Work to improve AI discourse · 2024-06-09T06:17:23.117Z · LW · GW

I think yall will be okay if you make sure your twitter account isn't your primary social existence, and you don't have to play twitter the usual way. Write longform stuff. Retweet old stuff. Be reasonable and conciliatory while your opponents are being unreasonable and nasty, that's how you actually win.

Remember that the people who've fallen in deep and contracted twitter narcissism are actually insane, It's not an adaptive behavior, they're there to lose. Every day they're embarrassing themselves and alienating people and all you have to do is hang around, occasionally point it out, and be the reasonable alternative.

Comment by mako yass (MakoYass) on Access to powerful AI might make computer security radically easier · 2024-06-08T22:46:10.266Z · LW · GW

Remember that you were only proposing discreet auditing systems to mollify the elves. They think of this as a privacy-preserving technology, because it is one, and that's largely what we're using it for.

Though it's also going to cause tremendous decreases in transaction costs by allowing ledger state to be validated without requiring the validator to store a lot of data or replay ledger history. If most crypto investors could foresee how it's going to make it harder to take rent on ledger systems, they might not be so happy about it.

Oh! ""10x" faster than RISC Zero"! We're down to a 1000x slowdown then! Yay!

Previous coverage btw.

Comment by mako yass (MakoYass) on Access to powerful AI might make computer security radically easier · 2024-06-08T21:34:23.921Z · LW · GW

However, one big challenge of building this is how two adversarial parties could ever gain enough confidence to allow such a hardware/software package into a secure facility, especially if it's whole point is to have a communication channel to their adversary.

Isn't it enough to constrain the output format to so that no steganographic leaks would be possible? Wont the counterparty usually be satisfied just with an hourly signal saying either "Something is wrong" (encompassing "Auditor saw a violation" / "no signal, the host has censored the auditor's report" / "invalid signal, the host has tampered with the auditor system" / "auditor has been blinded to the host's operations, or has ascertained that there are operations which the auditor cannot see") or "Auditor confirms that all systems are nominal and without violation."?

The host can remain in control of their facilities, as long as the auditor is running on tamperproof hardware. It's difficult to prove that a physical device can't be tampered with, it may be possible to take some components of the auditor even further and run them in a zero knowledge virtual machine, which provides a cryptographic guarantee that the program wasn't tampered with, so long as you can make it lithe enough to fit (zero knowledge virtual machines currently run at a 10,000x slowdown, though I don't think specialized hardware for them is available yet, crypto may drive that work), though a ZKVM wont provide a guarantee that the inputs to the system aren't being controlled, the auditor is monitoring inputs of such complexity — either footage of the real world or logs of a large training run — that it may be able to prove algorithmically to itself that the sensory inputs weren't tampered with either and the algorithm does have a view into the real world (I'm contending that even large state actors could not create Descarte's evil demon).

Comment by mako yass (MakoYass) on How do you shut down an escaped model? · 2024-06-02T23:55:37.337Z · LW · GW

AWS probably wouldn't do anything, assuming a human has signed their name to the account, there might be policy against doing anything. But if you called the NSA/cybercom instead, they probably would do something given that they've been dealing with botnets and foreign hackers for decades.

Comment by mako yass (MakoYass) on MIRI 2024 Communications Strategy · 2024-06-02T04:23:32.352Z · LW · GW

I guess in this case I'm arguing that it's accidentally, accidentally, productive.

Comment by mako yass (MakoYass) on MIRI 2024 Communications Strategy · 2024-06-01T23:53:48.453Z · LW · GW

Agreed but initially downvoted due to being obviously unproductive, but then upvoted for being an exquisite proof by absurdity about what's productive: This is the first time I have seen clearly how good communication must forbid some amount of nuance.

The insight: You have a limited amount of time to communicate arguments and models; methods for reproducing some of your beliefs. With most people, you will never have enough time to transmit our entire technoeschatology or xenoeconomics stuff. It is useless to make claims about it, as the recipient has no way of checking them for errors or deceptions. You can only communicate approximations and submodules. No one will ever see the whole truth. (You do not see the whole truth. Your organization, even just within itself, will never agree about the whole truth.)

Comment by mako yass (MakoYass) on MIRI 2024 Communications Strategy · 2024-06-01T23:14:50.922Z · LW · GW

We are not investing in grass-roots advocacy, protests, demonstrations, and so on.

I like this, I'd be really interested to ask you, given that you're taking a first principles no bullshit approach to outreach, what do you think of protest in general?

Every protest I've witnessed seemed to be designed to annoy and alienate its witnesses, making it as clear as possible that there was no way to talk to these people, that their minds were on rails. I think most people recognize that as cult shit and are alienated by that.

A leftist friend once argued that protest is not really a means, but a reward, a sort of party for those who contributed to local movementbuilding. I liked that view. Perhaps we should frame our public gatherings to be closer to being that. If there is to be chanting of slogans, it must be an organic ebullition of the spirit of a group that was formed around some more productive purpose than that, maybe the purpose of building inclusive networks for shared moral purpose? (EA but broader?)

Comment by mako yass (MakoYass) on Cohabitive Games so Far · 2024-05-30T21:33:57.179Z · LW · GW

Hmm but I think it'll be solved like 5 years from now so I'd be eager to start working on VR boardgames/social role playing games today. I believe jon blow, when he says it usually takes that long to figure out what a really fresh kind of game wants to be.

And the first good VR RPGs are going to be really culturally impactful.

but there's also more design language built around 2D UIs. I still think there's a ton of unexplored design space around "tabletop games" that make use of modern web flows

Oh? I guess asynchrony is one of the things in that design language. And a web based game could leave players in a groupchat/forum after the game, they could make friends there. I used to play Neptune's Pride, which I guess is an example of that kind of game. It was a... good game... I think. It was emotionally brutal.

Actually, that experience with Neptune's Pride is probably a large part of the reason I want to make cohabitive games today. It was a game that forced you to forge friendships that were all absolutely destined to collapse. I forget whether there was any benefit to coming second or third, but if there was it wouldn't have resonated with the narrative, it was a war of domination, the mechanics of the gameworld were such that anyone with an advantage would be able to grow their advantage until there was nothing left for anyone else (and there was no flourishing along the way, just war) so second or third wouldn't have really meant anything within the narrative of the game.
Honest negotiation wasn't possible, every message we sent was laced with deception, and often the opponent would pick up on that and not admit it and that would be another deception of their own. This is the norm in diplomacy games. And I guess I became aware of how ruinously that misrepresents the diplomatic games we're playing in the real world (at least, post WWII, it is a misrepresentation. Maybe when nationalism was more of a thing our game was really like this. But today global culture is getting everywhere.).

Comment by mako yass (MakoYass) on Hardshipification · 2024-05-29T21:16:09.563Z · LW · GW

I guess there's a subcultural aspect in how comfortable people feel with declining to answer questions: In subcultures where people can (or know to) just say "don't worry about it" and people don't get offended by that, questions are free.

But for the most part I think whether people are happy or annoyed with being asked a question comes down to how many times they have been asked. Asked zero times → very happy, like summer rain. Asked 30 times → not so much, but idk maybe you can mitigate the annoyance by copy and pasting something/publishing a statement.

Comment by mako yass (MakoYass) on Hardshipification · 2024-05-29T05:16:01.952Z · LW · GW

I had fun by ridiculing the entire situation, waiting it out while asking the doctors questions like:

  • Can identical twin brothers transmit testicle cancer through sodomy?
  • Can I keep my surgically removed ball? (For storing in a jar of formaldehyde)
  • Does hair loss from chemotherapy proceed in the same stages as male pattern baldness?

If you were hoping we weren't going to want to know the answers to these questions, I gotta disappoint ya

Comment by mako yass (MakoYass) on Hardshipification · 2024-05-29T05:09:22.704Z · LW · GW

I think your response mostly didn't make sense. "What's that like" imposes a greater burden on you by requiring you to explain something again and again, and bluntly, I'm only going to be interested in that if me knowing helps you in some way, and I don't see why it would, but maybe it would, I wouldn't know, so it makes sense for me to just express willingness to help if needed, and that starts with "If you need to talk to someone, I’m here for you".

I get frustrated with imaginary social scripts too, but that part of the script is fine.

Comment by mako yass (MakoYass) on Cohabitive Games so Far · 2024-05-28T23:43:19.812Z · LW · GW

I might be interested in doing that, but to clarify; the reason you need VR for board games is that VR facilitates the same kind of sense of shared presence and quality of audio conversation that playing in person does. I'm fairly sure VR boardgaming is going to be better than physical boardgaming (larger tables, no upkeep, more immersion, venues that can be teleported to from home so more repeat interactions so more legacy games are possible). That is the sense in which you need VR. But for prototyping, sure, developers have pre-existing buyin and can put up with the limitations of 2d.

Comment by mako yass (MakoYass) on Cohabitive Games so Far · 2024-05-28T23:01:05.329Z · LW · GW

That's interesting thanks, but I hope you can understand how keeping all of the individual goals secret would make it much harder to practice negotiation. It's okay (great, even) if there's some way of exposing the secret goals. In most games with secret goals that doesn't happen during the game, but since, iirc, it's a legacy game, maybe players tend to figure out each others' secret goals as the campaign goes on. Is that the case? If so, I'd be very interested in seeing that stuff, and the late-game.

Comment by mako yass (MakoYass) on Is CDT with precommitment enough? · 2024-05-26T22:05:28.536Z · LW · GW

(only including downstream effects of your policy)

I'm not sure I know what you mean by this, but if you mean causal effects, no, it considers all pasts, and all timelines.

(A reader might balk, "but that's computationally infeasible", but we're talking about mathematic idealizations, the mathematical idealization of CDT is also computationally infeasible. Once we're talking about serious engineering projects to make implementable approximations of these things, you don't know what's going to be feasible.)

Comment by mako yass (MakoYass) on simeon_c's Shortform · 2024-05-26T01:05:18.050Z · LW · GW

I don't know how to convince anyone of this but just having low network latency to California may be worth a lot after VR gets good. Physical access will matter much less than the quality of your network connection, the delay between speaking and seeing and hearing others' reactions, and timezone overlap.

I'm not sure how to put a price on these things. People seem to be able to adjust to the delay in conversation, but adjusting requires becoming comfortable with talking over people sometimes, it can get noisy. The timezone overlap issue also seems important, if you don't get off work/have your mid day break at the same time as others, you get left out of things, but people can adjust their sleep/wake time (I certainly can) and synch with a remote timezone so idk.

Comment by mako yass (MakoYass) on Is CDT with precommitment enough? · 2024-05-25T23:11:27.889Z · LW · GW

I note that in the cooperative bargaining domain, a CDT agent will engage in commitment races, using the commitment mechanism to turn itself into a berzerker, a threatmaker. If they're sharing a world with other CDT agents, that is all they will do. Whoever's able to constitutionalize first will make a pre-commitment like "I'll initiate a nuclear apocalypse if you don't surrender all of your land to us."

If they're sharing the world with UDT agents, they will be able to ascertain that those sorts of threats will be ignored (reflected in the US's principle of "refusing to negotiate with terrorists"), and recognize that it would just lead to MAD with no chance of a surrender deal. I think commitment mechanisms only lead to good bargaining outcomes if UDT agents already hold a lot of power.

Comment by mako yass (MakoYass) on Is CDT with precommitment enough? · 2024-05-25T23:10:11.497Z · LW · GW

A general pre-commitment mechanism is just self-modification. CDT with self-modification has been named "Son of CDT", and seems to have been discussed most recently on this arbital article.

It behaves like a UDT agent about everything after the modifications are made, but not about anything that was determined before then.
I'm not aware of any exploits for that. I suspect that there will be some.

Comment by mako yass (MakoYass) on What comes after Roam's renaissance? · 2024-05-24T05:36:46.547Z · LW · GW

It seems like the anki for notetaking is https://www.remnote.com/ ? Suggested here https://www.lesswrong.com/posts/CoqFpaorNHsWxRzvz/what-comes-after-roam-s-renaissance?commentId=9LY2cTTAKbaJsBCNf

Comment by mako yass (MakoYass) on What comes after Roam's renaissance? · 2024-05-24T05:22:37.874Z · LW · GW

I think the whole roam fad may have been an example of a phenomenon I'm noticing where these fundamentally social apes that we are cannot conceive of an activity failing to become social, the idea that what I see isn't what others see, or the idea that there are activities that can't be shared, is unnatural and unintuitive to us. Roam would have made a lot of sense as a new kind of social network oriented around collaboratively, iteratively building evergreen knowledge together, and it vibed like it was that, and I think that's why there was so much excitement, but due to the way Roam works and the shoddy way it was built, it actually couldn't grow into that. The human sense for social fun isn't smart enough to notice that it wasn't going to happen and so it took a long time to wind down.

Specifically, multiplayer roam has no notification system, and it's siloed into groups. They don't have per-block/per-page read/write permissions, meaning that you kind of have to be sharing all of the notes in a graph or none of them, and as far as I'm aware there are no communities that makes sense for. Most peoples' notes end up being mostly private, rough or intimate.

But massively multiplayer roam as a concept is totally possible, it just has to be built in a different way, with different systems.
It's one of the things I want to build.

Comment by mako yass (MakoYass) on What comes after Roam's renaissance? · 2024-05-24T05:06:49.049Z · LW · GW

From what it sounds like, the Roam community would never stand for that

Not sure exactly what's meant by roam community but Subconscious are thinking about that stuff. I forget what exactly they're doing with it, I haven't been paying attention to it (maybe I'm the roam community), but I think it was something like... it goes and surfaces something from the past to remind you of it to make sure things you might have forgotten about get linked together. (Gosh that would be bad for my productivity.)

Comment by mako yass (MakoYass) on What comes after Roam's renaissance? · 2024-05-24T04:58:26.043Z · LW · GW

At least for me—and most of the people I know—we got a garbage dump full of crufty links and pieces of text we hardly ever revisit.

For me, some stuff fell into disrepair when I realized no one else was going to read it or add to it, which I think is mostly due to a bad model for shared use, but a lot of other stuff kept growing and turned out to be really transformative. I've continued using it as a very good, very long todo list, a pensieve to relinquish and defer burdensome thoughts into, to sort every possible idea I could be pursuing and to prioritize the one currently most important thing.

Some other pages that've remained active over the years:

  • recipes
  • tasteweb engineering notes, pitch concepts
  • UFO stuff (some interesting evidence, but mostly debunkings). I guess this could add up to a post eventually.
  • "people", a list of various things to talk about with people next time we're in a room together, if we ever are. The uncertainty, that we may not meet, is the reason I think it's important to externalize these thoughts.
Comment by mako yass (MakoYass) on What comes after Roam's renaissance? · 2024-05-24T04:42:07.467Z · LW · GW

while Subconscious has taken the blockchain/protocol approach.

Subconscious isn't blockchain. The noosphere protocol has the foundational features of a smart contract system, but it doesn't seem quite secure enough against inconsistencies/double-spends to run finance (and that's not an aspiration they seem to have), and the sacrifices it makes will make it cheap and easy to scale horizontally/federate. The protocol it's most similar to is bluesky's ATProtocol. I actually wanted to complain to them that they're too similar and they should merge, but noosphere started developing before ATProtocol existed, so who can blame them really.

Although a part of me does wonder, if a protocol like atproto took off, whether people would start doing finance on it, security be damned, and then legal enforcement and auditing would come along and it would be effectively secured through international social technologies and trusted computing instead of cryptographic technologies and game theory and we might just end up in the same place. It would be a weird future.

Comment by mako yass (MakoYass) on mesaoptimizer's Shortform · 2024-05-19T21:51:15.989Z · LW · GW

That isn't anyone's first/preferred plan. I assure you everyone born in a liberal democracy has considered another plan before arriving at that one.

Comment by mako yass (MakoYass) on We are headed into an extreme compute overhang · 2024-05-15T22:10:13.687Z · LW · GW

I'm not sure why people would think LLMs understand their own output, we know they're not up to spotting sometimes human-obvious inconsistencies in it (as soon as they are, things will start moving very quickly).

Comment by mako yass (MakoYass) on Cooperation is optimal, with weaker agents too  -  tldr · 2024-05-08T22:13:56.401Z · LW · GW

:( that isn't what cooperation would look like. The gazelles can reject a deal that would lead to their extinction (they have better alternatives) and impose a deal that would benefit both species.

Cooperation isn't purely submissive compliance.

Comment by mako yass (MakoYass) on Cohabitive Games so Far · 2024-05-08T20:30:40.649Z · LW · GW

(I'm aware of most of these games)

I made it pretty clear in the article that it isn't about purely cooperative games. (Though I wonder if they'd be easier to adapt. Cooperative + complications seems closer to the character of a cohabitive game than competitive + non-zero-sum score goals do...)

Gloomhaven seems, and describes itself as being a cooperative game. What competitive elements are you referring to?

The third tier is worth talking about. I think these sorts of games might, if you played them enough, teach the same skills, but I think you'd have to play them for a long time. My expectation is that basically all of them end with a ranking? as you said, first, second, third. The ranking isn't scored, (ie, we aren't told that being second is half as good as being first) so there's not much clarity about how much players should value them, which is one obstacle to learning. Rankings also keep the game zero sum on net, and zero sum dynamics between first and second or between first and the alliance have the focus of your attention most of the time. The fewer or the more limited mutually beneficial deals are, the less social learning there will be. Zero sum dynamics need to be discussed in cohabitive games, but the games will support more efficient learning if they're reduced.
And there really are a lot of people who think that the game that humans are playing in the real world is zero sum, that all real games are zero sum, so, I also suspect that these sorts of games might never teach the skill, because to teach the skill you have to show them a way out of that mindset, and all they do is reinforce it.

competitive [...] not usually permanent alliances are critical to victory: Diplomacy, Twilight Imperium (all of them), Cosmic Encounter

This category is really interesting, because the alliances expire and have to be remade multiple times per game, and I've been meaning to play some games from this category, but they're also a lot more foggy, the agreements are of poor quality, they invite only limited amounts of foresight and social creativity, in contrast, writing good legislation in the real world seems to require more social creativity than we can currently produce.

Comment by mako yass (MakoYass) on quila's Shortform · 2024-05-08T06:57:15.749Z · LW · GW

Imagining a pivotal act of generating very convincing arguments for like voting and parliamentary systems that would turn government into 1) an working democracy 2) that's capable of solving the problem. Citizens and congress read arguments, get fired up, problem is solved through proper channels.

Comment by mako yass (MakoYass) on Industrial literacy · 2024-05-07T01:32:01.603Z · LW · GW

Yeah.

Well that's the usual reason to invoke it, I was more talking about the reason it lands as a believable or interesting explanation.

Notably, Terra Ignota managed to produce a mcguffin by having the canner device be extremely illegal by having even knowledge of its existence be a threat to the world's information infrastructure, so I'd guess that's the reason, iirc, they only made one.

Comment by mako yass (MakoYass) on KAN: Kolmogorov-Arnold Networks · 2024-05-05T18:43:56.091Z · LW · GW

I'm guessing they mean that the performance curve seems to reach much lower loss before it begins to trail off, while MLPs lose momentum much sooner. So even if MLPs are faster per unit of performance at small parameter counts and data, there's no way they will be at scale, to the extent that it's almost not worth comparing in terms of compute? (which would be an inherently rough measure anyway because, as I touched on, the relative compute will change as soon as specialized spline hardware starts to be built. Due to specialization for matmul|relu the relative performance comparison today is probably absurdly unfair to any new architecture.)

Comment by mako yass (MakoYass) on KAN: Kolmogorov-Arnold Networks · 2024-05-04T04:30:11.194Z · LW · GW

Theoretically and em-
pirically, KANs possess faster neural scaling laws than MLPs

What do they mean by this? Isn't that contradicted by this recommendation to use the an ordinary architecture if you want fast training:

A section from their diagram where they disrecommend KANs if you want fast training

It seems like they mean faster per parameter, which is an... unclear claim given that each parameter or step, here, appears to represent more computation (there's no mention of flops) than a parameter/step in a matmul|relu would? Maybe you could buff that out with specialized hardware, but they don't discuss hardware.

One might worry that KANs are hopelessly expensive, since each MLP’s weight
parameter becomes KAN’s spline function. Fortunately, KANs usually allow much smaller compu-
tation graphs than MLPs. For example, we show that for PDE solving, a 2-Layer width-10 KAN
is 100 times more accurate than a 4-Layer width-100 MLP (10−7 vs 10−5 MSE) and 100 times
more parameter efficient (102 vs 104 parameters) [this must be a typo, this would only be 1.01 times more parameter efficient].

I'm not sure this answers the question. What are the parameters, anyway, are they just single floats? If they're not, pretty misleading.

Comment by mako yass (MakoYass) on "AI Safety for Fleshy Humans" an AI Safety explainer by Nicky Case · 2024-05-04T02:41:11.305Z · LW · GW

often means "train the model harder and include more CoT/code in its training data" or "finetune the model to use an external reasoning aide", and not "replace parts of the neural network with human-understandable algorithms". 

The intention of this part of the paragraph wasn't totally clear but you seem to be saying this wasn't great? From what I understand, these actually did all made the model far more interpretable?

Chain of thought is a wonderful thing, it clears a space where the model will just earnestly confess its inner thoughts and plans in a way that isn't subject to training pressure, and so it, in most ways, can't learn to be deceptive about it.

Comment by mako yass (MakoYass) on "AI Safety for Fleshy Humans" an AI Safety explainer by Nicky Case · 2024-05-04T02:29:53.541Z · LW · GW

This is good! I would recommend it to a friend!

Some feedback.

  • An individual human can be inhumane, but the aggregate of human values kind of visibly isn't and in most ways couldn't be: Human cultures are getting more humane reliably as transparency/reflection and coordination increases over time, but also inevitably if you aggregate a bunch of concave values it will produce a value system that treats all of the subjects of the aggregation pretty decently.
    A lot of the time, when people accuse us of conflating something, we equate those things because we have an argument that they're going to turn out to be equivalent.
    So emphasizing a difference between these two things could be really misleading, and possibly kinda harmful, given that it could undermine the implementation of the simplest/most arguably correct solutions to alignment (which are just aggregations of humans' values). This could be a whole conversation, but could we just not define humane values as being necessarily distinct from human values? How about this:
    • People are sometimes confused by 'Human values', as it seems to assume that all humans value the same things, but many humans have values that conflict with the preferences of other humans. When we say 'Humane values', we're defining a value system that does a decent job at balancing and reconciling the preferences of every human (Humans, Every one).
  • [graph point for "systems programmer with mlp shirt"] would it be funny if there were another point, "systems programmer without mlp shirt", and it was pareto-inferior
  • "What if System 2 is System 1". This is a great insight, I think it is, and I think the main reason nerdy types often fail to notice how permeable and continuous the boundary is a kind of tragic habitual cognitive autoimmune disease, and I have a post brewing about this after I used a repaired relationship with the unconscious bulk to cure my astigmatism (I'm going to let it sit for a year just to confirm that the method actually worked and myopia really was averted)
  • Exponential growth is usually not slow, and even if it were slow, it wouldn't entail that "we'll get "warning shots" & a chance to fight back", it only takes a small sustained advantage to be able to utterly win a war (though contemporary humans don't like to carry wars to completion these days, the 20th century should have been a clear lesson that such things are within our abilities at current tech levels). Even if progress in capabilities over time continued to be linear, impact over capabilities is not going to be linear, it never has been.

But overall I think it addresses a certain audience who I know much better than my version of this that I hastily wrote last year when I was summoned to speak at a conference would have (and so I never showed it to them. Maybe one day I will show them yours.).

Comment by MakoYass on [deleted post] 2024-05-03T21:31:48.860Z

Uh I'm saying I think henry's is better. Except for the title maybe.

Comment by MakoYass on [deleted post] 2024-05-03T21:09:39.552Z

this one is better

Comment by mako yass (MakoYass) on Please stop publishing ideas/insights/research about AI · 2024-05-02T23:14:23.131Z · LW · GW

Possibly incidental, but if people were successfully maintaining continuous secure access to their signal account you wouldn't even notice because it doesn't even make an attempt to transfer encrypted data to new sessions.

Comment by mako yass (MakoYass) on Please stop publishing ideas/insights/research about AI · 2024-05-02T22:48:00.179Z · LW · GW

I don't think e2e encryption is warranted here for the first iteration. Generally, keypair management is too hard, today, everyone I know who used encrypted Element chat has lost their keys lmao. (I endorse element chat, but I don't endorse making every channel you use encrypted, you will lose your logs!), and keypairs alone are a terrible way of doing secure identity. Keys can be lost or stolen, and though that doesn't happen every day, the probability is always too high to build anything serious on top of them. I'm waiting for a secure identity system with key rotation and some form of account recovery process (which can be an institutional service or a "social recovery" thing) before building anything important on top of e2e encryption.

Comment by mako yass (MakoYass) on Please stop publishing ideas/insights/research about AI · 2024-05-02T22:47:12.937Z · LW · GW

Then, users can put in their own private key to see a post

This was probably a typo but just in case: you should never send a private key off your device. The public key is the part that you send.

Comment by mako yass (MakoYass) on Please stop publishing ideas/insights/research about AI · 2024-05-02T22:35:10.890Z · LW · GW

So I wrote a feature recommendation: https://www.lesswrong.com/posts/55rc6LJcqRmyaEr9T/please-stop-publishing-ideas-insights-research-about-ai?commentId=6fxN9KPeQgxZY235M

Comment by mako yass (MakoYass) on Please stop publishing ideas/insights/research about AI · 2024-05-02T22:34:23.371Z · LW · GW

On infrastructures for private sharing:

Feature recommendation: Marked Posts (name intentionally bland. Any variant of "private" (ie, secret, sensitive, classified) would attract attention and partially negate the point) (alternate names: permissioned posts, declared posts)

This feature prevents leaks, without sacrificing openness.

A marked post will only be seen by members in good standing. They'll be able to see the title and abstract in their feed, but before they're able to read it, they have to click "I declare that I'm going to read this", and then they'll leave a read receipt (or a "mark") visible to the post creator, admins, other members in good standing. (these would also just serve a useful social function of giving us more mutual knowledge of who knows what, while making it easier to coordinate to make sure every post gets read by people who'd understand it and be able to pass it along to interested parties.)

If a member "reads" an abnormally high number of these posts, the system detects that, and they may have their ability to read more posts frozen. Admins, and members who've read many of the same posts, are notified, and you can investigate. If other members find that this person actually is reading this many posts, that they seem to truly understand the content, they can be given an expanded reading rate. Members in good standing should be happy to help with this, if that person is a leaker, well that's serious, if they're not a leaker, what you're doing in the interrogation setting is essentially you're just getting to know a new entrant to the community who reads and understands a lot, talking about the theory with them, and that is a happy thing to do.

Members in good standing must be endorsed by another member in good standing before they will be able to see Marked posts. The endorsements are also tracked. If someone issues too many endorsements too quickly (or the people downstream of their endorsements are collectively doing so in a short time window), this sends an alert. The exact detection algorithm here is something I have funding to develop so if you want to do this, tell me and I can expedite that project.

Comment by mako yass (MakoYass) on Please stop publishing ideas/insights/research about AI · 2024-05-02T22:02:03.731Z · LW · GW

There never will be an infrastructure for this.

I should be less resolute about this. It would kind of be my job to look for a design that could do it.

One thing we've never seen is a system where read receipts are tracked and analyzed on the global level and read permissions are suspended and alerts are sent to admins if an account is doing too many unjustified reads.
This would prevent a small number of spies from extracting a large number of documents.
I suppose we could implement that today.

Comment by mako yass (MakoYass) on Please stop publishing ideas/insights/research about AI · 2024-05-02T21:38:24.827Z · LW · GW

You think that studying agency and infrabayesianism wont make small contributions to capabilities? Even just saying "agency" in the context of AI makes capabilities progress.

Comment by mako yass (MakoYass) on Please stop publishing ideas/insights/research about AI · 2024-05-02T21:35:33.900Z · LW · GW

"So where do I privately share such research?" — good question! There is currently no infrastructure for this.

This is why I currently think you're completely wrong about this. There never will be an infrastructure for this. Privacy of communities isn't a solvable problem in general, as soon as your community is large enough to compete with the adversary, it's large enough and conspicuous enough that the adversary will pay attention to it and send in spies and extract leaks. If you make it compartmented enough to prevent leaks/weed out the spies, it wont have enough intellectual liveliness to solve the alignment problem.

There is nothing that makes differentially helping capabilities "fine if you're only differentially helping them a little bit".

If your acceptable lower limit for basically anything is zero you wont be allowed to do anything, really anything. You have to name some quantity of capabilities progress that's okay to do before you'll be allowed to talk about AI in a group setting.

Comment by mako yass (MakoYass) on Alien neuropunk slaver civilizations · 2024-05-01T05:43:12.967Z · LW · GW

It would seem to me that in this world brains would be much more expensive (or impossible) to copy. Which is worth talking about, because there are designs in our own era for very efficient very dense neural networks that have that same quality. They can be trained, but the weights can't be accessed.

Comment by mako yass (MakoYass) on Tamsin Leake's Shortform · 2024-04-28T20:51:48.231Z · LW · GW

what does it even mean?

There actually is a meaningful question there: Would you enter the experience machine? Or do you need it to be real. Do you just want the experience of pleasing others or do you need those people being pleased out there to actually exist.

There are a lot of people who really think, and might truly be experience oriented. If given the ability, they would instantly self-modify into a Victory Psychopath Protecting A Dream.

Comment by mako yass (MakoYass) on Tamsin Leake's Shortform · 2024-04-27T22:10:34.390Z · LW · GW

An interesting question for me is how much true altruism is required to give rise to a generally altruistic society under high quality coordination frameworks. I suspect it's quite small.

Another question is whether building coordination frameworks to any degree requires some background of altruism. I suspect that this is the case. It's the hypothesis I've accreted for explaining the success of post-war economies (guessing that war leads to a boom in nationalistic altruism, generally increased fairness and mutual faith).

Comment by mako yass (MakoYass) on Losing Faith In Contrarianism · 2024-04-25T22:45:35.392Z · LW · GW

It may be useful to write about how a consumer can distinguish contrarian takes from original insights. Until that's a common skill, there will remain a market for contrarians.

Comment by mako yass (MakoYass) on "You're the most beautiful girl in the world" and Wittgensteinian Language Games · 2024-04-21T15:31:19.353Z · LW · GW

I didn't, but I often want to downvote articles that seem to be lecturing a group who wouldn't read or be changed by the article. I know a lot of idiots will upvote such articles out of a belief that by doing so they are helping or attacking that group. On reddit, it often felt like that is the main reason people upvote things, to engage indirectly with others, and it kills the sub, clogging it with posts that the people who visit the sub are not themselves getting anything from.

If you engaged with the target group successfully, they would upvote the post themselves, so a person should generally never upvote on others' behalf, because they don't actually know what would work for them.

Unfortunately, the whole anonymous voting thing makes it impossible to properly address voting norm issues like this. So either I address it improperly by making deep guesses about why people are voting, in this way (no, don't enjoy) or I prepare to depose lesswrong.com with a better system (that's what I'm doing)

Comment by mako yass (MakoYass) on "You're the most beautiful girl in the world" and Wittgensteinian Language Games · 2024-04-20T21:56:59.831Z · LW · GW

On reflection, it must have played out more than once that a kiwi lad, in a foreign country, drunk, has asked a girl if she wants to get a kebab. The girl thinks he means shish-kebab but says yes enthusiastically because she likes him and assumes he wouldn't ask that unless it was an abnormally good shish-kebab. The kiwi realizes too late that there are no kebabs in america, but they end up going ahead and getting shish-kebabs out of a combination of face-saving, and an infatuation-related coordination problem: The girl now truly wants a shish-kebab, it is too late to redirect the desires of the group.

So that detail might have just been inspired by a true story.

Comment by mako yass (MakoYass) on "You're the most beautiful girl in the world" and Wittgensteinian Language Games · 2024-04-20T20:46:31.597Z · LW · GW

Americans don't know how much they had to compromise in this video by using shish-kebabs instead of what a new zealander would really mean when someone at a party says "do you want to get a kebab with me", which are instead like, the turkish version of burritos, instead of mince, beans and cheese; turkish meat, hummus, veges and wider choice of sauces. They're a fixture of nightlife and tend to be open late.