Announcing Encultured AI: Building a Video Game

How will you compete in the market while optimizing for various things other than what players want?

I'd guess that the main AI-exacerbating thing that the game industry does is provoke consumers to subsidize hardware development. I don't know if this is worth worrying about (have you weighed the numbers?), but do you plan on like, promoting low-spec art-styles to curb demand for increasing realism? :] I wonder if there's a tension between realism and user-customizability that you might be able to inflame (typically, very detailed artstyles are more expensive to work in and are harder to kitbash, but it's also possible that stronger hardware would simplify asset pipelines in some ways: raytracing could actually simplify a lot of lighting stuff, right?).

It sounds like the game will be an MMO, is that correct?

@Andrew_Critch [LW · GW] 2 years later, it looks like Encultured.ai has pivoted away from the game concept to the (unrelated?) healthcare domain (as mentioned here: Encultured AI | Blog). I'd be really interested in reading about a postmortem of the game / agent-environment concept and whether you think it would be a worthwhile project for another team to pick up (either in broad concept or literally starting from Encultured's designs & prototype(s)).  

What will you do that makes your game better than the many other computer games out there? 

I presume that random members of the public will be playing this game, not just a handful of experts.

Once the AI realizes its in a game and being tested, you have basically no reason to suspect its behaviour in the game to be correlated with behaviour in reality. Given random people who are making basically no attempt to keep this secret, any human level AI will realize this. Freely interacting with large numbers of gullible internet randos isn't the best from a safety standpoint either. I mean a truely superintelligent AI will persuade MIRI to let it out of any box, but for a top human level AI, it could more easily trick internet randos.

 If you kept the controls much tighter, and only had a few AI experts interacting with it, you could possibly have an AI smart enough to be useful, and dumb enough to not realize its in a box being tested.  But this makes getting enough training data hard.

Does any of you actually have game design or development experience, or at least e.g. years of daydreaming and failed prototypes behind you? I'd like to point out that making a video game, particularly an MMO (which it looks like you're thinking of), is very hard, and there are many problems that can ruin their success quite apart from the actual quality of the game.

You also haven't said anything about the actual design itself yet; my prior, from lots and lots of personal experience, is to expect people who are making a Great Game That Will Help Change The World!!! to have tons of Cool Ideas and no discipline and to try to turn it into a kitchen sink and lose all momentum. (I've been those people. Over and over again.)

So: what are the actual game mechanics? What do players actually do in your game? And what makes that action fun?

looks at your website

Oh dear. You don't even have a job listing for a game designer. And lots of talk about the AI you want to make, and as far as I can see, nothing about the game you want to make. Your goals are admirable; but my prediction as of right now is ~80% that you will be shocked at the difficulty of the undertaking and your own lack of readiness for it and give up within a year.

My suggestion is that you help other game studios that already exist build AI for their games in return for their agreement to allow you to use those contexts to explore the alignment-relevant questions, rather than attempting to make games yourself. I think that would be much, much more likely to succeed and require far less effort and struggle on your part.

Evidence that my opinion ought to matter to you: I've wanted to make games since I read Chris Crawford's "On Game Design" when I was 11, and I have had a PERFECT EPIC GAME in my head for years that constantly changes form and never settles down, and I've never managed to actually build even a small prototype without giving up in frustration and self hate. I've also been part of several "small, passionate" teams of people who'd never made a game before and were trying to do so, two of which actually had a professional software developer at the head, and all of them failed miserably within a month or two with nothing to show for it.

How are your goals not met by existing cooperative games? E.g. Stardew Valley is a cooperative farming simulator, Satisfactory is about building a factory together. No violence or suffering there.

Your emphasis on how 'fun' your organization is is kind of off-putting (I noticed this on your website as well). I think it gives me the impression that you are not very serious about what you are doing. Maybe it's just me though.

In order to be the kind of healthy participant in the world AI ecosystem that you are describing, I have the sense that the object-level product you build must be good for the world independent of the goodness of the experiments that it enables or datasets that it generates. So I think you face the challenge of building a game that in addition to being successful, is on its own good for the world.

Seems like this guy has already started trying to use GPT-3 in a videogame: GPT3 AI Game Prototype

Maybe you have seen it before, but Veloren looks like a project with people you should talk with. They are building an open source voxel MMO in Rust, and you might be able to collaborate with them. I think most people working on it are doing it as a side hobby project.

Is this the beginning of Friendship is Optimal?

Are you considering starting with a text-based (or primarily text-only, with optional graphical elements) game? I think that could help a lot with the initial development hurdles and integration with LLMs. There's already a lot of open source code for multiplayer text-based games which could give you a jumping off point. Using that to start with would let you get right into the important part of game design and integration with LLMs. Also, there's some neat work that has already been done with single and multiplayer text-based games and LLMs.
As for how to make a text-based game appealing to modern gamers, I've been thinking that the new art-creating models combined with some old text-based games to auto-generate art to optionally accompany the existing text descriptions of 'rooms' would be a very cool revamp of an old genre. Especially if the players can use in-game currency to create their own text-based game objects which the AI would then illustrate for them. (I've played multiplayer online text-based games where in-game currency was used to purchase creation of 'rooms' and 'objects', so the code for this is already available and well play-tested). I think a game where text-based generative creation aspects in combination with strategic challenges and resource acquisition / trade economy elements sounds like an ideal milieu for building experimental mini-games for human & AI agents within.

Safety Concern: I feel compelled to point out that you'd have to monitor the AI agents' interactions with human players very closely and continuously because this has the potential to turn very dangerous very fast. An AI agent which strategically plans and takes actions including bargaining with humans is on a clear path to dangerous territory if left unchecked.

Wouldn't the kind of alignment you'd be able to test behaviorally in a game be unrelated to scalable alignment?