Pantheon Interface

post by NicholasKees (nick_kees), Sofia Vanhanen · 2024-07-08T19:03:51.681Z · LW · GW · 22 comments

Contents

  How do I use Pantheon?
    What do the daemons see?
    Trees, branching, and sections
    How do I save my progress?
    What are “AI Suggestions”?
    What is “Ask AI”?
    Can I control which daemon responds?
  What is the target use case?
  How do I make my own daemons?
  How does this relate to Loom?
  Does Pantheon use base models?
  Is Pantheon open source?
  Should I be concerned about data privacy?
  Acknowledgments
None
22 comments

Pantheon is an experimental LLM interface exploring a different type of human-AI interaction. We created this as a part of the cyborgism [LW · GW] project, with the abstract motivation of augmenting the human ability to think by integrating human and AI generated thoughts.

How it works:

1. A human user “thinks out loud” by typing out their thoughts one at a time. This leaves a text trace of their stream of thought.
2. AI characters (called daemons) read this trace, and interact with the user by responding asynchronously with comments and questions.

The core distinguishing philosophy is that, while most apps are about a human prompting an AI to do useful mental work, Pantheon is the opposite. Here, AI does the prompting, and the goal is for the AI generated questions or comments to cause the human user to think in ways they would not have on their own. At worst, the app is a rubber duck. At best, the app is a court of advisors, each using their own unique skills to push you to think your best thoughts.

Pantheon can be found at pantheon.chat, and we would really appreciate any and all feedback you have.

The app is set up for you to customize your own daemons. We have set up some default daemons to provide inspiration, but we expect the tool to be a lot more useful when they are customized to specific users. If the default daemons don’t feel useful, we highly encourage you to try to make your own.

How do I use Pantheon?

First, go to settings and provide an OpenAI API key.^[1]

Next, begin typing out your thoughts on some topic. It helps to keep each thought relatively short, sending them to the stream of thought as often as you can. This gives the daemons lots of opportunities to interject and offer their comments. Furthermore, it’s usually best to treat this more like a diary or personal notes, rather than as a conversation. In this spirit, it’s better not to wait for them to respond, but to instead continue your train of thought, keeping your focus on your own writing.

What do the daemons see?

Your stream of thought appears in the interface as a chain of individual thoughts. Daemons are called to respond to specific thoughts. When they do, they are given access to all preceding thoughts in the chain, up to and including the thought they were called to. Daemons can only see text the user has written, and they can’t see any of the comments made by themselves or other daemons. We are looking into ways to give the daemons access to their own comment history, but we have not yet made this possible.

After a daemon generates a comment, you can inspect the full chain of thought by clicking on that comment. This will open up a window which will show you everything the LLM saw in the process of generating that response. You can also edit the daemons in settings, as well as toggle them on or off.

Trees, branching, and sections

The text in the interface appears to you as a chain of thoughts, but it is actually a tree. If you hover over a thought, a plus icon will appear. If you click this icon, you can branch the chain. This is often useful if you feel that you have gone down a dead end, or would like to explore a tangent. When there are multiple branches, arrows will appear next to their parent thought, and you can use those arrows to navigate the tree.

If you would like a fresh context, you can make an entirely new tree by opening the “Collection view” in the top left. Furthermore, you can also create a new “section” by clicking the “New Section” button below the input box. This will create a hard section break such that daemons can no longer see any context which came before the break.

How do I save my progress?

Everything you do is automatically saved in local storage. You can also import/export the full app state in settings.

What are “AI Suggestions”?

These are continuations of your stream of thought generated by the base mode. You can understand them as the base model predicting what you will write next.

What is “Ask AI”?

In order to avoid you ever having to copy and paste your stream of thought into another chat bot interface, the button “Ask AI” will turn the text you are currently writing into an instruction for a chat model, and all of your past thoughts will be given to it for context. You can use it to, for example:

• Summarize everything so far
• Answer a specific question
• Brainstorm a list of ideas
• Rephrase something you said

These instructions and the model’s responses are “meta” in that they will not be shown to the daemons, or be used to generate base model completions (as such they also appear differently in the interface).

Can I control which daemon responds?

Yes, there is a mentions feature. By default, daemons are selected at random, but if you type @ and then the daemon's name, this will ensure that specific daemon will respond to your thought.

What is the target use case?

We built this as an experimental interface, and our goal from the beginning was just to explore new ways of interacting with LLMs. Therefore, we didn’t have a target use case in mind. That said, some examples for how we’ve personally used it include:

• Brainstorming ideas for a project
• Thinking through a personal problem
• Brain dumping after listening to a presentation
• Journaling
• Explaining a topic while learning about it
• Keeping a work log

We’ve especially found it useful for both overcoming the paralysis of an empty page, and maintaining the momentum to continue pursuing a train of thought.

How do I make my own daemons?

When you go to settings, you will find the option to edit existing daemons, or to create your own. A daemon consists of two things:

1. a system prompt
2. a list of chain of thought prompts

The list of chain of thought prompts outline half of a chatbot conversation. Each prompt is shown to the chat model one at a time, and the chat model will generate a response to each. The response to the final prompt is shown to the user as a comment.

In this pipeline, a daemon doesn’t see the user’s thoughts by default. The user’s thoughts have to be included explicitly by inserting variables into the prompts. These variables will be replaced by the user’s thoughts at the time the daemon is called. There are currently two variables:

• {PAST} : Contains all past thoughts in the current branch up to but not including the thought which the daemon was called to respond to.
• {CURRENT} : Only contains the thought which the daemon was called to respond to.

Below is an example of how these variables could be used:

A daemon may have arbitrarily many chain of thought prompts, though the longer the chain of thought gets, the longer it will take to generate a response (each chat model response requires a separate API request).

The formula we used for the default daemons is a variant of babble and prune [? · GW]. The first prompt gets the model to generate a long list of possible responses, and later prompts are used to narrow down which response is best. This isn’t necessarily the best way to do this, and we would encourage anyone making their own daemons to experiment with alternative formulas.

How does this relate to Loom?

In general, Loom is what we think of as an “LLM-centered” interface, where the burden of adapting to the constraints of the interaction falls primarily on the human user. Loom users will typically minimize the extent to which they directly intervene on LLMs in order to avoid contaminating their “natural behavior [LW · GW]”, and to make it possible to observe them thinking in a wide range of contexts which are often much less intuitive to most people by default.

In this frame, Pantheon is explicitly a “human-centered” interface, where the burden on the human user is minimized at the expense of making LLMs adapt to the unnatural context of having to comment on a human’s stream of thought. The intention is to ensure that, during a session, the human user does not have to spend any additional mental energy managing their relationship to the LLM, or adjusting their behavior to match the LLM’s expectations.

Does Pantheon use base models?

Yes, but in a much less significant way than we originally intended.

We spent a lot of time creating prompts for base model simulacra which had interesting/useful personalities, and found some prompts which worked fairly well. Base models, however, are extremely sensitive to changes in tone, style, or really anything that departs from the expectations set in the prompt. When these simulacra came into contact with the user’s text, which looked nothing like the prompt, they would always collapse into chaotic nonsense. It was also quite hard to find contexts where it made sense for a character to be responding to user text at all.

We do use a base model to predict the user’s next thought, and those predictions are continuously shown to the user as they write. These predictions are sometimes interesting, but this is far from the best way to use base models. Some ideas for how they might be used in the future:

• Dynamic prompting/sampling: One idea would be to use LLMs to dynamically construct a prompt which more naturally integrates the user’s text into a base model prompt to avoid breaking the flow of a story. Furthermore, we might be able to use alternate sampling methods [LW · GW] to help steer base models in such a way that they accept difficult to prompt premises (see janus’s comment [LW(p) · GW(p)] on diegetic interfaces).
• Hybrid daemons: Another idea we experimented with a little was to create daemons which used both a chat model and a base model to generate parts of their chain of thought. In this way, we might be able to use base models to “inject creativity” into chat models, and help them overcome mode collapse [LW · GW].

Is Pantheon open source?

Yes! Pantheon is open source and licensed under GNU General Public License version 3 (GPLv3). We considered the possible harms of open sourcing the tool and expect them to be fairly minimal. If you would like to contribute to Pantheon, the code is hosted on Github.

Should I be concerned about data privacy?

The app does not have a backend, and so any and all data you share with the app will only be shared with the OpenAI servers, or be kept in your own local storage. Therefore, your worries about data privacy should be about the same as your concerns with using the ChatGPT interface. 

Acknowledgments

Finally, we’d like to thank some of the people who helped make this project happen. Firstly, @Ethan Edwards [LW · GW] , @clem_acs [LW · GW] , Gianluca Pontonio, @Quentin FEUILLADE--MONTIXI [LW · GW] , @Filip Sondej [LW · GW] , and @remember [LW · GW] all gave really valuable help with brainstorming the initial design. We also got a lot of valuable user feedback from @eggsyntax [LW · GW], @clem_acs [LW · GW], @Filip Sondej [LW · GW] , Dmitriy Kripkov, Akash Arora, Vinay Hiremath, and @Aay17ush [LW · GW] . Aayush also helped a lot by providing a character generator we used to produce the default daemons.

1. ^{^}

   To get a working OpenAI key, log into your OpenAI account and go here: https://platform.openai.com/api-keys 
   You do not need to have an OpenAI subscription, but using the key will cost money every time the app makes a request. Detailed pricing information can be found here: https://openai.com/api/pricing/ 
   An extremely rough estimate is something like ~1USD / hour of continuous use, though this depends a lot on how quickly you type, and how long the chain of thought is for each daemon. 

22 comments

Comments sorted by top scores.