Meltdown: Interface for llama.cpp and ChatGPT

post by nextcaller · 2024-03-26T18:29:39.344Z · LW · GW · 1 comments

Contents

1 comment

I'm afraid linking what I've been working on for a while as my first post might not be greatly received, but I think you might find it interesting none the less.

I'm making a text interface to chat with local and remote models. It is made in 100% python, it uses tkinter/tcl which should be bundled with a normal python installation. 

I made it because I wasn't able to find an interface that felt right to me. I didn't try them all though. I like adding "power user" features when I think of one.

Repo: https://github.com/Merkoba/Meltdown

Some features:

• Load llama.cpp models (only gguf tested for now).
• Use your ChatGPT api key with a specific model of openai.
• Model configuration tweaks like temperature, top-p etc.
• Sessions with conversations spread in tabs. These can be saved and loaded.
• Configurations can be saved and loaded.
• Markdown support, including syntax highlighting for code snippets.
• Click, right click, or double click words to either Copy, Explain, Search, or open a new conversation.
• Dark and light themes available.
• Commands with tab completion and similarity check.
• Command line arguments to set how the program works.
• Saved context to use with the models.
• Save logs to either json or text.
• Run a command upon saving a log, like opening it with a text editor.
• Compact mode which hides some panels.
• Scrollable panel to pack more configs.
• Prepend and Append to your prompt automatically.
• Close tabs in different ways like old, others, all, etc.
• Display CPU, RAM, and Temperature. Clicking these opens a task manager. This can be expanded to work on more systems.
• Input history to go back to previous prompts by using up/down arrows, buttons, or mousewheel.
• Keyboard shortcuts to perform various actions.
• Variables to use for the system. For example \@name_user, \@name_ai, and \@date.
• Responses are streamed live.

I don't know if this works in systems different to mine. But you are encouraged to try. 

1 comments

Comments sorted by top scores.