SOLAR model paper questions 2023-12-29T20:34:38.254Z

Comment by Bartlomiej Lewandowski (bartlomiej-lewandowski) on Why no major LLMs with memory? · 2023-03-29T13:28:48.466Z · LW · GW

I think there has been a lot of research in the past in this space. The first thing that popped into my mind was https://huggingface.co/docs/transformers/model_doc/rag 

Currently, there are some approaches using langchain to persist the history of a conversation into an embeddings database, and retrieve the relevant parts performing a similar query / task.

Comment by Bartlomiej Lewandowski (bartlomiej-lewandowski) on GPT can write Quines now (GPT-4) · 2023-03-14T22:48:15.876Z · LW · GW

OpenAI has hired a lot of software engineers to code simple tasks, maybe these quines were a part of the fine tuning set? 

Comment by Bartlomiej Lewandowski (bartlomiej-lewandowski) on Deepmind's Gopher--more powerful than GPT-3 · 2021-12-09T04:54:25.360Z · LW · GW

How is retro different from https://ai.facebook.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models/ ?

Comment by Bartlomiej Lewandowski (bartlomiej-lewandowski) on A closer look at chess scalings (into the past) · 2021-07-18T11:19:29.903Z · LW · GW

Isn't ELO a reference metric that changes with time? I would assume that 2800 ELO in the 90s is a different level to today's 2800. Can we still make the same conclusions with that in mind?