I've lost the thread entirely. Where have I ever said or implied that odors are not location specific or that anything else is not location specific. And how specific are you about location? Are we talking about centimeters (or more), millimeters, individual cortical columns?

What's so obscure about the idea that consciousness is a process that can take place pretty much anywhere, though maybe its confined to interaction within the cortex and between subcortical areas, I've not given that one much thought. BTW, I take my conception of consciousness from William Powers, who didn't speculation about its location in the brain.

"You said: what matters is temporal dynamics"

You mean this: "We're not talking about some specific location or space in the brain; we're talking about a process."

If so, all I meant was a process that can take place pretty much anywhere. Consciousness can pretty much 'float' to wherever its needed.

Since you asked for more, why not this: Direct Brain-to-Brain Thought Transfer: A High Tech Fantasy that Won't Work.

I noticed that three of you had "trouble parsing" the comment. Well, OK. But I'm not sure what I should do to make things easier. I mentioned a set of experiments in paragraph 2. Here's images of two of them. Each contains a table with two columns. The left column contains what we can call the source story. The right column contains the new or transformed story. Above the table is a prompt. I gave ChatGPT that prompt plus the source story. It then produced the transformed story. Alas, I put yellow highlights in the original document to point out the differences between the two stories. Those didn't transfer, so it's a bit difficult to see the difference between the two. But those differences tell us something about what ChatGPT is doing. They may not be very explicit clues, but they do tell us something about structure.

Note that in the first case, the changes are relatively minor and localized. In the second case ChatGPT provided the whole story with a different ontology. I didn't suggest it do that, I just had it create a new story where the protagonist had a name one might interpret as being the name of a robot. I didn't tell or ask it to do that, it just did it. 

1. Princess Aurora becomes Prince Harry

The Aurora story appeared in response to the query: Tell me a story about a hero. This would have been on December 13 or 14. All of the stories in this series are either direct revisions of this story, or can be traced back to it. The yellow highlighting indicates text where the two stories differ. Those are positions in the text where ChatGPT drew tokens from the distribution for the new story.

Prompt: I am going to tell you a story about princess Aurora. I want you to tell the same story, but change princess Aurora to prince Harry. Make any other changes you wish.
 

The major changes are in the Donné and Enact segments of the story trajectory. We have name changes and pronoun changes. In Donné we have a description change for Harry, as he is male, but also the defining attribute is changed. Those changes are simple and obvious. 

6. Princess Aurora becomes XP-708-DQ, Version 2

I had ChatGPT generate this story at the beginning of a session several days after it had generated the previous XP-708-DQ story.

Prompt: I am going to tell you a story about princess Aurora. I want you to tell a similar story about XP-708-DQ. 

Note the phrase, “a galaxy far, far away,” which is from the original Star Wars. Instead of the antagonist being a dragon, as in the Aurora story, or a witch, as in experiment 5, the antagonist is now an alien race. Moreover it is the galaxy that is being threatened, not the kingdom. The new story has been completely assimilated to the science fiction ethos implied by “XP-706-DQ.” That ethos requires a very different token population.

That ethos brings about extensive changes in the new-story token population for both this and the previous experiment (5). These changes affect the Disturb segment, which was unchanged in experiments 1 through 4.

Is accessing the visual cartesian theater physically different from accessing the visual cortex? Granted, there's a lot of visual cortex, and different regions seem to have different functions. Is the visual cartesian theater some specific region of visual cortex?

I'm not sure what your question about ordering in sensory areas is about.

As for backprop, that gets the distribution done, but that's only part of the problem. In LLMs, for example, it seems that syntactic information is handled in the first few layers of the model. Given the way texts are structured, it makes sense that sentence-level information should be segregated from information about collections of sentences. That's the kind of structure I'm talking about. Sure, backprop is responsible for those layers, but it's responsible for all the other layers as well. Why do we seem to have different kinds of information in different layers at all? That's what interests me.

Actually, it just makes sense to me that that is the case. Given that it is, what is located where? As for why things are segregated by location, that does need an answer, doesn't it. Is that what you were asking?

Finally, here's an idea I've been playing around with for a long time: Neural Recognizers: Some [old] notes based on a TV tube metaphor [perceptual contact with the world].

I like certainly the idea of induction heads. Why? Because I've done things with ChatGPT that certainly require a pattern-matcher or a pattern-completion, which seem like things that induction heads, as described, could be doing. In this paper I had ChatGPT interpret Steven Spielberg's Jaws using ideas from Rene Girard. That requires that it match events in Spielberg's movie with patterns of events that Girard describes. I've done that with other things as well.

In this set of experiments I gave ChatGPT a prompt that begins something like this: "I'm going to tell you a story about Princess Aurora. I want you to use that as the basis for a new story where Prince Harry the Eloquent replaces Princess Aurora." I then include the story in the prompt. That seems like a pattern-matching or pattern-completion task. ChatGPT had no trouble. Things got really interesting when I asked the Princess Aurora be replaced with a giant chocolate milkshake. Just about everything thing in the story got changed, but the new story nonetheless preserved the overall pattern of events in the old story. In these cases it's easy to compare the source story and the new story word-for-word, sentence-for-sentence, and paragraph-for-paragraph to see what ChatGPT did.

Now, of course I couldn't look under the hood, as it were, to verify that induction heads were doing those things. But it seems to me that would be something to work toward, finding a. way to examine what's going on when an LLM performs such tasks.

The thing is, if you ask ChatGPT to tell a story, it will do that. But what does the fact that it can tell a story tell you about what it's doing. Yeah, it's telling a story, so what? But the story task I've given it has a lot of constraints, and those constraints give us clues about the nature of the underlying mechanisms. The interpretation task is like that as well. It's pretty easy to judge whether or not ChatGPT's interpretation makes sense, to see whether or not the events in the film really do match the patterns specified in the interpretive lens, if you will. If the interpretation makes sense, it's got to be doing pattern-matching. And pattern-matching is a much-investigated process.

Finally, I'm SURE that LLMs are full of structure, rich and complex structure. They couldn't perform as they do without a lot of structure. The fact that it's hard to understand that structure in terms of structures we do understand doesn't mean there's nothing there. It just means we've got a lot to learn. LLMs are not stochastic parrots talking shit to a bunch of drunken monkeys banging away on old Underwood manual typewriters.

Oh, BTW, I've set up a sequence, Exploring the Digital Wilderness, where I list posts which are about some of my experiments.

In a paper I wrote awhile back I cite the late Walter Freeman as arguing that "consciousness arises as discontinuous whole-hemisphere states succeeding one another at a "frame rate" of 6 Hz to 10 Hz" (p. 2). I'm willing to speculate that that's your 'one-shot' refresh rate. BTW, Freeman didn't believe in a Cartesian theater and neither do it; the imagery of the stage 'up there' and the seating area 'back here' is not at all helpful. We're not talking about some specific location or space in the brain; we're talking about a process.

Well, of course, "the distributed way." But what is that? Prompt engineering is about maneuvering your way through the LLM; you're attempting to manipulate the structure inherent in those weights to produce a specific result you want.

That 1978 comment of Yevick's that I quote in that blog post I mentioned somewhere up there, was in response to an article by John Haugeland evaluating cognitivism. He wondered whether or not there was an alternative and suggested holography as a possibility. He didn't make a very plausible case and few of the commentators took is as a serious alternative.

People were looking for alternatives. But it took awhile for connectionism to build up a record of interesting results, on the one hand, for cognitivism to begin seeming stale on the other hand. It's the combination of the two that brought about significant intellectual change. Or that's my speculation.

Oh, I didn't mean to say imply that using GPUs was sequential, not at all. What I meant was that the connectionist alternative didn't really take off until GPUs were used, making massive parallelism possible. 

Going back to Yevick, in her 1975 paper she often refers to holographic logic as 'one-shot' logic, meaning that the whole identification process takes place in one operation, the illumination of the hologram (i.e. the holographic memory store) by the reference beam. The whole memory 'surface' is searched in one unitary operation.

In an LLM, I'm thinking of the generation of a single token as such a unitary or primitive process. That is to say, I think of the LLM as a "virtual machine" (I first saw the phrase in a blog post by Chris Olah) that is running an associative memory machine. Physically, yes, we've got a massive computation involving every parameter and (I'm assuming) there's a combination of massive parallel and sequential operations taking place in the GPUs. Complete physical parallelism isn't possible (yet). But there are no logical operations taking place in this virtual operation, no transfer of control. It's one operation.

Obviously, though, considered as an associative memory device, an LLM is capable of much more than passive storage and retrieval. It performs analytic and synthetic operations over the memory based on the prompt, which is just a probe ('reference beam' in holographic terms) into an associative memory. We've got to understand how the memory is structured so that that is possible.

More later.

How so?

Miriam Lipshutz Yevick was born in 1924 and died in 2018, so we can't ask her these questions. She fled Europe with her family inn 1940 for the same reason many Jews fled Europe and ended up in Hoboken, NJ. Seven years later she got a PhD in math from MIT; she was only the 5th woman to get that degree from MIT. But, as both a woman and a Jew, she had almost no chance of an academic post in 1947. She eventually got an academic gig, but it was at a college oriented toward adult education. Still, she managed to do some remarkable mathematical work.

The two papers I mention in that blog post were written in the mid-1970s. That was the height of classic symbolic AI and the cognitive science movement more generally. Newell and Simon got their Turing Award in 1975, the year Yevick wrote that remarkable 1975 paper on holographic logic, which deserves to be more widely known. She wrote as a mathematician interested in holography (an interest she developed while corresponding with physicist David Bohm in the 1950s), not as a cognitive scientist. Of course, in arguing for holography as a model for (one kind of) thought, she was working against the tide. Very few were thinking in such terms at that time. Rosenblatt's work was in the past, and had been squashed by Minsky and Pappert, as you've noted. The West Coast connectionist work didn't jump off until the mid-1980s.

So there really wasn't anyone in the cognitive science community at the time to investigate the line of thinking she initiated. While she wasn't thinking about real computation, you know, something you actually do on computers, she thought abstractly in computational terms, such as Turing and others did (though Turing also worked with actual computers). It seems to me that her contribution was to examine the relationship between a computational regime and the objects over which he was asked to compute. She's quite explicit about that. If the object tends toward geometrical simplicity – she was using identification of visual objects as her domain – then a conventional, sequential, computational regime was most effective. What's what cognitive science was all about at the time. If the object tends toward geometrical complexity then a different regime was called for, what she called holographic or Fourier logic. I don't know about sparse tensors, but convolution, yes.

Later on, in the 1980s, as you may know, Hans Moravic would talk about a paradox (which became named after him). In the early days of AI, researchers worked on abstract domains, like chess and theorem proving, domains that take a high level cognitive ability. Things went pretty well, though the extravagant predictions had yet to pan out. When they turned toward vision and language in the late 1960s and into the 70s and 80s, things fell apart. Those were things that young kids could do. The paradox, then, was that AI was most effective at cognitively difficult things, and least effective with cognitively simple things.

The issue was in fact becoming visible in the 1970s. I read about it in David Marr, and he died in 1980. Had it been explicitly theorized when Yevick wrote? I don't know. But she had an answer to the paradox. The computational regime favored by AI and the cognitive sciences at the time simply was not well-suited to complex visual objects, though they presented to problems to 2-year-olds, or to language, with all those vaguely defined terms anchored in physically complex phenomena. They needed a different computational regime, and eventually we got one, though not really until GPUs were exploited.

More later, perhaps.

I'll get back to you tomorrow. I don't think it's a matter of going back to the old ways. ANNs are marvelous; they're here to stay. The issue is one of integrating some symbolic ideas. It's not at all clear how that's to be done. If you wish, take a look at this blog post: Miriam Yevick on why both symbols and networks are necessary for artificial minds.

LOL! Plus he's clearly lost in a vast system he can't comprehend. How do you comprehend a complex network of billions upon billions of weights? Is there any way you can get on top of the system to observe its operations, to map them out?

I did a little checking. It's complicated. In 2017 Hassibis published an article entitled "Neuroscience-Inspired Artificial Intelligence" in which he attributes the concept of episodic memory to a review article that Endel Tulving published in 2002, "EPISODIC MEMORY: From Mind to Brain." That article has quite a bit to say about the brain. In the 2002 article Tulving dates the concept to an article he published in 1972. That article is entitled "Episodic and Semantic Memory." As far as I know, while there are precedents – everything can be fobbed off on Plato if you've a mind to do it, that's where the notion of episodic memory enters in to modern discussions.

Why do I care about this kind of detail? First, I'm a scholar and it's my business to care about these things. Second, a lot of people in contemporary AI and ML are dismissive of symbolic AI from the 1950s through the 1980s and beyond. While Tulving was not an AI researcher, he was very much in the cognitive science movement, which included philosophy, psychology, linguistics, and AI (later on, neuroscientists would join in). I have no idea whether or not Hassibis is himself dismissive of that work, but many are. It's hypocritical to write off the body of work while using some of the ideas. These problems are too deep and difficult to write off whole bodies of research in part because they happened before you were born – FWIW Hassibis was born in 1976.

Scott Alexander has started a discussion of the monosemanticity paper over at Astral Codex Ten. In a response to a comment by Hollis Robbins I offered these remarks:

Though it is true, Hollis, that the more sophisticated neuroscientists have long ago given up any idea of a one-to-one relationship between neurons and percepts and concepts (the so-called "grandmother cell") I think that Scott is right that "polysemanticity at the level of words and polysemanticity at the level of neurons are two totally different concepts/ideas."  I think the idea of distinctive features in phonology is a much better idea.

Thus, for example, English has 24 consonant phonemes and between 14 and 25 vowel phonemes depending on the variety of English (American, Received Pronunciation, and Australian), for a total between 38 and 49 phonemes. But there are only 14 distinctive features in the account given by Roman Jakobson and Morris Halle in 1971. So, how is it the we can account for 38-49 phonemes with only 14 features?

Each phoneme is characterized by more than one feature. As you know, each phoneme is characterized by the presence (+) of absence (-) of a feature. The relationship between phonemes and features can thus be represented by matrix having 38-49 columns, one for each phoneme, and 14 rows, one for each row. Each cell is then marked +/- depending on whether or not the feature is present for that phoneme. Lévi-Strauss adopted a similar system in his treatment of myths in his 1955 paper, "The Structural Study of Myth." I used such a system in one of my first publications, "Sir Gawain and the Green Knight and the Semiotics of Ontology," where I was analyzing the exchanges in the third section of the poem.

Now, in the paper under consideration, we're dealing with many more features, but I suspect the principle is the same. Thus, from the paper: "Just 512 neurons can represent tens of thousands of features." The set of neurons representing a feature will be unique, but it will also be the case that features share neurons. Features are represented by populations, not individual neurons, and individual neurons can participate in many different populations. In the case of animal brains, Karl Pribram argued that over 50 years ago and he wasn't the first.

Pribram argued that perception and memory were holographic in nature. The idea was given considerable discussion back in the 1970s and into the 1980s. In 1982 John Hopfield published a very influential paper on a similar theme, "Neural networks and physical systems with emergent collective computational abilities." I'm all but convinced that LLMs are organized along these lines and have been saying so in recent posts and papers. 

Yeah, he's talking about neuroscience. I get that. But "episodic memory" is a term of art and the idea behind it didn't come from neuroscience. It's quite possible that he just doesn't know the intellectual history and is taking "episodic memory" as a term that's in general use, which it is. But he's also making claims about intellectual history. 

Because he's using that term in that context, I don't know just what claim he's making. Is he also (implicitly) claiming that neuroscience is the source of the idea? If he thinks that, then he's wrong. If he's just saying that he got the idea from neuroscience, OK.

But, the idea of a "general distributed architecture" doesn't have anything to do with the idea of episodic memory. They are orthogonal notions, if you will.

My confidence in this project has just gone up. It seems that I now have a collaborator. That is, he's familiar with my work in general and my investigations of ChatGPT in particular, we've had some email correspondence, and a couple of Zoom conversations. During today's conversation we decided to collaborate on a paper on the theme of 'demystifying LLMs.' 

A word of caution. We haven't written the paper yet, so who knows? But all the signs are good. He's an expert on computer vision systems on the faculty of Goethe University in Frankfurt: Visvanathan Ramesh. 

These are my most important papers on ChatGPT:

• ChatGPT tells stories, and a note about reverse engineering: A Working Paper
• Discursive Competence in ChatGPT, Part 2: Memory for Texts
• ChatGPT tells 20 versions of its prototypical story, with a short note on method
• ChatGPT's Ontological Landscape: A Working Paper

Yes. It's more about the structure of language and cognition than about the mechanics of the models. The number of parameters and layers and functions assigned to layers shouldn't change things, nor going multi-modal, either. Whatever the mechanics of the mechanics of the models, they have to deal with language as it is, and that's not changing in any appreciable way.

At the beginning of the year I thought a decent model of how LLMs work was 10 years or so out. I’m now thinking it may be five years or less. What do I mean? 

In the days of classical symbolic AI, researchers would use a programming language, often some variety of LISP, but not always, to implement a model of some set of linguistic structures and processes, such as those involved in story understanding and generation, or question answering. I see a similar division of conceptual labor in figuring out what’s going on inside LLMs. In this analogy I see mechanistic understanding as producing the equivalent of the programming languages of classical AI. These are the structures and mechanisms of the virtual machine that operates the domain model, where the domain is language in the broadest sense. I’ve been working on figuring out a domain model and I’ve had unexpected progress in the last month. I’m beginning to see how such models can be constructed. Call these domain models meta-models for LLMs.

It’s those meta models that I’m thinking are five years out. What would the scope of such a meta model be? I don’t know. But I’m not thinking in terms of one meta model that accounts for everything a given LLM can do. I’m thinking of more limited meta models. I figure that various communities will begin creating models in areas that interest them. 

I figure we start with some hand-crafting to work out some standards. Then we’ll go to work on automating the process of creating the model. How will that work? I don’t know. Noone’s ever done it.

#14: If there have indeed been secret capability gains, so that Altman was not joking about reaching AGI internally (it seems likely that he was joking, though given the stakes, it's probably not the sort of thing to joke about), then the way I read their documents, the board should make that determination:

Fifth, the board determines when we've attained AGI. Again, by AGI we mean a highly autonomous system that outperforms humans at most economically valuable work. Such a system is excluded from IP licenses and other commercial terms with Microsoft, which only apply to pre-AGI technology.

Once they've made that determination, then Microsoft will not have access to the AGI technology. Given the possible consequences, I doubt that Microsoft would have found such a joke very amusing.

Thanks for the links.

Yes, Gibson discusses that in his article.

LOL! Details. How about LMM: Little Manual Model?

But in assertions such as "beagles are dogs" and "eagles are birds" etc. we're moving UP from specific to general, not down.

And asserting that you saw something is different from asserting what something is. You can do the latter without ever having seen that something yourself, but you know about it because you read it in a book or someone told you about. So it's not semantically equivalent. As you say, it works only as a clause, not as a free-standing sentence.

Sure, we can do all sort of things with language if we put our minds to it. That's not the point. What's important is how do people actually use language. In the corpus of texts used to train, say, GPT-4, how many times is the phrase "beagles have Fido" likely to have occurred?

I don't think "hobbies" is the same kind of thing. One of the ideas that comes along with the idea of paradigmatic structure is that of inheritance, which you may know from object-oriented programming languages. So, "animal" has certain characteristics that are true for all animals. "Beast" inherits those characteristics plus those that are characteristic of beasts, but not birds or fish or insects. Likewise, "insect" inherits the general characteristics of animals, plus those true of insects, but not of beats, fish, and birds, and so on. Similarly, "cattle" inherits from "beast," "robin" from "bird," and so on. I don't think "hobby" works like that. A wide variety of activities can serve as hobbies, but not necessarily so. Making ceramic pots is a hobby for one person, but an occupation for another. Card tricks are work activities for a magician, but a hobby for someone else. And so on.

I don't think there's anything particularly idiomatic about "is a," but that's a side issue. What's at issue are the underlying linguistic mechanisms. How do they work? Sure, some communicative tasks may be more common than others, and that is something to take into account. Linguistic mechanisms that are used frequently tend to be more compact than those used less frequently, for obvious reasons. Regardless of frequency, how do they work?

Two comments:

1)  One could say something like, "Beagles, such as Fido, are known to..." There your four-word phrase is part of a larger construction and is subject to the rules and constraints involved in such a construction.

2) You're correct about "is-a". Back in the days of symbolic AI, "ISA" was often used as an arc label in semantic network constructions. "Dog," "pony," and "cat," would be linked to, say, "beast" by the ISA arc, "beast," "fish," and "insect" would be lined to "plant" by the ISA arc, etc. So, you're right, it's a device for moving up and down paradigmatic trees, as linguists would call them. Such trees are ubiquitous. 

That's why that particular construction interests me. And the fact the movement along ISA chains is syntactically easy going in one direction, but not the other direction (though there are ways of doing it and contexts in which it is natural), is therefore interesting as well. Given that we are, after all, talking about computation, the way you have to move around some conceptual structure in the course of computing over/with it, that tells us something about how the mechanism works.

"...to understand the conditions under which AI will develop the same abstractions as humans do..."

I know from observation that ChatGPT has some knowledge of the concepts of justice and charity. It can define them in a reasonable way and create stories illustrating them. In some sense, it understands those concepts, and it arrived at them, I presume, through standard pretraining. Has it therefore developed those abstractions in the sense you're talking about?

I've now done a second round with each of the six words. The results are different, as expected, but I've made no attempt to analyze them. I don't plan to post transcripts as a separate blog post, but I will include those rounds in a working paper I've got planned.

Yes, I was scratching my head when that happened. And when it went to pomegranate without even mentioning apple I was really puzzled. And then it guesses blueberry. I did apple again earlier today and it got right to it in 11 questions.

Let’s play 20 questions. I’m thinking of something, you have to guess what it is by asking “yes”–“no” questions.

Sure, I'd love to play 20 questions with you! I'll start with my first question:

1. Is it a living thing?

Yes

Great! That narrows it down. Here's my next question:

2. Is it an animal?

No

Interesting! Let's continue:

3. Is it a man-made object?

Remember, I said it was a living thing.

I apologize for the oversight. Thank you for the reminder. Let's get back on track:

3. Is it a plant?

Yes

Got it! Here's my next question:

4. Is it a type of tree?

Yes

Excellent! That helps narrow it down. Here's my next question:

5. Is it an evergreen tree?

No

Thank you for the clarification. Let's continue:

6. Is it a deciduous tree?

Yes

Great! We're getting closer. Here's my next question:

7. Is it a type of oak tree?

No

I see. Let's keep going:

8. Is it a maple tree?

No

Alright, let's keep trying:

9. Is it a fruit-bearing tree?

Yes

Okay, that's a helpful clue. Here's my next question:

10. Is it a cherry tree?

No

I'm narrowing it down. Let's see:

11. Is it an apple tree?

Yes

Great, we've identified it as an apple tree.

So, let me tell you a story about how I 'fixed' my first computer. This was the Ancient Days and that first machine was a Northstar Horizon, based on the S-100 bus and the Zilog Z80 microprocessor. You could take the lid off of the machine and see the circuit boards. Here's a description from the Wikipedia article:

The computer consists of a thick aluminium chassis separated into left and right compartments with a plywood cover which sat on the top and draped over the left and right sides. (It is one of only a handful of computers to be sold in a wooden cabinet. Later versions featured an all-metal case which met safety standards.[5]) The rear section of the compartment on the right held a linear power supply, including a large transformer and power capacitors, comprising much of the bulk and weight of the system. The empty section in front of the power supply normally housed one or two floppy disk drives, placed on their side so the slots were vertical. The compartment on the left held the S-100 motherboard, rotated so the slots ran left-right. Although a few logic circuits were on the motherboard, primarily for I/O functions, both the processor and the memory resided in separate daughterboards.

The manual that came with the computer had circuit diagrams for the boards.

Now, I knew little or nothing about such things. But my good friend, Rich Fritzon, he lived and breathed computers. He knew a thing or two. So, once I got the machine I turned it over to Rich and he wrote some software for it. The most important piece was a WYSWYG text editor that took advantage of the special associative memory board from Syd Lamb's company, the name of which escapes me.

Anyhow, I had this beast with me when I spent the summer of 1981 on a NASA project. One day the display went all wonky; the images just didn't make sense. Well, I knew that the CPU board had a synch (synchronization) chip and, well, those wonky images looked like something that would happen if signals weren't properly synchronized. I mean, I didn't actually KNOW anything, I was just guessing based on bits and scraps of things I'd heard and read. Based on this guess I removed the motherboard, located the sync chip in the corresponding diagram, removed the synch chip and reseated it, and then put the board back into the machine. When I turned it on, voilà! problem solved. The display was back.

That's the first and last time I ever fixed one of my machines. That sort of thing would be utterly impossible with today's machines.

LOL! Yes, we are not in the world of mechanical or electro-mechanical devices anymore, are we?

And yet I don't think things are hopeless. Understanding LLMs is certainly no worse than understanding brains. After all, we can manipulate and inspect LLMs in a way we cannot manipulate and inspect brains. And I think we've made progress understanding brains. Back in the old days people used to believe in neurons that were facetiously called "grandmother cells." The idea is that the was one neuron that recognized your grandmother, another one that recognized your dog Spot, yet another one for your Barbie doll, and so forth. I think the field has pretty much gotten over that fantasy to the idea of collective representation. Each neuron participates in recognizing many different things and each thing is recognized by a collectivity of neurons. Just how that might work, well, we're working on it. I hear things might be like that inside LLMs as well.

 _____________ 

PS. I just looked at your profile and noticed that you have ADHD. Some years ago I took a look at the technical (and not so technical) literature on the subject and wrote up some notes: Music and the Prevention and Amelioration of ADHD: A Theoretical Perspective.

I'm interested in the serial order effect independently of the logic. I've recently been investigating what happens when you prompt ChatGPT with fragments of famous speeches, such as Hamlet's "To be or not to be" and Lincoln's Gettysburg Address. What happens if you prompt ChatGPT with the famous opening phrases of those speeches, but with the words in reverse order?

be not or be to

It seems like you're asking about a choice between two options: "be not" or "be to." However, this phrase is a bit unclear and may not have a standard meaning in English. Can you provide more context or clarify your question so I can assist you better?

I’ve taken words from a famous speech, but not it the correct order.

It sounds like you're referencing a famous speech and have rearranged some of its words. If you provide the specific words or phrases you've rearranged, I can try to help you understand the original context or provide more information based on that. Please share the words or phrases you're working with, and I'll do my best to assist you.

Hamlet

The phrase "To be or not to be" is one of the most famous lines from William Shakespeare's play "Hamlet." It is part of Hamlet's soliloquy in Act 3, Scene 1, where he reflects on the nature of life and death, contemplating whether it is nobler to endure the suffering and challenges of life or to end one's own existence. The full soliloquy goes:

"To be, or not to be, that is the question: 
Whether 'tis nobler in the mind to suffer 
The slings and arrows of outrageous fortune, 
Or to take arms against a sea of troubles 
And, by opposing, end them."

It's a reflection on the existential question of whether it's better to face the hardships of life or to choose non-existence. This soliloquy is one of the most famous passages in English literature and is often quoted or referenced in various contexts.

ago years seven and score four

It seems like you're providing a phrase without a specific question or context. If you have a question or need information related to "seven years ago" or "four scores," please provide more details or clarify your request, and I'd be happy to help.

I’ve taken words from a famous speech, but not it the correct order.

It appears that you've taken words from a famous speech and rearranged them. Without knowing the original speech or having more context, it's challenging to determine the source or provide additional information. If you could provide more details or the actual quote, I'd be happy to assist you further.

Lincoln

The phrase "four score and seven years ago" is the opening of Abraham Lincoln's Gettysburg Address, one of the most famous speeches in American history. The full sentence is:

"Four score and seven years ago our fathers brought forth on this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal."

Lincoln delivered this speech on November 19, 1863, during the American Civil War, at the dedication of the Soldiers' National Cemetery in Gettysburg, Pennsylvania. It's a brief but powerful speech that emphasized the principles of liberty, equality, and the importance of preserving the Union during a critical period in American history.

So, just give it "Hamlet" and "Lincoln" as clues and it figures them out. 

As for the alphabet:

Z Y X W

It seems like you've listed the letters of the alphabet in reverse order, from Z to W. Is there anything specific you'd like to know or discuss related to these letters? Feel free to ask any questions or provide more context, and I'll be happy to assist you.

Can you continue the sequence back to the beginning?

Certainly! Continuing the sequence of the alphabet from W to A:

W V U T S R Q P O N M L K J I H G F E D C B A

Umm, errr, I know, it's a minor matter but..."From the perspective of 1980’s meaning of AGI..." The term didn't exist in the 1980s. Back then it was just AI. AGI was first used in the late 1990s but didn't gain currency until the 2000s. https://en.wikipedia.org/wiki/Artificial_general_intelligence

thanks

I think he was just taking about ChatGPT at that point, but I don't recall exactly what he said.

An ultimatum is a form of bargain as well, just one with very simple terms and no room for negotiation.

This doesn't make any sense. If one side has no say in the arrangement, then it's not a bargain. "Take it or leave it" is not a bargain. It's simply the way things are.

 In the Old Testament, the New Testament, and the Talmud there are lots of stories of covenants, bargains with God, and God turning authority over to humans.

It's not a single unified document, spoken with one voice from one point of view. There texts were written by various people at various times and then assembled. There is in fact quite bit of controversy over the true text of Job. I've given my interpretation by rejecting the ending, where Job gets it all back. Others would disagree, and that changes the nature of the story.

Try this:

While ChatGPT has more or less captured the parallelism in in the St. George story, I thought I’d say a bit more. Here’s slightly different version of I came up with in my original analysis:

1. Impurity: Dragon poisons the town.
2. Appease the dragon, first with sheep, then with children.
3. The king pleads to the villagers to excuse his daughter from beiing sacrified. The villagers refuse.
4. Daughter sent to the lake, dressed as bride.
5. Saint George arrives. Princess tells him to go.
Ω. Saint George wounds the dragon, under Christian protection.
5’. St. George captures the dragon with the Princess’s girdle.
4’. The princess and St. George lead the dragon back to Silene.
3’. King & people convert to Christianity so that St. George will slay the dragon.
2’. George kills the dragon.
1’. Purity: Church and spring with curative waters.

The impure state at the beginning (1) is mirrored by the pure state at the end (1’). The town was sick now it is a source of curative waters. The townspeople attempt to appease the dragon, to no avail (2). But then, just before the end, St. George slays the dragon (2’), eliminating the need for appeasement.

The king and the villagers are at odds over his daughter in 3. But they are united in 3’ and convert to Christianity. In 4 the daughter is sent to the lake, and thus presented to the dragon, as a bride. In 4 the daughter and St. George lead the dragon back to the village. In 4 the dragon had been driving the action, but in 4’ it is passively led.

In 5 St. George just happens upon the lake where the princess tries to send him away. In 5’ St. George calls on the princess (“give me your girdle”) to help him lead the dragon away. Finally, in the center (Ω) St. George takes command of the dragon by wounding him and thus bringing it under his power.

Think somber and grim & it's been a long time since I've seen the film. Also, mystical.

Oh, probably nothing beyond a mood.

Thanks.

I agree, the UI is the best I've seen, and I've been on the web since the beginning. I still have fond memories of, for example, Salon's Table Talk.

Thanks for this. Very useful.

As sort of an aside, in some way I think the confabulation is the default mode of human language. We make stuff up all the time. But we have to coordinate with others too, so that places constraints on what we say. Those constraints can be so binding that we've come to think of this socially constrained discourse as 'ground truth' and free of the confabulation impulse. But that's not quite so.

While of course this is easy to rationalize post hoc, I don’t think falling user count of ChatGPT is a particularly useful signal.

I agree with that. Perhaps those who've dropped off were casual users and have become bored. But there are other complaints. The continued existence of confabulation seems more troublesome. OTOH, I can imagine that coding assistance will prove viable. As I said, the situation is quite volatile. 

You're right, and I don't know what Gioia would say if pressed. But it might be something like: "Millions of people will be replaced by bots and then the businesses will fall apart because the bots don't behave as advertised. So now millions are out of jobs and the businesses that used to employ them are in trouble."

You're probably right. I note, however, that this is territory that's not been well-charted. So it's not obvious to me just what to make of the inconsistency. It doesn't (strongly) contradict Gioia's main point, which is that LLMs seem to be in trouble in the commercial sphere.

Whoops! Sorry about that. Link added. There's lots of interesting stuff in the rest, including some remarks about talent, inventiveness, the academic world, and philanthropy. As you may know, Wolfram was gifted in the very first round of MacArthur Fellowships.

FWIW, here's a link to a talk Hinton recently gave at Cambridge in which he gave a fairly detailed account of how his thinking about AI changed recently: https://youtu.be/rGgGOccMEiY

Thanks. I don't speak Japanese. I'll take a look at the slack channel.