https://www.youtube.com/watch?v=flXrLGPY3SU&t=348s&ab_channel=Wolfram

Understanding ChatGPT: How It Works and Why It Works

Introduction

In this discussion, we will be talking about the technology of neural nets, specifically ChatGPT, a large language model that can produce reasonable human-like essays. We will explore how ChatGPT works, and why it is able to produce such coherent responses.

How ChatGPT Works

Given an initial piece of text, ChatGPT is trying to continue that piece of text in a reasonable human-like way that is characteristic of typical human writing.
It uses its vast knowledge base of the web and millions of books to determine the statistically sensible continuation from the initial prompt.
ChatGPT writes its responses one word at a time, without a global plan for what will be written next.
Despite this, the responses produced by ChatGPT feel coherent and structured.

Why ChatGPT Works

ChatGPT is able to work so well because it is built upon decades of research in neural nets, and specifically on transformer models, which are able to process large amounts of sequential data with high accuracy.
It is also able to work because it has access to an immense knowledge base, allowing it to produce responses that are grounded in common sense and human knowledge.
ChatGPT's ability to produce coherent responses one word at a time is made possible by the use of attention mechanisms, which allow it to focus on relevant parts of the input text and the generated output.
Finally, ChatGPT is able to learn from its mistakes and continuously improve, thanks to its training on large amounts of data and its ability to fine-tune on specific tasks.

Chat GPT is an AI language model that can generate human-like text. It uses a neural network to predict the next word based on the input it receives. However, simply selecting the most probable word does not always result in coherent and interesting text. Therefore, Chat GPT uses a temperature parameter to vary the probability distribution and generate more diverse output.

Strategy for Text Generation

Chat GPT's strategy for generating text involves predicting the next word based on the input received. However, instead of always selecting the most probable word, it uses a temperature parameter to introduce randomness and diversity into the output. The higher the temperature, the more likely it is to select less probable words.

Temperature Parameter

The temperature parameter is a way to vary the probability distribution of words and introduce randomness into the text generation process. A higher temperature means more diversity in the output, while a lower temperature means a higher probability of selecting the most probable word.

Neural Network Architecture

Chat GPT uses a neural network architecture to generate text. The GPT-2 model is a popular model used for text generation. However, as the model gets larger, it becomes more difficult to run on a single computer. Therefore, smaller versions of the model are used for simpler tasks like chatbot development.

Using Chat GPT

To use Chat GPT, one can access the model through the Wolfram Neural Net repository. Input a starting prompt and the model will generate text based on the predicted probabilities of the next word. The output can be varied by adjusting the temperature parameter.

ChatGPT is a remarkable technology that is able to produce coherent and human-like responses to a wide range of prompts. Its success is due to a combination of factors, including its access to a vast knowledge base, its use of attention mechanisms, and its ability to continuously improve through training and fine-tuning.

///

Okay, hello everyone. Well, usually in this time slot each week, I do a science and technology Q&A for kids and others, which I've been doing for about three years now, where I try and answer arbitrary questions about science and technology. Today I thought I would do something slightly different. I just wrote a piece about chat GPT, what's it actually doing, why does it work? I thought I would talk a bit about that here and then throw this open for questions and I'm happy to try and talk about all things kind of chat GPT AI, large language models and so on that I might know about. Alright, so bursting onto the scene what a couple of months ago now was our friend chat GPT. I have to say it was a surprise to me that it worked so well. I've been following the technology of neural nets for I've looked out now 43 years or so. And there have been moments of significant improvement and a long periods of time where it was an interesting idea but it wasn't clear where it was going to go. The fact that chat GPT can work as well as it does, can produce reasonable human-like essays is quite remarkable, quite unexpected. I think even unexpected to its creators. And the thing that I want to talk about is, first of all, how does chat GPT basically work? And second of all, why does it work? Why is it even possible to do what has always seemed to be kind of a pinnacle of human kind of intellectual achievement of, you know, write that essay describing something? Why is that possible? I think what Chatchy PT is showing us is some things about science and about language and about thinking, things that we kind of might have suspected from long ago, but haven't really known and it's really showing us a piece of sort of scientific evidence for this. Okay, so what is Chatchy PT really doing? The starting point is it is trying to write reasonable, it is trying to take an initial piece of text that you might give and is trying to continue that piece of text in a reasonable human light way that is sort of characteristic of typical human writing. So you give it a prompt, you say something, you ask something, and it's kind of thinking to itself, I've read the whole web, I've read millions of books. How would those typically continue from this prompt that I've been given? What's the sort of the reasonable expected continuation based on kind of some kind of average of a few billion pages from the web, a few million books, and so on. That's what it's always trying to do. It's always trying to continue from the initial prompt that it's given. It's trying to continue in sort of a statistically sensible way. So let's say, let me start sharing here. Let's say that you had given it the you had said initially, the best thing about AI is its ability to then chat GPT has to ask what's it what's it going to say next now what one thing I should explain about chat GPT that's kind of shocking when you first hear about it is those essays that it's writing it's writing it one word at a time as as it writes each word it doesn't have a global plan about what's going to happen, it's simply saying what's the best word to put down next based on what I've already written. And it's remarkable that in the end one can get an essay that sort of feels like it's coherent and has a structure and so on, but really in a sense it's being written one word at a time. So let's say that the prompt has been the best thing about AI's its ability to.

Okay, what's Chattu BD going to do next? Well, what it's going to do is it's going to say, well, So what should the next word be? Based on everything I've seen on the web, etc., etc., etc. What's the most likely next word? And it knows certain probabilities. What it figures out are probabilities. It says, "Learn has probability 4.5%, predict 3.5%, and so on." And so then what it will then do is to put down the next word it thinks it should put down. So one strategy it could adopt is I'll always put down the word that has the highest probability based on what I've seen from the web and so on. Turns out that particular strategy of just saying put down the thing with the highest probability doesn't work very well. Nobody really knows why. One can have some guesses, but it's something where if you do that, you end up getting these kind of very flat, often repetitive, even sometimes word-for-word repetitive kinds of essays. So it turns out, and this is typical of what one sees in a kind of a large engineering system like this, there's a certain kind of touch of voodoo that's needed to make things work well. And one piece of that is saying, don't always take the highest probability word, take some with some probability, take a word of lower than lower than highest probability. And there's a whole mechanism, It's usually called its temperature parameter, temperature, sort of by analogy with statistical physics and so on, you're kind of jiggling things up to a certain extent. And the higher the temperature, the more you're kind of jiggling things up, and not just doing the most obvious thing of taking the highest probability would. So it turns out a temperature parameter of 0.8 apparently seems to work best for producing things like essays. So, okay, well let's see what it takes. One of the things that's nice to do is to kind of get some sort of concrete view of what's going on. We can actually start looking at sort of on our computer what's it doing. And I should say that this, what I'll talk about here is based on on this piece that I wrote that just came out a couple of days ago. And I should say that every piece of code there is clicked a copy. So if I click every picture is clicked a copy. If I click this, I will get a piece of Wolf and language code that will generate that. Let me go down here and start showing you kind of what how this really works. So what? Chat GPT in the end is, oops, not seeing screen. Interesting. Oh, well, okay, that's, oh, there we go. Okay, well, let me show you again then what I was showing showing before. This is the piece that I wrote and I just wanted to emphasize that every picture and so on that's in this piece has clicked a copy code. Just click it, paste it in to a word from language notebook on a desktop computer or in the cloud and you can just run it. Okay. So let's see how, let's see, let's actually run an approximation at least to chat GPT. So OpenAI produced a series of models over the last several years, and chat GPT is based on the GPT 3.5, I think model. These models got progressively bigger, progressively more impossible to run directly on one's local computer. This is a small version of the GPT2 model, which is something you can just run on your computer, and it's a part of our well from neural net repository and you can just pick it up from there. And this is now the kind of the neural net that's inside a simplified version of chat GPT. And we'll talk about what all of these innards really are later. But for now, we can just do something like say, Let's use that model and let's have it tell us the words with the top five probabilities based on the starting prompt, the best thing about AI is its ability to. So that's those are the top five words. Let me probably can ask it 20 words here. So let's say, these are probably sorted right. We probably want to sort these in reverse order. And this will now show us the, oh, I see, this is sorting. Okay, so this is showing us these words with different probabilities here. I'm actually confused by why this didn't, oh, I know why didn't I know why didn't do that. I know why didn't do that. Let me just make this do what I expect. Okay, here we go. So this is that sequence of words. It's now by the 20th word we're getting down to keep. I don't know, let's go just for fun. Let's go find out what the 50th word was.

Okay, so down here we're seeing words that were thought to be less likely. What does it mean to be less likely? It means that based on essentially chat GBT's extrapolation from what it has seen on billions of documents on the web, this is the word which these are the words which have certain probabilities of occurring next in that particular sentence. Okay, so now let's say we want to go on, we want to say let's say we want to say the best thing about it is ability to and the next word it might pick might be learn but then what's the word it's going to pick after that? Well we could figure that out by just saying here let's say the next word was learn okay then let's say that what we would get next will fill in the learn there and we just say let's get the next top five probabilities for the next word. Okay, so the next word is from. That's the most probable next word is from. So we could say learn from and then the next most probable word is experience. Alright, so let's write a piece of code that automates that we're going to nestedly apply this function that is just taking the most likely word, so to speak, let's do that 10 times. And this is now what we get. This is using the GPT-2 model. This is asking what the most likely continuation of that piece of text is. OK, so there it goes. Now, this is the case where it's always picking the most probable word. As I said before, it very quickly ends up in the zero temperature case, it very quickly ends up getting itself kind of tangled in some loop, let's see if I have the example of what it actually does in that case. Let's see. Yeah, here we go. And This is not a particularly good, impressive essay, and it gets itself quite tangled up. If you don't always pick the most probable word, things work much better. So, for example, and here are some examples of what happens when you use this temperature to kind of jiggle things up a bit, and not always pick the most, the word that's estimated is most probable. It's worth realizing that there's a, I showed you a few examples of less probable words. There's a huge spectrum of how of different words that can occur with progressively lower probabilities. It's kind of a typical observation about language that the, which you see here as well, that the enthmost common word has probability about one over n. And that's what you see for the word that will follow next. And you also see that in general for words and text. OK. Well, we could ask what happens in the zero temperature case for a-- let's see, for the actual GPT-3 model. This is what it does for zero temperature. Now one feature of this is if you use, well, for example, this is a link to the API for OpenAI that's in our packet repository. If you use that link and you simply call GPT3, it will, because this is always picking the most probable word, it'll be the same every time. So there's no randomness to this. happens usually when you're picking these words with, when you're picking non-zero temperature and you're picking words that aren't always the most probable word is there's a certain randomness that's being added and that randomness will cause you to get a different essay every time and that's why if you say regenerate this essay most likely you will get a different essay every time you press that regenerate button because it's going to pick different random numbers the decide which of the words ranked words it's going to use. So this is a typical example of a temperature 0.8 type S/A generated by GPT3. Okay, so the next big question is we've got these probabilities for words and so on. Where do those probabilities come from? So what I was saying is that the probabilities are basically a reflection of what's out there on the web. And those are the things that chat GPT has learnt from. It's trying to imitate the statistics of what it's seen. All right, so let's take some simple examples of that. Let's say we're dealing not with, so chat GPT essentially deals with putting down words at a time, actually they're pieces of words, but we can assume for the simpler cases, they're just words. But what if, let's start off to understand this, let's start off thinking about putting down individual letters at a time. So first question is, if we're going to just put down letters one at a time, what is the, with what probability should we put down what letter? How do we work that out? Okay, let's pick some random text, let's pick the Wikipedia article about cats and let's just count letters in the Wikipedia article about cats. And we see that E is the winner, A is the runner up, T comes next. So based on the sample of English text from the Wikipedia article about cats, this is what we would think about the statistics of different letters. Let's try the Wikipedia article about dogs. Okay, we have probably slightly different. We have an O shows up more with higher probability, probably because there's an O in the word dog and so on. But if we keep going and we say, well, what about really, so that's for these specific samples of English. Let's keep going. Let's make a, let's see, there we go. Let's use a very large sample of English. Let's say we have a few million books and we use that as our sample of English and ask what are the probabilities for different letters in that very large sample and we'll see what many people will immediately know that E is the most common letter followed by T, A, et cetera. Okay, so these are our probabilities. So now let's say that we want to just start generating text according to those probabilities. So this is probably just, yeah, just fill those in. Oh, there we go. There are the frequencies. And let's just fill in. Let's just have it start generating letters. This is just generating letters according to the probabilities that we get from the occurrence of those letters in English. So that was asking it to generate 500 letters with the correct probabilities to correspond to English text. That's really bad English text there, but that's that should have the number of ease should be about 12%. The number of T's should be about 9% and so on. Okay, we can make it a little bit more like English text by going ahead and let's append a certain probability to have a space. And now we can, let's make a bigger version of this. And now this is generating puts English text with the correct probabilities for letters and spaces and so on. We can make it a little bit more realistic by having it be the case that term, the word lengths, in this case here, we're just chopping it into words by saying there's an 18% chance that a character is a space, which is, here what we're doing is we're saying, let's insist that words have the correct distribution of lengths. And this is now the text that we get where the words have the correct distribution of length the letters have the correct probability of occurring with e being the most common and so on. Clearly, clearly not English. Clearly a lose if chat GPT was generating this, it would be a fail. But this is something which at the level of individual letters is statistically correct. If we said, if we asked, you know, can you tell that this isn't English by just looking at the chances of different letters, it would say this is English. And different languages, for example, have different characteristic signatures of frequencies. You know, if we were to pick this, or I don't know what, I'm sure if we'd pick this for English, and we were to do the corresponding thing for, let's say, which we'd pick, let's try, Spanish here, for example. and we'll get slightly different frequencies.

Okay, those are somewhat similar but not quite the same. Okay, so that's what happens if this is sort of generating English text with the correct single letter statistics. We could just plot the probabilities for those individual letters, oh boy, more complicated needed to be. Okay, that's just the probability for each letter to occur, so E is the most common, Q is very rare, etc. In this case, what we're assuming is that every letter is sort of picked at random independently. However, in actual English, we know that's not the case. For example, if we've had a queue that's been picked, then with overwhelming probability, the next letter that will occur is a U. And similarly, other kinds of combinations of letters, other kinds of two grams, other kinds of pairs of letters. So we can, instead of asking for the probability of just an an individual letter we could, for example, say, what's the probability for a pair of letters coming together? So here we go. So this is asking, this is saying, given that the letter B occurred, what's the probability for the next letter to be E? So it's fairly high. The probability for the next letter to be F is very low. Over here, when there's a Q, the probability for next letters is only substantial when there's a U as the next letter. So that's what it looks like to have. That's what this combination of pairs of letters, the probabilities for combinations of pairs of letters. So now, let's say that we try and generate text, letter at a time, with not just dealing with the individual probabilities of letters, but also the probabilities of pairs of letters. Okay, so now we do that and it's going to start looking a bit more, a little bit more like real English text. There's a couple of actual words here like on and the and well, Tesla I guess is a word of sorts. And this is now sort of getting a bit closer to actual English text because it's capturing more of the statistics of English. We can go on instead of just dealing with having the correct probabilities for individual letters, pairs of letters and so on, we can go on and say, "Let's have the correct probabilities for triples of letters, combinations of four letters and so on." And this is, actually, these numbers are probably off by one because those are really letters on their own. These are pairs of letters and so on. So this is six tuples of letters. And we can see that by the time you've got, by the time you're saying, I want to follow the probabilities for six tuples of letters, we're getting complete English words like average and so on. And the fact that that's how it finishes, that's why auto-complete when you type on a phone or something like that can work as well as it does. because by the time you have AVER, there's only a limited number of words that can follow that. And so you've pretty much determined it, and that's how the probabilities work when you're dealing with blocks of letters rather than small numbers of letters. Okay, so that's kind of the idea of sort of you're capturing the statistics of letters, the statistics of sequences of letters, and you're using that to randomly generate kind of text like things. So let's some, we can also do that not just with probabilities of individual letters with probabilities of words. So in English, there are maybe 40 or 50,000 sort of fairly commonly used words and we could simply say based on some large sample from millions of books or something, what are the probabilities of those different words and the probabilities of different words have changed over time and so on. But let's say we say what, what are, let's say, over the course of all books or for the current time, what are the probabilities for all those, let's say, 50,000 different words, and now just start generating sentences where we picked those words at random, but with the probabilities that correspond to the frequencies with which they occur in sort of the samples of English text. So there's a sentence we get by that method and it's a sentence where, well, these words are, you know, occurring with the right probability, this sentence doesn't really mean anything. It's just a collection of random words. Now, we can do the same thing we did with letters. Instead of just saying we use a certain probability for an individual word, we say, We correctly work out the probabilities for pairs of words based on our sample of English text and so on. We do that. It's actually computationally already comparatively difficult thing to do this even for pairs of words because we're dealing with sort of 50,000 square different possibilities, etc, etc. But now let's say we start with a particular word. Let's say we start with a word cat. That's our sort of prompt here. Then these are sentences that are generated with the correct probabilities for pairs of words. So we'll see things like the book and, well, throughout in is a little bit bizarre, but confirmation procedure, I guess. Those are, that's a pair of words that occur together a bunch, at least in the, in the, in the place where all this text was sampled from. So this is what you get when you're sampling text, sort of pairs of words at a time. This is kind of a very pre-chatchy PT. This is a very sort of super minimalist version in which it's just dealing with statistics of pairs of words as opposed to the much more elaborate stuff that it's that it's really doing. Now you could say, well, how about to do something more like what chat GPT does, let's just instead of picking pairs of words, let's pick combinations of five words or 20 words or 200 words, you know, let's ask it to given the prompt that we've specified, let's ask it to add in the next 200 words with the probability that at the, what you would expect based on what's out there on the web. So maybe we just make a table of what's the chance of having this three word combination, four word, five word combination. Okay, here's the problem with that. The problem is there just isn't enough English text that's ever been or text of any language that's ever been written to be able to estimate those probabilities in this direct way. Well, in other words, by the time you're at, you know, it's that there may be 40,000 common English words. That means the number of pairs of words that you have to ask the probability of is 1.6 billion. The number of triples is 60 trillion and you pretty quickly end up with something where you couldn't possibly there. There just isn't enough text that's been written in the few billion web pages that exist and so on to be able to sample all of those 60 trillion triples of words and say what's the probability of each one of these triples? By the time you get to like a 20 word essay, you're dealing with the number of possibilities being more than the number of particles in the universe, you wouldn't even be able to record those probabilities even if you had text written by sort of an infinite collection of monkeys or something imitating humans that was able to do that. So, how do we deal with this? How does ChatGPT, it's, how did, how does it deal with the fact that it, it can't sample from the web enough text to be able to just make a table of all those probabilities. Well, the key idea, which is a super old idea and the history of science, is to make a model. What is a model? A model is something where you're kind of summarizing data, you're summarizing things in a way where you don't have to have every piece of data, you can make, you can just have a model which allows you to predict more data even if you didn't immediately have it. So quintessential example, very early example of modeling was Galileo, late 1500s, trying to figure out things about objects falling under gravity and going up the tower of Pisa and dropping cannonballs off different levels on the tower of Pisa and saying how long does it take for these things to hit the ground? So he could make a plot. Oh, that's remarkably complicated way to make this plot, okay? could make a plot of, you know, I don't know how many floors there actually are on the tower piece, but imagine there were this number of floors, you could make a plot, and you could say, measure, you know, in those days by taking his pulse or something, how long did it take for the cannonball to hit the ground? And so this is as a function of what floor it was dropped from, how long it took the cannonball to hit the ground. So there's data about specific times for specific floors, but what if you want to know how long would it take for the cannonball to hit the ground if you were on the 35th floor which didn't happen to have been explicitly measured? So this is where kind of the idea of well let's make a model comes in and sort of a typical thing you might do is to say well let's just assume that it's a straight line, assume that that the time to hit the ground is a function of the floor. And this is the best straight line we can fit through that data. This allows us to predict what the time to hit the ground from a floor that we didn't explicitly visit will be. So essentially, this model is a way of summarizing the data and summarizing what we expect to do when we continue from this data. The reason this is going to be relevant to us is, as I mentioned, there isn't enough data to know these probabilities for different words just from actual texts that exists. So you have to have something where you're making a model where you're saying, assume this is sort of how things generally work. This is how we would figure out the answer when we haven't explicitly made a measurement. So we can make different models and we'll get different results. So for example, we could say, here's another model that we might pick. This is a quadratic curve through these particular data points. Now, it's worth realizing that there's no modelless model. You're always making certain assumptions about how things work. And in the case of these problems in physics, like dropping balls from from towers and so on. We have a pretty good expectation that these sort of simple mathematical models, mathematical formulas and so on, are likely to be things that will work. Doesn't always happen that way. This is another mathematical function. This is the best version. It has some parameters in this model. This is the best version of that model for fitting this data. And you can see it's a completely crummy fit to this data. So if we assume that this is sort of in general the way things work, we won't be able to correctly reproduce what this data is saying. In the case of this model, I think it has three parameters that are trying to fit this data and doesn't do very well. In the what Chattu BT is doing, it basically has 175 billion parameters that it's trying to fit to make a model of human language. trying to hope that when it has to estimate the probability of something in human language, that it does better than this, that with its 175 billion parameters, that the underlying structure it's using is such that it's going to be able to more correctly than this, for example, estimate the probabilities of things. So let's see. All right, so the next big thing to talk about is doing things like dropping balls from cows, a pizza and so on, that's something where we've learnt over the last 300 years since Galileo and so on that there are simple mathematical formulas that govern those kinds of processes, physical processes in nature. But when it comes to a task like what's the most probable next word or some other kind of human-like task, we don't have a simple kind of mathematics style model. So for example, we might say, here's a typical human-like task. We're asked to recognize from an array of image and an array of pixels, which digit out of the ten possibilities is this one? And so we humans do a pretty good job of saying, well, that's a four, that's a two and so on. But we need to ask sort of how do we think about this problem? So one thing we could say is, let's try and do the thing that we were doing where we say, let's just collect the data and figure out the answer based on collecting data. So we might say, well, let's get ourselves a whole collection of fours and let's just ask ourselves when we are presented with a particular array of pixel values, does that array of pixel values match one of the fours that we've gotten as sample? The chance of that happening is incredibly small and it's clear that we humans do something better than that. It doesn't matter where the individual pixels fell here, so long as it roughly is in the shape of the four, we're going to recognize it as a four. So the question then is how does that work? And what we found is that it's-- well, let's say this is using-- this is actually using this sort of a standard machine learning problem. This is using a simple neural net to recognize these handwritten digits. And so we see it gets the right answer there. But if we say, well, what's it really doing? Let's say we give it a set of progressively more blurred digits here. At the beginning it gets them right, then it quotes gets them wrong. What does it even mean that it gets them wrong? We know that this was a two that we put in here, and we know we just kept on blurring that too. And so we can say, well, it got it wrong because we knew it was supposed to be a two. But if we sort of zoom out and ask what's happening at a broader level, we say, well, if we were humans looking at those images, would we conclude that that's a two or not? By the time it gets blurred enough, we humans wouldn't even know it's a two. So to sort of assess whether the machine is doing the right thing, what we're really asking is, does it do something more or less what, what, like, what we humans do? So that becomes the question, is it not, we don't get to ask for these kind of human-like costs, there's no obvious right answer. It's just, does it do something that follows what us humans do? And, you know, that question of, of, uh, what's the right answer? Okay, for humans, we might say, well, up up there, you know, most humans would recognize that as a two. If instead we had visual systems like bees or octopuses or something like this, we might come to completely different conclusions. Once things get sort of blurred out, we might, the question of what we consider to be too like might be quite different. It's a very human answer that that to say that that that still looks like a two, for example, depends on our visual system. It's not something where there's sort of a mathematically precise definition of that has to be a two. Okay, so question is how do these models, how do these models, which we're using for things like image recognition, how do they actually work? the most popular by far and most successful at present time approach to doing this is to use neural nets. And so, okay, what is a neural net? It's kind of an idealization of what we think is going on in the brain. What's going on in the brain? Well, we all have about 100 billion neurons in our brains, which are nerve cells that have the feature that when they get excited, they produce electrical signals, maybe a thousand times a second. They and each nerve cell has it's taking that electrical signal and it's it has sort of wire like projections from the from the nerve cell that are connecting to maybe a thousand, maybe 10,000 other nerve cells. And so what happens in a sort of rough approximation is that you'll have electrical activity in one nerve cell, and that will kind of get, communicate itself to other nerve cells. And there's this whole network of nerves that has this elaborate pattern of electrical activity. So, and roughly the way it seems to work is that the extent to which one nerve cell will affect others is determined by sort of the weights associated with these different connections. And so one connection might have a very strong positive effect on a nerve cell. If the first nerve cell is fired, then it's like it makes it very likely the next nerve cell will or that connection might be an inhibitory connection where if one nerve cell fires, it makes it very unlikely for the next nerve cell to fire. There's some whole combination of these weights associated with these different connections between nerve cells. So, you know, what actually happens when we're trying to recognize a two in an image, for example. Well, the, you know, the light, the photons from the image fall on the cells at the back of our eye at a retina. These are photoreceptor cells. They convert that light into electrical signals, the electrical signals, end up going through nerves that get to the visual cortex at the back of our head. And there's an array of nerves that correspond to all the different, essentially pixel positions in the image. And then what's happening is that within our brains there's this sequence of connections, there's sort of layers of neurons that process the electrical signals that are coming in. And eventually we get to the point where we kind of form a thought that image that we're seeing in front of us is a two. And then we might say it's a two. But that process of sort of forming the thought, that's what we're talking about is kind of this process of recognition. I was talking about in the actual neural nets that we have in brains, but what is being done in all of these models, including things like chat GPT, is an idealization of that neural net. Okay, so for example, in the particular neural net we were using for image recognition, this is kind of a, well from language representation of that neural net, and we're going to talk about, not in total detail, but we're going to talk about all these pieces in here. It's very kind of engineering/biological. There's a lot of different funky little pieces here that go together to actually have the result of recognizing digits and so on. This particular neural net was constructed in 1998 and it's really was done as a piece of engineering. So how do we think about the way this neural networks? Essentially, the sort of the key idea is the idea of attractors. That's an idea that actually emerged from mathematical physics and so on. But it's a key idea when we're thinking about neural nets and such like. And so what is that idea? The idea is, let's say we've got all these different handwritten digits, the ones, the twos, etc., etc. What we want is if we lay all these digits out in some way, what we want is that if we are sort of near the ones, we are kind of attracted to the one spot. If the thing we have is kind of near the twos, we're attracted to the two spot. This is kind of the idea of attractors is Imagine that you have some I don't know mountain scape or something like this and you are you know You're a drop of water that falls somewhere on the mountain You are going to sort of roll down the mountain until you get to this minimum that is for From your particular part of the mountain, but then there'll be a watershed and if you're a raindrop that falls somewhere else You'll roll down to a different a different minimum a different lake And it's the same kind of thing here. When you move far enough away from the thing that looks like a one, you'll roll down into the, into the twos attractor rather than the ones attractor. That's kind of the idea there. Now, let's see, we can, let's, let's make a kind of idealized version of this. Let's say, we've got a bunch of points on the plane. Let's say those are coffee shops. And you say, I'm always going to go to the closest coffee shop to me. Well, this so-called Voronoy diagram shows you this sort of the division, the watersheds between coffee shops. If you're on this side of this watershade, you'll go to this coffee shop. If you're on that side, you'll go to this coffee shop. So that's kind of a minimal version of this idea of attractors. All right, so let's talk about neural nets and their relationship to attractors. So let's take an even simpler version. Let's just take these three attractors. there's the zero attractor, the plus one attractor, the minus one attractor. Well, it's going to say if we fall in this region, we'll have coordinates, x and y coordinates. So if we're in this region here, we're going to eventually, we're going to want to go to say the result is zero. We're in the zero, we're in the basin of the zero attractor, and we want to produce a zero. Okay, so we can kind of say, we can say as a function of the position x and y that we start from, what output do we want to get? Well, in this side, we want to get a 1, this one we want to get, what is that? A minus 1, there we want to get a 0. This is the thing that we are trying to, We're trying to set up as something where we'll have this kind of behavior. Okay, well now let's pull in a neural net. So this is a typical tiny neural net. Each of these dots represents an artificial neuron. Each of these lines represents a connection between neurons and the kind of the blue to redness represents the weight associated with that connection with blue being the most negative, red being the most positive here, and this is showing different, this is showing a neural net with particular choices for these weights by which one neuron affects others. Okay, so how do we use this neural net? Well, we feed in inputs at the top. We say those top two neurons got values 0.5 and minus 0.8, for example. Interpreting that in terms of the thing we're trying to work with, that's saying we're at position x equals 0.5, y equals minus 0.8 in that diagram that we had drawn. So now this neural net is basically just computing a certain function of these values x and y. And at each step what it's doing is it's taking these weights and it's simply taking for this neuron here what it's doing is it's saying I want this weight multiplied by this value here, this weight multiplied by this value here. And then what it says is I'm going to add those two numbers up, the numbers based on the weights multiplied by the original number. Then there's a thing we add. We add a constant offset, different offset for, for, for, we add this constant offset. And then we say we get some number out. And then the kind of the weird thing one does, which is sort of inspired by what seems to have them biologically is we have some kind of threshold in function. We say, for example, this is a very common one to use Relu. If that total number is less than zero, make it be not its actual value, but just zero. If it's greater than zero, make it be its actual value. And there are a variety of different so-called activation functions, activation because they're what determine what the activity of the next neuron sort of down the line will be based on the input to that neuron. So here, again, at every step, we're just collecting the values from the neurons at the previous layer, multiplying by weights, adding this offset, applying that activation function, RELU, to get this value minus 3.8 in this case. And what's happening here is we start off with this values 0.5 minus 0.8. We go through this whole neuron that, in this particular case, at the end, comes out with value minus one. Okay. What, what does that neural net, this neural net here, the one we've just been showing, what does that do as we change those inputs? Well, we can plot it. That's what that neural net actually does. So as a function, so remember, goal is to every time we have a value in this region, we want to give a zero, this region, we want to give a minus one and so on. This is what that particular neural net succeeds in doing. So it didn't quite make it to give the zero one minus one values, but it's kind of close. So this is a neural net that's been kind of set up to be as close as it can be for one of that size and shape and so on to giving us the exact function we wanted to compute. Well, how do we think about what this neural net is doing? The neural net is just computing some mathematical function. So for the particular neural net I was showing, if the W's are the weights and the B's are the offsets and so on, the F is the activation function. This is the messy sort of algebraic formula that says what the value of the output is going to be as a function of x and y, the values of the inputs. So now the question is, well as we look at simpler neural nets, what kinds of functions can we actually compute? So this is at the sort of minimum level. This is a single, this is a neuron here, it's getting input from two other neurons. What function is it computing? Well, it depends on the weights. These are the functions that get computed for for these different choices of weights. Very simple functions, in all cases, just these ramps. So now we can ask, well, okay, let's use a slightly more sophisticated neural net. Here's still a very small neural net. This is the best it can do in reproducing the function we want to get. Like a bigger neural net does slightly better, and even bigger neural net, up, it pretty much nailed it. Didn't quite nail it. Right at the boundary, it's a bit confused instead of going straight from red to blue. it's got this area where it's giving yellow and so on. But in a first approximation, this little neural net was a pretty good representation of the mathematical function that we wanted to compute. And this is the same story as what we're doing in that recognition of digits. We're again, we've got a neural net. It happens to have-- I don't know what it was. I think it's about 40,000 parameters in this particular case, that specify kind of-- that are doing the same kind of thing, of working out the function that goes from the array of pixels at the beginning to values 0 through 9 and so on. Well, again, we can ask the question, is it getting the right answer? Well, again, it's a hard question. That's really a human level question to because the question of whether it put a one in the wrong place, so to speak, it's a question of how we would define that. Well, we can do similar kinds of things. Let's say we have other kinds of images. We might try and make a neural in that that distinguishes cats from dogs And here we're showing sort of how it distinguishes those things. And mostly the cats are over in this corner, the dogs are over in this corner. But, you know, the question of what should it really ultimately do? You know, what should it do if we put a dog in a cat suit? Should it say that's a cat or should it say it's a dog? It's going to say some definite thing. The question is does it sort of agree with what we humans would, would assess it to, to, to be? Well, one question you might ask is, well, what's this neural net doing inside when it works out its sort of catness or its dogness? And let's say we start with, let's do this. And we can actually do this. Let's say we start with an image. Oh, maybe. Let's say we start with an image of a cat here. Now we can say what's going on inside the neural net when it decides that this is actually image of a cat. Well, what we can do normally when we are looking at the insides when you're on that, it's really hard to tell what's happening. In the case where the neural net corresponds to an image, we can at least, at least neural nets tend to be set up so that they sort of preserve the pixel structure of the image. So for example, here we can go, this is just going, what is this going? This is going 10 layers down. No, this is, no, this is actually sorry, this is actually going just one layer down in the neural net. And what happens in this particular neural net is it takes that image of a cat and it breaks it up into a lot of different kind of variants of that image. Now at this level, we can kind of say, well, it's doing things that we can sort of recognize. It's kind of looking at cat outlines without the background. It's trying to pull the cat out of the background. It's doing things that we can sort of imagine, you know, describing in words what's going on. And in fact, many of the things that it's doing are things that we know from studying neurophysiology, of brains are what the first levels of visual processing and brains actually do. By the time we're sort of deeper in the neural net, it's much harder to tell what's going on. Let's say we go 10 layers down in the neural net, then we've got, again, sort of this is in the mind of the neural net, this is what it's thinking about to try and decide, is a cat or a dog, things have become much more abstract, much harder to explicitly recognize. But that's kind of what sort of a representation for us of what's happening in the kind of mind of the neural net. And you know, if we say, well, what's a theory for how cat recognition works? It's not clear we can have a theory in the sense of sort of a narrative description, a simple way of describing how does the thing tell that it's a cat. You know, we can't, and if you even ask a sort of human, how do you tell? We say, well, it's got these pointy ears, it's got this and that thing. It's hard probably for a human to describe how they do that recognition. When we look inside the neuralnet, we don't get to have a... There's no guarantee that there's a simple narrative for what it's doing, and typically there isn't. Okay. So we've talked about how neuralnet can successfully go from a cat image to saying that's a cat versus that's a dog. How do you set the neuralnet up to do that? So the way we normally write programs is we say, well, I'm thinking about how should this program work? What should it do? Should it first take the image of the cat, figure out, does it have, what are the shape of its ears? Does it have whiskers? All these kinds of things. That's sort of the typical engineering way to make a program. That's what people did back 15 years ago, 20 years ago, in trying to make sort of recognize images of things. That was the typical kind of approach was to try and recognize sort of human explainable features of images and so on, to as a way to kind of recognize things. The big idea of machine learning is you don't have to do that. Instead, what you can do is just give a bunch of examples where you say this is a cat, this is a dog, and have it be the case that you have a system which can learn from those examples, and we just have to give it enough examples, and then when you show it a new cat image that's never seen before, it'll correctly say that's a cat versus that's a dog. So let's talk about how that's actually done. And what we're interested in is, can we take one of those neural nets? I showed that the neural nets where they have all these weights, and as you change the weights, change the function the neural net is computing, let's say you have a neural net and you want to make it compute a particular function. So let's say let's take a very simple case, let's say we have a neural net and we just want it to compute as a function of x, we want it to compute this particular function here. Okay, so let's pick a neural net. There's a neural net without weights. Let's fill in random weights in that neural net. For every random collection of weights and the neural net, the neural net will compute something. It won't be the function we want, but the law is compute something. It'll always be the case that when you feed in some value up here, you'll get out some value down here. And these are plots of the function that you get by doing that. Okay. The big idea is that if you do it the right way and you can give enough examples of what function you are trying to learn. You will be able to progressively tweak the weights in this neural net so that eventually you'll get a neural net that correctly computes this function. So again, what we're doing here is this is, we're just describing, if this is x, this is, let's say, you know, g of x down here. This is the value of x up here, and this is a g of x for some function g, and that function g that we want is this kind of square wave type thing here. Now, in this particular case, this neural net with these weights is not computing the function we wanted, it's computing this function here. But as we progressively train this neural net, we tweak the weights until eventually we get a neural net that actually computes the function we want. This particular case it took 10 million examples to get to the point where we have the neural net that we want. Okay, so the how does this actually work? How has this actually done? How does one, as I said at the beginning, we just had we started off a neural nets where we had random weights. With random weights, this function x to g of x, with that particular choice of weights, is this thing here which isn't even close to what we wanted. So even if we have, when we have examples of functions, examples of results, how do we go from those to train the neural nets? Essentially what we're doing is we run, we say we've got this neural net. We say let's pick a value of x. 0.2 for example. Let's run it through the neural net. Let's see what value we get. Okay, we get this value here. Oh, we say that value is not correct based on what we were trying to, based on the training data that we have, based on this function that we are trying to, we're trying to train the neural nets to generate that training. It isn't the correct results. It should have been, let's say, a minus one, and it was in fact a point seven or something. Okay. So then the idea is that knowing that we got it wrong, we can measure how much we got it wrong. And we can do that for many different samples. We can take, let's say, a thousand examples of this mapping from value x to function g of x that the neural net computes, and we can say, of those thousand examples, how far off were they? And we can compute what's often called the loss, which is take all those values of what we should have got versus what we actually got. And for example, take the sum of the squares of the differences between those values. And that gives us a sense of if all the values were right on, that would be zero, but in fact, it's not zero because we didn't actually get the right answer without neural net. And so then what we're trying to do is to progressively reduce that loss. We're trying to progressively tweak the neural net so that we reduce that loss. So for example, this is what it would typically look like. You typically have, this is the loss as a function of the number of examples you've shown. And what you see is that as you show more and more examples, the loss progressively decreases, reflecting the function that's being computed by the neural net is getting closer to the function we actually wanted. And eventually the loss is really quite small here, and then the function is really computed by the neural net is really close to the one we wanted. That's kind of the idea of training a neural net. We're trying to tweak the weights to reduce the loss to get to where we want. Okay, so let's say we've got a neural net, particular form of weights. We compute the loss, the loss is really bad. It's pretty far away. How do we arrange to incrementally get closer to the right answer? Well, we have to tweak the weights, but what direction do we tweak the weights in? Okay, so this is a tricky thing that got figured out well in the 1980s, but for you know, that's how to do this in a reasonably, it was known how to do in simple cases, I should say that the idea of neural nets originated in 1943, Warren McCulloch and Walter Petz, were the two guys who wrote this sort of original paper that described these idealized neural nets. And what's inside chat GPT is basically a big version of what was described in 1943. And there was sort of a long history of people doing things with just one layer of neural nets and that didn't work very well. And then early 1980s it started to be some knowledge of how to deal with more layers of neural nets. And then when GPUs started to exist and computers got faster, sort of big breakthrough around 2012, where it became possible to deal with sort of training and using sort of deep neural nets. By the way, for people who are interested, I did a discussion with a friend of mine named Teresonowski who's been involved with neural nets for about 45 years now and has been quite instrumental in many of the, many of the developments that have happened. I did a discussion with him that was live streamed a few days ago, which you can, you can find on the web and so on if you're interested in that, that history. But back to, back to sort of how these things work. What we want to do is we found the loss is bad. Let's reduce the loss. How do we reduce the loss? We need to tweak the weights. What direction do we tweak the weights in in order to reduce the loss? Well, this turns out to be a big application of calculus because basically what's happening is our neural net corresponds to a function. It has a function of the weights. It's a function of once when we compute the loss, We are basically working out the value of this neural on that function for lots of values, vex and y and so on. And that object, that thing we're computing, is a big complicated, we can think of it as an algebraic formula, that we can think of as being a function of all those weights. So how do we make the thing better? How do we reduce the overall value? How do we tweak the weights to reduce this overall loss quantity? Well, we can kind of use calculus. We can kind of say, we can think of this as sort of a surface as a function of all these weights. And we can say we want to minimize this function as a function of the weights. So for example, we might have a in a very simplified case. Oh, this is not good. In a very simplified case, we might have a So, as a function of just two weights, so for example, in those neural nets I was just showing they had, I don't know, 15 weights or something. In the real example of an image recognition network it might be 40,000 weights in Chatchy BT, it's 175 billion weights, but here we're just looking at two weights. And we're asking if this was the loss as a function of the value of those weights, how How would we find the minimum? How would we find the best values of those weights? See, oh, there we go. So this is a typical procedure to use. So called gradient descent. Basically, what you do is you say, I'm at this position on this loss surface, loss surface where the coordinates of the surface are weights. What I want to do is I want to get to a lower point on this loss surface. I want to do that by changing the weights, always following this gradient vector down the hill, the steepest descent down the hill. That's something that you just have to use calculus and you just work out derivatives at this point as a function of these weights and the direction where you are finding the maximum of these derivatives you're going down the hill as much as you can. So that's kind of how you try to minimize the loss is by tweaking the weights so that you follow this gradient descent thing to get to the minimum. Now there's a bit of a bug with this because the surface that corresponds to all the weights, it might have as this picture shows, it might have more than one minimum and actually these minima might not be all at the same height. So for example, if you're on a mountain scape, there might be a mountain lake, there might be a very high altitude mountain lake. And all of the water that's kind of following seapis descent down to get to the minimum only manages to get to that high altitude mountain lake. Even though there's a low altitude mountain lake that will be a much lower value of the loss, so to speak, that isn't reached by this gradient descent method. It's never, you get stuck in a local minimum, you never reach the more global minimum. And that's kind of what potentially happens in neural nets is you can be, okay, I'm going to reduce the loss, I'm going to tweak the weights, but whoops, I can't really get very far. I can't reduce the loss enough to be able to successfully reproduce my function with my neural net or whatever. I can't tweak the weights enough because I got stuck in a local minimum. I don't know how to get out of that local minimum. So this was a sort of a big breakthrough and surprise of 2012 in the development of neural nets was the following discovery. You might have thought that you'd have the best chance of getting a neural net to work well when it was a simple neural net. You kind of get your arms around it and figure out all these weights and do all these calculations and so on. But actually, it turns out things get easier when the neural net and the problem it's trying to solve gets more complicated. And roughly the intuition seems to be this, although one didn't expect nobody I think expected this, I certainly didn't expect this. That it's sort of obvious after the fact. Okay, the issue is, are you going to get stuck as you try and follow this gradient descent? Well, if you're in a kind of low dimensional space, it's quite easy to get stuck. You just get into the one of these mountain lakes you can't go any further. in a high dimensional space, there are many different directions you could go. And the chances are, any local minimum you get to, you'll be able to escape from that local minimum because there'll always be some dimension, some direction you can go that allows you to escape. And that's what seems to be happening. It's not totally obvious it would work that way, but that's what seems to be happening in these neural nets is there's always sort of a, when you have a complicated enough neural that there's always a way to escape, there's always a way to reduce the the loss and so on. Okay, so so that's kind of the um uh this idea of you tweak the weights to reduce the loss. That's what's going on in all neural nets and you can um uh uh there are different schemes for you know what how you do the gradient descent and how big the steps are and there are all kinds of different things. There are different ways you can calculate the loss when we're doing it for language, we're calculating probabilities of words based or probabilities of sequences of words based on the model versus based on what we actually see in the data as opposed to just distances between numbers and so on. But it's the same basic idea. Okay, so when that happens, let's see, we can potentially get every time we run one of these these neural nets, we do all this tweaking of weights and so on, we get something where yes, we got a neural net that reproduces the thing we want. Okay, so these are results from four different neural nets that all successfully pretty much reproduce this function. Now you might ask, well what happens if I go, let's see what happens if I, yeah what happens is I go outside the range where I had explicitly trained the neural net. I'm telling it, I told that my function x goes to g of x for this range here, the one and the white, but then I say, well, I've got the neural net. Now let me try running it for a value of x that I never trained it for. What's it going to give? Well, that will depend on which particular set of choices about which minimum, which weight tweaking, etc., etc., etc. It went to. And so when the neural net tries to figure out things that it wasn't explicitly trained on, it's going to give completely different answers depending on the details of how the neural net happened to get trained. It's kind of like, it knows the things which it's already seen examples of. It's kind of, it's going to be constrained to basically reproduce those examples when you're dealing with things that are kind of out of the box. It might think differently out of the box, so to speak. on the details of that neural net. Alright, so let's see. This whole question about training neural nets is, it's a giant modern art, so to speak, of how to train a neural net. And the over the last, particularly the last decade, there's been sort of increasingly elaborate sort of knowledge of that art of training neural nets, and there's been a certain amount of lore about how these neural nets should get trained that's developed. So how does that, what's sort of in that lore? Well kind of the first question is, you know, what kind of architecture of neural net, how How should you, how many neurons, how many neurons at each layer, how should they be connected together? What should you use? And there have been a number of kind of observations in sort of the art of neural nets that have emerged. So what was believed at the beginning was every different task you want a neural net to do, you would need a different architecture to do it. You would somehow optimize the architecture for each task. It's turned out that that hasn't, that isn't the case. much more that you that there are generic neural net architectures that seem to go across a lot of different tasks. And you might say, isn't that just like what happens with computers? And universal computers, you need only, you can run different software on the same computers, same hardware, different software. That was the kind of idea from the 1930s that launched the whole computer revolution, the whole notion of software and so on. Is this a repetition of that? I don't actually think so. I think this is actually something slightly different. I think that the reason that the neural nets that sort of a small number of architectures cover a lot of the toss neural nets can do is because those tasks that neural nets can do are tasks that we humans are also pretty good at doing. And these neural nets are kind of reproducing something about the way we humans do tasks. And so while the tasks you're asking the neural net to do are tasks that are sort of human like any human like neural net is going to be able to do those tasks. Now there are other tasks that are different kinds of computations that neural nets and humans are pretty bad at doing and those will be sort of out of this zone of it doesn't really matter what architecture you have. Well, okay, so there are all kinds of other things that people sort of wondered about like they said, well, let's make instead of making these very simple neurons that were just like the ones from 1943, let's make more complicated assemblies of things and let's put more detail into the internal operations of the neural net, turns out most of that stuff doesn't seem to matter. And I think that's unsurprising from a lot of science that I've done, not specifically related to neural net, so I think that that's a... that's something that isn't too surprising. Now when it comes to neural nets and sort of how they're architected, there are a few features that it is useful to sort of capture a few features. This is not the right thing. That's the right thing. The there are few features of the data that you're looking at with the neural net that it is useful to that it seems useful to capture in the actual architecture of the neural net. It's probably not in the end ultimately completely necessary. It's probably the case that you could use a much more generic neural net and with enough training enough enough kind of sort of tweaking from the actual data, you'd be able to learn all these things. But for example, if you've got a neural net that's dealing with images, it is useful to initially arrange the neurons in an array that's like the pixels. And so this is sort of representation for the particular network called lennet that we were showing. for image recognition, this sort of representation of there's a first layer of neurons here, then it sort of thickens up into multiple, multiple different copies of the image, which we actually saw. We're looking at those pictures and then it keeps going and eventually it rearranges. What one thing about neural nets to understand is that neural nets take everything they're dealing with and grinds it up into numbers. Computers take everything they're dealing with and eventually grinds it up, grind it up into zeros and ones and to bits. Neural nets right now are grinding things up into arbitrary numbers, you know, 3.72, they're real numbers, not necessarily just zeros and ones. It's not clear how important that is. It is necessary when you're going to incrementally improve weights and kind of use calculus-like things to do that. It's necessary to have these continuous numbers to be able to do that. But in any case, whether you're showing the neural net a picture, a piece of text, whatever, in the end, it's got to be represented in terms of numbers. And that's sort of a-- But how those numbers are arranged, like, for example, here, there's an array of numbers, which are sort of arranged in the pixel positions and so on. The whole array is reconstituted and rearranged and flattened and so on. And then in the end, you're going to get probabilities for each of each of the 10 digits, that will be just a sequence of numbers here, sort of a rearranged collection of numbers. Okay, so let's see. Like picture. There we go. Okay, so this is, so we're talking about sort of how complicated a neural net do you need to achieve it to perform a particular task. Sometimes pretty hard to estimate that because you don't really know how hard the task is. Let's say you want to know on that that plays a game. Well, you can compute the complete game tree for the game. All the possible sequences of games that could occur might be some as a huge game tree. But if you want to get human level play for that game, you don't need to reproduce that whole game tree. If you were going to do a very systematic computer computation and just play the game by looking at all the possibilities, you need that whole game tree, but or you need to be able to go through that whole game tree. But in the case of, if you're trying to achieve sort of human-like performance, the humans might have found some heuristic that dramatically simplifies it, and you might need just some much simpler, so much simpler neural networks. So this is an example of, well, if the neural net is way too simple, then it doesn't have the ability to reproduce in this case the function we wanted, But you'll see that as the neural net's got a bit more complicated, we eventually get to the point where we can indeed reproduce the function we wanted. All right, well, okay. So, and you can ask, you know, are there theorems about what functions you can reproduce with what neural net's? Basically, as soon as you have any neurons in the middle, you can at least in principle reproduce any function, but you might need an extremely large number of neurons to do that. And it's also the case that that neural net might not be trainable. It might not be the case that you can find some, for example, gradient that always makes the loss go down and so on just by tweaking weights. It might be that you couldn't incrementally get to that result. Well, okay. So, whoops, let's say you've got a, you've decided in your architecture of your neural net. and now you want to train your neural net. Okay, so the next big thing is you have to have the data to train your neural net from. And there are two basic categories of training that one does for neural nets, supervised learning and unsupervised learning. So in supervised learning, you give the neural net a bunch of examples of what you want it to learn. So you might say, here are 10,000 pictures of cats, 10,000 pictures of dogs, the pictures of cats are all tagged is being this is a picture of a cat, dogs or this is a picture of a dog, and you're feeding the neural net, these things that are kind of explicit things that you want it to learn. Now, that's what one has to do for many forms of machine learning. It can be non-trivial to get the data. Often there are sources of data that way you're sort of piggybacking on something else like you might get images from the web and they might have alt tags that were text, describing the image and that's how you might be able to associate the, you know, the description of the image, the fact this is a cat to the actual image, or you might have, you know, if you're doing audio kinds of things, you might have something where you, might say, let's get a bunch of videos which have closed captions and that will give us the supervised information on here's the audio, here's the text that correspond to that audio, that's what we have to learn. That's one style of teaching neural nets, supervised learning where you've got data which explicitly is examples of, here's the input you're supposed to get, here's the output you're supposed to give. And that's great when you can get it. Sometimes it's very, very difficult to get the necessary data to be able to train the machine learning system. And when people say, oh, can you use machine learning for this task? Well, if there's no training data, the answer is probably going to be no. unless that task is something that you can either get a sort of proxy with that task from somewhere else or you can or you just have to blindly hope that something that sort of was transferred from some other domain might might work just as when you're doing mathematical models, you might sort of say, well linear models or something worked in these places, maybe we can blindly hope they'll work here doesn't doesn't tend to work that well. Okay, the other form of, no, I should explain another thing about neural nets, it's kind of important, which is that there's something been very critical over the last decade or so, the notion of transfer learning. So that once you've kind of learnt a certain amount with a neural net, being able to transfer the learning that's happening one neural net to a new neural net, to give it a kind of head start is very important. Now that transfer might be, But the first neural net learnt the most important features to pick out an image. Let's feed the second neural net, those most important features and let it go on from there. Or it might be something where you're using one neural net to provide training data for another neural net. So you're making them compete against each other a variety of other things like that. That those actually have different names. The transfer learning thing is mostly the first thing I was talking about. Okay, so there is a lot of issues about how do you get enough training data? How many times do you show the same example to a neural net? You know, it's probably a little bit like humans. For us, when we memorize things, it's often useful to go back and just rethink about that exact same example that you were trying to memorize before again. So it is with neural nets. And there's also questions like, well, you know, you've got the image of a cat that looks like this, maybe you can get what is the equivalent of another image of a cat just by doing some simple image processing on the first cat and it turns out that that seems to work, that notion of data augmentation seems to work surprisingly well, even fairly simple transformations are almost as good as new in terms of providing more data. So, okay, the sort of a, the other big form of learning, that learning methodology that one tends to use is unsupervised learning, where you don't have to explicitly give sort of a thing you got as input, example of output. So for example, in the case of, let's try to keep track of, yeah, in the case of something like ChatGPT, there's a wonderful trick you can use. Let's say ChatGPT's mission is to continue a piece of text. Okay, how do you train it? You've just got a whole bunch of text and you say, okay, you know, chat GPT network. Here's the text up to this point. Let's mask out the text after that point. Can you predict what's going to come? What, you know, can you learn to predict what happens if you take off the mask? And that's the task that it you don't have to explicitly give it, you know, input, output, you're implicitly able to get that just from the original data that you've been provided. So essentially what's happening when you're training the neural net of a chat GPT is you're saying here's all the English text It's from billions of web pages now look at the text up to this point and say can you correctly predict what text will come later Okay, gets it wrong. You can say well, it's it's giving it getting it. It's getting it wrong So let's that's provides a you know that that means it has a that there's some loss associated with that. Let's see if we can tweak the weights and then you're all in that to get it closer to correctly predicting what's going to come next. So in any case, the end result of all of this is you make a neural net, I could show you, you're on that training and I could show you a more from language, it's very easy to train neural nets to, well, let's see. Maybe I shouldn't do this well. Let's see. Let's just do one. So here's a collection of handwritten digits. This is what is this going to be? This may be 50,000 handwritten digits. There we go. So this is a supervised training story where here all the zeros, and they say that's a hundred and zero, and it says it's a zero. Those are the nine. It says it's a nine. Okay, so let's take a random sample of, I don't know, two thousand of those. And now we're going to use that. Okay, there's a random sample of two thousand, a hundred and digit, and what it was supposed to be. Okay, so let's take it, let's get a neural net. Let's say, let's try taking this lanette, neural net. This is now a, um, um, an untrained neural net. Um, and now we can just say if we wanted to, we could should be able to say, uh, just train that neural net with this data, There's that data there. Ah, there you go. Go online 32. Um, let's say, uh, train this. And so what's going to happen is this is showing us the loss. And this is showing us as it's, as it's being presented with more and more of those examples. and it's being shown the same example many, many times, you'll see the loss is going down, and it's gradually learning. Okay, now we have a trained neural net, and now we could go back to our original collection of digits. Let's close that up. Let's go back to our original collection of digits. Let's pick a random digit here. Let's see whether from... Let's just pick another random sample here. Um, let's take five examples there. From, um, I should have not told it to do that. Okay, there we go. So now we can take this train neural net. Here's our train neural net. And let's take the train neural net and let's feed it. That particular nine there. Now, remember, we only trained it on 2000 examples. So it didn't have very much training, but, oops, I shouldn't have done that. should have just used that. Okay. Okay, it successfully told us it was a nine. That's kind of what it looks like to train this is, you know, Wolfman language version of training in neural net. This was a super simple neural net with only 2000 examples, but that's kind of what it looks like to do that, do that training. Okay. So let's see the thing with with chat GPT is that your um well let's let's yeah we can we can keep going and talk about the training of of of that but let's um Before we get to training of chat GPT, we need to talk about one more thing, which is we need to talk about this question of kind of, well, see, do we really need to talk about this? Yeah, I would probably should talk about this. The question of how you represent kind of things like words with numbers. So let's say we are going to have, we've got all these words and we could just number every word in English. We could say Apple is 75, Pear is 43, etc., etc., etc. But there's more useful ways to label words in English by numbers. And the more useful way is to get collections of numbers that have the property that words with nearby meanings have nearby collections of numbers. So it's as if we're placing every word somewhere in some meaning space and we're trying to set it up so that words will have a position in meaning space with the property that if two words if nearby in meaning space they must mean close to the same thing. So here for example our collection of words laid out in one of these meaning spaces, sort of actual meaning spaces like the one used by Chatchy Petia, like what is that one? It's probably 12,000 dimensional maybe. This one here is just too dimensional. We're just putting things like dog and cat, alligator crocodile, and then a bunch of fruits here. And what the main thing to notice about this is that things with similar meanings like alligator and crocodile wind up nearby in this meaning space. And you know, peach and apricot wind up nearby in meaning space. So in other words, we're representing these words by collections of numbers. In this case, just pairs of numbers coordinates, which have the property that those coordinates are some kind of representation of the meaning of these words. So, and we can do the same thing when it comes to images, for example, we could ask whether, when we looked at, and that's exactly what we had, when we were looking at a picture like this, we're sort of laying out different handwritten digits in some kind of meaning of the handwritten digit space, where in that meaning space, the ones that mean one were over here, the ones that mean three were over here and so on. So a question is how do you find, how do you actually generate coordinates that represent the so-called embeddings of things so that when they're sort of nearby in meaning, they will have nearby coordinates. Okay, so there's a number of neat tricks that are used to do this. So a typical kind of setup is, imagine we have this is just a representation of the neural net that we use to recognize digits. It has these multiple layers. Each one is just little wolf language representation of that. What actually does this network do? Well, in the end, what it's doing is it's taking that collection of pixels at the beginning. And in the end, what it's doing is it's computing. What are the probabilities for a particular configuration? It's going to produce a collection of numbers at the end because remember neural nets, all they ever deal with are collections of numbers. numbers. So what it's going to do is it's going to produce a collection of numbers at the end where each position in this collection of numbers, there'll be 10 numbers here, each position is the probability that the thing that the neural net was shown corresponded to a 0 or 1 or 2 or 3 or 4. So what you see here is the numbers are absurdly small except in the case of 4, so we can then deduce from this immediately, okay, that image was supposed to be a four. So this is kind of the output of the neural net is this collection probabilities where in this particular case it was really certain that the thing is a four, so that's what we deduce. Now the thing we can do is we say, well, let's back up one layer in the neural net before we get to that, that, let's just say before Before we had, there's a layer that kind of tries to make the neural net actually make a decision. It's, I think, it's a soft max layer that is at the end that's trying to force the decision. It's trying to exponentially pull apart these numbers so that the big number gets bigger and the small numbers get smaller. Okay, but one layer before, those numbers are a bit more sober in size. Before it's been sort of torn apart to make a decision, those numbers are much more sober in size. And these numbers at this layer give some pretty decent indication of the forness of what we're seeing. This has more information about what that thing that was shown actually is. And we can think about these numbers as giving some kind of signature, some kind of trace of what kind of a thing we were seeing. This is sort of specifying in some sense features of what we were seeing that later on we'll just decide that to four, but all these other subsidiary numbers are already useful. We go back, so this is, we can define these feature vectors that represent the feature vector representing that image there. That's the feature representing this image here. We see that the features for different fours, these vectors were a little bit different, but they're dramatically different between a 4 on 8. But we can use these vectors to represent kind of the important aspects of this 4 here for instance. And if we go back a couple more layers in that neural net, it turns out we can get an array of like 500 numbers that are a pretty good representation, a pretty good sort of feature signature of any of these images. And we do the same thing for pictures of cats and dogs, we can get this kind of signature of what the sort of feature vector associated with what is important about that image. And then we can take those feature vectors and we can say let's lay things out according to different values in those feature vectors and then we'll get this kind of embedding in the case of what we can think of as some kind of meaning space in the case of words. If we look at the raw, so how do we do that for words? Well, the idea is just like for the, for getting sort of a feature vector associated with, let's say, images, we have a task, like we're trying to recognize digits. And then we back up from the final answer, we're training a neural net to do that task. But what we end up doing is we back up from that final we nailed the task and we say what was the thing that was just before you you managed to nail the task that's our representation of the relevant features of the thing. Well you can do the same thing for words. So for example if we say the blank cat and we then ask in our training data what is that blank likely to be? Is it black, is it white, whatever else? that we could try and make a network that predicts what is that intermediate word likely to be? What are the probabilities for that intermediate word? We can train a network to be good at predicting the probabilities for blackness versus whiteness versus whatever other tabby-ness for cats or whatever it is. And once we've got that, we can then back up from the final answer and say, let's look at the innards of the network and let's see what it had done as it got towards coming up with that final result. That thing we get right before it gets to, a little bit before it gets the final result. That will be a good representation of features that were important about those words. And that's how we can then deduce what we can deduce these sort of feature vectors for words. So in the case of GPT-2, for example, we can compute those feature vectors. they're extremely uninformative when we look at them in the full feature vectors. If we, what is more informative is we sort of project these feature vectors down to a smaller number of dimensions, we'll discover that the cat one is closer to the dog one, probably than it is to the chair one. But that's, that's kind of, so what, what, what, chat GPT is doing when it deals with words is it, it's always representing them using these feature vectors that using this kind of embedding that turns them into these collections of numbers that have the property that nearby words have similar representations. Actually, I'm getting a little bit ahead of myself there because the way chat GPT works uses these kinds of embeddings, but it does so for whole chunks of text rather than for individual words. We'll get there. Okay. So I think we're getting on fairly well here. How about the actuality of chat GPT? Well, it's big neural net, millions of neurons, 175 billion connections between them, And what is its basic architecture? The... It's that this sort of a big idea that actually came out of language translation networks where the task was start from English and up with French or whatever else was this idea of what are called transformers. It's an architecture of neural nets that were more complicated architectures used before this actually a simpler one. And the notion is, as I mentioned, when one's dealing with images, it's convenient to have these neurons kind of attached to pixels, at least to sort of laid out in a kind of which pixel is next to which pixel kind of way. There's a so called convolutional neural nets or convenettes are the typical things that are used there. In the case of language, what transformers do is they are dealing with the fact that language is in a sequence. And with a convent for an image, one saying, there's this pixel here, what's happening in the neighboring nearby pixels in the image in a transformer. What one's doing is one saying, this is, here's a word. Let's look at the preceding words. Let's look at the words that came before this word. And in particular, let's pay attention differently to different ones of those words. So, I mean, this gets quite elaborate and engineering-y quite quickly. And it's very typical of a sophisticated engineering system that there's lots of detail here. And I'm not going to go into much of that detail. But this is a piece of the, this is sort of the, in a sense, a front-end of, okay, so remember, what is Chatt GBT ultimately doing? It's a neural net whose goal is to continue a piece of text. So it's going to essentially ingest the piece of text so far, reading in each token of the text. The tokens are either words or pieces of words, like things like the ING at the end of a word might be a separate token. They're sort of convenient pieces of words. They're about 50,000 different possible tokens. It's reading through the text, the prompt that you wrote, the text that it's generated so far. It's reading through all of those things. It is then going to generate, it's then going to, its goal is to then continue that text. In particular, it's going to tell you every time you run through this whole neural net, it's going to give you one new token. It's going to tell you what the next token should be or what the probability is for different choices of the next token should be. So one piece of this is the embedding part where what's happening is it's reading a token and it is doing, I mean this is just, you know, it gets into a lot of detail here. So for example, let's say that the sequence we were reading was hello, hello, hello, hello, hello, bye, bye, bye, bye, bye. This is showing the resulting, this is showing the embeddings that you get. Okay, this is showing you're trying to represent, I said before we were talking about embeddings for words. Now we're talking about embeddings for whole chunks of text and we're asking what is the sequence of numbers that should represent that collection of that piece of text and the way you set that up. I mean, again, this is getting pretty deep into the entrails of the creature. And, well, what you can think of is there are different components to this embedding vector. And let's see, what am I doing here? This picture is showing across the page, it's showing the contribution from each word. And down the page, it's showing the different different pieces of the feature vector that are being built up. And the way it works is, it takes each word and it has, then the position of the word is encoded by a, you could just encode it by saying the binary, the position of the word as a binary digit that says is word number seven, it's, you know, there is a, there is a one-on-one or something, but that doesn't work as well as essentially learning this sort of random looking collection of things which are essentially position tags for words. Anyway, the end result is you're going to make this thing that represents the where you have both where each level is a different sort of feature associated with each of these words and that's that's the thing that's going to be fed into the the next level of the neural net. Okay, so the next big piece is so-called attention block in which I don't know how much this is worth explaining. I talk about this a bit more in the thing that I wrote, but essentially what's happening is the, in the end, it's just a great big neural net, but that neural net has, doesn't have every possible connection in it. It has connections, for example, only connections that look back in the, that look to places that were earlier in that text. And the, it is in a sense concentrating differently on different parts of that text. And you can, you can make a picture here of the amount of attention that it is paying. And by attention, I mean, it's literally the number, the size of effectively the weights that it's using to with which it is weighting different parts of the sequence that came in. And the way it works, I think, for GPT-3, what it does is it, so first of all, it has this embedding vector, which for GPT-3 is about, is 12,288. I don't know why it's that particular, oh, I do know why it's that number. It's multiples of things. Long. And it's taking, it's trying to put together an embedding vector to represent the text so far, in which it has had contributions from words at different positions. And it's sort of, it's figured out how much contribution it should get from words at each different position. Um, well, okay, so it does that. Then it feeds the whole thing to a, a layer of neural net where sort of it has, um, it's a, um, what is it? It's a 12,000 by 12,000 array, which specifies where there are 12,000 by 12,000 weights, which specify for each incoming neuron, each neuron that's incoming has this weight to this outgoing neuron. And the result is you get this whole assembly of weights, which looks like nothing in particular. This is, but these are weights that have been learnt by chat GPT to be useful for its task of continuing text. And you can play little games, you can try and visualize those weights by kind of making moving averages. And you can kind of see that the weights sort of roughly are kind of like randomly chosen. But this is kind of showing you a little bit of the detail inside that randomness. In a sense, you can think of this as being sort of a view into the brain of the of chat GPT of showing you at the level of these individual weights that are in this neural net. What what its representation of human language is right down the level, you know, it's kind of like you take apart a computer and you look at individual bits inside the CPU. This is kind of the same sort of thing for the representation that chat GPT has of language. And turns out there isn't just one of these attention layers. Okay, what happens is the different elements of the feature vector for the text get there different blocks of that feature vector that gets separated out and handled differently. Nobody really knows what the interpretation of those blocks is. It's just been found to be a good thing to do to not treat the whole feature vector the same, but to break it into blocks and treat blocks of pieces in that feature vector differently. Maybe there's an interpretation of one piece of that feature vector that this is, I don't know, words that are about motion or something. It won't be anything like that. It won't be anything as human understandable as that. It's kind of like a human genome or something. It's all, all the traits are all mixed up in the specification. It's like what it's not something where we can easily have a sort of narrative description of what's going on. But what's been found is that you break this kind of feature vector of features of the text up and you have these separate attention heads that have this sort of re-weighting process going on for each one. You do that and this is where you know this is like it's crazy that things like this work. But you do that let's see 96 times for for chat GPT. You're making you're doing the same process 96 times over and this is for GPT2, the simpler version. This is kind of a representation of the things that come out of these attention layers, attention blocks, what the sort of weights that were used there were. And these may look, there is some regularity, I don't know what it means, But if you look at the size of the weights, they're not perfectly, for some layers, they're Gaussian distributed, for some layers, they're not. I have no idea what the significance of that is. It's just a feature of what ChatGPT learnt as it was trying to understand human language from from the web. So, okay. So, again, we've talked about what's in the end that what's happening is, it's just a great big neural net and it's being trained from, we're trying to deduce the waits for the neural net by showing it a whole bunch of text and saying, "What waits do you have to have in the neural net so that the continuation of the text will have the right probabilities for what word comes next?" That's its goal. And so I've sort of described the outline of how that's done. In the end, one has to feed it. The reason it's sort of even possible to do this is that there's a lot of training data to feed it. So it's been fed a significant fraction of what's on the web. There are maybe, I don't know, depends how you describe this. But there are maybe six billion, maybe ten billion kind of reasonably human written pages on the web, Humans actually typed that stuff. It wasn't mostly machine generated, et cetera, et cetera, et cetera. That's on the publicly visible web, not having programs go in and not selecting lots of different things and seeing what you get. That's just kind of raw, what's on the web page. Maybe there's 10, maybe a hundred times as much as that. If you were able to make selections to drill down to go into internal web pages, things like this. But so you've got something like some number of billions of human written pages. And there's a convenient collection called Common Crawl that's got where one goes, where it's some, you know, start from one web page, you follow all the links, you collect all those pages, you keep going, just following links, following links until you've, until you've visited all the connected parts of the web. But the result of this is there's a trillion words of text that you can readily get from the web. There are also probably 100 million books that have been published, maybe 100, I think the best test of maybe 130 million books that have been published, of which 5010 million exist in digitized form. You can use those as a training data as well. That's another 100 billion or so words of text. So you've got truly anish words of text. That's what, and there's probably much more than that in if you have the the transcriptions of videos and things like this. You know, for me personally, I've kind of been, you know, as a kind of a personal estimate of these things, I realized that the things I've written over my lifetime constitute about 3 million words. The emails I've sent over the last 30 years are another 15 million words. and the total number of words I've typed is around 50 million. Interestingly, in the live streams I've done, just in the last couple of years, I have spoken in other 10 million words. So it gives a sense of what human output is. But the main point is there's a trillion words available on that you can use to train a neural net to be able to do this task of continuing from things. It's, let's see. In, right. So the actual process of one thing to understand about training a neural net. There's sort of a question. Okay, there's a question. When we looked at those functions before and we said, how many neurons do we have to have to represent this function well? How many training examples do we have to give to get the neural net trained to represent that function? In those cases, we didn't need very big neural nets. We need a lot of training examples. There's all kinds of effort to understand how many training examples do you actually need? How big a neural net do you actually need to do something like do this text translation thing well. Well, it's not really known, but with 175 billion weights, the sort of the surprise is that chat GPT does pretty well. Now you can ask the question, what's the, how much training does it need? And how many times does it have to be shown those trillion words, what's the relationship between the trillion words and the number of weights in the network? And it seems to be the case that for text, that sort of the number of weights in the network is sort of comparable to the number of training examples. You sort of show it, the training examples about once, if you show it too many times, it actually gets worse in its performance. It's very different from what happens when you're training for mathematical functions and things like this. But one of the things that's an issue is that if you're every time then I should say every time I should explain by the way that the every time the neural net runs what's happening is you're giving it in the case of chat GPT, you're giving it this collection of numbers that represents the text it's gotten so far. And then that Collection Numbers is the input to the neural net. Then you sort of ripple through the neural net, layer after layer after layer, it's got about 400 layers, sort of core layers. It ripple through all those layers. And then at the end, you get some array of numbers, that array of numbers actually are probabilities for each of the 50,000 possible words in English. And that, based on that, it then picks them the next word. But so the main operation of ChatGPT is a very just straight through. You know, you've got this text so far, given that, calculate through this network, say what the next result should be. It's very, it just runs through one time. It's actually very different from the way computers tend to work for other purposes. Most non-trivial computations, you're taking the same piece of, of sort of computational material, the same piece of data, and you compute on it over and over and over again. In simple models of computation like touring machines, that's what's happening all the time. That's what makes computers able to do the non-trivial things computers do. That's is that they are taking maybe a small number of pieces of data and they're just reprocessing things over and over again. What's happening in something like ChatGPT is you've got this big network, you just put it through it once for every token. The only sense in which there's any feedback is that once you get an output, you add that token to the input that you feed it on the next step. So there's kind of an outer loop where you're giving feedback by adding tokens to the text. Then that percolates through, then you get another token that percolates through. So it's a very big outer loop. It's probably the case, certainly in computers, in lots of non-trivial computations that we do, there are lots of inside loops that are happening. quite possibly in the brain. There are inside loops that are happening as well, but the model that we have in chat GPT is this kind of just percolate through once kind of model with a very complicated network that's just percolating through once. So that's works, but one of the things that's tricky is that every time it percolates through, it has to use every single one of those weights. So every token that chat GPT is producing, It's essentially doing 175 billion mathematical operations to see how to use each of those weights to compute the results. Most likely that's not necessary actually, but we don't know how to get, do any better than that right now. But that's what it's doing. So every time it's percolating through doing that, well, when you train chat GPT and you are sort of, you're working out, how do you deal with, oh, making the weights change based on the loss? That's another, you're kind of, every time you make a training step, you're having to kind of do a reverse version of that, of that forward-so-called inference process. It turns out that reverse process isn't that much more expensive than the forward process, but you have to do it a whole lot of times in the training. So typically, if you have a model of size n roughly for text, it seems like you need about n squared amount of computational effort to do the training. And n is pretty big for the case when you're dealing with sort of language and things of the size of chat GPT. And so the training process that just a little bit mathematical square is a really big deal. And it means that you're, you know, to spend hundreds of millions of dollars potentially on doing the training with current GPUs and things like this is what you have to think about doing based on the current model of how neural nets work. Now, I mean, I have to say that there's a lot of aspects of the current model that probably aren't the final model. And, you know, we can plainly see that there are big differences between, for example, things the brain manages to do. For example, one big difference is most of the time when you're training a neural that most of the memory in the, you're doing that by having, you have a bunch of things in memory and you have some computation that's going on. But the things that are in memory are mostly idle most of the time and there's just a little bit of computation that's going on. In brains, every one of our neurons is both a place that stores memory and a place that computes. And it's a different kind of setup when we don't know how to do neural that training, there are various things that have been looked at from the distant past, actually, about how to do this. Even from the 1940s, people were starting to think about some distributed ways to do learning in neural nets, but that's not something that's landed yet as a thing we can do. Okay, case of chat GPT, an important thing was, and this was something, you know, six months ago, a year ago, there were kind of early versions of the GPT family, text, completion systems and so on. And they were kind of the text they produced was only so so. And then something was done by OpenAI with chat GPT, which was that there was an additional step, a reinforcement learning training step that was done, where essentially what was done was humans, told chat GPT, go and make an essay, go and be a chatbot, have a conversation with me. And the humans rated what came out and said, but that's terrible, that's better, that's terrible, et cetera, et cetera, et cetera. And the thing that was done was then that little bit poking turns out to have had it seems to have had a very big effect. That little bit of kind of human guidance of yes, you got the thing from the statistics of the web. Now when you look at what you got, this direction you're going in is a bad direction to go in. It's going to lead to a really boring essay or whatever else. And so that kind of, and by the way, there's a place where where a lot of kind of complication about well, what do the humans really think the network should be that the system should be producing it. The humans say, "We really don't want you to talk about this. We really don't want you to talk about that. That's the place that gets injected is in this reinforcement learning step at the end." But what you do is, for example, is given that sort of the way the humans poked at those essays, you can watch what they did when they They poked at those essays and rated what happened and so on. And you can try and machine learn that set of things that the humans did. Then you can use that to provide much more training data to then retrain a piece of this do retraining of the network to based on the sort of the tweaking that the humans did. You can do sort of fine tuning of this network based on the particular poking that the humans did turned into another network that can then be used to do the training, to produce the examples, to do the training of the main network. So that's a thing that seems to have had a big effect on the actual sort of human perception of what happens in Chattu Pt. And I think the other thing that is a sort of a surprise is that you can give it these long prompts in which you tell it all kinds of things and it will then sort of make use of that in a rather human kind of way in generating the text that comes later. Okay. Big question is, how come this works? Why is it that a thing with only 100 billion or so weights or something can reproduce this sort of amazing thing that seems to require all of the sort of depth of human thinking and brains and things like that human language? How does that manage to work? And I think the key thing to realize is what it's really telling us is a science fact. telling us there's more regularity in human language than we thought there was. It's telling us that this thing that is human language has a lot of structure in it and what it's done is it's learned a bunch of that structure and it's learned structure that we never even really noticed was there and that's what's allowing it to generate these kind of plausible pieces of text that are, you know, that are making use of the structure we know. So we know certain kinds of structure that exists in language. We know the, so for example, here's an example. So one piece of structure that we know, share this again, one piece of structure we know is grammatical syntax. the syntactic grammar. We know that sentences aren't random jumbles of words, sentences are made up with nouns in particular places, verbs in particular places, and we can represent that by a parse tree in which we say, you know, here's the whole sentence, there's a noun phrase, a verb phrase, another noun phrase, these are broken down in certain ways. This is the parse tree, and there are certain that in order for this to be a grammatically correct sentence, this has there are only certain possible forms of parse tree that correspond to a grammatically correct sentence. So this is a regularity of language that we've known for a couple of thousand years. It's only really been codified and it was big effort to codify it in 1956. But it was sort of known, this general idea was known for a long time. But then this that we can kind of represent the sort of syntactic grammar of language by these kinds of rules that say you can put nouns only together with verbs in this way and that way. And to any set of rules, and this has been a big source of controversial linguistics, to any set of rules you can define, there'll always be some weird exception where people typically say this rather than that. But if you, you know, it's at the much like happens in typical machine learning, you know, if you're interested in the 95% result, then there are just rigid rules and there are a few exceptions here and there. Okay, so that's one form of regularity that we know exists in language is this syntactic regularity. Now what one thing we can do, we can ask for sort of chat GPT has effectively implicitly learnt this syntactic grammar. Nobody ever told it. Verbs and nouns go this way in that way. It implicitly learnt it by virtue of seeing a trillion words of text on the web, which all have these properties. And when it's saying, well, what are the typical words that follow? Well, it's going to be words that followed in the examples it had, and those will follow mostly correct grammar. Now we can take a simpler version of this, we can just understand what's going on, we can take a very, very trivial grammar, we can take a grammar that's just a parenthesis, just open and close parentheses, and something is grammatically correct if we open parentheses and they always eventually close. This is a parse tree for a parenthesis, open open open close, open close, et cetera, et cetera. This is the parse tree that sort of shows how you can, it's a representation of the parsing of this sequence of open and closed parenthesis. Okay, so we might say, well, can we train a neural net to, what would it take to train a neural net to know even this particular kind of syntactic grammar? So we looked at a simple, how big was it? It was pretty small. Okay, we made a transformer net with eight heads and length 128. So, but our thing was a lot simpler than ChatGPT, but you can use one of these transformers. And if you look at the post, I made that there's the actual transformer is there, and you can play with it in a more of a language. But in a case, if you give that transformer this sequence here, you say, what comes next? It says, okay, well, 54% probability that there's a closed perennia. Based on, oh, it's training data was a randomly selected collection of correct open close, open close, parenthesis sequences. It has a little bit of a goof here because it says with .038% probability, this is the end of the sequence, which would of course be grammatically incorrect because there's no close for the open parentheses here. If we give something which is correctly closing, then it says, okay, great, there's a 34% probability, this is the end of the sequence, there were no further opens here. It has a little bit of a goof here because it says 15% probability, there's a closed parenthesis that should occur here, which can't possibly be right, because if we put a closed parenthesis here, it doesn't have a corresponding open parenthesis, it's not grammatically correct. But in any case, this gives a sense So what it takes for one of these transformonets, we can look inside this transformonet, we can see sort of what it took to learn this very simple grammar. Chat GPT is learning the much more complicated grammar of English. It's actually easier, probably, to learn the grammar of English because there's so many clues in the actual words that are used to how they're grammatically put together. And there's so many things that we humans wouldn't even notice as wrong in some sense of wrong because they're kind of just what we do. but in this more austere case of just this sort of mathematically defined parenthesis language we do notice. So if we just give it a bunch of open-prone, open-prone, etc. and we ask it what's the highest probability continuation, you'll see it does pretty good up to this point and then it starts losing it. And it's kind of a little bit like what would happen with humans. You know, we can tell at some point here that by just by i that these are correctly closed, it becomes more difficult to tell that when we get out here and it becomes more difficult for the from the network to tell that too. And this is a typical feature of these Nolongats that with these sort of shallow questions of, oh, you just have, you know, you can just see this block of things, you see another block of things, it does fine. When it has to go to much greater depth, it doesn't work so well. For a sort of regular computer that can do loops and things inside, it's very easy to figure out what's happening here, because you effectively just count up the number of open brands, count down the number of closed brands and so on. By the way, if you try this in actual chat GPT, it will confidently assert that it's match parentheses, but it will often be wrong for larger parenthesis sequences. It has the exact same problem. It fails at a slightly larger size, but it's still going to fail. That's just a feature of this kind of thing. Well, okay, so one type of regularity in language that ChatGPT has learnt is syntactic grammar. Another type of regularity, there's one more that you can readily identify, and that's logic. And what is logic? Well, originally when logic was invented by Aristotle, so far as we know, what Aristotle did was effectively, a bit like a machine learning system, he looked at lots of examples of rhetoric, lots of example speeches people gave, he said, "What are some forms of argument that appear repeatedly?" If somebody says, you know, something like people might have said, you know, "All men are mortal, socrates is a man, therefore socrates is mortal, all x is a y, Z is an X, therefore Z is a Y. That logic is taking forms of language and saying these are patterns that are repeated possible patterns in these pieces of language that are meaningful sequences. And originally in syllogistic logic, which is what Aristotle originally invented, it really was very language-based. And people would memorize, you know, in the middle ages, people would memorize these forms of syllogism, the barbarous syllogism, the cellarant syllogism and so on, which were just these patterns of word usage, where you could substitute in a different word for Socrates, but it was still that same pattern, that same structure. So that was that's kind of another form of regularity. And when Chatchy PT says it's, oh, it's it's figuring things out. Well, part of what's figuring out is it knows so just logic, because it's seen as zillion examples, just like Aristotle, who presumably seen a bunch of examples when he invented logic. It's seen a bunch of examples of this sentence follows this sentence in this way. And so it can it's going to do that too. And it says, what's the statistical thing that's going to happen based on based on the web? And so that's some. So by the way, when logic developed by the 1800s, when people like Boul were getting into the picture and making formal logic, it was no longer just these patterns. Boom, it's a pattern, it looks like this. It was more this thing, you could build up many, many layers of structure and you could build, you know, very complicated logical expressions where the whole thing was deeply nested. of course, our computers today are based on those deeply nested logical expressions. Chatchy PT doesn't stand a chance of decoding what's going on with one of those deeply nested kind of mathematical computational style Boolean expressions, but it does well at this kind of Aristotle level kind of structure of sort of templated structure of logic. Okay, well, I wanted to talk just a little bit and then we should wrap up here and I can try and answer some questions. The about kind of, so what are the regularities that Chatchy PT has discovered in this thing that we do, which is language and all the thinking that goes on around language? And I don't know the answer to this. I have some ideas about what's going on. I'll just give it a little bit of a tour. We talked about kind of meaning space, the sort of space of how words arrange in some, how you can arrange words in some kind of meaning space. And we can kind of see how words arrange these are different parts of speech. For a given word, there may be different places in meaning space where different instances of that word occur. This is the word crane and this is different sentences. There are two obvious meanings of crane, you know, the bird and the machine and they sort of break up in meaning space where they are. are, we can look at the sort of structure of meaning space. And nothing we can ask is, is meaning space like physical space? Is it the case that there are parallel lines in meaning space, are there things where we can go from place A to place B, and we, and then in parallel, we transport to new places? Well, so we can ask, you know, if we have analogies, is it the case that we can go, you know, from woman to man, from queen to king that those are sort of parallel parts in meaning space. The answer is, well, maybe a bit, not very convincingly. That's really the question in space, in physical space. This is the question of whether this is like flat space. It's like if we have things moving in flat space, you know, Newton's first law says if the thing is not acted on by a force it'll just keep going in a straight line. Well, then we have gravity and we can represent gravity by talking about the curvature of space. Here, this question is, when we go from, you know, ear to ear, eye to sea, those are sort of, we're moving in a certain direction in meaning space. And in a sense, the question of whether these things correspond to, whether we can do this kind of parallel transport idea is something like how flat is meaning space? How much effective gravity is there in meaning space or something like that? Meaning space is probably not something that's represented in terms of the kinds of things that physical space is represented in terms of, but that's a question. So now when it comes to the operation of ChatQbt, we can think about how is it moving around in meaning space? It's got its prompt, you know, the best thing about AI is, is its ability to, okay? And that's the prompt moving around in meaning space effectively. And then what Chatchy BT does is it continues that by continuing to move in meaning space. And so the question is, is there something like a semantic law of motion, an analog of kind of the laws of motion that we have in physical space, in the meaning space of concepts, words, something where we can say, okay, if it's gone, if it's moved around this way, it's like it's got momentum in this direction of meaning space, it's going to keep going in that mean space. It's nothing like that simple. But the question is, how do we think about? How do we represent kind of the sort of the process of going through meaning space? What we can start looking at that, we can say, for example, the different possible continuations that we get the best thing about AI's ability to, and then what's the next word, well, we can look at this kind of fan of different directions that it could go in meaning space at that point, and we can kind of see there's some direction in meaning space. It tends to go in that direction. It's not going all the way over here, at least not with high probability. Okay, if we keep going, we can kind of just see sort of how that fan develops as we go further out, as we continue that sentence, and we can kind of, this is kind of like our motion in meaning space kind of question. And, you know, I don't know what this exactly means yet, but this is kind of what what it looks like, what the trajectory in meaning space as ChatGPT tries to continue a sentence looks like. The green is the actual thing it chose. I think this is a zero temperature case and the gray things are the other things that were lower probability cases. So that's some, that's kind of what, that's some of you if we want to look, we don't want to do natural science on chat GPT and say what did it discover? What did it discover about how languages put together? One possibility is that there are these sort of semantic laws of motion that describe sort of how meaning, how you move through the space of meanings as you add words into a piece of text. I think a slightly different way to think about this is in terms of what I could call semantic grammar. So syntactic grammar is just about, you know, nouns, verbs, like that, parts of speech, things of that kind. But we can also ask, is there a generalization of that that is sort of more semantic that doesn't just look at that has finer gradations and just saying, it's a noun, it's a verb and says, oh, well, that verb means motion. And when we put this noun together with this noun that's a thing you can move together with this motion word, it does this. We kind of have buckets of meaning that are finer gradations than just parts of speech, but not necessarily individual words. Is there a kind of a semantic grammar that we can identify that is kind of this construction kit for how we put together not just sentences that are grammatically correct, that are syntactically grammatically correct, but sentences which are somehow semantically correct. Now that, that, I strongly think this is possible, and it's sort of what Aristotle was going for. He even talks about categories of sort of semantic categories and things like this. He talks about a variety of things he does it in a way that's based on the fact that it was 2000 years ago and we didn't know about computers and we didn't know about a lot of kinds of formal things that we know about now. Strangely enough, the amount of work that's been done trying to make kind of a semantic grammar in the last 2000 years has been rather small. There was a bit of an effort in the 1600s with people like Leibniz with his characteristic universe, Salas, and various other people trying to make what they call philosophical languages, sort of language, word, independent ways of describing meaning. And then there are more recent efforts, but they've tended to be fairly specific, fairly based on linguistics, and fairly based on the details of structure of human language and so on. And I think this idea that you can kind of have a semantic grammar is a, and that that's what sort of being discovered is that there are these rules that go beyond, that are just rules for how you put together a meaningful sentence. Now, you know, you can get a meaningful sentence, could be something like the elephant flew to the moon. Does that sentence mean something? Sure, it means something. It has a perfectly we can conjure up an image of what that means. Has it happened in the world? No, it hasn't happened so far as we know. And so there's a, but, you know, could it be in a story? Could it be in a fictional world? Absolutely. So this thing about this sort of semantic grammar will allow you to, allows you to put together things which are somehow, which are sort of meaningful things to describe about the world. The question of whether they are realized in the world or have been realized in the world is a separate question. But in any case, the thing that is, to me, interesting about this is it's something I'd long thought about because I've spent a large part of my life building a computational language, well from language system that is an effort to represent the world. world computationally, so to speak, to take the things that we know about about chemicals or lines or images or whatever else and have a computational representation for all those things and have a computational language which knows how all those things work. It knows how to compute the distance between two cities. It knows all of those kinds of things. And in, and so this is, I've been spending the last four decades or so trying to find a way to represent things in the world in this computational fashion so that you can then compute things about those things in an explicit computational way. It's something where, and we've been very successful at being able to do that. In a sense, the story of modern science is a story of being able to formalize lots of kinds of things in the world, and we're leveraging that in our computational language to be able to formalize things in the world to compute things about how they'll work. Now, one feature of that computing about how things work is that inevitably some of those computations are deep computations. They're computations that something like a chat GPT can't possibly do. And in a sense, there's sort of a difference between the things that are the kind of the sort of shallow computations that you can learn from examples in something like a chat GPT that you can say this piece of language that I saw on the web here is, you know, statistically I can sort of fit that in in this place. Just fitting together these sort of puzzle pieces of language is a very different thing from taking the world and actually representing it in some truly sort of formal way computationally so that you can compute things about how the world works. It's kind of like, well, back before people had kind of thought of this idea of formal formalism, maybe 400 years ago or more, everything that anybody figured out was just you think about it in terms of language, in terms of words, in terms of sort of immediate human thinking. What then sort of came in with with mathematical science at first and then computation was this idea of formalizing things and getting these much deeper sort of ways to deduce what happens. And thing I figured out about 30, 40 years ago now was this phenomenon of computational irreducibility. This idea that there really are things in the world where to compute what's going to happen, you have no choice but to follow all those computational steps. You can't just jump to the end and say, oh, I know what's going to happen. It's a shallow kind of thing. And so when we look at some like chat GPT, there are certain kinds of things it can do by sort of matching together, matching these pieces of language, there are other kinds of things it's not going to be able to do. It's not going to be able to do sort of the mathematical computation, the kind of the thing which requires an actual computational representation of the world. For those things like us humans, it's kind of a used tools type situation. And very conveniently, our Wolfmalfa system that used in a bunch of intelligent assistants and so on is has this feature that it's using a wolf and language computational language underneath, but it actually takes natural language input. So it's actually able to take the natural language that is produced by a chat GPT, for example, take that and then turn that into computational language, do a computation, work out the result, get the right answer, feed that back to chat GPT and then it can talk sense, so to speak, rather than just following sort of the statistics of words on the web. So it's a way of, you know, by allowing, but you can get sort of the best of both worlds, by having something where you have this sort of flow of language, as well as something where you have this sort of depth of computation, by having chat GPT use Wolfmalfa as a tool and I wrote a bunch of stuff about that and all kinds of things are happening with that. But the thing that, you know, talking about what did chat GPT discover, I think the thing that discovered is there is a semantic grammar to a lot of things. There is a way to represent using sort of computational primitives, lots of things that we talk about in text. And in our computational language, we've got representations of lots of kinds of things, whether it's foods or chemicals or stars or whatever else. But when it comes to something like, I'm going to eat a piece of chocolate. We have a great representation of the piece of chocolate. We know all its nutrition properties. We know everything about it. But we don't have a good representation yet of I'm going to eat the, I'm going to eat part. What I think Chattu PTA has shown us is that it's very plausible to get sort of this semantic grammar of how one has these pieces of representing these sort of lumps of meaning in language. And I think what's going to happen, and I've been interested in doing this for a long time, I think this is now finally the impetus to really, really roll up on sleeves and do it. It's a somewhat complicated project for a variety of reasons, not least that you have to make these kind of, well, it has to be, you have to make sort of, this process of designing a language something I happened to have been doing for 40 years, designing our computational language. This is a language design problem. And to my mind, the single most concentrated and intellectually difficult thing that I know is this problem of language design. So this is sort of a generalization of that. But I think ChatGPT has kind of shown us what I didn't know how hard it was going to be. I'm now convinced it's doable, so to speak. So what does this, you might ask the question, people might have said, okay, look, we've seen neural nets that do speech to text, we've seen neural nets do image identification. Now we've seen neural nets that can write essays. Surely if we have a big enough neural net, it can do everything. Well, not the neural nets of the kind we have so far that have the training structure that they have so far, not on their own. They will not be able to do these irreducible computations. Now these irreducible computations are not easy for us humans either. You know, when it comes to doing piece of math or worse, if somebody says, "Here's a program. Run this program in your head. Good luck." You know, very few people can do that. It's something where there is a difference between what is sort of immediate and easy for us humans and what is sort of computationally possible. Now another question is, maybe we don't care about the things that aren't easy for humans. It's turned out that we built an awful lot of good technology over the last few centuries based on what amounts to a much deeper level. We haven't really, in our technology, we're not actually going even that far into irreducible computation, but going far enough that it's beyond what we humans can readily do or what we can do with the neural nets that exist today. So I think that's the kind of the thing to understand that there's a certain set of things. What's happening in chat GBT is it's kind of taking the average of the web plus books and so on. And it's saying, I'm going to fit things together based on that. And that's how it's writing its essays. And when it's deducing things, when it's doing logic, things like that, what it's doing is it's doing logic like the way Aristotle discovered logic. It's figuring out, oh, there's a pattern of words. that looks like this and it tends to follow it like that because that's what I've seen in 100,000 examples on the web. So that's kind of what it's doing and it kind of that gives us some sense of what it's going to be able to do. And I think the most important thing it's able to do is it's a form of user interface. You know, we can get, I might get something where I know, Oh, what really matters is three bullet points. But if I'm going to communicate that to somebody else, they're really not going to understand my three bullet points. They need wrapping around that. They need something which is a whole essay, describing, you know, that's the human interface, so to speak. It's just like you could have, you know, the raw bits or something. And that wouldn't be useful to us humans. We have to wrap it in a human like in a sort of human compatible way. And language is sort of our richest human compatible medium. And what what chat GPT is doing is it's able to I think what the way to think about it is it's providing this interface that is well it is just it's generating pieces of language that are consistent but if you feed it specific things that it will talk about so to speak then it's kind of wrapping the thing the specifics with this interface that corresponds to kind of flowing human and language. All right, I went so much longer than I intended. And I see there are a bunch of questions here, and I'm going to go from-- and to try and address some of these. As a question from Antipas, our constructed languages, like Esperanto, more amenable to the semantic grammar AI approach, very good, very interesting question. So I think the one that I was experimenting with was the smallest of the constructed languages, a language called Tokipona that has only 130 words in it. It is not a language that allows one to express, everything one might want to express, but it's a good kind of small talk type language, a small language for doing small talks, so to speak. But it expresses a bunch of decent ideas. And so I was going to look at, yes, that it's a good clue, again, to semantic grammar, that there are these small constructed languages. It also helps. I think, well, I also think probably the largest that constructed languages, Ethquill, is another interesting source. It's a language which is trying to pull in all of the kind of language structures from all the all-knowing languages in some first approximation. That's, yeah, so I think the answer is that yes, I think they're a good stimulus for thinking about semantic grammar. In a sense, when people were trying to do this back in the 1600s, they were very confused about many things. But one gives them a lot of, they've gone a long way given that it was the 1600s. They were confused about things like whether the actual letters that were written as you wrote the language mattered and how that was more so than the structure of things. But there was the beginning of that kind of idea. Okay, I'm going to take these from the end, but I want to go back to some of these others. Okay, Tori is asking, "How come on study what's the best way of prompting chat GBT could a semantic law of motion be helpful?" Undoubtedly, yes, I don't know the answer that. I think it's a good question and I don't really know. I think, yeah, I don't know. Albert is asking, "Is the 4,000 token limits analogous to working memory with accessing larger memory being increasing the token limits or increasing such capability with with his reinforcement learning. Well, I think that the token limits that exists right now are, you know, if you want to have a coherent essay and you want it to know what it was talking about back in that early part of the essay, you better have enough tokens in that are being fed into the neuralnet every time it gets a new token. If it just doesn't know what it was talking about, if it forgot what it was talking about, 5,000 tokens ago, it may be saying totally silly things now, because it didn't know what was there before. So in some sense, I don't think it's like our short working memory, but I think it's kind of like you ramble on, I ramble on a lot, talking about things. And I might have forgotten half an hour later that I talked about that already. I might be telling the same story again. I hope I don't do that. I don't think I'd do that too badly. But you know that's a question of what that that's the kind of thing that happens with this token limit um Let's see let me go back to some of the questions that were asked earlier here um Okay, erim was asking talking more about the tension between superintelligence and computational irreducibility How far can lllm intelligence go? I think I talked a little bit about that. I think this question Oh boy, this is kind of complicated. I mean, so this question about, okay. The universe, the world is full of computational irreducibility. That's, it's full of situations where we know the underlying rules, but we run them as a computation and you can't shortcut the steps. What we've discovered from our physics project is it looks like The very lowest level of space time works just that way. In fact, just earlier today, so lovely work about doing practical simulation of space times and things using those ideas and very much supporting again, it's really computationally irreducible at the lowest level, just like in something like a gas, the molecules bouncing around in this computational irreducible way. What we humans do is we sample sort of aspects of the universe that have enough reducibility that we can predict enough that we can kind of go about our lives. Like we don't pay attention to all those individual gas molecules bouncing around. We only pay attention to the aggregate of the pressure of the gas or whatever else. We don't pay attention to all the atoms of space. We only pay attention to the fact that there's this thing that we can think of as more or less continuous space. So our story has been a story of finding slices of reduced ability slices places where we can predict things about the universe. There's a lot about the universe. We cannot predict. We don't know. So, and if our existence depended on those things, if we had not found kind of these slices of reducibility, we wouldn't be able to have a coherent existence of the kind that we do. So, if you ask sort of where do you go with that? Well, there are an infinite collection. There's an infinite kind of web of pieces of computational reducibility. There's sort of an infinite set of things to discover about that. have discovered some of them as we advance in our science and with our technology for things, we get to explore more of that kind of web-reducibility. But that's really the issue now that the problem is that the way that we humans kind of react to that is we have ways to describe what we can describe. We have words that describe things that are common in our world. We have a word for a camera. We have a word for a chair, those kinds of things. We don't have words, but things which have not yet been common in our world. And, you know, when we look at the innards of chat GPT, it's got all kinds of stuff going on in it. Maybe some of those things happen quite quite often, but we don't have words for those. We don't have a way, we haven't yet found a way to describe them. When we look at the natural world, we've, there are things that we've seen repeatedly in natural world we have words to describe them, we've built up this kind of descriptive layer for talking about things. But one of the things that happens is that if we kind of jump out to somewhere else in the son of universe of possible computations, there may be pieces of reduced ability there, but we don't have words to describe those things. We only have, we know about the things that are near us, so to speak. And so, and gradually as science advances, we get to expand the domain that we can talk about, so to speak, or everything advances. We get to have more words, we get to be able to talk about more things. But in a sense, to have something which operates, it's this gradual process of us sort of societally, in a sense, learning more concepts. We kind of can exchange concepts, we can build on those concepts and so on. But if you throw us out into some other place in what I call the Ruliat, the space of all possible computational processes, if you throw us out into an arbitrary place there, we will be completely confused because there will be things we can tell there are actual computations going on here. There are things happening. There's even pieces of the reducibility, but we don't relate to those things. So it's kind of like imagine that you were, you know, you're here now and you're, you know, chronically frozen for 500 years and you wake up again and there's all these other things in the world. And it's hard to reorient for all those other things without having seen the intermediate steps. And I think that that when you talk about kind of what where can you go from what we have now? How can you sort of add more? You basically intelligence is all about these kind of pieces of reducibility. These ways to jump ahead and not just say it's what we think of as human-like intelligence is about those kinds of things. And I think the, so what's the vision of what will happen when the world is full of AIs? sort of interesting because actually we've seen it before. I mean, when the world is full of AI's and they're doing all these things and there's all this computational irreducibility, there are all these pockets of irreducibility that we don't have access to because we haven't sort of you know incrementally got to that point. What's going to be happening is there's all this stuff happening among the AI's and it's happening in this layer that we don't understand. It's already happening in plenty of places on the web and you know bidding for ads or showing you content into the web, whatever. There's a layer of AI that's happening that we don't understand particularly. Well, we have a very clear model for that, which is nature. Nature is full of things going on that are often computational irreducible that we don't understand. What we've been able to do is to carve out an existence, so to speak, that is coherent for us. Even though there's all this computational irreducibility going on, we've got these little niches with respect to nature, which are convenient for us as human, so to speak. And I think it's sort of the same thing with the AI world as it becomes like the natural world and it becomes sort of not immediately comprehensible to us. That's we are kind of, our view of it has to be, oh, that's just the operation of nature. That's just something I'm not gonna understand. Oh, that's just the operation of the AI, I'm not gonna understand that. There's this piece that we've actually managed to humanize that we can understand. So that's a little bit of the thought about how that develops. In other words, you know, you can say, I'm gonna throw you out to some random place in the Ruliat. There's incredible computations happening. It's like, great, that's nice. I've spent a bunch of my life studying those kinds of things, but pulling them back, reeling them back into something which has sort of direct human understandability is a difficult thing. Aaron is asking more of a business question about Google and the transformer architecture. And why, you know, it's been a very interesting thing that the sort of neural nets with this Small field, very fragmented for many, many years, and then suddenly things started to work in 2012. And a lot of what worked and what was really worked on was done in a small number of large tech companies, some not so large tech companies. And it's sort of a different picture of where innovation is happening than has existed in other fields. And it's kind of interesting. It's kind of potentially a model for what will happen in other places. But it's always complicated. What causes one group to do this and another group to do that? And they're the entrepreneurial folk who are smaller and more agile. And they're the folks who have more, the more resources and so on. It's always complicated. OK, Nick, I was asking, do you think the pre-training logically inspired language model might be feasible in the future. I don't know. I think that the figuring out how to train something that is, you know, we don't know what parts of the biology are important. One of the one of the incredible important things we just learned is that probably there's not much more to brains that really matters for their information processing than the neurons and their connections and so on. It could have been the case that that every molecule has some quantum process that's going on, and that's where thinking really happens. But it doesn't seem to be the case, 'cause there's this pinnacle of kind of our sort of thinking powers of being able to write long essays and so on. It seems like that can be done with just a bunch of neurons with weights. Now, which other parts of biology are important? Like, you know, actually Teres and Offki just wrote this paper talking about how there are more backwards going neural connections and brains than forwards going ones. So in that sense, it looks like maybe we missed the point with these feed forward networks that something like chatGPT basically is and that the feedback is really important, but we haven't yet really got the right idealized model of that. I do think that the what's the next McCulloch Pets type thing? What's the next sort of simple meta model of this is important? I also think that there's probably a bunch of in-central mathematical structure to learn about general mathematical structure to learn. No, I was interested in neural nets back around 1980 and I kind of was trying to simplify, simplify, simplify models of things. And no, no, I went past them because they weren't simple enough for me. They had all these different weights and all these different network architectures and so on. And I ended up studying cellular automata and generalizations of that where you have something where everything is much simpler. There are no real numbers. There are no arbitrary connections. There are no this, that, and the other things. But what matters and what doesn't, we just don't know that. Yeah. All is asking, what about a five senses multimodal model to actually ground the system in a real world with real human life experience, I think that will be important and that will no doubt happen. And you'll be more human like, look, this chat GPT is pretty human like when it comes to text, because by the way, it just read a large fraction of the text that we humans at least publicly wrote. And, but it didn't know, it hasn't had the experience of walking upstairs and doing this or that thing. And so it's not going to be very human like when it comes to those sorts of things. it has those experiences, then I think we get to, you know, that that will be interesting. Okay. Someone's commenting on the fact that I should do the same kind of description for image generation, uh, generated AI for images. Um, the, uh, the thing that I like to think about there is I think that's one of our first moments of communication with an alien intelligence. In other words, we, in some sense, we're talking to the generative AI and English words or whatever, and it's going into its alien mind, so to speak, and plucking out the scuff that is these images and so on. It's less so, you know, with chatGPT, what the output is something that is already intended to be very human. It's human language. With an image generation system, it's more, it's really, it's producing something that has to be somewhat recognizable to us. It's not a random bunch of pixels. It's something that resonates with things we know, but in a sense, it can be be more completely creative in what it's showing us. In a sense, as one tries to sort of, navigate around its space of what it's going to show us, it feels a lot like kind of your communicating with an alien intelligence. And it's kind of showing you things about how it thinks about things by saying, "Oh, you said those words. I'm going to do this," and so on. I mean, I have to say that I'm if we can't, you know, the other examples of alien intelligence is that we have all around the planet, lots of craters from the cetaceans on, so to speak, that and I have to believe that if we could correlate kind of the experiences of those craters, cats, dogs, you know, cockatooes, whatever else, and the vocalizations that they have and so on, And we could, you know, that it's talked to the animals time, so to speak. I mean, I think that's a, that feels like that, that's, you know, the kinds of things we've learnt from TrackGPT about the structure of human language, I am quite certain that if there's any linguistic structure for other animals, it'll be similar because it's one of the lessons of biology is, you know, there are fewer ideas than you think, the, you know, these things that we have precursors in biology long, long ago. We may have made innovations in language, it's kind of the key innovation of our species, but whatever is there had precursors in other organisms, and that's what, and the fact that we now have this much better way of teasing out a model for language in humans means we should be able to do that elsewhere as well. Okay, David is saying, "Chatchy-PT's developers seem committed to injecting sort of political or cutelments into the code because to avoid it talking about controversial topics, how's that done?" It's done through this reinforcement learning stage. I think maybe there's also some actual, you know, if it's starting to use these words, just stop it type things. I think maybe maybe that's being done a little bit more with maybe with being than it is with with chat GPT at this point. I think that the I have to say the one thing that I consider a you know so far as I know chat GPT is a G rated you know thing and that's achievement in its own right that it doesn't turn maybe I shouldn't say that because probably maybe there are horrible counter examples to that but I think that was a you know in terms of One of the things that happens is well, you have a bunch of humans and they are giving it this training and those humans have opinions and they will have, you know, there'll be this kind of politics or that kind of politics or they'll believe in this or that or the other and they are, you know, whether purposefully or not, they're, you know, they're going to impose those opinions because there is no, you know, the opinion is what you're doing when you tell chat GPT that says good, that says isn't good, you know, it's not a lot an opinion. Now, that opinion may or may not be colored into something that is about politics or something like that, but it's sort of inevitable that you have that. I mean, I have to say, something I've thought about a little bit in connection with general sort of AI injection into sort of the things we see in the world like social media content and so on. I tend to think that the right way to solve this is to say, "Okay, let's have multiple chat bots or whatever," and they are in effect trained with different criteria by different groups under different banners, so to speak, and you know you get to pick the banner of chatbot that you want to be using and then you're happy because you're not seeing things that horrify you and so on. And you can discuss whether you want to pick the chatbot that accepts the most diverse views or whether you want to, you know, that's a, that's, that sort of throws one back into, um, into kind of standard issues of political philosophy and things like this. I mean, I think the thing to realize is that there is a, there's sort of an ethics, you know, one wants to put ethics somehow into what's going on. But one one says, let's have the AIs, you know, do the ethics. It's like, that's hopeless. Ethics is a, there is no sort of mathematically definable perfect ethics. Ethics is a the way humans want things to be. And then you have to choose, you know, well, is it the average ethics? Is it the, you know, the ethics, which makes only 5% of the people unhappy? Is it this and the other? These are old questions of political philosophy that don't really have so far as we know good answers. And but once thrown into those questions, there's no, you know, oh, we'll get a machine to do it and it'll be perfect. It won't happen because these are questions that aren't solvable for a machine because they're questions that in a sense come right from us. These are-- I mean, the thing to realize about chat GPT in general, chat GPT is a mirror on us. It's taken what we wrote on the web, so to speak, in an aggregate, and it's reflecting that back to us. So insofar as it does goofy things and it says goofy things, some-- that's really on us. I mean, that's, you know, it's our sort of, it's the average, kind of, the, the, the, the, sort of the average web that we're seeing here. So, Nesha is asking about a particular paper, which I sound interesting, but I don't know about it. Let's see. Up up soon here. Okay, Tragath is wondering how neural net AI compares to other living multicellular intelligence, plant roots, nerve nets and things like jellyfish and so on, biofilms, yeah, well, okay. So one of the big things that's come out of a bunch of science that I've done is this thing I call the principle of computational equivalence, which essentially says that as soon as you have a system that is not computationally trivial, it will ultimately be equivalent in its computational capabilities. And that's an important thing when you talk about computational irreducibility. So, because computational irreducibility arises, because you've got a system doing its computation, there's no system. You can't expect there to be, all other systems will just be equivalent in their computational sophistication. You can't expect a super system that's going to jump ahead and just say, "Oh, you went through all these computational steps, but I can jump ahead and just get to the answer." Now a question that is a really good question is when we look at, okay, one of the things that is characteristic of our consciousness, for example, relative to all the computational and the ability in the universe is the fact that we have coherent consciousness is a consequence of the fact that we are two things that seems to me. We are computationally bounded, we're not capable of looking at all those molecules bouncing around. We only see various aggregate effects. and point two that we are, we believe that we are persistent in time. We believe we have a persistent thread of existence through time. Turns out, big fact of our last few years for me is that the big facts of physics, general relativity, theory of gravity, quantum mechanics, and statistical mechanics, the second-row of thermodynamics, the law of entropy increase, All three of those big theories of physics that arose in the 20th century, all three of those can be derived from knowing that we human observers are noticing those laws and we human observers have those two characteristics I just mentioned. I consider this a very important, beautiful sort of profound result about kind of the fact that we observe the physics we observe because we are observers of the kind that we are. Now, interesting question, I suppose, is when we are limited, we are computational-limited things. And the very fact that we observe physics, the way we observe physics, is a consequence of those computational limitations. So a question is, how similar are the computational limitations in these other kinds of systems? In a sense, the fungus as observer, so to speak. How similar is that kind of observer to a human observer? and in terms of what computational capabilities it has and so on. My guess is it's pretty similar. And in fact, one of my next projects is a thing I'm calling observer theory, which is kind of a general theory of kinds of observers that you can have of things. And so maybe we'll learn something from that. But it's a very interesting question. Dugan is commenting. ChatGPT can be improved using an automated fact checking system like an adversarial network, for instance. Could one basically, could one train ChatGPT with Wolfmalfa and have it get better? The answer is surely up to a point, but then it will lose it, just like it does with parentheses. I mean, there's a certain, with a network of that architecture, There's a certain set of things one can learn, but one cannot learn what is computationally irreducible. I mean, in other words, you can learn the common cases, but there'll always be surprises. There'll always be unexpected things that you can only get to by just explicitly doing those computations. Barbers asking, "Can chat GPT play a text-based adventure game? "I bet it can. "I don't know. "I haven't seen anybody try that, but I bet it can." Um, I guess a question here from software. Aside from being trained on a huge corpus, what is it about GPT three that makes it so good at language? I think I tried to talk about that a bit about the fact that we, it's, it's some that there's, you know, there's regularity in language. I think the, the particulars of the transformer architecture of this kind of looking back on sequences and things. That's been helpful in refining the way that you can train it. the way that you can train it. And that seems to be important. Let's see. Atorias asking, could feature impact scores help us understand GPT better? Well, so what that's about is when you run a neural net, you can say how much did some particular feature affect the output that the neural in that gave? Chapaget GBG is just a really pretty complicated thing. I started digging around trying to understand what as a natural scientist. I couldn't do sort of neuroscience with actual brains because I'm 100 times, 1000 times to squeamish for that. But I can dig around inside an artificial brain, and I started trying to do that, and it's difficult. I mean, I didn't look at feature impact scores, I think one could. The... Okay. So by the way, I'm amused by these questions because I can still tell you guys are not bots, I think. And let's see. Ron is asking about implications. I have to work late tonight. What does that mean? Absolutely, chat GPT is learning stuff like that because it's seen a bunch of texts that says, "I have to work late tonight so I can't do this." It's seen examples of that. It's kind of doing the Aristotle again. It's just seeing these patterns of language and that's what it's learning from, so to speak. So yes, these things we might say, "How do we think about that formally?" it seems kind of complicated to us, but that pattern of language has occurred before. All right, last thing, perhaps. Okay, Albert is asking, do you think humans learn efficiently because they're born with the right networks to learn language more easily or is there some difference? I think it is important, the architecture of the brain undoubtedly is important. I mean, my impression is that there are, you know, it's a matter for the neuroscientists to go and find out. Now that we know that certain things can be made to work with artificial neural nets, did the actual brain discover those things too? And the answer will be often yes. I mean, just like there are things we probably have learnt from, you know, the flight of drones or the flight of planes that we can go back and say, "Oh, did we, did biology actually already have that idea?" I think that the, they're undoubtedly features of human language, which depend on aspects of the brain. I mean, like for example, one, you know, talking to Teriets Anoski, you know, we're talking about the the loop between the basal ganglia and the cortex and the possibility that, you know, the outer loop of chat GPT is a little bit like that loop. And it's kind of like, I'm turning things over in my mind, one might say, maybe that's actually a loop of data going around this literal loop from one part of the brain to another, maybe, maybe not. But sometimes those sayings have a habit of being more true than you think. And maybe the reason that when we think about things, we have these certain time frames, when you think about things, the certain times between when words come out and so on, maybe those times are literally associated with the amount of time it takes for signals to propagate through some number of layers in our our brains. And I think that in that sense, if that's the case, there will be features of language, which are, yes, we've got this brain architecture, we're going to have these features of language. And insofar as language evolves, as so far as it's adaptively worthwhile to have a different form of language that is optimized by having some different form of brain structure, that's what will have been driven by, by natural selection and so on. I mean, I think there are aspects of language like we know if you, you know, we tend to remember five chunks, you know, chunks of five, so to speak, things at a time. And we know that if we try and give a sentence which has more and more and more deeper, deeper sub clauses, we lose it off to some point. And that's presumably a hardware limitation of our brains. Okay. Dave is asking, this is a good last question. How difficult will it be for individuals to train something like a personal chat GPT that learns to behave more and more like a clone of the user. I think I don't know. I'm gonna try it. I have a lot of training data, as I mentioned, you know, 50 million typed words, yeah, typed words, for example, for me. And my guess is, I mean, I know somebody tried to train an earlier GPT-3 on stuff of mine. I didn't think it was terribly good. When I read ones trained for other people, I thought they were pretty decent. When I looked at one trained for myself, 'cause I kind of know myself better than I know anybody else, I think. The, you know, it didn't ring true, so to speak. And, but I do think that that will be a, you know, being able to write emails like I write emails, it'll do a decent job of that, I suspect. You know, I would like to believe that, you know, one still as a human one still has an edge because in a sense one knows what the goals are. The, you know, no, this system, its goal is to complete English text. And, you know, the bigger picture of what's going on is not going to be part of what it has, except in so far as it lends the aggregate bigger picture from just reading lots of text. So, you know, but I do think it'll be an interesting. I expect that as a person who gets a lot of email, some of which is fairly easy to answer in principle that maybe my bot will be able to answer the easier stuff for me. All right, that's probably a good place to wrap this up. Thanks for joining me and I would like to say that for those interested in more technical details, Some of the folks in our machine learning group are going to be doing some more detailed technical webinars about this material and really going into how you build these things from scratch and so on and what some of the more detail about what's happening actually is. But I should wrap up here for now and thanks for joining me and bye for now.

0 comments

The Perils and Regulations of Generative AI: Protecting Interests and Mitigating Risks

The Future of AI Regulation: Ensuring Safety and Inclusivity

The Impact of AI Technology on Privacy, Intellectual Property, and Misinformation

The Impact of Large Language Models on Jobs and Public Opinion

The Oversight of Artificial Intelligence: Ensuring Responsible Innovation

See all posts from LabSanta 🟡