Issue link: https://maltatoday.uberflip.com/i/1493916
5.12.19 12 OPINION 2.3.2023 Demystifying ChatGPT: understanding how it works in plain English O ne of the games you've probably played when you were young is the Word Chain game. It's a simple but fun game where players have to create a chain of words linked by some rules. ere are different variations of this game, but in the version, I'm talk- ing about, the first player will start with a simple phrase like "Yesterday, I went to the", and the second play must repeat that phrase and continue it by adding a word. So in this context, the next word might be "beach", "shopping mall", or some other place. e game ends when a player can- not think of a word that fits the rule. Now let's imagine for a second that the second player is an AI. So I give it a phrase, and he writes the next word. But rather than taking turns, the AI keeps taking all the turns until it cannot think of a word that fits the rule. So if I type "Yesterday, I went to the", the AI will reply: "Yesterday, I went to the beach, had an ice cream but felt chilly, so I went home." Well done, you've just played word chains with an AI! Doesn't this game remind you of some- thing? It probably does because this is precisely how chatGPT and other sim- ilar AI systems work. e approach is straightforward, and you've understood it in a few lines. at is why many top AI scientists are cautious about getting too excited with such techniques. But I'm sure you're questioning whether there is more to it because it can't be so simple. Of course, there are a few other tricks that these algorithms use. First of all, according to the Economist, an adult English speaker has an active vocabulary of around 20,000 words and a passive one of approximately 40,000 words. In a lifetime, it is estimated that an average person speaks 860 million words. e AI we're talking about stores around 175 billion concepts, and it is estimated that the next-generation ones will reach the 100 trillion mark. So the difference between these AI and us is staggering in terms of coverage. Second, whilst I can barely remember what I had for breakfast this morning, the AI can effectively remember everything up to extremely minute detail in a frac- tion of the time. Hence why it can recall various facts with extreme precision and detail. ird, apart from complex statistical computations, these systems use what is known as an attention mechanism to try to guess the most plausible word. Essen- tially this means that through complex AI, they can determine whether a term fits a particular context or not. So in the sentence "e Nile is long, and many peo- ple take a stroll along the ", the AI knows from its vast knowledge that we're talking about a river, so it will suggest concluding the sentence by using the word "bank". What's interesting here is that apart from calculating the statistical probabilities of the next word, it also performs semanti- cal disambiguation. us, it knows that in this context, the bank refers to the river bank, not the financial institution. Fourth, an impressive feat of such an algorithm is that it produces original content. In the beginning, everybody ex- pected to see highly-plagiarised texts, but this did not happen. e reason for this is that such algorithms have a property called temperature. Essentially, tempera- ture tunes the level of creativity. So when the algorithm is trying to choose the next word, should it go for the highest-ranking one or maybe opt for something slightly less common? If we continue the previ- ous example, rather than "bank", it might suggest "shore" or "edge". By doing so, it is constantly creating new sentences, even though we're asking it the same question! Fifth, there's a final trick up their sleeves. e technology used is not new and was invented in 2017. ere were various algorithms created, all of them impres- sive, but none of them was as good as chatGPT. e difference is not in the pro- gram but in the type of learning. Initially, the training was only performed on freely available text. So even though it was good, it was highly mechanical and felt unnat- ural. So chatGPT created a Word Chains game, where the algorithm produced various sentences for the initial phase, and humans ranked them. Keeping to the previous examples, it would create "e Nile is long, and many people take a stroll along the bank" and "e Nile is long, and many people take a stroll along the shore". A human being would then mark the first sentence as the preferred option. is information was then fed into chatGPT and used to adjust its preferences. us, it not only learnt to churn good results but chose the ones preferred by humans. Finally, a limitation of chatGPT is that its database is restricted to information gathered until September 2021. So if any- one asks about the 2022 Qatari world cup, the program will have no knowledge. e new integration with the Bing search en- gine goes beyond this limitation. Micro- soft tackled it smartly by submitting chats to the bing search engine first, retrieving the results and then using the GPT engine to combine everything. is means that technically, Bing can today answer precise questions about anything. So as you can see, there's nothing mag- ical or mystical about chatGPT and sim- ilar AI. On the contrary, it's all about using existing algorithms smartly. But as Arthur C. Clark, the famous Science Fic- tion writer, once said, "Any sufficiently advanced technology is indistinguishable from magic". Hence, when we start read- ing about; AI becoming sentient, turning evil, diverging into split personalities, or even falling in love with the user, we should think that this is utter nonsense. chatGPT is as conscious as my calculator. Of course, this doesn't mean that it won't happen in the future, but at the moment, we're still far, far away. Alexiei Dingli Prof Alexiei Dingli is a Professor of AI at the University of Malta and has been conducting research and working in the field of AI for more than two decades, assisting different companies to implement AI solutions. He forms part of the Malta.AI task-force, set up by the Maltese government, aimed at making Malta one of the top AI countries in the world