ChatGPT Can Do More, and More Reliable

Whether it’s negotiating parking tickets, making workout plans, or creating bedtime stories for children, chatbots have become an essential part of many of our lives. But what if we could get them to do more, and more reliably? Enter ChatGPT, the free bot that’s been taking the internet by storm since its release in November 2022. It’s the latest version of OpenAI’s GPT model, and it can take text and image prompts and produce text, video, and images in return—all with unprecedented speed and accuracy.

The key to its success, however, lies in the GPT model’s training. Using a process called reinforcement learning with human feedback (RLHF), the team behind ChatGPT trained it to understand and respond to prompts that would lead to the most productive outputs—in other words, the ones most likely to help users accomplish their goals.

This humongous dataset allowed it to learn patterns and relationships in text data, which, in turn, led to the ability to predict what type of text should come next in a given conversation. From there, it was able to optimize the network to be dialogue-ready, essentially teaching itself how to construct and convey messages that are safe, sensible, and coherent. It’s a complex process, but the end result has been astonishingly successful: since its launch, it has inspired users to imagine a host of new use cases for ChatGPT, including using it to negotiate a parking ticket or even write a bedtime story.

To do these things, ChatGPT update uses a fairly simple neural net. It starts with a “prompt”, breaks it down into tokens, and then runs a transformer-based neural network to try to understand what’s most important about it. It then uses this understanding to generate an appropriate sequence of tokens to continue a conversation—all based on the training it’s received from its massive sample of real-life text from the web, books, etc.

It’s worth noting, though, that the actual neural net itself is still pretty simple—it consists of a few hundred billion weights that are roughly comparable in size to the total number of words in the entire training set. And for each of these tokens, there are still a few thousand calculations going on—it’s not the kind of thing that brains do, so it’s hard to claim that ChatGPT has truly emulated human behavior.

The fact that it’s so remarkably effective at language-related tasks is probably what leads most people to think of it as “brain-like”. But there are plenty of other things that humans do well—especially those involving what amount to irreducible computations—that ChatGPT simply cannot do.

That said, there is something about the way that ChatGPT operates that does seem somewhat akin to the way that humans work, so perhaps we should think of it as something closer to a bullshit machine, rather than a robot that happens to be capable of producing some particularly interesting and useful types of output. And, if that’s the case, it also means that maybe there’s an insight to be gained by examining how the model actually works.