Much ink has been spilled of late about the tremendous promise of OpenAI’s ChatGPT program for generating natural-language utterances in response to human prompts.
The program strikes many people as so fresh and intriguing that ChatGPT must be unique in the universe.
Scholars of AI beg to differ.
“In terms of underlying techniques, ChatGPT is not particularly innovative,” said Yann LeCun, Meta’s chief AI scientist, in a small gathering of press and executives on Zoom last week.
“It’s nothing revolutionary, although that’s the way it’s perceived in the public,” said LeCun. “It’s just that, you know, it’s well put together, it’s nicely done.”
Such data-driven AI systems have been built in the past by many companies and research labs, said LeCun. The idea of OpenAI being alone in its type of work is inaccurate, he said.
“OpenAI is not particularly an advance compared to the other labs, at all,” said LeCun.
“It’s not only just Google and Meta, but there are half a dozen startups that basically have very similar technology to it,” added LeCun. “I don’t want to say it’s not rocket science, but it’s rarely shared, there’s no secret behind it, if you will.”
LeCun noted the many ways in which ChatGPT, and the program upon which it builds, OpenAI’s GPT-3, is composed of multiple pieces of technology developed over many years by many parties.
“You have to realize, ChatGPT uses Transformer architectures that are pre-trained in this self-supervised manner,” observed LeCun. “Self-supervised-learning is something I’ve been advocating for a long time, even before OpenAI existed,” he said.
January 23, 2023
Written by Tiernan Ray