ChatGPT, an artificial intelligence chatbot, can match the top 1% of human thinkers, according to a new study by the University of Montana.
Researchers administered the Torrance Tests of Creative Thinking, an oft-used creativity test, to the ChatGPT engine and recorded eight responses. They also collected responses from 24 University of Montana students. Those scores were compared with 2,700 students nationally who took the TTCT.
All responses were scored by the Scholastic Testing Service, which wasn't aware that AI responses were submitted.
The AI responses were as creative as the responses from the most creative real people who took the test, according to researchers. In fact, ChatGPT outperformed a majority of students nationally.
The findings were surprising, study author and assistant clinical professor at University of Montana's College of Business Erik Guzik told ScienceDaily.com. Guzik's areas of research include entrepreneurship and the economic aspect of creativity.
"For me, creativity is about doing things differently," Guzik said. "One of the definitions of entrepreneurship I love is that to be an entrepreneur is to think differently. So, AI may help us apply the world of creative thinking to business and the process of innovation, and that's just fascinating to me."
'Some of the responses were novel and surprising'
The TTCT contains two different assessments: one verbal and one figural. Both measure divergent thinking, or the thought process used to generate creative ideas.
In the verbal assessment, test-takers are provided with pictures and/or verbal prompts and asked to respond in written form. For example, they might be shown a picture of an event and asked to hypothesize the outcome. Or they might be shown a product and asked for ways to improve it. Their answers are used to assess three mental characteristics:
- Fluency: the number of relevant ideas
- Originality: the unusualness of ideas
- Flexibility: the variety of different types of ideas
The figural assessment requires the test-taker to draw their answers. For example, they might be asked to complete a picture. Along with fluency and originality the figural test evaluates the following skills:
- Elaboration: the addition of ideas beyond the minimum requirement
- Resistance to premature closure: the ability for a person to tolerate ambiguity enough to come up with a creative response
- Abstractness of titles: the ability to title a figure creatively
In the study, ChatGPT responses were in the top percentile for fluency and originality. Response scores slipped into the 97th percentile when it came to flexibility.
"We had all been exploring with ChatGPT, and we noticed it had been doing some interesting things that we didn't expect," Guzik said. "Some of the responses were novel and surprising. That's when we decided to put it to the test to see how creative it really is."
Guzik does not want to overstate the impact that ChatGPT might have on the economy, but he does suspect it will become a driver of innovation.
DON'T MISS: Want to be smarter and more successful with your money, work & life? Sign up for our new newsletter!
Get CNBC's free Warren Buffett Guide to Investing, which distills the billionaire's No. 1 best piece of advice for regular investors, do's and don'ts, and three key investing principles into a clear and simple guidebook.