Advertisement
  1. SEJ
  2.  ⋅ 
  3. Tools

OpenAI’s ChatGPT Update Brings Improved Accuracy

OpenAI's ChatGPT gets an update that includes more accurate responses and a new stop button.

  • OpenAI released an update to ChatGPT, which aims to improve accuracy and factuality.
  • ChatGPT's knowledge is still limited to 2021 data, which means it can't answer current questions.
  • We test the claim of increased accuracy, and find ChatGPT still doesn't get everything right.

OpenAI, the company behind the popular conversational AI model, ChatGPT, has released an update to improve the chatbot’s accuracy.

Following an extended period of downtime on Tuesday, ChatGPT is up and running with a new model.

This is the first update to ChatGPT this year and the second update to the model since its launch in November.

A popup message lists the changes in what OpenAI calls the “Jan 9 version” update.

Screenshot from: chat.openai.com/chat, January 2023.

The popup reads:

“We made more improvements to the ChatGPT model! It should be generally better across a wide range of topics and has improved factuality.”

In addition, you can now stop ChatGPT while it’s in the process of generating a response, which is a heavily requested feature.

Details in the release notes are limited, so I tested several prompts in ChatGPT to learn more about the new update. Here’s what I discovered.

Data Still Limited To 2021

While ChatGPT may be more accurate now, its knowledge is still limited to 2021 data.

When asked what’s in the new ChatGPT update, the AI tells me:

“ChatGPT is a large language model developed by OpenAI that is trained on a dataset of conversational text. Since it’s an OpenAI model, It’s updated regularly with new features and improvements. However, as my knowledge cut off is in 2021 I don’t have the latest updates and I cannot provide you a specific information on which new features or improvements have been added to the model in recent updates.”

When asked another way, I was able to get more insight into what’s involved in improving the accuracy of an AI model:

“In general, when an update is made to an AI model like ChatGPT, it can be aimed to improve its performance in one or multiple aspects. For example, it can have a bigger dataset to train on, or improvements in its architecture that allows it to generate more fluent and accurate responses. And such update also make it better in factuality, i.e its ability to understand and generate responses based on factual information.”

Accuracy Remains Questionable

OpenAI claims ChatGPT can now provide more factual answers.

I tested that claim by consulting a GitHub repository of ChatGPT failures and running several prompts to see if it would produce different answers.

Related: ChatGPT Examples: 5 Ways SEOs and Digital Marketers Can Use ChatGPT

Test One: Failed

Previously, ChatGPT could not accurately identify how many times Argentina won the FIFA World Cup.

Disregarding the 2022 World Cup win, because ChatGPT’s knowledge is limited to 2021, it should say Argentina has won it two times. Once in 1978 and again in 1986.

As shown in the tweet below, ChatGPT didn’t always return the right answer:

I ran the prompt through the updated version of ChatGPT, and it returned a different but still incorrect answer.

Screenshot from: chat.openai.com/chat, January 2023.

Test Two: Failed

Previously, ChatGPT was unable to provide a correct answer when asked who is the taller basketball player between Shaq and Yao Ming.

I ran the prompt through the updated version of ChatGPT, and it confidently returned the same incorrect answer.

Screenshot from: chat.openai.com/chat, January 2023.

Going through the ChatGPT failures linked above, I found it continues to struggle with the same prompts.

It’s difficult to pinpoint the areas in which ChatGPT can return more accurate responses. It would be helpful if OpenAI could provide specific details in the release notes of future updates.

That said, be careful when using ChatGPT as a source of information. Although it provides correct answers to many questions, it’s currently not dependable enough to replace Google.


Source: OpenAI

Featured Image: CHUAN CHUAN/Shutterstock

Category News Tools
ADVERTISEMENT
SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal

Matt G. Southern, Senior News Writer, has been with Search Engine Journal since 2013. With a bachelor’s degree in communications, ...

OpenAI’s ChatGPT Update Brings Improved Accuracy

Subscribe To Our Newsletter.

Conquer your day with daily search marketing news.