OpenAI is a world-leading artificial intelligence research laboratory that was founded in 2013 and has played a major role in the development of AI technologies, including the language AI model ChatGPT. It was the first of its kind and was developed by OpenAI’s team of brilliant scientists and engineers. From generative models to robotics, the applications of the technology are truly impressive and the public’s excitement in its capabilities is highly noteworthy.
The person mentioned in this article is I, who tested ChatGPT-4 on a Wordle game – the word game from the New York Times – by playing six rounds with the AI. The game is designed to identify a five-letter words by showing which letters are in the correct positions in the word.
Although the AI chatbot had been trained on 500 billion words, including all of Wikipedia, all public-domain books, huge volumes of scientific articles, and text from many websites, its capabilities to solve Wordle puzzles were unfortunately very low. ChatGPT-4 failed to find valid solutions oftentimes and gave out responses without any logical reason.
The underlying reason for such a poor performance is the need for AI chatbots to translate every input given and output generated into numbers, a process which is handled by a tokenizer. This process does not allow for the AI to recognize the letters and sequences of words, or reason about the letters’ positions, making it unable to produce any valid solutions for the puzzles.
ChatGPT-4 is quite good at reasoning pertaining to the first letters of words, which could be attributed to the fact that its training data includes indices from textbooks. Even with this strength, however, its capability to identify palindromes and work with the last letters of words is very low.
A more general and exciting solution is to produce an LLM that is able to generate code in order to solve word puzzles. This is an area where the grand potential of LLMs is yet to be explored. Using the example in the article, the AI chatbot was able to produce a program that calculated the valid words fitting a pattern after being pointed out an existing error in its code initially.
Therefore, we have understood that AI chatbots have a long way to go before they can provide a great service in language-related tasks. Although at its core, ChatGPT is a complex mathematical model and its performance on Wordle puzzles is limited, accompanied by the grand potential of this technology, its high performance in certain areas like writing poems is a sign of its capability to affect our lives for the better.
