Recently, a groundbreaking study was conducted to analyze the sources of information ChatGPT (GPT) relies on when composing scientific literature reviews. By focusing on the field of environmental science, the authors of this study sought to answer the question of whether GPT predominantly cites the most-cited publications, or if it selects information randomly.
The results of the said study showed that GPT tended to cite highly-cited publications in environmental science, with a median count of 1184.5 citations. Additionally, the study revealed that GPT had a preference for older publications, with a median publication year of 2010, while it mainly sourced information from esteemed journals within the field, with Nature being the most cited.
The authors of the study also identified an interesting observation, which is that GPT seems to exclusively draw on Google Scholar for citation information, rather than using other sources, such as Web of Science or Scopus. It is also worth noting that when it comes to literature research and reviews, Google Scholar is seen as the most prevalent information source.
Given the widespread use of GPT for literature reviews, we can assume that the citation dominance observed in this study will play an even larger role in future studies on the topic. In other words, this study provides evidence for the idea of the Matthew Effect in science, where the highly-cited publications tend to have the upper hand.
It is also worth noting that ChatGPT only began two years ago in 2018; however, it has already gained over 100 million users worldwide due to its innovative approach to helping people find information. With its advanced NLP capabilities, ChatGPT continues to help people uncover scientific literature faster than ever before.
