Revolutionizing Research: New Chatbot Surpasses PhDs in Literature Reviews
A groundbreaking study in the field of artificial intelligence has revealed a remarkable development in the capabilities of chatbots. Researchers have developed a new chatbot, OpenScholar, which can outperform PhD students and postdocs in conducting scientific literature reviews. The study, published in Nature, highlights the chatbot's ability to produce reliable summaries at a fraction of the cost, making it a game-changer for scholars and researchers.
The research team, comprising experts in computer science, physics, neuroscience, and biomedicine, evaluated OpenScholar and its spin-off version, ScholarQABench, against summaries written by PhD students. The results were astonishing. In over 50% of the cases, OpenScholar's summaries were preferred by domain-level experts, who are themselves PhDs and postdocs. This preference is attributed to the chatbot's ability to provide a more comprehensive and detailed overview of the literature.
OpenScholar's summaries were twice to three times longer than those written by PhD students, offering a greater breadth and depth of information. In contrast, ChatGPT-written summaries were preferred in only 31% of cases, as they struggled to cover all the essential information. The study, titled 'Synthesizing scientific literature with retrieval-augmented language models', also addressed the issue of 'hallucinations' in LLMs, where they produce false citations. OpenScholar, however, did not exhibit this problem, ensuring the accuracy of its citations.
The key to OpenScholar's success lies in its training data. Unlike other LLMs trained on the entire internet, OpenScholar's 8B model is based on a vast corpus of 45 million scientific papers. This unique training enables the chatbot to create a 'self-feedback loop', improving its factuality, coverage, and citation accuracy. Since its demo, OpenScholar has been used by over 30,000 people, generating nearly 90,000 user inquiries.
The cost-effectiveness of OpenScholar is another significant advantage. The study estimates that its literature reviews cost between 1 cent and 5 cents, allowing scholars to conduct thousands of searches every month. The authors of the study believe that OpenScholar has the potential to revolutionize research, supporting and accelerating future efforts.
Despite acknowledging the system's limitations, the authors have made both ScholarQABench and OpenScholar available to the research community. This move encourages ongoing research and refinement, inviting scholars to explore and contribute to the development of these advanced chatbots.
