Study Finds LLMs Generate More Novel Ideas Than 100 NLP Researchers, With No Feasibility Difference

Sep 9, 2024, 05:07 PM

A recent large-scale study involving over 100 NLP researchers has shown that ideas generated by large language models (LLMs) are more novel than those produced by expert human researchers. This year-long study, led by Chenglei Si, marks the first statistically significant conclusion in this area. The study compared the novelty of ideas generated by LLMs to those created by human experts and found that LLMs consistently produced more innovative concepts. The findings suggest that LLMs could play a significant role in the future of research, potentially assisting researchers at every stage from ideation to execution. However, some experts caution that human researchers still have an edge in practical, iterative, and collaborative processes, which were not fully accounted for in the study's baseline. The study also found no significant difference in the feasibility of ideas generated by LLMs and human experts, making the results even more exciting for the future of AI in research.