STaRK: New LLM Benchmark for Semi-Structured Data Released by Amazon, Researchers

Apr 29, 2024, 04:11 PM

In a collaborative effort with Amazon, researchers have developed and released STaRK, a new large-scale Language Model (LLM) retrieval benchmark designed to address the challenges of semi-structured knowledge bases. STaRK aims to reduce LLM hallucinations by utilizing Retrieval-Augmented Generation (RAG) over both textual and structured knowledge bases, enhancing the model's ability to handle complex real-world data. The benchmark includes datasets from diverse fields such as e-commerce, biomedicine, and academic research, facilitating advancements in semantic retrieval and reasoning while tackling more intricate user queries.