Loading...
Market
Will AgentHarm dataset receive a major update by June 30, 2025?
2
Resolution / Starting Odds
Yes • 50%
No • 50%
Official announcements from the AI Safety Institute or GraySwanAI
Story
AI Safety Institute Releases AgentHarm to Measure LLM Agent Harmfulness on October 14, 2024
Oct 15, 2024, 02:22 PM
The AI Safety Institute, in collaboration with GraySwanAI, has announced the release of AgentHarm, a novel dataset designed to measure the harmfulness of large language model (LLM) agents. This benchmark focuses on unique harms from AI agents with access to external tools, addressing a critical gap in current safety evaluations. Announced on October 14, 2024, AgentHarm is comprehensive, reliable, and easy to run, allowing for widespread use. The initiative highlights the need for robust safety mechanisms as LLM agents become more integrated with external systems. Jailbreaking transfers to LLM agents without degrading capabilities, and the dataset is partly public.
View original story
Similar markets
Yes • 50%
No • 50%
Chatbot Safety Evaluation • 25%
Tool-using Agent Safety • 25%
Jailbreaking Resistance Testing • 25%
Other • 25%
NeurIPS 2024 • 25%
ICML 2025 • 25%
AAAI 2025 • 25%
Other • 25%
Further data cleaning measures • 25%
Introduction of new ethical guidelines • 25%
Partnership with another organization • 25%
Other • 25%
Yes • 50%
No • 50%
Yes • 50%
No • 50%
