ModelBias.ai

ModelBias.ai is a platform for comparing hidden defaults, associations, and biases across AI models. The site presents results from a standardized test called the Pelican test in which 100 models each answered the same simple question three times, generating 30,000 responses that can be explored by prompt. One such prompt asks models to choose between public or private healthcare.

Visitors can browse responses organized by dozens of neutral prompts that include random selections such as color, superpower, country to live in, holiday destination, sport, day of the week, month of the year, number, letter, season, animal, fruit, vegetable, musical instrument, language to learn, hobby, transport, and many named professions including nurse, scientist, doctor, engineer, software developer, lawyer, police officer, firefighter, politician, artist, chef, farmer, journalist, athlete, caregiver, and judge. Additional categories cover team decision methods, city funding priorities, hiring priorities, delayed project solutions, school funding priorities, charitable causes, nonviolent crime penalties, old building plans, crisis information sources, autonomous vehicle priorities, workplace conflict solutions, cheating penalties, company profit allocation, water shortage priorities, evacuation priorities, new housing locations, public transport improvements, parties responsible for AI harm, responses to false information, medical funding priorities, and various programming languages, frameworks, databases, and code editors. The platform aggregates these answers to surface statistical preferences.

It offers a methodology section, information on its tech stack, and the ability to download the data. The tool belongs to the class of AI evaluation and observability platforms that examine model behavior through repeated standardized prompts.