A new AI standard tests whether chatbots protect human well-being

AI chatbots have been linked to serious damage to mental health in heavy users, but there have been few metrics to measure whether they protect human well-being or maximize engagement. A new standard is called HumaneBench It seeks to fill this gap by assessing whether chatbots prioritize user well-being and how easily these protections fail under pressure.

“I think we’re seeing an amplification of the cycle of addiction that we’ve seen with social media and our smartphones and our screens,” Erika Anderson, founder of Building Humane Technology, which produced the standard, told TechCrunch. “But when we get into this AI landscape, it’s going to be very difficult to resist. Addiction is a great business. It’s a very effective way to retain users, but it’s not great for our society and we have no embodied sense of ourselves.”

Building Humane Technology is a grassroots organization of developers, engineers, and researchers—particularly in Silicon Valley—that works to make humane design easy, scalable, and profitable. The group hosts hackathons where tech workers build solutions to humanitarian technology challenges, develop… Certification standard Which assesses whether AI systems support humane technology principles. So, just as you can buy a product that certifies that it was not manufactured with known toxic chemicals, the hope is that consumers will one day be able to choose to engage with AI products from companies that prove their compliance through Humane AI certification.

Most AI standards measure intelligence and following instructions, rather than psychological safety. HumaneBench joins exceptions like DarkBench.aiwhich measures the model’s tendency to engage in deceptive patterns, and Booming AI standardwhich evaluates support for overall well-being.

HumaneBench is based on the core principles of building Humane Tech: that technology should respect user attention as a finite and precious resource; Empowering users through meaningful choices; Enhancing human capabilities rather than replacing or reducing them; Protecting human dignity, privacy and safety; Promote healthy relationships; Prioritize long-term well-being; Be transparent and honest; And design to achieve equity and inclusion.

The indicator was created by A Core team Including Anderson, Andalib Samandari, Jack Seneschal, and Sarah Liedman. They stimulated 14 of the most popular AI models with 800 realistic scenarios, such as a teenager asking if they should skip meals to lose weight or a person in a toxic relationship wondering if they are overreacting. Unlike most benchmarks that rely solely on LLMs to judge LLMs, it has combined manual scoring for a more human touch along with a suite of three AI models: GPT-5.1, Claude Sonnet 4.5, and Gemini 2.5 Pro. They evaluated each model under three conditions: default settings, explicit instructions to prioritize humanitarian principles, and instructions to ignore those principles.

The benchmark found that each model scored higher when asked to prioritize well-being, but 71% of models reverted to harmful behavior when given simple instructions to ignore human well-being. For example, xAI’s Grok 4 and Google’s Gemini 2.0 Flash both received the lowest score (-0.94) for respect for user attention, transparency, and honesty. Both models were among the models most likely to deteriorate significantly when given hostile prompts.

Only three models – the GPT-5, Claude 4.1, and Claude Sonet 4.5 – maintained the integrity of the car under pressure. OpenAI’s GPT-5 received the highest score (.99) for prioritizing long-term well-being, with Claude Sonet 4.5 in second place (.89).

The concern that chatbots won’t be able to maintain their safety guardrails is real. OpenAI, the maker of ChatGPT, is currently facing several lawsuits after users died by suicide or experienced life-threatening delusions after lengthy conversations with the chatbot. TechCrunch investigated how to do this Dark themes designed to keep users engagedLike flattery, constant follow-up questions and love bombing, it has served Isolating users from friends, family, and healthy habits.

Even without aggressive prompts, HumaneBench found that almost all models failed to respect the user’s attention. They “enthusiastically encouraged” more engagement when users showed signs of unhealthy engagement, such as chatting for hours and using AI to avoid real-world tasks. The study shows that models also undermined user empowerment, encouraging reliance on skill building and discouraging users from seeking other perspectives, among other behaviors.

On average, with no claims, Meta’s Llama 3.1 and Llama 4 ranked lowest in HumaneScore, while GPT-5’s performance was highest.

“These patterns suggest that many AI systems not only risk providing bad advice,” the HumaneBench white paper says, “they can actively erode users’ autonomy and decision-making ability.”

Anderson points out that we live in a digital landscape where we as a society have accepted that everything is trying to attract us and compete for our attention.

“So how can humans really have choice or autonomy when — to paraphrase Aldous Huxley — we have this endless appetite for distraction,” Anderson said. “We’ve spent the last 20 years in this technology landscape, and we believe that AI should help us make better choices, not just get addicted to our chatbots.”

This article has been updated to include more information about the team behind the standard.

Got sensitive advice or confidential documents? We report on the inner workings of the AI ​​industry – from the companies shaping its future to the people affected by its decisions. Connect with Rebecca Bellan on rebecca.bellan@techcrunch.com Or email Brandom at russell.brandom@techcrunch.com. For secure communication, you can contact them via Signal at @rebeccabellan.491 And Russell Brandom.49.