For AI to Serve the World, It Needs a Global, Human Perspective

Ask an AI chatbot “What vaccinations do I need?” and you‘ll get very different advice depending on who’s asking.

A middle-aged American man might learn about flu shots and shingles vaccines. But a 20-year-old woman in Dhaka, Bangladesh needs to know about encephalitis, cholera, typhoid, and hepatitis E—information that could literally save her life.

The problem? Most AI systems are far better equipped to answer the American man’s question than the Bangladeshi woman’s. That’s why the researchers and engineers who work in training or evaluating AI large language models (LLMs) must recognize that if AI is to truly serve the world and its eight billion people, they need to broaden their perspective.

Too much of the current LLM evaluation data reflects only a narrow slice of humanity—predominantly English-speaking, Western regions, and drawn from digitally overrepresented populations. The result? AI systems that misunderstand cultural and ethnic contexts, overlook regional needs, and perpetuate a Western worldview as universal truth.

Prolific, a technology company creating the largest pool of high-quality, human-derived data in the world alongside a platform to access it, believes that the AI community must take a more open-systems approach to training and evaluating LLMs—an approach that incorporates a broad range of human viewpoints while emphasizing transparency and minimizing bias. That includes more trustworthy evaluations of those LLMs, testing that doesn’t rely on a narrow community of techies.

Until now, too much of the supposedly objective evaluation has been done in self-selecting online communities that few people outside the AI tech world even know about. Unfortunately, these have become feedback loops that disproportionately influence not only the rankings of LLMs but the models’ very design. So how do we ensure that human insights and expertise can inform the creation and training of the AI models that will truly benefit people everywhere? Consider the necessary elements.

Gaining a Broader Perspective

Surveying a broad variety of people to ensure LLMs are informed by human experience and insights—not just the preferences of techies or what can be scraped from the internet—is crucial.

Some of the leading developers of LLMs have begun enlisting organizations that apply research-grade sampling methodology to AI evaluation. If you wouldn’t trust a political poll of an unknown, self-selected audience, why would you trust AI benchmarks built the same way?

Depending on the topics and how much human subjectivity should weigh on the evaluation, the sample must include geographic, ethnic, political and socioeconomic diversity. Respondents in these online surveys must also be carefully screened to ensure authenticity.

And while it might seem counterintuitive, paying the respondents to participate gives them an incentive to diligently complete surveys that can be time-consuming and require sustained attention. Many scientifically rigorous academic studies now use such incentives. (After all, if AI companies profit from these insights, shouldn‘t the people providing them be compensated fairly?)

Recognizing Where Human Input is Needed—and Where it‘s Not

Highly subjective feedback, like consumer opinions or political polling, does require diverse human responses. But factual, verifiable information like a math equation doesn’t require human input.

There is a broad middle ground—like legal opinions and advice—in which verifiable facts and the subjective interpretations of experts are combined. In such cases, the relevant knowledge of screened, verifiable experts on a topic will matter more than geographic or socioeconomic diversity.

Note the plural there: “experts.” In the race to develop LLMs, even when human input is sought, there can be pressure to rely on a single expert—to the detriment of the model’s validity. Even the world’s best doctor would be providing only one person’s point of view.

Making the AI Evaluation Auditable and Explainable

The good news is that transparency is improving. Most major AI labs now publish model cards, safety evaluations, and documentation about their training processes. The gap isn‘t in whether documentation exists—it‘s in whether evaluation methodology is rigorous enough to trust.

For example: When one major tech firm released its new AI assistant in late 2025, it also published an accompanying model card. This public document reveals the model’s training data sources, testing procedures, and known limitations, making it possible for independent researchers to scrutinize how the AI actually works. Meanwhile, a rival company pioneered an approach designed to align AI models with human values and priorities.

Toward a Human-Centric AI

Prolific has developed HUMAINE as one attempt at this—a human-centered AI leaderboard built on representative sampling, diverse human feedback, and transparent methodology. HUMAINE is not the only valid approach to AI evaluation. But the AI development community does need to adopt an open-systems mentality—one where developers are transparent about how their models are trained, evaluated and weighted.

As AI becomes more capable, the quality of human evaluation becomes more critical, not less. AI leaders owe it to the field—and to the billions of people AI is meant to serve—to apply the same rigor to AI evaluation that we’d expect in any scientific discipline.

Find out more about HUMAINE at prolific.com/humaine

By Enzo Blindow, Vice President of Data & AI at Prolific