Click any country to explore its preferences

What people choose

Bradley–Terry scores adjusted for presentation order. Higher is better; 1,000 is the population baseline.

All respondents · worldwide
n = 331,112 votes · 6,864 people
  1. 1,022
    #1GPT Image 2OpenAI
  2. 1,017
    #2Gemini 3.1 Flash ImageGoogle
  3. 1,014
    #3MAI Image 2.5 ProMicrosoft
  4. 1,013
    #4Seedream 5.0 ProByteDance
  5. 1,010
    #5Gemini 3.1 Flash Lite ImageGoogle
  6. 1,008
    #6Qwen Image 3 ProAlibaba
  7. 1,008
    #7Gemini 3 Pro ImageGoogle
  8. 1,003
    #8MAI Image 2.5Microsoft
  1. #9FLUX.2 FlexBlack Forest Labs1,003
  2. #10Krea 2 LargeKrea1,002
  3. #11Qwen Image 3Alibaba1,001
  4. #12Seedream 4.5ByteDance1,001
  5. #13Seedream 5.0 LiteByteDance999
  6. #14Grok Imagine (quality)xAI999
  7. #15FLUX.2 ProBlack Forest Labs998
  8. #16FLUX.2 MaxBlack Forest Labs996
  9. #17Krea 2 MediumKrea995
  10. #18Krea 2 Medium TurboKrea995
  11. #19Recraft V4.1 UtilityRecraft991
  12. #20Recraft V4.1 Utility ProRecraft991
  13. #21Gemini 2.5 Flash ImageGoogle987
  14. #22Recraft V4Recraft984
  15. #23Recraft V4 ProRecraft983
  16. #24GPT Image 1OpenAI982

Preference, not an automated benchmark.

Scores come from blind A/B choices by people; 1,000 is the population baseline. Presentation order alone is worth +10 points in this modality, so every score is adjusted for it.

Groups are shown. Individuals never are.

A demographic view appears only after it reaches at least 150 votes and 25 participants.

Research

Human Preference Heterogeneity for Generative AI Model Outputs Across Different Modalities

Our results show that different populations can rank the same models differently. Aggregate results broadly align with established human-preference evaluations, but population views reveal differences hidden by a single global ranking. Retaining anonymized information about who provided each judgment lets us examine variation across geography, profession, age, gender, and other characteristics.

blind votes
465,356
participants
7,151
models
54
countries ranked
34

We present our human-preference evaluations of generative AI models. The dataset includes more than 450,000 blind pairwise comparisons from participants across countries, professions, and demographic groups, after quality filtering.

Worldwide
  1. 1GPT Image 2
  2. 2Gemini 3.1 Flash Image
  3. 3MAI Image 2.5 Pro
  4. 4Seedream 5.0 Pro
  5. 5Gemini 3.1 Flash Lite Image
China
  1. 1GPT Image 2
  2. 2Gemini 3.1 Flash Image
  3. 3MAI Image 2.5 Pro
  4. 4Seedream 5.0 Pro
  5. 5Gemini 3 Pro Image▲2

Why pooled rankings are not enough

Human preference is not a single global quantity. A pooled leaderboard reflects the particular distribution of people, prompts, and contexts that generated its votes. It does not imply that the same ordering will hold for every population or deployment.

Real AI systems are rarely deployed to an average user. A company may care about customers in a particular market, a professional tool about people with specific expertise, or a public institution about the population it serves.

Pairwise vote
Output A
Output B
Estimation
Blind comparison, models hiddenRecorded
Presentation-order term+10 Elo
Quality filtersApplied
UncertaintyAcross people

How we estimate preference

Participants compare two model outputs without seeing which models produced them. We estimate relative strength with a Bradley–Terry model and include a position term to account for systematic differences associated with where an output appears.

Worldwide confidence intervals come from a bootstrap that resamples participants rather than individual votes, so heavy voters do not make results look more certain than they are. Population-specific rankings are point estimates from the same model fitted within each group.

Before estimation we remove 248,734 votes from participants whose voting pattern did not reflect the content shown, such as choosing the same side almost every time, and traffic from sources that failed our quality checks.

where market = worldwideGPT Image 2#1 worldwide
where market = ChinaGPT Image 2#1 in China · #1 worldwide
where market = … and task = … and language = …next

From rankings to model selection

Most evaluation asks which model is best. Deployment decisions are narrower: which model is best for the users and task we care about? The answer can change with market, language, age, profession, and modality.

Preference is one dimension of quality. Accuracy, safety, latency, cost, reliability, and policy requirements can be equally important.

Model selection should be a conditional query, not a global number.

Where this goes

The goal is not only to identify the model with the highest aggregate score. It is to understand which models perform best for the specific populations and use cases an organization actually serves.

We want to map where different AI models work best across combinations of population, language, task, domain, market, modality, and deployment conditions. These results reflect the participants represented in the dataset and should not be read as census-representative estimates of entire countries.

We welcome collaboration with AI labs, researchers, companies, and public institutions on model evaluation, population-specific studies, additional modalities, and new model releases.

Collaborate with us