Skip to content
craze_Platform
← Research

Guide · 9 min read

Building a population

Sources, sampling, and how to tell whether the group you built is the group you meant.

A population is a set of digital twins and a record of where they came from. Every simulation runs against exactly one, so the population is the single largest lever on whether an answer is any good. A careful question put to the wrong people is still the wrong answer.

Define the group before you build it

Describe the population by the behaviour that matters to your decision, then add demographics only where they change that behaviour. “People who bought an energy drink in the last month” is a better starting point than “18 to 34 year olds”, because it names who actually makes the choice you are testing. Write down, too, who should be excluded: lapsed buyers, people outside your markets, people who never shop the category.

The four source types

  • Verified panelInterviewed, consented respondents. The most expensive to gather and the most reliable, because each twin is grounded in a real transcript. Use it as the base for anything about the general public or a paying customer.
  • Social graphSampled from the followers of one or more accounts. Fast, wide, and biased toward whoever follows that account. Good for reach and reaction questions among an audience you already have, weaker for anything about the general public.
  • ImportYour own character definitions as JSON. Use this when you already know the segment and want the twins to match it exactly, for example a persona set your team has agreed on.
  • FilterA narrowing expression over an existing pool. Cheap, and only as good as the attributes it filters on. Filtering on something the twins were never grounded in returns a group that looks right and behaves at random.

Stacking sources

Sources combine. A useful pattern is a panel base for grounding, a social graph for reach, and a filter to cut the result down to the behaviour you care about. Twins appearing in more than one source are deduplicated, so the total is smaller than the parts.

A population for a brand's launch test
Base
Verified panel: national, adults who buy the category
Reach
Social graph: followers of the brand and two rivals
Filter
Bought in the category in the last 30 days
Exclude
Outside the launch markets
The panel keeps the group anchored to real buyers; the graph adds the people most likely to see the launch first; the filter keeps only those who could plausibly buy it this month.

How big it needs to be

A run draws a weighted sample from the population, and around two thousand twins is usually enough to separate options more than a few points apart overall. The number that matters more is the smallest segment you plan to report on. If you intend to compare parents with non-parents, each group needs enough twins on its own to be read, and a segment of a few dozen will swing from run to run.

Size for the segment, not the headline. If one segment is small but important, build it as its own population and run it separately.

Checking your work

Every population carries an alignment score: how closely it reproduced held-out human responses on questions it had not seen. A population in the high eighties or low nineties is dependable. Below that, look at the source mix before you look at the model.

  1. 01Check the compositionCompare the population's make-up by age, region and the behaviour you filtered on against what you intended. A filter that quietly removed most of one region changes every answer after it.
  2. 02Read a few twinsOpen ten at random. Would each plausibly be in the group you described? One obviously wrong twin in ten is a source problem, not bad luck.
  3. 03Ask a question you knowRun one simulation whose answer you already have, such as a past launch or a known category share. If the population misses a question you can check, do not trust it on one you cannot.
  4. 04Then read the alignment scoreWith the first three checks done, the score tells you how far to trust the group on new questions.

A population built only from a social graph will usually align worse than one with a panel base. That is not a defect in the graph; it is the graph telling you it is a different group of people.

Common mistakes

  • Describing the customer you wantA population built from your target persona answers as that persona. If you want to know how the market will react, build the market.
  • Stacking filters until nothing is leftEach filter narrows the pool. Five in a row can leave a group too small to read and too odd to generalise from.
  • Editing a population mid-studySave a new population instead of changing the one you already ran. Otherwise earlier runs are no longer comparable with later ones.