When a UX study produces unreliable findings, the culprit isn’t always the methodology. Attracting the wrong people for your studies could mean drawing insights from users who don't represent your actual audience. And product decisions built on that data can miss the mark.
But getting the right participants isn't always easy, even for organizations that specialize in finding them. Data published in PLOS ONE shows that MTurk provided only about 26% high-quality participants, while Qualtrics, CloudResearch, and other platforms sourced closer to 50% to 60% high-quality participants
Screener surveys are your first line of defense against wasting budget on the wrong participants. To wield screeners well, you’ll need to identify the right screener questions, craft them carefully, then launch and test.
Key takeaways
A screener is a short pre-study survey that filters participants in or out before your research begins so you get the right data.
Solid screener questions start with your research goals: define who you want, who you don't, and which criteria will tell you the difference.
Well-crafted questions are hard to game, keep expectations reasonable, and give you a clear way to say yes or no to participants.
Test your screener by first sending it to a colleague, then reviewing early responses quickly.
What are screeners?
A screener is a type of survey that we use to help us identify who we do and don’t want to include in our research efforts.
You might hear it called a screener survey, a screener, or just screening questions, but all mean the same thing: a set of questions you ask potential participants to see if they’re the right fit for your study.
Most often, researchers will create screening questions and determine specific participant quotas as part of the planning process, and then choose how to launch the screener depending on the recruitment method.
If you’ll be using a recruitment vendor, they’ll likely take the questions you write and screen in their own way.
If you’re using an online recruiting tool, you’ll probably need to set up the screening questions yourself within the project setup, or you can use a traditional survey tool to publicize if you’ll be recruiting yourself.
Why are screener surveys important?
Screeners are how you sort through a pool of potential participants to find the ones that match your criteria. No matter how you recruit participants, you’ll need to send them through a screener so you spend research dollars only on the right respondents.
You may also want to include different types of users, even if you don’t have strict quotas.
For instance, let’s say you’re working with a gaming company that offers single-purchase and subscription pricing tiers, and your research goal is to understand more about who chooses each pricing tier and why so you can try to increase subscription purchases. While planning the best method, your team decides to run one-on-one interviews with people who have chosen each of the different tiers within the last month.
They want to include at least some participants who purchased a subscription after previously paying for a single use.
Even if you receive a list of contact information for customers who have opted-in to provide feedback, you may not have all the information about their past purchases, so your screener will be focused on identifying what sort of previous purchases folks have made and when.
Consider using your screener to identify and exclude folks who will not be good participants for your study.
You want participants to be eligible based on the criteria you set (i.e., has made a recent purchase), but there may also be things you’re not interested in hearing about.
Thinking of the example above, maybe your team has determined there’s a group of users who only make purchases when there are deep-discount flash sales, and they aren’t who you want to target. Whenever anyone indicates they’ve made one of these purchases, you can exclude your study.
Screeners can also help you find and weed out “serial participants” who try to make it into every potential study and don’t truly qualify.
Identifying your screener questions
These days, research fraud threatens to make up around 70% of online survey responses. Writing smart screener questions helps reduce fraud in your data.
Smart screener questions don't start with writing. They start with building a clear picture of your ideal participant. Anchor in your research goals, define who's in and who's out, and narrow your criteria down to only what matters.
Start with your research goals
Every decision about who to include or exclude in your study should trace back to what you're trying to learn.
If you already have personas and you want to include representative users, you can use the characteristics of your personas as the basis for screening questions.
If not, think through your target participants in the same way that you think about personas: consider what elements make them unique, what motivations drive their actions, and what behaviors differentiate them from others.
Also consider characteristics that you purposefully don’t want to include. For instance, if you’re running a usability test on an update to an-iOS mobile app, you may want to exclude people who don’t have and regularly use iOS devices.
Identify who you want (and who you don't)
Remember that people may not always be consistent, and they behave differently in different contexts.
Let’s say you’re working on an airline website. The way an administrative assistant looks for and books tickets for their staff will vary from the way they book travel for their family vacations, so if that matters for your study, be sure to add questions about their use context.
Think through all situations and characteristics that you need to consider, and start a list of what must be true or not to be included.
Start by listing out all the potential criteria, and then try to narrow down to only the most crucial items that will identify your target audience.
Narrow down and prioritize
Demographics aren’t necessarily what identify our target audience in UX. Things like age, race, and geography may be relevant, but you may also want to include or exclude people based on their usage or experience with a product, knowledge or opinions about a topic area, past behaviors, or roles.
Only include demographic screening questions if they truly help identify who you want to talk to. Prioritize the list and look for dependencies so you can decide the order in which to ask the questions.
Because you’re asking potential participants to fill out a survey before they’re promised a role (and therefore compensation), aim to keep the list as brief as possible, eliminate unqualified participants, and maintain a logical flow of questions.
How to craft effective screening questions
1. Translate each criterion into its own question
Once you decide what to screen in and out, you can start translating your criteria into questions. Focus on one screening criterion per question and provide precise, unambiguous answer options.
Example: You want to be sure participants frequently use your tool. Ask them how often they use it and provide answers that are specific time periods, like a number of days per week, rather than vague phrases like “often” or “usually.”
2. Default to closed-ended questions
Most of your questions should be closed-ended or multiple choice, meaning you’ll provide a list of answers for respondents to choose from. This makes it faster for them to fill out and easier for you to review responses.
Just be sure to include options like “other,” “not applicable,” “none of the above,” and “all of the above” so that you don’t force people to choose an incorrect answer and generate false positives.
3. Keep questions within realistic memory range
Aim for questions that a participant would realistically know or remember. Most people can recall what they ate for breakfast today and maybe even last week, but unless someone eats the exact same thing every day, they won’t be able to recall what their morning meal was two summers ago.
The same principle applies to product usage, purchase history, and behaviors. Be specific enough to be useful, but don’t ask for recall that doesn’t exist.
4. Obscure your target to deter fraudulent responses
Avoid being too obvious about who you’re targeting. You’ll encounter potential participants who try to guess what you’re looking for to appear like a good fit. To get around this, provide a few “correct” answers alongside some false signals that will tip you off if a participant is likely not telling the truth.
Example: You need someone who recently purchased a pair of your work boots. You frame the question as “Which of the following have you purchased in the last month?” and include your target product alongside many similar and dissimilar ones.
Someone who claims to have purchased work boots, ski boots, and cowboy boots in the same month is probably a red flag.
5. Use validation questions for criteria that really matter
If there’s a nuance you need to nail down, consider including multiple questions that ask variations of the same thing to validate responses. Be sure to keep it brief and avoid making it too obvious you’re looking for similar information across questions.
Validating with multiple questions is critical when a single misrepresented answer could compromise the quality of your entire study.
6. Add an open-ended articulation question
Open-ended “articulation” questions give you a sense of how participants will actually respond, and they can validate or add clarity to a closed-ended answer.
Let’s use our workboot example. While screening for work boot purchasers, you might add this question: “Describe the process of choosing the work boots you just purchased and tell us where you think you’ll be wearing them.”
Someone who describes what they were looking for, where they shopped, and mentions a plausible context (like a construction job) is likely a strong candidate. A very brief or vague response (“I needed new boots so I bought online”) isn’t necessarily disqualifying, but you might deprioritize that person in favor of participants who can give you more details.
Screener survey question examples
Screener questions tend to cluster around a few core criteria types. Here are examples organized by category. Pull from whichever types fit your research goals.
Behavioral questions
Behavioral questions screen for what participants actually do: the habits and actions that make them relevant (or not) to your study. These are among the most reliable screener criteria because behavior is harder to fake convincingly than self-reported attitudes.
Examples:
Which of the following have you done in the past 30 days? (Select all that apply; include your target behavior alongside plausible distractors)
What typically prompts you to [perform specific task]? (A work requirement / Personal choice / A recommendation / Other)
When you [perform specific task], which of the following do you typically use? (Select all that apply; list platforms, tools, or approaches)
Role and context questions
Role and context questions identify whether participants use a product or make decisions in the right capacity. The same person can interact with a product differently depending on whether they're acting as an individual user, a team administrator, or a purchasing decision-maker.
Examples:
When you use [product or tool], is it usually for personal use, work, or both?
What is the primary reason you [perform a specific task]? (List 4–5 plausible motivations)
Which best describes your role at your organization? (Individual contributor / Team lead or manager / Department head / Executive / Other)
Are you involved in purchasing or recommending tools for your team or family? (Yes, I make the final call / Yes, I have significant influence / I provide input but don't decide / No)
Knowledge and experience questions
These questions gauge familiarity with a product category, tool, or subject area. Use them when your study requires a minimum level of expertise or when you want a mix of novice and experienced users.
Examples:
How would you describe your experience level with [product or category]? (Never used it / Beginner / Intermediate / Advanced)
How long have you been using [product or tool]? (Less than 3 months / 3–12 months / 1–3 years / More than 3 years)
Which of the following have you done with [product or tool]? (Select all that apply; list specific features or tasks, from basic to advanced)
Have you ever encountered a problem with [product or task] that you couldn't resolve on your own? (Yes, frequently / Yes, occasionally / Rarely / Never)
Additional tips
Screeners are a special kind of survey, so most survey best practices still apply:
Write neutral, non-leading questions. There should be no hint at what you’re looking to find and no flavor of trying to validate assumptions. Remove any language that could potentially bias your participants, even unintentionally.
Don’t ask participants to predict their own future behavior. People are notoriously awful at this, but they’ll try anyway and often end up giving false information.
Keep it simple. Interdependent questions can complicate the participant experience and result in incorrect data. Simplify by creating multiple lean screeners instead of one, or use a tool that lets you build the logic behind the scenes. SurveyMonkey and Typeform are solid options.
Treat the screener as a first impression. A screener is often the first interaction a potential participant has with you or your client, so participant experience matters. Provide a warm introduction that sets context and invites participation.
Set clear expectations. Be upfront about when you’ll notify participants they’re in, how long sessions last, where or how they’ll take place, exactly what earns them a research incentive, and how they should prepare.
Distributing your screener
Once you write your screener questions, it’s time to launch and test.
How you launch the screener will depend on your recruiting method. Recruiting services will likely just need you to write the screening questions in text, and then they will follow their own screening methods.
But if you’re recruiting on your own, you’ll need to code the questions into a tool. Services like User Interviews have built-in screener tools, but you can use any survey tool to code your screener.
Pro tip: Always test your screener before sending it to participants. You’ll catch typos, technical issues, and more.
Send it to a colleague to be sure the technology works smoothly, you’ve correctly coded any branching, and that you’ve written clear questions. When in doubt, aim for simpler language and fewer questions.
If you’re using a DIY recruiting tool, you can then launch your study. The results should start trickling in.
If you’re fully recruiting on your own, you’ll then need to embed the survey URL into whatever communications channels you’ll use for outreach. I recommend doing an additional test of emails and social media posts with the link before pushing live.
The length of time you’ll need to keep your screener open can vary wildly, so try checking every few hours the first few days after launch.
Review the responses to see who best matches your criteria, and start inviting them to your study. Reach out to qualified respondents quickly while you’re still top of mind.
Conclusion
A strong screener helps you net the right participants for your study. That’s critical for stretching your research budget and delivering genuinely useful insights for your organization. And that’s why we do research in the first place, right?
To build a strong screener, clearly define who you do and don’t want to talk to based on your research goals, and then follow survey question best practices to get the best results. Then entice those people to participate in your studies by offering compensation for their time.
