Skip to main content
Back to Frameworks
Growth · Scale-up

Survey Design Framework

Bad surveys produce bad data, and bad data leads to bad decisions. This framework helps you design surveys that are methodologically sound: define screening criteria to reach the right respondents, structure question flow to minimise bias, choose appropriate scales, and plan your analysis before writing a single question. The 'analysis-first' approach ensures every question earns its place.

When to use this framework

  • →You're designing a customer satisfaction, NPS, or brand tracking survey
  • →You need to validate qualitative findings with a larger quantitative sample
  • →You're running a concept test, ad test, or pricing study
  • →You want to segment your audience based on attitudes or behaviours
  • →You're setting up a recurring survey (quarterly tracker, post-purchase, etc.)
Related:Research Design Template STP Model (Segmentation, Targeting, Positioning) Product-Market Fit Scorecard

Sign in to unlock full access

You're browsing as a visitor. Create a free account to fill in worksheets, download PDFs, and save your progress.

Get startedSign in
Sign in to download
Worked Example

Spotify

1. Survey Objective & Audience

What one thing must this survey tell you? Keep it focused — surveys that try to answer everything answer nothing.

Measure brand perception and feature awareness among free-tier users to identify the most compelling upgrade triggers for Premium conversion.

Who exactly should take this survey? Define demographics, behaviours, and any qualifying criteria.

Spotify Free users aged 18-35 who have used the app at least 3x in the past 30 days. Mix of markets: US, UK, Germany, Brazil. Exclude anyone who has ever been a Premium subscriber.

What questions will filter out unqualified respondents? These run before the main survey.

1. Which music streaming services do you currently use? (must select Spotify) 2. Which best describes your Spotify account? (must select 'Free / ad-supported') 3. How often have you used Spotify in the past 30 days? (must select 3+ times)

2. Question Design

Plan question types and flow before writing specific questions.

What are the 3-5 numbers you need from this survey? NPS score, awareness %, purchase intent, etc.

1. Unaided awareness of Premium features (%) 2. Feature appeal rating (1-7 scale for each feature) 3. Likelihood to upgrade in next 6 months (1-10 scale) 4. Maximum willingness to pay (open numeric) 5. Primary barrier to upgrading (single select)

Plan the logical flow: general → specific, unaided → aided, attitudes → behaviour → demographics. List the sections.

Section 1: Music listening habits (general, warm-up) Section 2: Spotify usage patterns (frequency, features used) Section 3: Unaided awareness of Premium features (open-ended) Section 4: Aided feature evaluation (show features, rate appeal) Section 5: Upgrade intent and barriers Section 6: Pricing and willingness to pay Section 7: Demographics

What scales will you use? 5-point Likert, 7-point agreement, 1-10 satisfaction, ranking, MaxDiff? Be consistent.

Feature appeal: 7-point scale (1=Not at all appealing to 7=Extremely appealing) — 7 points gives better discrimination than 5 for feature comparison. Upgrade likelihood: 0-10 NPS-style scale for consistency with existing tracking. Barriers: Single-select from pre-tested list + 'Other' with open-end.

3. Bias & Quality Checks

Which questions need randomised answer options? Which sections need rotation? Where could question order influence answers?

1. Feature list in Section 4: randomise order for each respondent 2. Barrier list in Section 5: randomise order 3. Sections 1-3 must stay in order (general → specific) 4. Unaided awareness (S3) MUST come before aided evaluation (S4) — never reverse

Review each question: does it suggest the 'right' answer? Replace 'How much do you love X?' with 'How would you rate X?'

Changed: 'How much do you enjoy Spotify's ad-free experience?' → removed (they're free users, haven't experienced it). Changed: 'Would you agree that Premium is good value?' → 'How would you rate the value of Premium at [price]?'

Attention checks, minimum completion time, straight-line detection, open-end quality checks.

1. Attention check at Q12: 'Please select Strongly Disagree for this question' 2. Minimum completion time: 3 minutes (flag responses under 3 min) 3. Straight-line detection: flag respondents who give identical answers to 8+ consecutive grid items 4. Open-end quality: require minimum 10 characters for open-ended responses

4. Pre-Planned Analysis

Decide your analysis approach BEFORE collecting data, not after.

What subgroup comparisons matter? By segment, by usage level, by demographics?

1. Feature appeal by market (US vs UK vs DE vs BR) 2. Upgrade intent by usage frequency (light vs heavy users) 3. Barriers by age group (18-24 vs 25-35) 4. WTP by market and usage frequency 5. Feature awareness by tenure (new users vs 2+ year users)

How many responses do you need for statistical significance at the subgroup level?

Total: 2,000 (500 per market). At n=500 per market, margin of error is ±4.4% at 95% confidence. Subgroup analysis (e.g., heavy users in UK) requires minimum n=100 per cell — may need to over-sample heavy users.

How will results be reported? Dashboard, slide deck, automated alerts for key thresholds?

Interactive dashboard (Looker) with market-level filters. Executive slide deck (15 slides) for leadership review. Automated Slack alert if upgrade intent drops below 30% in any market.
Your Worksheet

Fill in for your brand

Sign in to use this worksheet

Create a free account to fill in frameworks with your own brand details, download completed worksheets, and save your progress.

Create free accountAlready have an account? Sign in

Take this framework offline

Download a blank PDF to fill in by hand, use it in workshops, or pin it to your wall. If you've filled in the worksheet above, you can download your completed version too.

Create free account to downloadAlready have an account? Sign in