5B.2  Survey Programming

Qualtrics Basics

Qualtrics is the most common survey platform in academic research. You can build surveys through the visual interface or import them programmatically.

Key Features for Experiments

• Branch logic: Show different questions/treatments based on responses or embedded data
• Embedded data: Store variables passed via URL (treatment assignment, participant IDs)
• Randomization: Built-in randomizer for within-survey treatment assignment
• Piped text: Display dynamic content based on previous responses
• Quotas: Control how many participants see each condition

URL Parameters for Treatment Assignment

The most reliable way to assign treatments is through URL parameters. When participants click a link with embedded parameters, Qualtrics captures those values.

URL Structure

# Base survey URL with parameters
https://yourschool.qualtrics.com/jfe/form/SV_XXXX?treatment=1&pid=ABC123

# Parameters:
treatment=1    # Treatment condition (e.g., 0=control, 1=treatment)
pid=ABC123    # Participant ID from recruitment platform
source=prolific # Track recruitment source

Setting Up Embedded Data in Qualtrics

1. Go to Survey Flow
2. Add an Embedded Data element at the top
3. Add fields matching your URL parameters: treatment, pid
4. Leave values blank (they'll be filled from URL)

Using Embedded Data for Branching

In Survey Flow, add a Branch that checks the embedded data:

If treatment = 1
  → Show Treatment Block
If treatment = 0
  → Show Control Block

Prolific Integration

Prolific is a participant recruitment platform that provides high-quality respondents for online experiments. Integration with Qualtrics involves:

1. Passing Prolific IDs to Qualtrics

Prolific automatically appends participant info to your survey URL. Configure your study to include:

# Your survey URL in Prolific:
https://yourschool.qualtrics.com/jfe/form/SV_XXXX?PROLIFIC_PID={{%PROLIFIC_PID%}}&STUDY_ID={{%STUDY_ID%}}&SESSION_ID={{%SESSION_ID%}}

2. Redirecting Back to Prolific

At survey end, redirect participants to Prolific's completion URL:

# End of survey redirect URL (in Qualtrics):
https://app.prolific.co/submissions/complete?cc=XXXXXXXX

# The completion code (cc) is provided by Prolific for your study

3. Python Script for Pre-Assigning Treatments

# Python: Generate treatment-assigned URLs for Prolific
import pandas as pd
import numpy as np

# Base survey URL
base_url = "https://yourschool.qualtrics.com/jfe/form/SV_XXXX"

# List of participant IDs (from Prolific or pre-registration)
participant_ids = ["P001", "P002", "P003", "P004", "P005", "P006"]

# Random assignment
np.random.seed(42)
n = len(participant_ids)
treatments = np.random.choice([0, 1], size=n)

# Generate personalized URLs
urls = [
    f"{base_url}?pid={pid}&treatment={t}"
    for pid, t in zip(participant_ids, treatments)
]

# Save assignment for analysis
df = pd.DataFrame({
    'participant_id': participant_ids,
    'treatment': treatments,
    'survey_url': urls
})
print(df.to_string(index=False))
df.to_csv('treatment_assignment.csv', index=False)

participant_id  treatment                                                        survey_url
          P001          0  https://yourschool.qualtrics.com/jfe/form/SV_XXXX?pid=P001&treatment=0
          P002          1  https://yourschool.qualtrics.com/jfe/form/SV_XXXX?pid=P002&treatment=1
          P003          0  https://yourschool.qualtrics.com/jfe/form/SV_XXXX?pid=P003&treatment=0
          P004          0  https://yourschool.qualtrics.com/jfe/form/SV_XXXX?pid=P004&treatment=0
          P005          1  https://yourschool.qualtrics.com/jfe/form/SV_XXXX?pid=P005&treatment=1
          P006          1  https://yourschool.qualtrics.com/jfe/form/SV_XXXX?pid=P006&treatment=1

Saved to: treatment_assignment.csv

Embedded Data and Randomization

Within-Survey Randomization

If you prefer to randomize within Qualtrics rather than via URLs:

1. In Survey Flow, add a Randomizer element
2. Set to "Evenly Present Elements"
3. Add your treatment blocks as elements
4. Store the assignment in an embedded data field for analysis