6E  Synthetic Control

~2 hours SCM, Placebo Tests, Extensions

Table of Contents

Basic Synthetic Control

Synthetic control constructs a weighted combination of control units that matches the treated unit's pre-treatment characteristics. The post-treatment difference between the treated unit and synthetic control is the estimated treatment effect. 

# Python: Synthetic Control with SyntheticControlMethods
# pip install SyntheticControlMethods
from SyntheticControlMethods import Synth
import pandas as pd

# Data structure: unit, time, outcome, covariates
# treated_unit: the single treated unit
# treatment_time: when treatment occurs

# Initialize and fit
sc = Synth(df,
           outcome_var="gdp",
           id_var="country",
           time_var="year",
           treatment_period=1990,
           treated_unit="West Germany")

# Fit the synthetic control
sc.fit()

# Get results
print(sc.summary())

# Plot
sc.plot(["original", "pointwise"])

# Get weights
print("Donor weights:")
print(sc.W)
* Stata: Synthetic Control with synth
* ssc install synth

* Set panel structure
tsset country_id year

* Basic synthetic control
synth gdp gdp(1985) gdp(1986) gdp(1987) gdp(1988) gdp(1989) ///
    population trade_openness, ///
    trunit(7) trperiod(1990) ///
    figure

* With more options
synth gdp gdp(1985) gdp(1986) gdp(1987) gdp(1988) gdp(1989) ///
    population trade_openness inflation, ///
    trunit(7) trperiod(1990) ///
    xperiod(1985(1)1989) ///
    mspeperiod(1985(1)1989) ///
    resultsperiod(1985(1)2000) ///
    keep(synth_results) replace

* Get weights and gaps
matrix list e(W_weights)  // donor weights
matrix list e(V_matrix)   // predictor weights
# R: Synthetic Control with Synth package
library(Synth)

# Prepare data for Synth
dataprep.out <- dataprep(
  foo = df,
  predictors = c("gdp", "population", "trade_openness"),
  predictors.op = "mean",
  time.predictors.prior = 1985:1989,
  dependent = "gdp",
  unit.variable = "country_id",
  time.variable = "year",
  treatment.identifier = 7,  # treated unit
  controls.identifier = c(1:6, 8:20),  # donor pool
  time.optimize.ssr = 1985:1989,
  time.plot = 1985:2000
)

# Run synthetic control
synth.out <- synth(dataprep.out)

# Results
synth.tables <- synth.tab(dataprep.res = dataprep.out,
                          synth.res = synth.out)
print(synth.tables$tab.w)  # weights

# Plot
path.plot(synth.res = synth.out,
          dataprep.res = dataprep.out,
          Ylab = "GDP",
          Xlab = "Year",
          tr.intake = 1990)
Python Output 
Synthetic Control Method Results
=================================

Treated Unit: West Germany
Treatment Period: 1990
Pre-treatment periods: 1985-1989
Post-treatment periods: 1990-2000

Donor Weights:
--------------
Austria          0.423
Netherlands      0.312
Switzerland      0.198
Belgium          0.067
France           0.000
Italy            0.000
...
(remaining countries have weight = 0)

Pre-treatment Fit (RMSPE): 234.56
Post-treatment RMSPE: 1,456.78

Predictor Balance:
                    Treated    Synthetic
GDP (1985)          21,456      21,389
GDP (1986)          22,134      22,067
GDP (1987)          23,012      22,945
GDP (1988)          24,567      24,512
GDP (1989)          25,890      25,823
Population (mil)     61.2        58.9
Trade Openness       0.456       0.448

Average Treatment Effect (post-1990): -1,234.5
Stata Output 
Synthetic control method

Treated unit: 7 (West Germany)
Control units: 1-6, 8-20
Treatment period: 1990

Optimization method: Nested
MSPE minimization period: 1985-1989

Unit weights (W):
--------------------------------------------------------------------------------
         unit |     weight
--------------+----------------------------------------------------------------
      Austria |      0.423
  Netherlands |      0.312
  Switzerland |      0.198
      Belgium |      0.067
       France |      0.000
        Italy |      0.000
--------------------------------------------------------------------------------

Predictor weights (V):
--------------------------------------------------------------------------------
    predictor |     weight
--------------+----------------------------------------------------------------
    gdp(1985) |      0.156
    gdp(1986) |      0.178
    gdp(1987) |      0.189
    gdp(1988) |      0.201
    gdp(1989) |      0.187
   population |      0.056
trade_openness|      0.033
--------------------------------------------------------------------------------

Pre-treatment RMSPE: 234.56
[Graph saved: synth_germany.gph]
R Output 
Synthetic Control Method Results

Treated Unit: West Germany (ID = 7)
Treatment Period: 1990

****************
 UNIT WEIGHTS
****************
        unit.names w.weights
1          Austria     0.423
2      Netherlands     0.312
3      Switzerland     0.198
4          Belgium     0.067
5           France     0.000
6            Italy     0.000
...

****************
 PREDICTOR BALANCE
****************
                 Treated Synthetic Sample Mean
gdp(1985)          21456     21389       19234
gdp(1986)          22134     22067       19876
gdp(1987)          23012     22945       20456
gdp(1988)          24567     24512       21234
gdp(1989)          25890     25823       22012
population          61.2      58.9        45.6
trade_openness     0.456     0.448       0.412

MSPE (pre-treatment): 55034.67
Root MSPE: 234.56

[Path plot displayed showing treated vs synthetic control trajectories]

Inference: Placebo Tests

Standard errors don't apply to synthetic control. Instead, use placebo tests: apply the method to each control unit as if it were treated and compare the treated unit's effect to the placebo distribution. 

# Python: Placebo Tests
from SyntheticControlMethods import Synth
import numpy as np
import matplotlib.pyplot as plt

# Run placebo tests (in-space placebos)
all_units = df['country'].unique()
control_units = [u for u in all_units if u != 'West Germany']

placebo_effects = {}
for unit in control_units:
    sc = Synth(df, outcome_var="gdp", id_var="country",
               time_var="year", treatment_period=1990,
               treated_unit=unit)
    sc.fit()
    placebo_effects[unit] = sc.gaps  # gap = treated - synthetic

# Plot all gaps
fig, ax = plt.subplots(figsize=(10, 6))
for unit, gaps in placebo_effects.items():
    ax.plot(gaps.index, gaps.values, color='gray', alpha=0.3)
ax.plot(treated_gaps.index, treated_gaps.values, color='black', linewidth=2)
ax.axvline(x=1990, linestyle='--', color='red')
ax.axhline(y=0, linestyle='-', color='gray')
plt.show()

# Calculate p-value
post_treatment_mspe = np.mean(treated_gaps[1990:]**2)
placebo_mspe = [np.mean(gaps[1990:]**2) for gaps in placebo_effects.values()]
pvalue = np.mean(placebo_mspe >= post_treatment_mspe)
print(f"p-value: {pvalue:.3f}")
* Stata: Placebo Tests with synth_runner
* ssc install synth_runner

* Run placebo tests for all control units
synth_runner gdp gdp(1985) gdp(1986) gdp(1987) gdp(1988) gdp(1989) ///
    population trade_openness, ///
    trunit(7) trperiod(1990) ///
    gen_vars

* Plot with placebos
effect_graphs

* P-values
pval_graphs

* RMSPE ratio (pre/post treatment fit)
single_treatment_graphs
# R: Placebo Tests
library(Synth)

# Function to run SC for any unit
run_sc <- function(treated_id, donor_ids) {
  dp <- dataprep(
    foo = df,
    predictors = c("gdp", "population", "trade_openness"),
    predictors.op = "mean",
    time.predictors.prior = 1985:1989,
    dependent = "gdp",
    unit.variable = "country_id",
    time.variable = "year",
    treatment.identifier = treated_id,
    controls.identifier = donor_ids,
    time.optimize.ssr = 1985:1989,
    time.plot = 1985:2000
  )
  synth(dp)
}

# Run placebo for each control unit
all_ids <- unique(df$country_id)
treated_id <- 7
control_ids <- setdiff(all_ids, treated_id)

placebo_gaps <- lapply(control_ids, function(id) {
  donors <- setdiff(all_ids, id)
  sc <- run_sc(id, donors)
  # Calculate gaps
  ...
})

# Using tidysynth (tidyverse-friendly)
library(tidysynth)

sc <- df %>%
  synthetic_control(
    outcome = gdp,
    unit = country,
    time = year,
    i_unit = "West Germany",
    i_time = 1990,
    generate_placebos = TRUE  # auto placebo tests
  ) %>%
  generate_predictor(...) %>%
  generate_weights() %>%
  generate_control()

# Plot with placebos
sc %>% plot_placebos()
Python Output 
Running placebo tests for 19 control units...
  Austria: RMSPE = 312.45
  Belgium: RMSPE = 456.78
  Denmark: RMSPE = 289.34
  Finland: RMSPE = 523.12
  France: RMSPE = 198.67
  ...
  Switzerland: RMSPE = 345.23

Placebo Test Results
====================

Treated unit (West Germany):
  Pre-treatment RMSPE:  234.56
  Post-treatment RMSPE: 1456.78
  RMSPE ratio:          6.21

RMSPE Ratios (post/pre):
  West Germany:   6.21  <-- TREATED
  Austria:        2.34
  Belgium:        1.89
  Denmark:        2.12
  Finland:        1.45
  France:         3.12
  ...

Rank of treated unit: 1 out of 20
p-value (based on RMSPE ratio): 0.050

[Placebo plot displayed: Black line (West Germany) diverges
 significantly from gray lines (placebos) after 1990]
Stata Output 
Synthetic control with placebo tests

Running synth for 20 units...
  Unit 1 (Austria): done
  Unit 2 (Belgium): done
  ...
  Unit 7 (West Germany): TREATED UNIT
  ...
  Unit 20 (UK): done

RMSPE Summary:
--------------------------------------------------------------------------------
        Unit |   Pre-RMSPE   Post-RMSPE      Ratio       Rank
-------------+----------------------------------------------------------
West Germany |     234.56      1456.78       6.21          1
      France |     198.67       620.12       3.12          2
     Austria |     312.45       731.34       2.34          3
     Denmark |     289.34       614.56       2.12          4
     Belgium |     456.78       863.45       1.89          5
     ...
--------------------------------------------------------------------------------

Exact p-value (rank-based): 1/20 = 0.050

[effect_graphs: Shows gaps for all units, treated in black]
[pval_graphs: Shows running p-value over time]
R Output 
Running placebo tests...
Processing unit 1 of 19: Austria
Processing unit 2 of 19: Belgium
...
Processing unit 19 of 19: UK

Placebo Test Summary
====================

RMSPE Comparison:
             Unit Pre.RMSPE Post.RMSPE   Ratio Rank
1    West Germany    234.56    1456.78    6.21    1
2          France    198.67     620.12    3.12    2
3         Austria    312.45     731.34    2.34    3
4         Denmark    289.34     614.56    2.12    4
5         Belgium    456.78     863.45    1.89    5
...

Inference:
  Treated unit rank: 1 out of 20
  Two-sided p-value: 2/20 = 0.100
  One-sided p-value: 1/20 = 0.050

Using tidysynth::plot_placebos():
[Spaghetti plot showing:
 - Black line: West Germany (treated)
 - Gray lines: 19 placebo units
 - Vertical line at 1990 (treatment)
 - West Germany shows clear negative divergence post-treatment]

Critiques and Limitations

Known Issues with Synthetic Control

Recent methodological work has identified several concerns:

  • Poor pre-treatment fit: If synthetic control can't match pre-treatment outcomes well, the counterfactual is unreliable
  • Specification searching: Researchers may choose predictors/time periods that produce desired results
  • Negative weights are implicitly forbidden: This restricts the counterfactual to the convex hull of donors, which may not include the treated unit's trajectory
  • No formal inference: Placebo tests provide intuition but don't give valid p-values in finite samples
  • Extrapolation: If treated unit is outside the support of controls, the method extrapolates

Key papers: Ferman & Pinto (2021), Kaul et al. (2022), Abadie (2021 survey).

Diagnostic Checks

# Python: SC Diagnostics

# 1. Check pre-treatment fit (RMSPE)
pre_rmspe = np.sqrt(np.mean((treated_pre - synthetic_pre)**2))
print(f"Pre-treatment RMSPE: {pre_rmspe:.3f}")

# 2. Check covariate balance
print("Predictor balance:")
print(sc.comparison)  # treated vs synthetic on predictors

# 3. Check weights (few large weights is concerning)
print("Number of donors with weight > 0.05:")
print((sc.W > 0.05).sum())

# 4. RMSPE ratio (for inference)
post_rmspe = np.sqrt(np.mean((treated_post - synthetic_post)**2))
rmspe_ratio = post_rmspe / pre_rmspe
print(f"RMSPE ratio: {rmspe_ratio:.2f}")
* Stata: SC Diagnostics

* After running synth:

* 1. Pre-treatment RMSPE
ereturn list
display "Pre-treatment RMSPE: " sqrt(e(RMSPE))

* 2. Predictor balance
matrix list e(Y_balance)

* 3. Donor weights
matrix list e(W_weights)

* 4. Leave-one-out robustness
foreach donor in $donor_list {
    synth gdp ..., trunit(7) trperiod(1990) ///
        counit(`=subinstr("$donor_list", "`donor'", "", .)')
}
# R: SC Diagnostics
library(Synth)

# 1. Pre-treatment RMSPE
synth.tables$tab.pred  # predictor comparison

# 2. Weights
synth.tables$tab.w  # donor weights

# 3. Check sparsity of weights
sum(synth.out$solution.w > 0.05)

# 4. Leave-one-out
loo_results <- lapply(major_donors, function(d) {
  new_donors <- setdiff(control_ids, d)
  # Re-run synth without donor d
  ...
})

# 5. Pre-treatment fit plot
path.plot(synth.res = synth.out, dataprep.res = dataprep.out)
# Visual check: do paths match well before treatment?
Python Output 
SC Diagnostics
==============

1. Pre-treatment RMSPE: 234.560
   (Outcome mean: 23,456, so RMSPE is ~1% of mean - good fit)

2. Predictor Balance:
   Predictor        Treated    Synthetic    Difference
   gdp(1985)         21,456      21,389          67  (0.3%)
   gdp(1986)         22,134      22,067          67  (0.3%)
   gdp(1987)         23,012      22,945          67  (0.3%)
   gdp(1988)         24,567      24,512          55  (0.2%)
   gdp(1989)         25,890      25,823          67  (0.3%)
   population         61.2        58.9         2.3  (3.8%)
   trade_openness    0.456       0.448       0.008  (1.8%)

3. Donor Weight Sparsity:
   Number of donors with weight > 0.05: 4
   (Austria: 0.423, Netherlands: 0.312, Switzerland: 0.198, Belgium: 0.067)

4. RMSPE Ratio:
   Pre-treatment RMSPE:  234.56
   Post-treatment RMSPE: 1456.78
   RMSPE ratio: 6.21

   Interpretation: Post-treatment divergence is 6.2x larger than
   pre-treatment fit error - suggests meaningful treatment effect.
Stata Output 
Diagnostics for Synthetic Control
==================================

. ereturn list
scalars:
              e(RMSPE) =  55034.67136

. display "Pre-treatment RMSPE: " sqrt(e(RMSPE))
Pre-treatment RMSPE: 234.56

. matrix list e(Y_balance)
Y_balance[7,3]
                     Treated    Synthetic    Sample Mean
     gdp(1985)         21456        21389          19234
     gdp(1986)         22134        22067          19876
     gdp(1987)         23012        22945          20456
     gdp(1988)         24567        24512          21234
     gdp(1989)         25890        25823          22012
     population         61.2         58.9           45.6
trade_openness         0.456        0.448          0.412

. matrix list e(W_weights)
W_weights[19,1]
           weight
  Austria   0.423
Netherlands 0.312
Switzerland 0.198
  Belgium   0.067
   France   0.000
    ...

Leave-one-out robustness:
  Without Austria:     Effect = -1,189 (vs -1,234 with Austria)
  Without Netherlands: Effect = -1,298 (vs -1,234 with Netherlands)
  Without Switzerland: Effect = -1,212 (vs -1,234 with Switzerland)
Results are robust to excluding any single donor.
R Output 
Synthetic Control Diagnostics
=============================

1. Predictor Balance (synth.tables$tab.pred):
                 Treated Synthetic Sample Mean
gdp(1985)          21456     21389       19234
gdp(1986)          22134     22067       19876
gdp(1987)          23012     22945       20456
gdp(1988)          24567     24512       21234
gdp(1989)          25890     25823       22012
population          61.2      58.9        45.6
trade_openness     0.456     0.448       0.412

2. Donor Weights (synth.tables$tab.w):
        unit.names w.weights
1          Austria     0.423
2      Netherlands     0.312
3      Switzerland     0.198
4          Belgium     0.067
5-19        Others     0.000

3. Weight Sparsity Check:
> sum(synth.out$solution.w > 0.05)
[1] 4

4. Leave-One-Out Results:
  Excluded       Estimate   Change
  Austria          -1189     +45
  Netherlands      -1298     -64
  Switzerland      -1212     +22
  Belgium          -1241      -7
  (Baseline: -1234)

5. Pre-treatment fit: Visual inspection shows excellent match
   [Path plot displayed]

Modern Extensions

Augmented Synthetic Control (Ben-Michael et al. 2021)

Combines synthetic control with outcome regression to correct for imperfect pre-treatment fit. 

# R: Augmented Synthetic Control
library(augsynth)

# Augmented SC with ridge regression
asyn <- augsynth(outcome ~ treatment | covariate1 + covariate2,
                 unit = unit_id,
                 time = year,
                 data = df,
                 progfunc = "Ridge")

# Summary and plot
summary(asyn)
plot(asyn)

# Compare to standard SC
syn <- augsynth(outcome ~ treatment,
                unit = unit_id,
                time = year,
                data = df,
                progfunc = "None")  # standard SC
R Output 
Augmented Synthetic Control Results
====================================

Call:
augsynth(outcome ~ treatment | covariate1 + covariate2,
         unit = unit_id, time = year, data = df, progfunc = "Ridge")

Average ATT Estimate (post-treatment):
        Estimate  Std.Error   t value     p value
ATT       -1,312      412.3    -3.182      0.0015 **

Time-varying effects:
  Year   Estimate   Std.Error     95% CI
  1990      -234       189.2    [-605,  137]
  1991      -567       234.5    [-1027,-107]
  1992      -892       298.1    [-1476,-308]
  1993     -1,234      356.7    [-1933,-535]
  ...
  2000     -2,156      489.2    [-3115,-1197]

Pre-treatment fit:
  L2 Imbalance (pre-treatment): 0.0234
  Percent improvement from augmentation: 34.2%

Comparison: Standard SC vs Augmented SC
                    Standard    Augmented
ATT Estimate          -1,234      -1,312
Std. Error               N/A       412.3
Pre-treatment RMSPE   234.56      154.23

[Plot displayed showing point estimates with confidence bands]

Synthetic Difference-in-Differences (Arkhangelsky et al. 2021)

Combines DiD and SC: uses SC weights on units AND time periods to construct the counterfactual. 

# Python: Synthetic DiD
# pip install synthdid
from synthdid.synthdid import SynthDID

# Prepare data: Y matrix (units x time), treatment indicator
sdid = SynthDID(Y, treatment_matrix)
estimate = sdid.fit()
print(f"SDID estimate: {estimate:.3f}")

# Get weights
print("Unit weights:", sdid.omega)
print("Time weights:", sdid.lambda_)
* Stata: Synthetic DiD
* ssc install sdid

* Basic SDID
sdid outcome unit_id year treatment, vce(bootstrap) seed(1234)

* With graph
sdid outcome unit_id year treatment, vce(bootstrap) graph
# R: Synthetic DiD with synthdid
library(synthdid)

# Prepare panel data
setup <- panel.matrices(df,
                        unit = unit_id,
                        time = year,
                        outcome = outcome,
                        treatment = treatment)

# Estimate SDID
sdid_est <- synthdid_estimate(setup$Y, setup$N0, setup$T0)
print(sdid_est)

# Standard errors via bootstrap
se <- sqrt(vcov(sdid_est, method = "bootstrap"))

# Compare to DiD and SC
did_est <- did_estimate(setup$Y, setup$N0, setup$T0)
sc_est <- sc_estimate(setup$Y, setup$N0, setup$T0)

# Plot
synthdid_plot(sdid_est)
Python Output 
Synthetic Difference-in-Differences (SDID)
==========================================

SDID estimate: -1,287.456

Unit weights (omega):
  Austria:      0.312
  Netherlands:  0.267
  Switzerland:  0.189
  Belgium:      0.134
  France:       0.098
  (Remaining units have small weights)

Time weights (lambda):
  1985: 0.034
  1986: 0.089
  1987: 0.156
  1988: 0.267
  1989: 0.454
  (More weight on periods closer to treatment)

Comparison of Methods:
                Estimate     SE
DiD             -1,456    523.4
SC              -1,234      N/A
SDID            -1,287    398.2

Note: SDID interpolates between DiD (equal unit weights)
and SC (zero time weights for pre-treatment).
Stata Output 
Synthetic Difference-in-Differences

Number of observations:        320 (20 units x 16 periods)
Treated unit:                  West Germany
Treatment period:              1990
Control units:                 19
Pre-treatment periods:         5
Post-treatment periods:        11

------------------------------------------------------------------------------
             |      SDID    Bootstrap        [95% Conf. Interval]
             |   Estimate    Std. Err.
-------------+----------------------------------------------------------------
   treatment |  -1287.456     398.234        -2067.98       -506.93
------------------------------------------------------------------------------
Bootstrap replications: 200

Unit Weights:
          Austria: 0.312
      Netherlands: 0.267
      Switzerland: 0.189
          Belgium: 0.134
           France: 0.098

Time Weights:
  1985: 0.034    1986: 0.089    1987: 0.156    1988: 0.267    1989: 0.454

[Graph saved: sdid_plot.gph]
R Output 
Synthetic Difference-in-Differences Estimate
============================================

> print(sdid_est)
synthdid: -1287.456

> se <- sqrt(vcov(sdid_est, method = "bootstrap"))
> se
[1] 398.234

95% Confidence Interval: [-2067.98, -506.93]

Comparison of Estimators:
> did_est
did: -1456.123

> sc_est
sc: -1234.567

Summary Table:
        Method  Estimate   Std.Err.     95% CI
1          DiD   -1456.1     523.4  [-2482, -430]
2           SC   -1234.6       N/A            N/A
3         SDID   -1287.5     398.2  [-2068, -507]

Unit weights (omega):
         Austria      Netherlands      Switzerland
           0.312            0.267            0.189

Time weights (lambda):
  1985   1986   1987   1988   1989
 0.034  0.089  0.156  0.267  0.454

[synthdid_plot displayed showing:
 - Treated unit path (solid black)
 - Synthetic control path (dashed)
 - Treatment time vertical line
 - Confidence bands]
Key References
  • Abadie, A., Diamond, A., & Hainmueller, J. (2010). "Synthetic Control Methods for Comparative Case Studies." JASA.
  • Abadie, A. (2021). "Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects." JEL.
  • Arkhangelsky, D., Athey, S., Hirshberg, D., Imbens, G., & Wager, S. (2021). "Synthetic Difference-in-Differences." AER.
  • Ben-Michael, E., Feller, A., & Rothstein, J. (2021). "The Augmented Synthetic Control Method." JASA.
  • Ferman, B. & Pinto, C. (2021). "Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets." JBES.