DAGs and
potential outcomes

Session 5

PMAP 8521: Program evaluation
Andrew Young School of Policy Studies

1 / 45

Plan for today2 / 45

Plan for today

do()ing observational
causal inference

2 / 45

Plan for today

do()ing observational
causal inference

Potential outcomes

2 / 45

do()ing observational
causal inference3 / 45

Structural models

The relationship between nodes can be described with equations

$\begin{aligned} \text{Loc} &= f_\text{Loc}(\text{U1}) \\ \text{Bkgd} &= f_\text{Bkgd}(\text{U1}) \\ \text{JobCx} &= f_\text{JobCx}(\text{Edu}) \\ \text{Edu} &= f_\text{Edu}(\text{Req}, \text{Loc}, \text{Year}) \\ \text{Earn} &= f_\text{Earn}(\text{Edu}, \text{Year}, \text{Bkgd}, \\ & \quad\quad\quad\quad \text{Loc}, \text{JobCx}) \\ \end{aligned}$

4 / 45

Structural models

dagify() in ggdag forces you to think this way

$\begin{aligned} \text{Earn} &= f_\text{Earn}(\text{Edu}, \text{Year}, \text{Bkgd}, \\ & \quad\quad\quad\quad \text{Loc}, \text{JobCx}) \\ \text{Edu} &= f_\text{Edu}(\text{Req}, \text{Loc}, \text{Year}) \\ \text{JobCx} &= f_\text{JobCx}(\text{Edu}) \\ \text{Bkgd} &= f_\text{Bkgd}(\text{U1}) \\ \text{Loc} &= f_\text{Loc}(\text{U1}) \end{aligned}$

dagify(
  Earn ~ Edu + Year + Bkgd + Loc + JobCx,
  Edu ~ Req + Loc + Bkgd + Year,
  JobCx ~ Edu,
  Bkgd ~ U1,
  Loc ~ U1
)

5 / 45

Causal identification

All these nodes are related; there's correlation between them all

We care about
Edu → Earn, but what do we do about all the other nodes?

6 / 45

Causal identification

A causal effect is identified if the association between treatment and outcome is propertly stripped and isolated

7 / 45

Paths and associations

Arrows in a DAG transmit associations

You can redirect and control those paths by "adjusting" or "conditioning"

8 / 45

Three types of associations

Confounding

Common cause

Causation

Mediation

Collision

Selection /
endogeneity

9 / 45

Interventions

do-operator

Making an intervention in a DAG

$P[Y\ |\ do(X = x)] \quad \text{or} \quad E[Y\ |\ do(X = x)]$

10 / 45

Interventions

do-operator

Making an intervention in a DAG

$P[Y\ |\ do(X = x)] \quad \text{or} \quad E[Y\ |\ do(X = x)]$

P = probability distribution, or E = expectation/expected value

10 / 45

Interventions

do-operator

Making an intervention in a DAG

$P[Y\ |\ do(X = x)] \quad \text{or} \quad E[Y\ |\ do(X = x)]$

P = probability distribution, or E = expectation/expected value

Y = outcome, X = treatment;
x = specific value of treatment

10 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

11 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

E[ Firm growth | do(Government R&D funding)]

11 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

E[ Firm growth | do(Government R&D funding)]

E[ Air quality | do(Carbon tax)]

11 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

E[ Firm growth | do(Government R&D funding)]

E[ Air quality | do(Carbon tax)]

E[ Juvenile delinquency | do(Truancy program)]

11 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

E[ Firm growth | do(Government R&D funding)]

E[ Air quality | do(Carbon tax)]

E[ Juvenile delinquency | do(Truancy program)]

E[ Malaria infection rate | do(Mosquito net)]

11 / 45

Interventions

When you do() X, delete all arrows into it

12 / 45

Interventions

When you do() X, delete all arrows into it

Observational DAG

12 / 45

Interventions

When you do() X, delete all arrows into it

Observational DAG

Experimental DAG

12 / 45

Interventions

$E[\text{Earnings}\ |\ do(\text{College education})]$

13 / 45

Interventions

$E[\text{Earnings}\ |\ do(\text{College education})]$

Observational DAG

13 / 45

Interventions

$E[\text{Earnings}\ |\ do(\text{College education})]$

Observational DAG

Experimental DAG

13 / 45

Undo()ing things

We want to know P[Y | do(X)]
but all we have is
observational data X, Y, and Z

14 / 45

Undo()ing things

We want to know P[Y | do(X)]
but all we have is
observational data X, Y, and Z

$P[Y\ |\ do(X)] \neq P(Y\ |\ X)$

14 / 45

Undo()ing things

We want to know P[Y | do(X)]
but all we have is
observational data X, Y, and Z

$P[Y\ |\ do(X)] \neq P(Y\ |\ X)$

Correlation isn't causation!

14 / 45

Undo()ing things

Our goal with observational data:
Rewrite P[Y | do(X)] so that it doesn't have a do() anymore (is "do-free")

15 / 45

do-calculus

A set of three rules that let you manipulate a DAG
in special ways to remove do() expressions

WAAAAAY beyond the score of this class!
Just know it exists and computer algorithms can do it for you!

16 / 45

https://arxiv.org/abs/1906.07125

Special cases of do-calculus

Backdoor adjustment

Frontdoor adjustment

17 / 45

Backdoor adjustment

$P[Y\ |\ do(X)] = \sum_Z P(Y\ |\ X, Z) \times P(Z)$

↑ That's complicated!

The right-hand side of the equation means "the effect of X on Y after adjusting for Z"

There's no do() on that side!

18 / 45

Frontdoor adjustment

S → T is d-separated; T → C is d-separated
combine the effects to find S → C

19 / 45

Moral of the story

If you can transform do() expressions to
do-free versions, you can legally make causal inferences from observational data

20 / 45

Moral of the story

If you can transform do() expressions to
do-free versions, you can legally make causal inferences from observational data

Backdoor adjustment is easiest to see +
dagitty and ggdag do this for you!

20 / 45

Moral of the story

If you can transform do() expressions to
do-free versions, you can legally make causal inferences from observational data

Backdoor adjustment is easiest to see +
dagitty and ggdag do this for you!

Fancy algorithms (found in the causaleffect package)
can do the official do-calculus for you too

20 / 45

Potential outcomes21 / 45

Program effect

22 / 45

Some equation translations

Causal effect = δ (delta)

$\delta = P[Y\ |\ do(X)]$

23 / 45

Some equation translations

Causal effect = δ (delta)

$\delta = P[Y\ |\ do(X)]$

$\delta = E[Y\ |\ do(X)] - E[Y\ |\ \hat{do}(X)]$

23 / 45

Some equation translations

Causal effect = δ (delta)

$\delta = P[Y\ |\ do(X)]$

$\delta = E[Y\ |\ do(X)] - E[Y\ |\ \hat{do}(X)]$

$\delta = (Y\ |\ X = 1) - (Y\ |\ X = 0)$

23 / 45

Some equation translations

Causal effect = δ (delta)

$\delta = P[Y\ |\ do(X)]$

$\delta = E[Y\ |\ do(X)] - E[Y\ |\ \hat{do}(X)]$

$\delta = (Y\ |\ X = 1) - (Y\ |\ X = 0)$

$\delta = Y_1 - Y_0$

23 / 45

24 / 45

https://www.thisamericanlife.org/691/gardens-of-branching-paths

Fundamental problem
of causal inference

$\delta_i = Y_i^1 - Y_i^0 \quad \text{in real life is} \quad \delta_i = Y_i^1 - ???$

Individual-level effects are impossible to observe!

There are no individual counterfactuals!

25 / 45

Average treatment effect (ATE)

Solution: Use averages instead

$\text{ATE} = E(Y_1 - Y_0) = E(Y_1) - E(Y_0)$

26 / 45

Average treatment effect (ATE)

Solution: Use averages instead

$\text{ATE} = E(Y_1 - Y_0) = E(Y_1) - E(Y_0)$

Difference between average/expected value when
program is on vs. expected value when program is off

$\delta = (\bar{Y}\ |\ P = 1) - (\bar{Y}\ |\ P = 0)$

26 / 45

Person
Age
Treated
Outcome
with program
Outcome
without program
Effect


1
Old
TRUE
80
60
20

2
Old
TRUE
75
70
5

3
Old
TRUE
85
80
5

4
Old
FALSE
70
60
10

5
Young
TRUE
75
70
5

6
Young
FALSE
80
80
0

7
Young
FALSE
90
100
-10

8
Young
FALSE
85
80
5


27 / 45

Person	Age	Treated	Outcome with program	Outcome without program	Effect
1	Old	TRUE	80	60	20
2	Old	TRUE	75	70	5
3	Old	TRUE	85	80	5
4	Old	FALSE	70	60	10
5	Young	TRUE	75	70	5
6	Young	FALSE	80	80	0
7	Young	FALSE	90	100	-10
8	Young	FALSE	85	80	5

$\delta = (\bar{Y}\ |\ P = 1) - (\bar{Y}\ |\ P = 0)$

$\text{ATE} = \frac{20 + 5 + 5 + 5 + 10 + 0 + -10 + 5}{8} = 5$

28 / 45

CATE

ATE in subgroups

29 / 45

CATE

ATE in subgroups

Is the program more
effective for specific age groups?

29 / 45

Person	Age	Treated	Outcome with program	Outcome without program	Effect
1	Old	TRUE	80	60	20
2	Old	TRUE	75	70	5
3	Old	TRUE	85	80	5
4	Old	FALSE	70	60	10
5	Young	TRUE	75	70	5
6	Young	FALSE	80	80	0
7	Young	FALSE	90	100	-10
8	Young	FALSE	85	80	5

$\delta = (\bar{Y}_\text{O}\ |\ P = 1) - (\bar{Y}_\text{O}\ |\ P = 0)$

$\delta = (\bar{Y}_\text{Y}\ |\ P = 1) - (\bar{Y}_\text{Y}\ |\ P = 0)$

$\text{CATE}_\text{Old} = \frac{20 + 5 + 5 + 10}{4} = 10$

$\text{CATE}_\text{Young} = \frac{5 + 0 - 10 + 5}{4} = 0$

30 / 45

ATT and ATU

Average treatment on the treated

ATT / TOT

Effect for those with treatment

31 / 45

ATT and ATU

Average treatment on the treated

ATT / TOT

Effect for those with treatment

Average treatment on the untreated

ATU / TUT

Effect for those without treatment

31 / 45

Person	Age	Treated	Outcome with program	Outcome without program	Effect
1	Old	TRUE	80	60	20
2	Old	TRUE	75	70	5
3	Old	TRUE	85	80	5
4	Old	FALSE	70	60	10
5	Young	TRUE	75	70	5
6	Young	FALSE	80	80	0
7	Young	FALSE	90	100	-10
8	Young	FALSE	85	80	5

$\delta = (\bar{Y}_\text{T}\ |\ P = 1) - (\bar{Y}_\text{T}\ |\ P = 0)$

$\delta = (\bar{Y}_\text{U}\ |\ P = 1) - (\bar{Y}_\text{U}\ |\ P = 0)$

$\text{CATE}_\text{Treated} = \frac{20 + 5 + 5 + 5}{4} = 8.75$

$\text{CATE}_\text{Untreated} = \frac{10 + 0 - 10 + 5}{4} = 1.25$

32 / 45

ATE, ATT, and ATU

The ATE is the weighted average
of the ATT and ATU

33 / 45

ATE, ATT, and ATU

The ATE is the weighted average
of the ATT and ATU

$\text{ATE} = (\pi_\text{Treated} \times \text{ATT}) + (\pi_\text{Untreated} \times \text{ATU})$

$(\frac{4}{8} \times 8.75) + (\frac{4}{8} \times 1.25)$

$4.375 + 0.625 = 5$

π here means "proportion," not 3.1415

33 / 45

Selection bias

ATE and ATT aren't always the same

ATE = ATT + Selection bias

$\begin{aligned} 5 &= 8.75 + x \\ x &= -3.75 \end{aligned}$

Randomization fixes this, makes x = 0

34 / 45

Actual data

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

Treatment not
randomly assigned

We can't see
unit-level causal effects

What do we do?!

35 / 45

Actual data

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

Treatment seems to be correlated with age

36 / 45

Actual data

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

We can estimate the ATE by finding the weighted average of age-based CATEs

As long as we assume/pretend treatment was randomly assigned within each age = unconfoundedness

$\widehat{\text{ATE}} = \pi_\text{Old} \widehat{\text{CATE}_\text{Old}} + \pi_\text{Young} \widehat{\text{CATE}_\text{Young}}$

37 / 45

Actual data

$\color{#FF851B}{\widehat{\text{ATE}}} = \pi_\text{Old} \color{#2ECC40}{\widehat{\text{CATE}_\text{Old}}} + \pi_\text{Young} \color{#0074D9}{\widehat{\text{CATE}_\text{Young}}}$

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

$\color{#2ECC40}{\widehat{\text{CATE}_\text{Old}}} = \frac{80 + 75 + 85}{3} - \frac{60}{1} = \color{#2ECC40}{20}$

$\color{#0074D9}{\widehat{\text{CATE}_\text{Young}}} = \frac{75}{1} - \frac{80 + 100 + 80}{3} = \color{#0074D9}{-11.667}$

$\color{#FF851B}{\widehat{\text{ATE}}} = (\frac{4}{8} \times \color{#2ECC40}{20}) + (\frac{4}{8} \times \color{#0074D9}{-11.667}) = \color{#FF851B}{4.1667}$

38 / 45

¡¡¡DON'T DO THIS!!!

$\color{#FF851B}{\widehat{\text{ATE}}} = \color{#F012BE}{\widehat{\text{CATE}_\text{Treated}}} - \color{#AAAAAA}{\widehat{\text{CATE}_\text{Untreated}}}$

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

$\color{#F012BE}{\widehat{\text{CATE}_\text{Treated}}} = \frac{80 + 75 + 85 + 75}{4} = \color{#F012BE}{78.75}$

$\color{#AAAAAA}{\widehat{\text{CATE}_\text{Untreated}}} = \frac{60 + 80 + 100 + 80}{4} = \color{#AAAAAA}{80}$

$\color{#FF851B}{\widehat{\text{ATE}}} = \color{#F012BE}{78.75} - \color{#AAAAAA}{80} = \color{#FF851B}{-1.25}$

You can only do this if treatment is random!

39 / 45

Matching and ATEs

$\widehat{\text{ATE}} = \pi_\text{Old} \widehat{\text{CATE}_\text{Old}} + \pi_\text{Young} \widehat{\text{CATE}_\text{Young}}$

We used age here because it correlates with (and confounds) the outcome

And we assumed unconfoundedness;
that treatment is
randomly assigned within the groups

40 / 45

Does attending a private university cause an increase in earnings?

41 / 45

This is tempting!

Average private − Average public

$\begin{aligned} \frac{110 + 100 + 60 + 115 + 75}{5} &= \color{#0074D9}{92} \\ \frac{110 + 30 + 90 + 60}{4} &= \color{#2ECC40}{72.5} \\ (\color{#0074D9}{92} \times \color{#7FDBFF}{\frac{5}{9}}) - (\color{#2ECC40}{72.5} \times \color{#01FF70}{\frac{4}{9}}) &= \color{#FF851B}{18,888} \end{aligned}$

This is wrong!

$\color{#FF851B}{\widehat{\text{ATE}}} = \color{#7FDBFF}{\pi_\text{Private}} \color{#0074D9}{\widehat{\text{CATE}_\text{Private}}} + \color{#01FF70}{\pi_\text{Public}} \color{#2ECC40}{\widehat{\text{CATE}_\text{Public}}}$

42 / 45

Grouping and matching

These groups look like they have similar characteristics

Unconfoundedness?

43 / 45

CATE Group A + CATE Group B

$\begin{aligned} \frac{110 + 100}{2} - 110 &= \color{#0074D9}{-5,000} \\ 60 - 30 &= \color{#2ECC40}{30,000} \\ (\color{#0074D9}{-5} \times \color{#7FDBFF}{\frac{3}{5}}) + (\color{#2ECC40}{30} \times \color{#01FF70}{\frac{2}{5}}) &= \color{#FF851B}{9,000} \end{aligned}$

This is less wrong!

$\color{#FF851B}{\widehat{\text{ATE}}} = \color{#7FDBFF}{\pi_\text{Group A}} \color{#0074D9}{\widehat{\text{CATE}_\text{Group A}}} + \color{#01FF70}{\pi_\text{Group B}} \color{#2ECC40}{\widehat{\text{CATE}_\text{Group B}}}$

44 / 45

Matching with regression

$\text{Earnings} = \alpha + \beta_1 \text{Private} + \beta_2 \text{Group} + \epsilon$

45 / 45

Matching with regression

$\text{Earnings} = \alpha + \beta_1 \text{Private} + \beta_2 \text{Group} + \epsilon$

model_earnings <- lm(earnings ~ private + group_A, data = schools_small)

45 / 45

Matching with regression

$\text{Earnings} = \alpha + \beta_1 \text{Private} + \beta_2 \text{Group} + \epsilon$

model_earnings <- lm(earnings ~ private + group_A, data = schools_small)

term	estimate	std.error	statistic	p.value
(Intercept)	40000	11952.29	3.35	0.08
privateTRUE	10000	13093.07	0.76	0.52
group_ATRUE	60000	13093.07	4.58	0.04

45 / 45

Matching with regression

$\text{Earnings} = \alpha + \beta_1 \text{Private} + \beta_2 \text{Group} + \epsilon$

model_earnings <- lm(earnings ~ private + group_A, data = schools_small)

term	estimate	std.error	statistic	p.value
(Intercept)	40000	11952.29	3.35	0.08
privateTRUE	10000	13093.07	0.76	0.52
group_ATRUE	60000	13093.07	4.58	0.04

β₁ = $10,000 This is less wrong! Significance details!

45 / 45

DAGs and
potential outcomes

Session 5

PMAP 8521: Program evaluation
Andrew Young School of Policy Studies

1 / 45

Plan for today2 / 45

Plan for today

do()ing observational
causal inference

2 / 45

Plan for today

do()ing observational
causal inference

Potential outcomes

2 / 45

do()ing observational
causal inference3 / 45

Structural models

The relationship between nodes can be described with equations

4 / 45

Structural models

dagify() in ggdag forces you to think this way

dagify(
  Earn ~ Edu + Year + Bkgd + Loc + JobCx,
  Edu ~ Req + Loc + Bkgd + Year,
  JobCx ~ Edu,
  Bkgd ~ U1,
  Loc ~ U1
)

5 / 45

Causal identification

All these nodes are related; there's correlation between them all

We care about
Edu → Earn, but what do we do about all the other nodes?

6 / 45

Causal identification

A causal effect is identified if the association between treatment and outcome is propertly stripped and isolated

7 / 45

Paths and associations

Arrows in a DAG transmit associations

You can redirect and control those paths by "adjusting" or "conditioning"

8 / 45

Three types of associations

Confounding

Common cause

Causation

Mediation

Collision

Selection /
endogeneity

9 / 45

Interventions

do-operator

Making an intervention in a DAG

$P[Y\ |\ do(X = x)] \quad \text{or} \quad E[Y\ |\ do(X = x)]$

10 / 45

Interventions

do-operator

Making an intervention in a DAG

$P[Y\ |\ do(X = x)] \quad \text{or} \quad E[Y\ |\ do(X = x)]$

P = probability distribution, or E = expectation/expected value

10 / 45

Interventions

do-operator

Making an intervention in a DAG

$P[Y\ |\ do(X = x)] \quad \text{or} \quad E[Y\ |\ do(X = x)]$

P = probability distribution, or E = expectation/expected value

Y = outcome, X = treatment;
x = specific value of treatment

10 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

11 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

E[ Firm growth | do(Government R&D funding)]

11 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

E[ Firm growth | do(Government R&D funding)]

E[ Air quality | do(Carbon tax)]

11 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

E[ Firm growth | do(Government R&D funding)]

E[ Air quality | do(Carbon tax)]

E[ Juvenile delinquency | do(Truancy program)]

11 / 45

Interventions

$E[Y\ |\ do(X = x)]$

E[ Earnings | do(One year of college)]

E[ Firm growth | do(Government R&D funding)]

E[ Air quality | do(Carbon tax)]

E[ Juvenile delinquency | do(Truancy program)]

E[ Malaria infection rate | do(Mosquito net)]

11 / 45

Interventions

When you do() X, delete all arrows into it

12 / 45

Interventions

When you do() X, delete all arrows into it

Observational DAG

12 / 45

Interventions

When you do() X, delete all arrows into it

Observational DAG

Experimental DAG

12 / 45

Interventions

$E[\text{Earnings}\ |\ do(\text{College education})]$

13 / 45

Interventions

$E[\text{Earnings}\ |\ do(\text{College education})]$

Observational DAG

13 / 45

Interventions

$E[\text{Earnings}\ |\ do(\text{College education})]$

Observational DAG

Experimental DAG

13 / 45

Undo()ing things

We want to know P[Y | do(X)]
but all we have is
observational data X, Y, and Z

14 / 45

Undo()ing things

We want to know P[Y | do(X)]
but all we have is
observational data X, Y, and Z

$P[Y\ |\ do(X)] \neq P(Y\ |\ X)$

14 / 45

Undo()ing things

We want to know P[Y | do(X)]
but all we have is
observational data X, Y, and Z

$P[Y\ |\ do(X)] \neq P(Y\ |\ X)$

Correlation isn't causation!

14 / 45

Undo()ing things

Our goal with observational data:
Rewrite P[Y | do(X)] so that it doesn't have a do() anymore (is "do-free")

15 / 45

do-calculus

A set of three rules that let you manipulate a DAG
in special ways to remove do() expressions

WAAAAAY beyond the score of this class!
Just know it exists and computer algorithms can do it for you!

16 / 45

https://arxiv.org/abs/1906.07125

Special cases of do-calculus

Backdoor adjustment

Frontdoor adjustment

17 / 45

Backdoor adjustment

$P[Y\ |\ do(X)] = \sum_Z P(Y\ |\ X, Z) \times P(Z)$

↑ That's complicated!

The right-hand side of the equation means "the effect of X on Y after adjusting for Z"

There's no do() on that side!

18 / 45

Frontdoor adjustment

S → T is d-separated; T → C is d-separated
combine the effects to find S → C

19 / 45

Moral of the story

If you can transform do() expressions to
do-free versions, you can legally make causal inferences from observational data

20 / 45

Moral of the story

If you can transform do() expressions to
do-free versions, you can legally make causal inferences from observational data

Backdoor adjustment is easiest to see +
dagitty and ggdag do this for you!

20 / 45

Moral of the story

If you can transform do() expressions to
do-free versions, you can legally make causal inferences from observational data

Backdoor adjustment is easiest to see +
dagitty and ggdag do this for you!

Fancy algorithms (found in the causaleffect package)
can do the official do-calculus for you too

20 / 45

Potential outcomes21 / 45

Program effect

22 / 45

Some equation translations

Causal effect = δ (delta)

$\delta = P[Y\ |\ do(X)]$

23 / 45

Some equation translations

Causal effect = δ (delta)

$\delta = P[Y\ |\ do(X)]$

$\delta = E[Y\ |\ do(X)] - E[Y\ |\ \hat{do}(X)]$

23 / 45

Some equation translations

Causal effect = δ (delta)

$\delta = P[Y\ |\ do(X)]$

$\delta = E[Y\ |\ do(X)] - E[Y\ |\ \hat{do}(X)]$

$\delta = (Y\ |\ X = 1) - (Y\ |\ X = 0)$

23 / 45

Some equation translations

Causal effect = δ (delta)

$\delta = P[Y\ |\ do(X)]$

$\delta = E[Y\ |\ do(X)] - E[Y\ |\ \hat{do}(X)]$

$\delta = (Y\ |\ X = 1) - (Y\ |\ X = 0)$

$\delta = Y_1 - Y_0$

23 / 45

24 / 45

https://www.thisamericanlife.org/691/gardens-of-branching-paths

Fundamental problem
of causal inference

$\delta_i = Y_i^1 - Y_i^0 \quad \text{in real life is} \quad \delta_i = Y_i^1 - ???$

Individual-level effects are impossible to observe!

There are no individual counterfactuals!

25 / 45

Average treatment effect (ATE)

Solution: Use averages instead

$\text{ATE} = E(Y_1 - Y_0) = E(Y_1) - E(Y_0)$

26 / 45

Average treatment effect (ATE)

Solution: Use averages instead

$\text{ATE} = E(Y_1 - Y_0) = E(Y_1) - E(Y_0)$

Difference between average/expected value when
program is on vs. expected value when program is off

$\delta = (\bar{Y}\ |\ P = 1) - (\bar{Y}\ |\ P = 0)$

26 / 45

Person
Age
Treated
Outcome
with program
Outcome
without program
Effect


1
Old
TRUE
80
60
20

2
Old
TRUE
75
70
5

3
Old
TRUE
85
80
5

4
Old
FALSE
70
60
10

5
Young
TRUE
75
70
5

6
Young
FALSE
80
80
0

7
Young
FALSE
90
100
-10

8
Young
FALSE
85
80
5


27 / 45

Person	Age	Treated	Outcome with program	Outcome without program	Effect
1	Old	TRUE	80	60	20
2	Old	TRUE	75	70	5
3	Old	TRUE	85	80	5
4	Old	FALSE	70	60	10
5	Young	TRUE	75	70	5
6	Young	FALSE	80	80	0
7	Young	FALSE	90	100	-10
8	Young	FALSE	85	80	5

$\delta = (\bar{Y}\ |\ P = 1) - (\bar{Y}\ |\ P = 0)$

$\text{ATE} = \frac{20 + 5 + 5 + 5 + 10 + 0 + -10 + 5}{8} = 5$

28 / 45

CATE

ATE in subgroups

29 / 45

CATE

ATE in subgroups

Is the program more
effective for specific age groups?

29 / 45

Person	Age	Treated	Outcome with program	Outcome without program	Effect
1	Old	TRUE	80	60	20
2	Old	TRUE	75	70	5
3	Old	TRUE	85	80	5
4	Old	FALSE	70	60	10
5	Young	TRUE	75	70	5
6	Young	FALSE	80	80	0
7	Young	FALSE	90	100	-10
8	Young	FALSE	85	80	5

$\delta = (\bar{Y}_\text{O}\ |\ P = 1) - (\bar{Y}_\text{O}\ |\ P = 0)$

$\delta = (\bar{Y}_\text{Y}\ |\ P = 1) - (\bar{Y}_\text{Y}\ |\ P = 0)$

$\text{CATE}_\text{Old} = \frac{20 + 5 + 5 + 10}{4} = 10$

$\text{CATE}_\text{Young} = \frac{5 + 0 - 10 + 5}{4} = 0$

30 / 45

ATT and ATU

Average treatment on the treated

ATT / TOT

Effect for those with treatment

31 / 45

ATT and ATU

Average treatment on the treated

ATT / TOT

Effect for those with treatment

Average treatment on the untreated

ATU / TUT

Effect for those without treatment

31 / 45

Person	Age	Treated	Outcome with program	Outcome without program	Effect
1	Old	TRUE	80	60	20
2	Old	TRUE	75	70	5
3	Old	TRUE	85	80	5
4	Old	FALSE	70	60	10
5	Young	TRUE	75	70	5
6	Young	FALSE	80	80	0
7	Young	FALSE	90	100	-10
8	Young	FALSE	85	80	5

$\delta = (\bar{Y}_\text{T}\ |\ P = 1) - (\bar{Y}_\text{T}\ |\ P = 0)$

$\delta = (\bar{Y}_\text{U}\ |\ P = 1) - (\bar{Y}_\text{U}\ |\ P = 0)$

$\text{CATE}_\text{Treated} = \frac{20 + 5 + 5 + 5}{4} = 8.75$

$\text{CATE}_\text{Untreated} = \frac{10 + 0 - 10 + 5}{4} = 1.25$

32 / 45

ATE, ATT, and ATU

The ATE is the weighted average
of the ATT and ATU

33 / 45

ATE, ATT, and ATU

The ATE is the weighted average
of the ATT and ATU

$\text{ATE} = (\pi_\text{Treated} \times \text{ATT}) + (\pi_\text{Untreated} \times \text{ATU})$

$(\frac{4}{8} \times 8.75) + (\frac{4}{8} \times 1.25)$

$4.375 + 0.625 = 5$

π here means "proportion," not 3.1415

33 / 45

Selection bias

ATE and ATT aren't always the same

ATE = ATT + Selection bias

$\begin{aligned} 5 &= 8.75 + x \\ x &= -3.75 \end{aligned}$

Randomization fixes this, makes x = 0

34 / 45

Actual data

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

Treatment not
randomly assigned

We can't see
unit-level causal effects

What do we do?!

35 / 45

Actual data

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

Treatment seems to be correlated with age

36 / 45

Actual data

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

We can estimate the ATE by finding the weighted average of age-based CATEs

As long as we assume/pretend treatment was randomly assigned within each age = unconfoundedness

$\widehat{\text{ATE}} = \pi_\text{Old} \widehat{\text{CATE}_\text{Old}} + \pi_\text{Young} \widehat{\text{CATE}_\text{Young}}$

37 / 45

Actual data

$\color{#FF851B}{\widehat{\text{ATE}}} = \pi_\text{Old} \color{#2ECC40}{\widehat{\text{CATE}_\text{Old}}} + \pi_\text{Young} \color{#0074D9}{\widehat{\text{CATE}_\text{Young}}}$

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

$\color{#2ECC40}{\widehat{\text{CATE}_\text{Old}}} = \frac{80 + 75 + 85}{3} - \frac{60}{1} = \color{#2ECC40}{20}$

$\color{#0074D9}{\widehat{\text{CATE}_\text{Young}}} = \frac{75}{1} - \frac{80 + 100 + 80}{3} = \color{#0074D9}{-11.667}$

$\color{#FF851B}{\widehat{\text{ATE}}} = (\frac{4}{8} \times \color{#2ECC40}{20}) + (\frac{4}{8} \times \color{#0074D9}{-11.667}) = \color{#FF851B}{4.1667}$

38 / 45

¡¡¡DON'T DO THIS!!!

$\color{#FF851B}{\widehat{\text{ATE}}} = \color{#F012BE}{\widehat{\text{CATE}_\text{Treated}}} - \color{#AAAAAA}{\widehat{\text{CATE}_\text{Untreated}}}$

Person	Age	Treated	Actual outcome
1	Old	TRUE	80
2	Old	TRUE	75
3	Old	TRUE	85
4	Old	FALSE	60
5	Young	TRUE	75
6	Young	FALSE	80
7	Young	FALSE	100
8	Young	FALSE	80

$\color{#F012BE}{\widehat{\text{CATE}_\text{Treated}}} = \frac{80 + 75 + 85 + 75}{4} = \color{#F012BE}{78.75}$

$\color{#AAAAAA}{\widehat{\text{CATE}_\text{Untreated}}} = \frac{60 + 80 + 100 + 80}{4} = \color{#AAAAAA}{80}$

$\color{#FF851B}{\widehat{\text{ATE}}} = \color{#F012BE}{78.75} - \color{#AAAAAA}{80} = \color{#FF851B}{-1.25}$

You can only do this if treatment is random!

39 / 45

Matching and ATEs

$\widehat{\text{ATE}} = \pi_\text{Old} \widehat{\text{CATE}_\text{Old}} + \pi_\text{Young} \widehat{\text{CATE}_\text{Young}}$

We used age here because it correlates with (and confounds) the outcome

And we assumed unconfoundedness;
that treatment is
randomly assigned within the groups

40 / 45

Does attending a private university cause an increase in earnings?

41 / 45

This is tempting!

Average private − Average public

This is wrong!

42 / 45

Grouping and matching

These groups look like they have similar characteristics

Unconfoundedness?

43 / 45

CATE Group A + CATE Group B

This is less wrong!

44 / 45

Matching with regression

$\text{Earnings} = \alpha + \beta_1 \text{Private} + \beta_2 \text{Group} + \epsilon$

45 / 45

Matching with regression

$\text{Earnings} = \alpha + \beta_1 \text{Private} + \beta_2 \text{Group} + \epsilon$

model_earnings <- lm(earnings ~ private + group_A, data = schools_small)

45 / 45

Matching with regression

$\text{Earnings} = \alpha + \beta_1 \text{Private} + \beta_2 \text{Group} + \epsilon$

model_earnings <- lm(earnings ~ private + group_A, data = schools_small)

term	estimate	std.error	statistic	p.value
(Intercept)	40000	11952.29	3.35	0.08
privateTRUE	10000	13093.07	0.76	0.52
group_ATRUE	60000	13093.07	4.58	0.04

45 / 45

Matching with regression

$\text{Earnings} = \alpha + \beta_1 \text{Private} + \beta_2 \text{Group} + \epsilon$

model_earnings <- lm(earnings ~ private + group_A, data = schools_small)

term	estimate	std.error	statistic	p.value
(Intercept)	40000	11952.29	3.35	0.08
privateTRUE	10000	13093.07	0.76	0.52
group_ATRUE	60000	13093.07	4.58	0.04

β₁ = $10,000 This is less wrong! Significance details!

45 / 45

↑, ←, Pg Up, k	Go to previous slide
↓, →, Pg Dn, Space, j	Go to next slide
Home	Go to first slide
End	Go to last slide
Number + Return	Go to specific slide
b / m / f	Toggle blackout / mirrored / fullscreen mode
c	Clone slideshow
p	Toggle presenter mode
t	Restart the presentation timer
?, h	Toggle this help
o	Tile View: Overview of Slides

DAGs andpotential outcomes

Plan for today

Plan for today

Plan for today

do()ing observationalcausal inference

Structural models

Structural models

Causal identification

Causal identification

Paths and associations

Three types of associations

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Interventions

Undo()ing things

Undo()ing things

Undo()ing things

Undo()ing things

do-calculus

Special cases of do-calculus

Backdoor adjustment

Frontdoor adjustment

Moral of the story

Moral of the story

Moral of the story

Potential outcomes

Program effect

Some equation translations

Some equation translations

Some equation translations

Some equation translations

Average treatment effect (ATE)

Average treatment effect (ATE)

CATE

CATE

ATT and ATU

ATT and ATU

ATE, ATT, and ATU

ATE, ATT, and ATU

Selection bias

Actual data

Actual data

Actual data

Actual data

¡¡¡DON'T DO THIS!!!

Matching and ATEs

Grouping and matching

Matching with regression

Matching with regression

Matching with regression

Matching with regression

Plan for today

Help

DAGs and potential outcomes

DAGs andpotential outcomes

Plan for today

Plan for today

Plan for today

do()ing observationalcausal inference

Structural models

Structural models

Causal identification

Causal identification

Paths and associations

Three types of associations

Interventions

Interventions

Interventions

Interventions

DAGs and
potential outcomes

do()ing observational
causal inference

DAGs and
potential outcomes

do()ing observational
causal inference