Meta-analysis .med[– with examples]

class: center, middle, inverse, title-slide

# Meta-analysis <br> .med[– with examples]
### Chaochen Wang (CWAN)
### 2020-05-29 via hangout 15:00~16:30

---

class: middle
# Contents

- Introduction to meta-analysis 
- The fixed-effect model 
- The random-effect model 
- Forest plots
- Assessing heterogeneity
- Bias and small study effects
- Exploring heterogeneity

---
class: middle

# Systematic Review (1-5) and Meta-Analysis (6)

1. Formulate question: population, outcome, treatment
1. Define search strategy
1. Perform literature search
1. Select studies that extract results <br> (from authors and papers)
1. Report results of selected studies 
1. Meta-analyze the results if possible/needed

--
*Not every systematic review results in a meta-analysis.*

---
class: middle

# What is a meta-analysis?

- "A statistical analysis that combines or integrates the results of several independent clinical trials considered by the analyst to be 'combinable'". <sup>[1]</sup>
- Meta: Greek word meaning *beyond* or *after*.
- Two-stage approach (analysis of analyses)
    - Stage 1: compute individual studies results
    - Stage 2: produce overall summary (weighted)
- Assessment of whether results can be combined.

.small[[1] M. Huque, “Experiences with meta-analysis in NDA submissions,” in Proceedings of the Biopharmaceutical Section of the American Statistical Association, 1988, vol. 2, no. 1, pp. 28–33.]

---
class: middle

## Weighted Average

.pull-left[
Unweighted:
<style type="text/css">
.tg  {border-collapse:collapse;border-color:#bbb;border-spacing:0;}
.tg td{background-color:#E0FFEB;border-color:#bbb;border-style:solid;border-width:1px;color:#594F4F;
  font-family:Arial, sans-serif;font-size:14px;overflow:hidden;padding:10px 5px;word-break:normal;}
.tg th{background-color:#9DE0AD;border-color:#bbb;border-style:solid;border-width:1px;color:#493F3F;
  font-family:Arial, sans-serif;font-size:14px;font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}
.tg .tg-6ibf{border-color:inherit;font-size:24px;text-align:center;vertical-align:top}
</style>
<table class="tg">
<thead>
  <tr>
    <th class="tg-6ibf"> X</th>
  </tr>
</thead>
<tbody>
  <tr>
    <td class="tg-6ibf">5</td>
  </tr>
  <tr>
    <td class="tg-6ibf">10</td>
  </tr>
  <tr>
    <td class="tg-6ibf">15</td>
  </tr>
  <tr>
    <td class="tg-6ibf">20</td>
  </tr>
</tbody>
</table>

.small[$$\frac{5 + 10 + 15 + 20}{4} = 12.5$$]
]

.pull-right[
Weighted:
<style type="text/css">
.tg  {border-collapse:collapse;border-color:#bbb;border-spacing:0;}
.tg td{background-color:#E0FFEB;border-color:#bbb;border-style:solid;border-width:1px;color:#594F4F;
  font-family:Arial, sans-serif;font-size:14px;overflow:hidden;padding:10px 5px;word-break:normal;}
.tg th{background-color:#9DE0AD;border-color:#bbb;border-style:solid;border-width:1px;color:#493F3F;
  font-family:Arial, sans-serif;font-size:14px;font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}
.tg .tg-hjji{font-size:24px;text-align:center;vertical-align:top}
</style>
<table class="tg">
<thead>
  <tr>
    <th class="tg-hjji">Freq<br></th>
    <th class="tg-hjji">X</th>
  </tr>
</thead>
<tbody>
  <tr>
    <td class="tg-hjji">40</td>
    <td class="tg-hjji">5</td>
  </tr>
  <tr>
    <td class="tg-hjji">30</td>
    <td class="tg-hjji">10</td>
  </tr>
  <tr>
    <td class="tg-hjji">20</td>
    <td class="tg-hjji">15</td>
  </tr>
  <tr>
    <td class="tg-hjji">10</td>
    <td class="tg-hjji">20</td>
  </tr>
</tbody>
</table>

.small[
$$
`\begin{aligned}
& \frac{40 \times 5 + 30 \times 10 + 20 \times 15 + 10 \times 20}{100}\\
& = 10 \\
& = \frac{\sum w_iX_i}{\sum w_i}
\end{aligned}`
$$
]
]

---
class: middle

# Potential Advantages of  <br> Meta-Analysis

- Increase in statistical power
- Increase in precision of estimates
- More robust results 
- Resolve conflicting results 
- Enable answering of questions not addressed by individual studies e.g. subgroups.
- Results more generalisable

---
class: middle

# Potential Pitfalls with <br> Meta-Analysis

- Quality of systematic review
- Quality of included trials
    - design, analysis and reporting quality vary
    - good meta-analysis still **cannot correct** poor design or conduct in individual studies
- Small study effects
    - results from smaller trials often differ from larger ones
- Variation between studies: what is 'combinable'?

---
class: middle

# Two main analytic approaches

.pull-left[Fixed effect:
- Assumes each study is estimating the same effect<br>(singular "effect")
- Source of variation<br> within-study sampling error
]
.pull-right[
Random effects:
- Assumes studies are estimating different (but similar) effects
- Effects follow a distribution - estimate mean of distribution
- Sources of variation<br>within- and between- studies
]

---
class: middle

# Controversies

- Very different views on how to do meta-analysis

- (Almost) always use fixed-effect

- (Almost) always use random-effects

- Test for heterogeneity - if found then random-effects otherwise use fixed-effect

- Use knowledge of systematic review/studies for choice

---
class: middle

# The fixed-effect model

- Assumes each study is measuring a common true effect `$\theta$`

$$
`\begin{aligned}
\hat{\theta}_i & = \theta_F + \epsilon_i \\
\end{aligned}`
$$

Where,

- `$\hat{\theta}_i$` is the estimate for the `$i$`th study
- `$\epsilon_i$` is the within study sampling error

$$
\epsilon_i \sim N(0, v_i)
$$

---
class: top, center
background-image: url("./fig/Fixedeffect.jpeg")
background-position: 50% 100%
background-size: contain

---
class: middle

## Fixed-effect pooled estimate

- Weighted average:

$$
\hat{\theta}_F = \frac{\sum w_i \hat\theta_i}{\sum w_i}
$$

for `$i = 1, \dots, k$` where `$k$` is the number of studies.

---
class: middle

## Weights for fixed-effect model

- Inverse variance:  `$w_i = \frac{1}{v_i}$` <br> Where `$v_i$` is the error variance of the `$i$`th study

- Mantel-Haenszel method (for 2 `$\times$` 2 count data) `$w_i = \frac{d_{0i} h_{1i}}{n_i}$` 
<br> where, 
<br> `$d_{0i}$` is `$N$` with event in control group 
<br> `$h_{1i}$` is `$N$` without event in intervention group
<br> `$n_i$` is number of participants in study

- [Peto's weight](https://handbook-5-1.cochrane.org/chapter_9/9_4_4_2_peto_odds_ratio_method.htm) (for odds ratios, click to see details)

---
class: middle

## Inference for fixed-effect model

$$
\hat\theta_F \sim N(\theta_F, \frac{1}{\sum w_i})
$$

- Under null hypothesis, `$\theta = 0$`

$$
Z = \frac{\hat\theta_F}{SE(\hat\theta_F)} \sim N(0, 1)
$$

- 95% confidence interval for `$\theta$`

$$
\hat\theta_F \pm 1.96 \times SE(\hat\theta_F)
$$

---
class: middle

# The random-effects model

- Assumes individual studies are measuring different (but related) true effects which follow some distribution with mean `$\theta_R$`

$$
\hat\theta_i = \theta_R + u_i + \epsilon_i
$$

Where, 
<br> `$\hat\theta_i$` = the estimated effect for the `$i$`th study.
<br> `$u_i$` = the study specific random variable: `$u_i \sim N(0, \tau^2)$`
<br> `$\epsilon_i$` = the within study sampling error `$\epsilon_i \sim N(0, v_i)$`
<br> `$\theta_R$` = the average treatment effect

---
class: top, center
background-image: url("./fig/Randomeffects.jpeg")
background-position: 50% 100%
background-size: contain

---
class: middle

## Random-Effects Pooled Estimate

$$
\hat\theta_{R} = \frac{\sum w_i \hat\theta_i}{\sum w_i}
$$

for `$i = 1, \dots, k$` where `$k$` is the number of studies.

## Weights for Random-Effects Model -- Inverse variance:

$$
w_i = \frac{1}{v_i + \tau^2}
$$

Where, `$v_i$` is the error variance of the `$i$`th study <br> `$\tau^2$` is the between study variance.

---
class: middle

## Inference for Random-effects estimate

$$
\hat\theta_R \sim N(\theta_R, \frac{1}{\sum w_i})
$$

- Under null hypothesis `$\theta_R =0$`

$$
Z = \frac{\hat\theta_R}{SE(\hat\theta_R)} \sim N(0, 1)
$$

- 95% confidence interval for `$\theta_R$`

$$
\hat\theta_R \pm 1.96 \times SE(\hat\theta_R)
$$

---
class: middle

## Estimation of `$\tau^2$`

- DerSimonian and Laird method (weighted method of moments) <sup>[1]</sup>

$$
\hat{\tau}^2 = \frac{Q -df}{C}
$$

Where,

`$Q = \sum w_i(\hat\theta_i - \hat\theta_F)^2, \; \; df = k-1$` 
<br> `$C = \sum w_i - \frac{\sum w_i^2}{\sum w_i}$`; `$k$` is the number of studies

.small[ [1] DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Control Clin Trials, 7(3), 177-188. doi:10.1016/0197-2456(86)90046-2
]

---
class: middle

## Example 1: trials with counts

- 10 trials with binary outcome (yes/no)

---
class: middle

# Meta-Analysis in R

.tiny[

```r
meta::metabin(yes_int, yes_int + no_int, 
              yes_con, yes_con + no_con, sm = "RR", data = dat1, studlab = Studynum)
```

```
##              RR           95%-CI %W(fixed) %W(random)
## Study 1  0.9000 [0.3840; 2.1095]       2.5        3.5
## Study 2  0.7658 [0.5864; 1.0001]      27.9       22.4
## Study 3  0.6623 [0.4737; 0.9262]      19.3       16.8
## Study 4  1.0000 [0.4162; 2.4025]       2.3        3.3
## Study 5  1.8750 [0.8368; 4.2013]       2.0        3.9
## Study 6  0.6667 [0.2848; 1.5607]       3.0        3.5
## Study 7  0.6212 [0.4293; 0.8989]      16.6       14.6
## Study 8  1.1250 [0.6681; 1.8945]       6.0        8.5
## Study 9  0.7250 [0.4621; 1.1375]      10.1       10.8
## Study 10 1.0244 [0.6816; 1.5395]      10.3       12.6
## 
## Number of studies combined: k = 10
## 
##                          RR           95%-CI     z p-value
## Fixed effect model   0.7940 [0.6904; 0.9130] -3.24  0.0012
## Random effects model 0.8075 [0.6845; 0.9527] -2.54  0.0112
## 
## Quantifying heterogeneity:
##  tau^2 = 0.0131; tau = 0.1146; I^2 = 19.2% [0.0%; 59.9%]; H = 1.11 [1.00; 1.58]
## 
## Test of heterogeneity:
##      Q d.f. p-value
##  11.13    9  0.2666
## 
## Details on meta-analytical method:
## - Mantel-Haenszel method
## - DerSimonian-Laird estimator for tau^2
## - Mantel-Haenszel estimator used in calculation of Q and tau^2 (like RevMan 5)
```
]

---
class: middle

## Forest Plots

- Displays treatment effects and CIs for each study
- Box areas proportional to the weight for each study
- Pooled effect estimate and its CI - often a diamond
- Usually also include: 
    - study identifier (name/date)
    - statistics: summary statistics, effect estimate
    - relative weighting given to each study
    - measure of heterogeneity
    
---
class: middle

## Plot command in R

.small[

```r
m1.mh <- meta::metabin(yes_int, yes_int + no_int, 
              yes_con, yes_con + no_con, sm = "RR", 
              data = dat1, studlab = Studynum)
meta::forest(m1.mh, comb.random = FALSE, 
             text.fixed = "MH estimate")
```
]

---
class: middle

## Interpreting Forest Plots

- Look for consistency:
  - magnitude and direction of estimates
  - confidence intervals <br> -- how much do they overlap?
  - consistency between larger and smaller studies
- Look for unusual patterns: 
  - outlying estimates
  - estimates from smaller studies <br> -- do they tend to be more positive?
  - do the event rates differ markedly?
  - any time trends depend on the dates of trials?

---
class: middle

## Assessing Heterogeneity

### Cochran's Q statistic

$$
Q = \sum w_i (\hat\theta_i - \hat\theta_F)^2
$$

Where, `$\hat\theta_i$` is the estimated effect for the `$i$`th study <br>
`$\hat\theta_F$` is the fixed-effect pooled estimate <br>
`$w_i$` is the weights from fixed-effect model

Under `$H_0$`, `$Q$` follows a `$\chi^2$` distribution with `$k-1$` degrees of freedom ( `$k=$` number of studies)

--
.small[
- Test has low power when studies are small or `$k$` is small
- Test has high power to detect a small degree of heterogeneity when `$k$` is large or included very large studies
]

---
class: middle

### `$I^2$`

- `$I^2$` attempts to quantify **between study heterogeneity**

- Describes the percentage of the variability in effect estimates that is due to heterogeneity rather than sampling error.

- Takes values from 0 to 100% with increasing values indicating greater heterogeneity

$$
I^2 = \frac{Q - df}{Q} \times 100 \%
$$

---
class: middle

### Interpretation of `$I^2$` (1)

- Higgins et al (2003)<sup>[1]</sup> suggest a rough guide:

> A naive categorisation of values for `$I^2$` would not be appropriate for all circumstances, although we would tentatively assign adjective of low, moderate, and high to `$I^2$` values of 25%, 50%, and 75%.

.small[
[1] Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327(7414), 557-560. doi:10.1136/bmj.327.7414.557
]

---
class: middle

### Interpretation of `$I^2$` (2)

- Cochran Collaboration guide:

- The importance of the observed value of `$I^2$` depends on the magnitude and direction of effects and the strength of evidence for heterogeneity.

---
class: middle

### Example 1: <br>(output from the previous command)

.med[
```
## Quantifying heterogeneity:
##  tau^2 = 0.0131; tau = 0.1146; 
##  I^2 = 19.2% [0.0%; 59.9%]; H = 1.11 [1.00; 1.58]
## Test of heterogeneity:
##      Q d.f. p-value
##  11.13    9  0.2666
```
]

- `$Q = 11.13$` compare to a `$\chi^2$` distribution with 9 degrees of freedom
- p = 0.267
- No evidence of heterogeneity
- `$I^2 = 19.2\%$` might not be important

---
class: middle

### Fixed-effect vs. Random-effects (1)

- The summary estimate from random-effects model can be smaller, the same as, or larger than the summary estimate from the fixed-effect model.

- Therefore, although the CI from the random-effects model is almost always wider than that from the fixed-effect model, the result is not necessarily more conservative i.e. the p value may be larger or smaller.

- **If the pooled estimates from the two models are very different, then it is important to investigate why.**

---
class: middle

### Fixed-effect vs. Random-effects (2)

- Random-effects models will almost always have wider confidence invervals (as long as `$\tau^2 >0$`)

.pull-left[
Fixed-Effect model
`$$\hat\theta_F \sim N(\theta_F, \frac{1}{\sum w_i}) \\
w_i = \frac{1}{v_i}$$`
]

.pull-right[
Random-Effects model
`$$\hat\theta_R \sim N(\theta_R, \frac{1}{\sum w_i}) \\
w_i = \frac{1}{v_i + \tau^2}$$`
]

.med[
```
##                          RR           95%-CI     z p-value
## Fixed effect model   0.7940 [0.6904; 0.9130] -3.24  0.0012
## Random effects model 0.8075 [0.6845; 0.9527] -2.54  0.0112
```
]

---
class: middle

### Fixed-effect vs. Random-effects (3)

- Random-effects model gives greater weight to smaller studies. 
- The larger `$\tau^2$` the more weight is given to smaller studies. 
- Smaller studies are more influential in random-effects models - **can accentuate bias**.

.small[
```
##               %W(fixed) %W(random)
##study n = 180       2.5        3.5
##study n = 1800     27.9       22.4
##study n = 1200     19.3       16.8
##study n = 180       2.3        3.3
##study n = 180       2.0        3.9
##study n = 200       3.0        3.5
##study n = 1000     16.6       14.6
##study n = 500       6.0        8.5
##study n = 600      10.1       10.8
##study n = 800      10.3       12.6
```
]

---
class: middle

## Bias is a key issue in Meta-Analysis

- Often referred to as small-study effects. <br> Smaller studies tend to report different **(often larger)** treatment effects. 
- Within study bias
    - Clinical heterogeneity - smaller studies recruit patients more likely to respond to treatment 
    - Methodological quality
    - Reporting bias - selective reporting
- Selection bias 
    - Publication bias
    - Positive results more likely to be more visible
    
    
---
class: middle

### Funnel Plots

- Graphical method for investigating small study effects
- Plots a measure of effect size against precision for each study
- If no bias then a triangular (funnel) shaped symmetric distribution is expected 
    - Studies should produce estimates that are normally distributed around the truth (symmetrically)
    - larger studies with more precise estimates should be scattered closely around the truth
    - smaller studies with less precise estimates should be scattered more widely

---
class: top, center
background-image: url("./fig/FunnelPlot.png")
background-position: 50% 50%
background-size: contain

---
class: top, center
background-image: url("./fig/Funnelplottruth.png")
background-position: 50% 50%
background-size: contain

---
class: top, center
background-image: url("./fig/Funnelplotbiased.png")
background-position: 50% 50%
background-size: contain

---
class: middle

### Funnel plots in R

.small[

```r
m1.mh <- meta::metabin(yes_int, yes_int + no_int, 
              yes_con, yes_con + no_con, sm = "RR", 
              data = dat1, studlab = Studynum)
meta::funnel(m1.mh)
```

![](index_files/figure-html/unnamed-chunk-5-1.png)
]

---
class: middle
## Formal tests

- Assessment of bias using a Funnel plot is subjective
- Formal statistical tests for funnel plot asymmetry include:
    - Egger's, Harbord, Peter's test - regression based
    - Begg's test - non-parametric test based
    - etc.
- All flawed and sometimes no consensus. 
- Sensitivity analyses can be carried out to assess robustness of the combined result.

---
class: middle

## Tests for small study effect <br> Egger's test

.med[

```r
meta::metabias(m1.mh)
```

```
## 
## 	Linear regression test of funnel plot asymmetry
## 
## data:  m1.mh
## t = 1.6119, df = 8, p-value = 0.1457
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
##       bias    se.bias  intercept 
##  1.3650302  0.8468671 -0.5169690
```
]

---
class: middle

## Tests for small study effect <br> Begg's test

.med[

```r
meta::metabias(m1.mh, method = "rank")
```

```
## 
## 	Rank correlation test of funnel plot asymmetry
## 
## data:  m1.mh
## z = 1.3416, p-value = 0.1797
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
##       ks    se.ks 
## 15.00000 11.18034
```
]

---
class: middle

## Tests for small study effect <br> Thompson and Sharp

.med[

```r
meta::metabias(m1.mh, method = "mm")
```

```
## 
## 	Linear regression test of funnel plot asymmetry (methods of moment)
## 
## data:  m1.mh
## t = 1.6212, df = 8, p-value = 0.1436
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
##       bias    se.bias  intercept 
##  1.3690309  0.8444507 -0.5179303
```

]

---
class: middle

## Tests for small study effect <br> Harbord's test

.med[

```r
meta::metabias(m1.mh, method = "score")
```

```
## 
## 	Linear regression test of funnel plot asymmetry (efficient score)
## 
## data:  m1.mh
## t = 1.6503, df = 8, p-value = 0.1375
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
##       bias    se.bias  intercept 
##  1.4065959  0.8523048 -0.5199974
```
]

---
class: middle

## Tests for small study effect <br> Peter's test

.med[

```r
meta::metabias(m1.mh, method = "peters")
```

```
## 
## 	Linear regression test of funnel plot asymmetry (based on sample size)
## 
## data:  m1.mh
## t = 1.729, df = 8, p-value = 0.1221
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
##       bias    se.bias  intercept 
## 86.6618744 50.1232713 -0.3631411
```
]

---
class: middle

## What if there is significant heterogeneity?

- If fixed-effect and random-effects models produce similar point estimates then account for the heterogeneity using the random-effects model -- wider CIs reflect the uncertainty.

- If fixed-effect and random-effects models produce different estimates then it is important to explore causes of heterogeneity. 
    - publication bias
    - subgroup - stratified meta-analysis
    - meta-regression

---
class: middle

## Example 2: 11 trials of BCG vaccine to prevent TB

.small[
<table class="table table-striped table-hover table-condensed" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> trialnam </th>
   <th style="text-align:right;"> latitude </th>
   <th style="text-align:right;"> cases1 </th>
   <th style="text-align:right;"> pop1 </th>
   <th style="text-align:right;"> cases0 </th>
   <th style="text-align:right;"> pop0 </th>
   <th style="text-align:right;"> logrr </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Northern USA </td>
   <td style="text-align:right;"> 52 </td>
   <td style="text-align:right;"> 4 </td>
   <td style="text-align:right;"> 123 </td>
   <td style="text-align:right;"> 11 </td>
   <td style="text-align:right;"> 139 </td>
   <td style="text-align:right;"> -0.8893113 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Canada </td>
   <td style="text-align:right;"> 55 </td>
   <td style="text-align:right;"> 6 </td>
   <td style="text-align:right;"> 306 </td>
   <td style="text-align:right;"> 29 </td>
   <td style="text-align:right;"> 303 </td>
   <td style="text-align:right;"> -1.5853887 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> UK </td>
   <td style="text-align:right;"> 53 </td>
   <td style="text-align:right;"> 62 </td>
   <td style="text-align:right;"> 13598 </td>
   <td style="text-align:right;"> 248 </td>
   <td style="text-align:right;"> 12867 </td>
   <td style="text-align:right;"> -1.4415512 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Madanapalle </td>
   <td style="text-align:right;"> 13 </td>
   <td style="text-align:right;"> 33 </td>
   <td style="text-align:right;"> 5069 </td>
   <td style="text-align:right;"> 47 </td>
   <td style="text-align:right;"> 5808 </td>
   <td style="text-align:right;"> -0.2175473 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Haiti </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:right;"> 2545 </td>
   <td style="text-align:right;"> 10 </td>
   <td style="text-align:right;"> 629 </td>
   <td style="text-align:right;"> -1.6208982 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Madras </td>
   <td style="text-align:right;"> 13 </td>
   <td style="text-align:right;"> 505 </td>
   <td style="text-align:right;"> 88391 </td>
   <td style="text-align:right;"> 499 </td>
   <td style="text-align:right;"> 88391 </td>
   <td style="text-align:right;"> 0.0119523 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South Africa </td>
   <td style="text-align:right;"> 27 </td>
   <td style="text-align:right;"> 29 </td>
   <td style="text-align:right;"> 7499 </td>
   <td style="text-align:right;"> 45 </td>
   <td style="text-align:right;"> 7277 </td>
   <td style="text-align:right;"> -0.4694177 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Chicago </td>
   <td style="text-align:right;"> 42 </td>
   <td style="text-align:right;"> 17 </td>
   <td style="text-align:right;"> 1716 </td>
   <td style="text-align:right;"> 65 </td>
   <td style="text-align:right;"> 1665 </td>
   <td style="text-align:right;"> -1.3713448 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Puerto Rico </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 186 </td>
   <td style="text-align:right;"> 50634 </td>
   <td style="text-align:right;"> 141 </td>
   <td style="text-align:right;"> 27338 </td>
   <td style="text-align:right;"> -0.3393588 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Georgia (Sch) </td>
   <td style="text-align:right;"> 33 </td>
   <td style="text-align:right;"> 5 </td>
   <td style="text-align:right;"> 2498 </td>
   <td style="text-align:right;"> 3 </td>
   <td style="text-align:right;"> 2341 </td>
   <td style="text-align:right;"> 0.4459134 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Georgia (Comm) </td>
   <td style="text-align:right;"> 33 </td>
   <td style="text-align:right;"> 27 </td>
   <td style="text-align:right;"> 16913 </td>
   <td style="text-align:right;"> 29 </td>
   <td style="text-align:right;"> 17854 </td>
   <td style="text-align:right;"> -0.0173139 </td>
  </tr>
</tbody>
</table>
]

---
class: middle

## Example 2

.small[

```r
m2.mh <- meta::metabin(cases1, pop1, 
                       cases0, pop0, sm = "RR", 
              data = bcgtrial, studlab = trialnam)
m2.mh
```
]

.tiny[
```
                   RR           95%-CI %W(fixed) %W(random)
Northern USA   0.4109 [0.1343; 1.2574]       0.9        6.3
Canada         0.2049 [0.0863; 0.4864]       2.5        7.7
UK             0.2366 [0.1793; 0.3121]      21.6       11.0
Madanapalle    0.8045 [0.5163; 1.2536]       3.7       10.3
Haiti          0.1977 [0.0784; 0.4989]       1.4        7.4
Madras         1.0120 [0.8946; 1.1449]      42.3       11.5
South Africa   0.6254 [0.3926; 0.9962]       3.9       10.1
Chicago        0.2538 [0.1494; 0.4310]       5.6        9.8
Puerto Rico    0.7122 [0.5725; 0.8860]      15.5       11.3
Georgia (Sch)  1.5619 [0.3737; 6.5284]       0.3        4.9
Georgia (Comm) 0.9828 [0.5821; 1.6593]       2.4        9.8

Number of studies combined: k = 11

RR           95%-CI     z  p-value
Fixed effect model   0.6972 [0.6384; 0.7614] -8.02 < 0.0001
Random effects model 0.5079 [0.3356; 0.7689] -3.20   0.0014

* Quantifying heterogeneity:
*  tau^2 = 0.3851; tau = 0.6206; I^2 = 92.1% [87.9%; 94.9%]; H = 3.56 [2.87; 4.41]

Test of heterogeneity:
      Q d.f.  p-value
 126.63   10 < 0.0001

Details on meta-analytical method:
- Mantel-Haenszel method
- DerSimonian-Laird estimator for tau^2
- Mantel-Haenszel estimator used in calculation of Q and tau^2 (like RevMan 5)
```
]

---
class: middle

## Example 2 - Forest plot

.small[

```r
meta::forest(m2.mh, comb.fixed = FALSE, 
             text.random = "Overall estimate")
```
]

---
class: middle

## Exploring heterogeneity - stratification

- Subgroup method
- Depends on some element of study design: 
    - randomised vs. observation, 
    - country,etc.
- Do the study effect differ in the subgroups?

---
class: middle

## Example 2 <br>- stratified by latitude < 40

.small[

```r
m3.mh <- update(m2.mh, `byvar = latitude <40`)
m3.mh
```
]

.small[
```
Results for subgroups (fixed effect model):
                k     RR           95%-CI     Q   I^2
*byvar = FALSE   4 0.2421 [0.1922; 0.3050]  1.06  0.0%
*byvar =  TRUE   7 0.8974 [0.8132; 0.9904] 21.45 72.0%

Test for subgroup differences (fixed effect model):
                      Q d.f.  p-value
Between groups   104.60    1 < 0.0001

Results for subgroups (random effects model):
                k     RR           95%-CI  tau^2    tau
byvar = FALSE   4 0.2430 [0.1928; 0.3061]      0      0
byvar =  TRUE   7 0.7644 [0.5824; 1.0034] 0.0741 0.2722

Test for subgroup differences (random effects model):
                     Q d.f.  p-value
Between groups   39.60    1 < 0.0001
```
]

---
class: top, left
background-image: url("./fig/ForestEx2str.png")
background-position: 50% 100%
background-size: contain

## Forest plot by subgroup

---
class: middle

## Meta-regression

- Regression based method

- Dependent variable is the effect size in the individual studies e.g. log risk ratio

- Explanatory variables are study level covariates, e.g. country, mean age

- To what extent can between-study variation (heterogeneity) in effects be explained by study level covariates?

---
class: middle

## Example 2 - meta-regression on the latitude

.between[

```r
m4.mh <- meta::`metareg(m2.mh, latitude)`; m4.mh
```
]

.between[
```

Mixed-Effects Model (k = 11; tau^2 estimator: DL)

tau^2 (estimated amount of residual heterogeneity):     0.0652 (SE = 0.0654)
tau (square root of estimated tau^2 value):             0.2553
I^2 (residual heterogeneity / unaccounted variability): `61.80%`
H^2 (unaccounted variability / sampling variability):   2.62
R^2 (amount of heterogeneity accounted for):            `82.93%`

Test for Residual Heterogeneity:
QE(df = 9) = 23.5614, p-val = 0.0051

Test of Moderators (coefficient 2):
QM(df = 1) = 17.7319, p-val < .0001

Model Results:

estimate      se     zval    pval    ci.lb    ci.ub 
intrcpt     0.2539  0.2333   1.0883  0.2765  -0.2034   0.7112      
*latitude   -0.0298  0.0071  -4.2109  <.0001  -0.0436  -0.0159  ***

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
```
]

???
- Latitude significantly associated with log risk ratio
- Regression estimate is -0.0298 (-0.0436, -0.0159), suggesting higher latitude associated with greater treatment effect.
- Latitude explains 82.93% of between study variance
- Percentage of residual variation due to between study heterogeneity is 61.8%

---
class: middle

## Example 2 - bubble graph

.between[

```r
m4.mh <-  meta::metareg(m2.mh, 
                        latitude)
meta::bubble(m4.mh)
```

![](index_files/figure-html/unnamed-chunk-17-1.png)
]

.between[
```
* Bigger treatment effect in higher latitudes.
```
]

---
class: middle

# Summary

- Meta-analysis is a two-stage process in which results from >1 study are combined to produce a summary estimate
- Two main models used to produce a summary estimate
    - Fixed-effect model assumes a single common effect
    - Random-effects model assumes studies estimating different (but related) effects
- Assessment of small-study effects
- Importance of assessing and (if present) investigating heterogeneity

---
class: middle

# Advanced topics

- Adjusting for small-study effects
- Individual patient data meta-analysis
- Multivariate meta-analysis
- Network meta-analysis
- etc.

---
class: top, center, inverse
background-image: url("./fig/Meta-Analysis-with-R.png")
background-position: 50% 50%
background-size: contain

---
class: middle, center, inverse

# Thanks