mverse

class: middle, left, title-slide

.title[
# mverse
]
.subtitle[
## An R library for Teaching and Conducting<br/>Multiverse Analysis
]
.author[
### Michael Jongho Moon
]
.institute[
### PhD Student, DoSS, University of Toronto
]
.date[
### SSC 2021 Annual Meeting, June 7, 2021
]

---

.nav-left[# Collaborators]

.main-right[
.third[
## .accent2[`mverse` Education Development]
.accent2[
Alison Gibbs, PhD  
Nathan Taback, PhD  
Fanny Chevalier, PhD

University of Toronto]
]
.third[
## .accent3[`mverse` Package Development]
.accent3[
Mingwei Xu  
Haoda Li

University of Toronto]
]
.third.accent1[
## .accent1[`multiverse` Package Development]
.accent1[
Abhraneel Sarma  
Matthew Kay, PhD

Northwestern University]
]]

---

layout:true

# Example: Hurricane Names and Gender-based Expectations

---

.nav-right[

### Do people underestimate the severity of a hurricane when it's named with a female name?

]

.main-right[.half[

]

.half[

```
## Warning: Removed 1 rows containing missing values (geom_rich_text).
```

]]

.footnote[
Original data (n=94) retrieved from Jung, K., Shavitt, S., Viswanathan, M., and Hilbe, J.M. (2014) _Female hurricanes are deadlier than male hurricanes._  
Proceedings of the National Academy of Sciences, 111(24), 8782-8787, https://doi.org/10.1073/pnas.1402786111
]

---

.nav-right[

### Do people underestimate the severity of a hurricane when it's named with a female name?

.selected[Did hurricanes with *feminine* names lead to __more fatalities given equal strength__ because their names didn't motivate as much preparedness as hurricanes with *masculine* names?]

]

.main-right[.half[

]

.half[

```
## Warning: Removed 1 rows containing missing values (geom_rich_text).
```

]]

---

.nav-right[

### Do people underestimate the severity of a hurricane when it's named with a female name?

Consider a data set with the following information:

.small[
+   Femininity rating on hurricane names (1:very masculine; 11: very feminine)
+   Total damage dollar amount normalized to 2013 USD
+   Total fatality counts
+   Maximum wind speed
+   Minimum pressure
+   Year
]
]

.main-right[.half[

]

.half[

```
## Warning: Removed 1 rows containing missing values (geom_rich_text).
```

]]

---

.nav-right[

Should we remove _Katrina_ or _Katrina_ and _Audrey_ from data based on the distribution of fatalities?

]

.main-right[

]

---

.half[

### Michael

<b>1.</b> Remove _Katrina_ and _Audrey_.

]

.half[

### Mary

<b>1.</b> Remove _Katrina_ only.

]

<hr/>

<b>2.</b> Fit a regression model with _Femininity_, _Damage_, and their interaction.

`$$\text{Fatalities}_i\sim \text{Pois}\left(\mu_i\right)$$`
`$$\log\left(\mu_i\right) = \beta_0 + \beta_1\text{Damage}_i + \beta_2\text{Femininity}_i + \beta_3\text{Damage}_i\times\text{Femininity}_i$$`

<hr/>

<b>3.</b> Make an inference based on the interaction effect `$\beta_3$`.

---

.nav-right[

**Michael**: stronger hurricanes with a feminine name are under estimated

]

.main-right[

]

---

.nav-right[

**Michael**: stronger hurricanes with a feminine name are under estimated

**Mary**: there is no evidence of gender-based expectations for hurricanes

]

.main-right[

]

---

.nav-right[

**Michael**: stronger hurricanes with a feminine name are under estimated

**Mary**: there is no evidence of gender-based expectations for hurricanes

# .selected[_Which result should they report?_]

]

.main-right[

]

---
layout:false

.nav-left[

# Multiverse Analysis (Steegen et al. 2016)<sup>1</sup>

+   Performing and communicating all alternative analyses based on reasonable alternative paths
+   Improves **transparency** and **robustness** of the reported findings

]

.footnote[
1. Steegen, S., Tuerlinckx, F., Gelman, A., and Vanpaemel, W. (2016) _Increasing transparency through a multiverse analysis._ Perspectives on Psychological Science, 11(5), 702-712,  https://doi.org/10.1177/1745691616658637
]

## Example: A Multiverse of Hurricane Name Analaysis

<table>
 <thead>
  <tr>
   <th style="text-align:left;">  </th>
  </tr>
 </thead>
<tbody>
  <tr grouplength="3"><td colspan="1" style="border-bottom: 1px solid;"><strong>Which hurricanes should we analyse?</strong></td></tr>
<tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> &gt; Exclude 2 outliers with most fatalities </td>
  </tr>
  <tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> &gt; Exclude 1 outlier with most fatalities </td>
  </tr>
  <tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> &gt; Exclude 2 outliers with most fatalities, 2 with greatest damages </td>
  </tr>
  <tr grouplength="3"><td colspan="1" style="border-bottom: 1px solid;"><strong>Which variable to use to capture hurricane strengths?</strong></td></tr>
<tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> &gt; Normalized damage amount </td>
  </tr>
  <tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> &gt; Maximum wind speed </td>
  </tr>
  <tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> &gt; Minimum pressure </td>
  </tr>
  <tr grouplength="2"><td colspan="1" style="border-bottom: 1px solid;"><strong>Which effect should the inference be based on?</strong></td></tr>
<tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> &gt; Femininity x hurricane strength interaction </td>
  </tr>
  <tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> &gt; Femininity main effect </td>
  </tr>
</tbody>
</table>

---

.nav-left[

# Multiverse Analysis (Steegen et al. 2016)<sup>1</sup>

+   Performing and communicating all alternative analyses based on reasonable alternative paths
+   Improves **transparency** and **robustness** of the reported findings

]

]
---

.nav-left[

# Multiverse Analysis (Steegen et al. 2016)<sup>1</sup>

+   Performing and communicating all alternative analyses based on reasonable alternative paths
+   Improves **transparency** and **robustness** of the reported findings

]

.nav-right[

+   Jung et al. (2014)<sup>2</sup> reported a statistically significant result

+   Simonsohn et al. (2020)<sup>3</sup> considered 1,728 reasonable paths with the same data set

]

2. Jung, K., Shavitt, S., Viswanathan, M., and Hilbe, J.M. (2014) _Female hurricanes are deadlier than male hurricanes._ Proceedings of the National Academy of Sciences, 111(24), 8782-8787, https://doi.org/10.1073/pnas.1402786111

3. Simonsohn, U., Simmons, J.P., and Nelson, L.D. (2020) _Specification curve analysis._ Nature Human Behavior, 4 1208-1214, https://doi.org/10.1038/s41562-020-0912-z
]

---

.nav-left[

# Multiverse Analysis (Steegen et al. 2016)<sup>1</sup>

+   Performing and communicating all alternative analyses based on reasonable alternative paths
+   Improves **transparency** and **robustness** of the reported findings

]

.nav-right[

+   Jung et al. (2014)<sup>2</sup> reported a statistically significant result

+   Simonsohn et al. (2020)<sup>3</sup> considered 1,728 reasonable paths with the same data set

# _Multiverse analysis can get very complex and difficult to manage._
]

3. Simonsohn, U., Simmons, J.P., and Nelson, L.D. (2020) _Specification curve analysis._ Nature Human Behavior, 4 1208-1214, https://doi.org/10.1038/s41562-020-0912-z
]

---
layout:false
class:middle

# .selected[`mverse` aims to help students and analysts  
create, explore, and examine multiverse with ease]

.third[
## .hidden[Systematic Syntax]
]
.third[
## .hidden[Familiar Syntax]
]
.third[
## .hidden[Interactive Learning Modules]
]

---
class:middle

# .block[`mverse` aims to help students and analysts  
create, explore, and examine multiverse with ease]

.third[
## .accent1[Systematic Syntax]
]
.third[
## .accent3[Familiar Syntax]
]
.third[
## .accent2[Interactive Learning Modules]
]

---
.nav-left[

# .accent1[Systematic Syntax]

## Without `mverse`

+   Redundant codes
+   Prone to errors
]

.main-right.half[

```r
# universe 1 (Michael) & 2
data_1 <- hdata %>%
  filter(! name %in% c("Katrina", "Audrey"))
fit_1 <- glm(
  deaths ~ damage * femininity, 
  data = data_1, family = "poisson")
# extract interaction
est_1 <- summary(fit_1)$coef[4,] 
# extract main effect
est_2 <- summary(fit_1)$coef[3,]

# universe 3 & 4
fit_2 <- glm(
  deaths ~ max_wind_speed * femininity, 
  data = data_1, family = "poisson")
# extract interaction
est_3 <- summary(fit_2)$coef[4,] 
# extract main effect
est_4 <- summary(fit_2)$coef[3,]

# universe 5 & 6
fit_3 <- glm(
  deaths ~ min_pressure * femininity, 
  data = data_1, family = "poisson")
# extract interaction
est_5 <- summary(fit_3)$coef[4,] 
# extract main effect
est_6 <- summary(fit_3)$coef[3,]

# universe 7 (Mary) & 8
data_2 <- hdata %>%
  filter(! name %in% c("Katrina"))
fit_4 <- glm(
  deaths ~ damage * femininity, 
  data = data_2, family = "poisson")
# extract interaction
est_7 <- summary(fit_4)$coef[4,] 
# extract main effect
est_8 <- summary(fit_4)$coef[3,]

# universe 9 & 10
fit_5 <- glm(
  deaths ~ max_wind_speed * femininity, 
  data = data_2, family = "poisson")
# extract interaction
est_9 <- summary(fit_5)$coef[4,] 
# extract main effect
est_10 <- summary(fit_5)$coef[3,]

# universe 11 & 12
fit_6 <- glm(
  deaths ~ min_pressure * femininity, 
  data = data_2, family = "poisson")
# extract interaction
est_11 <- summary(fit_6)$coef[4,] 
# extract main effect
est_12 <- summary(fit_6)$coef[3,]

# universe 13 & 14
data_3 <- hdata %>%
  filter(! name %in% c("Katrina", "Audrey", "Andrew"))
fit_7 <- glm(
  deaths ~ damage * femininity, 
  data = data_3, family = "poisson")
# extract interaction
est_13 <- summary(fit_7)$coef[4,] 
# extract main effect
est_14 <- summary(fit_7)$coef[3,]

# universe 15 & 16
fit_8 <- glm(
  deaths ~ max_wind_speed * femininity, 
  data = data_3, family = "poisson")
# extract interaction
est_15 <- summary(fit_8)$coef[4,] 
# extract main effect
est_16 <- summary(fit_8)$coef[3,]

# universe 17 & 18
fit_9 <- glm(
  deaths ~ min_pressure * femininity, 
  data = data_3, family = "poisson")
# extract interaction
est_17 <- summary(fit_9)$coef[4,] 
# extract main effect
est_18 <- summary(fit_9)$coef[3,]
```
]

---
.nav-left[

# Systematic Syntax

## With `mverse`

.accent1[
+   **Avoids redundant codes**
+   **Less prone to errors**
+   **Easier to reproduce**
]]

.main-right.half[

```r
library(mverse)
# create decision points, or branches
hurricane_outliers <- filter_branch(
  ! name %in% c("Katrina", "Audrey"),
  ! name %in% c("Katrina"),
  ! name %in% c("Katrina", "Audrey", "Andrew")
)
hurricane_strength <- mutate_branch(
  damage, max_wind_speed, min_pressure
)
model <- formula_branch(
  deaths ~ hurricane_strength * femininity
)
distribution <- family_branch(
  poisson
)
# create a mverse object add branches
mv <- create_multiverse(hdata) %>%
  add_filter_branch(hurricane_outliers) %>%
  add_mutate_branch(hurricane_strength) %>%
  add_formula_branch(model) %>%
  add_family_branch(distribution) %>%
  glm_mverse() # fit a glm model
```
]

---
.nav-left[

# .accent3[Familiar Syntax]

]

.main-right.half[

```r
library(mverse)
# create decision points, or branches
hurricane_outliers <- filter_branch( 
  ! name %in% c("Katrina", "Audrey"),
  ! name %in% c("Katrina"),
  ! name %in% c("Katrina", "Audrey", "Andrew")
)
hurricane_strength <- mutate_branch( 
  damage, max_wind_speed, min_pressure
)
model <- formula_branch( 
  deaths ~ hurricane_strength * femininity
)
distribution <- family_branch( 
  poisson
)
# create a mverse object add branches
mv <- create_multiverse(hdata) %>%
  add_filter_branch(hurricane_outliers) %>% 
  add_mutate_branch(hurricane_strength) %>% 
  add_formula_branch(model) %>%
  add_family_branch(distribution) %>%
  glm_mverse() # fit a glm model
```
]

---

.nav-left[

# Familiar Syntax

+   .accent3[**Familiar verbs from `dplyr`**]

]

.main-right.half[

```r
library(mverse)
# create decision points, or branches
*hurricane_outliers <- filter_branch(
  ! name %in% c("Katrina", "Audrey"),
  ! name %in% c("Katrina"),
  ! name %in% c("Katrina", "Audrey", "Andrew")
)
*hurricane_strength <- mutate_branch(
  damage, max_wind_speed, min_pressure
)
model <- formula_branch( 
  deaths ~ hurricane_strength * femininity
)
distribution <- family_branch( 
  poisson
)
# create a mverse object add branches
mv <- create_multiverse(hdata) %>%
* add_filter_branch(hurricane_outliers) %>%
* add_mutate_branch(hurricane_strength) %>%
  add_formula_branch(model) %>%
  add_family_branch(distribution) %>%
  glm_mverse() # fit a glm model
```
]

---

.nav-left[

# Familiar Syntax

+   Familiar verbs from `dplyr`
+   .accent3[**Familiar syntax from base R**]

]

.main-right.half[

```r
library(mverse)
# create decision points, or branches
hurricane_outliers <- filter_branch( 
  ! name %in% c("Katrina", "Audrey"),
  ! name %in% c("Katrina"),
  ! name %in% c("Katrina", "Audrey", "Andrew")
)
hurricane_strength <- mutate_branch( 
  damage, max_wind_speed, min_pressure
)
*model <- formula_branch(
  deaths ~ hurricane_strength * femininity
)
*distribution <- family_branch(
  poisson
)
# create a mverse object add branches
mv <- create_multiverse(hdata) %>%
  add_filter_branch(hurricane_outliers) %>% 
  add_mutate_branch(hurricane_strength) %>% 
* add_formula_branch(model) %>%
* add_family_branch(distribution) %>%
* glm_mverse() # fit a glm model
```
]

---

.nav-left[

# Familiar Syntax

+   Familiar verbs from `dplyr`
+   Familiar syntax from base R
+   .accent3[**Familiar `tibble` format for summary of results**]

]

.main-right[

```r
summary(mv, conf.int = TRUE)
```
]
.tiny.two-third[

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> universe </th>
   <th style="text-align:left;"> hurricane_outliers_branch </th>
   <th style="text-align:left;"> hurricane_strength_branch </th>
   <th style="text-align:left;"> model_branch </th>
   <th style="text-align:left;"> distribution_branch </th>
   <th style="text-align:left;"> term </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> std.error </th>
   <th style="text-align:right;"> statistic </th>
   <th style="text-align:right;"> p.value </th>
   <th style="text-align:right;"> conf.low </th>
   <th style="text-align:right;"> conf.high </th>
   <th style="text-align:left;"> hurricane_outliers_branch_code </th>
   <th style="text-align:left;"> hurricane_strength_branch_code </th>
   <th style="text-align:left;"> model_branch_code </th>
   <th style="text-align:left;"> distribution_branch_code </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> 2.1333019 </td>
   <td style="text-align:right;"> 0.0804493 </td>
   <td style="text-align:right;"> 26.5173408 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 1.9731343 </td>
   <td style="text-align:right;"> 2.2885686 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> 0.0000302 </td>
   <td style="text-align:right;"> 0.0000026 </td>
   <td style="text-align:right;"> 11.4818671 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0000249 </td>
   <td style="text-align:right;"> 0.0000352 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0622634 </td>
   <td style="text-align:right;"> 0.0100554 </td>
   <td style="text-align:right;"> 6.1920656 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0427265 </td>
   <td style="text-align:right;"> 0.0821530 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 1 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0000008 </td>
   <td style="text-align:right;"> 0.0000003 </td>
   <td style="text-align:right;"> 2.4901342 </td>
   <td style="text-align:right;"> 0.0127695 </td>
   <td style="text-align:right;"> 0.0000002 </td>
   <td style="text-align:right;"> 0.0000014 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 2 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> -0.0125683 </td>
   <td style="text-align:right;"> 0.2716360 </td>
   <td style="text-align:right;"> -0.0462690 </td>
   <td style="text-align:right;"> 0.9630959 </td>
   <td style="text-align:right;"> -0.5471019 </td>
   <td style="text-align:right;"> 0.5179669 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 2 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> 0.0228428 </td>
   <td style="text-align:right;"> 0.0022515 </td>
   <td style="text-align:right;"> 10.1454268 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0183988 </td>
   <td style="text-align:right;"> 0.0272273 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 2 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0956374 </td>
   <td style="text-align:right;"> 0.0334948 </td>
   <td style="text-align:right;"> 2.8552928 </td>
   <td style="text-align:right;"> 0.0042997 </td>
   <td style="text-align:right;"> 0.0301345 </td>
   <td style="text-align:right;"> 0.1614503 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 2 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> -0.0002048 </td>
   <td style="text-align:right;"> 0.0002779 </td>
   <td style="text-align:right;"> -0.7369456 </td>
   <td style="text-align:right;"> 0.4611554 </td>
   <td style="text-align:right;"> -0.0007473 </td>
   <td style="text-align:right;"> 0.0003422 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 3 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> 46.9184165 </td>
   <td style="text-align:right;"> 3.2702981 </td>
   <td style="text-align:right;"> 14.3468316 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 40.5367120 </td>
   <td style="text-align:right;"> 53.3565656 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 3 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> -0.0463697 </td>
   <td style="text-align:right;"> 0.0034424 </td>
   <td style="text-align:right;"> -13.4701282 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> -0.0531522 </td>
   <td style="text-align:right;"> -0.0396577 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 3 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> -1.0239489 </td>
   <td style="text-align:right;"> 0.3973512 </td>
   <td style="text-align:right;"> -2.5769366 </td>
   <td style="text-align:right;"> 0.0099680 </td>
   <td style="text-align:right;"> -1.8050132 </td>
   <td style="text-align:right;"> -0.2474276 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 3 </td>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0011448 </td>
   <td style="text-align:right;"> 0.0004183 </td>
   <td style="text-align:right;"> 2.7370483 </td>
   <td style="text-align:right;"> 0.0061993 </td>
   <td style="text-align:right;"> 0.0003279 </td>
   <td style="text-align:right;"> 0.0019674 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 4 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> 2.1333019 </td>
   <td style="text-align:right;"> 0.0804493 </td>
   <td style="text-align:right;"> 26.5173408 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 1.9731343 </td>
   <td style="text-align:right;"> 2.2885686 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 4 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> 0.0000302 </td>
   <td style="text-align:right;"> 0.0000026 </td>
   <td style="text-align:right;"> 11.4818671 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0000249 </td>
   <td style="text-align:right;"> 0.0000352 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 4 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0622634 </td>
   <td style="text-align:right;"> 0.0100554 </td>
   <td style="text-align:right;"> 6.1920656 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0427265 </td>
   <td style="text-align:right;"> 0.0821530 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 4 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0000008 </td>
   <td style="text-align:right;"> 0.0000003 </td>
   <td style="text-align:right;"> 2.4901342 </td>
   <td style="text-align:right;"> 0.0127695 </td>
   <td style="text-align:right;"> 0.0000002 </td>
   <td style="text-align:right;"> 0.0000014 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 5 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> -0.0859063 </td>
   <td style="text-align:right;"> 0.2648374 </td>
   <td style="text-align:right;"> -0.3243737 </td>
   <td style="text-align:right;"> 0.7456551 </td>
   <td style="text-align:right;"> -0.6073774 </td>
   <td style="text-align:right;"> 0.4310230 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 5 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> 0.0234898 </td>
   <td style="text-align:right;"> 0.0021657 </td>
   <td style="text-align:right;"> 10.8464892 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0192195 </td>
   <td style="text-align:right;"> 0.0277112 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 5 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0530890 </td>
   <td style="text-align:right;"> 0.0319783 </td>
   <td style="text-align:right;"> 1.6601573 </td>
   <td style="text-align:right;"> 0.0968828 </td>
   <td style="text-align:right;"> -0.0094170 </td>
   <td style="text-align:right;"> 0.1159550 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 5 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0003322 </td>
   <td style="text-align:right;"> 0.0002601 </td>
   <td style="text-align:right;"> 1.2771749 </td>
   <td style="text-align:right;"> 0.2015405 </td>
   <td style="text-align:right;"> -0.0001756 </td>
   <td style="text-align:right;"> 0.0008444 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 6 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> 47.3960640 </td>
   <td style="text-align:right;"> 3.1698280 </td>
   <td style="text-align:right;"> 14.9522510 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 41.2094730 </td>
   <td style="text-align:right;"> 53.6353868 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 6 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> -0.0468620 </td>
   <td style="text-align:right;"> 0.0033376 </td>
   <td style="text-align:right;"> -14.0407023 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> -0.0534367 </td>
   <td style="text-align:right;"> -0.0403532 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 6 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> -0.8451817 </td>
   <td style="text-align:right;"> 0.3796521 </td>
   <td style="text-align:right;"> -2.2262008 </td>
   <td style="text-align:right;"> 0.0260007 </td>
   <td style="text-align:right;"> -1.5914463 </td>
   <td style="text-align:right;"> -0.1032312 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 6 </td>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0009801 </td>
   <td style="text-align:right;"> 0.0003998 </td>
   <td style="text-align:right;"> 2.4516637 </td>
   <td style="text-align:right;"> 0.0142197 </td>
   <td style="text-align:right;"> 0.0001993 </td>
   <td style="text-align:right;"> 0.0017664 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 7 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> 1.7453185 </td>
   <td style="text-align:right;"> 0.1042185 </td>
   <td style="text-align:right;"> 16.7467234 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 1.5368317 </td>
   <td style="text-align:right;"> 1.9455123 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 7 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> 0.0000800 </td>
   <td style="text-align:right;"> 0.0000073 </td>
   <td style="text-align:right;"> 10.9180044 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0000656 </td>
   <td style="text-align:right;"> 0.0000943 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 7 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.1049076 </td>
   <td style="text-align:right;"> 0.0123934 </td>
   <td style="text-align:right;"> 8.4647991 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0809661 </td>
   <td style="text-align:right;"> 0.1295657 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 7 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> -0.0000047 </td>
   <td style="text-align:right;"> 0.0000008 </td>
   <td style="text-align:right;"> -5.7418828 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> -0.0000063 </td>
   <td style="text-align:right;"> -0.0000031 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> damage </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 8 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> 0.3185825 </td>
   <td style="text-align:right;"> 0.3541973 </td>
   <td style="text-align:right;"> 0.8994491 </td>
   <td style="text-align:right;"> 0.3684135 </td>
   <td style="text-align:right;"> -0.3808318 </td>
   <td style="text-align:right;"> 1.0079617 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 8 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> 0.0195837 </td>
   <td style="text-align:right;"> 0.0031959 </td>
   <td style="text-align:right;"> 6.1278367 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 0.0133076 </td>
   <td style="text-align:right;"> 0.0258388 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 8 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0593501 </td>
   <td style="text-align:right;"> 0.0417436 </td>
   <td style="text-align:right;"> 1.4217787 </td>
   <td style="text-align:right;"> 0.1550905 </td>
   <td style="text-align:right;"> -0.0220195 </td>
   <td style="text-align:right;"> 0.1416428 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 8 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0001531 </td>
   <td style="text-align:right;"> 0.0003732 </td>
   <td style="text-align:right;"> 0.4103570 </td>
   <td style="text-align:right;"> 0.6815441 </td>
   <td style="text-align:right;"> -0.0005776 </td>
   <td style="text-align:right;"> 0.0008855 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> max_wind_speed </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 9 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:right;"> 50.3727835 </td>
   <td style="text-align:right;"> 4.3576710 </td>
   <td style="text-align:right;"> 11.5595655 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> 41.9548313 </td>
   <td style="text-align:right;"> 59.0422262 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 9 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength </td>
   <td style="text-align:right;"> -0.0499623 </td>
   <td style="text-align:right;"> 0.0045657 </td>
   <td style="text-align:right;"> -10.9429999 </td>
   <td style="text-align:right;"> 0.0000000 </td>
   <td style="text-align:right;"> -0.0590519 </td>
   <td style="text-align:right;"> -0.0411489 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 9 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> -1.4019880 </td>
   <td style="text-align:right;"> 0.5071268 </td>
   <td style="text-align:right;"> -2.7645708 </td>
   <td style="text-align:right;"> 0.0056998 </td>
   <td style="text-align:right;"> -2.4080914 </td>
   <td style="text-align:right;"> -0.4196809 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 9 </td>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> model_1 </td>
   <td style="text-align:left;"> distribution_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0015379 </td>
   <td style="text-align:right;"> 0.0005316 </td>
   <td style="text-align:right;"> 2.8930034 </td>
   <td style="text-align:right;"> 0.0038158 </td>
   <td style="text-align:right;"> 0.0005088 </td>
   <td style="text-align:right;"> 0.0025932 </td>
   <td style="text-align:left;"> !name %in% c(&quot;Katrina&quot;, &quot;Audrey&quot;, &quot;Andrew&quot;) </td>
   <td style="text-align:left;"> min_pressure </td>
   <td style="text-align:left;"> deaths ~ hurricane_strength * femininity </td>
   <td style="text-align:left;"> poisson </td>
  </tr>
</tbody>
</table>

]

---
.nav-left[

# Familiar Syntax

+   Familiar verbs from `dplyr`
+   Familiar syntax from base R
+   .accent3[**Familiar `tibble` format for summary of results**]

]

.main-right[

```r
summary(mv, conf.int = TRUE)  %>%
  filter(term %in% c("hurricane_strength:femininity", "femininity")) %>%
  select(hurricane_outliers_branch, hurricane_strength_branch, 
         term, estimate, conf.low, conf.high)
```
]
.tiny.two-third[

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> hurricane_outliers_branch </th>
   <th style="text-align:left;"> hurricane_strength_branch </th>
   <th style="text-align:left;"> term </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> conf.low </th>
   <th style="text-align:right;"> conf.high </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0622634 </td>
   <td style="text-align:right;"> 0.0427265 </td>
   <td style="text-align:right;"> 0.0821530 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0000008 </td>
   <td style="text-align:right;"> 0.0000002 </td>
   <td style="text-align:right;"> 0.0000014 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0956374 </td>
   <td style="text-align:right;"> 0.0301345 </td>
   <td style="text-align:right;"> 0.1614503 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> -0.0002048 </td>
   <td style="text-align:right;"> -0.0007473 </td>
   <td style="text-align:right;"> 0.0003422 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> -1.0239489 </td>
   <td style="text-align:right;"> -1.8050132 </td>
   <td style="text-align:right;"> -0.2474276 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_1 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0011448 </td>
   <td style="text-align:right;"> 0.0003279 </td>
   <td style="text-align:right;"> 0.0019674 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0622634 </td>
   <td style="text-align:right;"> 0.0427265 </td>
   <td style="text-align:right;"> 0.0821530 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0000008 </td>
   <td style="text-align:right;"> 0.0000002 </td>
   <td style="text-align:right;"> 0.0000014 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0530890 </td>
   <td style="text-align:right;"> -0.0094170 </td>
   <td style="text-align:right;"> 0.1159550 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0003322 </td>
   <td style="text-align:right;"> -0.0001756 </td>
   <td style="text-align:right;"> 0.0008444 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> -0.8451817 </td>
   <td style="text-align:right;"> -1.5914463 </td>
   <td style="text-align:right;"> -0.1032312 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_2 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0009801 </td>
   <td style="text-align:right;"> 0.0001993 </td>
   <td style="text-align:right;"> 0.0017664 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.1049076 </td>
   <td style="text-align:right;"> 0.0809661 </td>
   <td style="text-align:right;"> 0.1295657 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_1 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> -0.0000047 </td>
   <td style="text-align:right;"> -0.0000063 </td>
   <td style="text-align:right;"> -0.0000031 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> 0.0593501 </td>
   <td style="text-align:right;"> -0.0220195 </td>
   <td style="text-align:right;"> 0.1416428 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_2 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0001531 </td>
   <td style="text-align:right;"> -0.0005776 </td>
   <td style="text-align:right;"> 0.0008855 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> femininity </td>
   <td style="text-align:right;"> -1.4019880 </td>
   <td style="text-align:right;"> -2.4080914 </td>
   <td style="text-align:right;"> -0.4196809 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> hurricane_outliers_3 </td>
   <td style="text-align:left;"> hurricane_strength_3 </td>
   <td style="text-align:left;"> hurricane_strength:femininity </td>
   <td style="text-align:right;"> 0.0015379 </td>
   <td style="text-align:right;"> 0.0005088 </td>
   <td style="text-align:right;"> 0.0025932 </td>
  </tr>
</tbody>
</table>

]

---

.nav-left[

# Familiar Syntax

+   Familiar verbs from `dplyr`
+   Familiar syntax from base R
+   .accent3[**Familiar `tibble` format for summary of results**]

]

.main-right[

]

---
layout:false
class: dark, full
.nav-left[
# .accent2[Interactive Learning Modules]

+   Interactive modules to guide mutliverse analysis education
+   They do not require knowledge of coding in R

_The demo is a prototype and development is in progress_

]

.main-right[
<iframe src="https://micbon.shinyapps.io/mverse-ssc21-demo/" title="Interactive Presentation" style="border:none;width:90%;height:100%;"></iframe>
]

---
class:middle

.block[
.third[
## .accent1[Systematic Syntax]
]
.third[
## .accent3[Familiar Syntax]
]
.third[
## .accent2[Interactive Learning Modules]
]
]

# .selected[`mverse` can be used to introduce the idea of  
_multiverse_ to students at any level of their study]

---
layout:false
.nav-left[
# .selected[Development]
]
.main-right[
.timeline[
.item[
### In Development

Interactive learning modules are being developed

Visualizations for exploring and reporting multiverse analysis are also in development
]
.item[
### GitHub
      
`remote::install_github('mverseanalysis/mverse')` for the latest stable version

Currently supports multiverse analysis with `t.test`, `lm`, and `glm`
]
.item[
### CRAN

Plan to submit the first version in summer 2021 (no interactive learning modules)
]]]
---
class: full, middle, center

# Thank you
## .selected[
michael jongho moon  
June 7, 2021
]