mverse: How the R package is designed to help students explore the multiverse

CANSSI Ontario Statistical Software Conference

Michael Jongho Moon

PhD Student, DoSS, University of Toronto

November 10, 2022

Multiverse
analysis

Example

Are darker-skinned
soccer players
more likely
to receive a red card?

Data1

  • Card counts received by 2,053 soccer players who played in the first male divisions of England, France, and Spain in the 2012-2013 season throughout their careers
  • Contains skin tone information along with other player information
  • Which attributes are important to take into account as covariates?

1Silberzahn, R., Uhlmann, E.L., …, & Nosek, B.A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 1(3), 337-356. https://doi.org/10.1177/2515245917747646

Position

As-is

Fielder vs. goalkeeper

Fielder groups

Position

Which groupings should we use?

Size

Should we use height, weight, or both to represent size?

League country

Should we take the league country into account?

  • Reporting results from a single analysis doesn’t reveal the full scholarship involved in the analysis
  • Multiverse analysis2

    involves performing and reporting all analyses across the whole set of reasonable alternatives
  • There are at least 29 unique analytic approaches with the same data set1
  • A multiverse analysis would consider all reasonable combinations of alternatives for each analytic decision

2Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016).
Increasing Transparency Through a Multiverse Analysis.
Perspectives on Psychological Science, 11(5), 702–712.
https://doi.org/10.1177/1745691616658637

1Silberzahn, R., Uhlmann, E.L., …, & Nosek, B.A. (2018).
Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results.
Advances in Methods and Practices in Psychological Science, 1(3), 337-356.
https://doi.org/10.1177/2515245917747646

3Figure by Richie King, Aug 19, 2015, Science Sin’t Broken.
FiveThirtyEight. https://fivethirtyeight.com/features/science-isnt-broken/

Exploring a multiverse
is complex and difficult,
especially for students
with limited technical competency

mverse

An R package for tidy multiverse analysis

Collaborators

Haoda Li,
Mingwei Xu,
Nathan Taback, PhD, &
Fanny Chevalier, PhD

How we approached the design

  • Systematic syntax
  • Familiar syntax
  • Interactive exploration tool

Some of the features presented are in the experimental stage.

Systematic syntax

Multiverse definition to execution

Without mverse

library(tidyverse)
.
.
# data preparation steps
.
.
# create position alternatives
soccer <- mutate(
  soccer,
  # fielder vs goalkeeper
  position_fld_vs_gk = if_else(position == "Goalkeeper",
                               "Goalkeeper", "Fielder"),
  # grouping into 4 groups
  position_group = forcats::fct_collapse(
    position,
    Forward = c("Center Forward", "Right Winger", "Left Winger"),
    Midfielder = c("Defensive Midfielder", "Attacking Midfielder",
                   "Right Midfielder", "Center Midfielder", 
                   "Left Midfielder"),
    Defender = c("Center Back", "Right Fullback", "Left Fullback"),
    Goalkeeper = c("Goalkeeper")
  )
)

# specify variable inclusion and exclusion
model_1 <- formula(redCards ~ skintone)
model_2 <- formula(redCards ~ skintone + position)
model_3 <- formula(redCards ~ skintone + position_fld_vs_gk)
model_4 <- formula(redCards ~ skintone + position_group)

model_5 <- formula(redCards ~ skintone + 
                     weight)
model_6 <- formula(redCards ~ skintone + position + 
                     weight)
model_7 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                     weight)
model_8 <- formula(redCards ~ skintone + position_group + 
                     weight)

model_9 <- formula(redCards ~ skintone + 
                     height)
model_10 <- formula(redCards ~ skintone + position + 
                      height)
model_11 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      height)
model_12 <- formula(redCards ~ skintone + position_group + 
                      height)

model_13 <- formula(redCards ~ skintone + weight + 
                      height)
model_14 <- formula(redCards ~ skintone + position + 
                      weight + height)
model_15 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      weight + height)
model_16 <- formula(redCards ~ skintone + position_group + 
                      weight + height)

model_17 <- formula(redCards ~ skintone + 
                      leagueCountry)
model_18 <- formula(redCards ~ skintone + position + 
                      leagueCountry)
model_19 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      leagueCountry)
model_20 <- formula(redCards ~ skintone + position_group + 
                      leagueCountry)

model_21 <- formula(redCards ~ skintone + 
                      weight + leagueCountry)
model_22 <- formula(redCards ~ skintone + position + 
                      weight + leagueCountry)
model_23 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      weight + leagueCountry)
model_24 <- formula(redCards ~ skintone + position_group + 
                      weight + leagueCountry)

model_25 <- formula(redCards ~ skintone + 
                      height + leagueCountry)
model_26 <- formula(redCards ~ skintone + position + 
                      height + leagueCountry)
model_27 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      height + leagueCountry)
model_28 <- formula(redCards ~ skintone + position_group + 
                      height + leagueCountry)

model_29 <- formula(redCards ~ skintone + 
                      weight + height + leagueCountry)
model_30 <- formula(redCards ~ skintone + position + 
                      weight + height + leagueCountry)
model_31 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      weight + height + leagueCountry)
model_32 <- formula(redCards ~ skintone + position_group + 
                      weight + height + leagueCountry)

# bring the multiverse together and fit
models <- list(
  model_1, model_2, model_3, model_4,
  model_5, model_6, model_7, model_8,
  model_9, model_10, model_11, model_12,
  model_13, model_14, model_15, model_16,
  model_17, model_18, model_19, model_20,
  model_21, model_22, model_23, model_24,
  model_25, model_26, model_27, model_28,
  model_29, model_30, model_31, model_32
)
multiverse_fit <- lapply(
  models, glm, family = binomial, data = soccer)

Systematic syntax

Designed around analytical decision points

With mverse

library(mverse)
.
.
# data preparation steps
.
.
# create position alternatives
position_alternative <- mutate_branch(
  as_is = position, # use position as is
  # fielder vs goalkeeper
  position_fld_vs_gk = if_else(position == "Goalkeeper",
                               "Goalkeeper", "Fielder"),
  # grouping into 4 groups
  `Fielder Grouping` = forcats::fct_collapse(
    position,
    Forward = c("Center Forward", "Right Winger", "Left Winger"),
    Midfielder = c("Defensive Midfielder", "Attacking Midfielder",
                   "Right Midfielder", "Center Midfielder", 
                   "Left Midfielder"),
    Defender = c("Center Back", "Right Fullback", "Left Fullback"),
    Goalkeeper = c("Goalkeeper")
  )
)

# create size alternatives
size <- mutate_branch(height, weight)

# specify variable inclusion and exclusion
frmls <- formula_branch(
  `No Covariate` = redCards ~ skintone,
  `Position` = redCards ~ skintone + position_alternative,
  `Position + Size` = redCards ~ skintone + position_alternative + size,
  `Position + Weight + Height` = redCards ~ skintone + position_alternative + weight + height,
  `Size` = redCards ~ skintone + size,
  `Weight + Height` = redCards ~ skintone + weight + height,
  `Country` = redCards ~ skintone + leagueCountry,
  `Position + Country` = redCards ~ skintone + position_alternative + leagueCountry,
  `Position + Size + Country` = redCards ~ skintone + position_alternative + size + leagueCountry,
  `Position + Weight + Height + Country` = redCards ~ skintone + position_alternative + weight + height + leagueCountry,
  `Size + Country` = redCards ~ skintone + size + leagueCountry,
  `Weight + Height + Country` = redCards ~ skintone + weight + height + leagueCountry,
)
# remove unnecessary branch combinations
condition_position_1 <- branch_condition(
  redCards ~ skintone, position)
condition_position_2 <- branch_condition(
  redCards ~ skintone + size, position)
condition_position_3 <- branch_condition(
  redCards ~ skintone + size + leagueCountry, position)
condition_position_4 <- branch_condition(
  redCards ~ skintone + leagueCountry, position)
condition_position_5 <- branch_condition(
  redCards ~ skintone + weight + height, position)
condition_position_6 <- branch_condition(
  redCards ~ skintone + weight + height + leagueCountry, position)

# bring the multiverse together and fit
mv <- mverse(soccer) %>%
  add_mutate_branch(position_alternative, size) %>%
  add_formula_branch(frmls) %>%
  add_family_branch(
    family_branch(logistic = binomial, name = "fam")
  ) %>%
  add_branch_condition(
    condition_position_1, condition_position_2, condition_position_3,
    condition_position_4, condition_position_5, condition_position_6
  )
glm_mverse(mv)

Systematic syntax

Requires a fewer lines of codes

With mverse

.
.
.
# specify variable inclusion and exclusion
frmls <- formula_branch(
  `No Covariate` = redCards ~ skintone,
  `Position` = redCards ~ skintone + position_alternative,
  `Position + Size` = redCards ~ skintone + position_alternative + size,
  `Position + Weight + Height` = redCards ~ skintone + position_alternative + weight + height,
  `Size` = redCards ~ skintone + size,
  `Weight + Height` = redCards ~ skintone + weight + height,
  `Country` = redCards ~ skintone + leagueCountry,
  `Position + Country` = redCards ~ skintone + position_alternative + leagueCountry,
  `Position + Size + Country` = redCards ~ skintone + position_alternative + size + leagueCountry,
  `Position + Weight + Height + Country` = redCards ~ skintone + position_alternative + weight + height + leagueCountry,
  `Size + Country` = redCards ~ skintone + size + leagueCountry,
  `Weight + Height + Country` = redCards ~ skintone + weight + height + leagueCountry,
)
# remove unnecessary branch combinations
condition_position_1 <- branch_condition(
  redCards ~ skintone, position)
condition_position_2 <- branch_condition(
  redCards ~ skintone + size, position)
condition_position_3 <- branch_condition(
  redCards ~ skintone + size + leagueCountry, position)
condition_position_4 <- branch_condition(
  redCards ~ skintone + leagueCountry, position)
condition_position_5 <- branch_condition(
  redCards ~ skintone + weight + height, position)
condition_position_6 <- branch_condition(
  redCards ~ skintone + weight + height + leagueCountry, position)

# bring the multiverse together and fit
mv <- mverse(soccer) %>%
  add_mutate_branch(position_alternative, size) %>%
  add_formula_branch(frmls) %>%
  add_family_branch(
    family_branch(logistic = binomial, name = "fam")
  ) %>%
  add_branch_condition(
    condition_position_1, condition_position_2, condition_position_3,
    condition_position_4, condition_position_5, condition_position_6
  )
glm_mverse(mv)

Without mverse

.
.
.
# specify variable inclusion and exclusion
model_1 <- formula(redCards ~ skintone)
model_2 <- formula(redCards ~ skintone + position)
model_3 <- formula(redCards ~ skintone + position_fld_vs_gk)
model_4 <- formula(redCards ~ skintone + position_group)

model_5 <- formula(redCards ~ skintone + 
                     weight)
model_6 <- formula(redCards ~ skintone + position + 
                     weight)
model_7 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                     weight)
model_8 <- formula(redCards ~ skintone + position_group + 
                     weight)

model_9 <- formula(redCards ~ skintone + 
                     height)
model_10 <- formula(redCards ~ skintone + position + 
                      height)
model_11 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      height)
model_12 <- formula(redCards ~ skintone + position_group + 
                      height)

model_13 <- formula(redCards ~ skintone + weight + 
                      height)
model_14 <- formula(redCards ~ skintone + position + 
                      weight + height)
model_15 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      weight + height)
model_16 <- formula(redCards ~ skintone + position_group + 
                      weight + height)

model_17 <- formula(redCards ~ skintone + 
                      leagueCountry)
model_18 <- formula(redCards ~ skintone + position + 
                      leagueCountry)
model_19 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      leagueCountry)
model_20 <- formula(redCards ~ skintone + position_group + 
                      leagueCountry)

model_21 <- formula(redCards ~ skintone + 
                      weight + leagueCountry)
model_22 <- formula(redCards ~ skintone + position + 
                      weight + leagueCountry)
model_23 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      weight + leagueCountry)
model_24 <- formula(redCards ~ skintone + position_group + 
                      weight + leagueCountry)

model_25 <- formula(redCards ~ skintone + 
                      height + leagueCountry)
model_26 <- formula(redCards ~ skintone + position + 
                      height + leagueCountry)
model_27 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      height + leagueCountry)
model_28 <- formula(redCards ~ skintone + position_group + 
                      height + leagueCountry)

model_29 <- formula(redCards ~ skintone + 
                      weight + height + leagueCountry)
model_30 <- formula(redCards ~ skintone + position + 
                      weight + height + leagueCountry)
model_31 <- formula(redCards ~ skintone + position_fld_vs_gk + 
                      weight + height + leagueCountry)
model_32 <- formula(redCards ~ skintone + position_group + 
                      weight + height + leagueCountry)

# bring the multiverse together and fit
models <- list(
  model_1, model_2, model_3, model_4,
  model_5, model_6, model_7, model_8,
  model_9, model_10, model_11, model_12,
  model_13, model_14, model_15, model_16,
  model_17, model_18, model_19, model_20,
  model_21, model_22, model_23, model_24,
  model_25, model_26, model_27, model_28,
  model_29, model_30, model_31, model_32
)
multiverse_fit <- lapply(
  models, glm, family = binomial(), data = soccer)

Systematic syntax

Automates multiplexing

With mverse

.
.
.
# bring the multiverse together and fit
mv <- mverse(soccer) %>%
  add_mutate_branch(position_alternative, size) %>%
  add_formula_branch(frmls) %>%
  add_family_branch(
    family_branch(logistic = binomial, name = "fam")
  ) %>%
  add_branch_condition(
    condition_position_1, condition_position_2, 
    condition_position_3, condition_position_4, 
    condition_position_5, condition_position_6
  )
glm_mverse(mv)

Without mverse

.
.
.
# bring the multiverse together and fit
models <- list(
  model_1, model_2, model_3, model_4,
  model_5, model_6, model_7, model_8,
  model_9, model_10, model_11, model_12,
  model_13, model_14, model_15, model_16,
  model_17, model_18, model_19, model_20,
  model_21, model_22, model_23, model_24,
  model_25, model_26, model_27, model_28,
  model_29, model_30, model_31, model_32
)
multiverse_fit <- lapply(
  models, glm, family = binomial, data = soccer)

Systematic syntax

library(mverse)
.
.
# data preparation steps
.
.
# create position alternatives
position_alternative <- mutate_branch(
  as_is = position, # use position as is
  # fielder vs goalkeeper
  position_fld_vs_gk = if_else(position == "Goalkeeper",
                               "Goalkeeper", "Fielder"),
  # grouping into 4 groups
  `Fielder Grouping` = forcats::fct_collapse(
    position,
    Forward = c("Center Forward", "Right Winger", "Left Winger"),
    Midfielder = c("Defensive Midfielder", "Attacking Midfielder",
                   "Right Midfielder", "Center Midfielder", 
                   "Left Midfielder"),
    Defender = c("Center Back", "Right Fullback", "Left Fullback"),
    Goalkeeper = c("Goalkeeper")
  )
)

# create size alternatives
size <- mutate_branch(height, weight)

# specify variable inclusion and exclusion
frmls <- formula_branch(
  `No Covariate` = redCards ~ skintone,
  `Position` = redCards ~ skintone + position_alternative,
  `Position + Size` = redCards ~ skintone + position_alternative + size,
  `Position + Weight + Height` = redCards ~ skintone + position_alternative + weight + height,
  `Size` = redCards ~ skintone + size,
  `Weight + Height` = redCards ~ skintone + weight + height,
  `Country` = redCards ~ skintone + leagueCountry,
  `Position + Country` = redCards ~ skintone + position_alternative + leagueCountry,
  `Position + Size + Country` = redCards ~ skintone + position_alternative + size + leagueCountry,
  `Position + Weight + Height + Country` = redCards ~ skintone + position_alternative + weight + height + leagueCountry,
  `Size + Country` = redCards ~ skintone + size + leagueCountry,
  `Weight + Height + Country` = redCards ~ skintone + weight + height + leagueCountry,
)
# remove unnecessary branch combinations
condition_position_1 <- branch_condition(
  redCards ~ skintone, position)
condition_position_2 <- branch_condition(
  redCards ~ skintone + size, position)
condition_position_3 <- branch_condition(
  redCards ~ skintone + size + leagueCountry, position)
condition_position_4 <- branch_condition(
  redCards ~ skintone + leagueCountry, position)
condition_position_5 <- branch_condition(
  redCards ~ skintone + weight + height, position)
condition_position_6 <- branch_condition(
  redCards ~ skintone + weight + height + leagueCountry, position)

# bring the multiverse together and fit
mv <- mverse(soccer) %>%
  add_mutate_branch(position_alternative, size) %>%
  add_formula_branch(frmls) %>%
  add_family_branch(
    family_branch(logistic = binomial, name = "fam")
  ) %>%
  add_branch_condition(
    condition_position_1, condition_position_2, condition_position_3,
    condition_position_4, condition_position_5, condition_position_6
  )
glm_mverse(mv)

Less prone to errors

compared to copy-and-paste approach

Familiar syntax

Defining alternative options for a variable

With mverse

# create position alternatives
position_alternative <- mutate_branch(
  as_is = position, # use position as is
  # fielder vs goalkeeper
  position_fld_vs_gk = if_else(position == "Goalkeeper",
                               "Goalkeeper", "Fielder"),
  # grouping into 4 groups
  `Fielder Grouping` = forcats::fct_collapse(
    position,
    Forward = c("Center Forward", "Right Winger", "Left Winger"),
    Midfielder = c("Defensive Midfielder", "Attacking Midfielder",
                   "Right Midfielder", "Center Midfielder", 
                   "Left Midfielder"),
    Defender = c("Center Back", "Right Fullback", "Left Fullback"),
    Goalkeeper = c("Goalkeeper")
  )
)

# create size alternatives
size <- mutate_branch(height, weight)

Without mverse

# create position alternatives
soccer <- mutate(
  soccer,
  # fielder vs goalkeeper
  position_fld_vs_gk = if_else(position == "Goalkeeper",
                               "Goalkeeper", "Fielder"),
  # grouping into 4 groups
  position_group = forcats::fct_collapse(
    position,
    Forward = c("Center Forward", "Right Winger", "Left Winger"),
    Midfielder = c("Defensive Midfielder", "Attacking Midfielder",
                   "Right Midfielder", "Center Midfielder", "Left Midfielder"),
    Defender = c("Center Back", "Right Fullback", "Left Fullback"),
    Goalkeeper = c("Goalkeeper")
  )
)

Familiar syntax

Uses dplyr grammar

With mverse

# create position alternatives
position_alternative <- mutate_branch(
  as_is = position, # use position as is
  # fielder vs goalkeeper
  position_fld_vs_gk = if_else(position == "Goalkeeper",
                               "Goalkeeper", "Fielder"),
  # grouping into 4 groups
  `Fielder Grouping` = forcats::fct_collapse(
    position,
    Forward = c("Center Forward", "Right Winger", "Left Winger"),
    Midfielder = c("Defensive Midfielder", "Attacking Midfielder",
                   "Right Midfielder", "Center Midfielder", 
                   "Left Midfielder"),
    Defender = c("Center Back", "Right Fullback", "Left Fullback"),
    Goalkeeper = c("Goalkeeper")
  )
)

# create size alternatives
size <- mutate_branch(height, weight)

Without mverse

# create position alternatives
soccer <- mutate(
  soccer,
  # fielder vs goalkeeper
  position_fld_vs_gk = if_else(position == "Goalkeeper",
                               "Goalkeeper", "Fielder"),
  # grouping into 4 groups
  position_group = forcats::fct_collapse(
    position,
    Forward = c("Center Forward", "Right Winger", "Left Winger"),
    Midfielder = c("Defensive Midfielder", "Attacking Midfielder",
                   "Right Midfielder", "Center Midfielder", "Left Midfielder"),
    Defender = c("Center Back", "Right Fullback", "Left Fullback"),
    Goalkeeper = c("Goalkeeper")
  )
)

Familiar syntax

Uses dplyr grammar

With mverse

# create position alternatives
position_alternative <- mutate_branch(
  as_is = position, # use position as is
  # fielder vs goalkeeper
  position_fld_vs_gk = if_else(position == "Goalkeeper",
                               "Goalkeeper", "Fielder"),
  # grouping into 4 groups
  `Fielder Grouping` = forcats::fct_collapse(
    position,
    Forward = c("Center Forward", "Right Winger", "Left Winger"),
    Midfielder = c("Defensive Midfielder", "Attacking Midfielder",
                   "Right Midfielder", "Center Midfielder", 
                   "Left Midfielder"),
    Defender = c("Center Back", "Right Fullback", "Left Fullback"),
    Goalkeeper = c("Goalkeeper")
  )
)

# create size alternatives
size <- mutate_branch(height, weight)

Less to learn

for those learning R and tidyverse

Interactive exploration tool

extract()

multiverse_tree()

spec_curve()

Interactive exploration tool

extract()

multiverse_tree()

spec_curve()

extract(
  mv, 
  columns = c("universe", "skintone", "position_alternative", 
              "redCards_raw", "games")
) %>%
  group_by(universe, skintone, position_alternative) %>%
  summarise(redCards_prop = sum(redCards_raw) / sum(games))

Interactive exploration tool

extract()

multiverse_tree()

spec_curve()

extract(
  mv, 
  columns = c("universe", "skintone", "position_alternative", 
              "redCards_raw", "games")
) %>%
  group_by(universe, skintone, position_alternative) %>%
  summarise(redCards_prop = sum(redCards_raw) / sum(games))

# A tibble: 3,144 × 4
# Groups:   universe, skintone [432]
     universe skintone position_alternative redCards_prop
        <dbl>    <dbl> <chr>                        <dbl>
   1        1      1   Attacking Midfielder      0.00229 
   2        1      1   Center Back               0.00642 
   3        1      1   Center Forward            0.00267 
   4        1      1   Center Midfielder         0.00307 
   5        1      1   Defensive Midfielder      0.00350 
   6        1      1   Goalkeeper                0.00506 
   7        1      1   Left Fullback             0.00448 
   8        1      1   Left Midfielder           0.00192 
   9        1      1   Left Winger               0.00158 
  10        1      1   Right Fullback            0.00370 
  11        1      1   Right Midfielder          0.00196 
  12        1      1   Right Winger              0.00136 
  13        1      1.5 Attacking Midfielder      0.00258 
  14        1      1.5 Center Back               0.00360 
  15        1      1.5 Center Forward            0.00243 
  16        1      1.5 Center Midfielder         0.00317 
  17        1      1.5 Defensive Midfielder      0.00500 
  18        1      1.5 Goalkeeper                0.00487 
  19        1      1.5 Left Fullback             0.00504 
  20        1      1.5 Left Midfielder           0.00333 
  21        1      1.5 Left Winger               0.00164 
  22        1      1.5 Right Fullback            0.00294 
  23        1      1.5 Right Midfielder          0.00421 
  24        1      1.5 Right Winger              0.00360 
  25        1      2   Attacking Midfielder      0.00248 
  26        1      2   Center Back               0.00769 
  27        1      2   Center Forward            0.00453 
  28        1      2   Center Midfielder         0.00474 
  29        1      2   Defensive Midfielder      0.00366 
  30        1      2   Goalkeeper                0.00372 
  31        1      2   Left Fullback             0.00318 
  32        1      2   Left Midfielder           0.00404 
  33        1      2   Left Winger               0.00373 
  34        1      2   Right Fullback            0.00356 
  35        1      2   Right Midfielder          0.00160 
  36        1      2   Right Winger              0.00443 
  37        1      2.5 Attacking Midfielder      0.00604 
  38        1      2.5 Center Back               0.00682 
  39        1      2.5 Center Forward            0.00494 
  40        1      2.5 Center Midfielder         0       
  41        1      2.5 Defensive Midfielder      0.00417 
  42        1      2.5 Goalkeeper                0.00316 
  43        1      2.5 Left Fullback             0.00804 
  44        1      2.5 Left Midfielder           0       
  45        1      2.5 Left Winger               0       
  46        1      2.5 Right Fullback            0.00438 
  47        1      2.5 Right Midfielder          0.00450 
  48        1      2.5 Right Winger              0.00657 
  49        1      3   Attacking Midfielder      0.00112 
  50        1      3   Center Back               0.00771 
  51        1      3   Center Forward            0.00304 
  52        1      3   Center Midfielder         0.00580 
  53        1      3   Defensive Midfielder      0.00681 
  54        1      3   Goalkeeper                0       
  55        1      3   Left Fullback             0.00320 
  56        1      3   Left Midfielder           0.00951 
  57        1      3   Left Winger               0.00691 
  58        1      3   Right Fullback            0.00667 
  59        1      3   Right Midfielder          0.00259 
  60        1      3   Right Winger              0.00511 
  61        1      3.5 Attacking Midfielder      0.002   
  62        1      3.5 Center Back               0.00395 
  63        1      3.5 Center Forward            0       
  64        1      3.5 Defensive Midfielder      0.00441 
  65        1      3.5 Goalkeeper                0       
  66        1      3.5 Left Fullback             0.00402 
  67        1      3.5 Left Midfielder           0       
  68        1      3.5 Left Winger               0       
  69        1      3.5 Right Fullback            0       
  70        1      3.5 Right Winger              0.00862 
  71        1      4   Attacking Midfielder      0.00183 
  72        1      4   Center Back               0.00587 
  73        1      4   Center Forward            0.00538 
  74        1      4   Center Midfielder         0.00639 
  75        1      4   Defensive Midfielder      0.00493 
  76        1      4   Goalkeeper                0.00463 
  77        1      4   Left Fullback             0.00375 
  78        1      4   Left Midfielder           0.00913 
  79        1      4   Left Winger               0.000892
  80        1      4   Right Fullback            0.00519 
  81        1      4   Right Midfielder          0.00876 
  82        1      4   Right Winger              0.00458 
  83        1      4.5 Center Back               0.00549 
  84        1      4.5 Center Forward            0.00470 
  85        1      4.5 Defensive Midfielder      0.00822 
  86        1      4.5 Goalkeeper                0.00196 
  87        1      4.5 Left Fullback             0.00937 
  88        1      4.5 Left Midfielder           0       
  89        1      4.5 Left Winger               0.00588 
  90        1      4.5 Right Fullback            0.0225  
  91        1      4.5 Right Midfielder          0       
  92        1      4.5 Right Winger              0.00103 
  93        1      5   Attacking Midfielder      0       
  94        1      5   Center Back               0.00781 
  95        1      5   Center Forward            0.00382 
  96        1      5   Center Midfielder         0.00251 
  97        1      5   Defensive Midfielder      0.00365 
  98        1      5   Goalkeeper                0.00870 
  99        1      5   Left Fullback             0.0113  
 100        1      5   Left Midfielder           0.00680 
 101        1      5   Left Winger               0.00265 
 102        1      5   Right Fullback            0.00242 
 103        1      5   Right Midfielder          0.00926 
 104        1      5   Right Winger              0.00299 
 105        2      1   Attacking Midfielder      0.00229 
 106        2      1   Center Back               0.00642 
 107        2      1   Center Forward            0.00267 
 108        2      1   Center Midfielder         0.00307 
 109        2      1   Defensive Midfielder      0.00350 
 110        2      1   Goalkeeper                0.00506 
 111        2      1   Left Fullback             0.00448 
 112        2      1   Left Midfielder           0.00192 
 113        2      1   Left Winger               0.00158 
 114        2      1   Right Fullback            0.00370 
 115        2      1   Right Midfielder          0.00196 
 116        2      1   Right Winger              0.00136 
 117        2      1.5 Attacking Midfielder      0.00258 
 118        2      1.5 Center Back               0.00360 
 119        2      1.5 Center Forward            0.00243 
 120        2      1.5 Center Midfielder         0.00317 
 121        2      1.5 Defensive Midfielder      0.00500 
 122        2      1.5 Goalkeeper                0.00487 
 123        2      1.5 Left Fullback             0.00504 
 124        2      1.5 Left Midfielder           0.00333 
 125        2      1.5 Left Winger               0.00164 
 126        2      1.5 Right Fullback            0.00294 
 127        2      1.5 Right Midfielder          0.00421 
 128        2      1.5 Right Winger              0.00360 
 129        2      2   Attacking Midfielder      0.00248 
 130        2      2   Center Back               0.00769 
 131        2      2   Center Forward            0.00453 
 132        2      2   Center Midfielder         0.00474 
 133        2      2   Defensive Midfielder      0.00366 
 134        2      2   Goalkeeper                0.00372 
 135        2      2   Left Fullback             0.00318 
 136        2      2   Left Midfielder           0.00404 
 137        2      2   Left Winger               0.00373 
 138        2      2   Right Fullback            0.00356 
 139        2      2   Right Midfielder          0.00160 
 140        2      2   Right Winger              0.00443 
 141        2      2.5 Attacking Midfielder      0.00604 
 142        2      2.5 Center Back               0.00682 
 143        2      2.5 Center Forward            0.00494 
 144        2      2.5 Center Midfielder         0       
 145        2      2.5 Defensive Midfielder      0.00417 
 146        2      2.5 Goalkeeper                0.00316 
 147        2      2.5 Left Fullback             0.00804 
 148        2      2.5 Left Midfielder           0       
 149        2      2.5 Left Winger               0       
 150        2      2.5 Right Fullback            0.00438 
 151        2      2.5 Right Midfielder          0.00450 
 152        2      2.5 Right Winger              0.00657 
 153        2      3   Attacking Midfielder      0.00112 
 154        2      3   Center Back               0.00771 
 155        2      3   Center Forward            0.00304 
 156        2      3   Center Midfielder         0.00580 
 157        2      3   Defensive Midfielder      0.00681 
 158        2      3   Goalkeeper                0       
 159        2      3   Left Fullback             0.00320 
 160        2      3   Left Midfielder           0.00951 
 161        2      3   Left Winger               0.00691 
 162        2      3   Right Fullback            0.00667 
 163        2      3   Right Midfielder          0.00259 
 164        2      3   Right Winger              0.00511 
 165        2      3.5 Attacking Midfielder      0.002   
 166        2      3.5 Center Back               0.00395 
 167        2      3.5 Center Forward            0       
 168        2      3.5 Defensive Midfielder      0.00441 
 169        2      3.5 Goalkeeper                0       
 170        2      3.5 Left Fullback             0.00402 
 171        2      3.5 Left Midfielder           0       
 172        2      3.5 Left Winger               0       
 173        2      3.5 Right Fullback            0       
 174        2      3.5 Right Winger              0.00862 
 175        2      4   Attacking Midfielder      0.00183 
 176        2      4   Center Back               0.00587 
 177        2      4   Center Forward            0.00538 
 178        2      4   Center Midfielder         0.00639 
 179        2      4   Defensive Midfielder      0.00493 
 180        2      4   Goalkeeper                0.00463 
 181        2      4   Left Fullback             0.00375 
 182        2      4   Left Midfielder           0.00913 
 183        2      4   Left Winger               0.000892
 184        2      4   Right Fullback            0.00519 
 185        2      4   Right Midfielder          0.00876 
 186        2      4   Right Winger              0.00458 
 187        2      4.5 Center Back               0.00549 
 188        2      4.5 Center Forward            0.00470 
 189        2      4.5 Defensive Midfielder      0.00822 
 190        2      4.5 Goalkeeper                0.00196 
 191        2      4.5 Left Fullback             0.00937 
 192        2      4.5 Left Midfielder           0       
 193        2      4.5 Left Winger               0.00588 
 194        2      4.5 Right Fullback            0.0225  
 195        2      4.5 Right Midfielder          0       
 196        2      4.5 Right Winger              0.00103 
 197        2      5   Attacking Midfielder      0       
 198        2      5   Center Back               0.00781 
 199        2      5   Center Forward            0.00382 
 200        2      5   Center Midfielder         0.00251 
 201        2      5   Defensive Midfielder      0.00365 
 202        2      5   Goalkeeper                0.00870 
 203        2      5   Left Fullback             0.0113  
 204        2      5   Left Midfielder           0.00680 
 205        2      5   Left Winger               0.00265 
 206        2      5   Right Fullback            0.00242 
 207        2      5   Right Midfielder          0.00926 
 208        2      5   Right Winger              0.00299 
 209        3      1   Attacking Midfielder      0.00229 
 210        3      1   Center Back               0.00642 
 211        3      1   Center Forward            0.00267 
 212        3      1   Center Midfielder         0.00307 
 213        3      1   Defensive Midfielder      0.00350 
 214        3      1   Goalkeeper                0.00506 
 215        3      1   Left Fullback             0.00448 
 216        3      1   Left Midfielder           0.00192 
 217        3      1   Left Winger               0.00158 
 218        3      1   Right Fullback            0.00370 
 219        3      1   Right Midfielder          0.00196 
 220        3      1   Right Winger              0.00136 
 221        3      1.5 Attacking Midfielder      0.00258 
 222        3      1.5 Center Back               0.00360 
 223        3      1.5 Center Forward            0.00243 
 224        3      1.5 Center Midfielder         0.00317 
 225        3      1.5 Defensive Midfielder      0.00500 
 226        3      1.5 Goalkeeper                0.00487 
 227        3      1.5 Left Fullback             0.00504 
 228        3      1.5 Left Midfielder           0.00333 
 229        3      1.5 Left Winger               0.00164 
 230        3      1.5 Right Fullback            0.00294 
 231        3      1.5 Right Midfielder          0.00421 
 232        3      1.5 Right Winger              0.00360 
 233        3      2   Attacking Midfielder      0.00248 
 234        3      2   Center Back               0.00769 
 235        3      2   Center Forward            0.00453 
 236        3      2   Center Midfielder         0.00474 
 237        3      2   Defensive Midfielder      0.00366 
 238        3      2   Goalkeeper                0.00372 
 239        3      2   Left Fullback             0.00318 
 240        3      2   Left Midfielder           0.00404 
 241        3      2   Left Winger               0.00373 
 242        3      2   Right Fullback            0.00356 
 243        3      2   Right Midfielder          0.00160 
 244        3      2   Right Winger              0.00443 
 245        3      2.5 Attacking Midfielder      0.00604 
 246        3      2.5 Center Back               0.00682 
 247        3      2.5 Center Forward            0.00494 
 248        3      2.5 Center Midfielder         0       
 249        3      2.5 Defensive Midfielder      0.00417 
 250        3      2.5 Goalkeeper                0.00316 
 251        3      2.5 Left Fullback             0.00804 
 252        3      2.5 Left Midfielder           0       
 253        3      2.5 Left Winger               0       
 254        3      2.5 Right Fullback            0.00438 
 255        3      2.5 Right Midfielder          0.00450 
 256        3      2.5 Right Winger              0.00657 
 257        3      3   Attacking Midfielder      0.00112 
 258        3      3   Center Back               0.00771 
 259        3      3   Center Forward            0.00304 
 260        3      3   Center Midfielder         0.00580 
 261        3      3   Defensive Midfielder      0.00681 
 262        3      3   Goalkeeper                0       
 263        3      3   Left Fullback             0.00320 
 264        3      3   Left Midfielder           0.00951 
 265        3      3   Left Winger               0.00691 
 266        3      3   Right Fullback            0.00667 
 267        3      3   Right Midfielder          0.00259 
 268        3      3   Right Winger              0.00511 
 269        3      3.5 Attacking Midfielder      0.002   
 270        3      3.5 Center Back               0.00395 
 271        3      3.5 Center Forward            0       
 272        3      3.5 Defensive Midfielder      0.00441 
 273        3      3.5 Goalkeeper                0       
 274        3      3.5 Left Fullback             0.00402 
 275        3      3.5 Left Midfielder           0       
 276        3      3.5 Left Winger               0       
 277        3      3.5 Right Fullback            0       
 278        3      3.5 Right Winger              0.00862 
 279        3      4   Attacking Midfielder      0.00183 
 280        3      4   Center Back               0.00587 
 281        3      4   Center Forward            0.00538 
 282        3      4   Center Midfielder         0.00639 
 283        3      4   Defensive Midfielder      0.00493 
 284        3      4   Goalkeeper                0.00463 
 285        3      4   Left Fullback             0.00375 
 286        3      4   Left Midfielder           0.00913 
 287        3      4   Left Winger               0.000892
 288        3      4   Right Fullback            0.00519 
 289        3      4   Right Midfielder          0.00876 
 290        3      4   Right Winger              0.00458 
 291        3      4.5 Center Back               0.00549 
 292        3      4.5 Center Forward            0.00470 
 293        3      4.5 Defensive Midfielder      0.00822 
 294        3      4.5 Goalkeeper                0.00196 
 295        3      4.5 Left Fullback             0.00937 
 296        3      4.5 Left Midfielder           0       
 297        3      4.5 Left Winger               0.00588 
 298        3      4.5 Right Fullback            0.0225  
 299        3      4.5 Right Midfielder          0       
 300        3      4.5 Right Winger              0.00103 
 301        3      5   Attacking Midfielder      0       
 302        3      5   Center Back               0.00781 
 303        3      5   Center Forward            0.00382 
 304        3      5   Center Midfielder         0.00251 
 305        3      5   Defensive Midfielder      0.00365 
 306        3      5   Goalkeeper                0.00870 
 307        3      5   Left Fullback             0.0113  
 308        3      5   Left Midfielder           0.00680 
 309        3      5   Left Winger               0.00265 
 310        3      5   Right Fullback            0.00242 
 311        3      5   Right Midfielder          0.00926 
 312        3      5   Right Winger              0.00299 
 313        4      1   Attacking Midfielder      0.00229 
 314        4      1   Center Back               0.00642 
 315        4      1   Center Forward            0.00267 
 316        4      1   Center Midfielder         0.00307 
 317        4      1   Defensive Midfielder      0.00350 
 318        4      1   Goalkeeper                0.00506 
 319        4      1   Left Fullback             0.00448 
 320        4      1   Left Midfielder           0.00192 
 321        4      1   Left Winger               0.00158 
 322        4      1   Right Fullback            0.00370 
 323        4      1   Right Midfielder          0.00196 
 324        4      1   Right Winger              0.00136 
 325        4      1.5 Attacking Midfielder      0.00258 
 326        4      1.5 Center Back               0.00360 
 327        4      1.5 Center Forward            0.00243 
 328        4      1.5 Center Midfielder         0.00317 
 329        4      1.5 Defensive Midfielder      0.00500 
 330        4      1.5 Goalkeeper                0.00487 
 331        4      1.5 Left Fullback             0.00504 
 332        4      1.5 Left Midfielder           0.00333 
 333        4      1.5 Left Winger               0.00164 
 334        4      1.5 Right Fullback            0.00294 
 335        4      1.5 Right Midfielder          0.00421 
 336        4      1.5 Right Winger              0.00360 
 337        4      2   Attacking Midfielder      0.00248 
 338        4      2   Center Back               0.00769 
 339        4      2   Center Forward            0.00453 
 340        4      2   Center Midfielder         0.00474 
 341        4      2   Defensive Midfielder      0.00366 
 342        4      2   Goalkeeper                0.00372 
 343        4      2   Left Fullback             0.00318 
 344        4      2   Left Midfielder           0.00404 
 345        4      2   Left Winger               0.00373 
 346        4      2   Right Fullback            0.00356 
 347        4      2   Right Midfielder          0.00160 
 348        4      2   Right Winger              0.00443 
 349        4      2.5 Attacking Midfielder      0.00604 
 350        4      2.5 Center Back               0.00682 
 351        4      2.5 Center Forward            0.00494 
 352        4      2.5 Center Midfielder         0       
 353        4      2.5 Defensive Midfielder      0.00417 
 354        4      2.5 Goalkeeper                0.00316 
 355        4      2.5 Left Fullback             0.00804 
 356        4      2.5 Left Midfielder           0       
 357        4      2.5 Left Winger               0       
 358        4      2.5 Right Fullback            0.00438 
 359        4      2.5 Right Midfielder          0.00450 
 360        4      2.5 Right Winger              0.00657 
 361        4      3   Attacking Midfielder      0.00112 
 362        4      3   Center Back               0.00771 
 363        4      3   Center Forward            0.00304 
 364        4      3   Center Midfielder         0.00580 
 365        4      3   Defensive Midfielder      0.00681 
 366        4      3   Goalkeeper                0       
 367        4      3   Left Fullback             0.00320 
 368        4      3   Left Midfielder           0.00951 
 369        4      3   Left Winger               0.00691 
 370        4      3   Right Fullback            0.00667 
 371        4      3   Right Midfielder          0.00259 
 372        4      3   Right Winger              0.00511 
 373        4      3.5 Attacking Midfielder      0.002   
 374        4      3.5 Center Back               0.00395 
 375        4      3.5 Center Forward            0       
 376        4      3.5 Defensive Midfielder      0.00441 
 377        4      3.5 Goalkeeper                0       
 378        4      3.5 Left Fullback             0.00402 
 379        4      3.5 Left Midfielder           0       
 380        4      3.5 Left Winger               0       
 381        4      3.5 Right Fullback            0       
 382        4      3.5 Right Winger              0.00862 
 383        4      4   Attacking Midfielder      0.00183 
 384        4      4   Center Back               0.00587 
 385        4      4   Center Forward            0.00538 
 386        4      4   Center Midfielder         0.00639 
 387        4      4   Defensive Midfielder      0.00493 
 388        4      4   Goalkeeper                0.00463 
 389        4      4   Left Fullback             0.00375 
 390        4      4   Left Midfielder           0.00913 
 391        4      4   Left Winger               0.000892
 392        4      4   Right Fullback            0.00519 
 393        4      4   Right Midfielder          0.00876 
 394        4      4   Right Winger              0.00458 
 395        4      4.5 Center Back               0.00549 
 396        4      4.5 Center Forward            0.00470 
 397        4      4.5 Defensive Midfielder      0.00822 
 398        4      4.5 Goalkeeper                0.00196 
 399        4      4.5 Left Fullback             0.00937 
 400        4      4.5 Left Midfielder           0       
 401        4      4.5 Left Winger               0.00588 
 402        4      4.5 Right Fullback            0.0225  
 403        4      4.5 Right Midfielder          0       
 404        4      4.5 Right Winger              0.00103 
 405        4      5   Attacking Midfielder      0       
 406        4      5   Center Back               0.00781 
 407        4      5   Center Forward            0.00382 
 408        4      5   Center Midfielder         0.00251 
 409        4      5   Defensive Midfielder      0.00365 
 410        4      5   Goalkeeper                0.00870 
 411        4      5   Left Fullback             0.0113  
 412        4      5   Left Midfielder           0.00680 
 413        4      5   Left Winger               0.00265 
 414        4      5   Right Fullback            0.00242 
 415        4      5   Right Midfielder          0.00926 
 416        4      5   Right Winger              0.00299 
 417        5      1   Attacking Midfielder      0.00229 
 418        5      1   Center Back               0.00642 
 419        5      1   Center Forward            0.00267 
 420        5      1   Center Midfielder         0.00307 
 421        5      1   Defensive Midfielder      0.00350 
 422        5      1   Goalkeeper                0.00506 
 423        5      1   Left Fullback             0.00448 
 424        5      1   Left Midfielder           0.00192 
 425        5      1   Left Winger               0.00158 
 426        5      1   Right Fullback            0.00370 
 427        5      1   Right Midfielder          0.00196 
 428        5      1   Right Winger              0.00136 
 429        5      1.5 Attacking Midfielder      0.00258 
 430        5      1.5 Center Back               0.00360 
 431        5      1.5 Center Forward            0.00243 
 432        5      1.5 Center Midfielder         0.00317 
 433        5      1.5 Defensive Midfielder      0.00500 
 434        5      1.5 Goalkeeper                0.00487 
 435        5      1.5 Left Fullback             0.00504 
 436        5      1.5 Left Midfielder           0.00333 
 437        5      1.5 Left Winger               0.00164 
 438        5      1.5 Right Fullback            0.00294 
 439        5      1.5 Right Midfielder          0.00421 
 440        5      1.5 Right Winger              0.00360 
 441        5      2   Attacking Midfielder      0.00248 
 442        5      2   Center Back               0.00769 
 443        5      2   Center Forward            0.00453 
 444        5      2   Center Midfielder         0.00474 
 445        5      2   Defensive Midfielder      0.00366 
 446        5      2   Goalkeeper                0.00372 
 447        5      2   Left Fullback             0.00318 
 448        5      2   Left Midfielder           0.00404 
 449        5      2   Left Winger               0.00373 
 450        5      2   Right Fullback            0.00356 
 451        5      2   Right Midfielder          0.00160 
 452        5      2   Right Winger              0.00443 
 453        5      2.5 Attacking Midfielder      0.00604 
 454        5      2.5 Center Back               0.00682 
 455        5      2.5 Center Forward            0.00494 
 456        5      2.5 Center Midfielder         0       
 457        5      2.5 Defensive Midfielder      0.00417 
 458        5      2.5 Goalkeeper                0.00316 
 459        5      2.5 Left Fullback             0.00804 
 460        5      2.5 Left Midfielder           0       
 461        5      2.5 Left Winger               0       
 462        5      2.5 Right Fullback            0.00438 
 463        5      2.5 Right Midfielder          0.00450 
 464        5      2.5 Right Winger              0.00657 
 465        5      3   Attacking Midfielder      0.00112 
 466        5      3   Center Back               0.00771 
 467        5      3   Center Forward            0.00304 
 468        5      3   Center Midfielder         0.00580 
 469        5      3   Defensive Midfielder      0.00681 
 470        5      3   Goalkeeper                0       
 471        5      3   Left Fullback             0.00320 
 472        5      3   Left Midfielder           0.00951 
 473        5      3   Left Winger               0.00691 
 474        5      3   Right Fullback            0.00667 
 475        5      3   Right Midfielder          0.00259 
 476        5      3   Right Winger              0.00511 
 477        5      3.5 Attacking Midfielder      0.002   
 478        5      3.5 Center Back               0.00395 
 479        5      3.5 Center Forward            0       
 480        5      3.5 Defensive Midfielder      0.00441 
 481        5      3.5 Goalkeeper                0       
 482        5      3.5 Left Fullback             0.00402 
 483        5      3.5 Left Midfielder           0       
 484        5      3.5 Left Winger               0       
 485        5      3.5 Right Fullback            0       
 486        5      3.5 Right Winger              0.00862 
 487        5      4   Attacking Midfielder      0.00183 
 488        5      4   Center Back               0.00587 
 489        5      4   Center Forward            0.00538 
 490        5      4   Center Midfielder         0.00639 
 491        5      4   Defensive Midfielder      0.00493 
 492        5      4   Goalkeeper                0.00463 
 493        5      4   Left Fullback             0.00375 
 494        5      4   Left Midfielder           0.00913 
 495        5      4   Left Winger               0.000892
 496        5      4   Right Fullback            0.00519 
 497        5      4   Right Midfielder          0.00876 
 498        5      4   Right Winger              0.00458 
 499        5      4.5 Center Back               0.00549 
 500        5      4.5 Center Forward            0.00470 
 501        5      4.5 Defensive Midfielder      0.00822 
 502        5      4.5 Goalkeeper                0.00196 
 503        5      4.5 Left Fullback             0.00937 
 504        5      4.5 Left Midfielder           0       
 505        5      4.5 Left Winger               0.00588 
 506        5      4.5 Right Fullback            0.0225  
 507        5      4.5 Right Midfielder          0       
 508        5      4.5 Right Winger              0.00103 
 509        5      5   Attacking Midfielder      0       
 510        5      5   Center Back               0.00781 
 511        5      5   Center Forward            0.00382 
 512        5      5   Center Midfielder         0.00251 
 513        5      5   Defensive Midfielder      0.00365 
 514        5      5   Goalkeeper                0.00870 
 515        5      5   Left Fullback             0.0113  
 516        5      5   Left Midfielder           0.00680 
 517        5      5   Left Winger               0.00265 
 518        5      5   Right Fullback            0.00242 
 519        5      5   Right Midfielder          0.00926 
 520        5      5   Right Winger              0.00299 
 521        6      1   Attacking Midfielder      0.00229 
 522        6      1   Center Back               0.00642 
 523        6      1   Center Forward            0.00267 
 524        6      1   Center Midfielder         0.00307 
 525        6      1   Defensive Midfielder      0.00350 
 526        6      1   Goalkeeper                0.00506 
 527        6      1   Left Fullback             0.00448 
 528        6      1   Left Midfielder           0.00192 
 529        6      1   Left Winger               0.00158 
 530        6      1   Right Fullback            0.00370 
 531        6      1   Right Midfielder          0.00196 
 532        6      1   Right Winger              0.00136 
 533        6      1.5 Attacking Midfielder      0.00258 
 534        6      1.5 Center Back               0.00360 
 535        6      1.5 Center Forward            0.00243 
 536        6      1.5 Center Midfielder         0.00317 
 537        6      1.5 Defensive Midfielder      0.00500 
 538        6      1.5 Goalkeeper                0.00487 
 539        6      1.5 Left Fullback             0.00504 
 540        6      1.5 Left Midfielder           0.00333 
 541        6      1.5 Left Winger               0.00164 
 542        6      1.5 Right Fullback            0.00294 
 543        6      1.5 Right Midfielder          0.00421 
 544        6      1.5 Right Winger              0.00360 
 545        6      2   Attacking Midfielder      0.00248 
 546        6      2   Center Back               0.00769 
 547        6      2   Center Forward            0.00453 
 548        6      2   Center Midfielder         0.00474 
 549        6      2   Defensive Midfielder      0.00366 
 550        6      2   Goalkeeper                0.00372 
 551        6      2   Left Fullback             0.00318 
 552        6      2   Left Midfielder           0.00404 
 553        6      2   Left Winger               0.00373 
 554        6      2   Right Fullback            0.00356 
 555        6      2   Right Midfielder          0.00160 
 556        6      2   Right Winger              0.00443 
 557        6      2.5 Attacking Midfielder      0.00604 
 558        6      2.5 Center Back               0.00682 
 559        6      2.5 Center Forward            0.00494 
 560        6      2.5 Center Midfielder         0       
 561        6      2.5 Defensive Midfielder      0.00417 
 562        6      2.5 Goalkeeper                0.00316 
 563        6      2.5 Left Fullback             0.00804 
 564        6      2.5 Left Midfielder           0       
 565        6      2.5 Left Winger               0       
 566        6      2.5 Right Fullback            0.00438 
 567        6      2.5 Right Midfielder          0.00450 
 568        6      2.5 Right Winger              0.00657 
 569        6      3   Attacking Midfielder      0.00112 
 570        6      3   Center Back               0.00771 
 571        6      3   Center Forward            0.00304 
 572        6      3   Center Midfielder         0.00580 
 573        6      3   Defensive Midfielder      0.00681 
 574        6      3   Goalkeeper                0       
 575        6      3   Left Fullback             0.00320 
 576        6      3   Left Midfielder           0.00951 
 577        6      3   Left Winger               0.00691 
 578        6      3   Right Fullback            0.00667 
 579        6      3   Right Midfielder          0.00259 
 580        6      3   Right Winger              0.00511 
 581        6      3.5 Attacking Midfielder      0.002   
 582        6      3.5 Center Back               0.00395 
 583        6      3.5 Center Forward            0       
 584        6      3.5 Defensive Midfielder      0.00441 
 585        6      3.5 Goalkeeper                0       
 586        6      3.5 Left Fullback             0.00402 
 587        6      3.5 Left Midfielder           0       
 588        6      3.5 Left Winger               0       
 589        6      3.5 Right Fullback            0       
 590        6      3.5 Right Winger              0.00862 
 591        6      4   Attacking Midfielder      0.00183 
 592        6      4   Center Back               0.00587 
 593        6      4   Center Forward            0.00538 
 594        6      4   Center Midfielder         0.00639 
 595        6      4   Defensive Midfielder      0.00493 
 596        6      4   Goalkeeper                0.00463 
 597        6      4   Left Fullback             0.00375 
 598        6      4   Left Midfielder           0.00913 
 599        6      4   Left Winger               0.000892
 600        6      4   Right Fullback            0.00519 
 601        6      4   Right Midfielder          0.00876 
 602        6      4   Right Winger              0.00458 
 603        6      4.5 Center Back               0.00549 
 604        6      4.5 Center Forward            0.00470 
 605        6      4.5 Defensive Midfielder      0.00822 
 606        6      4.5 Goalkeeper                0.00196 
 607        6      4.5 Left Fullback             0.00937 
 608        6      4.5 Left Midfielder           0       
 609        6      4.5 Left Winger               0.00588 
 610        6      4.5 Right Fullback            0.0225  
 611        6      4.5 Right Midfielder          0       
 612        6      4.5 Right Winger              0.00103 
 613        6      5   Attacking Midfielder      0       
 614        6      5   Center Back               0.00781 
 615        6      5   Center Forward            0.00382 
 616        6      5   Center Midfielder         0.00251 
 617        6      5   Defensive Midfielder      0.00365 
 618        6      5   Goalkeeper                0.00870 
 619        6      5   Left Fullback             0.0113  
 620        6      5   Left Midfielder           0.00680 
 621        6      5   Left Winger               0.00265 
 622        6      5   Right Fullback            0.00242 
 623        6      5   Right Midfielder          0.00926 
 624        6      5   Right Winger              0.00299 
 625        7      1   Attacking Midfielder      0.00229 
 626        7      1   Center Back               0.00642 
 627        7      1   Center Forward            0.00267 
 628        7      1   Center Midfielder         0.00307 
 629        7      1   Defensive Midfielder      0.00350 
 630        7      1   Goalkeeper                0.00506 
 631        7      1   Left Fullback             0.00448 
 632        7      1   Left Midfielder           0.00192 
 633        7      1   Left Winger               0.00158 
 634        7      1   Right Fullback            0.00370 
 635        7      1   Right Midfielder          0.00196 
 636        7      1   Right Winger              0.00136 
 637        7      1.5 Attacking Midfielder      0.00258 
 638        7      1.5 Center Back               0.00360 
 639        7      1.5 Center Forward            0.00243 
 640        7      1.5 Center Midfielder         0.00317 
 641        7      1.5 Defensive Midfielder      0.00500 
 642        7      1.5 Goalkeeper                0.00487 
 643        7      1.5 Left Fullback             0.00504 
 644        7      1.5 Left Midfielder           0.00333 
 645        7      1.5 Left Winger               0.00164 
 646        7      1.5 Right Fullback            0.00294 
 647        7      1.5 Right Midfielder          0.00421 
 648        7      1.5 Right Winger              0.00360 
 649        7      2   Attacking Midfielder      0.00248 
 650        7      2   Center Back               0.00769 
 651        7      2   Center Forward            0.00453 
 652        7      2   Center Midfielder         0.00474 
 653        7      2   Defensive Midfielder      0.00366 
 654        7      2   Goalkeeper                0.00372 
 655        7      2   Left Fullback             0.00318 
 656        7      2   Left Midfielder           0.00404 
 657        7      2   Left Winger               0.00373 
 658        7      2   Right Fullback            0.00356 
 659        7      2   Right Midfielder          0.00160 
 660        7      2   Right Winger              0.00443 
 661        7      2.5 Attacking Midfielder      0.00604 
 662        7      2.5 Center Back               0.00682 
 663        7      2.5 Center Forward            0.00494 
 664        7      2.5 Center Midfielder         0       
 665        7      2.5 Defensive Midfielder      0.00417 
 666        7      2.5 Goalkeeper                0.00316 
 667        7      2.5 Left Fullback             0.00804 
 668        7      2.5 Left Midfielder           0       
 669        7      2.5 Left Winger               0       
 670        7      2.5 Right Fullback            0.00438 
 671        7      2.5 Right Midfielder          0.00450 
 672        7      2.5 Right Winger              0.00657 
 673        7      3   Attacking Midfielder      0.00112 
 674        7      3   Center Back               0.00771 
 675        7      3   Center Forward            0.00304 
 676        7      3   Center Midfielder         0.00580 
 677        7      3   Defensive Midfielder      0.00681 
 678        7      3   Goalkeeper                0       
 679        7      3   Left Fullback             0.00320 
 680        7      3   Left Midfielder           0.00951 
 681        7      3   Left Winger               0.00691 
 682        7      3   Right Fullback            0.00667 
 683        7      3   Right Midfielder          0.00259 
 684        7      3   Right Winger              0.00511 
 685        7      3.5 Attacking Midfielder      0.002   
 686        7      3.5 Center Back               0.00395 
 687        7      3.5 Center Forward            0       
 688        7      3.5 Defensive Midfielder      0.00441 
 689        7      3.5 Goalkeeper                0       
 690        7      3.5 Left Fullback             0.00402 
 691        7      3.5 Left Midfielder           0       
 692        7      3.5 Left Winger               0       
 693        7      3.5 Right Fullback            0       
 694        7      3.5 Right Winger              0.00862 
 695        7      4   Attacking Midfielder      0.00183 
 696        7      4   Center Back               0.00587 
 697        7      4   Center Forward            0.00538 
 698        7      4   Center Midfielder         0.00639 
 699        7      4   Defensive Midfielder      0.00493 
 700        7      4   Goalkeeper                0.00463 
 701        7      4   Left Fullback             0.00375 
 702        7      4   Left Midfielder           0.00913 
 703        7      4   Left Winger               0.000892
 704        7      4   Right Fullback            0.00519 
 705        7      4   Right Midfielder          0.00876 
 706        7      4   Right Winger              0.00458 
 707        7      4.5 Center Back               0.00549 
 708        7      4.5 Center Forward            0.00470 
 709        7      4.5 Defensive Midfielder      0.00822 
 710        7      4.5 Goalkeeper                0.00196 
 711        7      4.5 Left Fullback             0.00937 
 712        7      4.5 Left Midfielder           0       
 713        7      4.5 Left Winger               0.00588 
 714        7      4.5 Right Fullback            0.0225  
 715        7      4.5 Right Midfielder          0       
 716        7      4.5 Right Winger              0.00103 
 717        7      5   Attacking Midfielder      0       
 718        7      5   Center Back               0.00781 
 719        7      5   Center Forward            0.00382 
 720        7      5   Center Midfielder         0.00251 
 721        7      5   Defensive Midfielder      0.00365 
 722        7      5   Goalkeeper                0.00870 
 723        7      5   Left Fullback             0.0113  
 724        7      5   Left Midfielder           0.00680 
 725        7      5   Left Winger               0.00265 
 726        7      5   Right Fullback            0.00242 
 727        7      5   Right Midfielder          0.00926 
 728        7      5   Right Winger              0.00299 
 729        8      1   Attacking Midfielder      0.00229 
 730        8      1   Center Back               0.00642 
 731        8      1   Center Forward            0.00267 
 732        8      1   Center Midfielder         0.00307 
 733        8      1   Defensive Midfielder      0.00350 
 734        8      1   Goalkeeper                0.00506 
 735        8      1   Left Fullback             0.00448 
 736        8      1   Left Midfielder           0.00192 
 737        8      1   Left Winger               0.00158 
 738        8      1   Right Fullback            0.00370 
 739        8      1   Right Midfielder          0.00196 
 740        8      1   Right Winger              0.00136 
 741        8      1.5 Attacking Midfielder      0.00258 
 742        8      1.5 Center Back               0.00360 
 743        8      1.5 Center Forward            0.00243 
 744        8      1.5 Center Midfielder         0.00317 
 745        8      1.5 Defensive Midfielder      0.00500 
 746        8      1.5 Goalkeeper                0.00487 
 747        8      1.5 Left Fullback             0.00504 
 748        8      1.5 Left Midfielder           0.00333 
 749        8      1.5 Left Winger               0.00164 
 750        8      1.5 Right Fullback            0.00294 
 751        8      1.5 Right Midfielder          0.00421 
 752        8      1.5 Right Winger              0.00360 
 753        8      2   Attacking Midfielder      0.00248 
 754        8      2   Center Back               0.00769 
 755        8      2   Center Forward            0.00453 
 756        8      2   Center Midfielder         0.00474 
 757        8      2   Defensive Midfielder      0.00366 
 758        8      2   Goalkeeper                0.00372 
 759        8      2   Left Fullback             0.00318 
 760        8      2   Left Midfielder           0.00404 
 761        8      2   Left Winger               0.00373 
 762        8      2   Right Fullback            0.00356 
 763        8      2   Right Midfielder          0.00160 
 764        8      2   Right Winger              0.00443 
 765        8      2.5 Attacking Midfielder      0.00604 
 766        8      2.5 Center Back               0.00682 
 767        8      2.5 Center Forward            0.00494 
 768        8      2.5 Center Midfielder         0       
 769        8      2.5 Defensive Midfielder      0.00417 
 770        8      2.5 Goalkeeper                0.00316 
 771        8      2.5 Left Fullback             0.00804 
 772        8      2.5 Left Midfielder           0       
 773        8      2.5 Left Winger               0       
 774        8      2.5 Right Fullback            0.00438 
 775        8      2.5 Right Midfielder          0.00450 
 776        8      2.5 Right Winger              0.00657 
 777        8      3   Attacking Midfielder      0.00112 
 778        8      3   Center Back               0.00771 
 779        8      3   Center Forward            0.00304 
 780        8      3   Center Midfielder         0.00580 
 781        8      3   Defensive Midfielder      0.00681 
 782        8      3   Goalkeeper                0       
 783        8      3   Left Fullback             0.00320 
 784        8      3   Left Midfielder           0.00951 
 785        8      3   Left Winger               0.00691 
 786        8      3   Right Fullback            0.00667 
 787        8      3   Right Midfielder          0.00259 
 788        8      3   Right Winger              0.00511 
 789        8      3.5 Attacking Midfielder      0.002   
 790        8      3.5 Center Back               0.00395 
 791        8      3.5 Center Forward            0       
 792        8      3.5 Defensive Midfielder      0.00441 
 793        8      3.5 Goalkeeper                0       
 794        8      3.5 Left Fullback             0.00402 
 795        8      3.5 Left Midfielder           0       
 796        8      3.5 Left Winger               0       
 797        8      3.5 Right Fullback            0       
 798        8      3.5 Right Winger              0.00862 
 799        8      4   Attacking Midfielder      0.00183 
 800        8      4   Center Back               0.00587 
 801        8      4   Center Forward            0.00538 
 802        8      4   Center Midfielder         0.00639 
 803        8      4   Defensive Midfielder      0.00493 
 804        8      4   Goalkeeper                0.00463 
 805        8      4   Left Fullback             0.00375 
 806        8      4   Left Midfielder           0.00913 
 807        8      4   Left Winger               0.000892
 808        8      4   Right Fullback            0.00519 
 809        8      4   Right Midfielder          0.00876 
 810        8      4   Right Winger              0.00458 
 811        8      4.5 Center Back               0.00549 
 812        8      4.5 Center Forward            0.00470 
 813        8      4.5 Defensive Midfielder      0.00822 
 814        8      4.5 Goalkeeper                0.00196 
 815        8      4.5 Left Fullback             0.00937 
 816        8      4.5 Left Midfielder           0       
 817        8      4.5 Left Winger               0.00588 
 818        8      4.5 Right Fullback            0.0225  
 819        8      4.5 Right Midfielder          0       
 820        8      4.5 Right Winger              0.00103 
 821        8      5   Attacking Midfielder      0       
 822        8      5   Center Back               0.00781 
 823        8      5   Center Forward            0.00382 
 824        8      5   Center Midfielder         0.00251 
 825        8      5   Defensive Midfielder      0.00365 
 826        8      5   Goalkeeper                0.00870 
 827        8      5   Left Fullback             0.0113  
 828        8      5   Left Midfielder           0.00680 
 829        8      5   Left Winger               0.00265 
 830        8      5   Right Fullback            0.00242 
 831        8      5   Right Midfielder          0.00926 
 832        8      5   Right Winger              0.00299 
 833        9      1   Attacking Midfielder      0.00229 
 834        9      1   Center Back               0.00642 
 835        9      1   Center Forward            0.00267 
 836        9      1   Center Midfielder         0.00307 
 837        9      1   Defensive Midfielder      0.00350 
 838        9      1   Goalkeeper                0.00506 
 839        9      1   Left Fullback             0.00448 
 840        9      1   Left Midfielder           0.00192 
 841        9      1   Left Winger               0.00158 
 842        9      1   Right Fullback            0.00370 
 843        9      1   Right Midfielder          0.00196 
 844        9      1   Right Winger              0.00136 
 845        9      1.5 Attacking Midfielder      0.00258 
 846        9      1.5 Center Back               0.00360 
 847        9      1.5 Center Forward            0.00243 
 848        9      1.5 Center Midfielder         0.00317 
 849        9      1.5 Defensive Midfielder      0.00500 
 850        9      1.5 Goalkeeper                0.00487 
 851        9      1.5 Left Fullback             0.00504 
 852        9      1.5 Left Midfielder           0.00333 
 853        9      1.5 Left Winger               0.00164 
 854        9      1.5 Right Fullback            0.00294 
 855        9      1.5 Right Midfielder          0.00421 
 856        9      1.5 Right Winger              0.00360 
 857        9      2   Attacking Midfielder      0.00248 
 858        9      2   Center Back               0.00769 
 859        9      2   Center Forward            0.00453 
 860        9      2   Center Midfielder         0.00474 
 861        9      2   Defensive Midfielder      0.00366 
 862        9      2   Goalkeeper                0.00372 
 863        9      2   Left Fullback             0.00318 
 864        9      2   Left Midfielder           0.00404 
 865        9      2   Left Winger               0.00373 
 866        9      2   Right Fullback            0.00356 
 867        9      2   Right Midfielder          0.00160 
 868        9      2   Right Winger              0.00443 
 869        9      2.5 Attacking Midfielder      0.00604 
 870        9      2.5 Center Back               0.00682 
 871        9      2.5 Center Forward            0.00494 
 872        9      2.5 Center Midfielder         0       
 873        9      2.5 Defensive Midfielder      0.00417 
 874        9      2.5 Goalkeeper                0.00316 
 875        9      2.5 Left Fullback             0.00804 
 876        9      2.5 Left Midfielder           0       
 877        9      2.5 Left Winger               0       
 878        9      2.5 Right Fullback            0.00438 
 879        9      2.5 Right Midfielder          0.00450 
 880        9      2.5 Right Winger              0.00657 
 881        9      3   Attacking Midfielder      0.00112 
 882        9      3   Center Back               0.00771 
 883        9      3   Center Forward            0.00304 
 884        9      3   Center Midfielder         0.00580 
 885        9      3   Defensive Midfielder      0.00681 
 886        9      3   Goalkeeper                0       
 887        9      3   Left Fullback             0.00320 
 888        9      3   Left Midfielder           0.00951 
 889        9      3   Left Winger               0.00691 
 890        9      3   Right Fullback            0.00667 
 891        9      3   Right Midfielder          0.00259 
 892        9      3   Right Winger              0.00511 
 893        9      3.5 Attacking Midfielder      0.002   
 894        9      3.5 Center Back               0.00395 
 895        9      3.5 Center Forward            0       
 896        9      3.5 Defensive Midfielder      0.00441 
 897        9      3.5 Goalkeeper                0       
 898        9      3.5 Left Fullback             0.00402 
 899        9      3.5 Left Midfielder           0       
 900        9      3.5 Left Winger               0       
 901        9      3.5 Right Fullback            0       
 902        9      3.5 Right Winger              0.00862 
 903        9      4   Attacking Midfielder      0.00183 
 904        9      4   Center Back               0.00587 
 905        9      4   Center Forward            0.00538 
 906        9      4   Center Midfielder         0.00639 
 907        9      4   Defensive Midfielder      0.00493 
 908        9      4   Goalkeeper                0.00463 
 909        9      4   Left Fullback             0.00375 
 910        9      4   Left Midfielder           0.00913 
 911        9      4   Left Winger               0.000892
 912        9      4   Right Fullback            0.00519 
 913        9      4   Right Midfielder          0.00876 
 914        9      4   Right Winger              0.00458 
 915        9      4.5 Center Back               0.00549 
 916        9      4.5 Center Forward            0.00470 
 917        9      4.5 Defensive Midfielder      0.00822 
 918        9      4.5 Goalkeeper                0.00196 
 919        9      4.5 Left Fullback             0.00937 
 920        9      4.5 Left Midfielder           0       
 921        9      4.5 Left Winger               0.00588 
 922        9      4.5 Right Fullback            0.0225  
 923        9      4.5 Right Midfielder          0       
 924        9      4.5 Right Winger              0.00103 
 925        9      5   Attacking Midfielder      0       
 926        9      5   Center Back               0.00781 
 927        9      5   Center Forward            0.00382 
 928        9      5   Center Midfielder         0.00251 
 929        9      5   Defensive Midfielder      0.00365 
 930        9      5   Goalkeeper                0.00870 
 931        9      5   Left Fullback             0.0113  
 932        9      5   Left Midfielder           0.00680 
 933        9      5   Left Winger               0.00265 
 934        9      5   Right Fullback            0.00242 
 935        9      5   Right Midfielder          0.00926 
 936        9      5   Right Winger              0.00299 
 937       10      1   Attacking Midfielder      0.00229 
 938       10      1   Center Back               0.00642 
 939       10      1   Center Forward            0.00267 
 940       10      1   Center Midfielder         0.00307 
 941       10      1   Defensive Midfielder      0.00350 
 942       10      1   Goalkeeper                0.00506 
 943       10      1   Left Fullback             0.00448 
 944       10      1   Left Midfielder           0.00192 
 945       10      1   Left Winger               0.00158 
 946       10      1   Right Fullback            0.00370 
 947       10      1   Right Midfielder          0.00196 
 948       10      1   Right Winger              0.00136 
 949       10      1.5 Attacking Midfielder      0.00258 
 950       10      1.5 Center Back               0.00360 
 951       10      1.5 Center Forward            0.00243 
 952       10      1.5 Center Midfielder         0.00317 
 953       10      1.5 Defensive Midfielder      0.00500 
 954       10      1.5 Goalkeeper                0.00487 
 955       10      1.5 Left Fullback             0.00504 
 956       10      1.5 Left Midfielder           0.00333 
 957       10      1.5 Left Winger               0.00164 
 958       10      1.5 Right Fullback            0.00294 
 959       10      1.5 Right Midfielder          0.00421 
 960       10      1.5 Right Winger              0.00360 
 961       10      2   Attacking Midfielder      0.00248 
 962       10      2   Center Back               0.00769 
 963       10      2   Center Forward            0.00453 
 964       10      2   Center Midfielder         0.00474 
 965       10      2   Defensive Midfielder      0.00366 
 966       10      2   Goalkeeper                0.00372 
 967       10      2   Left Fullback             0.00318 
 968       10      2   Left Midfielder           0.00404 
 969       10      2   Left Winger               0.00373 
 970       10      2   Right Fullback            0.00356 
 971       10      2   Right Midfielder          0.00160 
 972       10      2   Right Winger              0.00443 
 973       10      2.5 Attacking Midfielder      0.00604 
 974       10      2.5 Center Back               0.00682 
 975       10      2.5 Center Forward            0.00494 
 976       10      2.5 Center Midfielder         0       
 977       10      2.5 Defensive Midfielder      0.00417 
 978       10      2.5 Goalkeeper                0.00316 
 979       10      2.5 Left Fullback             0.00804 
 980       10      2.5 Left Midfielder           0       
 981       10      2.5 Left Winger               0       
 982       10      2.5 Right Fullback            0.00438 
 983       10      2.5 Right Midfielder          0.00450 
 984       10      2.5 Right Winger              0.00657 
 985       10      3   Attacking Midfielder      0.00112 
 986       10      3   Center Back               0.00771 
 987       10      3   Center Forward            0.00304 
 988       10      3   Center Midfielder         0.00580 
 989       10      3   Defensive Midfielder      0.00681 
 990       10      3   Goalkeeper                0       
 991       10      3   Left Fullback             0.00320 
 992       10      3   Left Midfielder           0.00951 
 993       10      3   Left Winger               0.00691 
 994       10      3   Right Fullback            0.00667 
 995       10      3   Right Midfielder          0.00259 
 996       10      3   Right Winger              0.00511 
 997       10      3.5 Attacking Midfielder      0.002   
 998       10      3.5 Center Back               0.00395 
 999       10      3.5 Center Forward            0       
1000       10      3.5 Defensive Midfielder      0.00441 
# … with 2,144 more rows
# ℹ Use `print(n = ...)` to see more rows

Interactive exploration tool

extract()

multiverse_tree()

spec_curve()

multiverse_tree(
  mv, branches = c("frmls", "position_alternative"),
  label = "name", label_size = 4
  ) +
  ggraph::scale_edge_colour_brewer(
    palette = "Dark2", labels = c("Model", "Position"))

Interactive exploration tool

extract()

multiverse_tree()

spec_curve()

spec_curve(mv, "skintone", branch_order = frmls_branch)

Design adopted from Simonsohn, U., Simmons, J.P. & Nelson, L.D. Specification curve analysis. (2020). Nature Human Behaviour, 4, 1208–1214. https://doi.org/10.1038/s41562-020-0912-z

Thank you

  • The package is available on CRAN (install.packages("mverse"))
  • Experimental features are available on GitHub (remotes::install_github("mverseanalysis/mverse", ref = "test"))
  • For those more adventurous, see multiverse package by Abhraneel Sarma, Matthew Kay, et al. at https://mucollective.github.io/multiverse/ on which mverse is built
  • Demo codes for the slides available at https://github.com/mjmoon/presentations/tree/main/CANSSI-Ontario-Stat-SW-Conf-22