# Models missing random intercepts or random correlations

## Table of Contents

## 1 Random slopes only (no random intercepts)

A conventional misconception is that random intercepts are more important for protecting against Type I error than random slopes. While random intercepts are clearly important when item effects are confounded with the treatment variable (i.e., between-items designs), it is less clear what the consequences would be for removing them in within-unit designs. This could have important implications for how one goes about simplifying the random effects structure when a model does not converge. Thus, the current simulations only include random intercepts when the comparision is between units. We used two different random effects parameterizations for between-items designs, and one for within-items designs.

- between-items design:
- parameterization 1:
`y~c+(0+c|Subject)+(1|Item)`

- parameterization 2:
`y~c+(0+c|Subject)+(0+c|Item)`

- parameterization 1:
- within-items design:
`y~c+(0+c|Subject)+(0+c|Item)`

Results are reported in table format, with \(p\) values obtained using either the \(t\)-as-\(z\) heuristic, or a likelihood-ratio test (\(\chi^2\)). Results from the maximal model ("max") are included in all tables as a benchmark. We considered two different coding schemes for the experimental treatment variable, an ANOVA-style "deviation" coding scheme (-.5, .5), and a treatment coding scheme (0, 1).

Figure 1 (below) illustrates the differences in expressivity of models with and without correlation between random intercept and random slope. Although the maximal model (Model 3 in the manuscript) is the more general model for "within" designs, the model with no random-correlations is also of interest because it has fewer parameters and may thus be easier to fit stably. However, the performance of the model depends on the coding scheme used to express one's experimental treatment, a characteristic which is absent from the other models we consider in the paper. Figure 2 (below) illustrates this dependence by showing that the distributions of random effects with no correlation under deviation coding have a non-zero correlation under treatment coding. As shown in our simulations below, this dependence on coding can affect the degree of anti-conservativity of \(p\)-values in such models.

Like the no-random-correlation model, the no-within-unit-intercepts model will express different model families depending on the coding used for experimental condition: here, treatment coding would imply that subject-specific means differ only for condition B, not condition A, whereas deviation coding would imply that subject-specific means differ equally and in opposite directions from the grand mean in the two conditions.

Figure 1. Joint distributions allowed for random intercepts and random slopes for models with no-within unit intercepts, and models with no random correlations, in the deviation-coding representation. Here the S_{0} axis is the subject's overall deviation from the grand mean, and the S_{1} axis is half the subject-specific discrepancy between experimental conditions (above and beyond the population-average discrepancy). Both models allow distributions plotted in black, where subject's overall reaction time is uncorrelated with subject's idiosyncratic sensitivity to word type; only a maximal model allows distributions plotted in gray, where the two are correlated.

Figure 2. Joint distributions allowed for random intercepts and random slopes for models with no-within unit intercepts, and models with no random correlations, in the treatment-coding representation. Here the S_{0} axis is the subject's condition-A deviation from the condition-A grand mean, and the S_{1} axis is the subject-specific discrepancy between experimental conditions (above and beyond the population-average discrepancy). The distributions in black i Figure 1 cannot be fitted using the no-random-correlations model with a treatment coding, since in each case there is a correlation between random slope and intercept in the treatment-coding representation.

### 1.1 Results (Within-items and Between-items parameterization 1)

- Type I Error

Deviation Treatment Maximal Deviation Treatment Maximal \(t\) as \(z\) \(t\) as \(z\) \(t\) as \(z\) \(\chi^2\) \(\chi^2\) \(\chi^2\) wsbi.12 .09973 .08752 .08619 .08085 .07135 .07026 wsbi.24 .07321 .06553 .06485 .06489 .05856 .05750 wswi.12 .07007 .06625 .07185 .05601 .05811 .05885 wswi.24 .06366 .05892 .06265 .05536 .05405 .05594

RS only (Deviation) RS only (Treatment) Maximal wsbi.12 wsbi.24 wswi.12 wswi.24

- Power

Deviation Treatment Maximal Deviation Treatment Maximal \(t\) as \(z\) \(t\) as \(z\) \(t\) as \(z\) \(\chi^2\) \(\chi^2\) \(\chi^2\) wsbi.12 .32280 .29611 .30022 .28825 .26401 .26725 wsbi.24 .40103 .36817 .38249 .38010 .35039 .36361 wswi.12 .47739 .40137 .49641 .43655 .37146 .46032 wswi.24 .61984 .53852 .62941 .59571 .51949 .61037

RS only (Deviation) RS only (Treatment) Maximal wsbi.12 wsbi.24 wswi.12 wswi.24

### 1.2 Results (Between-items parameterization 2)

- Type I Error

Deviation Treatment Maximal Deviation Treatment Maximal \(t\) as \(z\) \(t\) as \(z\) \(t\) as \(z\) \(\chi^2\) \(\chi^2\) \(\chi^2\) wsbi.12 .09958 .17314 .08619 .08102 .13980 .07026 wsbi.24 .07320 .13312 .06485 .06508 .11797 .05750

RS only (Deviation) RS only (Treatment) Maximal wsbi.12 wsbi.24

- Power

Deviation Treatment Maximal Deviation Treatment Maximal \(t\) as \(z\) \(t\) as \(z\) \(t\) as \(z\) \(\chi^2\) \(\chi^2\) \(\chi^2\) wsbi.12 0.32274 0.42197 0.30022 0.28872 0.37181 0.26725 wsbi.24 0.40103 0.49264 0.38249 0.38064 0.46814 0.36361

RS only (Deviation) RS only (Treatment) Maximal wsbi.12 wsbi.24

## 2 Independent random intercepts and random slopes

In a maximal model, random intercepts and slopes are considered to be drawn from a multivariate distribution, the intercept and slope variances are estimated, as well as their covariance. On some occasions where a model does not converge, the question arises as to what the consequences would be of removing the random correlation parameters.

To investigate this, we fit models with the following random effects
structure (where `c`

indexes experimental condition):

- between-items design:
`y~c+(1|Subject)+(0+c|Subject)+(1|Item)`

- within-items design:
`y~c+(1|Subject)+(0+c|Subject)+(1|Item)+(0+c|Item)`

### 2.1 Type I Error

Deviation | Treatment | Maximal | Deviation | Treatment | Maximal | |
---|---|---|---|---|---|---|

\(t\) as \(z\) | \(t\) as \(z\) | \(t\) as \(z\) | \(\chi^2\) | \(\chi^2\) | max \(\chi^2\) | |

wsbi.12 | .08611 | .09052 | .08619 | .06933 | .07239 | .07026 |

wsbi.24 | .06433 | .06728 | .06485 | .05696 | .05937 | .05750 |

wswi.12 | .07246 | .08974 | .07185 | .05919 | .07465 | .05885 |

wswi.24 | .06245 | .07463 | .06265 | .05563 | .06650 | .05594 |

No corr (Deviation) | No corr (Treatment) | Maximal | |
---|---|---|---|

wsbi.12 | |||

wsbi.24 | |||

wswi.12 | |||

wswi.24 |

### 2.2 Power

Deviation | Treatment | Maximal | Deviation | Treatment | Maximal | |
---|---|---|---|---|---|---|

\(t\) as \(z\) | \(t\) as \(z\) | \(t\) as \(z\) | \(\chi^2\) | \(\chi^2\) | \(\chi^2\) | |

wsbi.12 | .29953 | .30704 | .30022 | .26656 | .27283 | .26725 |

wsbi.24 | .38159 | .38759 | .38249 | .36254 | .36848 | .36361 |

wswi.12 | .49710 | .53355 | .49641 | .46116 | .49853 | .46032 |

wswi.24 | .62856 | .65386 | .62941 | .60956 | .63640 | .61037 |

No corr (Deviation) | No corr (Treatment) | Maximal | |
---|---|---|---|

wsbi.12 | |||

wsbi.24 | |||

wswi.12 | |||

wswi.24 |

## 3 Discussion

Models excluding random intercepts for within-unit manipulations showed good control of the Type I error rate, although they showed decreasing power versus a maximal model as slopes became small relative to the by-unit intercept variance and residual variance. Models excluding random correlations exhibited performance that was virtually indistinguishable from maximal models. This suggests that when one is facing decisions about how to simplify the random effects structure for a nonconverging model, then removing correlations will not harm the overall performance of the model. It may also be worth considering removing random intercepts, especially if the estimates from the nonconverged model suggest they may be associated with relatively small amounts of variability. These analyses indicate that, at least for within-unit designs, the conventional view is clearly wrong: random slopes are far more important than random intercepts.

## 4 R Scripts

### 4.1 Random slopes only (no random intercepts)

library(simgen) dev2treat <- function(nitem, wsbi, mcr.params) { # this function converts the default "deviation" coding # of mkDf to treatment coding x <- mkDf(nitem=nitem, wsbi=wsbi, mcr.params=mcr.params) x$Cond <- ifelse(x$Cond==-.5,0,1) return(x) } param.mx <- createParamMx() # Type I Error param.mx[,"eff"] <- 0 mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wsbi.12.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=TRUE) mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wsbi.24.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=TRUE) mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wswi.12.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=FALSE) mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wswi.24.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=FALSE) mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wsbi.12.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=TRUE) mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wsbi.24.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=TRUE) mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wswi.12.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=FALSE) mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wswi.24.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=FALSE) # Power param.mx[,"eff"] <- .8 mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wsbi.12.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=TRUE) mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wsbi.24.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=TRUE) mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wswi.12.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=FALSE) mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wswi.24.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=FALSE) mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wsbi.12.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=TRUE) mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wsbi.24.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=TRUE) mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wswi.12.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=FALSE) mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wswi.24.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=FALSE)

### 4.2 Independent slopes and intercepts

library(simgen) dev2treat <- function(nitem, wsbi, mcr.params) { # this function converts the default "deviation" coding # of mkDf to treatment coding x <- mkDf(nitem=nitem, wsbi=wsbi, mcr.params=mcr.params) x$Cond <- ifelse(x$Cond==-.5,0,1) return(x) } param.mx <- createParamMx() # Type I Error param.mx[,"eff"] <- 0 mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wsbi.12.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=TRUE) mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wsbi.24.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=TRUE) mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wswi.12.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=FALSE) mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wswi.24.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=FALSE) mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wsbi.12.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=TRUE) mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wsbi.24.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=TRUE) mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wswi.12.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=FALSE) mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wswi.24.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=FALSE) # Power param.mx[,"eff"] <- .8 mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wsbi.12.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=TRUE) mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wsbi.24.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=TRUE) mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wswi.12.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=FALSE) mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wswi.24.csv", mcr.xdatFnc="mkDf", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=FALSE) mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wsbi.12.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=TRUE) mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wsbi.24.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=TRUE) mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wswi.12.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=12, wsbi=FALSE) mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wswi.24.csv", mcr.xdatFnc="dev2treat", mcr.varying=param.mx, nsubj=24, nitem=24, wsbi=FALSE)