UP | HOME

Models missing random intercepts or random correlations

Table of Contents

1 Random slopes only (no random intercepts)

A conventional misconception is that random intercepts are more important for protecting against Type I error than random slopes. While random intercepts are clearly important when item effects are confounded with the treatment variable (i.e., between-items designs), it is less clear what the consequences would be for removing them in within-unit designs. This could have important implications for how one goes about simplifying the random effects structure when a model does not converge. Thus, the current simulations only include random intercepts when the comparision is between units. We used two different random effects parameterizations for between-items designs, and one for within-items designs.

  • between-items design:
    • parameterization 1: y~c+(0+c|Subject)+(1|Item)
    • parameterization 2: y~c+(0+c|Subject)+(0+c|Item)
  • within-items design: y~c+(0+c|Subject)+(0+c|Item)

Results are reported in table format, with \(p\) values obtained using either the \(t\)-as-\(z\) heuristic, or a likelihood-ratio test (\(\chi^2\)). Results from the maximal model ("max") are included in all tables as a benchmark. We considered two different coding schemes for the experimental treatment variable, an ANOVA-style "deviation" coding scheme (-.5, .5), and a treatment coding scheme (0, 1).

Figure 1 (below) illustrates the differences in expressivity of models with and without correlation between random intercept and random slope. Although the maximal model (Model 3 in the manuscript) is the more general model for "within" designs, the model with no random-correlations is also of interest because it has fewer parameters and may thus be easier to fit stably. However, the performance of the model depends on the coding scheme used to express one's experimental treatment, a characteristic which is absent from the other models we consider in the paper. Figure 2 (below) illustrates this dependence by showing that the distributions of random effects with no correlation under deviation coding have a non-zero correlation under treatment coding. As shown in our simulations below, this dependence on coding can affect the degree of anti-conservativity of \(p\)-values in such models.

Like the no-random-correlation model, the no-within-unit-intercepts model will express different model families depending on the coding used for experimental condition: here, treatment coding would imply that subject-specific means differ only for condition B, not condition A, whereas deviation coding would imply that subject-specific means differ equally and in opposite directions from the grand mean in the two conditions.

img/allowable-ellipses-deviation.png

Figure 1. Joint distributions allowed for random intercepts and random slopes for models with no-within unit intercepts, and models with no random correlations, in the deviation-coding representation. Here the S0 axis is the subject's overall deviation from the grand mean, and the S1 axis is half the subject-specific discrepancy between experimental conditions (above and beyond the population-average discrepancy). Both models allow distributions plotted in black, where subject's overall reaction time is uncorrelated with subject's idiosyncratic sensitivity to word type; only a maximal model allows distributions plotted in gray, where the two are correlated.

img/allowable-ellipses-treatment.png

Figure 2. Joint distributions allowed for random intercepts and random slopes for models with no-within unit intercepts, and models with no random correlations, in the treatment-coding representation. Here the S0 axis is the subject's condition-A deviation from the condition-A grand mean, and the S1 axis is the subject-specific discrepancy between experimental conditions (above and beyond the population-average discrepancy). The distributions in black i Figure 1 cannot be fitted using the no-random-correlations model with a treatment coding, since in each case there is a correlation between random slope and intercept in the treatment-coding representation.

1.1 Results (Within-items and Between-items parameterization 1)

  • Type I Error
    DeviationTreatmentMaximalDeviationTreatmentMaximal
    \(t\) as \(z\)\(t\) as \(z\)\(t\) as \(z\)\(\chi^2\)\(\chi^2\)\(\chi^2\)
    wsbi.12.09973.08752.08619.08085.07135.07026
    wsbi.24.07321.06553.06485.06489.05856.05750
    wswi.12.07007.06625.07185.05601.05811.05885
    wswi.24.06366.05892.06265.05536.05405.05594


    RS only (Deviation)RS only (Treatment)Maximal
    wsbi.12
    wsbi.24
    wswi.12
    wswi.24
  • Power
    DeviationTreatmentMaximalDeviationTreatmentMaximal
    \(t\) as \(z\)\(t\) as \(z\)\(t\) as \(z\)\(\chi^2\)\(\chi^2\)\(\chi^2\)
    wsbi.12.32280.29611.30022.28825.26401.26725
    wsbi.24.40103.36817.38249.38010.35039.36361
    wswi.12.47739.40137.49641.43655.37146.46032
    wswi.24.61984.53852.62941.59571.51949.61037


    RS only (Deviation)RS only (Treatment)Maximal
    wsbi.12
    wsbi.24
    wswi.12
    wswi.24

1.2 Results (Between-items parameterization 2)

  • Type I Error
    DeviationTreatmentMaximalDeviationTreatmentMaximal
    \(t\) as \(z\)\(t\) as \(z\)\(t\) as \(z\)\(\chi^2\)\(\chi^2\)\(\chi^2\)
    wsbi.12.09958.17314.08619.08102.13980.07026
    wsbi.24.07320.13312.06485.06508.11797.05750


    RS only (Deviation)RS only (Treatment)Maximal
    wsbi.12
    wsbi.24
  • Power
    DeviationTreatmentMaximalDeviationTreatmentMaximal
    \(t\) as \(z\)\(t\) as \(z\)\(t\) as \(z\)\(\chi^2\)\(\chi^2\)\(\chi^2\)
    wsbi.120.322740.421970.300220.288720.371810.26725
    wsbi.240.401030.492640.382490.380640.468140.36361


    RS only (Deviation)RS only (Treatment)Maximal
    wsbi.12
    wsbi.24

2 Independent random intercepts and random slopes

In a maximal model, random intercepts and slopes are considered to be drawn from a multivariate distribution, the intercept and slope variances are estimated, as well as their covariance. On some occasions where a model does not converge, the question arises as to what the consequences would be of removing the random correlation parameters.

To investigate this, we fit models with the following random effects structure (where c indexes experimental condition):

  • between-items design: y~c+(1|Subject)+(0+c|Subject)+(1|Item)
  • within-items design: y~c+(1|Subject)+(0+c|Subject)+(1|Item)+(0+c|Item)

2.1 Type I Error

DeviationTreatmentMaximalDeviationTreatmentMaximal
\(t\) as \(z\)\(t\) as \(z\)\(t\) as \(z\)\(\chi^2\)\(\chi^2\)max \(\chi^2\)
wsbi.12.08611.09052.08619.06933.07239.07026
wsbi.24.06433.06728.06485.05696.05937.05750
wswi.12.07246.08974.07185.05919.07465.05885
wswi.24.06245.07463.06265.05563.06650.05594


No corr (Deviation)No corr (Treatment)Maximal
wsbi.12
wsbi.24
wswi.12
wswi.24

2.2 Power

DeviationTreatmentMaximalDeviationTreatmentMaximal
\(t\) as \(z\)\(t\) as \(z\)\(t\) as \(z\)\(\chi^2\)\(\chi^2\)\(\chi^2\)
wsbi.12.29953.30704.30022.26656.27283.26725
wsbi.24.38159.38759.38249.36254.36848.36361
wswi.12.49710.53355.49641.46116.49853.46032
wswi.24.62856.65386.62941.60956.63640.61037


No corr (Deviation)No corr (Treatment)Maximal
wsbi.12
wsbi.24
wswi.12
wswi.24

3 Discussion

Models excluding random intercepts for within-unit manipulations showed good control of the Type I error rate, although they showed decreasing power versus a maximal model as slopes became small relative to the by-unit intercept variance and residual variance. Models excluding random correlations exhibited performance that was virtually indistinguishable from maximal models. This suggests that when one is facing decisions about how to simplify the random effects structure for a nonconverging model, then removing correlations will not harm the overall performance of the model. It may also be worth considering removing random intercepts, especially if the estimates from the nonconverged model suggest they may be associated with relatively small amounts of variability. These analyses indicate that, at least for within-unit designs, the conventional view is clearly wrong: random slopes are far more important than random intercepts.

4 R Scripts

4.1 Random slopes only (no random intercepts)

library(simgen)

dev2treat <- function(nitem, wsbi, mcr.params) {
  # this function converts the default "deviation" coding
  # of mkDf to treatment coding
  x <- mkDf(nitem=nitem, wsbi=wsbi, mcr.params=mcr.params)
  x$Cond <- ifelse(x$Cond==-.5,0,1)
  return(x)
}

param.mx <- createParamMx()

# Type I Error
param.mx[,"eff"] <- 0

mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wsbi.12.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wsbi.24.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wswi.12.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wswi.24.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wsbi.12.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wsbi.24.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wswi.12.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wswi.24.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=FALSE)

# Power
param.mx[,"eff"] <- .8

mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wsbi.12.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wsbi.24.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wswi.12.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wswi.24.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wsbi.12.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wsbi.24.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wswi.12.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wswi.24.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=FALSE)

4.2 Independent slopes and intercepts

library(simgen)

dev2treat <- function(nitem, wsbi, mcr.params) {
  # this function converts the default "deviation" coding
  # of mkDf to treatment coding
  x <- mkDf(nitem=nitem, wsbi=wsbi, mcr.params=mcr.params)
  x$Cond <- ifelse(x$Cond==-.5,0,1)
  return(x)
}

param.mx <- createParamMx()

# Type I Error
param.mx[,"eff"] <- 0

mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wsbi.12.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wsbi.24.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wswi.12.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wswi.24.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wsbi.12.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wsbi.24.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wswi.12.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wswi.24.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=FALSE)

# Power
param.mx[,"eff"] <- .8

mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wsbi.12.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wsbi.24.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wswi.12.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wswi.24.csv",
      mcr.xdatFnc="mkDf", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=FALSE)    

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wsbi.12.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wsbi.24.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wswi.12.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wswi.24.csv",
      mcr.xdatFnc="dev2treat", mcr.varying=param.mx, 
      nsubj=24, nitem=24, wsbi=FALSE)    

Author: Dale J. Barr, Roger Levy, Christoph Scheepers, and Harry J. Tily (daleb@daleb-pc)

Date: March 27, 2012

Generated by Org version 7.8.06 with Emacs version 23

Validate XHTML 1.0