Models missing random intercepts or random correlations

1 Random slopes only (no random intercepts)

A conventional misconception is that random intercepts are more important for protecting against Type I error than random slopes. While random intercepts are clearly important when item effects are confounded with the treatment variable (i.e., between-items designs), it is less clear what the consequences would be for removing them in within-unit designs. This could have important implications for how one goes about simplifying the random effects structure when a model does not converge. Thus, the current simulations only include random intercepts when the comparision is between units. We used two different random effects parameterizations for between-items designs, and one for within-items designs.

• between-items design:
• parameterization 1: y~c+(0+c|Subject)+(1|Item)
• parameterization 2: y~c+(0+c|Subject)+(0+c|Item)
• within-items design: y~c+(0+c|Subject)+(0+c|Item)

Results are reported in table format, with $$p$$ values obtained using either the $$t$$-as-$$z$$ heuristic, or a likelihood-ratio test ($$\chi^2$$). Results from the maximal model ("max") are included in all tables as a benchmark. We considered two different coding schemes for the experimental treatment variable, an ANOVA-style "deviation" coding scheme (-.5, .5), and a treatment coding scheme (0, 1).

Figure 1 (below) illustrates the differences in expressivity of models with and without correlation between random intercept and random slope. Although the maximal model (Model 3 in the manuscript) is the more general model for "within" designs, the model with no random-correlations is also of interest because it has fewer parameters and may thus be easier to fit stably. However, the performance of the model depends on the coding scheme used to express one's experimental treatment, a characteristic which is absent from the other models we consider in the paper. Figure 2 (below) illustrates this dependence by showing that the distributions of random effects with no correlation under deviation coding have a non-zero correlation under treatment coding. As shown in our simulations below, this dependence on coding can affect the degree of anti-conservativity of $$p$$-values in such models.

Like the no-random-correlation model, the no-within-unit-intercepts model will express different model families depending on the coding used for experimental condition: here, treatment coding would imply that subject-specific means differ only for condition B, not condition A, whereas deviation coding would imply that subject-specific means differ equally and in opposite directions from the grand mean in the two conditions.

1.1 Results (Within-items and Between-items parameterization 1)

• Type I Error
DeviationTreatmentMaximalDeviationTreatmentMaximal
$$t$$ as $$z$$$$t$$ as $$z$$$$t$$ as $$z$$$$\chi^2$$$$\chi^2$$$$\chi^2$$
wsbi.12.09973.08752.08619.08085.07135.07026
wsbi.24.07321.06553.06485.06489.05856.05750
wswi.12.07007.06625.07185.05601.05811.05885
wswi.24.06366.05892.06265.05536.05405.05594

RS only (Deviation)RS only (Treatment)Maximal
wsbi.12
wsbi.24
wswi.12
wswi.24
• Power
DeviationTreatmentMaximalDeviationTreatmentMaximal
$$t$$ as $$z$$$$t$$ as $$z$$$$t$$ as $$z$$$$\chi^2$$$$\chi^2$$$$\chi^2$$
wsbi.12.32280.29611.30022.28825.26401.26725
wsbi.24.40103.36817.38249.38010.35039.36361
wswi.12.47739.40137.49641.43655.37146.46032
wswi.24.61984.53852.62941.59571.51949.61037

RS only (Deviation)RS only (Treatment)Maximal
wsbi.12
wsbi.24
wswi.12
wswi.24

1.2 Results (Between-items parameterization 2)

• Type I Error
DeviationTreatmentMaximalDeviationTreatmentMaximal
$$t$$ as $$z$$$$t$$ as $$z$$$$t$$ as $$z$$$$\chi^2$$$$\chi^2$$$$\chi^2$$
wsbi.12.09958.17314.08619.08102.13980.07026
wsbi.24.07320.13312.06485.06508.11797.05750

RS only (Deviation)RS only (Treatment)Maximal
wsbi.12
wsbi.24
• Power
DeviationTreatmentMaximalDeviationTreatmentMaximal
$$t$$ as $$z$$$$t$$ as $$z$$$$t$$ as $$z$$$$\chi^2$$$$\chi^2$$$$\chi^2$$
wsbi.120.322740.421970.300220.288720.371810.26725
wsbi.240.401030.492640.382490.380640.468140.36361

RS only (Deviation)RS only (Treatment)Maximal
wsbi.12
wsbi.24

2 Independent random intercepts and random slopes

In a maximal model, random intercepts and slopes are considered to be drawn from a multivariate distribution, the intercept and slope variances are estimated, as well as their covariance. On some occasions where a model does not converge, the question arises as to what the consequences would be of removing the random correlation parameters.

To investigate this, we fit models with the following random effects structure (where c indexes experimental condition):

• between-items design: y~c+(1|Subject)+(0+c|Subject)+(1|Item)
• within-items design: y~c+(1|Subject)+(0+c|Subject)+(1|Item)+(0+c|Item)

2.1 Type I Error

DeviationTreatmentMaximalDeviationTreatmentMaximal
$$t$$ as $$z$$$$t$$ as $$z$$$$t$$ as $$z$$$$\chi^2$$$$\chi^2$$max $$\chi^2$$
wsbi.12.08611.09052.08619.06933.07239.07026
wsbi.24.06433.06728.06485.05696.05937.05750
wswi.12.07246.08974.07185.05919.07465.05885
wswi.24.06245.07463.06265.05563.06650.05594

No corr (Deviation)No corr (Treatment)Maximal
wsbi.12
wsbi.24
wswi.12
wswi.24

2.2 Power

DeviationTreatmentMaximalDeviationTreatmentMaximal
$$t$$ as $$z$$$$t$$ as $$z$$$$t$$ as $$z$$$$\chi^2$$$$\chi^2$$$$\chi^2$$
wsbi.12.29953.30704.30022.26656.27283.26725
wsbi.24.38159.38759.38249.36254.36848.36361
wswi.12.49710.53355.49641.46116.49853.46032
wswi.24.62856.65386.62941.60956.63640.61037

No corr (Deviation)No corr (Treatment)Maximal
wsbi.12
wsbi.24
wswi.12
wswi.24

3 Discussion

Models excluding random intercepts for within-unit manipulations showed good control of the Type I error rate, although they showed decreasing power versus a maximal model as slopes became small relative to the by-unit intercept variance and residual variance. Models excluding random correlations exhibited performance that was virtually indistinguishable from maximal models. This suggests that when one is facing decisions about how to simplify the random effects structure for a nonconverging model, then removing correlations will not harm the overall performance of the model. It may also be worth considering removing random intercepts, especially if the estimates from the nonconverged model suggest they may be associated with relatively small amounts of variability. These analyses indicate that, at least for within-unit designs, the conventional view is clearly wrong: random slopes are far more important than random intercepts.

4 R Scripts

4.1 Random slopes only (no random intercepts)

library(simgen)

dev2treat <- function(nitem, wsbi, mcr.params) {
# this function converts the default "deviation" coding
# of mkDf to treatment coding
x <- mkDf(nitem=nitem, wsbi=wsbi, mcr.params=mcr.params)
x$Cond <- ifelse(x$Cond==-.5,0,1)
return(x)
}

param.mx <- createParamMx()

# Type I Error
param.mx[,"eff"] <- 0

mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wsbi.12.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wsbi.24.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wswi.12.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h0.wswi.24.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wsbi.12.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wsbi.24.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wswi.12.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h0.wswi.24.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=FALSE)

# Power
param.mx[,"eff"] <- .8

mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wsbi.12.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wsbi.24.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wswi.12.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonly.h1.wswi.24.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wsbi.12.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wsbi.24.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wswi.12.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitrsonly", mcr.outfile="fitrsonlyTreat.h1.wswi.24.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=FALSE)



4.2 Independent slopes and intercepts

library(simgen)

dev2treat <- function(nitem, wsbi, mcr.params) {
# this function converts the default "deviation" coding
# of mkDf to treatment coding
x <- mkDf(nitem=nitem, wsbi=wsbi, mcr.params=mcr.params)
x$Cond <- ifelse(x$Cond==-.5,0,1)
return(x)
}

param.mx <- createParamMx()

# Type I Error
param.mx[,"eff"] <- 0

mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wsbi.12.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wsbi.24.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wswi.12.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h0.wswi.24.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wsbi.12.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wsbi.24.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wswi.12.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h0.wswi.24.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=FALSE)

# Power
param.mx[,"eff"] <- .8

mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wsbi.12.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wsbi.24.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wswi.12.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorr.h1.wswi.24.csv",
mcr.xdatFnc="mkDf", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wsbi.12.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wsbi.24.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=TRUE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wswi.12.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=12, wsbi=FALSE)

mcRun("fitnocorr", mcr.outfile="fitnocorrTreat.h1.wswi.24.csv",
mcr.xdatFnc="dev2treat", mcr.varying=param.mx,
nsubj=24, nitem=24, wsbi=FALSE)


Date: March 27, 2012

Generated by Org version 7.8.06 with Emacs version 23

Validate XHTML 1.0