UP | HOME

Datasets with high random correlations

Introduction

In our paper, we ran simulations where the range for the random correlation was [-.8, .8]. A reviewer asked us to consider high correlations (>.8), based on the fact that high correlations are often seen in the output from lmer. (Note, however, that a high correlation may reflect numerical estimation problems rather than high random correlations "in the world.") Here, we verify that our main results hold for datasets with high correlations (>.8). To simplify matters, we consider only datasets including a single within-subject/within-item factor, and with 24 subject and 12 or 24 items. We also considered only the best performing stepwise model (the backwards "best path" model).

Method

We created 10000 additional datasets. In the first 5000 datasets, the by-subject random correlation varied from .8 to 1 or -.8 to -1, while the by-item random correlation varied from -1 to 1. For the second 5000 datasets, the by-item random correlation varied from .8 to 1 or -.8 to -1, while the by-subject random correlation varied from -1 to 1. All other parameters had the same ranges as in the paper.

Results

Pre-defined random effects structure

SimulationModeltypeI.12itypeI.24ipower.12ipower.24i
High Subject CorrelationLMEM, Maximal, \(\chi^2_{LR}\)0.0620.0560.4600.614
High Subject CorrelationLMEM, No Random Correlations \(\chi^2_{LR}\)0.0640.0550.4600.612
High Item CorrelationLMEM, Maximal, \(\chi^2_{LR}\)0.0590.0540.4560.611
High Item CorrelationLMEM, No Random Correlations \(\chi^2_{LR}\)0.0600.0550.4580.608

Note that all results for Type I error are within .006 of the original simulations, and all results for power are within .004 of the original simulations (results from the original simulations can be found in Table 6 of the main paper).

Data-driven random effects structure

img/datadriven.jpg

The Type I error / power of the best stepwise model when random correlations are high is not appreciably different from its performance from the original parameter space (see Figure 2 of our paper), and asymptotes toward the performance of the maximal model.

Author: Dale J. Barr, Roger Levy, Christoph Scheepers, and Harry J. Tily (daleb@daleb-pc)

Date: March 27, 2012

Generated by Org version 7.8.06 with Emacs version 23

Validate XHTML 1.0