# Datasets with high random correlations

## Introduction

In our paper, we ran simulations where the range for the random correlation was [-.8, .8]. A reviewer asked us to consider high correlations (>.8), based on the fact that high correlations are often seen in the output from lmer. (Note, however, that a high correlation may reflect numerical estimation problems rather than high random correlations "in the world.") Here, we verify that our main results hold for datasets with high correlations (>.8). To simplify matters, we consider only datasets including a single within-subject/within-item factor, and with 24 subject and 12 or 24 items. We also considered only the best performing stepwise model (the backwards "best path" model).

## Method

We created 10000 additional datasets. In the first 5000 datasets, the by-subject random correlation varied from .8 to 1 or -.8 to -1, while the by-item random correlation varied from -1 to 1. For the second 5000 datasets, the by-item random correlation varied from .8 to 1 or -.8 to -1, while the by-subject random correlation varied from -1 to 1. All other parameters had the same ranges as in the paper.

## Results

### Pre-defined random effects structure

SimulationModeltypeI.12itypeI.24ipower.12ipower.24i
High Subject CorrelationLMEM, Maximal, $$\chi^2_{LR}$$0.0620.0560.4600.614
High Subject CorrelationLMEM, No Random Correlations $$\chi^2_{LR}$$0.0640.0550.4600.612
High Item CorrelationLMEM, Maximal, $$\chi^2_{LR}$$0.0590.0540.4560.611
High Item CorrelationLMEM, No Random Correlations $$\chi^2_{LR}$$0.0600.0550.4580.608

Note that all results for Type I error are within .006 of the original simulations, and all results for power are within .004 of the original simulations (results from the original simulations can be found in Table 6 of the main paper).

### Data-driven random effects structure

The Type I error / power of the best stepwise model when random correlations are high is not appreciably different from its performance from the original parameter space (see Figure 2 of our paper), and asymptotes toward the performance of the maximal model.

Date: March 27, 2012

Generated by Org version 7.8.06 with Emacs version 23

Validate XHTML 1.0